Create API keys on ElasticSearch with limited search capabilities - elasticsearch

I want to create API keys on elasticsearch via POST _security/api_key API, I am able to create these but I want to limit search capability for the generated key which I am unable to do.
Essentially, what I want to achieve is let's say that all my records have a field like "username":"username1" or another one like "username":"username2" i.e. all the records will have a valid value for username field
Now what I want is to be able to create a key where I specify something like "username":"username2" which then gets appended to every search query made using that key as an AND case like if this key searches (/index_name/_search) for
{
"query": {
"match": {
"key_zz":"value_aa"
}
}
}
this query would actually be an AND of both of the below (i.e. if a record with this combination exists "key_zz":"value_aa" but the value for username is not username2, the API would not return that object)
{
"query": {
"match": {
"key_zz":"value_aa"
}
}
}
AND
{
"query": {
"must": {
"match" : {
"username":"username2"
}
}
}
}
I have tried creating the key with all of the following combinations:
"name": "key-name",
"role_descriptors": {
"role_name": {
"indices": [
{
"names": [
"index_name"
],
"privileges": [
"read"
],
"query": {
"match": {
"username": "value_custom"
}
}
}
]
}
}
}
In the query field, I have tried all the following combinations:
"query": {
"and": {
"username": "value_custom"
}
}
"query": {
"bool": {
"must": {
"match": {
"username": "value_custom"
}
}
}
}
"query": {
"bool": {
"must": {
"bool": {
"must": {
"match": {
"username": "value_custom"
}
}
}
}
}
}
But none of the above worked. Also, earlier the mapping type of username field was text but I then updated it to be keyword
In a nutshell, what I am trying to achieve is some kind of document level security. I know ElasticSearch has some kind of Document Level Security (https://www.elastic.co/guide/en/elastic-stack-overview/current/document-level-security.html) but in our architecture, we want to achieve this using API keys with restricted search capabilities. We are currently using Algolia and we were achieving this using the exact implementation I described above. ElasticSearch documentation has references for how to limit a role but not how to limit API keys. Need help to achieve this.
Also, I am using ElasticSearch v7
Some reference links :
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role.html

Related

multi fields search query for elasticsearch golang

I have a situation where I need to do elastic search based on multi-field. For Example: I have multiple fields in my postindex and I want to apply condition on four these fields (i.e. userid, channelid, createat, teamid) to meet my search requirement. When value of all these fields matched then search query displays results and if one of these is not match with values in postindex then it display no result.
I am trying to make a multifield search query for go-elasticsearch to search data from my post index. For the searcquery result four field must match otherwise it display 0 hit/no-result.
So, I think you need to write a following query :
GET postindex/_search
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"term": {
"userid": {
"value": "mcqmycxpyjrddkie9mr13txaqe"
}
}
},
{
"term": {
"channelid": {
"value": "dnoihmrinins3qrm6bb9175ume"
}
}
},
{
"range": {
"createat": {
"gt": 1672909114890
}
}
}
]
}
},
{
"term": {
"teamid": {
"value": "qomrg11o8b8ijxoy8hrcnweoay"
}
}
}
]
}
}
}
In here, there is a bool query with should in parent scope, which is like OR. And inside the should there is another bool query with must which is like AND. We can also write the query shorter, but this will be better for you to understand.

Elastic Search 1.7 - Full Like search like SQL

Im using old version of elastic search and I can't find a way to allow full like search like sql: '%%'
I have to search a non pre-known data so the "columns" are not known.
Ex.
I have this columns ad an example of row:
source,dest,message
eth0, eth1, DELIVERY 3015801: SUCCESS: DID
In the search form, a user write something like this:
+message:"delivery"
or
+message:"delivery" and +source:"eht0"
The row is returned succesfully.
Unfortunally if they write:
+message:"success"
it return no row because of ":"
I need to allow user to write somethig like:
+message:"success*"
This is the query submitted. I have timerange filter and then the query :
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"from": 1561615860000,
"to": 1561619519000
}
}
}
]
}
},
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "+message:\"success\""
}
}
]
}
}
}
},
"sort": [
{
"#timestamp": {
"order": "desc"
}
}
]
}
Is there a way to translate my text search in something like sql
message like 'success%'
Thanks.
Is that an analyzed or a not-analyzed field? Maybe a prefix query is what you want for that.
But generally:
This version is more than dead. Please upgrade.
This is not a relational database. If you try to translate the concepts 1:1 you will not get the most out of it. Documentation or courses will help you to move into proper search and away from LIKE searches.

Creating a security filter in elasticsearch

I'm attempting to create a security filter of sorts to exclude certain users from seeing certain documents in elasticsearch. As an example, if a document contains "ABC:123" and "ABC:XYZ", the user must have both of those in their profile to see the document. We are creating this on the fly using mustache templates. My first attempt was along these lines:
"bool": {
"filter": {
"bool": {
"minimum_should_match": 1,
"should": {
"bool": [{
"must_not": {
"prefix": {
"controlSet": "ABC:"
}
}
},{
"must": {
"terms": {
"controlSet": ["ABC:123","ABC:XYZ"]
}
}
}]
}
}
}
}
However, I quickly realized that this will allow a user with one control to view a document that has multiple. A document must have a subset of the controls that the user has to be matched. So if the user has "ABC:XYZ" only they should not be able to see a document that has "ABC:123" even if the document also contains "ABC:XYZ".
Is there a way to accomplish this that I am missing? Currently we enumerate ever control in the system and add them to a must_not but controls change periodically and I'd rather not maintain that listing manually.
Assuming your document looks like the following:
{
...
"controlSet": ["ABC:123", "ABC:XYZ"],
...
}
and controlSet is a keyword field, the following query should do the trick:
{
"bool": {
"filter": {
"terms": {
"controlSet": ["ABC:123", "ABC:XYZ"]
}
}
}
}
It will match documents with controlSet having at least ABC:123 and ABC:XYZ
I may have found a solution...
{
"bool": {
"must_not": {
"regexp": {
"value": "ABC:~(XYZ|123)",
"flags": "COMPLEMENT"
}
}
}
}
That should allow documents with either ABC:XYZ or ABC:123 (or both) while excluding documents with ABC:[anything else]
Of course I am worried about the speed of the regexp but I think the lack of wildcards will make it relatively fast.

Elasticsearch: Search in an array of JSONs

I'm using Elasticsearch with the python library and I have a problem using the search query when the object become a little bit complex. I have objects build like that in my index:
{
"id" : 120,
"name": bob,
"shared_status": {
"post_id": 123456789,
"text": "This is a sample",
"urls" : [
{
"url": "http://test.1.com",
"displayed_url": "test.1.com"
},
{
"url": "http://blabla.com",
"displayed_url": "blabla.com"
}
]
}
}
Now I want to do a query that will return me this document only if in one of the displayed URL's a substring "test" and there is a field "text" in the main document. So I did this query:
{
"query": {
"bool": {
"must": [
{"exists": {"field": "text"}}
]
}
}
}
}
But I don't know what query to add for the part: one of the displayed URL's a substring "test"
Is that posssible? How does the iteration on the list works?
If you didn't define an explicit mapping for your schema, elasticsearch creates a default mapping based on the data input.
urls will be of type object
displayed_url will be of type string and using standard analyzer
As you don't need any association between url and displayed_url, the current schema will work fine.
You can use a match query for full text match
GET _search
{
"query": {
"bool": {
"must": [
{
"exists": {
"field": "text"
}
},
{
"match": {
"urls.displayed_url": "test"
}
}
]
}
}
}

Search in every field with a fixed parameter

Perhaps it's a basic question; by the way, I need to search in every indexed field and to have a specific fixed value for another field.
How can I do it?
Currently I have a simple: query( "aValue", array_of_models )
I tried many options without success, for example:
query({
"query": {
"bool": {
"query": "aValue",
"filter": {
"term": {
"published": "true"
}
}
}
}
})
I would prefer to avoid to specify the fields to search in because I use the same search params for different models.
I found a solution, perhaps it's not optimized but works:
{
"query": {
"bool": {
"should": [
{
"match": {
"_all": "aValue"
}
}
],
"filter": {
"term": {
"published": true
}
}
}
}
}
Not sure if I understood correctly your intention.
The _all field is as default enabled. So if you have no special mapping every indexed field value is added as text string to the _all field.
You can use the
Query String Query, https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
Simple Query String Query, https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html
With a simple query like this, that should work for you.
GET my_index/_search
{
"query": {
"simple_query_string": {
"query": "aValue",
"fields": []
}
}
}
Both query types contains parameters, that should suffice your use case IMHO.

Resources