We were having some issues where our queries weren't returning items with specific version IDs using ElasticSearch 2.3. After some investigation, it looks like our current elasticsearch query is not behaving when there is a '#' in the version ID.
The query I am trying to perform is something like the following:
{
"query": {
"constant_score": {
"filter": {
"terms": {
"_parent": [
"faro-deployments-webservice-infrastructure|#abc123",
"faro-deployments-webservice-infrastructure|xyz321"
]
}
}
}
}
}
This works fine but excludes any results where the parent ID has a '#' character in it.
I can't seem to find it again, but I recall reading somewhere that # has a specific meaning in this context. I have tried a variety of ways to attempt to escape the #, is there a way to support versions with a # character in it for this or perform a similar query with similar results?
The following seems to work for me. I changed the query to do something similar and did not use the "_parent" field.
{
"query": {
"has_parent": {
"type": "deck",
"query": {
"constant_score": {
"filter": {
"terms": {
"_id": [
"faro-deployments-webservice-infrastructure|#abc123",
"faro-deployments-webservice-infrastructure|xyz321"
]
}
}
}
}
}
}
}
Related
I'm trying to boost matches on a certain field over another.
This works fine:
{
"query": {
"bool": {
"should": [
{
"terms": {
"boost": 2,
"mainField": "foo"
}
},
{
"terms": {
"otherField": "foo"
}
}
]
}
}
}
When i see the documents matched on mainField, i see they have a _score of 2.0 as expected.
But when i wrap this same query in a filter:
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"terms": {
"boost": 2,
"mainField": "foo"
}
},
{
"terms": {
"otherField": "foo"
}
}
]
}
}
]
}
}
}
The _score for all documents is 0.0.
The same thing happens for multi_match. By itself (e.g inside a query) it works fine, but inside a bool + filter, it doesn't work.
Can someone explain why this is the case? I need to wrap in a filter due to the way my app composes queries.
Some context might also help: I'm trying to return documents that match on either mainField or otherField, but sort the ones matching on mainField first, so i figured boost would be the most appropriate choice here. But let me know if there is a better way.
The filter queries are always executed in the filter context. It will always return a score of zero and only contribute to the filtering of documents.
Refer to this documentation, to know more about filter context
Due to this, you are not getting a _score of 2.0, even after applying boost, in the second query
I am using the latest version of Elastic Search (7.9) and i'm trying to find a good way to do a multiple term query as an AND.
Essentially what i want do is:
select * where field1 === 'word' AND field2 === 'different word'
I am currently using term to do an exact keyword match on field1. But adding in the second field is causing me some jip.
{
"query": {
"term": {
"field1": {
"value": "word"
}
}
}
}
This is my current query, i have tried using BOOL. I came across answers in previous versions where i could maybe use a filtered query. But i cant seem to get that to work either.
Can someone help me please. It's really doing my nut.
EDIT
Here is what i've tried with a bool / must with multiple terms. But i get no results even though i know in this case this query should return data
{
"query": {
"bool": {
"must": [
{
"term": {
"field1": {
"value": "word"
}
}
},
{
"term": {
"field2": {
"value": "other word"
}
}
}
]
}
}
}
Ok, so, fun fact. You can do the bool / match as i have tried. What you should remember is that keyword matches (which is what i'm using) are case sensitive. ES doesnt do any analyzing or filtering of the data when you set the type of a field to keyword.
It's also possible to use a bool with a filter to get the same results (when you factor in a type)
{
"query": {
"bool": {
"must":
{
"term": {
"field1": {
"value": "word"
}
}
},
"filter":
{
"term": {
"field2": {
"value": "other word"
}
}
}
}
}
}
I want to create API keys on elasticsearch via POST _security/api_key API, I am able to create these but I want to limit search capability for the generated key which I am unable to do.
Essentially, what I want to achieve is let's say that all my records have a field like "username":"username1" or another one like "username":"username2" i.e. all the records will have a valid value for username field
Now what I want is to be able to create a key where I specify something like "username":"username2" which then gets appended to every search query made using that key as an AND case like if this key searches (/index_name/_search) for
{
"query": {
"match": {
"key_zz":"value_aa"
}
}
}
this query would actually be an AND of both of the below (i.e. if a record with this combination exists "key_zz":"value_aa" but the value for username is not username2, the API would not return that object)
{
"query": {
"match": {
"key_zz":"value_aa"
}
}
}
AND
{
"query": {
"must": {
"match" : {
"username":"username2"
}
}
}
}
I have tried creating the key with all of the following combinations:
"name": "key-name",
"role_descriptors": {
"role_name": {
"indices": [
{
"names": [
"index_name"
],
"privileges": [
"read"
],
"query": {
"match": {
"username": "value_custom"
}
}
}
]
}
}
}
In the query field, I have tried all the following combinations:
"query": {
"and": {
"username": "value_custom"
}
}
"query": {
"bool": {
"must": {
"match": {
"username": "value_custom"
}
}
}
}
"query": {
"bool": {
"must": {
"bool": {
"must": {
"match": {
"username": "value_custom"
}
}
}
}
}
}
But none of the above worked. Also, earlier the mapping type of username field was text but I then updated it to be keyword
In a nutshell, what I am trying to achieve is some kind of document level security. I know ElasticSearch has some kind of Document Level Security (https://www.elastic.co/guide/en/elastic-stack-overview/current/document-level-security.html) but in our architecture, we want to achieve this using API keys with restricted search capabilities. We are currently using Algolia and we were achieving this using the exact implementation I described above. ElasticSearch documentation has references for how to limit a role but not how to limit API keys. Need help to achieve this.
Also, I am using ElasticSearch v7
Some reference links :
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role.html
Im using old version of elastic search and I can't find a way to allow full like search like sql: '%%'
I have to search a non pre-known data so the "columns" are not known.
Ex.
I have this columns ad an example of row:
source,dest,message
eth0, eth1, DELIVERY 3015801: SUCCESS: DID
In the search form, a user write something like this:
+message:"delivery"
or
+message:"delivery" and +source:"eht0"
The row is returned succesfully.
Unfortunally if they write:
+message:"success"
it return no row because of ":"
I need to allow user to write somethig like:
+message:"success*"
This is the query submitted. I have timerange filter and then the query :
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"from": 1561615860000,
"to": 1561619519000
}
}
}
]
}
},
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "+message:\"success\""
}
}
]
}
}
}
},
"sort": [
{
"#timestamp": {
"order": "desc"
}
}
]
}
Is there a way to translate my text search in something like sql
message like 'success%'
Thanks.
Is that an analyzed or a not-analyzed field? Maybe a prefix query is what you want for that.
But generally:
This version is more than dead. Please upgrade.
This is not a relational database. If you try to translate the concepts 1:1 you will not get the most out of it. Documentation or courses will help you to move into proper search and away from LIKE searches.
I have data on ElasticSearch index that looks like this
{
"title": "cubilia",
"people": [
"Ling Deponte",
"Dana Madin",
"Shameka Woodard",
"Bennie Craddock",
"Sandie Bakker"
]
}
Is there a way for me to do a search for all the people whos name starts with
"ling" (should be case insensitive) and get distinct terms properly cased "Ling Deponte" not "ling deponte"?
I am find with changing mappings on the index in any way.
Edit does what I want but is really bad query:
{
"size": 0,
"aggs": {
"person": {
"filter": {
"bool":{
"should":[
{"regexp":{
"people.raw":"(.* )?[lL][iI][nN][gG].*"
}}
]}
},
"aggs": {
"top-colors": {
"terms": {
"size":10,
"field": "people.raw",
"include":
{
"pattern": ["(.* )?[lL][iI][nN][gG].*"]
}
}
}
}
}
}
}
people.raw is not_analyzed
Yes, and you can do it without a regular expression by taking advantage of Elasticsearch's full text capabilities.
GET /test/_search
{
"query": {
"match_phrase": {
"people": "Ling"
}
}
}
Note: This could also be match or match_phrase_prefix in this case. The match_phrase* queries imply an order of the values in the text. match simply looks for any of the values. Since you only have one value, it's pretty much irrelevant.
The problem is that you cannot limit the document responses to just that name because the search API returns documents. With that said, you can use nested documents and get the desired behavior via inner_hits.
You do not want to do wildcard prefixing whenever possible because it simply does not work at scale. To put it in SQL terms, that's like doing a full table scan; you effectively lose the benefit of the inverted index because it has to walk it entirely to find the actual start.
Combining the two should work pretty well though. Here, I use the query to widdle down results to what you are interested in, then I use your inner aggregation to only include based on the value.
{
"size": 0,
"query": {
"match_phrase": {
"people": "Ling"
}
}
"aggs": {
"person": {
"terms": {
"size":10,
"field": "people.raw",
"include": {
"pattern": ["(.* )?[lL][iI][nN][gG].*"]
}
}
}
}
}
Hi Please find the query it may help for your request
GET skills/skill/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"wildcard": {
"skillNames.raw": "jav*"
}
}
]
}
}
}
}
}
My intention is to find documents starting with the "jav"