Use Elasticsearch to search "
productID of a or (productID of b and price of c)
" with devp of kibana
This is my code: （what is right?）
GET my_store/products/_search
{
"query":{
"bool":{
"should":[
{
"match":{
"productId":a
}},
{"match":{
"productId":b
}
}
],
"must":{
"match":{
"price":c
}
}
}
}
}
You want productID of a or (productID of b and price of c). It sounds like
productId=a OR (productId=b AND price=C)
{
"query": {
"bool": {
"should": [
{
"match": {"productID": "a"}
},
{
"bool": {
"must": [
{"match": {"productID": "b"}},
{"match": {"price": "c"}}
]
}
}
]
}
}
}
You consider the below
OR = should
AND = must
Since you have not mentioned anything about the data that you have taken, I have indexed the following data:
Sample Index Data
{
"productId":"a",
"price": 100
}
{
"productId":"b",
"price": 200
}
{
"productId":"c",
"price": 300
}
Search Query:
{
"query": {
"bool": {
"should": [
{
"match": {"productId": "a"}
},
{
"bool": {
"must": [
{"match": {"productId": "b"}},
{"match": {"price": 700}}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "foo10",
"_type": "_doc",
"_id": "1",
"_score": 0.9808292,
"_source": {
"productId": "a",
"price": 100
}
}
]
Here, since no data match with "productId": "b" and "price": 700`, in the Search result only the data with "productId": "a" is shown.
You can simply consider Must to be equivalent to logical AND and should as logical OR
Refer this to know about Elasticsearch difference between MUST and SHOULD bool query and to get detailed explanation about Boolean Query refer this official documentation
Related
I have a index in elastic search called professor
If for cross field i need "AND" condition
for same field array i need to OR condition
I need to search subject which is Physics or Accounting this is array of fields(OR) statement
AND
I need to search type is Permanent or GUEST condition this is array of fields(OR) statement
AND
I need to search Location is NY(&) condition
test = [{'id':1,'name': 'A','subject': ['Maths','Accounting'],'type':'Contract', 'Location':'NY'},
{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'},
{'id':3,'name': 'ABC','subject': ['Maths','Engineering'],'type':'Permanent','Location':'NY'},
{'id':4,'name':'ABCD','subject': ['Physics','Engineering'],'type':['Contract','Guest'],'Location':'NY'}]
Query is below,3rd one got it, How to add 1 and 2
content_search = es.search(index="professor", body={
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"Location.keyword": "NY"
}
}
]
}
}
})
content_search ['hits']['hits']
Expected out is id [{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'},{'id':4,'name':'ABCD','subject': ['Physics','Engineering'],'type':['Contract','Guest'],'Location':'NY'}]
The filter clause (query) must appear in matching documents. However
unlike must the score of the query will be ignored. Filter clauses are
executed in filter context, meaning that scoring is ignored and
clauses are considered for caching.
Please go through this Elasticsearch documentation on bool queries, to get a detailed understanding about it.
Adding a working example with index data(same as that in question), search query, and search result
Search Query:
{
"query": {
"bool": {
"must": {
"match": {
"Location.keyword": "NY"
}
},
"filter": [
{
"bool": {
"should": [
{
"match": {
"subject.keyword": "Accounting"
}
},
{
"match": {
"subject.keyword": "Physics"
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"type.keyword": "Permanent"
}
},
{
"match": {
"type.keyword": "Guest"
}
}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64370980",
"_type": "_doc",
"_id": "2",
"_score": 0.10536051,
"_source": {
"id": 2,
"name": "AB",
"subject": [
"Physics",
"Engineering"
],
"type": "Permanent",
"Location": "NY"
}
},
{
"_index": "stof_64370980",
"_type": "_doc",
"_id": "4",
"_score": 0.10536051,
"_source": {
"id": 4,
"name": "ABCD",
"subject": [
"Physics",
"Engineering"
],
"type": [
"Contract",
"Guest"
],
"Location": "NY"
}
}
]
Another Search Query:
You can even use terms query that returns documents that contain
one or more exact terms in a provided field.The terms query is the
same as the term query, except you can search for multiple values.
{
"query": {
"bool": {
"must": [
{
"terms": {
"subject.keyword": [
"Physics",
"Accounting"
]
}
},
{
"terms": {
"type.keyword": [
"Guest",
"Permanent"
]
}
},
{
"match": {
"Location.keyword": "NY"
}
}
]
}
}
}
Update 1:
{
"query": {
"bool": {
"must": [
{
"terms": {
"subject.keyword": [
"Physics",
"Accounting"
]
}
},
{
"terms": {
"type.keyword": [
"Guest",
"Permanent"
]
}
},
{
"match": {
"Location.keyword": "NY"
}
},
{
"query_string": {
"query": "ABCD"
}
}
]
}
}
}
I've got two different queries against my elasticsearch. The difference between these two queries is that the first one got the two search criteria in one boolean should query and he second splits it into two single bool should queries. The first one return the expected response but the second one doesnt match to any document even if there are documents which contains both criteria. If i refactor the second one so that the two splitted bool should queries are encapsulatec by a bool should querie it returns the expected response like it is for querie 1.
The question is why does query 2 doesn't return the response as 1 and 3 do? Am i missing something?
EDIT: provided example data
EDIT: my problem solved, it was just a spelling mistake while building the range query in my code and i doesnt recognize it -.- but maybe the explanation from the answer here will help somebody else.
1.
GET _search
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"range": {
"streetNr": {
"from": "1",
"to": "100",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
},
{
"match": {
"geographicAddress.city": {
"query": "Berlin"
}
}
}
],
"minimum_should_match": "1"
}
}
]
}
}
}
GET _search
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"range": {
"streetNr": {
"from": "1",
"to": "100",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"minimum_should_match": "1"
}
},
{
"bool": {
"should": [
{
"match": {
"geographicAddress.city": {
"query": "Berlin"
}
}
}
],
"minimum_should_match": "1"
}
}
]
}
}
}
GET _search
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"bool": {
"should": [
{
"range": {
"streetNr": {
"from": "1",
"to": "100",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"minimum_should_match": "1"
}
},
{
"bool": {
"should": [
{
"match": {
"geographicAddress.city": {
"query": "Berlin"
}
}
}
],
"minimum_should_match": "1"
}
}
],
"minimum_should_match": "1"
}
}
]
}
}
}
Example data:
{
"_index": "stof_64371064",
"_type": "_doc",
"_id": "1",
"_score": 0.0,
"_source": {
"streetNr": 90,
"geographicAddress": {
"city": "Berlin"
}
}
},
{
"_index": "stof_64371064",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"streetNr": 10,
"geographicAddress": {
"city": "Berlin"
}
}
}
Please refer ES official documentation on bool query, to get a detailed understanding of various clauses.
The structure of your first search query is like -
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{},
{}
],
"minimum_should_match": 1
}
}
]
}
}
}
filter clause is wrapping should query, but at the end of should clause, "minimum_should_match": 1 is added which indicates that 1 should clause must be mandatory.
The structure of your second search query is like -
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": {},
"minimum_should_match": "1"
}
},
{
"bool": {
"should": {},
"minimum_should_match": "1"
}
}
]
}
}
}
Here since you have added "minimum_should_match": "1" after every should clause, then in a way, it acts like a must clause only, as there is only one condition that needs to be matched in the should clause. filter clause is applied enclosing both the bool should clause, so when both the should clause match, then only you will get the result.
The structure of your third search query is like -
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"bool": {
"should": [
{}
],
"minimum_should_match": 1
}
},
{
"bool": {
"should": [
{}
],
"minimum_should_match": 1
}
}
],
"minimum_should_match": 1
}
}
]
}
}
}
In this, you have used multiple combinations of the bool should clause. The first outer bool should clause, wraps two more bool should clause. But here at the end of the outer should clause you have added "minimum_should_match": 1. So though here filter clause is there but it will return a result even if one bool should clause satisfy the condition.
Adding a working example with index data, search query, and search result
Index Data:
{
"streetNr":0,
"geographicAddress":{
"city":"Berlin"
}
}
{
"streetNr":90,
"geographicAddress":{
"city":"Berlin"
}
}
Search Query: (Second search query acc to your question)
{
"query": {
"bool": {
"should": [ <-- note this
{
"bool": {
"should": [
{
"range": {
"streetNr": {
"from": "1",
"to": "100",
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"minimum_should_match": "1"
}
},
{
"bool": {
"should": [
{
"match": {
"geographicAddress.city": {
"query": "Berlin"
}
}
}
],
"minimum_should_match": "1"
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64371064",
"_type": "_doc",
"_id": "1",
"_score": 0.0,
"_source": {
"streetNr": 90,
"geographicAddress": {
"city": "Berlin"
}
}
},
{
"_index": "stof_64371064",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"streetNr": 0,
"geographicAddress": {
"city": "Berlin"
}
}
}
]
In the should with minimum should match=1 you say that if one of the criteria is right return the document as you have set in query 1 and 3 . But in the second query you have set two criteria inside filter and elasticsearch search and returns those documents which both criterias are valid on them. Because of that your second query behaves such as a must in comparison with should in your other queries.
I have a index in elastic search called professor
If for cross field i need "AND" condition
for same field array i need to OR condition
I need to search subject which is Physics or Accounting this is array of fields(OR) statement
I need to search type is Permanent(&) condition
I need to search Location is NY(&) condition
There is chance that {'type':['Contract','Guest']} type also coming as list
test = [{'id':1,'name': 'A','subject': ['Maths','Accounting'],'type':'Contract', 'Location':'NY'},
{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'},
{'id':3,'name': 'ABC','subject': ['Maths','Engineering'],'type':'Permanent','Location':'NY'}]
Query is below,3rd one got it, How to add 1 and 2
content_search = es.search(index="professor", body={
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"Location.keyword": "NY"
}
}
]
}
}
})
content_search ['hits']['hits']
Expected out is id [{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'}]
You need to use the bool query, to wrap all your conditions
Adding a working example with index data(same as that in question), search query, and search result
Search Query:
{
"query": {
"bool": {
"must": [
{
"match": {
"type.keyword": "Permanent"
}
},
{
"match": {
"Location.keyword": "NY"
}
}
],
"should": [
{
"match": {
"subject.keyword": "Accounting"
}
},
{
"match": {
"subject.keyword": "Physics"
}
}
],
"minimum_should_match": 1,
"boost": 1.0
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64370980",
"_type": "_doc",
"_id": "2",
"_score": 1.8365774,
"_source": {
"id": 2,
"name": "AB",
"subject": [
"Physics",
"Engineering"
],
"type": "Permanent",
"Location": "NY"
}
}
]
Query is below
{
"from" : 0,
"size" : 100,
"query": {
"match_all": {}
}
}
I need to filter from the match_all if name is test
i tried with
{
"from" : 0,
"size" : 100,
"query": {
"match_all": {}
},
"filter": [ "term": { "name": "test" }}]
}
I got error 'Unknown key for a START_ARRAY in [filter].')
You will need to wrap your query in a bool query , try out this search query:
{
"from":0,
"size":10,
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"grocery_name": "elastic"
}
}
]
}
}
}
Update 1:
According to the comment mentioned by #Nons
Search Query:
Terms query return documents that contain an exact term in a provided
field.
{
"from":0,
"size":10,
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"parentName.keyword": "Developer" <-- note this
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64275684",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"id": "1",
"name": "A",
"parentName": "Developer",
"Data": [
{
"id": "455",
"name": "Google",
"lastUpdatedDate": "2020-09-10",
"parent_id": "1"
}
],
"Function": [
{
"id": "1",
"name": "Major"
}
]
}
}
]
You can even use a match query where the provided text is analyzed
before matching.
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": {
"match": {
"parentName": "developer"
}
}
}
}
}
I would recommend to use the Chrome ElasticSearch Head plugin. It allows to test and run searches against Elastic very easily (functionality is similar to MySql Workbech).
Please find example of usage of plugin below (combination of condition and aggregation).
I used the following filter and then searched for query string using Lucene to get the view that I was looking for.
{
"query": {
"match": {
"eventSource": {
"query": "ec2.amazonaws.com",
"type": "phrase"
}
}
}
}
I do not want to return event names those start with the word describe or get. Rest of the event names from ec2 event source should be returned.
!(eventName.keyword: Describe* OR eventName.keyword:
Get* )
The question is how to combine these 2 search requests into one?
I need to use that query from my application.
Update:
The Inspect menu of Kibana Discover tab generates this query. I am just trying to rewrite query_string part with usual match or match_phrase using boolean OR clause.
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "!(eventName.keyword: Describe* OR eventName.keyword: Get* )",
"analyze_wildcard": true
}
},
{
"match_phrase": {
"eventSource": {
"query": "ec2.amazonaws.com"
}
}
},
{
"range": {
"#timestamp": {
"format": "strict_date_optional_time",
"gte": "2020-07-09T08:39:15.947Z",
"lte": "2020-07-24T08:39:15.947Z"
}
}
}
],
"filter": [],
"should": [],
"must_not": []
}
}
You can easily use the boolean query's must_not clause to exclude the documents which you don't want in your search result and you can add as many as must_not as you want, it's fairly easy to do and can be done in a single query.
Please refer the example in the same link to get more info. Created sample in my local to show your the correct query, Please note instead of wildcard I am using the prefix query which is better and server your use-case.
Create index mapping
{
"mappings": {
"properties": {
"eventName": {
"type": "keyword"
}
}
}
}
Index sample doc
{
"eventName" : "Describe the events"
}
{
"eventName" : "the Describe events"
}
{
"eventName" : "Get the event"
}
{
"eventName" : "event Get"
}
Now search query to get only 2 and 3rd doc according to your req
{
"query": {
"bool": {
"must_not": [
{
"prefix": {
"eventName": "Desc"
}
},
{
"prefix": {
"eventName": "Get"
}
}
]
}
}
}
Search result
"hits": [
{
"_index": "ngramkey",
"_type": "_doc",
"_id": "2",
"_score": 0.0,
"_source": {
"eventName": "the Describe events"
}
},
{
"_index": "ngramkey",
"_type": "_doc",
"_id": "4",
"_score": 0.0,
"_source": {
"eventName": "event Get"
}
}
]
As suggested by the user "Opster Elasticsearch Ninja", I have merged must not boolean query like this...
{
"query": {
"bool": {
"must": [
{
"bool": {
"must_not": [
{
"prefix": {
"eventName.keyword": "Desc"
}
},
{
"prefix": {
"eventName.keyword": "Get"
}
}
]
}
},
{
"match_phrase": {
"eventSource": {
"query": "ec2.amazonaws.com"
}
}
},
{
"range": {
"#timestamp": {
"format": "strict_date_optional_time",
"gte": "2020-07-09T08:39:15.947Z",
"lte": "2020-07-24T08:39:15.947Z"
}
}
}
],
"filter": [],
"should": [],
"must_not": []
}
}
}