I'm facing some challenges with Elastic Search. I want to query for by some text and then filter based on a category. I followed the Elastic Search 6.3 Documentation for Queries but my response for ES is always empty. I know for a fact that I have at least one entry that should match the request. Below I have posted my query to Elastic Search and the entry that I know is present in my Elastic Search index. Any help is very much appreciated.
Query
{
"from": 0,
"size": 300,
"query": {
"bool": {
"filter": {
"term": {"category": "Soups"}
},
"should": [
{"term": {"instructions": "Matt"}},
{"term": {"introduction": "Matt"}},
{"term": {"recipe_name": "Matt"}},
],
"minimum_should_match": 1,
"boost": 1.0
}
}
}
Record Present in Elastic Search
{
"_index": "recipes",
"_type": "_doc",
"_id": "QMCScWoBkkkjW61rD81v",
"_score": 0.2876821,
"_source": {
"calories": 124,
"category": "Soups",
"cook_time": {
"hour": "2",
"min": "4"
},
"cooking_temp": "375",
"cooking_temp_units": "°F",
"creator_username": "virtualprodigy",
"ingredients": [
{
"majorQuantity": "1 ",
"measuring_units": "teaspoon",
"minorQuantity": " ",
"name": "mett"
}
],
"instructions": "instructions",
"introduction": "intro",
"prep_time": {
"hour": "1",
"min": "2"
},
"recipe_name": "Matt Test",
"servings": 1
}
}
Your fields are probably indexed using a standard analyser, which means they are split into tokens and lowercased. The term query is an exact match and does not perform this analysis, so you are looking for 'Matt' and it only has 'matt'. You look for 'Soups' and it only has 'soups'. The easiest fix is to change your term queries into match queries. e.g:
{
"from": 0,
"size": 300,
"query": {
"bool": {
"filter": {
"match": {
"category": "Soups"
}
},
"should": [
{"match": {"instructions": "Matt"}},
{"match": {"introduction": "Matt"}},
{"match": {"recipe_name": "Matt"}}
],
"minimum_should_match": 1,
"boost": 1.0
}
}
}
Related
I have a email elastic search db created uses following mappings for email sender and receipients:
"mappings": {
...
"recipients": {
"type": "keyword"
},
"sender": {
"type": "keyword"
},
...
I am given a list of emails and I try to query the emails if the any of the email is either the sender OR recipient. For example, I try to use following query:
{
"query": {
"multi_match" : {
"query": "abc#apple.com defg#samsung.com",
"operator": "OR",
"fields": [ "recipients", "sender" ],
"type": "cross_fields"
}
}
}
to query the emails if (abc#apple.com exists in the sender or receipient) OR (defg#samsung.com exists in the sender or receipient). But it doesn't return any result.. (But it do exists)
Does anyone know how to query the emails if any of the email in sender or receipient?
Thanks
It's good that you have found the solution, but understanding why multi_match didn't work and why query_string worked, and why you should avoid the query_string if possible important.
As mentioned, in the official Elasticsearch documentation,
Also, your multi_match query didn't work as you provided the two mails input in the same query like abc#apple.com defg#samsung.com and this term is analyzed depending on the fields analyzer(keyword in your example), So, it would try to find abc#apple.com defg#samsung.com in your fields, not abc#apple.com or defg#samsung.com.
If you want to use the multi_match, right query would be
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "abc#apple.com",
"operator": "OR",
"fields": [
"recipients",
"sender"
],
"type": "cross_fields"
}
},
{
"multi_match": {
"query": "defg#samsung.com",
"operator": "OR",
"fields": [
"recipients",
"sender"
],
"type": "cross_fields"
}
}
]
}
}
}
which returns below documents.
"hits": [
{
"_index": "71367024",
"_id": "1",
"_score": 0.6931471,
"_source": {
"recipients": "abc#apple.com",
"sender": "foo#bar.com"
}
},
{
"_index": "71367024",
"_id": "2",
"_score": 0.6931471,
"_source": {
"recipients": "defg#samsung.com",
"sender": "baz#bar.com"
}
}
]
I think I may find the answer. Using the following query will work:
{
"query": {
"query_string" : {
"query": "abc#apple.com OR defg#samsung.com",
"fields": [ "recipients", "sender" ]
}
}
I have a index in elastic search called professor
If for cross field i need "AND" condition
for same field array i need to OR condition
I need to search subject which is Physics or Accounting this is array of fields(OR) statement
I need to search type is Permanent(&) condition
I need to search Location is NY(&) condition
There is chance that {'type':['Contract','Guest']} type also coming as list
test = [{'id':1,'name': 'A','subject': ['Maths','Accounting'],'type':'Contract', 'Location':'NY'},
{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'},
{'id':3,'name': 'ABC','subject': ['Maths','Engineering'],'type':'Permanent','Location':'NY'}]
Query is below,3rd one got it, How to add 1 and 2
content_search = es.search(index="professor", body={
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"Location.keyword": "NY"
}
}
]
}
}
})
content_search ['hits']['hits']
Expected out is id [{ 'id':2,'name': 'AB','subject': ['Physics','Engineering'],'type':'Permanent','Location':'NY'}]
You need to use the bool query, to wrap all your conditions
Adding a working example with index data(same as that in question), search query, and search result
Search Query:
{
"query": {
"bool": {
"must": [
{
"match": {
"type.keyword": "Permanent"
}
},
{
"match": {
"Location.keyword": "NY"
}
}
],
"should": [
{
"match": {
"subject.keyword": "Accounting"
}
},
{
"match": {
"subject.keyword": "Physics"
}
}
],
"minimum_should_match": 1,
"boost": 1.0
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64370980",
"_type": "_doc",
"_id": "2",
"_score": 1.8365774,
"_source": {
"id": 2,
"name": "AB",
"subject": [
"Physics",
"Engineering"
],
"type": "Permanent",
"Location": "NY"
}
}
]
Query is below
{
"from" : 0,
"size" : 100,
"query": {
"match_all": {}
}
}
I need to filter from the match_all if name is test
i tried with
{
"from" : 0,
"size" : 100,
"query": {
"match_all": {}
},
"filter": [ "term": { "name": "test" }}]
}
I got error 'Unknown key for a START_ARRAY in [filter].')
You will need to wrap your query in a bool query , try out this search query:
{
"from":0,
"size":10,
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"grocery_name": "elastic"
}
}
]
}
}
}
Update 1:
According to the comment mentioned by #Nons
Search Query:
Terms query return documents that contain an exact term in a provided
field.
{
"from":0,
"size":10,
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": [
{
"term": {
"parentName.keyword": "Developer" <-- note this
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "stof_64275684",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"id": "1",
"name": "A",
"parentName": "Developer",
"Data": [
{
"id": "455",
"name": "Google",
"lastUpdatedDate": "2020-09-10",
"parent_id": "1"
}
],
"Function": [
{
"id": "1",
"name": "Major"
}
]
}
}
]
You can even use a match query where the provided text is analyzed
before matching.
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": {
"match": {
"parentName": "developer"
}
}
}
}
}
I would recommend to use the Chrome ElasticSearch Head plugin. It allows to test and run searches against Elastic very easily (functionality is similar to MySql Workbech).
Please find example of usage of plugin below (combination of condition and aggregation).
I have different types indexed in elastic search.
but, if I want to boost my results on some selected types then what should I do?
I could use type filter in boosting query, but type filter allows me only one type to be used in filter. I need results to be boosted on the basis of multiple types.
Example:
I have Person, Event, Location data indexed in elastic search where Person, Location and Event are my types.
I am searching for keyword 'London' in all types but i want Person and Event type records to be boosted than Location.
How could I achieve the same?
One of the ways of getting the desired functionality is by wrapping your query inside a bool query and then make use of the should clause, in order to boost certain documents
Small example:
POST test/person
{
"title": "london elise moore"
}
POST test/event
{
"title" : "london is a great city"
}
Without boost:
GET test/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "london"
}
}
]
}
}
}
With the following response:
"hits": {
"total": 2,
"max_score": 0.2972674,
"hits": [
{
"_index": "test",
"_type": "person",
"_id": "AVVx621GYvUb9aQn6r5X",
"_score": 0.2972674,
"_source": {
"title": "london elise moore"
}
},
{
"_index": "test",
"_type": "event",
"_id": "AVVx63LrYvUb9aQn6r5Y",
"_score": 0.26010898,
"_source": {
"title": "london is a great city"
}
}
]
}
And now with the added should clause:
GET test/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": "london"
}
}
],
"should": [
{
"term": {
"_type": {
"value": "event",
"boost": 2
}
}
}
]
}
}
}
Which gives back the following response:
"hits": {
"total": 2,
"max_score": 1.0326607,
"hits": [
{
"_index": "test",
"_type": "event",
"_id": "AVVx63LrYvUb9aQn6r5Y",
"_score": 1.0326607,
"_source": {
"title": "london is a great city"
}
},
{
"_index": "test",
"_type": "person",
"_id": "AVVx621GYvUb9aQn6r5X",
"_score": 0.04235228,
"_source": {
"title": "london elise moore"
}
}
]
}
You could even leave out the extra boost in the should clause, cause if the should clause matches it will boost the result :)
Hope this helps!
I see two ways of doing that using that but both is using scripts
1. using sorting
POST c1_1/_search
{
"from": 0,
"size": 10,
"sort": [
{
"_script": {
"order": "desc",
"type": "number",
"script": "double boost = 1; if(doc['_type'].value == 'Person') { boost *= 2 }; if(doc['_type'].value == 'Event') { boost *= 3}; return _score * boost; ",
"params": {}
}
},
{
"_score": {}
}
],
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "*",
"default_operator": "and"
}
}
],
"minimum_should_match": "1"
}
}
}
Second option Using function score.
POST c1_1/_search
{
"from": 0,
"size": 10,
"query": {
"function_score": {
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "*",
"default_operator": "and"
}
}
],
"minimum_should_match": "1"
}
},
"script_score": {
"script": "_score * (doc['_type'].value == 'Person' || doc['_type'].value == 'Event'? 2 : 1)"
}
}
}
}
I have a fallback in my queries to a popularity ranking if no hits are found. Every week I calculate a popRank field based on the number of times the doc is visited in the last month. This means that not all docs will have a popRank, only the ones visited in the last month.
The query below does not work with the must clause even though there are items that contain that category
GET /index/docs/_search
{
"size": 10,
"query": {
"bool": {
"should": [{
"terms": {
"body": [<array of keyword strings>]
}
}, {
"constant_score": {
"filter": {
"match_all": {}
},
"boost": 0
}
}],
"must": [{
"terms": {
"category": ["DIY"],
"boost": 0
}
}],
"minimum_should_match": 1
}
},
"sort": [{
"_score": {
"order": "desc"
}
}, {
"popRank": {
"unmapped_type": "double",
"order": "desc"
}
}]
}
This query is supposed to return resulting docs if the should clause is fulfilled, if not then the popularity ranking will take over, in either case it must be filtered by the category. This works if something other than the match_all returns results but does not work if only the match_all returns results.
This is an example doc.
{
"_index": "index",
"_type": "docs",
"_id": "Fridays",
"_score": 1,
"_source": {
"id": "Fridays",
"body": "text...",
"category": [
"DIY",
"Kitchen"
],
"popRank": 1
}
}