I am trying to OR together two clauses in a single query but I am seeing some strange behavior and am looking for guidance
Given an index "index1" with two documents -
{"column1": "A"} with _id=1
{"column1": "B"} with _id=2
When run a POST index1/_search with following body -
{"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{"term": {"_id": "1"}},
{"term": {"column1": "A"}}
]
}
},
{
"bool": {
"must": [
{"term": {"_id": "2"}},
{"term": {"column1": "B"}}
]
}
}
]
}
}}
I get no results. What I was expecting was both documents to be returned. If I remove one of the column1 term matches, I will get the other document. If I remove both column1 term matches, I get both documents?
What am I doing wrong?
As #Val already suggested, you should have used column1.keword, if you are taking default mapping of column1
Search Query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"term": {
"_id": "1"
}
},
{
"term": {
"column1.keyword": "A"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_id": "2"
}
},
{
"term": {
"column1.keyword": "B"
}
}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "fd_cb",
"_type": "_doc",
"_id": "1",
"_score": 1.6931472,
"_source": {
"column1": "A"
}
},
{
"_index": "fd_cb",
"_type": "_doc",
"_id": "2",
"_score": 1.6931472,
"_source": {
"column1": "B"
}
}
]
If you have already defined explicit mapping for column1, like the one shown below :
{
"mappings": {
"properties": {
"column1": {
"type": "keyword"
}
}
}
}
Then the search query, mentioned in your question works perfectly fine.
Related
I'm trying to perform a search by country, or city or both using ElasticSearch.
When I perform a search by country using USA as the search term, I get these results, which are correct.
email
country
city
mike#example.com
USA
Portland
You Can Also
USA
Chicago
The query looks like so:
{
"body": {
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"match": {
"country": {
"query": "USA",
"operator": "and"
}
}
}
]
}
}
]
}
}
}
}
The problem is that if I want to also search by city using the term Portland I'm expecting to get only one result
email
country
city
mike#example.com
USA
Portland
but I get both results again, just like when I'm only searching by country.
The query for both fields looks like this:
{
"body": {
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"match": {
"country": {
"query": "USA",
"operator": "and"
}
}
}
]
}
},
{
"bool": {
"should": [
{
"match": {
"country": {
"query": "USA",
"operator": "and"
}
}
},
{
"match": {
"city": {
"query": "Portland",
"operator": "and"
}
}
}
]
}
}
]
}
}
}
}
What am I doing wrong?
There is no need to use multiple bool/should clause and operator (with match query), in your case.
Adding a working example
Search by country
{
"query": {
"bool": {
"must": {
"match": {
"country": {
"query": "USA"
}
}
}
}
}
}
Search Result:
"hits": [
{
"_index": "67676851",
"_type": "_doc",
"_id": "1",
"_score": 0.18232156,
"_source": {
"email": "mike#example.com",
"country": "USA",
"city": "Portland"
}
},
{
"_index": "67676851",
"_type": "_doc",
"_id": "2",
"_score": 0.18232156,
"_source": {
"email": "You Can Also",
"country": "USA",
"city": "Chicago"
}
}
]
Search by city:
{
"query": {
"bool": {
"must": {
"match": {
"city": {
"query": "Portland"
}
}
}
}
}
}
Search Result:
"hits": [
{
"_index": "67676851",
"_type": "_doc",
"_id": "1",
"_score": 0.6931471,
"_source": {
"email": "mike#example.com",
"country": "USA",
"city": "Portland"
}
}
]
Search by city and country:
{
"query": {
"bool": {
"must": [
{
"match": {
"city": "Portland"
}
},
{
"match": {
"country": "USA"
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "67676851",
"_type": "_doc",
"_id": "1",
"_score": 0.8754687,
"_source": {
"email": "mike#example.com",
"country": "USA",
"city": "Portland"
}
}
]
suppose I have two indexed documents in my elasticsearch cluster --
{"APPLE": "TEST"}
{"TEST": "APPLE"}
Suppose I want to search on an APPLE term. What query would return to me both of these documents?
You can use match query to search for documents having APPLE as the field value, and exists query to search for documents having field APPLE. Try out this below query
{
"query": {
"bool": {
"should": [
{
"match": {
"TEST": "APPLE"
}
},
{
"exists": {
"field": "APPLE"
}
}
]
}
}
}
Search Result will be
"hits": [
{
"_index": "67048809",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"APPLE": "TEST"
}
},
{
"_index": "67048809",
"_type": "_doc",
"_id": "2",
"_score": 0.2876821,
"_source": {
"TEST": "APPLE"
}
}
]
Update 1:
You can query the field's existence with wildcard characters also.
{
"query": {
"bool": {
"should": [
{
"match": {
"TEST": "APPLE"
}
},
{
"exists": {
"field": "*APPLE*" // note this
}
}
]
}
}
}
I'm very new to elastic search, how do I write a query which search for a keyword (ie. test keyword) in all fields in the document, and one more keyword which search in a specific field.
this can be done using query_string but we can't do search in nested fields with nested field specified, So i'm using LUQUM to convert lucene query to Elasticsearch DSL.
Below is the sample usecase:
I have a mapping:
"mappings": {
"properties": {
"grocery_name":{
"type": "text"
},
"items": {
"type": "nested",
"properties": {
"name": {
"type": "text"
},
"stock": {
"type": "integer"
},
"category": {
"type": "text"
}
}
}
}
}
}
and the data looks like below
{
"grocery_name": "Elastic Eats",
"items": [
{
"name": "Red banana",
"stock": "12",
"category": "fruit"
},
{
"name": "Cavendish banana",
"stock": "10",
"category": "fruit"
},
{
"name": "peach",
"stock": "10",
"category": "fruit"
},
{
"name": "carrot",
"stock": "9",
"category": "vegetable"
},
{
"name": "broccoli",
"stock": "5",
"category": "vegetable"
}
]
}
How can I query to get all items where the item name matches banana from grocery_name: Elastic Eats ?
tried with * and _all, it didn't work.
example query:
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"match": {
"*": {
"query": "banana",
"zero_terms_query": "all"
}
}
}
]
}
}
}
I'm sure I'm missing something obvious, but I have read the manual and I'm getting no joy at all.
UPDATE:
enabling include_in_parent for nested object works for below query, but it will internally duplicates data which will definitely impact on memory.
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"multi_match": {
"query": "banana"
}
}
]
}
}
}
Is there any other way to do this?
You need to use a nested match query with inner_hits resulting in an inner nested query to automatically match the relevant nesting level, rather than root
Search Query
{
"query": {
"bool": {
"filter": [
{
"term": {
"grocery_name": "elastic"
}
},
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"match": {
"items.name": "banana"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
Search Result:
"inner_hits": {
"items": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.744874,
"hits": [
{
"_index": "stof_64273970",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "items",
"offset": 0
},
"_score": 0.744874,
"_source": {
"name": "Red banana",
"stock": "12",
"category": "fruit"
}
},
{
"_index": "stof_64273970",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "items",
"offset": 1
},
"_score": 0.744874,
"_source": {
"name": "Cavendish banana",
"stock": "10",
"category": "fruit"
}
}
]
}
Update 1:
On the basis of your comments, you can use multi match query, for your use case
If no fields are provided, the multi_match query defaults to the
index.query.default_field index settings, which in turn defaults to *.
(*) extracts all fields in the mapping that are eligible to term queries and filters the metadata fields. All extracted fields are then
combined to build a query.
Search Query:
{
"query": {
"bool": {
"filter": [
{
"term": {
"grocery_name": "elastic"
}
},
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "banana" <-- note this
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
Update 2:
You need to use a combination of multiple bool queries like this:
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"multi_match": {
"query": "banana"
}
}
]
}
},
{
"bool": {
"must": [
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "banana"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
]
}
}
]
}
}
}
I am trying to exclude some field with source filtering.
I create an index:
put testindex
{
"mappings": {
"type1": {
"properties":{
"name": { "type": "text" },
"age": { "type": "integer" }
}
}
}
}
insert a document:
put testindex/type1/a
{
"name":"toto",
"age":23
}
and try a filtered query:
get testindex/_search
{
"_source": {
"excludes": [ "age" ]
},
"query": {
"bool": {
"should": []
}
}
}
the result is:
"hits": [
{
"_index": "testindex",
"_type": "type1",
"_id": "a",
"_score": 1,
"_source": {
"name": "toto",
"age": 23
}
}
]
I don't understand why it does not hide the "age" field in _source.
_source: false give the same result.
I used elasticsearch & kibana 5.6
Ok I found it.
It's probably due to Kibana.
When I use lowercase for the "get". It does not work.
get testindex/_search
{
"_source": {
"excludes": [ "age" ]
},
"query": {
"bool": {
"should": []
}
}
}
When I use uppercase, it work. I don't really know why but that's it..
GET testindex/_search
{
"_source": {
"excludes": [ "name" ]
},
"query": {
"bool": {
"should": []
}
}
}
elastic version: 5.0.1
define mappingļ¼
PUT test
{
"mappings": {
"my_parent": {
"properties": {
"key": {
"type": "keyword"
}
}
},
"my_child": {
"_parent": {
"type": "my_parent"
},
"properties": {
"key": {
"type": "keyword"
}
}
}
}
}
add demo data:
POST _bulk
{"update": {"_index": "test","_type": "my_parent","_id": "1"}}
{"doc": {"key": 1},"doc_as_upsert": true}
{"update": {"_index": "test","_type": "my_child","_parent": 1,"_id": "11"}}
{"doc": {"key": 11},"doc_as_upsert": true}
{"update": {"_index": "test","_type": "my_child","_parent": 1,"_id": "12"}}
{"doc": {"key": 12},"doc_as_upsert": true}
query:
POST test/my_parent/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"key": 3
}
},
{
"has_child": {
"type": "my_child",
"inner_hits": {
"name": "a"
},
"query": {
"term": {
"key": 11
}
}
}
}
]
}
},
{
"has_child": {
"type": "my_child",
"inner_hits": {
"name": "b"
},
"query": {
"term": {
"key": 12
}
}
}
}
]
}
}
}
}
}
result:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0,
"hits": [
{
"_index": "test",
"_type": "my_parent",
"_id": "1",
"_score": 0,
"_source": {
"key": 1
},
"inner_hits": {
"a": {
"hits": {
"total": 1,
"max_score": 0.9808292,
"hits": [
{
"_type": "my_child",
"_id": "11",
"_score": 0.9808292,
"_routing": "1",
"_parent": "1",
"_source": {
"key": 11
}
}
]
}
},
"b": {
"hits": {
"total": 1,
"max_score": 0.9808292,
"hits": [
{
"_type": "my_child",
"_id": "12",
"_score": 0.9808292,
"_routing": "1",
"_parent": "1",
"_source": {
"key": 12
}
}
]
}
}
}
}
]
}
}
question here:
Do 'must'\'should'\'must_not' clause have the same meaning between plain search and parent\child search?
Why the result of inner_hits with name 'a' is returned?
'must'|'should'|'must_not' clauses have different meaning. Let me explain you with example of the plain search.
Understand these clause with equivalent SQL query.
must: The clause (query) must appear in matching documents and will contribute to the score.
SQL: select * from user where country_code = 'US' AND state_code = 'NY'
Query DSL:
POST _search
{
"query": {
"bool": {
"must": [
{"term": {"country_code": "US"}},
{"term": {"state_code": "NY"}}
]
}
}
}
should: At least one of these clauses must match, like logical OR.
SQL: select * from user where country_code = 'US' OR state_code = 'NY'
Query DSL:
POST _search
{
"query": {
"bool": {
"should": [
{"term": {"country_code": "US"}},
{"term": {"state_code": "NY"}}
]
}
}
}
must_not: Condition must not match the documents.
SQL: select * from user where country_code != 'US' AND state_code != 'NY'
Query DSL:
POST _search
{
"query": {
"bool": {
"must_not": [
{"term": {"country_code": "US"}},
{"term": {"state_code": "NY"}}
]
}
}
}
Why the result of inner_hits with name 'a' is returned?
Because you put two has_child condition inside the should filter. As explain above it is matching the document from (inner_hits.name =a ..) OR ( inner_hits.name=b ..)