I can't understand why I have no results? Using ES 2.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"term": {
"technical.techcolor": "red"
}
}
}
}
And here is the info from db that I am searching against.
{"technical":
[{
"techname22": "test",
"techcolor":"red",
"techlocation": "usa"
}],
"audio":
{
"someAudioMetadata": "test"
}
}
Since you have not specified your mapping, I am considering the following mapping.
Mapping:
{
"mappings": {
"company": {
"properties": {
"technical": {
"type": "nested"
}
}
}
}
}
Search Query:
{
"query": {
"filtered": {
"query": {
"match_all": {
}
},
"filter": {
"nested": {
"path": "technical",
"filter": {
"term": {
"technical.techcolor": "red"
}
}
}
}
}
}
}
Search Result:
"hits": {
"total": 1,
"max_score": 1.0,
"hits": [
{
"_index": "demos",
"_type": "company",
"_id": "1",
"_score": 1.0,
"_source": {
"technical": [
{
"techname22": "test",
"techcolor": "red",
"techlocation": "usa"
}
],
"audio": {
"someAudioMetadata": "test"
}
}
}
]
}
To know more about nested datatype you can refer to this official documentation and for Query and Filter Context refer this
Related
I am stuck on one of my tasks.
Overview:
There are some records on elastic search. Which includes information about the candidates and their employment.
There is a field that stores information about the statuses in which the candidate got submitted.
{
"submittedJobs": [
{
"status": "PendingPM", "jobId": "ABC", ...
},
{
"status": "PendingClient", "jobId": "XYZ", ...
},
{
"status": "PendingPM", "jobId": "WXY", ...
},
...
]
}
I want to write an es query to fetch all the records in which submitted jobs array "only" have "pendingPM" statuses and no other statuses.
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "submittedJobs",
"query": {
"bool": {
"must": [
{
"term": {
"submittedJobs.status.keyword": "PendingPM"
}
}
]
}
}
}
}
]
}
}
I tried this query, and it returns the records which include "pendingPM" along with other statuses - might use contains() logic.
here is the mapping
"submittedJobs": {
"type": "nested",
"properties": {
"statusId": {
"type": "long"
},
"status": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256,
"normalizer": "lowercase_normalizer"
}
}
},
"jobId": {
"type": "keyword"
}
}
}
For example. let's suppose there are two documents
document #1:
{
"submittedJobs": [
{
"status": "PendingPM", "jobId": "ABC", ...
},
{
"status": "PendingClient", "jobId": "XYZ", ...
},
{
"status": "PendingPM", "jobId": "WXY", ...
},
...
]
},
document #2:
{
"submittedJobs": [
{
"status": "PendingPM", "jobId": "ABC", ...
},
{
"status": "PendingPM", "jobId": "WXY", ...
},
...
]
}
Only document #2 should be returned, as the entire array contains only "PendingPM" and no other statuses.
Document #1 will be filtered-out since it includes mixed statuses.
Any help will be appreciated.
Try this:
Will be return only document with all item of array with status PendingPM.
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "submittedJobs",
"query": {
"bool": {
"must_not": [
{
"match": {
"submittedJobs.status": {
"query": "PendingPM"
}
}
},
{
"match": {
"submittedJobs.status": {
"query": "PendingClient"
}
}
}
]
}
}
}
}
]
}
}
}
You can use inner_hits along with nested query to get only the matched results from the document
Adding a working example
Index Mapping:
{
"mappings": {
"properties": {
"submittedJobs": {
"type": "nested"
}
}
}
}
Search Query:
{
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "submittedJobs",
"query": {
"bool": {
"must": [
{
"term": {
"submittedJobs.status.keyword": "PendingPM"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
Search Result would be:
"hits": [
{
"_index": "73062439",
"_id": "1",
"_score": 0.0,
"_source": {
"submittedJobs": [
{
"status": "PendingPM",
"jobId": "ABC"
},
{
"status": "PendingClient",
"jobId": "XYZ"
},
{
"status": "PendingPM",
"jobId": "WXY"
}
]
},
"inner_hits": { // note this
"submittedJobs": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.4700036,
"hits": [
{
"_index": "73062439",
"_id": "1",
"_nested": {
"field": "submittedJobs",
"offset": 0
},
"_score": 0.4700036,
"_source": {
"jobId": "ABC",
"status": "PendingPM"
}
},
{
"_index": "73062439",
"_id": "1",
"_nested": {
"field": "submittedJobs",
"offset": 2
},
"_score": 0.4700036,
"_source": {
"jobId": "WXY",
"status": "PendingPM"
}
}
]
}
}
}
}
]
Having this template (abbreviated version).
{
"index_patterns": "index_pattern*",
"order": 1,
"version": 1,
"aliases": {
"some_alias": {}
},
"settings": {
"number_of_shards": 5,
},
"mappings": {
"dynamic": "false",
"properties": {
"someId": {
"type": "keyword"
},
"audience": {
"type": "object",
"properties": {
....
"ageRanges": {
"type": "nested",
"properties": {
"ageTo": {
"type": "integer"
},
"ageFrom": {
"type": "integer"
}
}
}
}
}
}
}
}
I need to query if the audience.ageRanges does not exist or if it does exist apply other filters.
Let's say we want to search if a document with specific someId value fits into the audience.ageRanges query clauses (removed for clarity).
It has some audience properties but no ageRanges.
"audience": {
"genders": [
"any"
],
"deviceType": "any"
}
Shouldn't the below query return the specific document?
{
"query": {
"bool": {
"must": [
{
"term": {
"someId": {
"value": "03183f31"
}
}
},
{
"nested": {
"path": "audience.ageRanges",
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "audience.ageRanges"
}
}
]
}
}
}
}
]
}
}
}
My results are 0, it is a bit confusing how it works.
Trying with a document id that does have audience.ageRanges items and changing the must_not nested query to must will return results.
Instead of putting must_not inside the nested query, you should put the nested query inside the must_not.
Consider a sample index data as
{
"someId":123,
"audience": {
"genders": [
"any"
],
"deviceType": "any"
}
}
You need to modify your search query as shown below -
Search Query:
{
"query": {
"bool": {
"must": [
{
"term": {
"someId": {
"value": "123"
}
}
},
{
"bool": {
"must_not": {
"nested": {
"path": "audience.ageRanges",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "audience.ageRanges"
}
}
]
}
}
}
}
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "65852173",
"_type": "_doc",
"_id": "1",
"_score": 0.2876821,
"_source": {
"someId": 123,
"audience": {
"genders": [
"any"
],
"deviceType": "any"
}
}
}
]
This is the query to get the Top 10 records. There is a Field name Answer inside this we have a record "UNHANDLED". I want to exclude the UNHANDLED inside the Answer field.
How to write the query to get both Top 10 and Exclude UNHANDLED
GET /logstash-sdc-mongo-abcsearch/_search?size=0
{
"aggs": {
"top_tags": {
"terms": {
"field": "question.keyword"
},
"aggs": {
"top_faq_hits": {
"top_hits": {
"_source": {
"includes": [
"answer"
]
},
"size": 1
}
}
}
}
}
}
You can use the must_not clause, to exclude the documents that containsUNHANDLED in the answer field. Try out the below query -
Index Mapping:
{
"mappings": {
"properties": {
"question": {
"type": "keyword"
},
"answer": {
"type": "keyword"
}
}
}
}
Index Data:
{
"question": "a",
"answer": "b"
}
{
"question": "c",
"answer": "UNHANDLED"
}
Search Query:
{
"query": {
"bool": {
"must_not": {
"term": {
"answer": "UNHANDLED"
}
}
}
},
"aggs": {
"top_tags": {
"terms": {
"field": "question"
},
"aggs": {
"top_faq_hits": {
"top_hits": {
"_source": {
"includes": [
"answer"
]
},
"size": 1
}
}
}
}
}
}
Search Result:
"aggregations": {
"top_tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "a",
"doc_count": 1,
"top_faq_hits": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.0,
"hits": [
{
"_index": "65563925",
"_type": "_doc",
"_id": "1",
"_score": 0.0,
"_source": {
"answer": "b"
}
}
]
}
}
}
]
}
}
Update 1:
Based on the comments below, try out the below query:
{
"query": {
"bool": {
"must_not": {
"term": {
"answer": "UNHANDLED"
}
},
"must": {
"term": {
"source": "sonax"
}
}
}
},
"aggs": {
"top_tags": {
"terms": {
"field": "question"
},
"aggs": {
"top_faq_hits": {
"top_hits": {
"_source": {
"includes": [
"answer"
]
},
"size": 1
}
}
}
}
}
}
I'm very new to elastic search, how do I write a query which search for a keyword (ie. test keyword) in all fields in the document, and one more keyword which search in a specific field.
this can be done using query_string but we can't do search in nested fields with nested field specified, So i'm using LUQUM to convert lucene query to Elasticsearch DSL.
Below is the sample usecase:
I have a mapping:
"mappings": {
"properties": {
"grocery_name":{
"type": "text"
},
"items": {
"type": "nested",
"properties": {
"name": {
"type": "text"
},
"stock": {
"type": "integer"
},
"category": {
"type": "text"
}
}
}
}
}
}
and the data looks like below
{
"grocery_name": "Elastic Eats",
"items": [
{
"name": "Red banana",
"stock": "12",
"category": "fruit"
},
{
"name": "Cavendish banana",
"stock": "10",
"category": "fruit"
},
{
"name": "peach",
"stock": "10",
"category": "fruit"
},
{
"name": "carrot",
"stock": "9",
"category": "vegetable"
},
{
"name": "broccoli",
"stock": "5",
"category": "vegetable"
}
]
}
How can I query to get all items where the item name matches banana from grocery_name: Elastic Eats ?
tried with * and _all, it didn't work.
example query:
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"match": {
"*": {
"query": "banana",
"zero_terms_query": "all"
}
}
}
]
}
}
}
I'm sure I'm missing something obvious, but I have read the manual and I'm getting no joy at all.
UPDATE:
enabling include_in_parent for nested object works for below query, but it will internally duplicates data which will definitely impact on memory.
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"multi_match": {
"query": "banana"
}
}
]
}
}
}
Is there any other way to do this?
You need to use a nested match query with inner_hits resulting in an inner nested query to automatically match the relevant nesting level, rather than root
Search Query
{
"query": {
"bool": {
"filter": [
{
"term": {
"grocery_name": "elastic"
}
},
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"match": {
"items.name": "banana"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
Search Result:
"inner_hits": {
"items": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 0.744874,
"hits": [
{
"_index": "stof_64273970",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "items",
"offset": 0
},
"_score": 0.744874,
"_source": {
"name": "Red banana",
"stock": "12",
"category": "fruit"
}
},
{
"_index": "stof_64273970",
"_type": "_doc",
"_id": "1",
"_nested": {
"field": "items",
"offset": 1
},
"_score": 0.744874,
"_source": {
"name": "Cavendish banana",
"stock": "10",
"category": "fruit"
}
}
]
}
Update 1:
On the basis of your comments, you can use multi match query, for your use case
If no fields are provided, the multi_match query defaults to the
index.query.default_field index settings, which in turn defaults to *.
(*) extracts all fields in the mapping that are eligible to term queries and filters the metadata fields. All extracted fields are then
combined to build a query.
Search Query:
{
"query": {
"bool": {
"filter": [
{
"term": {
"grocery_name": "elastic"
}
},
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "banana" <-- note this
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
Update 2:
You need to use a combination of multiple bool queries like this:
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"grocery_name": {
"query": "Elastic Eats"
}
}
},
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"multi_match": {
"query": "banana"
}
}
]
}
},
{
"bool": {
"must": [
{
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "banana"
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
]
}
}
]
}
}
}
I have a multi-level nested document. I want to query based on multiple nested queries with all the conditions must match.
Example
Document 1
{
"publishing_rule": {
"publishing_orders": [{
"transporters": [{
"fteid": "81"
}],
"active": false
},
{
"transporters": [{
"fteid": "82"
}],
"active": true
}
]
}
}
Document 2
{
"publishing_rule": {
"publishing_orders": [{
"transporters": [{
"fteid": "81"
}],
"active": true
},
{
"transporters": [{
"fteid": "82"
}],
"active": false
}
]
}
}
I want to fetch all the documents which match below condition
publishing_rule.publishing_orders.active = true
AND
publishing_rule.publishing_orders.transporters.fteid = '81'
Both active and transporters.fteid should be part of same object.
I have tried creating below mapping
{
"mappings": {
"_doc": {
"properties": {
"publishing_rule.publishing_orders": {
"type": "nested",
"properties": {
"transporters": {
"type": "nested"
}
}
}
}
}
}
}
And used below query
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "publishing_rule.publishing_orders",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.active": true
}
}
]
}
}
}
},
{
"nested": {
"path": "publishing_rule.publishing_orders.transporters",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.transporters.fteid": "81"
}
}
]
}
}
}
}
]
}
}
}
But I am not getting expected result. The query returning both the documents.
I am expecting only Document 2 in the result.
Your query actually will look at any document that matches either active = true or fteid = 81 but not both of them. Those criteria are met in document 1 and document 2. That's why you got those two.
This query should work
{
"query": {
"nested": {
"path": "publishing_rule.publishing_orders",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.active": true
}
},
{
"nested": {
"path": "publishing_rule.publishing_orders.transporters",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.transporters.fteid": "81"
}
}
]
}
}
}
}
]
}
}
}
}
}
Notice that I use a single nested as entry point then inner nested, this is to enable two filters altogether to search in the document.
UPDATE
mapping
GET /myindex/_mapping
{
"mappings": {
"_doc": {
"properties": {
"publishing_rule": {
"properties": {
"publishing_orders": {
"type": "nested",
"properties": {
"active": {
"type": "boolean"
},
"transporters": {
"type": "nested",
"properties": {
"fteid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
}
}
}
ES result
Document 2
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.89712,
"hits": [
{
"_index": "myindex",
"_type": "_doc",
"_id": "AWzu__bgqsCjtPMt7kG_",
"_score": 1.89712,
"_source": {
"publishing_rule": {
"publishing_orders": [
{
"transporters": [
{
"fteid": "81" // matched
}
],
"active": true // matched
},
{
"transporters": [
{
"fteid": "82"
}
],
"active": true
}
]
}
}
}
]
}
}
Hope it helps
I saw that #deerawan answer is adequate to the question that you have asked. Since you didn't accept his answer, I believe what you are looking for is to get the nested document alone in the result. I have modified #deerawan's query to include the nested document alone that gets matched by the query
{
"_source": false,
"query": {
"nested": {
"path": "publishing_rule.publishing_orders",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.active": true
}
},
{
"nested": {
"path": "publishing_rule.publishing_orders.transporters",
"query": {
"bool": {
"must": [
{
"match": {
"publishing_rule.publishing_orders.transporters.fteid": "81"
}
}
]
}
}
}
}
]
}
},
"inner_hits": {}
}
}
}
This should give you the below response
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.3862944,
"hits": [
{
"_index": "nest",
"_type": "doc",
"_id": "1234",
"_score": 1.3862944,
"inner_hits": {
"publishing_rule.publishing_orders": {
"hits": {
"total": 1,
"max_score": 1.3862944,
"hits": [
{
"_nested": {
"field": "publishing_rule.publishing_orders",
"offset": 0
},
"_score": 1.3862944,
"_source": {
"transporters": [
{
"fteid": "81"
}
],
"active": true
}
}
]
}
}
}
}
]
}
}