Query with filter array field - elasticsearch

Question

I want to return documents that include only some of array field members.
For example, I have of two order documents:\
{
"orderNumber":"ORD-111",
"items":[{"name":"part-1","status":"new"},
{"name":"part-2","status":"paid"}]
}
{
"orderNumber":"ORD-112",
"items":[{"name":"part-3","status":"paid"},
{"name":"part-4","status":"supplied"}]
}
I want to create a query so that my result will include all the order documents but only with items that match {"status":"supplied"}.
The result should look like:\
{
"orderNumber":"ORD-111",
"items":[]
}
{
"orderNumber":"ORD-112",
"items":[{"name":"part-4","status":"supplied"}]
}

Answer 1

You can use a nested query along with inner_hits to get only matching array values in the result
Adding a working example
Index Mapping:
{
"mappings": {
"properties": {
"items": {
"type": "nested"
}
}
}
}
Search Query:
{
"query": {
"nested": {
"path": "items",
"query": {
"bool": {
"must": [
{
"match": {
"items.status": "supplied"
}
}
]
}
},
"inner_hits": {}
}
}
}
Search Result:
"hits": [
{
"_index": "67890614",
"_type": "_doc",
"_id": "2",
"_score": 1.2039728,
"_source": {
"orderNumber": "ORD-112",
"items": [
{
"name": "part-3",
"status": "paid"
},
{
"name": "part-4",
"status": "supplied"
}
]
},
"inner_hits": {
"items": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.2039728,
"hits": [
{
"_index": "67890614",
"_type": "_doc",
"_id": "2",
"_nested": {
"field": "items",
"offset": 1
},
"_score": 1.2039728,
"_source": {
"name": "part-4",
"status": "supplied" // note this
}
}
]
}
}
}
}
]

Answer 2

Elasticsearch flats the matching field so is unable to tell which was the actual element in the array that matches.
As previously answered you could use nested queries.
How arrays of objects are flattened
Elasticsearch has no concept of inner objects. Therefore, it flattens object hierarchies into a simple list of field names and values. For instance, consider the following document:
PUT my-index-000001/_doc/1
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
The user field is dynamically added as a field of type object.
The previous document would be transformed internally into a document that looks more like this:
{
"group" : "fans",
"user.first" : [ "alice", "john" ],
"user.last" : [ "smith", "white" ]
}
The user.first and user.last fields are flattened into multi-value fields, and the association between alice and white is lost. This document would incorrectly match a query for alice AND smith:
GET my-index-000001/_search
{
"query": {
"bool": {
"must": [
{ "match": { "user.first": "Alice" }},
{ "match": { "user.last": "Smith" }}
]
}
}
}
To answer your question:
If you need to index arrays of objects and to maintain the independence of each object in the array, use the nested data type instead of the object data type.
Internally, nested objects index each object in the array as a separate hidden document, meaning that each nested object can be queried independently of the others with the nested query:
PUT my-index-000001
{
"mappings": {
"properties": {
"user": {
"type": "nested"
}
}
}
}
PUT my-index-000001/_doc/1
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
GET my-index-000001/_search
{
"query": {
"nested": {
"path": "user",
"query": {
"bool": {
"must": [
{ "match": { "user.first": "Alice" }},
{ "match": { "user.last": "Smith" }}
]
}
}
}
}
}
GET my-index-000001/_search
{
"query": {
"nested": {
"path": "user",
"query": {
"bool": {
"must": [
{ "match": { "user.first": "Alice" }},
{ "match": { "user.last": "White" }}
]
}
},
"inner_hits": {
"highlight": {
"fields": {
"user.first": {}
}
}
}
}
}
}
The user field is mapped as type nested instead of type object.
This query doesn’t match because Alice and Smith are not in the same nested object.
This query matches because Alice and White are in the same nested object.
inner_hits allow us to highlight the matching nested documents.
Interacting with nested documents
Nested documents can be:
queried with the nested query.
analyzed with the nested and reverse_nested aggregations.
sorted with nested sorting.
retrieved and highlighted with nested inner hits.
Because nested documents are indexed as separate documents, they can only be accessed within the scope of the nested query, the nested/reverse_nested aggregations, or nested inner hits.
consider performance when taking this approach as it is by magnitudes more expensive.
for more details
ou can check the source:
https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html