elasticsearch nested functionScoreQuery cannot access parent properties - elasticsearch

I have a type in elasticsearch that looks like this:
"hotel" : {
"field" : 1,
"rooms" : [
{
"type" : "single",
"magicScore" : 1
},
{
"type" : "double",
"magicScore" : 2
}
]
}
where rooms is of type nested. I sort using a nested functionScoreQuery:
{
"query" : {
"filtered" : {
"query" : {
"nested" : {
"query" : {
"function_score" : {
"filter" : {
"match_all" : { }
},
"functions" : [ {
"script_score" : {
"script" : "return doc['hotel.field'].value"
}
} ]
}
},
"path" : "rooms",
"score_mode" : "max"
}
}
}
}
Problem is hotel.field returns 0 always. Is there a way to access the parent field inside a nested query? I know I can always pack the field inside the nested document but its a hack not a solution. Would using a dismax query help me? https://discuss.elastic.co/t/nested-value-on-function-score/29935
The query I am actually using looks something like this:
{
"query" : {
"bool" : {
"must" : {
"nested" : {
"query" : {
"function_score" : {
"query" : {
"not" : {
"query" : {
"terms" : {
"rooms.type" : [ "single", "triple" ]
}
}
}
},
"functions" : [ {
"script_score" : {
"script" : {
"inline" : "return doc['rooms.magicScore'].value;",
"lang" : "groovy",
"params" : {
"ratings" : {
"sample" : 0.5
},
"variable" : [ 0.0, 0.0, 0.0, 0.0, -0.5, -2.5]
}
}
}
} ],
"score_mode" : "max"
}
},
"path" : "rooms"
}
},
"filter" : {
"bool" : {
"filter" : [ {
"bool" : {
"should" : [ {
"term" : {
"cityId" : "166"
}
}, {
"term" : {
"cityId" : "165"
}
} ]
}
}, {
"nested" : {
"query" : {
"not" : {
"query" : {
"terms" : {
"rooms.type" : [ "single", "triple" ]
}
}
}
},
"path" : "rooms"
}
} ]
}
}
}
}
}
What I am trying to achieve is to access for example the cityId inside the function_score query which is nested.

The question is why are you accessing the parent values in a nested query. Once you are in the nested context, you cannot access parent fields or other fields from other nested fields.
From the documentation:
The nested clause “steps down” into the nested comments field. It no longer has access to fields in the root document, nor fields in any other nested document.
So, rewrite your queries so that the nested part touches the fields in that nested field and anything else is accessed outside the nested part.

Related

Elasticsearch DSL for all null values between 2 timestamps

I have this JSON structure in Elasticsearch. I am having trouble creating a DSL to search for all null values of awsKafkaTimestamp that are in between a post.timestamp range of A and B. How can I do this?
{
"tracer": {
"post": {"timestamp": 123123},
"awsKafkaTimestamp": null,
"qdcKafkaTimestamp": null
}
}
Try this: (works for ES 2.4 - will not work for 2.2 and below)
{
"fields" : ["your_field"],
"query" : {
"bool" : {
"must_not" : {
"exists" : {
"field" : "awsKafkaTimestamp "
}
},
"must" : [{
"nested" : {
"path" : "post",
"filter" : {
"bool" : {
"must" : {
"range" : {
"post.timestamp" : {
"lte" : A,
"gte" : B
}
}
}
}
}
}
}
]
}
}
}

Elasticsearch bool filter for multiple conditions on same element of array

I'm trying to create a query/filter that matches a document only if a number of conditions are met on the same item of an array.
Let's say this is the document:
{
arr: [
{ "f1" : "a" , f2 : true },
{ "f1" : "b" , f2 : false}
]
}
I want to be able to retrieve documents that have N conditions matching on the same element. For example: arr.f1 == "a" AND arr.f2 == true should match the document but arr.f1 == "b" AND arr.f2 == true should not.
I'm trying nested bool filters (I have other filters apart from this one) but it doesn't work, something in the lines of
"bool" : {
"must": [
{ some other filter },
{"bool": {"must" : [
{"term" : {"arr.f1" : "a"}},
{"term" : {"arr.f2" : true}},
] }}
]
}
Any idea how to do that?
thanks
edit:
I changed the mapping and now a nested query works as per Val's response. I'm now not able to do an "exists" filter on the nested field:
A simple { "filter" : {"exists" : { "field" : "arr" } } } search returns no hits. How do I do that?
edit: It looks like I need to do a nested exists filter to check that a field inside the nested object exists.
something like:
"filter" : {
"nested" : {"path" : "arr", "filter" : {"exists" : { "field" : "f1" } }}
}
edit:
argh - now highlight doesn't work anymore:
"highlight" : {
"fields" : [
{"arr.f1" : {}},
]
}
Worked around that by adding include_in_parent : true and querying both the nested field and the root object. It's just awful. If anyone has a better idea, they're more than welcome!
{
"query" : {
"bool" : {
"must": [
{"term" : { "arr.f1" : "a" }},
{ "nested" : { "path" : "arr",
"query" : { "bool" : { "must" : [
{"term" : { "arr.f1" : "a" }},
{"term" : { "arr.f2" : true }}
] } }
}}
]
}
},
"highlight" : {
"fields" : [
{"arr.f1" : {}},
]
}
}
In case you're wondering: it's legacy stuff. I can't reindex right now (that would be the obvious solution) and I need a quick & dirty workaround
You need to set the type of your arr field as nested like this:
{
"your_type": {
"properties": {
"arr": {
"type": "nested",
"properties": {
"f1": {"type":"string"},
"f2": {"type":"boolean"}
}
}
}
}
}
Then you need to use a nested query:
{
"nested" : {
"path" : "arr",
"query" : {
"bool" : {
"must" : [
{
"term" : {"arr.f1" : "a"}
},
{
"term" : {"arr.f2" : true}
}
]
}
}
}
}
Your exists filter needs to specify the full field path
"filter" : {
"nested" : {"path" : "arr", "filter" : {"exists" : { "field" : "arr.f1" } }}
}

Elasticsearch match_phrase doesn't perform the same as multi_match with type phrase?

I'm having some trouble turning a match_phrase query into a multi_match query for multiple fields. My original query:
{
"from" : 0,
"size" : 50,
"query" : {
"filtered" : {
"query" : {
"match_phrase" : {
"metadata.description" : "Search Terms"
}
},
"filter" : {
"bool" : {
"must" : [ {
"terms" : {
"collectionId" : [ "1", "2" ]
}
} ]
}
}
}
}
}
Returns results correctly, but when I rewrite the match_phrase piece as a multi_match to run against multiple fields:
{
"from" : 0,
"size" : 50,
"query" : {
"filtered" : {
"query" : {
"multi_match" : {
"query" : "Search Terms",
"fields" : [ "metadata.description", "metadata.title" ],
"type" : "phrase"
}
},
"filter" : {
"bool" : {
"must" : [ {
"terms" : {
"collectionId" : [ "1", "2" ]
}
} ]
}
}
}
}
}
I am not getting any results. Is there anything obvious I am doing wrong here?
EDIT:
It must be something to do with the filter, as
{
"from" : 0,
"size" : 50,
"query" : {
"match_phrase" : {
"metadata.description" : "Search Terms"
}
}
}
and
{
"from" : 0,
"size" : 50,
"query" : {
"multi_match" : {
"query" : "Search Terms",
"fields" : [ "metadata.description", "metadata.title" ],
"type" : "phrase"
}
}
}
both perform as expected.
I am not sure why, exactly, but not using a filtered query, and applying the filter at the top level
{
"from" : 0,
"size" : 50,
"query" : {
"multi_match" : {
"query" : "Search Terms",
"fields" : [ "metadata.description", "metadata.title" ],
"type" : "phrase"
}
},
"filter" : {
"bool" : {
"must" : [ {
"terms" : {
"collectionId" : [ "1", "2" ]
}
} ]
}
}
}
resolves the problem.

How to exclude a filter from a facet?

I have come from a Solr background and am trying to find the equivalent of "tagging" and "excluding" in Elasticsearch.
In the following example, how can I exclude the price filter from the calculation of the prices facet? In other words, the prices facet should take into account all of the filters except for price.
{
query : {
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"and" : [
{
"term" : {
"colour" : "Red"
}
},
{
"term" : {
"feature" : "Square"
}
},
{
"term" : {
"feature" : "Shiny"
}
},
{
"range" : {
"price" : {
"from" : "10",
"to" : "20"
}
}
}
]
}
}
},
"facets" : {
"colours" : {
"terms" : {
"field" : "colour"
}
},
"features" : {
"terms" : {
"field" : "feature"
}
},
"prices" : {
"statistical" : {
"field" : "price"
}
}
}
}
You can apply price filter as a top level filter to your query and add it to all facets expect prices as a facet_filter:
{
query : {
"filtered" : {
"query" : {
"match_all" : {}
},
"filter" : {
"and" : [
{
"term" : {
"colour" : "Red"
}
},
{
"term" : {
"feature" : "Square"
}
},
{
"term" : {
"feature" : "Shiny"
}
}
]
}
}
},
"facets" : {
"colours" : {
"terms" : {
"field" : "colour"
},
"facet_filter" : {
"range" : { "price" : { "from" : "10", "to" : "20" } }
}
},
"features" : {
"terms" : {
"field" : "feature"
},
"facet_filter" : {
"range" : { "price" : { "from" : "10", "to" : "20" } }
}
},
"prices" : {
"statistical" : {
"field" : "price"
}
}
},
"filter": {
"range" : { "price" : { "from" : "10", "to" : "20" } }
}
}
Btw, important change since ES 1.0.0. Top-level filter was renamed to post_filter (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_search_requests.html#_search_requests). And filtered queries using is still preferred as described here: http://elasticsearch-users.115913.n3.nabble.com/Filters-vs-Queries-td3219558.html
And there is global option for facets to avoid filtering by query filter (elasticsearch.org/guide/en/elasticsearch/reference/current/search-facets.html#_scope).

Elastic search has_child query

We have a parent-child (one to many) relation in elastic search, and we want to check for all parent objects where it's child object attribute(child_attr) has any value in it.
we are generating json-queries as below:
1) For Has value condition.
{
"has_child" : {
"query" : {
"filtered" : {
"query" : {
"match_all" : { }
},
"filter" : {
"and" : {
"filters" : [ {
"exists" : {
"field" : "child_attr"
}
}, {
"not" : {
"filter" : {
"term" : {
"child_attr" : ""
}
}
}
} ]
}
}
}
},
"type" : "child"
}
}
2) For Has No Value Condition
{
"has_child" : {
"query" : {
"filtered" : {
"query" : {
"match_all" : { }
},
"filter" : {
"or" : {
"filters" : [ {
"missing" : {
"field" : "child_attr"
}
}, {
"term" : {
"child_attr" : ""
}
} ]
}
}
}
},
"type" : "child"
}
}
These queries are returning only those parent objects where either all child objects have some value or all child objects have no value the searched attribute.
It doesn't return anything where this condition is met partially which covers majority of data.
I have also toyed with keyword analyzer to index this child_attribute but no joy.
Look forward to your expert suggestions please.
You are getting unexpected results because the query
"missing" : {
"field" : "child_attr"
}
matches both records that were indexed with empty string in the child_attr and records in which child_attr was missing.
The query
"exists" : {
"field" : "child_attr"
}
is exact oposite of the first query, it matches all records that were indexed with a non-empty child_attr field.
The query
"term" : {
"child_attr" : ""
}
doesn't match anything.

Resources