Term query on nested fields returns no result in Elasticsearch - elasticsearch

I have a nested type field in my mapping. When I use Term search query on my nested field no result is returned from Elasticsearch whereas when I change Term to Match query, it works fine and Elasticsearch returns expected result
here is my mapping, imagine I have only one nested field in my type mapping
{
"homing.estatefiles": {
"mappings": {
"estatefile": {
"properties": {
"DynamicFields": {
"type": "nested",
"properties": {
"Name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"ValueBool": {
"type": "boolean"
},
"ValueDateTime": {
"type": "date"
},
"ValueInt": {
"type": "long"
}
}
}
}
}
}
}
}
And here is my term query (which returns no result)
{
"from": 50,
"size": 50,
"query": {
"bool": {
"filter": [
{
"nested": {
"query": {
"bool": {
"must": [
{
"term": {
"DynamicFields.Name":{"value":"HasParking"}
}
},
{
"term": {
"DynamicFields.ValueBool": {
"value": true
}
}
}
]
}
},
"path": "DynamicFields"
}
}
]
}
}
}
And here is my query which returns expected result (by changing Term query to Match query)
{
"from": 50,
"size": 50,
"query": {
"bool": {
"filter": [
{
"nested": {
"query": {
"bool": {
"must": [
{
"match": {
"DynamicFields.Name":"HasParking"
}
},
{
"term": {
"DynamicFields.ValueBool": {
"value": true
}
}
}
]
}
},
"path": "DynamicFields"
}
}
]
}
}
}

This is happening because the capital letters with the analyzer of elastic.
When you are using term the elastic is looking for the exact value you gave.
up until now it sounds good, but before it tries to match the term, the value you gave go through an analyzer of elastic which manipulate your value.
For example in your case it also turn the HasParking to hasparking.
And than it will try to match it and of course will fail. They have a great explanation in the documentation in the "Why doesn’t the term query match my document" section. This analyzer not being activated on the value when you query using match and this why you get your result.

Related

Need help combining wildcard search with range query within elasticsearch?

I am trying to combine wildcard with date range in elastic search query but is not giving response based upon the wildcard search. It is returning response with items which have incorrect date range.
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"wildcard": {
"hostName": "*abc*"
}
},
{
"range": {
"requestDate": {
"gte": "2019-10-01T08:00:00.000Z"
}
}
}
]
}
}
]
}
}
}
The index mapping looks as below:
{
"index_history": {
"mappings": {
"applications_datalake": {
"properties": {
"query": {
"properties": {
"term": {
"properties": {
"server": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
},
"index-data-type": {
"properties": {
"attributes": {
"properties": {
"wwnListForServer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"hostName": {
"type": "keyword"
},
"requestDate": {
"type": "date"
},
"requestedBy": {
"properties": {
"id": {
"type": "keyword"
},
"name": {
"type": "keyword"
}
}
}
}
}
}
}
}
You missed minimum_should_match parameter,
Check this out :
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html.
I think your query should looklike this:
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"wildcard": {
"hostName": "*abc*"
}
},
{
"range": {
"requestDate": {
"gte": "2019-10-01T08:00:00.000Z"
}
}
}
],
"minimum_should_match" : 2
}
}
]
}
}
}
From the documentation :
You can use the minimum_should_match parameter to specify the number
or percentage of should clauses returned documents must match.
If the bool query includes at least one should clause and no must or
filter clauses, the default value is 1. Otherwise, the default value
is 0.
According to your mappings, you have to call-out the fully qualified property for hostName and requestDate fields. Example:
"wildcard": {
"index-data-type.hostName": {
"value": "..."
}
}
Also, could also consider reducing your compound queries to just the main bool query, using the must clause, and apply a filter. Example:
{
"from": 0,
"size": 20,
"query": {
"bool": {
"must": [
{
"wildcard": {
"index-data-type.hostName": {
"value": "*abc*"
}
}
}
],
"filter": {
"range": {
"index-data-type.requestDate": {
"gte": "2019-10-01T08:00:00.000Z"
}
}
}
}
}
}
The filter context doesn't contribute to the _score yet it reduces your number of hits.
Warnining:
Using the leading asterisk (*) on a wildcard query can have severe performance impacts to your queries.

Can you reference other queries in Elasticsearch percolator?

can percolator queries reference other stored query docs in a percolator index? For example, given I have the following Boolean query, with _id=1, already indexed in the percolator:
{
"query": {
"bool": {
"must": [
{ "term": { "tag": "wow" } }
]
}
}
}
Could I have another query, with _id=2, indexed (note that I'm making up the _percolator_ref_id terms query key):
{
"query": {
"bool": {
"should": [
{ "term": { "tag": "elasticsearch" } },
{ "terms" : { "_percolator_ref_id": [1] } }
]
}
}
}
If I percolated the following document:
{ "tag": "wow" }
I would expect both _id=1 and _id=2 queries to match. Does some functionality like _percolator_ref_id exist?
Thanks!
Edit: To clarify, I do not know beforehand how many query references appear in a given query (e.g., the _id=2 query could reference 10 other queries potentially).
You can do something like below
2 queries are registered in below index
PUT myindex
{
"mappings": {
"properties": {
"query1": {
"type": "percolator"
},
"query": {
"type": "percolator"
},
"field": {
"type": "text"
}
}
}
}
You can use bool and must/should to combine different queries
GET /myindex/_search
{
"query": {
"bool": {
"must": [
{
"percolate": {
"field": "query",
"document": {
"field": "fox jumps over the lazy dog"
}
}
},
{
"percolate": {
"field": "query1",
"document": {
"field": "fox jumps over the lazy dog"
}
}
}
]
}
}
}

Elastic Search query for an AND condition on two properties of a nested object

I have the post_filter as below, Where I am trying to filter records where the school name is HILL SCHOOL AND containing a nested child object with name JOY AND section A.
school is present in the parent object, Which is holding children list of nested objects.
All of the above are AND conditions.
But the query doesn't seem to work. Any idea why ? And is there a way to combine the two nested queries?
GET /test_school/_search
{
"query": {
"match_all": {}
},
"post_filter": {
"bool": {
"must_not": [
{
"bool": {
"must": [
{
"term": {
"schoolname": {
"value": "HILL SCHOOL"
}
}
},
{
"nested": {
"path": "children",
"query": {
"bool": {
"must": [
{
"match": {
"name": "JACK"
}
}
]
}
}
}
},
{
"term": {
"children.section": {
"value": "A"
}
}
}
]
}
}
]
}
}
}
The schema is as below:
PUT /test_school
{
"mappings": {
"_doc": {
"properties": {
"schoolname": {
"type": "keyword"
},
"children": {
"type": "nested",
"properties": {
"name": {
"type": "keyword",
"index": true
},
"section": {
"type": "keyword",
"index": true
}
}
}
}
}
}
}
Sample data as below:
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JOY",
"section":"A"
}
}
second record
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JACK",
"section":"B"
}
}
https://stackoverflow.com/a/17543151/183217 suggests special mapping is needed to work with nested objects. You appear to be falling foul of the "cross object matching" problem.

ElasticSearch filtered query and filter term

I'm trying to use a filter on a filtered query, this is what I'm trying with Sense:
GET myindex/catalog/_search
{
"query": {
"filtered": {
"query": {
"query_string": {
"analyze_wildcard": true,
"query": "test",
"fields": ["title^3.5", "contributions.authors.name^5", "publisher^2", "formats.productCode^0.5", "description^0.1"],
"use_dis_max": true
}
},
"filter": {
"term": {
"sku": "test-687"
}
}
}
}
}
This query hasn't any hit, but if I remove the filter property I get exactly the item with sku = test-687.
I cannot understand why the query with the filter doesn't give me the same result.
Mapping:
{
"myindex": {
"mappings": {
"catalog": {
"properties": {
"sku": {
"type": "string"
},
"title": {
"type": "string"
},
"updated_at": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
}
}
the full query is:
GET myindex/catalog/_search {
"query": {
"filtered": {
"query": {
"query_string": {
"analyze_wildcard": true,
"query": "test",
"fields": ["title^3.5", "contributions.authors.name^5", "publisher^2", "formats.productCode^0.5", "description^0.1"],
"use_dis_max": true
}
},
"filter": {
"bool": {
"must": {
"query": {
"match": {
"sku": "test-687"
}
}
}
}
}
}
}
}
With default mapping the "Standard Analyser is used" :
An analyzer of type standard is built using the Standard Tokenizer with the Standard Token Filter, Lower Case Token Filter, and Stop Token Filter.
(More details her )
Term is case sensitive, match not

How to get Elastic search to return both exact matched and then other matches in result

Need help with Elasticsearch. I try to get first exact match result then those documents that have one field matched using the following query but with no luck. Basically, trying to get top score hits first and then less accurate and only matched by one field in the total search result.
The mapping is as following:
{
"palsx1493": {
"mappings": {
"pals": {
"properties": {
"aboutme": {
"type": "string"
},
"dob": {
"type": "date",
"format": "date"
},
"fccode": {
"type": "string"
},
"fcname": {
"type": "string"
},
"learning": {
"type": "nested",
"properties": {
"skillslevel": {
"type": "string"
},
"skillsname": {
"type": "string"
}
}
},
"name": {
"type": "string"
},
"rating": {
"type": "string"
},
"teaching": {
"type": "nested",
"properties": {
"skillslevel": {
"type": "string"
},
"skillsname": {
"type": "string"
}
}
},
"trate": {
"type": "string"
},
"treg": {
"type": "string"
}
}
}
}
}
}
When Searching, I need the result to return the exact matched documents followed by lower score matched with the teaching skillname in that prioritized order. what happens now is that I get the exact matches correctly first and then I get the learning.skillname matched, and then teaching.skillname matched. I want these two last ones swapped having the teaching.skillname coming after the exact matched results.
Exact match:
1. fcname (is crom country name and can be either a specific name or just set to "Any Country".
2. dob: Date of birth is a range value - a range value is given as input
3. teaching: skillname
4. learning: skillname
This is what I have tried with no luck:
{
"query": {
"bool": {
"should": [
{ "match": { "fcname": "spain"}},
{ "range": {
"bod": {
"from": "1950-10-10",
"to": "1967-12-12"
}
}
},
{
"nested": {
"path": "learning",
"score_mode": "max",
"query": {
"bool": {
"must": [
{ "match": { "learning.skillname": learningSkillName}}
]
}
}
}
},
{
"nested": {
"path": "teaching",
"query": {
"bool": {
"must": [
{ "match": { "teaching.skillname": teachingSkillName}}
]
}
}
}
}
]
}
}
}
Please look into indices. The default is a full text search which does inverted indexing to store data. So it would store the string according to the analyzer.
Fo exact string match please use : index = 'not_analyzed'
eg.
"nick"{
"type": "string",
"index":"not_analyzed"
},
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html
I figured it out. Solution was to use function_score feature to override/ add score to a document with certain matched field. Replacing the nested part above with following gave me the correct result:
"nested": {
"path": "teaching",
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{ "match": { "teaching.skillname": "xxx"}}
]
}
},
"functions": [
{
"script_score": {
"script": "_score + 2"
}
}],

Resources