In some of our types, we have a parent child setup and we want to search on parent fields and also on the child fields (and return parent) and we do query like below. When there is a has_child match is there any way to get highlighting information from the child match even though the parent is being returned. As an example, if we have mapping like the following:
PUT nested2
{
"mappings":{
"discussion":{
"properties" : {
"title":{
"type":"string"
}
}
},
"discussionPost":{
"_parent":{
"type" : "discussion"
},
"properties" : {
"post" : {
"type" : "string"
}
}
}
}
}
And we issue a query like below, highlight information is returned if there is a match on parent field but not if the parent is being returned due to a has_child match:
POST nested2/discussion/_search
{
"query": {
"bool": {
"should": [
{
"prefix": {
"_all" : "cat"
}
},
{
"has_child" : {
"type" : "discussionPost",
"score_mode" : "sum",
"query" : {
"prefix": {
"_all" : "cat"
}
}
}
}
],
"minimum_should_match": 1
}
},
"highlight":{
"fields":{
"*":{}
}
}
}
Is it possible to get highlight information on what matched in the child when has_child query is being issued on the parent?
Regards
LT
It is possible to do this using inner_hits inside the has_child query clause:
{
"query": {
"bool": {
"should": [
{
"has_child" : {
"inner_hits": {
"_source": false,
"highlight":{
"order": "score",
"fields": {"*":{}}
}
},
"type" : "discussionPost",
"score_mode" : "sum",
"query" : {
"prefix": {
"_all" : "cat"
}
}
}
}
],
"minimum_should_match": 1
}
}
}
Related
I have the following mapping
"mappings":{
"properties":{
"name": {
"type": "text"
},
"age": {
"type": "integer"
},
"customProps":{
"type" : "nested",
"properties": {
"key":{
"type": "keyword"
},
"value": {
"type" : "keyword"
}
}
}
}
}
example data
{
"name" : "person1",
"age" : 10,
"customProps":[
{"hairColor":"blue"},
{"height":"120"}
]
},
{
"name" : "person2",
"age" : 30,
"customProps":[
{"jobTitle" : "software engineer"},
{"salaryAccount" : "AvGhj90AAb"}
]
}
so i want to be able to search for document by salary account case insensitive, i am also searching using wild card
example query is
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "customProps",
"query": {
"bool": {
"must": [
{ "match": { "customProps.key": "salaryAccount" } },
{ "wildcard": { "customProps.value": "*AvG*"
}
}
]}}}}]}}}
i tried adding analyzer with PUT using the following syntax
{
"settings":{
"index":{
"analysis":{
"analyzer":{
"analyzer_case_insensitive" : {
"tokenizer":"keyword",
"filter":"lowercase"
}
}
}
}
},
"mappings":{
"people":{
"properties":{
"customProps":{
"properties":{
"value":{
"type": "keyword",
"analyzer": "analyzer_case_insensitive"
}
}
}
}
}
}
}
im getting the following error
"type" : "mapper_parsing_exception",
"reason" : "Root mapping definition has unsupported parameters: [people: {properties={customProps={properties={value={analyzer=analyzer_case_insensitive, type=keyword}}}}}]"
any idea how to do the analyzer for the salary account object in the array when it exists?
Your use case is quite clear, that you want to search on the value of salaryAccount only when this key exists in customProps array.
There are some issues with your mapping definition :
You cannot define a custom analyzer for keyword type field, instead you can use a normalizer
Based on the mapping definition you added at the beginning of the question, it seems that you are using elasticsearch version 7.x. But the second mapping definition that you provided, in that you have added mapping type also (i.e people), which is deprecated in 7.x
There is no need to add the key and value fields in the index mapping.
Adding a working example with index mapping, search query, and search result
Index Mapping:
PUT myidx
{
"mappings": {
"properties": {
"customProps": {
"type": "nested"
}
}
}
}
Search Query:
You need to use exists query, to check whether a field exists or not. And case_insensitive param in Wildcard query is available since elasticsearch version 7.10. If you are using a version below this, then you need to use a normalizer, to achieve case insensitive scenarios.
POST myidx/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "customProps",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "customProps.salaryAccount"
}
},
{
"wildcard": {
"customProps.salaryAccount.keyword": {
"value": "*aVg*",
"case_insensitive": true
}
}
}
]
}
}
}
}
]
}
}
}
Search Result:
"hits" : [
{
"_index" : "myidx",
"_type" : "_doc",
"_id" : "2",
"_score" : 2.0,
"_source" : {
"name" : "person2",
"age" : 30,
"customProps" : [
{
"jobTitle" : "software engineer"
},
{
"salaryAccount" : "AvGhj90AAb"
}
]
}
}
]
When i do a multi-condition query and apply field collapsing to one of the field in the mentioned index i get following error
no mapping found for `search_type.keyword` in order to collapse on
Query Used :
GET /_search
{
"query": {
"bool" : {
"must" : [
{
"match" :
{
"id" : "123456"
}
},
{
"terms": {
"_index": ["history"]
}
}
]
}
},
"collapse" : {
"field" : "search_type.keyword",
"inner_hits": {
"name": "terms",
"size": 10
}
}
}
Error Trace:
{
"shard" : 0,
"index" : "test",
"node" : "UOA44HkATh61krg6ht3paA",
"reason" : {
"type" : "illegal_argument_exception",
"reason" : "no mapping found for `search_type.keyword` in order to collapse on"
}
}
Currently, am applying the query only for index - history but the result throws exception for indexes that i haven't mentioned. Please help how to narrow down field collapsing to a particular index.
It appears to be a bug, but if you notice your result carefully, you should be able to view the response you are looking for at the very end after all the such errors are observed.
But then again why not add the index name to the front and modify your query as below:
POST history/_search <---- Add index name here
{
"query": {
"bool": {
"must": [
{
"match": {
"id": "123456"
}
}
]
}
},
"collapse" : {
"field" : "search_type.keyword",
"inner_hits": {
"name": "terms",
"size": 10
}
}
}
The mapping contains nested fields which shouldn't be analyzed (not sure if the 'not_analyzed' value is accurate). Is it possible to do an exact match on a nested field? In the query below the "metadata.value": "2014.NWJSD.47" still gets analyzed. Elasticsearch breaks up the string into several terms ("2014", "NWJSD", "47"). I tried to use "term" instead of "match" but this didn't return any result.
"mappings" : {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "text",
"index" : "not_analyzed"
},
"value" : {
"type" : "text",
"index" : "not_analyzed"
}
}
}
}
The Query:
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"match": {
"metadata.name": "number"
}
},
{
"match": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}
Try to use keyword instead of text in your mapping like:
{
"mappings": {
"your_type_name": {
"properties": {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "keyword"
},
"value" : {
"type" :"keyword"
}
}
}
}
}
}
}
These fields won't be analyzed. Then you should reindex your data and to query your data you should replace match (which is analyzed query) with term.
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"term": {
"metadata.name": "number"
}
},
{
"term": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}
}
I think you are looking for a query string query.
You can freely disable "analyze" option for that field in mapping option and reindex everything again but you could also check this query out:
as written here:
GET /_search
{
"query": {
"query_string" : {
"query" : "your string"
}
}
}
I have trouble combining term, must_not queries on nested documents.
Sense example can be found here : http://sense.qbox.io/gist/be436a1ffa01e4630a964f48b2d5b3a1ef5fa176
Here my mapping :
{
"mappings": {
"docs" : {
"properties": {
"tags" : {
"type": "nested",
"properties" : {
"type": {
"type": "string",
"index": "not_analyzed"
}
}
},
"label" : {
"type": "string"
}
}
}
}
}
with two documents in this index :
{
"tags" : [
{"type" : "POST"},
{"type" : "DELETE"}
],
"label" : "item 1"
},
{
"tags" : [
{"type" : "POST"}
],
"label" : "item 2"
}
When I query this index like this :
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}
}
}
I've got one hit (which is correct)
When I want to get documents WHICH DON'T CONTAIN the tag "DELETE", with this query :
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "delete"
}
}
}
}
}
}
}
I've got 2 hits (which is incorrect).
This issue seems very close to this one (Elasticsearch array must and must_not) but it's not...
Can you give me some clues to resolve this issue ?
Thank you
Your original query would search in each individual nested object and eliminate the objects that don't match, but if there are some nested objects left, they do match with your query and so you get your results. This is because nested objects are indexed as a hidden separate document
Original code:
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "delete"
}
}
}
}
}
}
}
The solution is then quite simple really, you should bring the bool query outside the nested documents. Now all the documents are discarded who have a nested object with the "DELETE" type. Just what you wanted!
The solution:
{
"query": {
"bool": {
"must_not": {
"nested": {
"path": "tags",
"query": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}
}
}
NOTE: Your strings are "not analyzed" and you searched for "delete" instead of "DELETE". If you want to search case insensitive, make your strings analyzed
This should fix your problem: http://sense.qbox.io/gist/f4694f542bc76c29624b5b5c9b3ecdee36f7e3ea
Two most important things:
include_in_root on "tags.type". This will tell ES to index tag types as "doc.tags.types" : ['DELETE', 'POSTS'], so you can access an array of those values "flattened" on the root doc . This means you no longer need a nested query (see #2)
Drop the nested query.
{
"mappings": {
"docs" : {
"properties": {
"tags" : {
"type": "nested",
"properties" : {
"type": {
"type": "string",
"index": "not_analyzed"
}
},
"include_in_root": true
},
"label" : {
"type": "string"
}
}
}
}
}
{
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}
I currently have parents indexed in elastic search (documents) and child (comments) related to these documents.
My first objective was to search for a document with more than N comments, based on a child query. Here is how I did it:
documents/document/_search
{
"min_score": 0,
"query": {
"has_child" : {
"type" : "comment",
"score_type" : "sum",
"boost": 1,
"query" : {
"range": {
"date": {
"lte": 20130204,
"gte": 20130201,
"boost": 1
}
}
}
}
}
}
I used score to calculate the amount of comments a document has and then I filtered the documents by this amount, using "min_score".
Now, my objective is to search not just comments, but several other child documents related to the document, always based on frequency. Something like the query bellow:
documents/document/_search
{
"query": {
"match_all": {
}
},
"filter" : {
"and" : [{
"query": {
"has_child" : {
"type" : "comment",
"query" : {
"range": {
"date": {
"lte": 20130204,
"gte": 20130201
}
}
}
}
}
},
{
"or" : [
{"query": {
"has_child" : {
"type" : "comment",
"query" : {
"match": {
"text": "Finally"
}
}
}
}
},
{ "query": {
"has_child" : {
"type" : "comment",
"query" : {
"match": {
"text": "several"
}
}
}
}
}
]
}
]
}
}
The query above works fine, but it doesn't filter based on frequency as the first one does. As filters are computed before scores are calculated, I cannot use min_score to filter each child query.
Any solutions to this problem?
There is no score at all associated with filters. I'd suggest to move the whole logic to the query part and use a bool query to combine the different queries together.