i am trying to filter my result using nested filter but i am getting incorrect result
here is my mapping info
{
"stock" : {
"mappings" : {
"clip" : {
"properties" : {
"description" : {
"type" : "string"
},
"keywords" : {
"type" : "nested",
"properties" : {
"category" : {
"type" : "string"
},
"tags" : {
"type" : "string",
"index_name" : "tag"
}
}
},
"tags" : {
"type" : "string",
"index_name" : "tag"
},
"title" : {
"type" : "string"
}
}
}
}
}
}
clip document data
{
"_index" : "stock",
"_type" : "clip",
"_id" : "AUnsTOBBpafrKleQN284",
"_score" : 1.0,
"_source":{
"title": "journey to forest",
"description": "this clip contain information about the animals",
"tags": ["birls", "wild", "animals", "roar", "forest"],
"keywords": [
{
"tags": ["spring","summer","autumn"],
"category": "Weather"
},
{
"tags": ["Cloudy","Stormy"],
"category": "Season"
},
{
"tags": ["Exterior","Interior"],
"category": "Setting"
}
]
}
i am trying to filter tags inside nested field 'keywords'
here is my query
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "keywords",
"filter": {
"bool": {
"must": [
{
"terms": { "tags": ["autumn", "summer"] }
}
]
}
}
}
}
}
}
}
i am getting no result why ?
what's wrong with my query or schema please help
The above query is syntactically incorrect . You need to provide the full path to tags from root keywords in the term query i.e.keywords.tags
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "keywords",
"filter": {
"bool": {
"must": [
{
"terms": { "keywords.tags": ["autumn", "summer"] }
}
]
}
}
}
}
}
}
}
Related
I'm trying to implement the Multi-Term Auto Completion that's presented here.
Filtering down to the correct documents works, but when aggregating the completion_terms they are not filtered to those that match the current partial query, but instead include all completion_terms from any matched documents.
Here are the mappings:
{
"mappings": {
"dynamic" : "false",
"properties" : {
"completion_ngrams" : {
"type" : "text",
"analyzer" : "completion_ngram_analyzer",
"search_analyzer" : "completion_ngram_search_analyzer"
},
"completion_terms" : {
"type" : "keyword",
"normalizer" : "completion_normalizer"
}
}
}
}
Here are the settings:
{
"settings" : {
"index" : {
"analysis" : {
"filter" : {
"edge_ngram" : {
"type" : "edge_ngram",
"min_gram" : "1",
"max_gram" : "10"
}
},
"normalizer" : {
"completion_normalizer" : {
"filter" : [
"lowercase",
"german_normalization"
],
"type" : "custom"
}
},
"analyzer" : {
"completion_ngram_search_analyzer" : {
"filter" : [
"lowercase"
],
"tokenizer" : "whitespace"
},
"completion_ngram_analyzer" : {
"filter" : [
"lowercase",
"edge_ngram"
],
"tokenizer" : "whitespace"
}
}
}
}
}
}
}
I'm then indexing data like this:
{
"completion_terms" : ["Hammer", "Fortis", "Tool", "2000"],
"completion_ngrams": "Hammer Fortis Tool 2000"
}
Finally, the autocomplete search looks like this:
{
"query": {
"bool": {
"must": [
{
"term": {
"completion_terms": "fortis"
}
},
{
"term": {
"completion_terms": "hammer"
}
},
{
"match": {
"completion_ngrams": "too"
}
}
]
}
},
"aggs": {
"autocomplete": {
"terms": {
"field": "completion_terms",
"size": 100
}
}
}
}
This correctly returns documents matching the search string "fortis hammer too", but the aggregations include ALL completion terms that are included in any of the matched documents, e.g. for the query above:
"buckets": [
{ "key": "fortis" },
{ "key": "hammer" },
{ "key": "tool" },
{ "key": "2000" },
]
Ideally, I'd expect
"buckets": [
{ "key": "tool" }
]
I could filter out the terms that are already covered by the search query ("fortis" and "hammer" in this case) in the app, but the "2000" doesn't make any sense from a user's perspective, because it doesn't partially match any of the provided search terms.
I understand why this is happening, but I can't think of a solution. Can anyone help?
try filters agg please
{
"query": {
"bool": {
"must": [
{
"term": {
"completion_terms": "fortis"
}
},
{
"term": {
"completion_terms": "hammer"
}
},
{
"match": {
"completion_ngrams": "too"
}
}
]
}
},
"aggs": {
"findOuthammerAndfortis": {
"filters": {
"filters": {
"fortis": {
"term": {
"completion_terms": "fortis"
}
},
"hammer": {
"term": {
"completion_terms": "hammer"
}
}
}
}
}
}
}
I'm new to elasticsearch. I need to build the query dynamically, where for each field name the the corresponding file is fetched
I have the below query, can anyone say if its the right approach? Also with this query, the documents are just repeating for one particular file name
Please let me know how to go about it
GET index_name/_search
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "program"
}
},
{
"match_phrase": {
"field_value": "aaa-123"
}
}
]
}
},
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "species"
}
},
{
"match_phrase": {
"field_value": "mouse"
}
}
]
}
},
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "model name"
}
},
{
"match_phrase": {
"field_value": "b45"
}
}
]
}
}
]
}
},"aggs": {
"2": {
"terms": {
"field": "myfile_file_name.keyword",
"size": 1000,
"order": {
"_key": "asc"
}
},
"aggs": {
"3": {
"terms": {
"field": "field_name.keyword",
"size": 1000,
"order": {
"_key": "asc"
}
}
}
}
}
}
}
mapping and Output
{
"_index" : "test",
"_type" : "test_data",
"_id" : "123",
"_score" : 1.0,
"_source" : {
"document_id" : 123,
"m_id" : 1,
"source" : "ADDD",
"type" : "M",
"name" : "Animal",
"value" : "None",
"test_type" : "Test123",
"file_name" : "AA.zip",
"description" : "testing",
"program" : ["hello"],
"species" : ["mouse"],
"study" : ["Study1"],
"create_date" : "2020-08-20 11:51:21.152",
"update_date" : "2020-08-20 11:51:21.152",
"source_name" : "Anim",
"auth" : ["na"],
"treatment" : ["TR001", "TR002", "TR004"],
"timepoint" : ["72", "48"],
"findings_reports" : "na",
"model" : ["None",],
"additional" : "{'view': '', 'load': []}",
"data" : "Pre"
}
},
]
}
}
Using elasticsearch 7, I am trying to build facets for doubly nested objects.
So in the example below I would like to pull out the artist id codes from the artistMakerPerson field. I can pull out the association which is nested at a single depth but I can't get the syntax for the nested nested objects.
You could use the following code in Kibana to recreate an example.
My mapping looks like this:
PUT test_artist
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"properties": {
"object" : {
"type" : "text",
"fields" : {
"raw" : {
"type" : "keyword"
}
},
"copy_to" : [
"global_search"
]
},
"uniqueID" : {
"type" : "keyword",
"copy_to" : [
"global_search"
]
},
"artistMakerPerson" : {
"type" : "nested",
"properties" : {
"association" : {
"type" : "keyword"
},
"name" : {
"type" : "nested",
"properties" : {
"id" : {
"type" : "keyword"
},
"text" : {
"type" : "text",
"fields" : {
"raw" : {
"type" : "keyword"
}
},
"copy_to" : [
"gs_authority"
]
}
}
},
"note" : {
"type" : "text"
}
}
}
}
}
}
Index a document with:
PUT /test_artist/_doc/123
{
"object": "cup",
"uniquedID": "123",
"artistMakerPerson" : [
{
"name" : {
"text" : "Johann Kandler",
"id" : "A6734"
},
"association" : "modeller",
"note" : "probably"
},
{
"name" : {
"text" : "Peter Reinicke",
"id" : "A27702"
},
"association" : "designer",
"note" : "probably"
}
]
}
I am using this query to pull out facets or aggregations for artistMakerPerson.association
GET test_artist/_search
{
"size": 0,
"aggs": {
"artists": {
"nested": {
"path": "artistMakerPerson"
},
"aggs": {
"kinds": {
"terms": {
"field": "artistMakerPerson.association",
"size": 10
}
}
}
}
}
}
and I am rewarded with buckets for designer and modeller but I get nothing when I try to pull out the deeper artist id:
GET test_artist/_search
{
"size": 0,
"aggs": {
"artists": {
"nested": {
"path": "artistMakerPerson"
},
"aggs": {
"kinds": {
"terms": {
"field": "artistMakerPerson.name.id",
"size": 10
}
}
}
}
}
}
What am I doing wrong?
Change the path from artistMakerPerson to artistMakerPerson.name.
GET test_artist/_search
{
"size": 0,
"aggs": {
"artists": {
"nested": {
"path": "artistMakerPerson.name"
},
"aggs": {
"kinds": {
"terms": {
"field": "artistMakerPerson.name.id",
"size": 10
}
}
}
}
}
}
Using named queries, I can get a list of the matched_queries for boolean expressions such as:
(query1) AND (query2 OR query3 OR true)
Here is an example of using named queries to match on top-level document fields:
DELETE test
PUT /test
PUT /test/_mapping/_doc
{
"properties": {
"name": {
"type": "text"
},
"type": {
"type": "text"
},
"TAGS": {
"type": "nested"
}
}
}
POST /test/_doc
{
"name" : "doc1",
"type": "msword",
"TAGS" : [
{
"ID" : "tag1",
"TYPE" : "BASIC"
},
{
"ID" : "tag2",
"TYPE" : "BASIC"
},
{
"ID" : "tag3",
"TYPE" : "BASIC"
}
]
}
# (query1) AND (query2 or query3 or true)
GET /test/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": {
"query": "doc1",
"_name": "query1"
}
}
}
],
"should": [
{
"match": {
"type": {
"query": "msword",
"_name": "query2"
}
}
},
{
"exists": {
"field": "type",
"_name": "query3"
}
}
]
}
}
}
The above query correctly returns all three matched_queries in the response:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.5753641,
"hits" : [
{
"_index" : "test",
"_type" : "_doc",
"_id" : "TKNJ9G4BbvPS27u-ZYux",
"_score" : 1.5753641,
"_source" : {
"name" : "doc1",
"type" : "msword",
"TAGS" : [
{
"ID" : "ds1",
"TYPE" : "BASIC"
},
{
"ID" : "wb1",
"TYPE" : "BASIC"
}
]
},
"matched_queries" : [
"query1",
"query2",
"query3"
]
}
]
}
}
However, I'm trying to run a similar search:
(query1) AND (query2 OR query3 OR true)
only this time on the nested TAGS object rather than top-level document fields.
I've tried the following query, but the problem is I need to supply the inner_hits object for nested objects in order to get the matched_queries in the response, and I can only add it to one of the three queries.
GET /test/_search
{
"query": {
"bool": {
"must": {
"nested": {
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag1",
"_name": "tag1-query"
}
}
},
// "inner_hits" : {}
}
},
"should": [
{
"nested": {
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag2",
"_name": "tag2-query"
}
}
},
// "inner_hits" : {}
}
},
{
"nested": {
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag3",
"_name": "tag3-query"
}
}
},
// "inner_hits" : {}
}
}
]
}
}
}
Elasticsearch will complain if I add more than one 'inner_hits'. I've commented out the places above where I can add it, but each of these will only return the single matched query.
I want my response to this query to return:
"matched_queries" : [
"tag1-query",
"tag2-query",
"tag3-query"
]
Any help is much appreciated, thanks!
A colleague helpfully provided a solution to this; move the _named parameter to directly under each nested section:
GET /test/_search
{
"query": {
"bool": {
"must": {
"nested": {
"_name": "tag1-query",
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag1"
}
}
}
}
},
"should": [
{
"nested": {
"_name": "tag2-query",
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag2"
}
}
}
}
},
{
"nested": {
"_name": "tag3-query",
"path": "TAGS",
"query": {
"match": {
"TAGS.ID": {
"query": "tag3"
}
}
}
}
}
]
}
}
}
This correctly returns all three tags now in the matched_queries response:
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 2.9424875,
"hits" : [
{
"_index" : "test",
"_type" : "_doc",
"_id" : "TaNy9G4BbvPS27u--oto",
"_score" : 2.9424875,
"_source" : {
"name" : "doc1",
"type" : "msword",
"TAGS" : [
{
"ID" : "ds1",
"TYPE" : "DATASOURCE"
},
{
"ID" : "wb1",
"TYPE" : "WORKBOOK"
},
{
"ID" : "wb2",
"TYPE" : "WORKBOOK"
}
]
},
"matched_queries" : [
"tag1-query",
"tag2-query",
"tag3-query"
]
}
]
}
}
I have products index with following mapping:
{
"products" : {
"mappings" : {
"properties" : {
"#timestamp" : {
"type" : "date"
},
"name":{
"type": "text"
},
"price" : {
"type" : "integer"
},
"product_review_rel" : {
"type" : "join",
"eager_global_ordinals" : true,
"relations" : {
"product" : "review"
}
},
"rate" : {
"type" : "integer"
}
}
}
}
}
This index contains products and reviews, as you can see at product_review_rel field.
Products contain name, price, ... fields.
Reviews contain rate, ... fields.
I want to get average rating of each products. How to do that?
Another question, is it possible to return average rating from products returned by the following query in the same request:
{
"query": {
"nested": {
"path": "translations",
"query": {
"multi_match": {
"query": "kem chống nắng",
"fields": [
"name"
],
"analyzer":"vi_analyzer"
}
}
}
}
}
Update 1: Composite aggregation
GET products/_search
{
"query": {
"nested": {
"path": "translations",
"query": {
"multi_match": {
"query": "kem chống nắng",
"fields": [
"translations.name",
"translations.description"
],
"analyzer": "vi_analyzer"
}
}
}
},
"aggs": {
"products": {
"composite": {
"sources": [
{
"id": {
"terms": {
"field": "_id"
}
}
}
]
},
"aggs": {
"reviews": {
"children": {
"type": "review"
},
"aggs": {
"rating": {
"avg": {
"field": "rate"
}
}
}
}
}
}
}
}```