i have index in es and this is the corresponding mapping :
'''
GET /vid_detect2/_mapping
{
"properties": {
"date":{"type":"date"},
"time":{ "type": "text",
"fielddata": true},
"frame_id": {"type":"integer"},
"camera_id":{"type":"integer"},
"path":{"type":"text"},
"objects" : {"type": "nested",
"properties": {
"class": { "type": "text" ,"fielddata":true },
"confidence": { "type": "float" },
"coordinates":{ "type": "nested" ,
"properties": { "x" :{"type":"float"},
"y" :{"type":"float"},
"w" :{"type":"float"},
"h" :{"type":"float"}
} }
}
}
}
}'''
I want to run following query first :
"query": {
"bool": {
"must": [
{
"nested": {
"path": "objects",
"query": {
"bool": {
"must": [
{ "match": { "objects.class": "person" }}
]
}}}}
]
}}
and then aggregate the returned results with respect to camera_id and further aggregate those aggregated results with date histogram. Please help.
Good start! You can now simply add an aggregation section to achieve what you want:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "objects",
"query": {
"bool": {
"must": [
{
"match": {
"objects.class": "person"
}
}
]
}
}
}
}
]
}
},
"aggs": {
"camera": {
"terms": {
"field": "camera_id"
},
"aggs": {
"histo": {
"date_histogram": {
"field": "date",
"interval": "day"
}
}
}
}
}
}
Related
In my index I have a mapping that contains the locations property of type "nested". This property contains an array of locations. Each location is an object with the following structure:
{
"id": string,
"type": string
}
On my website I want to select multiple locations and find all documents that have ANY of these locations I selected. I was able to search by one location but I have to modify my query somehow to search for many locations.
I have the following index set up:
"mappings": {
"users": {
"properties": {
"locations": {
"type": "nested",
"properties": {
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"id": {
"type": "long"
}
}
}
}
Example document:
{
"locations": [
{
"id": "UA",
"type": "country"
},
{
"id": "RU",
"type": "country"
},
{
"id": "OC",
"type": "continent"
}
]
}
My query that searches for users by location:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "locations",
"query": {
"bool": {
"must": [
{
"match": { "locations.id": "RU" }
},
{
"match": { "locations.type": "country" }
}
]
}
}
}
}
]
}
}
}
EDIT:
I also want to retreive documents which have locations.type equal to continent. In SQL it would look like
WHERE (id IN ('UA', 'RU') AND type = 'country')
OR (id IN ('EU', 'OC', 'NA') AND type = 'continent')
for multiple value search use Terms and for OR condition use should (see here):
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "locations",
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"terms": {
"locations.id": [
"UA",
"RU"
]
}
},
{
"term": {
"locations.type": "country"
}
}
]
}
},
{
"bool": {
"must": [
{
"terms": {
"locations.id": [
"EU",
"OC"
]
}
},
{
"term": {
"locations.type": "continent"
}
}
]
}
}
]
}
}
}
}
]
}
}
}
I have this index mapping with nested fields:
"customerpropertieses": {
"_parent": {
"type": "customerprofile"
},
"_routing": {
"required": true
},
"properties": {
"id": {
"type": "string"
},
"parentId": {
"type": "string"
},
"properties": {
"type": "nested",
"properties": {
"extentionPropertyId": {
"type": "long"
},
"propertyName": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
I want to find customerpropertieses where propertyName=criteria1 & value=value1 & propertyName=criteria2 & value=value2
The query I'm generating by hand is this:
{
"from": 0,
"size": 10,
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"query": {
"nested": {
"query": {
"bool": {
"must": [
{
"match": {
"properties.propertyName": {
"query": "criteria1 "
}
}
},
{
"match": {
"properties.value": {
"boost": 10.0,
"query": "value1"
}
}
},
{
"match": {
"properties.propertyName": {
"query": "criteria2"
}
}
},
{
"match": {
"properties.value": {
"boost": 10.0,
"query": "value2"
}
}
}
]
}
},
"path": "properties",
"inner_hits": {
"explain": false
},
"_name": "nested_properties"
}
}
}
I get 0 results back and I certainly have data with those characteristics. The search works fine when I look only propertyName=criteria1 & value=value1 or with propertyName=criteria2 & value=value2'
The question is how can I stack search criteria using & in a nested query?
Try the following query. From the look of your query i take a guess you want to match documents which had two nested documents with values value1,criteria1 and value2,criteria2 in value and criteria field respectively.
{
"from": 0,
"size": 10,
"sort": [{
"_score": {
"order": "desc"
}
}],
"query": {
"bool": {
"must": [{
"nested": {
"query": {
"bool": {
"must": [{
"match": {
"properties.propertyName": {
"query": "criteria2"
}
}
}, {
"match": {
"properties.value": {
"boost": 10.0,
"query": "value2"
}
}
}]
}
},
"path": "properties",
"inner_hits": {
"explain": false
},
"_name": "nested_properties"
}
}, {
"nested": {
"query": {
"bool": {
"must": [{
"match": {
"properties.propertyName": {
"query": "criteria1 "
}
}
}, {
"match": {
"properties.value": {
"boost": 10.0,
"query": "value1"
}
}
}]
}
},
"path": "properties",
"inner_hits": {
"explain": false
},
"_name": "nested_properties"
}
}]
}
}
}
I indexed the following document making the guess and modified the query
The Following document will match the query above
{
"parentId" : "3434",
"properties" : [{
"extentionPropertyId" : 24,
"propertyName" : "criteria1",
"value" : "value1"
},{
"extentionPropertyId" : 24,
"propertyName" : "criteria2",
"value" : "value2"
}]
}
Hope this helps.
Thanks
How do I go about fetching all documents w/o any objects in a field?
I have the following mapping:
"properties" : {
"name": {
"type": "text"
},
"nestedArray": {
"type": "nested",
"properties": {
"prop1": {
"type": "text"
},
"prop2": {
"type": "text"
}
}
}
}
and I want to get all documents where "nestedArray" is empty or doesn't exist.
I'm using elasticSearch 5.0
I think exists query would solve this problem. Try following query
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "nestedArray",
"query": {
"bool": {
"filter": {
"exists": {
"field": "nestedArray"
}
}
}
}
}
}
]
}
}
}
Try this
{
"size": 5000,
"query": {
"bool": {
"filter": [],
"must_not": [
{
"nested": {
"path": "nestedArray",
"query": {
"exists": {
"field": "nestedArray"
}
}
}
}
]
}
},
"from": 0
}
we have following structure in an index - following is only a partial and doc relevant for this question.
"instance" : {
"id" : 1,
{"instFields": [
{
"sourceFieldId": 2684,
"fieldValue": "false",
"fieldBoolean": false
},
{
"sourceFieldId": 1736,
"fieldValue": "DODGE",
"fieldString": "DODGE"
},
{
"sourceFieldId": 1560,
"fieldValue": "GRAY",
"fieldString": "GRAY"
},
{
"sourceFieldId": 1558,
"fieldValue": "CHALLENGER",
"fieldString": "CHALLENGER"
},
{
"sourceFieldId": 1556,
"fieldValue": "2010",
"fieldDouble": 2010
}
]
}
first user query is give me all instances where sourceFieldId=1736 - this returns all the DODGE instances[] - all this is working fine with an appripriate Elastic Search query. now when user is seeing all DODGE records - user wants to sort by any of those sourceFieldIds for e.g. say user is wanting to sort results by - color - sourceFieldId=1560.
say we have following sort query
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"nested": {
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"instance.dataSourceId": "196"
}
},
{
"term": {
"instance.dsTypeId": "5"
}
},
{
"nested": {
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"instance.instFields.sourceFieldId": "1558"
}
},
{
"term": {
"instance.instFields.fieldString.raw": "challenger"
}
}
]
}
}
}
},
"path": "instance.instFields"
}
}
]
}
}
}
},
"path": "instance",
"inner_hits": {
"name": "inner_data"
}
}
},
{
"nested": {
"query": {
"bool": {
"should": {
"bool": {
"must": [
{
"match": {
"instance.entitlements.roleId": {
"query": "1",
"type": "boolean"
}
}
},
{
"match": {
"instance.entitlements.read": {
"query": "true",
"type": "boolean"
}
}
}
]
}
}
}
},
"path": "instance.entitlements"
}
}
]
}
}
}
},
"sort": {
"instance.instFields.fieldString.raw": {
"order": "asc",
"nested_path": "instance.instFields",
"nested_filter": {
"bool": {
"filter": {
"bool": {
"must": [
{
"nested": {
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"instance.dataSourceId": "196"
}
},
{
"term": {
"instance.dsTypeId": "5"
}
},
{
"nested": {
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"instance.instFields.sourceFieldId": "1558"
}
},
{
"term": {
"instance.instFields.fieldString.raw": "challenger"
}
}
]
}
}
}
},
"path": "instance.instFields"
}
}
]
}
}
}
},
"path": "instance",
"inner_hits": {
"name": "inner_data1"
}
}
},
{
"nested": {
"query": {
"bool": {
"should": {
"bool": {
"must": [
{
"match": {
"instance.entitlements.roleId": {
"query": "1",
"type": "boolean"
}
}
},
{
"match": {
"instance.entitlements.read": {
"query": "true",
"type": "boolean"
}
}
}
]
}
}
}
},
"path": "instance.entitlements"
}
}
]
}
}
}
}
}
}
}
resulting docs must return entire instance with all the soureceFields - as on a user page it displays other values of DODGE as well.
now issue is- sort query still has to have knowledge to sort where - "sourceFieldId": 1560 (which is a sourceFieldId for color) to sort on color
is there a way to achieve such a sort query in ES without using dynamic scripting/dynamic templating? something like
"sort": {
"instance.instFields.fieldString.raw": (where sourceFieldId=1560?)
Should be able to achieve this using nested_filter option in sort
From the documentation:
nested_filter A filter that the inner objects inside the nested path
should match with in order for its field values to be taken into
account by sorting. Common case is to repeat the query / filter inside
the nested filter or query. By default no nested_filter is active.
For example to sort on color field it would be:
{
"sort": {
"instance.instFields.fieldValue.raws": {
"order": "asc",
"nested_path": "instance.instFields",
"nested_filter": {
"term": {
"instance.instFields.sourceFieldId": "1560"
}
}
}
}
}
Edited
"sort": [{
"instance.instFields.fieldValue": {
"order": "asc",
"nested_path": "instance.instFields",
"nested_filter": {
"term": {
"instance.instFields.sourceFieldId": "1560"
}
}
}
},
{
"instance.instFields.fieldValue": {
"order": "asc",
"nested_path": "instance.instFields",
"nested_filter": {
"term": {
"instance.instFields.sourceFieldId": "1558"
}
}
}
}
]
Using Elasticsearch 2, is it possible to return an aggregation where a document category matches a specific field value? For example, I want to get all the categories where categories.type = "application".
My mapping looks like this:
"mappings": {
"products": {
"_all": {
"enabled": true
},
"properties": {
"title": {
"type": "string"
},
"categories": {
"type":"nested",
"properties": {
"type": {
"type": "string",
"index": "not_analyzed"
},
"name": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
My query looks like this, which returns all category types, but I want to filter just the ones where categories.type = "application".
{
"query":{
"multi_match": {
"query": "Sound",
"fields": [
"title"
]
}
},
"aggs":{
"Applications": {
"nested": {
"path": "categories"
},
"aggs": {
"meta": {
"terms": {
"field": "categories.type"
},
"aggs": {
"name": {
"terms": {
"field": "categories.name"
}
}
}
}
}
}
}
}
You can use aggregation filter if I understand correctly:
{
size : 50,
"query":{
"multi_match": {
"query": "Sound",
"fields": [
"title"
]
}
},
"aggs":{
"Applications": {
"nested": {
"path": "categories"
},
"aggs": {
"meta": {
"filter" : {
"term" : {
"categories.type" : "application"
}
},
"aggs": {
"name": {
"terms": {
"field": "categories.name"
}
}
}
}
}
}
}
}
Hope that helpes.
You just need to replace "include": ".*" to "include": "application"
{
"query":{
"multi_match": {
"query": "Sound",
"fields": [
"title"
]
}
},
"aggs":{
"Applications": {
"nested": {
"path": "categories"
},
"aggs": {
"meta": {
"terms": {
"field": "categories.type"
, "include": "application"
},
"aggs": {
"name": {
"terms": {
"field": "categories.name"
}
}
}
}
}
}
}
}