Unable to run elastic search nested aggregate query - elasticsearch

I am trying to create a query that aggregates the sum of 3 different field and also matches three different conditions. I don't understand what the error message is saying.
The query below gives this specific error message:
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "Unknown key for a VALUE_NUMBER in [Type].",
"line": 1,
"col": 9
}
],
"type": "parsing_exception",
"reason": "Unknown key for a VALUE_NUMBER in [Type].",
"line": 1,
"col": 9
}
}
My query looks as follow:
{
"aggs": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"data.entity.productId": "45c29143b3bb4073a9fd325106784ce2"
}
},
{
"term": {
"data.entity.locationId": "c5f45ffc4fd94dcb926f96f1d5b9d835"
}
},
{
"term": {
"type.keyword": "StockLocationActivityAggregate"
}
}
]
}
}
},
"aggs": {
"directStock": {
"sum": { "field": "data.entity.inStock" },
"aggs": {
"directOutgoing": {
"sum": { "field": "data.entity.outgoing" },
"aggs": {
"directIncoming": { "sum": { "field": "data.entity.incoming" } }
}
}
}
}
}
},
"size": 0
}
Update
I am using the following index map
{
"mapping": {
"_doc": {
"properties": {
"active": {
"type": "boolean"
},
"data": {
"properties": {
"entity": {
"properties": {
"activityDate": {
"type": "date"
},
"creationDate": {
"type": "date"
},
"deleted": {
"type": "boolean"
},
"hash": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"inStock": {
"type": "float"
},
"incoming": {
"type": "float"
},
"locationId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"modifiedOn": {
"type": "date"
},
"modifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"orderId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"orderItemId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"outgoing": {
"type": "float"
},
"productId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"reservationDate": {
"type": "date"
},
"version": {
"type": "long"
}
}
},
"hash": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"modifiedOn": {
"type": "date"
},
"modifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tenantIdentifier": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"version": {
"type": "long"
}
}
},
"deleted": {
"type": "boolean"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"tenantId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"version": {
"type": "long"
}
}
}
}
}
I've also tried the example from the elastic search docs and the sample from Val below. They all give the same rror.

The sum aggregation is a metric aggregation that cannot have sub-aggregations... So you cannot do sum -> sum -> sum.
If you need the 3 different sums, you can do something like this:
{
...
"aggs": {
"directIncoming": {
"sum": {
"field": "data.entity.incoming"
}
},
"directStock": {
"sum": {
"field": "data.entity.inStock"
}
},
"directOutgoing": {
"sum": {
"field": "data.entity.outgoing"
}
}
}
}

Related

How to find objects with inner objects having multiple fields by specific values in Elastic Search

I have an index with objects named "DynamicFields" and each of them have inner objects named "Fields" like this:
{
"DynamicFields": [
{
"Fields": [
{
"DFieldVal": "Value1",
"Owned": 0,
"DFieldRelCode": 181254,
"DFieldCode": 1835
},
{
"DFieldVal": "Value2",
"Owned": 0,
"DFieldRelCode": 181255,
"DFieldCode": 1836
},
{
"DFieldVal": "Value3",
"Owned": 1,
"DFieldRelCode": 181256,
"DFieldCode": 1837
},
{
"DFieldVal": "Value4",
"Owned": 0,
"DFieldRelCode": 181257,
"DFieldCode": 1838
}
]
}
]
}
I need to find objects "DynamicFields" that has inner objects "Fields" with this exact values:
"DFieldCode": 1837
and
"Owned": 0
Im using this query for it, but it gives me wrong result, it should return an empty result because there isn't any inner object "Fields" having both of the values:
{
"from":0,
"size":10,
"query": {
"bool":{
"must":[
{ "terms": { "DynamicFields.Fields.Owned" : [0] } },
{ "terms": { "DynamicFields.Fields.DFieldCode" : [1837] } }
]
}
}
}
I think the problem is that Elastic search sees the inner objects properties as normal property for the Root Object so it returns the objects that have the mentioned fields in all inner objects no matter in the same inner object.
EDIT:
i have summarized the data to make it simpler
the mapping is full map of the data:
{
"marketplace": {
"mappings": {
"object": {
"properties": {
"Addresses": {
"properties": {
"AddrID": {
"type": "long"
},
"AddressText": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"AddressTree": {
"properties": {
"AddrFieldRelID": {
"type": "long"
},
"AddrTitleName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"AddrTitlePersianName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"AddrValName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Latitude": {
"type": "float"
},
"Longitude": {
"type": "float"
}
}
},
"Latitude": {
"type": "float"
},
"Longitude": {
"type": "float"
},
"Tel": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"DelFlag": {
"type": "long"
},
"DynamicFields": {
"properties": {
"DynamicDefCode": {
"type": "long"
},
"DynamicDefDataTypeName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"DynamicDefName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"DynamicValKind": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Fields": {
"properties": {
"DFieldCode": {
"type": "long"
},
"DFieldRelCode": {
"type": "long"
},
"DFieldVal": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Owned": {
"type": "boolean"
}
}
}
}
},
"GFRefCode": {
"type": "long"
},
"GoodsDesc": {
"properties": {
"FName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"GoodsFullName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Supplier": {
"properties": {
"Barcode": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"GPackDayPrice": {
"type": "long"
},
"GoodsEnterDate": {
"type": "date"
},
"GoodsFinalCode": {
"type": "long"
},
"GoodsFullName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"GoodsWHStock": {
"type": "long"
},
"StoreName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"UserName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"WHName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"WareHouseCode": {
"type": "long"
}
}
},
"UserName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"GoodsFinalCode": {
"type": "long"
},
"Images": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"IsMainObject": {
"type": "boolean"
},
"ObjectDetailPackID": {
"type": "long"
},
"ObjectKind": {
"type": "long"
},
"Prices": {
"properties": {
"Barcode": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"GPWeight": {
"type": "float"
},
"GpackDayPrice": {
"type": "long"
},
"PackingName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"RefGoodsFinalCode": {
"type": "long"
},
"TreePath": {
"properties": {
"DFieldCode": {
"type": "long"
},
"DFieldRelCode": {
"type": "long"
},
"DFieldVal": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
}
thanks.
As your index shows you saved your array as an object,
read more about this here
Basically unless specified otherwise elasticsearch flattens arrays when being saved, making objects in arrays lose their structure.
you should define the type of Fields as nested to avoid this.

Why is my term query not working?

Here is my settings file -
{
"settings": {
"mappings": {
"default": {
"properties": {
"AIRPORT_CODE": {
"type": "text"
},
"PROVINCE_NAME": {
"type": "text"
},
"AIRPORT_NAME": {
"type": "text"
},
"CITY_NAME": {
"type": "text"
},
"TYPE": {
"type": "keyword"
},
"COUNTRY_NAME": {
"type": "text"
}
}
}
}
}
}
Here are 2 documents that I ingested into the index -
Document - 1
{
"AIRPORT_CODE": "SQA",
"PROVINCE_NAME": "California",
"AIRPORT_NAME": "Santa Ynez Airport",
"CITY_NAME": "SANTA YNEZ",
"TYPE": "AIRPORT"
}
Document - 2
{
"PROVINCE_NAME": "SANTIAGO",
"CITY_NAME": "SANTIAGO",
"COUNTRY_NAME": "DOMINICAN REPUBLIC",
"TYPE": "HOTEL"
}
Here is my search query -
{
"size": 4,
"timeout": "2m",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "SQA"
}
}
],
"filter": [
{
"term": {
"TYPE": "AIRPORT"
}
}
]
}
},
"explain": false
}
The "TYPE" field has been defined as keyword. But the above query doesn't return anything. If change the "TYPE" value from "AIRPORT" to "airport" (lowercase), I get the results back. What am I doing wrong?
Note # My effort is to get this query working so I can take the advantage of filter cache (node query cache).
Thanks in advance!
Update - 1: adding out of GET index/_mapping
{
"rc-filter-cache": {
"mappings": {
"default": {
"properties": {
"AIRPORT_CODE": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"AIRPORT_NAME": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"CITY_NAME": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"COUNTRY_NAME": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"PROVINCE_NAME": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"TYPE": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"settings": {
"properties": {
"index": {
"properties": {
"analysis": {
"properties": {
"analyzer": {
"properties": {
"keylower": {
"properties": {
"tokenizer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
}
},
"mappings": {
"properties": {
"default": {
"properties": {
"properties": {
"properties": {
"AIRPORT_CODE": {
"properties": {
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"AIRPORT_NAME": {
"properties": {
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"CITY_NAME": {
"properties": {
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"COUNTRY_NAME": {
"properties": {
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"PROVINCE_NAME": {
"properties": {
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"TYPE": {
"properties": {
"analyzer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
Here is a way to solve that:
DELETE index
PUT index
{
"mappings": {
"default": {
"properties": {
"AIRPORT_CODE": {
"type": "text"
},
"PROVINCE_NAME": {
"type": "text"
},
"AIRPORT_NAME": {
"type": "text"
},
"CITY_NAME": {
"type": "text"
},
"TYPE": {
"type": "keyword"
},
"COUNTRY_NAME": {
"type": "text"
}
}
}
}
}
PUT index/default/1
{
"AIRPORT_CODE": "SQA",
"PROVINCE_NAME": "California",
"AIRPORT_NAME": "Santa Ynez Airport",
"CITY_NAME": "SANTA YNEZ",
"TYPE": "AIRPORT"
}
PUT index/default/2
{
"PROVINCE_NAME": "SANTIAGO",
"CITY_NAME": "SANTIAGO",
"COUNTRY_NAME": "DOMINICAN REPUBLIC",
"TYPE": "HOTEL"
}
GET index/_search
{
"query": {
"bool": {
"filter": [
{
"term": {
"TYPE": "AIRPORT"
}
}
]
}
}
}

Elasticsearch - using nested object value in Function Score

I currently have a nested object interest_scores in ES that looks like this:
[{
username: 'Somebody',
interest_scores: [
{ name: 'Running', score: 10 }
{ name: 'Food and drinks', score: 21 }
]
},
{
username: 'SomebodyElse',
interest_scores: [
{ name: 'Running', score: 7 }
{ name: 'Food and drinks', score: 29 }
]
}]
When I enter the search term Running I would like the user with the highest score for Running to get returned first.
I know the way to do this is to use a Function Score Query but I am not sure how to use the matching search term in the function / script. What I think is that the query will return all documents that have the interest "Running" and then I could use something like interest_scores.{match}.score to add to or multiply by the document score.
Any help with this would be greatly appreciated!
As requested, here is the mapping:
{
"influencers": {
"mappings": {
"influencer": {
"properties": {
"email": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"gender": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"geo": {
"type": "geo_point"
},
"hashtags": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"interest_scores": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"score": {
"type": "long"
}
}
},
"interests": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"language": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"location": {
"properties": {
"city": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"country_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"lat": {
"type": "float"
},
"lng": {
"type": "float"
},
"state_code": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"subdivision": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"network_data": {
"properties": {
"facebook": {
"properties": {
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"instagram": {
"properties": {
"bio": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"engagement": {
"type": "float"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"picture": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"reach": {
"type": "long"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"pinterest": {
"properties": {
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"twitter": {
"properties": {
"bio": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"engagement": {
"type": "float"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"picture": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"reach": {
"type": "long"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"youtube": {
"properties": {
"bio": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"engagement": {
"type": "float"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"picture": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"reach": {
"type": "long"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"videos": {
"type": "long"
},
"views": {
"type": "long"
},
"views_per_video": {
"type": "float"
}
}
}
}
},
"networks": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"picture": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"total_reach": {
"type": "long"
},
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
I do not have a function score query yet, I am only testing in the Dev Tools of Kibana - I do have all of the other filters working correctly though. I am just looking to say "If the search term matches a interest_scores.name then sort the hits by the interest_scores.score of that interest_scores.name
Update
The following seems to be working when I test it in Kibana dev tools:
{
"query": {
"nested": {
"path": "interest_scores",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"match": { "interest_scores.name": "Running" }
},
"script_score": {
"script": "_score + doc['interest_scores.score'].value"
}
}
}
}
}
}
I have tested it with a few different search terms and it always returns the highest score first, but what is weird is that I get the same results when I remove the script_score function. Can anyone tell me if this is a good solution, or why it works without the script_score?
As described here, you can sort by nested fields:
{
"_source": false, # for inner hits - you can remove it
"query": {
"nested": {
"path": "interest_scores",
"filter": {
"range": {
"interest_scores.score": {
"gte": "0"
}
}
},
"inner_hits": {} # for inner hits - you can remove it
}
},
"sort": {
"interest_scores.score": {
"order": "desc",
"mode": "max",
"nested_filter": {
"range": {
"interest_scores.score": {
"gte": "0"
}
}
}
}
}
}
*Pay attention that, you can use the inner_hits ability to show only relevant nested documents. If all inner hits documents are relevant - please remove the marked lines.
**Use the filter on score field or on any other field (e.g: name you would like to filter by).
EDIT 1:
If you want to get the sorted scores of specific name, try:
{
"_source": false,
"query": {
"nested": {
"path": "interest_scores",
"filter": {
"term": {
"interest_scores.name": "SCORE_NAME"
}
},
"inner_hits": {}
}
},
"sort": {
"interest_scores.score": {
"order": "desc",
"mode": "max",
"nested_filter": {
"range": {
"interest_scores.score": {
"gte": "0"
}
}
}
}
}
}
Put the desired score name instead SCORE_NAME.

Missing query for nested property in ElasticSearch

I am trying to write a query that returns all docs which don't have a particular field which is a nested property but it doesn't work.
I am using ES 5.4
{
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "shares"
}
}
]
}
}
}
What am I doing wrong?
This is my mapping
{
"test": {
"aliases": {},
"mappings": {
"vendor": {
"properties": {
"address": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"categories": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"city": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"displayTypes": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"emailId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"facebookAppId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"foursquareType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"googleTypes": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"hours": {
"properties": {
"day": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"from": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"to": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"isOnBoarded": {
"type": "boolean"
},
"kiaskCategories": {
"type": "keyword"
},
"latitude": {
"type": "float"
},
"location": {
"type": "geo_point"
},
"longitude": {
"type": "float"
},
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"phoneNumber": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"pictures": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"placeId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"price": {
"type": "float"
},
"rating": {
"type": "float"
},
"shares": {
"type": "nested",
"properties": {
"_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"content": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"createdDate": {
"type": "date"
},
"endDate": {
"type": "date"
},
"facebookLikes": {
"type": "long"
},
"images": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"price": {
"type": "long"
},
"quantity": {
"type": "long"
},
"shareType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"startDate": {
"type": "date"
},
"tags": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"source": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"state": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timezone": {
"type": "float"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"website": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"yelpTypes": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"creation_date": "1496319860700",
"number_of_shards": "5",
"number_of_replicas": "1",
"uuid": "0S0A9YN-S-SrW4eeOHX06w",
"version": {
"created": "5040099"
},
"provided_name": "test"
}
}
}
}
this should work
You have enforce nested data type mappings and use nested query.
{
"query" : {
"nested": {
"path": "shares",
"query": {
"bool": {
"must_not": [
{
"exists" : {
"field" : "shares._id"
}
}
]
}
}
}
}
}
Note: The following setup works for me.
PUT test_index
{
"mappings": {
"document_type" : {
"properties": {
"name" : {
"type": "text"
},
"shares" : {
"type": "nested"
}
}
}
}
}
POST test_index/document_type
{
"name" : "vicky"
}
POST test_index/_search
{
"query": {
"nested": {
"path": "shares",
"query": {
"bool": {
"must_not": [
{
"exists" : {
"field" : "shares.city"
}
}
]
}
}
}
}
}

Sort a nested array and return top 10 in elastic

I have a nested data type in an elastic index and want to sort this ascending for all returned results. I have tried the following:
GET indexname/_search
{
"_source" : ["m_iTopicID", "m_iYear", "m_Companies"],
"query": {
"terms":{
"m_iTopicID": [11,12,13]
}
},
"sort" : [
{
"m_Companies.value" : {
"order" : "asc",
"nested_path" : "m_Companies"
}
}
]
}
The mapping of the index as follows:
{
"indexname": {
"mappings": {
"topicyear": {
"properties": {
"m_Companies": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "float"
}
}
},
"m_People": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "float"
}
}
},
"m_Places": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "float"
}
}
},
"m_Subtopics": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"m_fActivation": {
"type": "float"
},
"m_iDocBodyWordCnt": {
"type": "long"
},
"m_iNodeID": {
"type": "long"
},
"m_iTopicID": {
"type": "long"
},
"m_iYear": {
"type": "long"
},
"m_szDocID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"m_szDocTitle": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"m_szGeo1": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "float"
}
}
},
"m_szSourceType": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"value": {
"type": "float"
}
}
},
"m_szSrcUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"m_szTopicNames": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
This returns all topics with ID 11, 12 or 13 with a list of m_Companies... but the lists aren't sorted ascending by the value field.
I would then like to only return the top 10 of each list. So the list doesn't return hundreds like currently but just n. If I can't achieve this option I will just obtain the top 10 at the front-end with a javascript splice(0,10) but it would be great if elastic could do this for me.
Thanks in advance.
Since you provided the sort in the main/parent level query, this will sort only the parent/root documents. As you might have observed with the results that documents are sorted with minimum value for m_Companes.value.
To sort the nested documents for each document you have to go deep inside the nested document and apply sort as m_Companies are subdocuments in the parent document. You have to use nested inner_hits and then sort the inner_hits.
This github issue has very good example of what i was trying to explain as how this sorts only the parent/root document based on values in nested documents.
Since you want all documents in nested, so you can let the nested query to fetch all nested documents using match_all and sort based on value field.
you can use the following query
{
"_source": ["m_iYear", "m_Companies"],
"query": {
"bool": {
"must": [{
"terms": {
"m_iTopicID": [11, 12, 13]
}
},
{
"nested": {
"path": "m_Companies",
"query": {
"match_all": {}
},
"inner_hits": {
"sort": [{
"m_Companies.value": "asc"
}]
}
}
}
]
}
},
"sort": [{
"m_Companies.value": {
"order": "asc",
"nested_path": "m_Companies"
}
}]
}
Hope this helps,
Thanks

Resources