I would like to receive the lowest prices for the next and previous 15 days from my chosen date in my products index.
How can I get this prices in ES? What kind of query should I write?
My mapping:
{
"product-data": {
"mappings": {
"mine-apple": {
"properties": {
"date": {
"type": "date"
},
"productName": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"productDescription": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"price": {
"type": "long"
},
"query": {
"properties": {
"match_all": {
"type": "object"
}
}
}
}
}
}
}
}
Thanks in advance.
The solution I found: I added date-histogram to my query.In this way, grouping my query with date-histogram. At the latest I get minimum prices with minimum aggregation.
{
"query": {
"bool": {
"must": [
{
"range": {
"date": {
"gte": "2017-05-11",
"lte": "2017-05-14"
}
}
}
]
}
},
"size": 0,
"aggs": {
"sales_per_month": {
"date_histogram": {
"format": "YYYY-MM-dd",
"field": "date",
"interval": "day"
},
"aggs": {
"sales": {
"min": {
"field": "price"
}
}
}
}
}
}
Related
I have the following mapping:
{
"accountId": {
"type": "long"
},
"storeProductId": {
"type": "long"
},
"storeSchemaId": {
"type": "long"
},
"yoyoValues": {
"type": "nested",
"properties": {
"yoyoNameId": {
"type": "long"
},
"dataType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "long"
},
"languageId": {
"type": "long"
},
"value_Number": {
"type": "float"
},
"value_Raw": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
and I'm trying to get the max and min values for value_number for all nested documents with yoyoNameId of 3 that also has a parent document with an accountId of 1285 and storeSchemaId of 241.
Everytime I've tried, I've been unable to properly filter the nested documents so it ends up being the min and max values for all nested documents with the correct parent document values.
I've tried several different queries but my most recent one is as follows:
{
"size": 0,
"aggs": {
"filter-layer": {
"filters": {
"filters": [
{
"term": {
"accountId": 1285
}
},
{
"term": {
"yoyoSchemaId": 241
}
},
{
"nested": {
"path": "yoyoValues",
"query": {
"bool": {
"filter": [
{
"term": {
"yoyoValues.yoyoNameId": 3
}
}
]
}
}
}
}
]
},
"aggs": {
"yoyoValues": {
"nested": {
"path": "yoyoValues"
},
"inner": {
"filter": {
"term": {
"yoyoValues.yoyoNameId": 3
}
},
"aggs": {
"min_value": {
"min": {
"field": "yoyoValues.value_Number"
}
},
"max_value": {
"max": {
"field": "yoyoValues.value_Number"
}
}
}
}
}
}
}
}
}
Can someone please help me correct this query? I'm limited to elastic v7.13.
Below is the query. Inside this query, I need to add one more field which is category.keyword.
I am finding it difficult to add an additional field. Please modify my query and add the category keyword field also inside the Query.
My requirement is to show the list of questions and list of categories with the count.
{
"size": 0,
"aggs": {
"genres": {
"terms": {
"field": "question.keyword",
"order": {
"_count": "desc"
}
},
"aggs": {
"bucket_truncate": {
"bucket_sort": {
"from": 0,
"size": 10
}
}
}
}
}
}
Index Mapping details
"mapping": {
"properties": {
"answer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"category": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"question": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"relavence_score": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"source": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"timestamp": {
"type": "long"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
You can have multiple aggregations at the same level, in your case bucket_truncate and categories:
{
"size": 0,
"aggs": {
"questions": {
"terms": {
"field": "question.keyword",
"order": {
"_count": "desc"
}
},
"aggs": {
"categories": {
"terms": {
"field": "category.keyword",
"order": {
"_count": "desc"
}
}
},
"bucket_truncate": {
"bucket_sort": {
"from": 0,
"size": 10
}
}
}
}
}
}
I want to get all the buckets available for a particular aggregate. Is there any query or endpoint to get the buckets?
Below is my Mapping. If I query with any filter then the related buckets are coming up, but I want all the buckets to show it on the frontend to have or operations.
Example: If we have 2 records, one is with category as chair and the other is in the table. If I select a chair it is returning table count is zero but it should show as table count as 1. So user can select both.
MyMapping:
{
"properties": {
"australiasellable": {
"type": "boolean"
},
"avgRating": {
"type": "float"
},
"categories": {
"type": "nested"
},
"category": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"categorycode": {
"type": "text",
"fielddata": true
},
"categoryname": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"colour": {
"type": "text",
"fielddata": true
},
"commercialuse": {
"type": "boolean"
},
"customisable": {
"type": "boolean"
},
"depth": {
"type": "float"
},
"freedelivery": {
"type": "boolean"
},
"height": {
"type": "float"
},
"listprice": {
"type": "float"
},
"location": {
"type": "geo_point"
},
"material": {
"type": "text",
"fielddata": true
},
"materialcode": {
"type": "text",
"fielddata": true
},
"message": {
"type": "geo_point"
},
"numberOfRating": {
"type": "long"
},
"online": {
"type": "boolean"
},
"outdooruse": {
"type": "boolean"
},
"productid": {
"type": "long"
},
"productimageurl": {
"type": "text",
"fielddata": true
},
"productname": {
"type": "text",
"fielddata": true
},
"producttypecode": {
"type": "text",
"fielddata": true
},
"sellercode": {
"type": "text",
"fielddata": true
},
"sellerdescription": {
"type": "text",
"fielddata": true
},
"shortdescription": {
"type": "text",
"fielddata": true
},
"sku": {
"type": "text",
"fielddata": true
},
"state": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"stylecode": {
"type": "text",
"fielddata": true
},
"warrantycode": {
"type": "text",
"fielddata": true
},
"weight": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"width": {
"type": "float"
}
}
}
Regards,
Sreenivas
A possible solution would be not to set the filter in the query section of your payload but rather perform filtered aggregations and use the top_hits to get the _sources of the matched docs.
Long story short, if you apply a query, it'll of course affect your aggregations. So the trick is to not apply any query (either match_all or remove the whole query object) and perform the queries in the sub-aggregations as follows:
Using your category field:
GET your_index/_search
{
"size": 0,
"query": {
"match_all": {}
},
"aggs": {
"actual_query_agg": {
"filter": {
"term": {
"category.keyword": {
"value": "chair"
}
}
},
"aggs": {
"actual_query_agg_top_hits": {
"top_hits": {
"_source": [
"category"
],
"size": 10
}
}
}
},
"excluding_my_query_filtered_agg": {
"filter": {
"bool": {
"must_not": {
"term": {
"category.keyword": "chair"
}
}
}
},
"aggs": {
"by_other_categories_agg": {
"terms": {
"field": "category.keyword",
"size": 10
},
"aggs": {
"categorized_other_docs_agg_top_hits": {
"top_hits": {
"_source": [
"category"
],
"size": 10
}
}
}
}
}
}
}
}
You can get rid of the top_hits sub-aggregations if you're just interested in the counts and not the underlying docs, i.e.:
GET your_index/_search
{
"size": 0,
"query": {
"match_all": {}
},
"aggs": {
"actual_query_agg": {
"filter": {
"term": {
"category.keyword": {
"value": "chair"
}
}
}
},
"excluding_my_query_filtered_agg": {
"filter": {
"bool": {
"must_not": {
"term": {
"category.keyword": "chair"
}
}
}
},
"aggs": {
"by_other_categories_agg": {
"terms": {
"field": "category.keyword",
"size": 10
}
}
}
}
}
}
I have following mappings
PUT prod_nested
{
"mappings": {
"default": {
"properties": {
"pkey": {
"type": "keyword"
},
"original_price": {
"type": "float"
},
"tags": {
"type": "nested",
"properties": {
"category": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 30
}
}
},
"attribute": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 30
}
}
},
"original_price": {
"type": "float"
}
}
}
}
}
}
}
I am trying to do something like following sql aggregation
select tag_attribute,
tag_category,
avg(original_price)
FROM products
GROUP BY tag_category, tag_attribute
I am able to do the group-by part using nested aggregation on tags, but its not able to access the original_price in sub-aggregation. One option might be to duplicate the original_price inside the tags nested document, but I have millions of records to handle. My current aggregation is
GET prod_nested/_search?size=0
{
"aggs": {
"tags": {
"nested": {
"path": "tags"
},
"aggs": {
"categories": {
"terms": {
"field": "tags.category.keyword",
"size": 30
},
"aggs": {
"attributes": {
"terms": {
"field": "tags.attribute.keyword",
"size": 30
},
"aggs": {
"price": {
"avg": {
"field": "original_price"
}
}
}
}
}
}
}
}
}
}
Thanks, in advance.
I was able to get the desired results by using reverse_nested aggregation.
GET prod_nested/_search?size=0
{
"aggs": {
"tags": {
"nested": {
"path": "tags"
},
"aggs": {
"categories": {
"terms": {
"field": "tags.category.keyword",
"size": 10
},
"aggs": {
"attributes": {
"terms": {
"field": "tags.attribute.keyword",
"size": 10
},
"aggs": {
"parent_doc_price": {
"reverse_nested": {},
"aggs": {
"avg_price": {
"avg": {
"field": "original_price"
}
}
}
}
}
}
}
}
}
}
}
}
i think what you want is not possible.
but how about change your mapping?
{
"mappings": {
"default": {
"properties": {
"pkey": {
"type": "keyword"
},
"original_price": {
"type": "float"
},
"tags": {
"type": "nested",
"properties": {
"category_attribute": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 30
}
}
},
"category": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 30
}
}
},
"attribute": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 30
}
}
},
"original_price": {
"type": "float"
}
}
}
}
}
}
}
you can use category_attribute.
and your aggregation will be below.
GET prod_nested/_search?size=0
{
"aggs": {
"tags": {
"nested": {
"path": "tags"
},
"aggs": {
"category_attribute": {
"terms": {
"field": "tags.category_attribute.keyword",
"size": 30
},
"aggs": {
"price": {
"avg": {
"field": "original_price"
}
}
}
}
}
}
}
}
I have an index which contains CustomerProfile documents. Each of this document in the CustomerInsightTargets(with the properties Source,Value) property can be an array with x items. What I am trying to achieve is an autocomplete (of top 5) on CustomerInsightTargets.Value grouped by CustomerInisghtTarget.Source.
It will be helpful if anyone gives me hint about how to select only a subset of nested objects from each document and use that nested obj in aggregations.
{
"customerinsights": {
"aliases": {},
"mappings": {
"customerprofile": {
"properties": {
"CreatedById": {
"type": "long"
},
"CreatedDateTime": {
"type": "date"
},
"CustomerInsightTargets": {
"type": "nested",
"properties": {
"CustomerInsightSource": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"CustomerInsightValue": {
"type": "text",
"term_vector": "yes",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
},
"analyzer": "ngram_tokenizer_analyzer"
},
"CustomerProfileId": {
"type": "long"
},
"Guid": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Id": {
"type": "long"
}
}
},
"DisplayName": {
"type": "text",
"term_vector": "yes",
"analyzer": "ngram_tokenizer_analyzer"
},
"Email": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Id": {
"type": "long"
},
"ImageUrl": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"number_of_shards": "1",
"provided_name": "customerinsights",
"creation_date": "1484860145041",
"analysis": {
"analyzer": {
"ngram_tokenizer_analyzer": {
"type": "custom",
"tokenizer": "ngram_tokenizer"
}
},
"tokenizer": {
"ngram_tokenizer": {
"type": "nGram",
"min_gram": "1",
"max_gram": "10"
}
}
},
"number_of_replicas": "2",
"uuid": "nOyI0O2cTO2JOFvqIoE8JQ",
"version": {
"created": "5010199"
}
}
}
}
}
Having as example a document:
{
{
"Id": 9072856,
"CreatedDateTime": "2017-01-12T11:26:58.413Z",
"CreatedById": 9108469,
"DisplayName": "valentinos",
"Email": "valentinos#mail.com",
"CustomerInsightTargets": [
{
"Id": 160,
"CustomerProfileId": 9072856,
"CustomerInsightSource": "Tags",
"CustomerInsightValue": "Tag1",
"Guid": "00000000-0000-0000-0000-000000000000"
},
{
"Id": 160,
"CustomerProfileId": 9072856,
"CustomerInsightSource": "ProfileName",
"CustomerInsightValue": "valentinos",
"Guid": "00000000-0000-0000-0000-000000000000"
},
{
"Id": 160,
"CustomerProfileId": 9072856,
"CustomerInsightSource": "Playground",
"CustomerInsightValue": "Wiki",
"Guid": "00000000-0000-0000-0000-000000000000"
}
]
}
}
If i ran an aggregation on the top_hits the result will include all targets from a document -> if one of them match my search text.
Example
GET customerinsights/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "CustomerInsightTargets",
"query": {
"bool": {
"must": [
{
"match": {
"CustomerInsightTargets.CustomerInsightValue": {
"query": "2017",
"operator": "AND",
"fuzziness": 2
}
}
}
]
}
}
}
}
]
}
} ,
"aggs": {
"root": {
"nested": {
"path": "CustomerInsightTargets"
},
"aggs": {
"top_tags": {
"terms": {
"field": "CustomerInsightTargets.CustomerInsightSource.keyword"
},
"aggs": {
"top_tag_hits": {
"top_hits": {
"sort": [
{
"_score": {
"order": "desc"
}
}
],
"size": 5,
"_source": "CustomerInsightTargets"
}
}
}
}
}
}
},
"size": 0,
"_source": "CustomerInsightTargets"
}
My question is how I should use the aggregation to get the "autocomplete" Values grouped by Source and order by the _score. I tried to use a significant_terms aggregation but doesn't work so well, also terms aggs doesn't sort by score (and by _count) and having fuzzy also adds complexity.