I'm using the Elasticsearch date_histogram aggregation for binning/bucketing my data. This works fine when plotting the results of a single query:
{
"query": {...},
"aggs" : {
"timeline" : {
"date_histogram" : {
"field" : "date",
"interval" : "month"
}
}
}
}
However, I now want to use ES for binning/bucketing the results of multiple queries. At the end, I need a line chart with each query representing a single line on the chart.
So, is it possible to use a single bucketing for multiple queries?
Ok, ended up defining a custom range for the date field and executed multiple queries with the same custom range. Probably not the most efficient way, but works fine.
{
"query": {...},
"aggs" : {
"ranges" : {
"date_range" : {
"field": date,
"format": yyyyMMdd,
"ranges": ranges}
}
}
}
Related
Is it possible to save time fields in elastic search, in format like HH:mm and search then based on some time query range like HH:mm-HH:mm?
Yes, you can store time in elastic in this format check the related doc about the different date format here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html
hour_minute or strict_hour_minute
A formatter for a two digit hour of day and two digit minute of hour: HH:mm.
You will have a mapping like this if you use the build in format:
PUT my_index
{
"mappings": {
"properties": {
"date": {
"type": "date",
"format": "hour_minute"
}
}
}
}
To search you can use the build in format in your range query.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
GET /_search
{
"query": {
"range" : {
"age" : {
"gte" : "10:15",
"lte" : "20:13",
"format" : "hour_minute"
}
}
}
}
I am creating the date based elastic indexes like - logs-2017-06-10, logs-2018-07-10, logs-2019-06-11, date suffix can be any valid date.
How can i limit my search query to only search against specific days index.
for example if i want to seach between 2018-06-09 to 2018-06-11 then only below mentioned indexes should get searched against my query
logs-2018-06-09, logs-2018-06-10 and logs-2018-06-11
I tried wildcard * but it will not help here.
logs-2018-06-* will search in indexes logs-2018-06-01 to logs-2018-06-30 which is not my query.
how can i only limit it to
logs-2018-06-09,logs-2018-06-10 and logs-2018-06-11
GET /_search
{
"query": {
"indices" : {
"indices" : ["index1", "index2"],
"query" : { "term" : { "tag" : "wow" } },
"no_match_query" : { "term" : { "tag" : "kow" } }
}
}
}
From: https://www.elastic.co/guide/en/elasticsearch/reference/5.4/query-dsl-indices-query.html
I did not find any way to decide the dynamic list of indices against which I could run my search query.
As an alternative now I will run my search query against all the indices and based on date range query.
GET logs-2019-*/_search
{
"query": {
"range" : {
"timestamp" : {
"time_zone": "+01:00",
"gte": "2015-01-01 00:00:00",
"lte": "now"
}
}
}
}```
In my previous question, I was introduced to the fields in a query_string query and how it can help me to search nested fields of a document.
{
"query": {
"query_string": {
"fields": ["*.id","id"],
"query": "2"
}
}
}
But it only works for matching, what if I want to do some comparison? After some reading and testing, it seems queries like range do not support fields. Is there any way I can perform a range query, e.g. on a date, over a field that can be scattered anywhere in the document hierarchy?
i.e. considering the following document:
{
"id" : 1,
"Comment" : "Comment 1",
"date" : "2016-08-16T15:22:36.967489",
"Reply" : [ {
"id" : 2,
"Comment" : "Inner comment",
"date" : "2016-08-16T16:22:36.967489"
} ]
}
Is there a query searching over the date field (like date > '2016-08-16T16:00:00.000000') which matches the given document, because of the nested field, without explicitly giving the address to Reply.date? Something like this (I know the following query is incorrect):
{
"query": {
"range" : {
"date" : {
"gte" : "2016-08-16T16:00:00.000000",
},
"fields": ["date", "*.date"]
}
}
}
The range query itself doesn't support it, however, you can leverage the query_string query (again) and the fact that you can wildcard fields and that it supports range queries in order to achieve what you need:
{
"query": {
"query_string": {
"query": "\*date:[2016-08-16T16:00:00.000Z TO *]"
}
}
}
The above query will return your document because Reply.date matches *date
My document has the following mapping property:
"sid" : {"type" : "long", "store": "yes", "index": "not_analyzed"},
This property has only one long value for each record. I would like to query this property. I tried the following two queries:
{
"query" : {
"term" : {
"sid" : 10
}
}
}
{
"query" : {
"match" : {
"sid" : 10
}
}
}
Both queries work and return the target document. My question: which one is more efficient? And why?
You want to use a term query, and if you want to be even more effecient, use a filtered query so your results get cached.
GET index1/test/_search
{
"query": {
"filtered": {
"filter": {
"term": {
"sid": 10
}
}
}
}
}
Both work like the same way as you mentioned. As distinguished from match query the term query matches documents that have fields that contain a term (not analyzed!). So my opinion is that term query is more efficient in your case, because no analyzing have to be done.See:http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-query.html
I have an elasticsearch index containing products, which I can query for different search terms. Every product contains a field shop_id to reference the shop it belongs to. Now I try to display a list of all shops holding products for my query. (To filter by shops)
As far as I read on similar questions, I have to use an aggregation. Finally I built this query:
curl -XGET 'http://localhost:9200/searchindex/_search?search_type=count&pretty=true' -d '{
"query" : {
"match" : {
"_all" : "playstation"
}
},
"aggregations": {
"shops_count": {
"terms": {
"field": "shop_id"
}
}
}
}'
This should search for playstation and aggregate the results based on shop_id. Sadly it only returns
Data too large, data would be larger than limit of [8534150348]
bytes].
I also tried it with queries returning only 2 results.
The index contains more than 90,000,000 products.
I would suggest thats a job for a filter aggregation.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-filter-aggregation.html
Note: I don't know your product mapping in your index, so if that filter below doesn't work, try another filter from http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-filters.html
{
"aggs" : {
"in_stock_playstation" : {
"filter" : { "term" : { "change_me_to_field_for_product" : "playstation" } } },
"aggs" : {
"shop_count" : { "terms" : { "field" : "shop_id" } }
}
}
}
}