I'm looking in kibana to get the average speed each of four road ids within a specific date range. I have this aggregation code that doesn't bring up an errors, but it also Gateway Time-out every time I try. The two fields are there, but I can't seem to build the aggregation well enough to get my intended averages. Can someone see what I'm doing incorrectly? Here is my query:
GET herev322_*/_search
{
"aggs": {
"ns2ids":{
"filter": {
"bool":{
"must":[
{
"terms":{
"nS2ID": [
"102-4893",
"102-4894",
"102+10103",
"102+10104"
]
}
},
{
"range":{
"pBT":{
"from": "2021-01-01T07:00:00",
"to": "2021-02-01T19:00:00"
}
}
}
]
}
},
"aggs":{
"avg_speed":{
"avg":{
"field": "speed"
}
}
}
}
}
}
Related
I'm using Elasticsearch 6. I want to calculate sales differences (grouped by categories) across 2 date ranges and sort it afterwards.
In the example query below, I used Bucket Script aggregation to evaluate the sales differences before sorting it with Bucket Sort aggregation to obtain the top 10 largest sales difference.
{
"size":0,
"aggs":{
"categories":{
"terms":{
"field":"category"
},
"aggs":{
"months_range":{
"date_range":{
"field":"date",
"ranges":[
{ "from":"01-2015", "to":"03-2015", "key":"start_month" },
{ "from":"03-2015", "to":"06-2015", "key":"end_month" }
],
"keyed":true
},
"aggs":{
"sales":{
"sum":{
"field":"sales_amount"
}
}
}
},
"sales_difference":{
"bucket_script":{
"buckets_path":{
"startMonthSales":"months_range.start_month>sales", // correct syntax?
"endMonthSales":"months_range.end_month>sales" // correct syntax?
},
"script":"params.endMonthSales - params.startMonthSales"
}
},
"sales_bucket_sort":{
"bucket_sort":{
"sort":[
{
"sales_difference":{
"order":"desc"
}
}
],
"size":10
}
}
}
}
}
}
My questions:
Is my bucket path's syntax correct? Is it possible to access individual date_range bucket, e.g. months_range.end_month, in the bucket path?
How's the performance of executing a custom script in Elasticsearch as compared to running similar business logic in the application server?
I'm trying to compose a query in Elasticsearch that filters out documents with a specific field value, and also returns the number of documents that has been filtered out as an aggregation.
What I have so far is below, however, with my solution it seems that the documents are filtered out first, then after the filtering, the count is performed, which is making it always be 0.
{
"query":{
"bool":{
"must_not":[
{
"terms":{
"gender":[
"male"
]
}
}
]
}
},
"aggs":{
"removed_docs_count":{
"filter":{
"term":{
"gender":"male"
}
}
}
}
}
You don't need a query block, just aggs will provide you expected results.
{
"aggs":{
"removed_docs_count":{
"filter":{
"term":{
"gender":"male"
}
}
}
}
}
I have a query like
{
"query":{
"bool":{
"must":[
{
"range":{
"created_date":{
"gte":1801301,
"lte":1807061
}
}
}
]
}
},
"aggs":{
"rating":{
"filters":{
"filters":{
"neutral":{
"match":{
"rating":0
}
},
"positive":{
"match":{
"rating":1
}
},
"negative":{
"match":{
"rating":2
}
}
}
}
}
},
"size":0
}
The query filters documents by created_date. I use date range that covers two date ranges: current and previous. Like data for this month and previous month. This is needed in other calculations(original query is much bigger).
This query works, but it calculates the rating for current and previous date ranges. I need to calculate rating in shorter date range: created_date: 1804181-1807061.
Is there a way how I can do this?
You can use
{
"range: {
"created_date": {
"gte":"now-10d/d",
"lte":"now/d"
}
}
}
I'm thinking this will help for you. Let me know if you any questions
I have this query to get data from AWS elasticSearch instance v6.2
{
"query": {
"bool": {
"must": [
{
"term": {"logLevel": "error"}
},
{
"bool": {
"should": [
{
"match": {"EventCategory": "Home Management"}
}
]
}
}
],
"filter": [{
"range": { "timestamp": { "gte": 155254550880 }}
}
]
}
},
"size": 10,
"from": 0
}
My data has multiple EventCategories for example 'Home Management' and 'User Account Management'. Problem with this is inside should having match returns all data because phrase 'Management' is in both categories. If I use term instead of match, it don't returns anything at all even when the given value is exactly same as in document.
I need to get data when any of given category is matched with rest of filters.
EDIT:
There may none, one or more than one EventCategory be passed to should clause
I'm not sure why you added a should within a must. Do you expect to have more than one should cases? It looks a bit odd.
As for your question, you can't use the term query on an analysed field, but only on keyword typed fields. If your EventCategory field has the default mapping, you can run the term query against the default non-analysed multi-field of EventCategory as follows:
...
{
"term": { "EventCategory.keyword": "Home Management" }
}
...
Furthermore, if you just want to filter in/out documents without caring about their relevance, I'd recommend you to move all the conditions in the filter block, to speed-up your query and make a better use of the cache.
Below query should work.
I've just removed should and created two must clauses one for each of event and management. Note that the query is meant for text datatypes.
{
"query":{
"bool":{
"must":[
{
"term":{
"logLevel":"error"
}
},
{
"match":{
"EventCategory":"home"
}
},
{
"match":{
"EventCategory":"management"
}
}
],
"filter":[
{
"range":{
"timestamp":{
"gte":155254550880
}
}
}
]
}
},
"size":10,
"from":0
}
Hope it helps!
I feel like this shouldn't be as difficult as its turning out to be, I've been attempting to use the:
index/_search
and
index/_count
endpoints, using query, bool, must filter etc. It seems no matter how I construct it, I cannot use range and date, with the match filter. The elasticsearch documentation doesn't seem to show complex queries like this so I'm not exactly sure how to construct it. The main query I've been manipulating is:
{
"query":{
"bool":{
"must":{
"range":{
"date":{
"gte":"now-1d/d",
"lt" :"now/d"
}
},
"match":{
"KEY":"VALUE"
}
}
}
}
}
I either get "no query registered for date", or "unknown key for a start_object in match" Been all over stackoverflow and can't seem to find an answer to this, it seems like it should be quite a simple query to make against a data store such as this. What am I missing here?
must can take an array of conditions if you want to combine them. Try this format :
{
"query": {
"bool": {
"must": [
{
"range": {
"date": {
"gte": "now-1d/d",
"lt": "now/d"
}
}
},
{
"match": { "KEY": "VALUE" }
}
]
}
}
}