I have a query that i use to filter and get sum of a column for each group in the dev tool.
GET readers/_search
{
"size": 0,
"aggs": {
"usr_agg": {
"terms": {
"field": "assetType.keyword"
},
"aggs": {
"by_device_os": {
"sum": {
"field": "count"
}
}
}
}
}
}
how can i apply this as a filter to get this as a table?
Related
I'm using Elasticsearch and I see questions now and then about doing some aggregations with sorting or aggregations with paging, but I never see anything GROUP_BY, SUM, SORT, and PAGINATION together. If I were to write what I'm looking for as SQL, here it is (without the PAGINATION).
select invoice_date, address_2, company_name, sum(amount)
from my_table
group by invoice_date, address_2, company_name
order by sum(amount) desc
I tried doing this using many different techniques like composite aggregation, however it appears I can't do the ORDER_BY with this on the summation.
# composite aggregation
POST /746ee3a6-2b87-4288-9f20-3bf3a9e47e93/_search
{
"size": 0,
"aggs": {
"my_buckets": {
"composite": {
"sources": [
{ "Address2": { "terms": { "field": "Address2" } } },
{ "Company_Description": { "terms": { "field": "Company_Description" } } },
{ "InvoiceDate": { "terms": { "field": "InvoiceDate" } } }
]
},
"aggregations": {
"summation": {
"sum": { "field": "GrossValue" }
}
}
}
}
}
I tried repeated nested aggregations but I saw a comment somewhere that with many nested levels you can't ORDER_BY either.
POST /746ee3a6-2b87-4288-9f20-3bf3a9e47e93/_search
{
"size":0,
"from":0,
"sort":[{"Address2":"asc"}],
"query":{"bool":{"must":[{"match":{"taxonomy_full_code":-1}}]}},
"track_total_hits":true,
"aggs":{
"agg0":{
"terms":{"field":"Address2"},
"aggs":{
"agg1":{
"terms":{"field":"Company_Description"},
"aggs":{
"agg2":{
"terms":{"field":"InvoiceDate"},
"aggs":{"sum(GrossValue)":{"sum":{"field":"GrossValue"}}
}
}
}
}
}
}
}
}
Same with multi-term aggregation
# multi-term aggregation
POST /746ee3a6-2b87-4288-9f20-3bf3a9e47e93/_search
{
"size": 0,
"aggs": {
"rule_builder": {
"multi_terms": {
"terms": [
{"field": "Address2"},
{"field": "Company_Description"},
{"field": "InvoiceDate"}
]
},
"aggs":{
"sum(GrossValue)":{"sum":{"field":"GrossValue"}}
}
}
}
}
I'm using Elasticsearch 7.14. Is what I'm looking for possible in this version?
SELECT
max( timestamp ) AS first_time,
min( timestamp ) AS last_time,
src_ip,
threat_target ,
count(*) as count
FROM
traffic
GROUP BY
src_ip,
threat_target
ORDER BY
first_time desc
LIMIT 0 ,10
I want to get this result, but I don't know how to get limit size and where to use sort
{
"size": 0,
"aggregations": {
"src_ip": {
"aggregations": {
"threat_target": {
"aggregations": {
"last_time": {
"max": {
"field": "`timestamp`"
}
},
"first_time": {
"min": {
"field": "`timestamp`"
}
}
},
"terms": {
"field": "threat_target.keyword"
}
}
},
"terms": {
"field": "src_ip.keyword"
}
}
}
}
Aggregation Pagination is generally not supported in Elastic Search, however, composite aggregation provides a way to paginate your aggregation.
Unlike the other multi-bucket aggregation the composite aggregation can be used to paginate all buckets from a multi-level aggregation efficiently.
Excerpt from Composite-Aggregation ES Docs.
CHECK: THIS
Except "ORDER BY first_time desc", below query should run fine for you. I don't think ordering on any fields other than the grouping fields (src_ip,
threat_target) is possible.
GET traffic/_search
{
"size": 0,
"aggs": {
"my_bucket": {
"composite": {
"size": 2, //<=========== PAGE SIZE
/*"after":{ // <========== INCLUDE THIS FROM Second request onwards, passing after_key of the last output here for next page
"src_ip" : "1.2.3.5",
"threat_target" : "T3"
},*/
"sources": [
{
"src_ip": {
"terms": {
"field": "source_ip",
"order": "desc"
}
}
},
{
"threat_target": {
"terms": {
"field": "threat_target"
}
}
}
]
},
"aggs": {
"first_time": {
"max": {
"field": "first_time"
}
}
}
}
}
}
I am trying to implement query which will sort aggregated results by the formula.
For example, we have the next entities:
{
"price":"1000",
"zip":"77777",
"field1":"1",
"field2":"5"
},
{
"price":"2222",
"zip":"77777",
"field1":"2",
"field2":"5"
},
{
"price":"1111",
"zip":"77777",
"field1":"1",
"field2":"5"
}
Now, my query without sorting looks like:
POST /entities/_search {
"size": 0,
"query": {
"term": {
"zip": {
"value": "77777"
}
}
},
"aggs": {
"my composite": {
"composite": {
"size": 500,
"sources": [{
"field1_term": {
"terms": {
"field": "field1"
}
}
},
{
"field2_term": {
"terms": {
"field": "field2"
}
}
}
]
},
"aggs": {
"avg_price_per_group": {
"avg": {
"field": "price"
}
},
"results_per_group": {
"top_hits": {
"size": 100,
"_source": {
"include": ["entity_id", "price"]
}
}
}
}
}
}
}
The first one I need to group result by field1 and field2 and then calculate the average price for each group.
Then I need to divide the price of each doc by average price value and sort documents based on this value.
Is it possible to do this somehow?
I am using elastic search 1.6.0.
Here is my aggregation query :
GET /a/dummydata/_search
{
"size": 0,
"aggs": {
"sum_trig_amber": {
"terms": {
"field": "TRIGGER_COUNT_AMBER"
}
},
"sum_trig_green": {
"terms": {
"field": "TRIGGER_COUNT_GREEN"
}
},
"sum_trig-red": {
"terms": {
"field": "TRIGGER_COUNT_RED"
}
}
}
}
Is there any way by which i can add three sum_trig_amber + sum_trig_red + sum_trig_green ?
This is a portion of the data I have indexed in elasticsearch:
{
"country" : "India",
"colour" : "white",
"brand" : "sony"
"numberOfItems" : 3
}
I want to get the total sum of numberOfItems on a per country basis, per colour basis and per brand basis. Is there any way to do this in elasticsearch?
The following should land you straight to the answer.
Make sure you enable scripting before using it.
{
"aggs": {
"keys": {
"terms": {
"script": "doc['country'].value + doc['color'].value + doc['brand'].value"
},
"aggs": {
"keySum": {
"sum": {
"field": "numberOfItems"
}
}
}
}
}
}
To get a single result you may use sum aggregation applied to a filtered query with term (terms) filter, e.g.:
{
"query": {
"filtered": {
"filter": {
"term": {
"country": "India"
}
}
}
},
"aggs": {
"total_sum": {
"sum": {
"field": "numberOfItems"
}
}
}
}
To get statistics for all countries/colours/brands in a single pass over the data you may use the following query with 3 multi-bucket aggregations, each of them containing a single-bucket sum sub-aggregation:
{
"query": {
"match_all": {}
},
"aggs": {
"countries": {
"terms": {
"field": "country"
},
"aggs": {
"country_sum": {
"sum": {
"field": "numberOfItems"
}
}
}
},
"colours": {
"terms": {
"field": "colour"
},
"aggs": {
"colour_sum": {
"sum": {
"field": "numberOfItems"
}
}
}
},
"brands": {
"terms": {
"field": "brand"
},
"aggs": {
"brand_sum": {
"sum": {
"field": "numberOfItems"
}
}
}
}
}
}