I'm trying the below format but it is returning an empty dictionary.
result_dict = es.search(index="comp*", body={"from": 0, "size": 10000,"query": {
"constant_score" : {
"filter" : {
"bool" : {
"must" : [
{ "term" : { "tags" :"prod" } },
{"term" :{ "severity" :"INFO" }}
{"term" :{ "service" :"abc-service" }}
]
}
}
}
} )
im making a Query on elasticSearch, of metricbeat, to rate the most used process per hourly, in these moment i'm aggregating per process start time, and process name, i need to "divide" these groups using field "#timestamp" hourly
that's my actual query
GET metricbeat*/_search?
{"query": {
"bool": {
"must": [
{ "wildcard" : { "beat.hostname" : "ibmcx*" }},
{ "range": {
"#timestamp": {
"gte": "2019-03-22T00:00:00",
"lte": "2019-03-23T00:00:00"}}},
{"terms" : { "beat.hostname" : ["ibmcxapp101", "ibmcxapp102", "ibmcxapp103",
"ibmcxapp104", "ibmcxapp105", "ibmcxapp106", "ibmcxapp107",
"ibmcxapp108", "ibmcxapp109", "ibmcxapp110", "ibmcxapp111",
"ibmcxapp112", "ibmcxapp113", "ibmcxapp114", "ibmcxapp115",
"ibmcxapp116", "ibmcxapp117", "ibmcxapp118", "ibmcxapp119",
"ibmcxapp120", "ibmcxapp121", "ibmcxapp122", "ibmcxxaa100",
"ibmcxxaa101", "ibmcxxaa102", "ibmcxxaa103", "ibmcxxaa104",
"ibmcxxaa105", "ibmcxxaa106", "ibmcxxaa107", "ibmcxxaa108",
"ibmcxxaa109", "ibmcxxaa110", "ibmcxxaa111", "ibmcxxaa112",
"ibmcxxaa201", "ibmcxxaa202", "ibmcxxaa203", "ibmcxxaa204"
] }},
{"exists": {"field": "system.process.cmdline"}}
],
"must_not": [
{"term" : { "system.process.username" : "NT AUTHORITY\\SYSTEM" }},
{"term" : { "system.process.username" : "NT AUTHORITY\\NETWORK SERVICE" }},
{"term" : { "system.process.username" : "NT AUTHORITY\\LOCAL SERVICE" }},
{"term" : { "system.process.username" : "NT AUTHORITY\\Servicio de red"}},
{"term" : { "system.process.username" : "" }}
]
}
},
"size": 0,
"aggs": {
"group_by_start_time": {
"terms": {
"field": "system.process.cpu.start_time"
},
"aggs": {
"group_by_name": {
"terms": {
"field": "system.process.name.keyword"
}
}
}
}
},
"size": 0,
"sort" : [
{ "system.process.cpu.start_time" : {"order" : "asc"}},
{ "#timestamp" : {"order" : "asc"}},
{ "system.process.pid" : {"order" : "desc"}}
]}
It's a bit hard to follow and reproduce — a minimal example (I think the entire query is not really needed) and sample docs would go a long way.
If you want to have an hourly aggregation, the first thing you'll need to do is that aggregation and then run the others inside.
The minimal example for an hourly aggregation would be:
POST /metricbeat*/_search?size=0
{
"aggs" : {
"metrics_per_hour" : {
"date_histogram" : {
"field" : "#timestamp",
"interval" : "hour"
}
}
}
}
Folding in the other aggregation would look like this:
POST /metricbeat*/_search?size=0
{
"aggs" : {
"metrics_per_hour" : {
"date_histogram" : {
"field" : "#timestamp",
"interval" : "hour"
},
"aggs" : {
...
}
}
}
}
PS: If you are using a daily index pattern, you could just use the right day instead of the wildcard one and then skip this part of the query:
"range": {
"#timestamp": {
"gte": "2019-03-22T00:00:00",
"lte": "2019-03-23T00:00:00"
}
}
My documents in Elasticsearch index have following format:
{
timestamp: "123456789",
tags: [
{ key:"tag1", "value": "val1" }, ...
]
}
I want get all documents which contain for example { key:"tag1" } and { key:"tag2", "value": "val2" } in their tags field.
How can I do this?
You can try with a bool query, where you specify how many nested query you need in the must section:
GET test_nested/test/_search
{
"query": {
"bool": {
"must": [
{"nested" : {
"path" : "tags",
"query" : {
"bool" : {
"must" : [
{ "match" : {"tags.key" : "tag1"} }
]
}
}
}},
{"nested" : {
"path" : "tags",
"query" : {
"bool" : {
"must" : [
{ "match" : {"tags.key" : "tag2"} },
{ "match" : {"tags.value" : "val2"} }
]
}
}
}}
]
}
}
}
In this case i have one nested query for selecting all documents with key "tag1" and the second nested query to select all documents with the "tag2" and "value2".
I am using elasticsearch 1.5.2. I stored some products with a field named "allergic" and some others without this field. And the values of this field can be fish or milk or nuts etc. I want to make a query and to get as a result only products which doesn't have at all this field called "allergic" and to integrate this to an other aggregation query. I want to make just one query: first eliminate products which have "allergic" field and then execute the aggregation query of the second block.
How to integrate this :
{
"constant_score" : {
"filter" : {
"missing" : { "field" : "allergic" }
}
}
}
to this aggregation query:
POST tes1/_search?search_type=count
{
"aggs" : {
"fruits" : {
"filter" : {
"query":{
"query_string": {
"query": "Fruits",
"fields": [
"category"
]
}
}},
"aggs" : {
"minprice": {
"top_hits": {
"sort": [
{
"prix en €/kg": {
"order": "asc"
}
}
], "size":400
}
}
}
}} }
You need to add the query part before the aggregation call. This will filter the results and then run aggregation on the resultset.
POST tes1/_search
{
"_source": false,
"size": 1000,
"query":
{ "constant_score" : {
"filter" : {
"missing" : { "field" : "allergic" }
}
}
},
"aggs" : {
"fruits" : {
"filter" : {
"query":{
"query_string": {
"query": "Fruits",
"fields": [
"category"
]
}
}},
"aggs" : {
"minprice": {
"top_hits": {
"sort": [
{
"prix en €/kg": {
"order": "asc"
}
}
], "size":400
}
}
}
}} }
On a side note please consider upgrading ElasticSearch to the latest version as 1.x is no longer supported.
I want to search data on ES in this order by index-> by index_type-> text search data.
When I'am using the below query on "_index" I expected to get list of index_types under that particular _index and also the related data but it returned nothing. On the other hand when I searched by _type I got the data pertaining to the index_type. Where have I gone wrong?
curl -XGET 'http://localhost:9200/_all/_search?pretty' -d '{
"facets": {
"terms": {
"terms": {
"field": "_index",
"size": 10,
"order": "count",
"exclude": []
},
"facet_filter": {
"fquery": {
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "*"
}
}
]
}
},
"filter": {
"bool": {
"must": [
{
"terms": {
"_index": [
"<index_name>"
]
}
}
]
}
}
}
}
}
}
}
},
"size": 0
}'
Note: I faced this problem first on Kibana, where I used the filter "_index":"name_of_index"; it returned no results but "_type":"name_of_index_type" returned the expected result. I found Kibana uses the above query behind the scenes to get the results of the filter I tried.
this is an example of query with pre filter ( "query" : "*" ) and then a must&mustnot query. then the resutlt is used to make the aggregations :
curl -XGET 'http://localhost:9200/YOUR_INDEX_NAME/_search?size=10' -d '{
"query" : {
"filtered" : {
"query" : {
"query_string" : {
"query" : "*"
}
},
"filter" : {
"bool" : {
"must" : [
{ "term" : { "E_RECORDEDBY" : "malençon, g."} },
{ "term" : { "T_SCIENTIFICNAME" : "peniophora incarnata" } }
],
"must_not" : [
{"term" : { "L_CONTINENT" : "africa" } },
{"term" : { "L_CONTINENT" : "europe" } }
]
}
}
}
},
"aggs" : {
"L_CONTINENT" : {
"terms" : {
"field" : "L_CONTINENT",
"size" : 20
}
}
},
"sort" : "_score"
}'