Apply filter in Elastic search query - elasticsearch

Question

I want to apply filter after aggregate query. For example, with the below aggregate query, I want to get only those entries where we have all the windows.
Note: we do not have to use include because it uses regular expression which is time consuming and we cannot ignore the case.
Query:
GET /record_new/_search
{"size":0, "aggs" : {
"software_tags" : {
"terms" : {
"field" : "software_tags.keyword",
"size" : 100
}
}
}
}
Response:
{
"took": 77,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 5706542,
"max_score": 0,
"hits": []
},
"aggregations": {
"software_tags": {
"doc_count_error_upper_bound": 5514,
"sum_other_doc_count": 581800,
"buckets": [
{
"key": "Microsoft Windows",
"doc_count": 70641
},
{
"key": "Bitcoin",
"doc_count": 35423
},
{
"key": "Linux",
"doc_count": 33230
},
{
"key": "ICQ",
"doc_count": 21934
},
{
"key": "PHP",
"doc_count": 20562
},
{
"key": "Windows XP",
"doc_count": 19720
},
{
"key": "Android (operating system)",
"doc_count": 17774
},
{
"key": "C++",
"doc_count": 14792
},
{
"key": "Pretty Good Privacy",
"doc_count": 14307
},
{
"key": "Tor (anonymity network)",
"doc_count": 14110
}
]
}
}
}
I tried to do filter as well but I am not getting incorrect output. In output we are getting linux as well. I don't know what is happening here.
GET /record_new/_search
{"size":0, "query": {
"constant_score": {
"filter":
{ "term": { "software_tags": "windows" }}
}
}, "aggs" : {
"software_tags" : {
"terms" : {
"field" : "software_tags.keyword",
"size" : 10
}
}
}
}
Output:
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 93181,
"max_score": 0,
"hits": []
},
"aggregations": {
"software_tags": {
"doc_count_error_upper_bound": 1640,
"sum_other_doc_count": 171831,
"buckets": [
{
"key": "Microsoft Windows",
"doc_count": 70641
},
{
"key": "Windows XP",
"doc_count": 19720
},
{
"key": "Windows 7",
"doc_count": 12692
},
{
"key": "Linux",
"doc_count": 12311
},
{
"key": "Windows Vista",
"doc_count": 10172
},
{
"key": "Windows NT",
"doc_count": 5417
},
{
"key": "Windows Registry",
"doc_count": 5055
},
{
"key": "Windows 8",
"doc_count": 4829
},
{
"key": "Windows 2000",
"doc_count": 4738
},
{
"key": "Windows 10",
"doc_count": 4611
}
]
}
}
}

Answer 1

Try this query, it should look for records with windows in the software_tag:
{
"size":0,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "software_tags: *windows* AND NOT *linux* AND NOT *<next OS name to exclude>*",
"analyze_wildcard": true
}
}
]
}
}, "aggs" : {
"software_tags" : {
"terms" : {
"field" : "software_tags.keyword",
"size" : 10
}
}
}
}
It might be a bit slower than the usual queries but thats because of the wildcard character in the query.