I am using Elasticsearch to search the Packetbeat indices to identify if two IP addresses communicate. If IP xx.xx.xx.xx talks to IP yy.yy.yy.yy OR if IP yy.yy.yy.yy talk to IP xx.xx.xx.xx, I want to know about it. Below is my DSL but all the returned results are not relevant at all. What am I doing wrong? Thanks!
GET /packetbeat-*/_search?size=100&pretty
{
"query": {
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
}
],
"must_not": [
{
"term": {
"source.ip": "127.0.0.1"
}
},
{
"term": {
"dest.ip": "127.0.0.1"
}
}
],
"should": [
{
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"term": {
"source.ip": "xx.xx.xx.xx"
}
},
{
"term": {
"dest.ip": "yy.yy.yy.yy"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"term": {
"source.ip": "yy.yy.yy.yy"
}
},
{
"term": {
"dest.ip": "xx.xx.xx.xx"
}
}
]
}
}
],
"filter": {
"range": {
"#timestamp": {
"gte": "now-30d/d",
"lte": "now-1d/d"
}
}
}
}
}
}
To simplify your query:
_type: flow
not localhost
source.ip != dest.ip
source.ip OR dest.ip equal to IP_X OR IP_Y
Have a look, according to this answer:
{
"query": {
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"script": {
"script": "doc['source.ip'].value != doc['dest.ip'].value"
}
},
{
"terms": {
"source.ip": [
"IP_X",
"IP_Y"
]
}
},
{
"terms": {
"dest.ip": [
"IP_X",
"IP_Y"
]
}
}
],
"must_not": [
{
"term": {
"source.ip": "127.0.0.1"
}
},
{
"term": {
"dest.ip": "127.0.0.1"
}
}
],
"filter": {
"range": {
"#timestamp": {
"gte": "now-30d/d",
"lte": "now-1d/d"
}
}
}
}
}
}
Related
I am trying to have this query in elasticsearch
statement:
(ip=xx.xx.xx.xx and status="running") or (ip=xx.xx.xx.xx and status="running")
can anyone tell me how can i write this in elasticsearch query.
Read about Bool Queries
Try this query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"term": {
"ip": {
"value": "xx.xx.xx.xx"
}
}
},
{
"term": {
"status": {
"value": "running"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"ip": {
"value": "bb.bb.bb.bb"
}
}
},
{
"term": {
"status": {
"value": "running"
}
}
}
]
}
}
]
}
}
}
Hi i have recently been busy reworking our old and outdated ES now um running 8.4 and im trying to create a similar query to what i had in the old one. But i have a hard time finding examples or the right documentation
This is an example what we had on version 1.7:
I have tried te exact same in 8.4 now
{
"index": "vehicles",
"type": "vehicle",
"body": {
"from": "0",
"size": 30,
"query": {
"filtered": {
"query": { "match_all": [] },
"filter": {
"bool": {
"must": [
{
"bool": {
"should": [
[
{ "term": { "make.untouched": "IVECO" } },
{ "term": { "make.untouched": "VOLKSWAGEN" } }
]
]
}
},
{
"bool": {
"should": [[{ "term": { "type_serie.untouched": "DAILY" } }]]
}
},
{
"bool": {
"should": [[{ "term": { "filters.header.camper": "true" } }]]
}
},
{ "range": { "days_sold": { "gte": 0, "lt": 2 } } },
{ "range": { "days_on_stock": { "gte": 3 } } },
{ "range": { "price": { "gt": 0 } } },
{ "range": { "price_trader": { "gt": 0 } } },
{ "term": { "show_in_search": true } }
],
"must_not": [
{ "term": { "offer_code": "z" } },
{ "term": { "stockingtype": "a" } },
{ "term": { "stockingtype": "v" } },
{ "term": { "stockingtype": "z" } },
{ "term": { "stockingtype": "p" } }
]
}
}
}
},
"sort": [
{ "_score": "desc" },
{ "days_on_stock": "asc" },
{ "price": "asc" }
]
}
}
You have double arrays and that's the issue (older versions were more permissive):
"should": [
remove this -->[
{ "term": { "make.untouched": "IVECO" } },
{ "term": { "make.untouched": "VOLKSWAGEN" } }
remove this -->]
]
Same here
same here same here
{ | |
"bool": { v v
"should": [[{ "term": { "type_serie.untouched": "DAILY" } }]]
}
},
{
"bool": {
"should": [[{ "term": { "filters.header.camper": "true" } }]]
}
},
After upgrading to Elasticsearch 2.x I got an issue with the following query:
{ "query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"terms": {
"_type": [
"xxx",
"yyy"
]
}
},
{
"exists": {
"field": "aaa"
}
},
{
"exists": {
"field": "bbb"
}
},
{
"exists": {
"field": "ccc"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "eee"
}
},
{
"term": {
"f": 0
}
}
]
}
}
]
}
}
} } }
Basically, I do not know how to replace the 'must' inside the 'should' filter with the new query DSL rules in Elasticsearch 2.x.
Thanks in advance.
You can simply remove the filtered/filter part and modify your query like this:
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"terms": {
"_type": [
"xxx",
"yyy"
]
}
},
{
"exists": {
"field": "aaa"
}
},
{
"exists": {
"field": "bbb"
}
},
{
"exists": {
"field": "ccc"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "eee"
}
},
{
"term": {
"f": 0
}
}
]
}
}
]
}
}
}
I cannot find examples of the syntax anywhere and the following does not work:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{ "term": { "category": "catname" }}
]
}
}
}
},
"post_filter": {
"terms": {"type": ["foo1", "foo2"] },
"range": { "price": { "gte": 300, "lte": 600 } }
}
}
You just need to wrap all your terms filters in an additional bool/must filter:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"category": "catname"
}
}
]
}
}
}
},
"post_filter": {
"bool": {
"must": [
{
"terms": { "type": [ "foo1", "foo2" ] }
},
{
"range": { "price": { "gte": 300, "lte": 600 } }
}
]
}
}
}
How can I order the results by _score?
I can't figure out how to calculate the score for each result, also :)
I managed to write this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"type_licitatie": "3"
}
},
{
"term": {
"tip_sursa": "5"
}
}
]
}
}
}
},
"sort": [
{
"_score": {
"order": "desc"
}
}
]
}
and this:
{
"query": {
"function_score": {
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"country_id": "1"
}
},
{
"term": {
"industry_id": "3"
}
}
]
}
}
}
},
"script_score" : {
"script": "(doc['country_id'].values=1) + (doc['industry_id'].values=3)"
},
"boost_mode": "replace"
}
}
}