I need to search my index based on a timestamp.
The documents have these field combinations:
start_time and end_time
or
just start_time (no end_time field)
Pseudo query: .
For a given timestamp, I wish to return all documents where an id matches, and also:
timestamp >= start_time && timestamp < end_time
but if there is no end_time field, then the query needs to be this:
(not exists end_time) && (timestamp > start_time)
Elastic query .
This is where I am going mad. I can't get an elastic query equivilent to that pseudo query above. Perhaps I am approaching it the wrong way (entirely possible). Here is what I have:
{
"query": {
"bool": {
"must": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"must": [
{
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
]
},
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
},
{
"filter": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
]
}
}
]
}
}
}
But this gives me [must] query malformed, no start_object after query name
How do I construct this query? Am I on the right track?
thanks in advance!
Your query is syntactically wrong. The correct query would be:
{
"query": {
"bool": {
"filter": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
}
]
}
}
]
}
}
}
There is a slight mistake in the logic. Ideally, comparison should be like this gte start_time_dt and lte end_time_dt. You did other way round so that translates to timestamp <= start_time && timestamp > end_time.
The correct query is
{
"query": {
"bool": {
"filter": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"start_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
}
]
}
}
]
}
}
}
Hope this helps!!
I believe this block should be must not should as mentioned in the answer. The reason I say is both those conditions: must ( not exists AND range ) is what the OP intention I believe
{
"bool": {
"must": [ <====== mentioned it as should
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
},
Related
My current elasticsearch query is-
{
"must": [
{
"range": {
"firstClosedAt": {
"gte": 1667948400000,
"lte": 1668034800000
}
}
},
{
"term": {
"status": "CLOSED"
}
}
I want to modify it such that if "firstClosedAt" is null or not present then look for "closedAt".
Just like we have coalesce("firstClosedAt","closedAt") in sql
Help would be appreciated
There's no coalesce equivalent in ES, but you can do the query like below, which can read like: "either use firstClosedAt OR use closedAt if firstClosedAt does not exist":
{
"query": {
"bool": {
"filter": [
{
"term": {
"status": "CLOSED"
}
},
{
"bool": {
"minimum_should_match": 1,
"should": [
{
"range": {
"firstClosedAt": {
"gte": 1667948400000,
"lte": 1668034800000
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "firstClosedAt"
}
},
"filter": {
"range": {
"closedAt": {
"gte": 1667948400000,
"lte": 1668034800000
}
}
}
}
}
]
}
}
]
}
}
}
You could, however, create a much simpler query if you create another date field at indexing time which would either take the value of firstClosedAt or closedAt if firstClosedAt does not exist
We have a requirement if newId is there then we have to get the data less than todays date
and if newId field is not there in the data then we have to get the data till expiry date + 2Months.
I was trying below query but result has not come as expected.
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":"2345"
},
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":null
},
{
"id":"235",
"startDate":"23/07/2020",
"endDate":"24/06/2020",
"newId":"2345"
},
Query that I was trying
{
"query": {
"bool": {
"must": [
{
"match_all": {}
},
{
"bool": {
"must": [
{
"bool": {
"must": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte":"now/d"
}
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte": "now-2M"
}
}
}
]
}
}
]
}
}
]
}
}
}
Expected result
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":"2345"
},
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":null
},
Great start! Your query is almost right, but you need a few more tweaks, namely to use should instead of must, because both sub-queries will never be true at the same time:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte": "now/d"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"endDate": {
"gte": "now-2M"
}
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "newId"
}
}
]
}
}
]
}
}
]
}
}
}
I have this following query with elastic:
{
"query": {
"bool": {
"filter": [{
"terms": {
"participants.group": ["group1","group2"]
}
}, {
"range": {
"recordDate": {
"gte": "2020-05-14 00:00:00.000",
"lte": "2020-07-22 20:30:56.566"
}
}
}]
}
}
}
Currently, this finds records with participants with group "group1" and "group2".
How to change the query so it finds records with participants from "group1" or "group2?
Is it possible to do it without changing the structure of the query?
I'm assuming that the field participants.group is of keyword type and not text type.
Assuming that, the query you have roughly translates to (group1) or (group2) or (group1 and group2).
All you need to do is modify the query as below and add a must_not clause like below:
POST my_filter_index/_search
{
"query": {
"bool": {
"filter": [
{
"bool": {
"must": [
{
"range": {
"recordDate": {
"gte": "2020-05-14 00:00:00.000",
"lte": "2020-07-22 20:30:56.566"
}
}
}
],
"should": [
{
"terms": {
"participants.group": ["group1", "group2"]
}
}
]
}
}
],
"must_not": [
{
"bool": {
"must": [
{
"term": {
"participants.group": "group1"
}
},
{
"term": {
"participants.group": "group2"
}
}
]
}
}
]
}
}
}
Let me know if that works!
I have a "date_created_tranx" and "phone_number_cust" fields. Few entries of date_created_tranx are null . I want to have particular phone_number within date_range and with null value.
a = {
"query": {
"bool": {
"must": [
{
"range": {
"date_created_tranx": {
"gte": "2019-12-01",
"lte": "2020-05-07"
}
}
},
{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}
]
}
}
}
b = {
"query": {
"bool": {
"must": [{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}],
"must_not": [{
"exists": {
"field": "date_created_tranx"
}
}
]
}
}
}
How to combine these ??
I cannot call it twice because The result is paginated
I am totally new to elastic search . Any leads will be helpful.
I tried
doc2 = {
"query" :{
"bool" : {
"must":[
a,
b
]
}
}
}
It throws
Error: RequestError: RequestError(400, 'parsing_exception', 'no [query] registered for [query]')
The query you're looking for is this one, i.e.:
We have a constraint on the phone number and we also check that either the date_created_tranx is within bounds or does not exist (i.e. is null).
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"range": {
"date_created_tranx": {
"gte": "2019-12-01",
"lte": "2020-05-07"
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "date_created_tranx"
}
}
}
}
],
"filter": [
{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}
]
}
}
}
I am upgrading to elasticsearch 5.2 and have the following query, which now fails because the "missing" filter is deprecated:
{
"query": {
"bool": {
"should": [
{
"missing": {
"field": "birthday"
}
},
{
"range": {
"birthday": {
"lte": "20131231"
}
}
}
]
}
}
}
So, I am looking for documents that are either missing the birthday field or have a birthday less than 12/31/2013. The suggested replacement for "missing" is to use "must_not". I get that but how do I now do the same "or" query I had going on before? I have:
{
"query": {
"bool": {
"should": {
"range": {
"birthday": {
"lte": "20131231"
}
}
},
"must_not": {
"exists": {
"field": "birthday"
}
}
}
}
}
You're on the right path and almost there:
{
"query": {
"bool": {
"should": [
{
"range": {
"birthday": {
"lte": "20131231"
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "birthday"
}
}
}
}
]
}
}
}