Elastic search - handling the condition using must or must not query - elasticsearch

We have a requirement if newId is there then we have to get the data less than todays date
and if newId field is not there in the data then we have to get the data till expiry date + 2Months.
I was trying below query but result has not come as expected.
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":"2345"
},
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":null
},
{
"id":"235",
"startDate":"23/07/2020",
"endDate":"24/06/2020",
"newId":"2345"
},
Query that I was trying
{
"query": {
"bool": {
"must": [
{
"match_all": {}
},
{
"bool": {
"must": [
{
"bool": {
"must": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte":"now/d"
}
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte": "now-2M"
}
}
}
]
}
}
]
}
}
]
}
}
}
Expected result
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":"2345"
},
{
"id":"234",
"startDate":"23/07/2020",
"endDate":"24/09/20202",
"newId":null
},

Great start! Your query is almost right, but you need a few more tweaks, namely to use should instead of must, because both sub-queries will never be true at the same time:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "newId"
}
},
{
"range": {
"endDate": {
"gte": "now/d"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"endDate": {
"gte": "now-2M"
}
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "newId"
}
}
]
}
}
]
}
}
]
}
}
}

Related

Filter on field value or not exists

I'm trying to filter a field for a specific value OR that it does not exist.
I have the part for the specific value
{
"query": {
"match": {
"app.serviceType": {
"query": "MY_VALUE",
"type": "phrase"
}
}
}
}
I'd also like to add to this any case where the field serviceType doesn't exist at all.
Essentially I'd like the equivalent of this:
serviceType == "MY_VALUE" || string.IsNullOrEmpty(serviceType)
This request must match with your use case :
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "serviceType"
}
},
{
"match_phrase": {
"serviceType": "MY_VALUE"
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "serviceType"
}
}
]
}
}
]
}
}
}
Based on the previous answer (which didn't work but got me close) I was able to get it to work.
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "app.serviceType"
}
},
{
"match_phrase": {
"app.serviceType": "MY_VALUE"
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "app.serviceType"
}
}
]
}
}
]
}
}
}

combine two queries of elasticsearch?

I have a "date_created_tranx" and "phone_number_cust" fields. Few entries of date_created_tranx are null . I want to have particular phone_number within date_range and with null value.
a = {
"query": {
"bool": {
"must": [
{
"range": {
"date_created_tranx": {
"gte": "2019-12-01",
"lte": "2020-05-07"
}
}
},
{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}
]
}
}
}
b = {
"query": {
"bool": {
"must": [{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}],
"must_not": [{
"exists": {
"field": "date_created_tranx"
}
}
]
}
}
}
How to combine these ??
I cannot call it twice because The result is paginated
I am totally new to elastic search . Any leads will be helpful.
I tried
doc2 = {
"query" :{
"bool" : {
"must":[
a,
b
]
}
}
}
It throws
Error: RequestError: RequestError(400, 'parsing_exception', 'no [query] registered for [query]')
The query you're looking for is this one, i.e.:
We have a constraint on the phone number and we also check that either the date_created_tranx is within bounds or does not exist (i.e. is null).
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"range": {
"date_created_tranx": {
"gte": "2019-12-01",
"lte": "2020-05-07"
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "date_created_tranx"
}
}
}
}
],
"filter": [
{
"regexp": {
"phone_number_cust": ".*702625.*"
}
}
]
}
}
}

Elasticsearch query: range query on two fields, but one is optional field

I need to search my index based on a timestamp.
The documents have these field combinations:
start_time and end_time
or
just start_time (no end_time field)
Pseudo query: .
For a given timestamp, I wish to return all documents where an id matches, and also:
timestamp >= start_time && timestamp < end_time
but if there is no end_time field, then the query needs to be this:
(not exists end_time) && (timestamp > start_time)
Elastic query .
This is where I am going mad. I can't get an elastic query equivilent to that pseudo query above. Perhaps I am approaching it the wrong way (entirely possible). Here is what I have:
{
"query": {
"bool": {
"must": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"must": [
{
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
]
},
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
},
{
"filter": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
]
}
}
]
}
}
}
But this gives me [must] query malformed, no start_object after query name
How do I construct this query? Am I on the right track?
thanks in advance!
Your query is syntactically wrong. The correct query would be:
{
"query": {
"bool": {
"filter": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
}
]
}
}
]
}
}
}
There is a slight mistake in the logic. Ideally, comparison should be like this gte start_time_dt and lte end_time_dt. You did other way round so that translates to timestamp <= start_time && timestamp > end_time.
The correct query is
{
"query": {
"bool": {
"filter": [
{
"term": {
"id_s": "SomeIdValue"
}
},
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"range": {
"start_time_dt": {
"gte": "2019-07-12T03:20:22"
}
}
},
{
"range": {
"end_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
}
]
}
}
]
}
}
}
Hope this helps!!
I believe this block should be must not should as mentioned in the answer. The reason I say is both those conditions: must ( not exists AND range ) is what the OP intention I believe
{
"bool": {
"must": [ <====== mentioned it as should
{
"bool": {
"must_not": [
{
"exists": {
"field": "end_time_dt"
}
}
],
"must": [
{
"range": {
"start_time_dt": {
"lte": "2019-07-12T03:20:22"
}
}
}
]
}
},

Filter the records from nested query - Elastic Search

I am having nested array fields, I need to query and filter the records for that.
Sample
"test":{
"name":[
{
"name": "vanaraj",
"Age" : 26
},
{
"name": "vanaraj",
"Age" : 10
},
{
"name": "ranjit",
"Age" : 26
},
]
}
Here how I need the query for below conditions,
1. Where Name is equal to both ["vanaraj","ranjit"] to fetch
2. Add condition where Age > 25 for only "vanaraj"
I need a query like below, but it is not working.
{
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "data.test.name",
"query": {
"bool": {
"filter": [
{
"terms": {
"data.test.name.name": ["vanaraj","ranjit"]
}
}
]
}
}
}
},
{
"nested": {
"path": "data.test.name",
"query": {
"bool": {
"filter": [
{
"term": {
"data.test.name.name": "vanaraj"
}
},
{
"range": {
"data.test.name.Age": {
"gt": 25
}
}
}
]
}
}
}
}
]
}
}
}
Mapping :
{
"mappings":{
"properties":{
"test":{
"properties":{
"name":{
"type":"nested",
"properties":{
"Age":{
"type":"long"
},
"name":{
"type":"text",
"fields":{
"keyword":{
"type":"keyword",
"ignore_above":256
}
}
}
}
}
}
}
}
}
}
Based on your further clarification, the below code should help you:
Solution:
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "test.name",
"query": {
"bool": {
"must": [
{
"match": {
"test.name.name": "vanaraj"
}
},
{
"range": {
"test.name.Age": {
"gte": 26
}
}
}
]
}
}
}
},
{
"nested": {
"path": "test.name",
"query": {
"bool": {
"must": [
{
"match": {
"test.name.name": "ranjith"
}
}
]
}
}
}
}
]
}
}
}
The above solution would return you all the documents having name as ranjith OR if name is vanaraj and age > 25
Summary of query:
Bool
- Should
- Must clause for name=vanaraj and age >= 26
- Must clause for name=ranjith
Updated Solution:
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "test.name",
"query": {
"bool": {
"must": [
{
"match": {
"test.name.name": "vanaraj"
}
},
{
"range": {
"test.name.Age": {
"gte": 26
}
}
}
]
}
}
}
},
{
"nested": {
"path": "test.name",
"query": {
"bool": {
"must": [
{
"terms": {
"test.name.name": [
"abc",
"ranjit"
]
}
}
]
}
}
}
}
],
"must_not": [
{
"nested": {
"path": "test.name",
"query": {
"bool": {
"must": [
{
"terms": {
"test.name.name": [
"vanaraj"
]
}
},{
"range": {
"test.name.Age": {
"lte": 25
}
}
}
]
}
}
}
}
]
}
}
}
Please run the above and let me know if this helps!

Elasticsearch- nested conditional statements

I would like to develop multiple if else condition like this :
if(condition 1)
{
process 1
}
else
{
if(condition 2.1)
{
process 2
}
else (condition 2.2)
{ process 3
}
}
is bool with must and should the optimized way to do it or can script be used? As my query is already huge, since it has fuzziness and wildcard already.
Thanks
I think you can use painless script query for your use case. Bool must query will not work in this case I think.
You can refer this page for how to use if else in the script query
.https://www.elastic.co/guide/en/elasticsearch/painless/6.0/painless-examples.html
GET /books/_search
{
"_source": [
"id",
"name",
"user",
"privacy"
],
"query": {
"bool": {
"must": [
{
"term": {
"status": {
"value": 1
}
}
},
{
"bool": {
"minimum_should_match": 1,
"should": [
{ //if
"bool": {
"must": [
{
"term": {
"user.privacy.mode": {
"value": 0
}
}
},
{
"term": {
"privacy.mode": {
"value": 0
}
}
}
]
}
},
{//else if
"bool": {
"must": [
{
"term": {
"user.privacy.mode": {
"value": 2
}
}
},
{
"bool": {
"minimum_should_match": 1,
"should": [
{// if
"nested": {
"path": "readers",
"query": {
"match": {
"readers.id": "621120dc86b8920019295363"
}
}
}
},
{ // else
"nested": {
"path": "buyers",
"query": {
"match": {
"buyers.purchase.id": "621120dc86b8920019290f50"
}
}
}
}
]
}
}
]
}
},
{// else if
"bool": {
"must": [
{
"term": {
"privacy.mode": {
"value": 2
}
}
},
{
"bool": {
"minimum_should_match": 1,
"should": [
{
"nested": {
"path": "readers",
"query": {
"match": {
"readers.id": "621120dc86b89200195373"
}
}
}
},
{
"nested": {
"path": "buyers",
"query": {
"match": {
"buyers.purchase.id": "621120dc86b892001929036350"
}
}
}
}
]
}
}
]
}
}
]
}
}
],
"filter": {
"bool": {
"must_not": [
{
"term": {
"user.privacy.mode": 1
}
},
{
"term": {
"privacy.mode": 1
}
}
]
}
}
}
}
}

Resources