How to add multiple exists fields with OR operation in elasticsearch - elasticsearch

I have the following query:
{
"query": {
"exists": {
"field": "field1"
}
}
}
I would like my query to be modified in a way that I could add another filter with existing field having 'OR' operation between them.
Any idea how to do so?
Thanks.


You can use Bool Query with should.
{
"query": {
"bool": {
"should": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}


Use should Bool Query if either field1 OR field2 should exist from the resulting documents.
Example:
POST your_index/_doc/_search
{
"query": {
"bool": {
"should": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}
Use must Bool Query if either field1 AND field2 must exist in the output documents.
Example:
POST your_index/_doc/_search
{
"query": {
"bool": {
"must": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}

Related

Elasticsearch: Multiple fields, return true if any exists

ES version: 5.2 alpine
I have a document like this:
{
"field1": null,
"field2": "xyz"
"field3": null
}
I want to return the document if any of the fields above exists/not null.
"filter": {
"bool": {
"must": [
{
"exists": {
"field": ["field1","field2"]
}
}]
}
}
but I get following error.
[exists] unknown token [START_ARRAY] after [field]
Any idea how to do so with this ES version?
Thanks.

You cannot pass multiple fields in exists query. Use exists query for each field and wrap them in should clause as below:
{
"query": {
"bool": {
"should": [
{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
},
{
"exists": {
"field": "field3"
}
}
]
}
}
}

I am not sure you want to return the doc if field value exist, or also the value. This is for returning documents which satisfies the condition.
This can help,
You can use should query with minimum should match as 1.
"filter": {
"bool": {
"should": [
{ "exists": { "field": "field1 } },
{ "exists": { "field": "field2 } },
],
"minimum_should_match" : 1,
}
}
Try it, I hope it will work in your version as well.

Error in my elastic average query - malformed

I am getting a reason": "[query] query malformed, no start_object after query name" error, not sure why.
The query is meant to grab the difference between two date fields and calculate average of all the results, I believe this should work but it may not work.
Any help would be greatly appreciated.
I am on elastic version 5.6.12
query below:
POST index_my.test/_search
{
"size":10,
"query": {
"bool": {
"must": [
{
"query":
"match_all": {}
}
}
]
}
"filter": {
"and": [
{
"exists": {
"field": "activity.timeline.found"
}
}
{
"exists": {
"field": "activity.timeline.sent"
}
}
]
},
"aggs": {
"avg_timedifference": {
"avg": {
"script" : "Math.ceil(doc['activity.timeline.found'].value - doc['activity.timeline.sent'].value)"
}
}
}
}

You forgot a comma before "filter". Try this:
POST index_my.test/_search
{
"size":10,
"query": {
"bool": {
"must": [
{
"query":
"match_all": {}
}
}
]
},
"filter": {
"and": [
{
"exists": {
"field": "activity.timeline.found"
}
}
{
"exists": {
"field": "activity.timeline.sent"
}
}
]
},
"aggs": {
"avg_timedifference": {
"avg": {
"script" : "Math.ceil(doc['activity.timeline.found'].value - doc['activity.timeline.sent'].value)"
}
}
}
}

Get all docs which not contains the key?

For example,i have 2 type docs,such as
{
"field2":"xx",
"field1","x"
}
{
"field1","x"
}
The one has 2 fields(field1 and field2),another one just has 1 field(field1).
Now,i want to query all docs which do not have field2 field?
EIDT
dsl:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "LableToMember"
}
}
]
}
}
}
doc:
{
"LableToMember": [
{
"xxx": "xxx",
"id": "1"
}
],
"field2":"xxx"
}
LableToMember is a nested field.I find exists api can't be used for nested field?

Note that in ES 5.x the missing query has been removed in favor of the exists one.
So if you want to be forward compatible, you should prefer using this:
POST /_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "field2"
}
}
}
}
}
UPDATE
If you want to retrieve all docs which don't have field2 or have field2 with a given value, you can do it like this:
POST /_search
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "field2"
}
}
}
},
{
"term": {
"field2": "somevalue"
}
}
]
}
}
}

In short you want to query those documents which have field2 missing. You can use Missing Query like:
"filter" : {
"missing" : { "field" : "field2" }
}
Hope it helps

ElasticSearch query - exists but doesn't contain

I'm trying to build a query that will return results only if they contain a certain field BUT only if that fields doesn't equal a specific value.
I can't manage the proper syntax:
POST webdata/interaction/_search
{
"query": {
"filtered": {
"filter": {
"exists": {
"field": "mediaType"
},
"and": {
"not" {
"term" : { "mediaType" : "none" }
}
}
}
}
}
}

Use the Bool Filter with must and must_not clauses.
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"exists": {
"field": "mediaType"
}
},
"must_not": {
"term": {
"mediaType": "none"
}
}
}
}
}
}
}

Create Elasticsearch curl query for not null and not empty("")

How can i create Elasticsearch curl query to get the field value which are not null and not empty(""),
Here is the mysql query:
select field1 from mytable where field1!=null and field1!="";

A null value and an empty string both result in no value being indexed, in which case you can use the exists filter
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"constant_score" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
}
}
}
}
'
Or in combination with (eg) a full text search on the title field:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"filtered" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
},
"query" : {
"match" : {
"title" : "search keywords"
}
}
}
}
}
'

As #luqmaan pointed out in the comments, the documentation says that the filter exists doesn't filter out empty strings as they are considered non-null values.
So adding to #DrTech's answer, to effectively filter null and empty string values out, you should use something like this:
{
"query" : {
"constant_score" : {
"filter" : {
"bool": {
"must": {"exists": {"field": "<your_field_name_here>"}},
"must_not": {"term": {"<your_field_name_here>": ""}}
}
}
}
}
}

On elasticsearch 5.6, I have to use command below to filter out empty string:
GET /_search
{
"query" : {
"regexp":{
"<your_field_name_here>": ".+"
}
}
}

Wrap a Missing Filter in the Must-Not section of a Bool Filter. It will only return documents where the field exists, and if you set the "null_value" property to true, values that are explicitly not null.
{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"bool":{
"must":{},
"should":{},
"must_not":{
"missing":{
"field":"field1",
"existence":true,
"null_value":true
}
}
}
}
}
}
}

You can do that with bool query and combination of must and must_not like this:
GET index/_search
{
"query": {
"bool": {
"must": [
{"exists": {"field": "field1"}}
],
"must_not": [
{"term": {"field1": ""}}
]
}
}
}
I tested this with Elasticsearch 5.6.5 in Kibana.

The only solution here that worked for me in 5.6.5 was bigstone1998's regex answer. I'd prefer not to use a regex search though for performance reasons. I believe the reason the other solutions don't work is because a standard field will be analyzed and as a result have no empty string token to negate against. The exists query won't help on it's own either since an empty string is considered non-null.
If you can't change the index the regex approach may be your only option, but if you can change the index then adding a keyword subfield will solve the problem.
In the mappings for the index:
"myfield": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
Then you can simply use the query:
{
"query": {
"bool": {
"must": {
"exists": {
"field": "myfield"
}
},
"must_not": {
"term": {
"myfield.keyword": ""
}
}
}
}
}
Note the .keyword in the must_not component.

You can use not filter on top of missing.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"missing": {
"field": "searchField"
}
}
}
}
}
}

Here's the query example to check the existence of multiple fields:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "field_1"
}
},
{
"exists": {
"field": "field_2"
}
},
{
"exists": {
"field": "field_n"
}
}
]
}
}
}

You can use a bool combination query with must/must_not which gives great performance and returns all records where the field is not null and not empty.
bool must_not is like "NOT AND" which means field!="", bool must exist means its !=null.
so effectively enabling: where field1!=null and field1!=""
GET IndexName/IndexType/_search
{
"query": {
"bool": {
"must": [{
"bool": {
"must_not": [{
"term": { "YourFieldName": ""}
}]
}
}, {
"bool": {
"must": [{
"exists" : { "field" : "YourFieldName" }
}]
}
}]
}
}
}
ElasticSearch Version:
"version": {
"number": "5.6.10",
"lucene_version": "6.6.1"
}

ES 7.x
{
"_source": "field",
"query": {
"bool": {
"must": [
{
"exists": {
"field":"field"
}
}
],
"must_not": [
{
"term": {
"field.keyword": {
"value": ""
}
}
}
]
}
}
}

We are using Elasticsearch version 1.6 and I used this query from a co-worker to cover not null and not empty for a field:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "myfieldName"
}
},
{
"not": {
"filter": {
"term": {
"myfieldName": ""
}
}
}
}
]
}
}
}
}
}

You need to use bool query with must/must_not and exists
To get where place is null
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "place"
}
}
}
}
}
To get where place is not null
{
"query": {
"bool": {
"must": {
"exists": {
"field": "place"
}
}
}
}
}

Elastic search Get all record where condition not empty.
const searchQuery = {
body: {
query: {
query_string: {
default_field: '*.*',
query: 'feildName: ?*',
},
},
},
index: 'IndexName'
};

Resources