I'm trying to get Elasticsearch to do the same thing that MongoDB does with the findOneAndUpdate method, but this doesn't seem to be possible.
The use case is that multiple servers and threads will look into the specific index for the next task to complete.
Therefore my best bet would be to update the "next" task/document with a unique ID and then retrieve the document afterwards.
This query will give me the next document to retrieve:
GET /test_index/_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "next_id"
}
}
}
},
"sort": {
"next_update": {"order": "asc"}
},
"size": 1
}
But I can't seem to figure out how to use the Update By Query API to update only a single row. I've been trying this query, but it updates every found document:
POST /test_index/_update_by_query
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "next_id"
}
}
}
},
"sort": {
"next_update": {"order": "asc"}
},
"script": {
"source": "ctx._source['next_update'] = params.next_id",
"params": {
"next_id": "xxxx"
}
}
}
How can I solve this?
You can use max_docs param in _update_by_query and set value to 1 so it will be executed for only one document.
You can check this documentation.
POST /test_index/_update_by_query
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "next_id"
}
}
}
},
"max_docs": 1,
"sort": {
"next_update": {"order": "asc"}
},
"script": {
"source": "ctx._source['next_update'] = params.next_id",
"params": {
"next_id": "xxxx"
}
}
}
Im trying to do a query where the field does not exists or is in the array of values, in mongo it would be something like this.
$or: [
{
field: {
$exists: false
}
},
{
field: [val1, val2]
}
]
I've tried with a couple of variations but cant seem to find the solution for this.
Edit1:
Basically on my mind, the query should look like.
query: {
"bool": {
"should": [
"must": {...logic}
"must_not": {...logic}
]
}
}
this returns
"must_not" malformed
You need to do it like this:
{
"query": {
"bool": {
"should": [
{
"terms": {
"field": [
"val1",
"val2"
]
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "field"
}
}
}
}
]
}
}
}
I have the following query:
{
"query": {
"exists": {
"field": "field1"
}
}
}
I would like my query to be modified in a way that I could add another filter with existing field having 'OR' operation between them.
Any idea how to do so?
Thanks.
You can use Bool Query with should.
{
"query": {
"bool": {
"should": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}
Use should Bool Query if either field1 OR field2 should exist from the resulting documents.
Example:
POST your_index/_doc/_search
{
"query": {
"bool": {
"should": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}
Use must Bool Query if either field1 AND field2 must exist in the output documents.
Example:
POST your_index/_doc/_search
{
"query": {
"bool": {
"must": [{
"exists": {
"field": "field1"
}
},
{
"exists": {
"field": "field2"
}
}
]
}
}
}
For example,i have 2 type docs,such as
{
"field2":"xx",
"field1","x"
}
{
"field1","x"
}
The one has 2 fields(field1 and field2),another one just has 1 field(field1).
Now,i want to query all docs which do not have field2 field?
EIDT
dsl:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "LableToMember"
}
}
]
}
}
}
doc:
{
"LableToMember": [
{
"xxx": "xxx",
"id": "1"
}
],
"field2":"xxx"
}
LableToMember is a nested field.I find exists api can't be used for nested field?
Note that in ES 5.x the missing query has been removed in favor of the exists one.
So if you want to be forward compatible, you should prefer using this:
POST /_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "field2"
}
}
}
}
}
UPDATE
If you want to retrieve all docs which don't have field2 or have field2 with a given value, you can do it like this:
POST /_search
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "field2"
}
}
}
},
{
"term": {
"field2": "somevalue"
}
}
]
}
}
}
In short you want to query those documents which have field2 missing. You can use Missing Query like:
"filter" : {
"missing" : { "field" : "field2" }
}
Hope it helps
How can i create Elasticsearch curl query to get the field value which are not null and not empty(""),
Here is the mysql query:
select field1 from mytable where field1!=null and field1!="";
A null value and an empty string both result in no value being indexed, in which case you can use the exists filter
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"constant_score" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
}
}
}
}
'
Or in combination with (eg) a full text search on the title field:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"filtered" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
},
"query" : {
"match" : {
"title" : "search keywords"
}
}
}
}
}
'
As #luqmaan pointed out in the comments, the documentation says that the filter exists doesn't filter out empty strings as they are considered non-null values.
So adding to #DrTech's answer, to effectively filter null and empty string values out, you should use something like this:
{
"query" : {
"constant_score" : {
"filter" : {
"bool": {
"must": {"exists": {"field": "<your_field_name_here>"}},
"must_not": {"term": {"<your_field_name_here>": ""}}
}
}
}
}
}
On elasticsearch 5.6, I have to use command below to filter out empty string:
GET /_search
{
"query" : {
"regexp":{
"<your_field_name_here>": ".+"
}
}
}
Wrap a Missing Filter in the Must-Not section of a Bool Filter. It will only return documents where the field exists, and if you set the "null_value" property to true, values that are explicitly not null.
{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"bool":{
"must":{},
"should":{},
"must_not":{
"missing":{
"field":"field1",
"existence":true,
"null_value":true
}
}
}
}
}
}
}
You can do that with bool query and combination of must and must_not like this:
GET index/_search
{
"query": {
"bool": {
"must": [
{"exists": {"field": "field1"}}
],
"must_not": [
{"term": {"field1": ""}}
]
}
}
}
I tested this with Elasticsearch 5.6.5 in Kibana.
The only solution here that worked for me in 5.6.5 was bigstone1998's regex answer. I'd prefer not to use a regex search though for performance reasons. I believe the reason the other solutions don't work is because a standard field will be analyzed and as a result have no empty string token to negate against. The exists query won't help on it's own either since an empty string is considered non-null.
If you can't change the index the regex approach may be your only option, but if you can change the index then adding a keyword subfield will solve the problem.
In the mappings for the index:
"myfield": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
Then you can simply use the query:
{
"query": {
"bool": {
"must": {
"exists": {
"field": "myfield"
}
},
"must_not": {
"term": {
"myfield.keyword": ""
}
}
}
}
}
Note the .keyword in the must_not component.
You can use not filter on top of missing.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"missing": {
"field": "searchField"
}
}
}
}
}
}
Here's the query example to check the existence of multiple fields:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "field_1"
}
},
{
"exists": {
"field": "field_2"
}
},
{
"exists": {
"field": "field_n"
}
}
]
}
}
}
You can use a bool combination query with must/must_not which gives great performance and returns all records where the field is not null and not empty.
bool must_not is like "NOT AND" which means field!="", bool must exist means its !=null.
so effectively enabling: where field1!=null and field1!=""
GET IndexName/IndexType/_search
{
"query": {
"bool": {
"must": [{
"bool": {
"must_not": [{
"term": { "YourFieldName": ""}
}]
}
}, {
"bool": {
"must": [{
"exists" : { "field" : "YourFieldName" }
}]
}
}]
}
}
}
ElasticSearch Version:
"version": {
"number": "5.6.10",
"lucene_version": "6.6.1"
}
ES 7.x
{
"_source": "field",
"query": {
"bool": {
"must": [
{
"exists": {
"field":"field"
}
}
],
"must_not": [
{
"term": {
"field.keyword": {
"value": ""
}
}
}
]
}
}
}
We are using Elasticsearch version 1.6 and I used this query from a co-worker to cover not null and not empty for a field:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "myfieldName"
}
},
{
"not": {
"filter": {
"term": {
"myfieldName": ""
}
}
}
}
]
}
}
}
}
}
You need to use bool query with must/must_not and exists
To get where place is null
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "place"
}
}
}
}
}
To get where place is not null
{
"query": {
"bool": {
"must": {
"exists": {
"field": "place"
}
}
}
}
}
Elastic search Get all record where condition not empty.
const searchQuery = {
body: {
query: {
query_string: {
default_field: '*.*',
query: 'feildName: ?*',
},
},
},
index: 'IndexName'
};