How can i create Elasticsearch curl query to get the field value which are not null and not empty(""),
Here is the mysql query:
select field1 from mytable where field1!=null and field1!="";
A null value and an empty string both result in no value being indexed, in which case you can use the exists filter
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"constant_score" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
}
}
}
}
'
Or in combination with (eg) a full text search on the title field:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"filtered" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
},
"query" : {
"match" : {
"title" : "search keywords"
}
}
}
}
}
'
As #luqmaan pointed out in the comments, the documentation says that the filter exists doesn't filter out empty strings as they are considered non-null values.
So adding to #DrTech's answer, to effectively filter null and empty string values out, you should use something like this:
{
"query" : {
"constant_score" : {
"filter" : {
"bool": {
"must": {"exists": {"field": "<your_field_name_here>"}},
"must_not": {"term": {"<your_field_name_here>": ""}}
}
}
}
}
}
On elasticsearch 5.6, I have to use command below to filter out empty string:
GET /_search
{
"query" : {
"regexp":{
"<your_field_name_here>": ".+"
}
}
}
Wrap a Missing Filter in the Must-Not section of a Bool Filter. It will only return documents where the field exists, and if you set the "null_value" property to true, values that are explicitly not null.
{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"bool":{
"must":{},
"should":{},
"must_not":{
"missing":{
"field":"field1",
"existence":true,
"null_value":true
}
}
}
}
}
}
}
You can do that with bool query and combination of must and must_not like this:
GET index/_search
{
"query": {
"bool": {
"must": [
{"exists": {"field": "field1"}}
],
"must_not": [
{"term": {"field1": ""}}
]
}
}
}
I tested this with Elasticsearch 5.6.5 in Kibana.
The only solution here that worked for me in 5.6.5 was bigstone1998's regex answer. I'd prefer not to use a regex search though for performance reasons. I believe the reason the other solutions don't work is because a standard field will be analyzed and as a result have no empty string token to negate against. The exists query won't help on it's own either since an empty string is considered non-null.
If you can't change the index the regex approach may be your only option, but if you can change the index then adding a keyword subfield will solve the problem.
In the mappings for the index:
"myfield": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
Then you can simply use the query:
{
"query": {
"bool": {
"must": {
"exists": {
"field": "myfield"
}
},
"must_not": {
"term": {
"myfield.keyword": ""
}
}
}
}
}
Note the .keyword in the must_not component.
You can use not filter on top of missing.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"missing": {
"field": "searchField"
}
}
}
}
}
}
Here's the query example to check the existence of multiple fields:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "field_1"
}
},
{
"exists": {
"field": "field_2"
}
},
{
"exists": {
"field": "field_n"
}
}
]
}
}
}
You can use a bool combination query with must/must_not which gives great performance and returns all records where the field is not null and not empty.
bool must_not is like "NOT AND" which means field!="", bool must exist means its !=null.
so effectively enabling: where field1!=null and field1!=""
GET IndexName/IndexType/_search
{
"query": {
"bool": {
"must": [{
"bool": {
"must_not": [{
"term": { "YourFieldName": ""}
}]
}
}, {
"bool": {
"must": [{
"exists" : { "field" : "YourFieldName" }
}]
}
}]
}
}
}
ElasticSearch Version:
"version": {
"number": "5.6.10",
"lucene_version": "6.6.1"
}
ES 7.x
{
"_source": "field",
"query": {
"bool": {
"must": [
{
"exists": {
"field":"field"
}
}
],
"must_not": [
{
"term": {
"field.keyword": {
"value": ""
}
}
}
]
}
}
}
We are using Elasticsearch version 1.6 and I used this query from a co-worker to cover not null and not empty for a field:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "myfieldName"
}
},
{
"not": {
"filter": {
"term": {
"myfieldName": ""
}
}
}
}
]
}
}
}
}
}
You need to use bool query with must/must_not and exists
To get where place is null
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "place"
}
}
}
}
}
To get where place is not null
{
"query": {
"bool": {
"must": {
"exists": {
"field": "place"
}
}
}
}
}
Elastic search Get all record where condition not empty.
const searchQuery = {
body: {
query: {
query_string: {
default_field: '*.*',
query: 'feildName: ?*',
},
},
},
index: 'IndexName'
};
Related
I would like to match within a boolean query in Elasticsearch. I have the match query and boolean query working as expected now, but I am not sure how to have a AND to combine them.
nested boolean
{
"query": {
"constant_score" : {
"filter":{
"bool":{
"must":[
{"terms":{"address.keyword": addr}},
{"bool":{
"should":[
{"terms": {"state.keyword": state}}
,{"terms": {"city.keyword": city}}
]
}}
]
}
}
}}}
match
{"query": {
"match": {
"auct_title": {
"query": keyword,
"operator": "and"
}
}
}
, "collapse" : {
"field" : "id"
}
,"sort" : [
{ sort_field: {"order" : sort_order} }]
,"size":20
}
You can move natch to the must clause . So document has to satisfy three conditions
1.address
2.either of state/city
2.match on auct_title
It will then return one document per Id based on sort order passed
GET <index>/_search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"term": {
"address.keyword": "addr"
}
},
{
"bool": {
"should": [
{
"term": {
"state.keyword": "state"
}
},
{
"term": {
"city.keyword": "city"
}
}
]
}
},
{
"match": {
"auct_title": {
"query": "keyword",
"operator": "and"
}
}
}
]
}
}
}
},
"collapse": {
"field": "id"
},
"sort": [
{
"FIELD": {
"order": "desc"
}
}
],
"size": 20
}
I have a nested documents as:
"someField": "hello",
"users": [
{
"name": "John",
"surname": "Doe",
"age": 2
}
]
according to this https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html, the above should match:
GET /_search
{
"query": {
"exists" : { "field" : "users" }
}
}
whereas the following should not,
"someField": "hello",
"users": []
but unfortunately both do not match. any ideas?
The example mentioned on the Elasticsearch blog refers to string and array of string types, not for nested types.
The following query should work for you:
{
"query": {
"nested": {
"path": "users",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "users"
}
}
]
}
}
}
}
}
Also, you can refer to this issue for more info, which discusses this usage pattern.
This works for me
GET /type/_search?pretty=true
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "outcome",
"query": {
"exists": {
"field": "outcome.outcomeName"
}
}
}
}
]
}
}
}
With the following index mapping:
{
"index_name": {
"mappings": {
"object_name": {
"dynamic": "strict",
"properties": {
"nested_field_name": {
"type": "nested",
"properties": {
"some_property": {
"type": "keyword"
}
}
}
}
}
}
}
}
I needed to use this query:
GET /index_name/_search
{
"query": {
"nested": {
"path": "nested_field_name",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "nested_field_name.some_property"
}
}
]
}
}
}
}
}
Elasticsearch version 5.4.3
The answer from user3775217 has worked for me but I needed to tweak it to work as expected for must_not. Essentially the bool/must needed to be wrapped around the nested portion of the query:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "users",
"query": {
"exists": {
"field": "users"
}
}
}
}
}
]
}
}
I have a query in Elastic search to get unique values for specific field.
How to get Unique values using where clause.
where field1:xyz, field2:yzx etc
{
"size": 20,
"aggs" : {
"pf" : {
"terms" : { "field" : "platform" }
}
}}
I think you are looking for aggregations with filters
{
"size":0, // <-- If you are using ES 2.0 or above, setting size=0 will only return aggregations and no results
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [{
"term": {
"field1": "xyz"
}
}, {
"term": {
"field2": "abc"
}
}]
}
}
}
},
"aggregations": {
"pf": {
"terms": {
"field": "platform"
}
}
}
}
We are using Elasticsearch 1.6 and I have a working three term query that I need to modify with a stand alone working missing filter. Here is the current code:
The original term query with three entries
GET ...
{
"query": {
"nested": {
"path": "MAIN_FIELD",
"query": {
"bool": {
"must": [
{
"term": {
"MAIN_FIELD.ID": 1234
}
},
{
"term": {
"MAIN_FIELD.OTHER_IND": "false"
}
},
{
"term": {
"MAIN_FIELD.INDICATOR": "Y"
}
}
]
}
}
}
}
}
The stand alone missing query:
GET ...
{
"query" : {
"filtered" : {
"filter" : {
"missing" : { "field" : "MAIN_FIELD.OTHER_IND" }
}
}
}
}
How do I change the term query from the first query:
"term": {
"MAIN_FIELD.OTHER_IND": "false"
}
to use a missing filter?
I think what you want is below:
{
"query": {
"nested": {
"path": "MAIN_FIELD",
"query": {
"bool": {
"must": [
{
"term": {
"MAIN_FIELD.ID": 1234
}
},
{
"filtered": {
"filter": {
"missing": {
"field": "MAIN_FIELD.OTHER_IND"
}
}
}
},
{
"term": {
"MAIN_FIELD.INDICATOR": "Y"
}
}
]
}
}
}
}
}
I'm trying to build a query that will return results only if they contain a certain field BUT only if that fields doesn't equal a specific value.
I can't manage the proper syntax:
POST webdata/interaction/_search
{
"query": {
"filtered": {
"filter": {
"exists": {
"field": "mediaType"
},
"and": {
"not" {
"term" : { "mediaType" : "none" }
}
}
}
}
}
}
Use the Bool Filter with must and must_not clauses.
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"exists": {
"field": "mediaType"
}
},
"must_not": {
"term": {
"mediaType": "none"
}
}
}
}
}
}
}