Range filter gives values out of range - elasticsearch

I have got a problem with my query :
I recive hits with StarDate out of filter range, for e.q 2016-09-07 ...
"query" :{
"bool" : {
"must" : {
"range": {
"StartDate" : {
"gte" : "2016-09-18 00:18:32"
}
}
},
"must" :{
"query_string":{
"query":"WR_WRO_GGAJOWICKA_B"
}
}


Your second must clause overrides the first one with the constraint on StartDate, and hence only the query_string query kicks in. You need to write your query like this instead:
{
"query": {
"bool": {
"must": [
{
"range": {
"StartDate": {
"gte": "2016-09-18 00:18:32"
}
}
},
{
"query_string": {
"query": "WR_WRO_GGAJOWICKA_B"
}
}
]
}
}
}

Related

ElasticSearch filtered bool query not working with multi_match

I have this installed on Ubuntu running ElasticSearch 6.2.3
I am new to ES so if this is duplicate or obvious I am sorry.
I just need to do a contains query that filters out items not marked as PUBLISHED.
Here is what i think should be returning the correct results but it returns nothing (0 hits):
GET /index/type/_search
{
"query": {
"bool" : {
"must" : {
"multi_match" : {
"query": "Funny black cat",
"operator": "and",
"fields": [ "title", "description"]
}
},
"filter": {
"term" : { "publishStatus" : "PUBLISHED" }
}
}
}
}
Furthermore, this simple filter query is also return 0 records...
GET /index/type/_search
{
"query": {
"bool": {
"filter": {
"term": {
"publishStatus": "PUBLISHED"
}
}
}
}
}

Well this is what worked for me (stopped using filter, changed to match):
GET index/type/_search
{
"query": {
"bool":{
"must":[
{
"multi_match" : {
"query": "dinner sydney",
"fields": [ "title^3", "description" ],
"operator": "and"
}
},{
"match": { "publishStatus": "PUBLISHED"}
}
]
}
}
}

Elasticsearch: How to get an exact match in a nested field

The mapping contains nested fields which shouldn't be analyzed (not sure if the 'not_analyzed' value is accurate). Is it possible to do an exact match on a nested field? In the query below the "metadata.value": "2014.NWJSD.47" still gets analyzed. Elasticsearch breaks up the string into several terms ("2014", "NWJSD", "47"). I tried to use "term" instead of "match" but this didn't return any result.
"mappings" : {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "text",
"index" : "not_analyzed"
},
"value" : {
"type" : "text",
"index" : "not_analyzed"
}
}
}
}
The Query:
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"match": {
"metadata.name": "number"
}
},
{
"match": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}

Try to use keyword instead of text in your mapping like:
{
"mappings": {
"your_type_name": {
"properties": {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "keyword"
},
"value" : {
"type" :"keyword"
}
}
}
}
}
}
}
These fields won't be analyzed. Then you should reindex your data and to query your data you should replace match (which is analyzed query) with term.
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"term": {
"metadata.name": "number"
}
},
{
"term": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}
}

I think you are looking for a query string query.
You can freely disable "analyze" option for that field in mapping option and reindex everything again but you could also check this query out:
as written here:
GET /_search
{
"query": {
"query_string" : {
"query" : "your string"
}
}
}

How to search both in range and match query in one merged request using Elasticsearch?

I have two assembled queries that work as expected.
First one uses constant score, while matching range between two values:
GET /_search
{
"query" : {
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 101000349
}
}
}
}
}
}
The second one searches for bool.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "Barcelona"
}
}]
}
}
}
}
}
Now I need to merge them and I am struggling how, because tried many combinations of putting in different scopes, but not successful.
So this query returns an error.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "sídlisko"
}
}]
}
}
},
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 1000010349
}
}
}
}
}
}
Error:
... failed to parse search source. expected field name but got
[START_OBJECT]

You could just put constant score query inside bool must clause
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"match": {
"name": "sidlisko"
}
},
{
"constant_score": {
"filter": {
"range": {
"locationId": {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
]
}
}
}
}
}

I've managed to establish this query and it appears to work.
This looks as the most optimised one.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"fullAddress": "sidlisko"
}
}]
}
},
"filter" : {
"range" : {
"locationId" : {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
}

Must not query elasticsearch

I have my request:
{
"size": 10,
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{"term": {"event": "matchmaking_done"}}
]
}
},
"filter": {
"range": {
"#timestamp": {
"gt" : "2016-06-01T00:00:00.000Z",
"lte" : "2016-06-01T00:05:00.000Z"
}
}
}
}
},
"aggs" : {
"user-ids" : {
"terms" : { "field" : "user_id",
"size": 0
}
}
}
}
And I need to add into this request parameter - does not contain field pvp_league! I tried add must_not but can't understand how to do this correct.
Help please!

You answered it yourself, but the ES 2.x way to do this is to not use the filtered query because it has been deprecated and it will be removed in ES 5.0. ES 2.x introduces the concept of the "filter" context rather than every query being either just a query or a filter; now every query is both a filter or a query (scored), just depending on the context it's used in.
For your query, this therefore becomes a little simpler because of the simplified bool / filter syntax:
{
"size":10,
"query":{
"bool":{
"must":[
{
"term":{
"event":"matchmaking_done"
}
}
],
"must_not":[
{
"exists":{
"field":"pvp_league"
}
}
],
"filter":[
{
"range":{
"#timestamp":{
"gt":"2016-06-01T00:00:00.000Z",
"lte":"2016-06-01T00:05:00.000Z"
}
}
}
]
}
},
"aggs":{
"user-ids":{
"terms":{
"field":"user_id",
"size":0
}
}
}
}
As a very big aside, specifying "size" : 0 for the terms aggregation, you are requesting all unique terms, up to INT_MAX. That is not a scalable request (works great with 10 user_ids, or even 100, but not 10000 users).
As a not-so-bad aside, your request doesn't need a query context at all because nothing about the search side of it cares about relevance. Your term query ("event" : "matchmaking_done") either matches or it doesn't. Since you either want it to match or not, but you don't really care about order inherently, you should use this in the filter context. This changes the request to:
{
"size": 10,
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "pvp_league"
}
}
],
"filter":[
{
"range": {
"#timestamp": {
"gt":"2016-06-01T00:00:00.000Z",
"lte":"2016-06-01T00:05:00.000Z"
}
}
},
{
"term": {
"event": "matchmaking_done"
}
}
]
}
},
"aggs": {
"user-ids": {
"terms": {
"field": "user_id",
"size": 0
}
}
}
}

I've found solution! It looks like this:
{
"size": 10,
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{"term": {"event": "matchmaking_done"}}
],
"must_not": [
{"filtered": {
"filter": {
"exists": {
"field": "pvp_league"
}
}
}
}
]
}
},
"filter": {
"range": {
"#timestamp": {
"gt" : "2016-06-01T00:00:00.000Z",
"lte" : "2016-06-01T00:05:00.000Z"
}
}
}
}
},
"aggs" : {
"user-ids" : {
"terms" : { "field" : "user_id",
"size": 0
}
}
}
}

Create Elasticsearch curl query for not null and not empty("")

How can i create Elasticsearch curl query to get the field value which are not null and not empty(""),
Here is the mysql query:
select field1 from mytable where field1!=null and field1!="";

A null value and an empty string both result in no value being indexed, in which case you can use the exists filter
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"constant_score" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
}
}
}
}
'
Or in combination with (eg) a full text search on the title field:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"filtered" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
},
"query" : {
"match" : {
"title" : "search keywords"
}
}
}
}
}
'

As #luqmaan pointed out in the comments, the documentation says that the filter exists doesn't filter out empty strings as they are considered non-null values.
So adding to #DrTech's answer, to effectively filter null and empty string values out, you should use something like this:
{
"query" : {
"constant_score" : {
"filter" : {
"bool": {
"must": {"exists": {"field": "<your_field_name_here>"}},
"must_not": {"term": {"<your_field_name_here>": ""}}
}
}
}
}
}

On elasticsearch 5.6, I have to use command below to filter out empty string:
GET /_search
{
"query" : {
"regexp":{
"<your_field_name_here>": ".+"
}
}
}

Wrap a Missing Filter in the Must-Not section of a Bool Filter. It will only return documents where the field exists, and if you set the "null_value" property to true, values that are explicitly not null.
{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"bool":{
"must":{},
"should":{},
"must_not":{
"missing":{
"field":"field1",
"existence":true,
"null_value":true
}
}
}
}
}
}
}

You can do that with bool query and combination of must and must_not like this:
GET index/_search
{
"query": {
"bool": {
"must": [
{"exists": {"field": "field1"}}
],
"must_not": [
{"term": {"field1": ""}}
]
}
}
}
I tested this with Elasticsearch 5.6.5 in Kibana.

The only solution here that worked for me in 5.6.5 was bigstone1998's regex answer. I'd prefer not to use a regex search though for performance reasons. I believe the reason the other solutions don't work is because a standard field will be analyzed and as a result have no empty string token to negate against. The exists query won't help on it's own either since an empty string is considered non-null.
If you can't change the index the regex approach may be your only option, but if you can change the index then adding a keyword subfield will solve the problem.
In the mappings for the index:
"myfield": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
Then you can simply use the query:
{
"query": {
"bool": {
"must": {
"exists": {
"field": "myfield"
}
},
"must_not": {
"term": {
"myfield.keyword": ""
}
}
}
}
}
Note the .keyword in the must_not component.

You can use not filter on top of missing.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"missing": {
"field": "searchField"
}
}
}
}
}
}

Here's the query example to check the existence of multiple fields:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "field_1"
}
},
{
"exists": {
"field": "field_2"
}
},
{
"exists": {
"field": "field_n"
}
}
]
}
}
}

You can use a bool combination query with must/must_not which gives great performance and returns all records where the field is not null and not empty.
bool must_not is like "NOT AND" which means field!="", bool must exist means its !=null.
so effectively enabling: where field1!=null and field1!=""
GET IndexName/IndexType/_search
{
"query": {
"bool": {
"must": [{
"bool": {
"must_not": [{
"term": { "YourFieldName": ""}
}]
}
}, {
"bool": {
"must": [{
"exists" : { "field" : "YourFieldName" }
}]
}
}]
}
}
}
ElasticSearch Version:
"version": {
"number": "5.6.10",
"lucene_version": "6.6.1"
}

ES 7.x
{
"_source": "field",
"query": {
"bool": {
"must": [
{
"exists": {
"field":"field"
}
}
],
"must_not": [
{
"term": {
"field.keyword": {
"value": ""
}
}
}
]
}
}
}

We are using Elasticsearch version 1.6 and I used this query from a co-worker to cover not null and not empty for a field:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "myfieldName"
}
},
{
"not": {
"filter": {
"term": {
"myfieldName": ""
}
}
}
}
]
}
}
}
}
}

You need to use bool query with must/must_not and exists
To get where place is null
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "place"
}
}
}
}
}
To get where place is not null
{
"query": {
"bool": {
"must": {
"exists": {
"field": "place"
}
}
}
}
}

Elastic search Get all record where condition not empty.
const searchQuery = {
body: {
query: {
query_string: {
default_field: '*.*',
query: 'feildName: ?*',
},
},
},
index: 'IndexName'
};

Resources