Range in elasticsearch not really working - elasticsearch

Question

I run this query:
curl -X GET "localhost:9200/mydocs/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool" : { "must" : [{"wildcard": {"guid": "14744*"}}, {"range": {"availability.start": {"lt": "now"}}}] }
}
}
'
I then get this response:
"hits" : [
{
"_index" : "mydocs",
"_type" : "_doc",
"_id" : "14744",
"_score" : 2.0,
"_source" : {
"guid" : "14744",
"availability" : {
"start" : "2021-03-28T22:00:00.000Z",
"end" : "2021-12-31T22:59:00.000Z"
},
"title" : "Some title"
}
}
]
What I actually want is results where today is in the range for the availability's start and end.
The above results says the document is available between
2021-03-28T22:00:00.000Z
and
2021-12-31T22:59:00.000Z
Today is 2021-04-15:15:00.000Z
So, what I shoud do is to add:
{"range": {"availability.end": {"gt": "now"}}}
isn't it correct? But when I run:
curl -X GET "localhost:9200/mydocs/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool" : { "must" : [{"wildcard": {"guid": "14744*"}}, {"range": {"availability.start": {"lt": "now"}}}, {"range": {"availability.end": {"gt": "now"}}}] }
}
}
'
I got an empty hits list.
Partial mapping:
{
mappings: {
_doc: {
properties: {
availability: {
properties: {
end: {
type: "keyword"
},
start: {
type: "keyword"
}
}
},
properties: {
guid: {
type: "keyword"
}
}
}
}
}

Answer 1

Your query is perfectly correct! Good job with that!
The problem is that the availability.* fields are defined as keyword.
They MUST be of type date in order for range queries on date values to deliver accurate results, otherwise the range queries will just perform a lexical (i.e. string) comparison of now vs the date values expressed as strings:
availability: {
properties: {
end: {
type: "date" <--- change this
},
start: {
type: "date" <--- and this
}
}
},
You can't change the mapping of existing fields, but you can always create new fields. So, you can change your mapping to create new date sub-fields for both start and end, like this:
PUT mydocs/_mapping
{
"properties": {
"availability": {
"properties": {
"end": {
"type": "keyword",
"fields": {
"date": {
"type": "date"
}
}
},
"start": {
"type": "keyword",
"fields": {
"date": {
"type": "date"
}
}
}
}
}
}
}
Then you simply need to run the following command in order to update your index:
POST mydocs/_update_by_query
And then modify your query to use the new sub-fields and that will work:
POST mydocs/_search
{
"query": {
"bool": {
"must": [
{
"wildcard": {
"guid": "14744*"
}
},
{
"range": {
"availability.start.date": {
"lt": "now"
}
}
},
{
"range": {
"availability.end.date": {
"gt": "now"
}
}
}
]
}
}
}