Elasticsearch conditional sort on date field - sorting

I am trying to sort an Elastic Search query result on a date field, registeredAt. However, registeredAt doesn't exist in all documents returned. In that case, I would want the sort to look for the date on an alternative field, invitedAt.
If we have 3 hits which look like this:
hits = [
{
id: 'hit2'
registeredAt: '2021-06-01T23:00:00.000Z',
invitedAt: '2021-05-31T23:00:00.000Z'
},
{
id: 'hit3'
invitedAt: '2021-05-31T23:00:00.000Z'
},
{
id: 'hit1'
invitedAt: '2021-06-04T23:00:00.000Z'
},
],
then I would want the sort to return them in order from most recent to least recent: [hit1, hit2, hit3].
In each document, the sort script should look for the registeredAt field and take that date as the sort value and, if that field does not exist, look at the value for invitedAt and take that as the sort value.
In that sense, hit1 does not have a registeredAt and has the most recent date for invitedAt and, as such, should come first. hit2 has a registeredAt field and the date for that field is more recent than the invitedAt date of hit3 (which doesn't have a registeredAt field.
I have written the query as such:
client.search({
index: 'users',
track_total_hits: true,
sort: {
_script: {
type: 'number',
script: {
lang: 'painless',
source:
"if (!doc.containsKey('registeredAt') || doc['registeredAt'].empty) { return doc['invitedAt'].value; } else { return doc['registeredAt'].value }",
},
order: 'desc',
},
},
body: {
from: skip,
size: limit,
query: {...},
},
})
The query runs without errors but the sorting does not work and the documents are returned in the order that they were indexed in.

I assume that registeredAt and invitedAt are date in the mapping.
This query should work. What I added is calling .getMillis() after getting the value.
{
"sort": [
{
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": """
if (!doc.containsKey('registeredAt') || doc['registeredAt'].empty) {
return doc['invitedAt'].value.getMillis();
}
else {
return doc['registeredAt'].value.getMillis();
}
"""
},
"order": "desc"
}
}
]
}
Edit: .getMillis() is depricated in version 7.x. .toInstant().toEpochMilli() should be used instead.
This is the query:
{
"sort": [
{
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": """
if (!doc.containsKey('registeredAt') || doc['registeredAt'].empty) {
return doc['invitedAt'].value.toInstant().toEpochMilli();
}
else {
return doc['registeredAt'].value.toInstant().toEpochMilli();
}
"""
},
"order": "desc"
}
}
]
}

Related

ElasticSearch sorting based with wildcard in key

I have a data structure something like this from query. I want to apply a sort based on the date in the object values.
{
users: {
"1234": {
name: "User 1",
joining_date: "2022-12-28T11:37:00.000Z"
},
"3456": {
name: "User 2",
joining_date: "2022-12-18T11:37:00.000Z"
}
}
}
This is my query so far.
GET /_search
{
"sort" : [ {
"users.*.joining_date": {
"order": "desc",
"format": "date",
"unmapped_type": "long"
} }
],
"query": {
"query_string": {
"query": "_schema:users"
}
}
}
The problem is with using a wildcard in the key. I have tried multiple combinations from the documentation but nothing worked so far. I will be grateful for any help.

On ElasticSearch, how to search based on the number of items in a field?

On Elasticsearch I have a field named Itinerary that can contain multiple values (from 1 up to 6), for example in the picture below there's 2 items in the field.
"Itinerary": [
{
"Carrier": "LH",
"Departure": "2021-07-04T06:55:00Z",
"Number": "1493",
"Arrival": "2021-07-04T08:40:00Z",
},
{
"Carrier": "LH",
"Departure": "2021-07-04T13:30:00Z",
"Number": "422",
"Arrival": "2021-07-04T16:05:00Z",
}
}
]
Is there a way to query the number of results that contains a certain amount of items in this particular field? Something like:
query: {
match: {
number_of_items_in_Itinerary_field : 4
}
}
You can achieve this by using a script query like this:
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['my_array_field'].length > params.param1",
"lang": "painless",
"params": {
"param1": 1
}
}
}
}
}
}
}
Here is more Information on script queries: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-query.html
and specifically on array types in painless scripts: https://www.elastic.co/guide/en/elasticsearch/painless/current/painless-operators-array.html

Elasticsearch - Sort query based on collapse results

I'm trying to group/stack items based on their SKU.
Currently if sorting from high to low, an item thats being sold for $10 or $1, will show the $1 item first (because it's also sold for $10 it will be placed in front of the array ofcourse). The sorting should only respect the lowest_price for its sorting operation, for only that specific SKU.
Is there a way so I can do sorting based on the lowest_price of for every SKU and only return 1 single item per SKU?
If the results from the collapse could be used as variable for the sorting, this could be solved but I haven't been able to find out how this work.
My item object looks like this:
{
itemId: String,
sku: String,
price: Number
}
This is my query:
let itemsPerPage = 25;
let searchQuery = {
from: itemsPerPage * page,
size: itemsPerPage,
_source: ['itemId'],
sort: [{'sale.price': 'desc'}],
query: {
bool: {
must: [],
must_not: []
}
},
collapse: {
field: 'sku',
inner_hits: [{
name: 'lowest_price',
size: 1,
_source: ['itemId'],
sort: [{
'price': 'asc'
}]
}
],
}
};
You need to add sort underneeth collapse.
example:
GET /test/_search
{
"query": {
"function_score": {
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"match" : {
"job_status" : "SUCCESS"
}
}
]
}
}
}
}
}
},
"collapse": {
"field": "run_id.keyword"
},
"sort": [
{
"#timestamp": {
"order": "desc"
}
}
]
}
This may solve your issue.

Elasticsearch painless scripting with dates

I have a document which has a date field. I'd like to sort by documents by the this date ASC, but ones with a date in the past i'd like at the end.
In my end, it's like i want to assign the document value to a new value:
- If date is > "utc now", then assign value to whatever the date is
- If date is < "utc now", then assign value to max date
Then, i can sort by this field ASC.
So, it seems the only way to achieve this is with painless scripting.
This is what i've got so far, works.. but not sure if it's the correct approach.
GET /listings/_search
{
"track_total_hits": true,
"from": 0,
"query": {
"match_all": {}
},
"size": 48,
"sort": [
{
"_script" : {
"type": "string",
"script": {
"lang": "painless",
"source": "if (doc['auctionOn.utc'].size() == 0) { return params['maxTimestamp'].toString(); } else { long timestampDoc = doc['auctionOn.utc'].value.toInstant().toEpochMilli();long timestampNow = new Date().getTime();if (timestampDoc > timestampNow) { return timestampDoc.toString(); } else { return params['maxTimestamp'].toString(); } }",
"params": {
"maxTimestamp": 9223372036854776000
}
},
"order": "asc"
}
}
]
}
can someone please advise if this is the correct/performant approach?

Sorting results in Elastic

Trying to sort query results now. Neither of the queries below works:
{
"sort": [
{
"name": {
"order": "asc"
}
}
],
"query": {
"match_all": {}
}
}
{
"query": {
"match_all": {}
},
"sort": [
{
"name": {
"order": "asc"
}
}
]
}
even though they seem to be valid, according to the (documentation). What am I missing here? And by the way, does the order of query parts matters at all in Elastic?
i have faced same case.If your name field mapped as string and anlayzed then query never works. Use multi field type where you can map name twice, one as string and analyzed and another as string and not analyzed. For Ex
'user_id' =>array('type'=>'integer'),
'name' =>array(
'type'=>'multi_field',
'fields'=>array(
'name'=>array('type'=>'string','index'=>'analyzed'),
'sort_name'=>array('type'=>'string','index'=> 'not_analyzed')
)
)
using above mapping, you can search on name field and u can sort using sort_name.
{
'query': {
'query_string':{'query':'user_name*','fields':['name']}
},
'sort':[
{
'name.sort_name': {
'order': 'asc'
}
}
]
}
Reference Link :
1) http://awesomism.co.uk/536336#
2) http://blog.wiercinski.net/2011/uncategorized/elasticsearch-sorting-on-string-types-with-more-than-one-value-per-doc-or-more-than-one-token-per-field/

Resources