ORDER BY the IN value list position - elasticsearch

I need to build elasticsearch query with sorting by field value positioning from an array.
Similar to MySQL:
SELECT * FROM `comments` ORDER BY FIELD(`id`,'17','3','5','12') DESC, id DESC;
or Postgres:
SELECT * FROM comments
LEFT JOIN unnest('{12,5,3,17}'::int[]) WITH ORDINALITY t(id, ord) USING (id) ORDER BY t.ord, id DESC;

You are looking for custom sorting in elasticsearch
it is possible to achieve it via painless script
here is what I do
PUT my_test
{
"mappings": {
"properties": {
"animal": {
"type": "keyword"
}
}
}
}
Populate docs
POST my_test/_doc
{
"animal": "mouse"
}
POST my_test/_doc
{
"animal": "cat"
}
POST my_test/_doc
{
"animal": "dog"
}
Custom sort
GET my_test/_search
{
"query": {
"match_all": {}
},
"sort": {
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": "if(params.scores.containsKey(doc['animal'].value)) { return params.scores[doc['animal'].value];} return 100000;",
"params": {
"scores": {
"dog": 0,
"cat": 1,
"mouse": 2
}
}
},
"order": "asc"
}
}
}

Just to compliment the answer above.
You can combine the approach to promote some items to the top and keep the origin sorting by relevance for all other items of the search result.
"sort": {
"_script": {
"type": "number",
"script": {
"lang": "painless",
"source": "if(params.scores.containsKey(doc['sku'].value)) { return params.scores[doc['sku'].value];} return 10",
"params": {
"scores": {
"3JK76": 0,
"8UF78": 1
}
}
},
"order": "asc"
},
"_score": { "order": "desc" }
}

Related

Elasticsearch conditional sorting by different fields (some fields can be empty)

I have objects like this:
{
subStatus: {
updatedAt: 3
},
statusUpdatedAt: 1
},
{
subStatus: null,
statusUpdatedAt: 2
}
I need to sort the objects conditionally as follows:
relative to the subStatus.updatedAt field if the subStatus field exists,
or relative to the statusUpdatedAt field if the subStatus field does not exist
If both fields are absent, objects should be sorted to the end of the list
I used the following query:
"sort": {
"_script": {
"type": "number",
"order": "asc",
"script": {
"lang": "painless",
"source": "if(doc['subStatus'].value != null) { return doc['subStatus.updatedAt'].value } else { return doc['statusUpdatedAt'].value }"
}
}
}
But I get an Error: "No field found for [subStatus] in mapping with types []"
Could you advise me how to solve the problem?
Try check by "size".
POST idx_fields/_bulk
{"index":{}}
{"subStatus": { "updatedAt": 3 }, "statusUpdatedAt": 1 }
{"index":{}}
{"subStatus": null, "statusUpdatedAt": 2 }
GET idx_fields/_search
{
"sort": {
"_script": {
"type": "number",
"order": "asc",
"script": {
"lang": "painless",
"source": "if(doc['subStatus.updatedAt'].size() > 0) { return doc['subStatus.updatedAt'].value } else { return doc['statusUpdatedAt'].value }"
}
}
}
}

Querying multiple indices but limit the results from a single index - Elasticsearch 6.x

All, I am using ES 6.7 and trying to return the results from a single index while querying two indices( customer, payment) and doing a terms lookup against user-customers index. The index I want data from(customer) has more fields than the second index. But for some reason, I only see results from the payment index. The fields customerName, customerNumber, state, address only exist on customer index. But I only want customers that has totalCredits > 0(This exists only on payment index) ordered by the logic in the sort array. I tried adding an _index filter( setting this to customer) but dint help. Adding source filtering doesn't help either. Is this doable in ES 6.7?. Am I left with the option of adding the fields in the sort array to the payment index or are there some other options?
ES query
GET customer,payment/_search
{
"sort": [
{
"customerName": {
"order": "asc",
"unmapped_type": "keyword"
}
},
{
"customerNumber": {
"order": "asc"
}
},
{
"state": {
"order": "asc",
"unmapped_type": "keyword"
}
},
{
"address": {
"order": "asc",
"unmapped_type": "keyword"
}
}
],
"query": {
"bool": {
"filter": [
{
"bool": {
"must_not": [
{
"terms": {
"status": [
"pending"
]
}
}
]
}
}
],
"must": [
{
"terms": {
"customerNumber": {
"index": "user-customers",
"type": "_doc",
"id": "rennish#emial.com",
"path": "users"
}
}
},
{
"range": {
"totalCredits": {
"gt": 0
}
}
}
]
}
}
}

ElasticSearch - How can I reuse script_fields field in aggregation?

It is possible to use a script_field to compute a field, 'emp_salary', and use in an aggregation query? Here's an example.
I have a script_fields script to compute the 'emp_salary', and I want to use it in the aggregation sub query but I get
{
"query": {
"term": {
"name.keyword": "John"
}
},
"script_fields": {
"emp_salary": {
"script": {
"lang": "painless",
"source": """return 1"""
}
}
},
"aggs": {
"average": {
"avg": {
"field": "_field['emp_salary']"
}
}
}
}
but I get null for the 'emp_salary'. Am I accessing the field value wrong?
"aggregations": {
"average": {
"value": null
}
}
Thanks

MySql Order By Value equivalent in ElasticSearch 5.6

ElasticSearch Version: 5.6
I have imported MySQL data in ElasticSearch and I have added mapping to the elastic search as required. Following is one mapping for the column application_status.
Mappings:
{
"settings": {
"analysis": {
"analyzer": {
"case_insensitive": {
"type": "custom",
"tokenizer": "keyword",
"filter": ["lowercase"]
}
}
}
},
"mappings": {
"lead": {
"properties": {
"application_status": {
"type": "string",
"analyzer": "case_insensitive",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}}
On the above mapping, I am able to do simple sorting (asc or desc) using following query:
{
"size": 50,
"from": 0,
"sort": [{
"application_status.keyword": {
"order": "asc"
}
}]}
which is MySql equivalent of
select * from <table_name> order by application_status asc limit 50;
Need help on following problem:
I have MySQL query which sorts based on application_status:
select * from vLoan_application_grid order by CASE WHEN application_status = "IP_QUAL_REASSI" THEN application_status END desc, CASE WHEN application_status = "IP_COMPLE" THEN application_status END desc, CASE WHEN application_status LIKE "IP_FRESH%" THEN application_status END desc, CASE WHEN application_status LIKE "IP_%" THEN application_status END desc
Please help me write the same query in ElasticSearch. I am not able to find order by value equivalent for strings in ElasticSearch. Searching online, I understood that, I should use sorting scripts but not able to find any proper documentation.
I have following query which just does simple sort.
{
"size": 500,
"from": 0,
"query" : {
"match_all": {}
},
"sort": {
"_script": {
"type": "string",
"script": {
"source": "doc['application_status.keyword'].value",
"params": {
"factor": ["IP_QUAL_REASS", "IP_COMPLE"]
}
},
"order": "desc"
}
}}
In the above query, I am not using params section as I am not aware how to use it for type: string
I believe I am asking too much. Please help or any relevant documentation links would be greatly appreciated. Hope question is clear. I'll provide more details if necessary.
You have two options:
the most performant one is to index at indexing time another field that should be a number. This number (your choice) will be the numerical representation of that status. Then at search time, you simply sort by that number and not by the status
at search time use a script that will do almost the same thing as the first option, but dynamically, and less performant (but still quite fast)
Below you have the second option:
"sort": {
"_script": {
"type": "number",
"script": {
"source": "if (params.factor[0].containsKey(doc['application_status.keyword'].value)) return params.factor[0].get(doc['application_status.keyword'].value); else return 1000;",
"params": {
"factor": [{
"IP_QUAL_REASS":1,
"IP_COMPLE":2,
"whatever":3
}
]
}
},
"order": "asc"
}
}
If you also want things like LIKE WHATEVER%, my suggestion is to consider an indexing time change, rather than search time because the script gets more complex. But, this is the one for wildcard matches as well:
"sort": {
"_script": {
"type": "number",
"script": {
"source": "if (params.factor[0].containsKey(doc['application_status.keyword'].value)) return params.factor[0].get(doc['application_status.keyword'].value); else { params.wildcard_factors[0].entrySet().stream().filter(kv -> doc['application_status.keyword'].value.startsWith(kv.getKey())).map(Map.Entry::getValue).findFirst().orElse(1000)}",
"params": {
"factor": [
{
"IP_QUAL_REASS": 1,
"IP_COMPLE": 2,
"whatever": 3
}
],
"wildcard_factors": [
{
"REJ_": 66
}
]
}
},
"order": "asc"
}
}

elastic search sort not working

Hi This is my sample output from elastic search
"table":{
"data":[
{
"label":"First Label",
"value":"10"
},
{
"label":"1st Label",
"value":"9"
}
],
"details":"Examples set on MSRP, your actual payment may vary based on price set by dealer."
}
And i wish to sort this in the ascending order, either thru the colunm label or through value.
The search options i tried are
sort = [{"data.label" : {"order" : "asc", "mode" : "min", "nested_path" : "data"}}];
sort = [{ "table.data": {"order": "asc"}]
But, I am not getting the expected sorted result
Any help on this will be greatly appreciated
Your sort part of the query should be -
"sort": { "label": { "order": "desc" }}
or
"sort": { "value": { "order": "desc" }}
or
"sort": [
{ "label": { "order": "desc" }},
{ "value": { "order": "desc" }}
]
Below is the mapping, docs and subsequent query to get inner sorted elements.
PUT /table
{
"mappings": {
"data": {
"properties": {
"name": {"type": "string"},
"subjects": {
"type": "nested",
"properties": {
"name": { "type": "string"},
"marks":{ "type": "integer"}
}
}
}
}
}
}
PUT /table/data/1?pretty
{
"name":"Ram",
"subjects":[
{
"name":"English",
"marks":13
},
{
"name":"Hindi",
"marks":12
}
]
}
PUT /table/data/2?pretty
{
"name":"Sam",
"subjects":[
{
"name":"Biology",
"marks":83
},
{
"name":"Maths",
"marks":68
}
]
}
PUT /table/data/3?pretty
{
"name":"Jim",
"subjects":[
{
"name":"Chemistry",
"marks":96
},
{
"name":"Geology",
"marks":58
}
]
}
GET table/data/_search
{
"query":{
"nested":{
"path":"subjects",
"query": {
"match_all": {}
},
"inner_hits":{
"sort":{
"subjects.marks":{
"order":"asc"
}
}
}
}
}
}
You have two options.
Use URL parameter
POST /index/_search?sort=table.data:asc
{}
Use Body
POST /index/_search
{
"sort":[{"table.data": "asc"}],
...
}
or
POST /index/_search
{
"sort":[{"table.data": {"order": "asc"}}],
...
}

Resources