Can I reference the document itself in an elasticsearch query? - elasticsearch

I would like to get all documents where the value of fieldA is greater than the value of fieldB. What is the most efficient way to do this?
Kind of like this:
body: {
query: {
bool: {
must: {
range: {
"fieldA": { gt: "this.fieldB" }
}
}
}
}
}

The most efficient way to do it is by indexing fieldC with the value of fieldA - filedB and using a range filter to find all records with fieldC greater than 0. It can be also a boolean field that will have true if fieldA is greater than fieldB and false otherwise.
If reindexing is not an option it's possible to use script filter to perform this check, but it will essentially mean a full index scan for every search, so it is not going to scale.

Related

Python OpenSearch retrieve records based on element in a list

So I need to retrieve records based on a field called "cash_transfer_ids" which is a python list.
I want to retrieve all records whose cash_transfer_ids contain a specific id value (a string).
What should the query be look like? Should I use match or term query?
Example: I want to retrieve any record whose cash_transfer_ids field contains 'abc'
Then I may get record such as
record 1: cash_transfer_ids:['abc']
record 2: cash_transfer_ids:['dfdfd', 'abc']
etc...
Thanks very much for any help!
if cash_transfer_ids is type keyword I try filter with Term.
term = "abc"
query = {
"query": {
"term": {
"cash_transfer_ids": {
"value": term
}
}
}
}
response = get_client_es().search(index="idx_test", body=query)

Represent enum in Elastic Search for sorting

I have a use case to represent an enum for difficulty level (EASY, MEDIUM, DIFFICULT) in elastic search with support of sorting on this field. If this field is indexed as string the sorting will not work as expected.
One way to support this is to index integer values for each enumeration in ES and map it to string values when sorted results are returned by ES.
Are there other alternatives such that ES itself takes care of sorting in the enumeration order while this field is indexed as string? Can I specify custom sort function for a field? function_score is an option, but given that I have to sort based on enum ordering is there better way than defining custom function_score?
In my use case there are multiple such enumeration defining scale across dimensions like difficulty, height (low, medium, high), grades (good, average, poor), etc. Both the above solution requires custom work as a new dimension is introduced. Can either of the above approach be generalzied?
You can check the answer to the same question here. You will need to use script_score like below:
GET /my-index-2/_search
{
"query": {
"script_score": {
"query": {
"match_all":{}
},
"script": {
"source": "if (doc['field name'].value == 'EASY'){2} else if(doc['field name'].value == 'MEDIUM') {1} else if(doc['field name'].value == 'DIFFICULT') {0}"
}
}
}
}

Exact match over decimal values

I want to perform an exact match over decimal values.
I have submitted two applications , for first application with annual salary as 99999868.10 and the other as 99999868.99.
When I do a query for 99999868 or I search 99999868.10 it returns me both the data , whereas I expect it to return only the exact match for it
The query I am executing is :
GET index/_search
{"query": {
"term": {
"Annual Salary": {
"value": "99999868"
}
}
}
}
Change mapping of salary field to numeric type and re index data
Numeric type reference : - https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html
use match_phrase and let me know. Actually, it will solve your problem.

Searching for multiple values in a String array in Elastic

I have a field that I am indexing into Elasticsearch that is an array of strings. So, for example, here is what the string array will look like in two records:
Record 1: {"str1", str2", str3", "str4", "str5"}
Record 2: {"str1", str2", str6", "str7", "str8"}
Question 1: I want to be able to query for multiple strings in this array. For e.g. my query has "str1", "str2". "str3" as the search parameter. I want to search for records where the string array has any of these three strings
Question 2: For the scenario above will Record 1 return with a higher score than record 2 (since all three strings are in the array for record 1 but only two are there in record 2).
Is this possible at all? Can you please help with what the query should look like and if the scoring works the way I stated.
You can index them as an array, such as:
{
"myArrayField": [ "str1", str2", str3", "str4", "str5" ],
...
}
You would then be able to query a number of ways, the simplest for your case being a match query (which is analyzed):
{
"match" : {
"myArrayField" : "str1 str2 str3"
}
}
Or a terms query (which is not analyzed):
{
"terms" : {
"myArrayField" : [ "str1", "str2", "str3" ]
}
}
And Yes, matches against more query terms will receive a higher score, so Record 1 would be scored higher than Record 2.

Sorting by value in multivalued field in elasticsearch

I have a multivalue field with integers in the document, for example
{
values: [1,2,3,4,5]
}
I apply range filter, for example from 2 to 4 and get list of document with values, contains 2,3,4.
Now I'd like to sort results, and first return documents, which contains 3.
I could do it using script sorting:
{
sort:{
_script: {
script: "doc['values'].getValues().contains(3) ? 0 : 1",
type: "number"
}
}
}
But I don't like it's performance, because getValues() returns a List actually, and contains methods is O(n).
Are any better ways?

Resources