I'm trying to get the difference between two fields. I'm using Elasticsearch 5.5.
I have already tried
{
"query": {
"bool": {
"filter": {
"script": {
"script": "doc['students.total_fee'].value - doc['students.paid_fee'].value"
}
}
}
}
}
but it is returning empty "hits".
I have also tried
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "students",
"query": {
"script": {
"script": {
"inline": "doc['students.total_fee'].value - doc['students.paid_fee'].value",
"lang": "expression"
}
}
}
}
}
]
}
}
and it is returning "0".
also the "script_fields" did not worked.
{ "script_fields" :
{ "difference" :
{ "script" : "doc['students.total_fee'].value - doc['students.paid_fee'].value"
} } }
suppose I have data in following format.
"_source" : {
"students" : [
{
"name" : "A",
"total_fee" : 12345,
"paid_fee" : 12344.8
},
{
"name" : "B",
"total_fee" : 23456,
"paid_fee" : 23455.6
}
]
}
Now I want to get the difference between "total_fee" and "paid_fee" for each student.
I expect to get an array of differences for all students.
Thanks in advance :D
You need to use the below query and my es version is 6.2.2. It is giving a perfect result. But remember scripting is generally CPU intensive.
If you normal field then below query working fine.
{
"size": 10,
"script_fields": {
"fare_diff": {
"script": "doc[\"students.total_fee\"].value - doc[\"students.paid_fee\"].value"
}
}
}
If you are using a nested field parameter then the query would be like below.
{
"script_fields": {
"fare_diff": {
"script": {
"lang": "painless",
"source": "int total = 0; def l = new ArrayList(); for (int i = 0; i < params['_source']['students'].size(); ++i) { l.add(params['_source']['students'][i]['total_fee'] - params['_source']['students'][i]['paid_fee']);} return l.toArray();"
}
}
}
}
Reason: Because nested documents are indexed as separate documents, they can only be accessed within the scope of the nested query, the nested/reverse_nested aggregations, or nested inner hits.
Related
I have some data in an elasticsearch index that looks like this:
{
"field1" : {
"en-US" : {
"value1" : "example",
"value2" : "example2"
}
},
"field2" : "x"
}
I need to update the value of the nested fields by query and my update script looks like this:
{
"query": {
"bool": {
"must": [
{
"terms": {
"field2": [
"x"
]
}
}
]
}
},
"script": {
"lang": "painless",
"params": {
"value": example
},
"source": "ctx._source.field1.en-US.value1 = params.value"
}
}
This gives an error as it takes the '-' b/w en-US as the subtraction operation; I can update the whole object by making this change to the script:
"script": {
"lang": "painless",
"params": {
"value": {
"value1" : "example3"
}
},
"source": "ctx._source.field1['en-US'] = params.value"
}
This works but I have to update the whole object but not one particular key inside the object. Is there a way to achieve this?
I have a simple search using scripts:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "fieldX"
}
},
{
"exists": {
"field": "fieldY"
}
}
]
}
},
"aggs": {
"groupBy": {
"terms": {
"script": {
"source": "doc['fieldX.keyword'].value+','+doc['fieldY.keyword'].value"
}
}
}
}
}
Groups are the result of this search:
"buckets" : [
{
"key" : "valueX1,valueY1",
"doc_count" : 165
},
{
"key" : "valueX2,valueY2",
"doc_count" : 45
}
]
The main problem here is: If all the fields (fieldX and fieldY) in documents exist - everything will be fine. If at least one field is missing, the search will return nothing.
I tried to rewrite this to normal search using aggs, terms, field but unsuccessfully.
Any idea how I can rewrite this search to keep the original result but avoid the described problem?
You need to update your script and add some null pointer check as shown below:
{
"aggs": {
"groupBy": {
"terms": {
"script": {
"source": """
if(doc['fieldX.keyword'].size()!=0 &&doc['fieldY.keyword'].size()!=0){
doc['fieldX.keyword'].value+','+doc['fieldY.keyword'].value
}else if(doc['fieldX.keyword'].size()==0 && doc['fieldY.keyword'].size()!=0){
doc['fieldY.keyword'].value
}else if(doc['fieldY.keyword'].size()==0 && doc['fieldX.keyword'].size()!=0){
doc['fieldX.keyword'].value
}
"""
}
}
}
}
}
You can remove else if part if you want to generate aggregation only when both fields value is available. Also, exists query you can remove after updating script.
I want to search elasticsearch index based on length of text in one of the field. I tried various answers on SO like elasticsearch filter by length of a string field but it didn't work. Below query returns me document with longer value like %230f72e5. What is wrong with my query?
my mapping:
{
"myindex": {
"mappings": {
"properties": {
"colorcode": {
"type": "keyword"
}
my query GET myindex/_search
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : "doc['colorcode'].size() < 5"
}
}
}
}
}
For calculating the length of the string you need to use the below script
For version 7.*
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['colorcode.keyword'].value.length() < 5",
"lang": "painless"
}
}
}
}
}
}
That query is wrong!
Error: A document doesn't have a value for a field! Use doc[].size()==0 to check if a document is missing a field!
The correct query is:
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : "doc['colorcode'].size() > 0 && doc['colorcode'].value.length() < 5"
}
}
}
}
}
I need to get documents in an elasticseach index which have empty array for a field (which can be nested)?
Doing it the below way is not working
{
"query": {
"filter": {
"script": {
"script": "doc['arrayField'].length > 0"
}
}
}
}
or
{
"query": {
"bool": {
"must": { "match_all": {} },
"filter": { "term": { "arrayField": [] }}
}
}
}
It seems like I have to use the filter feature, alongwith some script but can't figure out how.
To get documents with the empty array we find the length of the field and see if it's length is less than 1. Please, the working query below.
"query": {
"bool" : {
"must" : {
"script" : {
"script" : {
"inline": "doc['arrayField'].length < 1",
"lang": "painless"
}
}
}
}
}
Let me know if you find any issues.
Thanks
I'm comparing 2 numeric fields, but it looks like it's not working.
The results caintain docs not filtered on the condition.
Did I miss something ?
GET crawl-panda-18-06-2018-2-2018/_search
{
"filtered": {
"filter": {
"script": {
"script": "doc[csv_hit].value > doc[csv_googlebot-desktop].value"
}
}
}
}
Mapping:
{
"crawl-panda-18-06-2018-2-2018": {
"aliases": {},
"mappings": {
"items": {
"properties": {
(...)
"csv_googlebot-desktop": {
"type": "long"
},
"csv_hit": {
"type": "long"
}
(...)
Results sample:
"csv_googlebot-desktop": 1,
"csv_hit": 0
1st error founds by #Val: Elasticsearch/Kibana don't allow line between GET and the query.
2nd error is that script filter has been replaced by Script Query:
"The script filter has been replaced by the Script Query. It behaves as a query in “query context” and as a filter in “filter context” (see Query DSL)."
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/query-dsl-script-filter.html#query-dsl-script-filter
Works with the following code:
GET index-name/_search
{
"query": {
"bool" : {
"must" : {
"script" : {
"script" : {
"source": "doc['field_a'].value > doc['field_b'].value",
"lang": "painless"
}
}
}
}
}
}