This behavior of running script with elastic search is quite puzzling to me. I am running a script to filter by time. I have got these methods from this elastic documentation.
This script with HoursOfDay works (date.hourOfDay)
{
"script": {
"script": {
"lang": "expression",
"source": "doc['#timestamp'].date.hourOfDay >= min && doc['#timestamp'].date.hourOfDay <= max",
"params": {
"min": 5,
"max": 8
}
}
}
}
Whereas, the script with Minutes doesnt return any results. (date.minuteOfDay)
{
"script": {
"script": {
"lang": "expression",
"source": "doc['#timestamp'].date.minuteOfDay >= min && doc['#timestamp'].date.minuteOfDay <= max",
"params": {
"min": 300,
"max": 480
}
}
}
}
Also i have tried few other way from this elastic document by using "date.getHourOfDay()" and "date.getMinutes()", and found similar behavior. getHourOfDay returns results and getMinutes() doesnt.
Am I missing something here or is there a specific configuration at the elastic level to enable this method. Thanks.
I think in source you can try params.min and params.max instead min and max respectively
Related
Using Elasticsearch 7.*, we have a field 'ElapsedTime' under the mapping and I am trying to write a query to generate output of that field as 'ElapsedTime' / 1000.
Tried below but no luck:
1)
GET /_search
{
"script_fields": {
"test1": {
"script": {
"lang": "painless",
"source": "params._source.ElapsedTime / 1000"
}
}
}
}
GET /_search
{
"script_fields": {
"test2": {
"script": {
"lang": "expression",
"source": "doc['ElapsedTime'] / 1000"
}
}
}
}
Errors:
null pointer exception
parse_exception: Field [ElapsedTime] does not exist in mappings
You need to run GET concrete-index/_search on a concrete-index and not on / which runs on all indexes of your cluster, where the chance of hitting an index which doesn't have ElapsedTime in its mapping is quite big.
Wondering if anybody knows why this update by query runs fine but nothing gets deleted even though task runs to completion and claims all records are updated? I can use the same syntax to delete all person objects without any issues. Why does ES not delete object properties?
POST /16cf303e902f4445a560a8e9a5b9ea51/_update_by_query?conflicts=proceed&wait_for_completion=false
{
"conflicts": "proceed",
"query": {
"exists": {
"field": "person.hair_color"
}
},
"script": {
"lang": "painless",
"source": "ctx._source.remove('person.hair_color');"
}
}
Now if I watch the task using the ID returned from the above call
GET /_tasks/qLeuj8jqQgOPFGsEzL7u9Q:1776664
I get this (shorted version) claiming all documented were updated. However all person's still have their hair color for some reason.
{
"completed" : true,
"task" : {
"status" : {
"updated" : 110345,
}
}
Thanks for any guidance!
Figured this out finally! Apparently the _source contains all objects in your document. Therefore to delete an object property you need to run:
POST /16cf303e902f4445a560a8e9a5b9ea51/_update_by_query?conflicts=proceed&wait_for_completion=false
{
"conflicts": "proceed",
"query": {
"exists": {
"field": "person.hair_color"
}
},
"script": {
"lang": "painless",
"source": "ctx._source.person.remove('hair_color');"
}
}
This works if anybody has the same issue!
I'm trying to make a single query for updating the one field value in ES index.
I have a index pages which contain information about the pages (id, name, time, parent_page_id, child_count etc)
I can update the field parent_page_id with number of documents which have this page id as parent_page_id
I can update the field with default single value like:
PUT HOST_ADDRESS/pages/_update_by_query
{
"script": {
"source": "def child_count = 0; ctx._source.child_count = child_count;",
"lang": "painless"
},
"query": {
"match_all": {}
}
}
I'm trying with this code to get child count but its not working.
"source": "def child_count = 0; client.prepareSearch('pages').setQuery(QueryBuilders.termQuery("parent_page_id", "ctx._source.id")).get().getTotal().getDocs().getCount(); ctx._source.child_count = child_count;",
"lang": "painless"
My question is, how can i make a sub count-query in script to have a real child count in variable child_count
Scripting doesn't work like this — you cannot use java DSL in there. There's no concept of client or QueryBuilders etc in the Painless contexts.
As such, you'll need to obtain the counts before you proceed to update the doc(s) with a script.
Tip: scripts are reusable when you store them:
POST HOST_ADDRESS/_scripts/update_child_count
{
"script": {
"lang": "painless",
"source": "ctx._source.child_count = params.child_count"
}
}
and then apply via the id:
PUT HOST_ADDRESS/pages/_update_by_query
{
"script": {
"id": "update_child_count", <-- no need to write the Painless code again
"params": {
"child_count": 987
}
},
"query": {
"term": {
"parent_page_id": 123
}
}
}
I have a document which has a date field. I'd like to sort by documents by the this date ASC, but ones with a date in the past i'd like at the end.
In my end, it's like i want to assign the document value to a new value:
- If date is > "utc now", then assign value to whatever the date is
- If date is < "utc now", then assign value to max date
Then, i can sort by this field ASC.
So, it seems the only way to achieve this is with painless scripting.
This is what i've got so far, works.. but not sure if it's the correct approach.
GET /listings/_search
{
"track_total_hits": true,
"from": 0,
"query": {
"match_all": {}
},
"size": 48,
"sort": [
{
"_script" : {
"type": "string",
"script": {
"lang": "painless",
"source": "if (doc['auctionOn.utc'].size() == 0) { return params['maxTimestamp'].toString(); } else { long timestampDoc = doc['auctionOn.utc'].value.toInstant().toEpochMilli();long timestampNow = new Date().getTime();if (timestampDoc > timestampNow) { return timestampDoc.toString(); } else { return params['maxTimestamp'].toString(); } }",
"params": {
"maxTimestamp": 9223372036854776000
}
},
"order": "asc"
}
}
]
}
can someone please advise if this is the correct/performant approach?
With Elasticsearch and painless is there a way to implement an exponential function? I can't seem to find anything. I have something like this.
bdy = {
"from" : 0,
"size" : 10,
"query": {
"function_score": {
"query": {
"bool": {
"must": must_terms
}
},
"script_score": {
"script": {
"lang": "expression",
"source": "doc['release_year'].value"
}
}
}
}
}
I want to add some more complex math in the source field, like this.
"source": "Math.exponential(1/doc['release_year'].value)"
Is that possible? Or is there another scripting language that you can do that in within elasticsearch?
UPDATE
Actually looks like I can use.
"lang": "expression"
"source": "_score/10 + 1/(1+ exp(-(doc['release_year'].value*a)))"
http://lucene.apache.org/core/6_0_0/expressions/index.html?org/apache/lucene/expressions/js/package-summary.html
If anyone has other options that would be cool.
You can do it in Painless the same way with Math.exp()
"source": "_score/10 + 1/(1+ Math.exp(-(doc['release_year'].value*a)))"
See the full Painless API here: https://www.elastic.co/guide/en/elasticsearch/painless/current/painless-api-reference.html