I have external ES instance which I need to query for documents older than 6 months. Problem is they store timestamp like that:
"timestamp": {
"year": 2018,
"monthValue": 5,
"dayValue": 1,
}
Is it possible to create a range query combining these fields and getting documents "lt" "now-6m" or something like that?
You should be able to accomplish this using a Script Query. That would enable you to create a date object using the field values, and then compare that date with the current date.
Notional example
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"params": {
"monthRange": 6
},
"source": """
def today = new Date();
def timestamp = new Date(doc['timestamp']['year'].value, doc['timestamp']['monthValue'].value, doc['timestamp']['dayValue'].value);
/* Date comparison magic (I don't know Java, so you're on your own here) */
/* return result of comparison */
""",
"lang": "painless"
}
}
}
}
}
}
I've only used Painless once before, so I'm not familiar enough to give a perfect answer. But this may help you get started. If you get stuck, just ask another question specific to the issue you're having, and someone who's more familiar with Java/Painless can help you out.
Related
I am trying to use a script to set several values of my Elastic document.
POST myindex/_update_by_query
{
"script" : {
"source": """
ctx._source.categories='categories';
ctx._source.myObject={};
""",
"lang": "painless"
},
"query": {
"term" : {
"name": "Tony"
}
}
}
But I can't set an object value with this painless language. However I write it, I get an error.
Is there a way to do this, maybe with a different script language ?
Thanks!
In order to create an object (i.e. a hash map), you should do it this way:
ctx._source.myObject = [:];
I'm trying to create a query to filter my documents by one (can be anyone) value from a field (in my case "host.name"). The point is that I don't know previously the unique values of this field. I need found these and choose one to be used in the query.
I had tried the below query using a painless script, but I have not been able to achieve the goal.
{
"sort" : [{"#timestamp": "desc"}, {"host.name": "asc"}],
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": """
String k = doc['host.name'][0];
return doc['host.name'].value == k;
""",
"lang": "painless"
}
}
}
}
}
I'll appreciate if any can help me improving this idea of suggesting me a new one.
TL;DR you can't.
The script query context operates on one document at a time and so you won't have access to the other docs' field values. You can either use a scripted_metric aggregation which does allow iterating through all docs but it's just that -- an aggregation -- and not a query.
I'd suggest to first run a simple terms agg to figure out what values you're working with and then build your queries accordingly.
Let's see if someone could shed a light on this one, which seems to be a little hard.
We need to correlate data from multiple index and various fields. We are trying painless script.
Example:
We make a search in an index to gather data about the queueid of mails sent by someone#domain
Once we have the queueids, we need to store the queueids in an array an iterate over it to make new searchs to gather data like email receivers, spam checks, postfix results and so on.
Problem: Hos can we store the data from one search and use it later in the second search?
We are testing something like:
GET here_an_index/_search
{
"query": {
"bool" : {
"must": [
{
"range": {
"#timestamp": {
"gte": "now-15m",
"lte": "now"
}
}
}
],
"filter" : {
"script" : {
"script" : {
"source" : "doc['postfix_from'].value == params.from; qu = doc['postfix_queueid'].value; return qu",
"params" : {
"from" : "someona#mdomain"
}
}
}
}
}
}
}
And, of course, it throws an error.
"doc['postfix_from'].value ...",
"^---- HERE"
So, in a nuttshell: is there any way ti execute a search looking for some field value based on a filter (like from:someone#dfomain) and use this values on later searchs?
We have evaluated using script fields or nested, but due to some architecture reasons and what those changes would entail, right now, can not be used.
Thank you very much!
I created parent-child in ElasticSearch.
I use has_parent for query data, but I want to sort this data follow some parent field which is a String.
I tried to find solutions, I found how to sort in ElasticSearch documents
"query": {
"has_parent" : {
"parent_type" : "blog",
"score" : true,
"query" : {
"function_score" : {
"script_score": {
"script": "_score * doc['view_count'].value"
}
}
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/query-dsl-has-parent-query.html#_sorting_2
It use "score" * doc['view_count'] for searching which is an integer, but I want to do some calculations with String for searching it (with Groovy), but I don't know what should I do.
If you have another idea, please tell me.
thanks for helping me.
Hi i am kinda new to elasticsearch. I need to get an aggregation with date comparison and a dynamic range filter.
Like i need to get documents count where created_at document is 1 week earlier than their identification_date.
So i tried something like this but my date param seems unused, actually changing it never changes my results.
"aggs": {
"identified": {
"terms": {
"script": "doc['created_at'].value > (doc['identification_date'].value - diff_date)
&& doc['created_at'].value < doc['identification_date'].value",
"params": {
"diff_date": 604800
}
}
}
}
Thank you for taking time helping.
since on the abstracted level you just want count of documents created 7 days before on some date field.you don't need aggregation for this if you don't want to group the results sets further on some fields.Simply you can a)use range filter query on your date field
{
"range" : {
"date" : {
"gte" : "now-7d/d",
"lt" : "now"
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html