Given an index in Elasticsearch (opensearch) with a field, "etl_at", whose entries are unix timestamp integers, how to I find all records that are newer than yesterday?
"request_id" : "20230208201622189E3A8AFF810DB804F3",
"etl_at" : 1675888029122
I'm trying to use a CAST, but am met with error;
GET users/_search
{
"query": {
"range": {
"etl_at": {
"gt": "CAST('now-1d/d' as int)"
}
}
}
}
Related
I'm looking for a solution to be able to store and query dates before Epoch (1st January 1970).
For example, I'm trying to store into Elasticsearch's index the date December, 25 1969 00:00:00 +0100.
I suppose it possible to store into into string or integer, but is there any solution to keep the field type date into Elasticsearch's mapping ?
You can definitely store dates before the epoch, simply by storing the date string or negative numbers.
If your index mapping looks like this:
PUT epoch
{
"mappings": {
"properties": {
"my_date": {
"type": "date"
}
}
}
}
Then you can index documents before the epoch, like this:
PUT epoch/_doc/1
{
"my_date": -608400
}
PUT epoch/_doc/2
{
"my_date": "1969-12-25T00:00:00"
}
When searching for document with a date before the epoch, both would be returned:
POST epoch/_search
{
"query": {
"range": {
"before": {
"lt": "1970-01-01"
}
}
}
}
Can I make a range query on default timestamp field ignoring date values i.e. using only time in timestamp - say 2 hours of each day?
My intentions are to search for all the documents but exclude the documents indexed between 9 PM and 12 AM (I have seen example with date ranges in filtering).
timestamp example stands following:
"#timestamp": [
"2015-12-21T15:18:17.120Z"
]
Elasticsearch version: 1.5.2
My first idea would be to use the date math in Elasticsearch query, e.g. if you run your query at 1PM, this would work:
{
"query": {
"range" : {
"#timestamp" : {
"gte": "now-16h/h",
"lte": "now-1h/h"
}
}
}
}
(watch out for the timezone though).
As far as I know, the only other possibility would be to use scripting.
Please note also that you are running a very old version of Elasticsearch.
Edit If you need simply absolute date, then check how your #timestamp field look, and use the same format, for instance on my Elasticsearch, it would be:
{
"query": {
"range" : {
"#timestamp" : {
"gte": "2015-03-20T01:21:00.01Z",
"lte": "2015-03-21T01:12:00.04Z"
}
}
}
}
Hi i am kinda new to elasticsearch. I need to get an aggregation with date comparison and a dynamic range filter.
Like i need to get documents count where created_at document is 1 week earlier than their identification_date.
So i tried something like this but my date param seems unused, actually changing it never changes my results.
"aggs": {
"identified": {
"terms": {
"script": "doc['created_at'].value > (doc['identification_date'].value - diff_date)
&& doc['created_at'].value < doc['identification_date'].value",
"params": {
"diff_date": 604800
}
}
}
}
Thank you for taking time helping.
since on the abstracted level you just want count of documents created 7 days before on some date field.you don't need aggregation for this if you don't want to group the results sets further on some fields.Simply you can a)use range filter query on your date field
{
"range" : {
"date" : {
"gte" : "now-7d/d",
"lt" : "now"
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html
I'm working with ES and I need a query that returns the difference between two datetime (mysql timediff), but have not found any function of ES to do that. Someone who can help me?
MySQL Query
SELECT SEC_TO_TIME(
AVG(
TIME_TO_SEC(
TIMEDIFF(r.acctstoptime,r.acctstarttime)
)
)
) as average_access
FROM radacct
Thanks!
Your best best is scripted fields. The above search query should work , provided you have enabled dynamic scripting and these date fields are defined as date in the mapping.
{
"script_fields": {
"test1": {
"script": "doc['acctstoptime'].value - doc['acctstarttime'].value"
}
}
}
Note that you would be getting result in epoch , which you need to convert to your denomination.
You can read about scripted field here and some of its examples here.
Here is another example using script fields. It converts dates to milli seconds since epoch, subtracts the two and converts the results into number of days between the two dates.
{
"query": {
"bool": {
"must": [
{
"exists": {
"field": "priorTransactionDate"
}
},
{
"script": {
"script": "(doc['transactionDate'].date.millis - doc['priorTransactionDate'].date.millis)/1000/86400 < 365"
}
}
]
}
}
}
I have a query that properly parses date ranges. However, my database has a default value that all dates have a timestamp of 00:00:00. This means that items that are still valid today are shown as expired even if they should still be valid. How can I adjust the following to look at just the date and not the time of the item (expirationDate).
{
"range": {
"expirationDate": {
"gte": "now"
}
}
}
An example of the data is:
"expirationDate": "2014-06-24T00:00:00.000Z",
Did you look into the different format options for dates stored in ElasticSearch? If this does not work for you or you don't want to store dates without the time you can try this query, which will work for your exact use case I guess:
{
"range": {
"expirationDate": {
"gt": "now-1d"
}
}
}
You can also round down the time so that your query returns anything that occurred since the beginning of the day:
Assuming that
now is 2017-03-07T07:00:00.000,
now/d is 2017-03-07T00:00:00.000
Your query would be:
{
"range": {
"expirationDate": {
"gte": "now/d"
}
}
}
elastic search documentation on rounding times