I have a field with the following mapping:
birthdate: { type: :date, format: :dateOptionalTime }
I need to find everyone who were born in month of May (including all years)
Another query is to find all people who were born on 'August 25' (including all years)
What would be the query for that?
You can achieve this with a script filter
All people born in May of any year:
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "doc.birthdate.date.monthOfYear == 5"
}
}
}
}
}
All people born on August 25th (any year)
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "doc.birthdate.date.monthOfYear == 8 && doc.birthdate.date.dayOfMonth == 25"
}
}
}
}
}
Just for the archives, ES 7 brought some braking changes https://www.elastic.co/guide/en/elasticsearch//reference/current/breaking-changes-7.0.html#_getdate_and_getdates_removed
It is now:
POST /people/_search?size=50
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"source": "doc['jour'].value.getMonthValue() == 5",
"lang": "painless"
}
}
}
}
}
}
GET index_name/doc_type/_search
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"source": "doc.field_name.date.getMonthOfYear() == month_number",
"lang": "painless"
}
}
}
}
}
}
First question:
POST /test_index/_search
{
"query": {
"filtered": {
"filter": {
"range": {
"birthdate": {
"gte": "2015-05-01",
"lt": "2015-06-01"
}
}
}
}
}
}
Second question:
POST /test_index/_search
{
"query": {
"filtered": {
"filter": {
"range": {
"birthdate": {
"gte": "2015-08-25",
"lte": "2015-08-25"
}
}
}
}
}
}
Here's the code I used to test it:
http://sense.qbox.io/gist/36c800cabbe4143ecf72144d02e58e267c1e761a
Related
I have documents in the following form
PUT test_index/_doc/1
{
"dates" : [
"2018-07-15T14:12:12",
"2018-09-15T14:12:12",
"2018-11-15T14:12:12",
"2019-01-15T14:12:12",
"2019-03-15T14:12:12",
"2019-04-15T14:12:12",
"2019-05-15T14:12:12"],
"message" : "hello world"
}
How do I query for documents such that there are n number of dates within the dates array falling in between two specified dates?
For example: Find all documents with 3 dates in the dates array falling in between "2018-05-15T14:12:12" and "2018-12-15T14:12:12" -- this should return the above document as "2018-07-15T14:12:12", "2018-09-15T14:12:12" and "2018-11-15T14:12:12" fall between "2018-05-15T14:12:12" and "2018-12-15T14:12:12".
I recently faced the same problem. However came up with two solutions.
1) If you do not want to change your current mapping, you could query for the documents using query_string. Also note you will have to create the query object according to the range that you have. ("\"2019-04-08\" OR \"2019-04-09\" OR \"2019-04-10\" ")
{
"query": {
"query_string": {
"default_field": "dates",
"query": "\"2019-04-08\" OR \"2019-04-09\" OR \"2019-04-10\" "
}
}
}
However,this type of a query only makes sense if the range is short.
2) So the second way is the nested method. But you will have to change your current mapping in such a way.
{
"properties": {
"dates": {
"type": "nested",
"properties": {
"key": {
"type": "date",
"format": "YYYY-MM-dd"
}
}
}
}
}
So your query will look something like this :-
{
"query": {
"nested": {
"path": "dates",
"query": {
"bool": {
"must": [
{
"range": {
"dates.key": {
"gte": "2018-04-01",
"lte": "2018-12-31"
}
}
}
]
}
}
}
}
}
You can create dates as a nested document and use bucket selector aggregation.
{
"empId":1,
"dates":[
{
"Days":"2019-01-01"
},
{
"Days":"2019-01-02"
}
]
}
Mapping:
"mappings" : {
"properties" : {
"empId" : {
"type" : "keyword"
},
"dates" : {
"type" : "nested",
"properties" : {
"Days" : {
"type" : "date"
}
}
}
}
}
GET profile/_search
{
"query": {
"bool": {
"filter": {
"nested": {
"path": "dates",
"query": {
"range": {
"dates.Days": {
"format": "yyyy-MM-dd",
"gte": "2019-05-01",
"lte": "2019-05-30"
}
}
}
}
}
}
},
"aggs": {
"terms_parent_id": {
"terms": {
"field": "empId"
},
"aggs": {
"availabilities": {
"nested": {
"path": "dates"
},
"aggs": {
"avail": {
"range": {
"field": "dates.Days",
"ranges": [
{
"from": "2019-05-01",
"to": "2019-05-30"
}
]
},
"aggs": {
"count_Total": {
"value_count": {
"field": "dates.Days"
}
}
}
},
"max_hourly_inner": {
"max_bucket": {
"buckets_path": "avail>count_Total"
}
}
}
},
"bucket_selector_page_id_term_count": {
"bucket_selector": {
"buckets_path": {
"children_count": "availabilities>max_hourly_inner"
},
"script": "params.children_count>=19;" ---> give the number of days that should match
}
},
"hits": {
"top_hits": {
"size": 10
}
}
}
}
}
}
I found my own answer to this, although I'm not sure how efficient it is compared to the other answers:
GET test_index/_search
{
"query":{
"bool" : {
"filter" : {
"script" : {
"script" : {"source":"""
int count = 0;
for (int i=0; i<doc['dates'].length; ++i) {
if (params.first_date < doc['dates'][i].toInstant().toEpochMilli() && doc['dates'][i].toInstant().toEpochMilli() < params.second_date) {
count += 1;
}
}
if (count >= 2) {
return true
} else {
return false
}
""",
"lang":"painless",
"params": {
"first_date": 1554818400000,
"second_date": 1583020800000
}
}
}
}
}
}
}
where the parameters are the two dates in epoch time. I've chosen 2 matches here, but obviously you can generalise to any number.
I have built two queries in ElasticSearch to get the counts for each error message. for example, the first query is to get how many error messages related to "was not found" error
GET /logstash*/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"match": {
"kubernetes.pod_name": "api"
}
},
{
"match": {
"log": "error"
}
},
{
"match": {
"log": {
"query": "was not found",
"operator": "and"
}
}
},
{
"range": {"#timestamp": {
"time_zone": "CET",
"gt": "now-7d",
"lte": "now"}}
}
]
}
}
}
},
"aggs" : {
"type_count" : {
"value_count" : {
"script" : {
"source" : "doc['log.keyword'].value"
}
}
}
}
}
The second query is to get the count of error messages related to "Duplicate Entry" error
GET /logstash*/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"match": {
"kubernetes.pod_name": "api"
}
},
{
"match": {
"log": "error"
}
},
{
"match": {
"log": {
"query": "Duplicate entry",
"operator": "and"
}
}
},
{
"range": {"#timestamp": {
"time_zone": "CET",
"gt": "now-7d",
"lte": "now"}}
}
]
}
}
}
},
"aggs" : {
"type_count" : {
"value_count" : {
"script" : {
"source" : "doc['log.keyword'].value"
}
}
}
}
}
My boss really wants me to combine these individual query into a one big query, then get the list of counts for each error messages in one output. Since we have a lot of error messages, which means we have to write each query for each error message, then we have to run each query to get the counts. Is there a way I can click one run to get the list of counts?
I have been trying use query string query and looking for solutions on either Stack Overflow and Documentation. However, there is no luck
You can use filter aggregation along with the value_count aggregation to combine these two queries. In both the queries, out of the 4 queries inside must clause only one differs. You can take this out and combine them with the two filter aggregations as below:
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"match": {
"kubernetes.pod_name": "api"
}
},
{
"match": {
"log": "error"
}
},
{
"range": {
"#timestamp": {
"time_zone": "CET",
"gt": "now-7d",
"lte": "now"
}
}
}
]
}
}
}
},
"aggs": {
"not_found_count": {
"filter": {
"match": {
"log": {
"query": "was not found",
"operator": "and"
}
}
},
"aggs": {
"count": {
"value_count": {
"script": {
"source": "doc['log.keyword'].value"
}
}
}
}
},
"duplicate_entry_count": {
"filter": {
"match": {
"log": {
"query": "Duplicate entry",
"operator": "and"
}
}
},
"aggs": {
"count": {
"value_count": {
"script": {
"source": "doc['log.keyword'].value"
}
}
}
}
}
}
}
Hi I want to join two queries in one in elasticsearch, but I don't know how to do it: I think I should do an aggregation but I don't know very clear how to do it. Could you help me? My ES version is 5.1.2.
First filter by status and name:
POST test_lite/_search
{
"aggs": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"match": {
"STATUS": "Now"
}
},
{
"match": {
"NAME": "PRUDENTL"
}
}
]
}
}
}
}
}
Look for in the filtered records for the word filtered in description:
POST /test_lite/_search
{
"query": {
"wildcard" : { "DESCRIPTION" : "*english*" }
}
}
The only query needed is:
POST test_lite/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"STATUS": "Now"
}
},
{
"match": {
"NAME": "PRUDENTL"
}
},
{"wildcard" : { "DESCRIPTION" : "*english*" }}
]
}
}
}
I have two assembled queries that work as expected.
First one uses constant score, while matching range between two values:
GET /_search
{
"query" : {
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 101000349
}
}
}
}
}
}
The second one searches for bool.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "Barcelona"
}
}]
}
}
}
}
}
Now I need to merge them and I am struggling how, because tried many combinations of putting in different scopes, but not successful.
So this query returns an error.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "sÃdlisko"
}
}]
}
}
},
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 1000010349
}
}
}
}
}
}
Error:
... failed to parse search source. expected field name but got
[START_OBJECT]
You could just put constant score query inside bool must clause
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"match": {
"name": "sidlisko"
}
},
{
"constant_score": {
"filter": {
"range": {
"locationId": {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
]
}
}
}
}
}
I've managed to establish this query and it appears to work.
This looks as the most optimised one.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"fullAddress": "sidlisko"
}
}]
}
},
"filter" : {
"range" : {
"locationId" : {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
}
i have a question with filter, range and or with elastic search.
I have de document in elasticSearch with
{
startDate : myDate
endDate : onOtherDateOr Nothing
}
I Want to search for a range where now is after startDate, or Between startDate and endDate if endDate is defined. How can i do that ?
You can do it like this with two bool/should filters containing two nested bool/must filters:
curl -XPOST localhost:9200/_search -d '{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"bool": {
"must": [ <--- if endDate exists, check both bounds
{
"exists": {
"field": "endDate"
}
},
{
"range": {
"startDate": {
"lte": "now"
}
}
},
{
"range": {
"endDate": {
"gte": "now"
}
}
}
]
}
},
{
"bool": {
"must": [ <--- if endDate missing, only check startDate
{
"missing": {
"field": "endDate"
}
},
{
"range": {
"startDate": {
"lte": "now"
}
}
}
]
}
}
]
}
}
}
}
}'
I find below query as best approach:
{
"query": {
"range": {
"order.dateCreated": {
"dateFrom": "2011-01-24",
"dateTo": "2011-01-24"
}
}
}
}
try the next query, at least it works for me.`
"query": {
"range": {
"date_ field ": { # this field_name should be the field where your date range
resides.
"gte": startDate,
"lte": endDate,
"boost": 2.0
}
}
}`