I have been trying to use a must query with bool but I am failing to get the results.
In pseudo-SQL:
SELECT * FROM info WHERE (ulevel= '1.3.10' or ulevel= '1.3.6') AND (#timestamp between '2017-06-05T07:00:00.000Z' and '2017-06-05T07:00:00.000Z')
Here is what I have:
"query": {
"bool": {
"must": [
{
"query_string": {
"default_field": "_all",
"query": "*"
},
"range": {
"#timestamp": {
"from": "2017-06-05T07:00:00.000Z",
"to": "2017-06-05T07:20:00.000Z"
}
},
"bool": {
"should": [
{"term": { "ulevel": "1.3.10"}},
{"term": { "ulevel": "1.3.6"}}
]
}
}
]
}
}
Does anyone have a solution?
Thank you so much.
You can use terms query for the first part and the range query for the second part
GET _search
{
"query": {
"bool": {
"must": [
{
"terms": {
"ulevel": [
"1.3.10",
"1.3.6"
]
}
},
{
"range": {
"#timestamp": {
"gte": "2017-06-05T07:00:00.000Z",
"lte": "2017-06-05T07:20:00.000Z"
}
}
}
]
}
},
"from": 0,
"size": 20
}
Some Notes :
Filters documents that have fields that match any of the provided terms (not analyzed)
Also you can use some date spesific formulation with rage filter. Please check the range query page https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html#ranges-on-dates more information.
Update:
Added from and size for comment question.
Related
I have ES documents structured in a flat data structure using the nested data type, as they accept arbitrary JSON that we don't control, and we need to avoid a mapping explosion. Here's an example document:
{
"doc_flat":[
{
"key":"timestamp",
"type":"date",
"key_type":"timestamp.date",
"value_date":[
"2023-01-20T12:00:00Z"
]
},
{
"key":"status",
"type":"string",
"key_type":"status.string",
"value_string":[
"warning"
]
},
... more arbitrary fields ...
],
}
I've figured out how to query this nested data set to find matches on this arbitrary nested data, using a query such as:
{
"query": {
"nested": {
"path": "doc_flat",
"query": {
"bool": {
"must": [
{"term": {"doc_flat.key": "status"}},
{"term": {"doc_flat.value_string": "warning"}}
]
}
}
}
}
}
And I figured out how to find documents matching a particular date range:
{
"query": {
"nested": {
"path": "doc_flat",
"query": {
"bool": {
"must": [
{"term": {"doc_flat.key": "timestamp"}},
{
"range": {
"doc_flat.value_date": {
"gte": "2023-01-20T00:00:00Z",
"lte": "2023-01-21T00:00:00Z"
}
}
}
]
}
}
}
}
}
But I'm struggling to combine these two queries together, in order to search for documents that have a nested documents which match these two conditions:
a doc_flat.key of status, and a doc_flat.value_string of warning
a doc_flat.key of timestamp, and a doc_flat.value_date in a range
Obviously I can't just shove the second set of query filters into the same must array, because then no documents will match. I think I need to go "one level higher" in my query and wrap it in another bool query? But I can't get my head around how that would look.
You tried two nested inside Bool query?
{
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "doc_flat",
"query": {
"bool": {
"must": [
{
"term": {
"doc_flat.key": "timestamp"
}
},
{
"range": {
"doc_flat.value_date": {
"gte": "2023-01-20T00:00:00Z",
"lte": "2023-01-21T00:00:00Z"
}
}
}
]
}
}
}
}
],
"must": [
{
"nested": {
"path": "doc_flat",
"query": {
"bool": {
"must": [
{
"term": {
"doc_flat.key": "status"
}
},
{
"term": {
"doc_flat.value_string": "warning"
}
}
]
}
}
}
}
]
}
}
}
I have query that is similar to union operation in SQL. What i need is to specify the size of result set for each index. For example i want to get 10 records from first index and 15 records from second index.
My query:
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [{
"match_phrase_prefix": {"userName": "ar" }
}]
}
},
{
"bool": {
"must": [{
"match_phrase_prefix": { "groupName": "ar" }
}]
}
}
]
}
}
}
Url to send query:
http://website.com:9200/user_data,group_data/_search
If you have any thoughts i'd be very grateful.
Thank you
I think you can't do that with a simple query.
But can do that with the Top Hits aggregation, which lets you group result sets by certain fields via a bucket aggregator. Your case should look like:
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [{
"match_phrase_prefix": {"userName": "ar" }
}]
}
},
{
"bool": {
"must": [{
"match_phrase_prefix": { "groupName": "ar" }
}]
}
}
]
}
}, #Your query stills the same
"size": 0, #This will bring back nothing within the field "hits", so you can focus in the "aggregations" field.
"aggs": {
"10_usernames": {
"top_hits": {
"_source": {
"includes": [ "userName" ]
},
"size" : 10
}
},
"15_groupames": {
"top_hits": {
"_source": {
"includes": [ "groupName" ]
},
"size" : 15
}
}
}
}
You'll see your results within the "aggregations" field.
Hope this is helpful! :D
Ok, thanks for help
Eventually i've chosen another approach. I use Multi Search API, which allows you executing several requests at once. My query is:
POST http://website.com:9200/_msearch
{"index": "user_data"}
{"size":10,"query":{"bool":{"must":[{"match_phrase_prefix":{"userName":"##USER_TEXT##"}}]}}}
{"index": "group_data"}
{"size":15,"query":{"bool":{"must":[{"match_phrase_prefix":{"groupName":"##USER_TEXT##"}}]}}}
Below is the elastic search query. I need to use both the range and missing in a query.How can I change the below query
{
"query": {
"bool": {
"must": [
{
"constant_score": {
"filter": {
"missing": {
"field": "url"
}
}
}
}
],
"should": []
}
},
"_source": [
"id",
"com_name",
"website",
"_foundation._rating"
]
}
I need to add range to the above query. Kindly help me add the below section to the above query
"range": {
"_foundation._rating": {
"gte": 1,
"lte": 4
}
I suspect that the query you need is the following, i.e. the url field must be missing and the _foundation._rating field must be between and 1 and 4 (inclusive):
{
"query": {
"bool": {
"must": [
{
"missing": {
"field": "url"
}
},
{
"range": {
"_foundation._rating": {
"gte": 1,
"lte": 4
}
}
}
]
}
},
"_source": [
"id",
"com_name",
"url",
"_foundation._rating"
]
}
Based on the version of your elastic search, if you are using 5.x, you must use exists inside a must_not clause.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
Try the below query:
{
"query": {
"range": {
"bool": {
"must": [
{
"term": {
"_foundation._rating": {
"gte": 1,
"lte": 4
}
}
}
],
"must_not": {
"exists": {
"field": "url"
}
}
}
}
}
}
I'm losing my mind over how to achieve a filtered query that matches on having 'all' the provided terms, not 'any', basically 'and' vs 'or'.
Given I have (a very simplified version) a document:
{
"content": {
"tags": [
{
"tag": "Express",
"type": "other"
},
{
"tag": "Baking",
"type": "other"
}
]
}
}
Currently I have the following for a search:
{
"min_score": 0.5,
"query": {
"filtered": {
"query": {},
"filter": {
"bool": {
"must": [{
"nested": {
"path": "content.tags",
"filter": {
"terms": {
"content.tags.tag": ["Express", "Baking"],
}
}
}
}],
"should": []
}
}
}
},
"sort": ["_score"],
"from": 0,
"size": 12
}
I understand that this will look for any documents that have 'Express' and also add to that any documents that have 'Baking'. What I need is a filter that matches only documents that have both tags present.
I've read the docs, and my understanding of them was that I needed to add "execution": "and" into the filter as such
{
"min_score": 0.5,
"query": {
"filtered": {
"query": {},
"filter": {
"bool": {
"must": [{
"nested": {
"path": "content.tags",
"filter": {
"terms": {
"content.tags.tag": ["Express", "Baking"],
"execution": "and"
}
}
}
}],
"should": []
}
}
}
},
"sort": ["_score"],
"from": 0,
"size": 12
}
Sadly, this matches 0 documents, when I know for a fact there are 28 of them.
My content.tags.tag field is not analyzed, and content.tags has many nested objects as described above, so I understood it to mean that it must contain one of each tag, but that doesn't seem to be the case.
Anyone know how I can achieve this?
What you're asking for is a nested document that contains both terms "Express" and "Baking", which does not match the (parent) document you listed. It has a nested document that matches the first term, and another nested document that matches the second, but no nested documents that match both. And as #Frederick pointed out, your second query won't work in Es 2.0 anyway.
On the other hand, when I tried your query, I got no results anyway, because your empty query clause matches nothing. But when I removed it the document was returned, as was a document that matched either of the terms but not both. (if the "execution": "and" hadn't been ignored, it would have returned none of the documents).
Anyway, it sounds like this is what you are asking for: find a parent document containing nested documents that match each of the two terms:
POST /test_index/_search
{
"filter": {
"bool": {
"must": [
{
"nested": {
"path": "content.tags",
"filter": {
"term": {
"content.tags.tag": "Express"
}
}
}
},
{
"nested": {
"path": "content.tags",
"filter": {
"term": {
"content.tags.tag": "Baking"
}
}
}
}
]
}
}
}
Here is some code I used to test all this (in ES 2.1):
http://sense.qbox.io/gist/e375721509acbb5170e07eb969ea4409921f42d3
Use two different must condition,
{
"min_score": 0.5,
"query": {
"filtered": {
"query": {},
"filter": {
"bool": {
"must": [{
"nested": {
"path": "content.tags",
"filter": {
"terms": {
"content.tags.tag": "Banking"
}
}
}
},"nested": {
"path": "content.tags",
"filter": {
"terms": {
"content.tags.tag": "Express"
}
}
}
}],
"should": []
}
}
}
},
"sort": ["_score"],
"from": 0,
"size": 12
}
I am trying to figure out how to AND my Elastic Search query. I've tried a few different variations but I am always hitting a parser error.
What I have is a structure like this:
{
"title": "my title",
"details": [
{ "name": "one", "value": 100 },
{ "name": "two", "value": 21 }
]
}
I have defined details as a nested type in my mappings. What I'm trying to achieve is a query where it matches a part of the title and it matches various details by the detail's name and value.
I have the following query which gets me nearly there but I haven't been able to figure out how to AND the details. As an example I'd like to find anything that has:
detail of one with value less than or equal to 100
AND detail of two with value less than or equal to 25
The following query only allows me to search by one detail name/value:
"query" : {
"bool": {
"must": [
{ "match": {"title": {"query": titleQuery, "operator": "and" } } },
{
"nested": {
"path": "details",
"query": {
"bool": {
"must": [
{ "match": {"details.name" : "one"} },
{ "range": {"details.value" : { "lte": 100 } } }
]
}
}
} // nested
}
] // must
}
}
As a second question, would it be better to query the title and then move the nested part of the query into a filter?
You were so close! Just add another "nested" clause in your outer "must":
POST /test_index/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"title": {
"query": "title",
"operator": "and"
}
}
},
{
"nested": {
"path": "details",
"query": {
"bool": {
"must": [
{"match": {"details.name": "one" } },
{ "range": { "details.value": { "lte": 100 } } }
]
}
}
}
},
{
"nested": {
"path": "details",
"query": {
"bool": {
"must": [
{"match": {"details.name": "two" } },
{ "range": { "details.value": { "lte": 25 } } }
]
}
}
}
}
]
}
}
}
Here is some code I used to test it:
http://sense.qbox.io/gist/1fc30d49a810d22e85fa68d781114c2865a7c92e
EDIT: Oh, the answer to your second question is "yes", though if you're using 2.0 things have changed a little.