I have indexed JSON like below format
JSON:
{"work":[{"organization":"abc", end:"present"},{"organization":"edf", end:"old"}]}
{"work":[{"organization":"edf", end:"present"},{"organization":"abc", end:"old"}]}
I want to query records where organization is "abc" and end is "present"
but below query is not working
work.0.organization: "abc" AND work.0.end:"present"
No records are matched
if I give query like below
work.organization: "abc" AND work.end:"present"
Both the records are matched. Whereas only the first record is what I want
The matched record should be only the below
{"work":[{"organization":"abc", end:"present"},{"organization":"edf", end:"old"}]}
You have to use nested_types. First map work as nested type in elastic using following mappings
PUT index_name_3
{
"mappings": {
"document_type" : {
"properties": {
"work" : {
"type": "nested",
"properties": {
"organization" : {
"type" : "text"
},
"end" : {
"type" : "text"
}
}
}
}
}
}
}
Use the following query to do nested filter match and innerhits
{
"query": {
"nested": {
"path": "work",
"inner_hits": {},
"query": {
"bool": {
"must": [{
"term": {
"work.organization": {
"value": "abc"
}
}
},
{
"term": {
"work.end": {
"value": "present"
}
}
}
]
}
}
}
}
}
Related
So I got this document with a
nested array that I want to filter with this query.
I want ES to return all documents where all items have changes = 0 and that only.
If document has even a single item in the list with a change = 1, that's discarded.
Is there any way I can achieve this starting from the query I have already wrote? Or should I use a script instead?
DOCUMENTS:
{
"id": "abc",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 1
}
]
}
},
{
"id": "def",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 0
}
]
}
}
QUERY:
GET trips_solutions/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"id": {
"value": "abc"
}
}
},
{
"nested": {
"path": "trips",
"query": {
"range": {
"trips.changes": {
"gt": -1,
"lt": 1
}
}
}
}
}
]
}
}
}
EXPECTED RESULT:
{
"id": "def",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 0
}
]
}
}
Elasticsearch version: 7.6.2
Already read this answers but they didn't help me:
https://discuss.elastic.co/t/how-to-match-all-item-in-nested-array/163873
ElasticSearch: How to query exact nested array
First off, if you filter by id: abc, you obviously won't be able to get id: def back.
Second, due to the nature of nested fields which are treated as separate subdocuments, you cannot query for all trips that have the changes equal to 0 -- the connection between the individual trips is lost and they "don't know about each other".
What you can do is return only the trips that matched your nested query using inner_hits:
GET trips_solutions/_search
{
"_source": "false",
"query": {
"bool": {
"must": [
{
"nested": {
"inner_hits": {},
"path": "trips",
"query": {
"term": {
"trips.changes": {
"value": 0
}
}
}
}
}
]
}
}
}
The easiest solution then is to dynamically save this nested info on a parent object like discussed here and using range/term query on the resulting array.
EDIT:
Here's how you do it using copy_to onto the doc's top level:
PUT trips_solutions
{
"mappings": {
"properties": {
"trips_changes": {
"type": "integer"
},
"trips": {
"type": "nested",
"properties": {
"changes": {
"type": "integer",
"copy_to": "trips_changes"
}
}
}
}
}
}
trips_changes will be an array of numbers -- I presume they're integers but more types are available.
Then syncing a few docs:
POST trips_solutions/_doc
{"trips":[{"type":"home","changes":0},{"type":"home","changes":1}]}
POST trips_solutions/_doc
{"trips":[{"type":"home","changes":0},{"type":"home","changes":0}]}
And finally querying:
GET trips_solutions/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "trips",
"query": {
"term": {
"trips.changes": {
"value": 0
}
}
}
}
},
{
"script": {
"script": {
"source": "doc.trips_changes.stream().filter(val -> val != 0).count() == 0"
}
}
}
]
}
}
}
Note that we first filter normally using the nested term query to narrow down our search context (scripts are slow so this is useful). We then check if there are any non-zero changes in the accumulated top-level changes and reject those that apply.
Our data structure looks as follows
{
"foo": "bar",
"baz": [
{
"qux": true
},
{
"qux": null
},
{
"qux": false
}
]
}
In other documents, the number of items in the baz array might vary.
We're looking to find documents that have at least one null value for qux in the baz array.
We have tried:
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "baz.qux"
}
}
}
}
}
However, this will not return documents that have e.g. one true and one null value for qux in the baz array.
Any help would be greatly appreciated!
You can simply check whether the field having null or not using the missing query.
{
"query" : {
"constant_score" : {
"filter": {
"missing" : { "field" : "baz.qux" }
}
}
}
}
You can check the elastic document as well on below link:
Documentation: Is null Query
You can achieve this with a nested mapping and nested query.
Mapping:
{
"properties": {
"foo": {
"type": "text"
},
"baz": {
"type": "nested",
"properties": {
"qux": {
"type": "boolean"
}
}
}
}
}
Query:
{
"query": {
"nested": {
"path": "baz",
"query": {
"bool": {
"must_not": {
"exists": {
"field": "baz.qux"
}
}
}
}
}
}
}
I would like to perform an AND operation in ElasticSearch using the URI Search (q=). How do I do it?
If I have document like:
[{ "name":"Test 1", "pub":"2"}, { "name":"Test 2", "pub":"1"}, { "name":"A", "pub":"1"}]
And I would like to query for documents containing with a name containing "Test" AND where pub equals "1". How do I do that?
Thanks!
Assuming your document looks like this:
{
"my_field": [
{ "name":"Test 1", "pub":"2"},
{ "name":"Test 2", "pub":"1"},
{ "name":"A", "pub":"1"}
]
}
And the mapping of my_field is of type nested similar to this:
{
"mappings": {
"doc_type": {
"properties": {
"my_field": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"pub": {"type": "integer" }
}
}
}
}
}
}
Then you can query your index and get the expected documents with the following nested query:
POST /_search
{
"query": {
"nested": {
"path": "my_field",
"query": {
"bool": {
"filter": [
{
"match": {
"name": "Test"
}
},
{
"match": {
"pub": 1
}
}
]
}
}
}
}
}
Actually you'd need nested fields. The following is a good resource.
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html
I am trying to use Elastic Search and I am stuck trying to query for the nested object.
Basically my object is of the following format
{
"name" : "Some Name",
"field2": [
{
"prop1": "val1",
"prop2": "val2"
},
{
"prop1": "val3",
"prop2":: "val4"
}
]
}
Mapping I used for the nested field is the following.
PUT /someval/posts/_mapping
{
"posts": {
"properties": {
"field2": {
"type": "nested"
}
}
}
}
Say now i insert elements for /field/posts/1 and /field/posts/2 etc. I have k values for field2.prop1 and i want a query which gets the posts sorted based on most match of field2.prop1 among the K values i have. What would be the appropriate query for that.
Also I tried a simple filter but even that doesnt seem to work right.
GET /someval/posts/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
}
},
"filter" : {
"nested" : {
"path" : "field2",
"filter" : {
"bool" : {
"must" : [
{
"term" : {"field2.prop1" : "val1"}
}
]
}
},
"_cache" : true
}
}
}
}
The above query should match atleast the first post. But it returns no match. Can anyone help to clarify whats wrong here ?
There was problem in your json structure, you used filtered query , but filter(object) was in different level than query.
Find the difference.
POST /someval/posts/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "field2",
"filter": {
"bool": {
"must": [
{
"term": {
"field2.prop1": "val1"
}
}
]
}
},
"_cache": true
}
}
}
}
}
I have trouble combining term, must_not queries on nested documents.
Sense example can be found here : http://sense.qbox.io/gist/be436a1ffa01e4630a964f48b2d5b3a1ef5fa176
Here my mapping :
{
"mappings": {
"docs" : {
"properties": {
"tags" : {
"type": "nested",
"properties" : {
"type": {
"type": "string",
"index": "not_analyzed"
}
}
},
"label" : {
"type": "string"
}
}
}
}
}
with two documents in this index :
{
"tags" : [
{"type" : "POST"},
{"type" : "DELETE"}
],
"label" : "item 1"
},
{
"tags" : [
{"type" : "POST"}
],
"label" : "item 2"
}
When I query this index like this :
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}
}
}
I've got one hit (which is correct)
When I want to get documents WHICH DON'T CONTAIN the tag "DELETE", with this query :
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "delete"
}
}
}
}
}
}
}
I've got 2 hits (which is incorrect).
This issue seems very close to this one (Elasticsearch array must and must_not) but it's not...
Can you give me some clues to resolve this issue ?
Thank you
Your original query would search in each individual nested object and eliminate the objects that don't match, but if there are some nested objects left, they do match with your query and so you get your results. This is because nested objects are indexed as a hidden separate document
Original code:
{
"query": {
"nested": {
"path": "tags",
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "delete"
}
}
}
}
}
}
}
The solution is then quite simple really, you should bring the bool query outside the nested documents. Now all the documents are discarded who have a nested object with the "DELETE" type. Just what you wanted!
The solution:
{
"query": {
"bool": {
"must_not": {
"nested": {
"path": "tags",
"query": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}
}
}
NOTE: Your strings are "not analyzed" and you searched for "delete" instead of "DELETE". If you want to search case insensitive, make your strings analyzed
This should fix your problem: http://sense.qbox.io/gist/f4694f542bc76c29624b5b5c9b3ecdee36f7e3ea
Two most important things:
include_in_root on "tags.type". This will tell ES to index tag types as "doc.tags.types" : ['DELETE', 'POSTS'], so you can access an array of those values "flattened" on the root doc . This means you no longer need a nested query (see #2)
Drop the nested query.
{
"mappings": {
"docs" : {
"properties": {
"tags" : {
"type": "nested",
"properties" : {
"type": {
"type": "string",
"index": "not_analyzed"
}
},
"include_in_root": true
},
"label" : {
"type": "string"
}
}
}
}
}
{
"query": {
"bool": {
"must_not": {
"term": {
"tags.type": "DELETE"
}
}
}
}
}