search in array of objects in elasticsearch - elasticsearch

I am using elasticsearch-6.4.3
I have created an index test_index with the following mapping:
PUT test_index
{
"mappings": {
"_doc": {
"properties": {
"name": {
"type": "keyword"
},
"segments": {
"type": "nested",
"properties": {
"f1": {
"type": "keyword"
},
"f2": {
"type": "integer"
}
}
}
}
}
}
}
I inserted a document in the index with the following command:
POST /test_index/_doc
{
"name": "ABC",
"segments": [{
"f1": "abc",
"f2": 123
},
{
"f1": "def",
"f2": 124
}]
}
I have an array of values say arr = ["def", "xyz"]. I want to match all the documents whose atleast one of the f1 field in segments field match with any of the values in the given array arr.
You can call it something like array_intersect to easier to understand.
I am trying something like this:
GET /test_index/_search
{
"query": {
"nested": {
"path": "segments",
"query": {
"bool": {
"must": [
{
"terms": {
"f1": [
"def",
"xyz"
]
}
}
]
}
}
}
}
}
I am getting no results as the output.
Expected output: The document should match as "def" is present as value of f1 in the document.

You would need to specify the full path in your terms query as segments.f1 instead of f1 for nested query.
Modify the query as below:
POST test_index/_search
{
"query":{
"nested":{
"path":"segments",
"query":{
"bool":{
"must":[
{
"terms": {
"segments.f1": ["def","xyz"] <---- Mention full path here
}
}
]
}
}
}
}
}
Hope this helps!

Related

ElasticSearch query nested path filter OR

I have following index:
PUT /ab11
{
"mappings": {
"properties": {
"product_id": {
"type": "keyword"
},
"data": {
"type": "nested",
"properties": {
"p_id": {
"type": "keyword"
}
}
}
}
}
}
PUT /ab11/_doc/1
{
"product_id": "123",
"data": [
{
"p_id": "a"
},
{
"p_id": "b"
},
{
"p_id": "c"
}
]
}
I want to do query like following sql does(NOTE: I want to do filter not query, because I don't care about score) :
select * from abc11 where data.pid = "a" or data.pid = "b"
You can do it like this because the terms query has OR semantics by default:
{
"query": {
"nested": {
"path": "data",
"query": {
"terms": {
"data.p_id": [
"a",
"b"
]
}
}
}
}
}
Basically, select all documents which have either "a" or "b" in their data.p_id nested docs.

Elasticsearch query by generic properties: keywords and numeric values

I have this mapping in ES 7.9:
{
"mappings": {
"properties": {
"cid": {
"type": "keyword",
"store": true
},
"id": {
"type": "keyword",
"store": true
},
"a": {
"type": "nested",
"properties": {
"attribute":{
"type": "keyword"
},
"key": {
"type": "keyword"
},
"num": {
"type": "float"
}
}
}
}
}
}
And some documents indexed like:
{
"cid": "177",
"id": "1",
"a": [
{
"attribute": "tags",
"key": [
"heel",
"thong",
"low_heel",
"economic"
]
},
{
"attribute": "weight",
"num": 15
}
]
}
Basically, an object can have multiple attributes (a property array).
Those attributes can be different for each client. In this example, I have 2 types of attributes: tag and weight, however other documents could have other attributes like vendor, size, power, etc., so the model has to be generic enough to support beforehand unknown attributes.
An attribute can be a list of keywords (like tags) or a numeric value (like weight).
I need an ES query to fetch the documents ids with this pseudo-query:
cid="177" and (tag="flat" or tag="heel") and tag="economic" and weight<20
I managed to reach this query that seems to be working as expected:
{
"_source": ["id"],
"query": {
"bool": {
"must" : [
{"term" : { "cid" : "177" }},
{
"nested": {
"path": "a",
"query": {
"bool":{
"must":[
{"term" : { "a.attribute": "tags"}},
{"terms" : { "a.key": ["flat","heel"]}}
]
}
}
}
},
{
"nested": {
"path": "a",
"query": {
"bool":{
"must":[
{"term" : { "a.attribute": "tags"}},
{"term" : { "a.key": "economic"}}
]
}
}
}
},
{
"nested": {
"path": "a",
"query": {
"bool":{
"must":[
{"term" : { "a.attribute": "weight" } },
{"range": { "a.num": {"lt": 20} } }
]
}
}
}
}
]
}
}
}
Is this query correct or I am getting the correct results by chance?
Is the query (or mapping) optimal or I should rethink something?
Can the query be simplified?
The query is correct.
The mapping is great and the query is optimal.
While the query can be simplified:
{
"_source": [
"id"
],
"query": {
"bool": {
"must": [
{
"term": {
"cid": "177"
}
},
{
"nested": {
"path": "a",
"query": {
"query_string": {
"query": "a.attribute:tags AND ((a.key:flat OR a.key:heel) AND a.key:economic)"
}
}
}
},
{
"nested": {
"path": "a",
"query": {
"query_string": {
"query": "a.attribute:weight AND a.num:<20"
}
}
}
}
]
}
}
}
it'd be less optimal due to the fact that these query_strings would still need to be internally compiled into essentially the query DSL that you've got above. Plus you'd still be needing the two separate nested groups so... You're good to roll with what you've got.

Elasticsearch nested query and sorting

can somebody help me to understand what Elastic means by nested. In documentations https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_nested_sorting_examples is an example which does not show how the document object looks like. It look like I should imagine the mapping from the search query. Query looks like:
POST /_search
{
"query": {
"nested": {
"path": "parent",
"query": {
"bool": {
"must": {"range": {"parent.age": {"gte": 21}}},
"filter": {
"nested": {
"path": "parent.child",
"query": {"match": {"parent.child.name": "matt"}}
}
}
}
}
}
},
"sort" : [
{
"parent.child.age" : {
"mode" : "min",
"order" : "asc",
"nested": {
"path": "parent",
"filter": {
"range": {"parent.age": {"gte": 21}}
},
"nested": {
"path": "parent.child",
"filter": {
"match": {"parent.child.name": "matt"}
}
}
}
}
}
]
}
Can somebody write a document structure on which this query will work?
Something like this.
{
"parent": {
"name": "Elasti Sorch",
"age": 23,
"child": [
{
"name": "Kibana Lion",
"age": 12
},
{
"name": "Matt",
"age": 15
}
]
}
}
In Elastic nested means it's an array of objects. To store an array of objects into a field in elastic search you have to map the field to a nested while creating the index.
PUT parent
{
"mappings": {
"doc":{
"properties": {
"name":{
"type": "text"
},
"age":{
"type": "integer"
},
"child":{
"type": "nested",
"properties": {
"name":{
"type":"text"
},
"age":{
"type":"integer"
}
}
}
}
}
}
}
and a sample nested document cab be inserted like this
POST parent/doc
{
"name":"abc",
"age":50,
"child":[
{
"name":"son1",
"age":25
},
{
"name":"adughter1",
"age":20
}
]
}

How to search on multiple fields in URI Search

I would like to perform an AND operation in ElasticSearch using the URI Search (q=). How do I do it?
If I have document like:
[{ "name":"Test 1", "pub":"2"}, { "name":"Test 2", "pub":"1"}, { "name":"A", "pub":"1"}]
And I would like to query for documents containing with a name containing "Test" AND where pub equals "1". How do I do that?
Thanks!
Assuming your document looks like this:
{
"my_field": [
{ "name":"Test 1", "pub":"2"},
{ "name":"Test 2", "pub":"1"},
{ "name":"A", "pub":"1"}
]
}
And the mapping of my_field is of type nested similar to this:
{
"mappings": {
"doc_type": {
"properties": {
"my_field": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"pub": {"type": "integer" }
}
}
}
}
}
}
Then you can query your index and get the expected documents with the following nested query:
POST /_search
{
"query": {
"nested": {
"path": "my_field",
"query": {
"bool": {
"filter": [
{
"match": {
"name": "Test"
}
},
{
"match": {
"pub": 1
}
}
]
}
}
}
}
}
Actually you'd need nested fields. The following is a good resource.
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html

Elasticsearch: nested object under path is not of nested type

I've been trying to search on my document which contains a nested field. I created the nested mapping like this:
{
"message": {
"properties": {
"messages": {
"type": "nested",
"properties": {
"message_id": { "type": "string" },
"message_text": { "type": "string" },
"message_nick": { "type": "string" }
}
}
}
}
}
My search looks like this:
curl -XGET 'localhost:9200/thread_and_messages/thread/_search' \
-d '{"query": {"bool": {"must": [{"match": {"thread_name": "Banana"}}, {"nested": {"path": "messages", "query": {"bool": {"must": [{"match": {"messages.message_text": "Banana"}}]}}}]}}}}'
Yet I am receiving this error message:
QueryParsingException[[thread_and_messages] [nested] nested object under path [messages] is not of nested type]
EDIT
I am still receiving this error. I am doing this via Java so this is the document I am trying to create:
{
"_id": {
"path": "3",
"thread_id": "3",
"thread_name": "Banana",
"created": "Wed Mar 25 2015",
"first_nick": "AdminTech",
"messages": [
{
"message_id": "9",
"message_text": "Banana",
"message_nick": "AdminTech"
}
]
}
}
Creating the index like so:
CreateIndexRequestBuilder indexRequest = client.admin().indices().prepareCreate(INDEX).addMapping("message", mapping);
I think I am possibly indexing the document incorrectly.
TLDR: Put "type": "nested", in your nested type.
Say we have a normal type, and another type nested in it:
{
"some_index": {
"mappings": {
"normal_type": {
"properties": {
"nested_type": {
"type": "nested",
"properties": {
"address": {
"type": "string"
},
"country": {
"type": "string"
}
}
},
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
}
}
The "type": "nested", line is required for the nested queries to work which have "path": assigned to nested_type, like this:
GET /some_index/normal_type/_search
{
"query": {
"nested": {
"query": {
"bool": {}
},
"path": "nested_type"
}
}
}
The "type": "nested", line seems to be required in newer Elasticsearch versions only (since 1.1.1 ?).
Syntax error in query DSL. Incorrect closing for must block query->bool->must
{
"query": {
"bool": {
"must": [
}// Should be ]
}
}
}
Correct version query are :
curl -XGET 'localhost:9200/thread_and_messages/thread/_search' -d '{
"query": {
"bool": {
"must": [
{
"match": {
"thread_name": "Banana"
}
},
{
"nested": {
"path": "messages",
"query": {
"bool": {
"must": [
{
"match": {
"messages.message_text": "Banana"
}
}
]
}
}
}
}
]
}
}
}'
If your field is type object you will need to use the flatten name <object name>.<object key> has if it was a normal variable. For example:
Not this
{
"nested":
{
"path":"album",
"query":{
"bool":{
"boost":5,
"should":[{"match":{"album.name":"lady"}}]
}
}
}
},
Yes to this
{
"match":{
"genre":{
"query":"lady",
"boost":2
}
}
},
In meme form:

Resources