Make query in elasticsearch - elasticsearch

I'm learning elastic for a new project.
I have a index with format:
{
"content_id": "bbbbbb",
"title": "content 2",
"data": [
{
"id": "xx",
"value": 3,
"tags": ["a","b","c"]
},
{
"id": "yy",
"value": 1,
"tags": ["e","d","c"]
}
]
}
How can i make a query to search contents that have at least one element in data that include tags "a" and "b" ?
thanks so much !!
How can i make query or re design my format data to easy make new query ?

Working with lists and nested documents in Elasticsearch is a bit tricky.
When creating the index, you need to specify the mapping for the nested documents. You can do this by adding a mapping for the data field.
PUT /my_index
{
"mappings": {
"doc": {
"properties": {
"data": {
"type": "nested"
}
}
}
}
}
If you have already created the index, you should delete it first, then create it again with the mapping.
Before deleting the index, you can reindex (copy) the data to a new index.
Now you can query the data field using the nested query:
GET /my_index/_search
{
"query": {
"nested": {
"path": "data",
"query": {
"terms": {
"data.tags": [
"j",
"b",
"c"
]
}
}
}
}
}

If the requirement is to get the document if at least one object in data contains a and b both, then you need to specify the nested mapping as suggested by #ChamRun.
But for getting the results having a and b both, you need to use the below query :
{
"query": {
"nested": {
"path": "data",
"query": {
"bool": {
"must": [
{
"term": {
"data.tags": "a"
}
},
{
"term": {
"data.tags": "b"
}
}
]
}
}
}
}
}

Related

terms query not working in Elastic search with value having space in it

We need to get the data based on multiple values.
So I am trying to use terms query in elastic search for modelNumber field.
But it is not working as expected.can anyone let me know what is wrong with the query.
POST index_name/_Search
{
"query": {
"bool": {
"must": [
{
"terms": {
"modelNumber": [
"test 1234rthg-1234-1234512-2345",
"testMode11l-123-rtyu-xyz11"
]
}
},
{
"terms": {
"userId": [
"123",
"VALUE2"
]
}
}
]
}
}
}
Terms query returns documents that contain one or more exact terms in
a provided field.
If you have not explicitly defined any index mapping, then you need to add .keyword to the modelNumber field. This uses the keyword analyzer instead of the standard analyzer (notice the ".keyword" after modelNumber field).
{
"query": {
"bool": {
"must": [
{
"terms": {
"modelNumber.keyword": [ // note this
"test 1234rthg-1234-1234512-2345",
"testMode11l-123-rtyu-xyz11"
]
}
},
{
"terms": {
"userId": [
"123",
"VALUE2"
]
}
}
]
}
}
}
OR you need to modify the mapping of modelNUmber field as -
{
"mappings": {
"properties": {
"modelNumber": {
"type": "keyword"
}
}
}
}

Multiple (AND) queries for a nested index structure in Elasticsearch

I have an index with the below mapping
{
"mappings": {
"xxxxx": {
"properties": {
"ID": {
"type": "text"
},
"pairs": {
"type": "nested"
},
"xxxxx": {
"type": "text"
}
}
}
}
}
the pairs field is essentially an array of objects - each object has a unique ID associated with it
What i'm trying to do is to get only one object from the pairs field for updates. To that extent , i've tried this
GET /sample/_search/?size=1000
{
"query": {
"bool": {
"must": [
{
"match": {
"ID": "2rXdCf5OM9g1ebPNFdZNqW"
}
},
{
"match": {
"pairs.id": "c1vNGnnQLuk"
}
}
]
}
},
"_source": "pairs"
}
but this just returns an empty object despite them being valid IDs. If i remove the pairs.id rule - i get the entire array of objects .
What do i need to add/edit to ensure that i can query via both IDS (original and nested)
Since pairs is of nested type, you need to use a nested query. Also you might probably want to leverage nested inner-hits as well:
GET /sample/_search/?size=1000
{
"query": {
"bool": {
"must": [
{
"match": {
"ID": "2rXdCf5OM9g1ebPNFdZNqW"
}
},
{
"nested": {
"path": "pairs",
"query": {
"match": {
"pairs.id": "c1vNGnnQLuk"
}
},
"inner_hits": {}
}
}
]
}
},
"_source": false
}

Elasticsearch sorting by matching array item

I have a following structure in indexed documents:
document1: "customLists":[{"id":8,"position":8},{"id":26,"position":2}]
document2: "customLists":[{"id":26,"position":1}]
document3: "customLists":[{"id":8,"position":1},{"id":26,"position":3}]
I am able to search matching documents that belong to a given list with match query "customLists.id = 26". But I need to sort the documents based on the position value within that list and ignore positions of the other lists.
So the expected results would be in order of document2, document1, document3
Is the data structure suitable for this kind of sorting and how to handle this?
One way to achieve this would be to set mapping type of customLists as nested and then use sorting by nested fields
Example :
1) Create Index & Mapping
put test
put test/test/_mapping
{
"properties": {
"customLists": {
"type": "nested",
"properties": {
"id": {
"type": "integer"
},
"position": {
"type": "integer"
}
}
}
}
}
2) Index Documents :
put test/test/1
{
"customLists":[{"id":8,"position":8},{"id":26,"position":2}]
}
put test/test/2
{
"customLists":[{"id":26,"position":1}]
}
put test/test/3
{
"customLists":[{"id":8,"position":1},{"id":26,"position":3}]
}
3) Query to sort by positon for given id
post test/_search
{
"filter": {
"nested": {
"path": "customLists",
"query": {
"term": {
"customLists.id": {
"value": "26"
}
}
}
}
},
"sort": [
{
"customLists.position": {
"order": "asc",
"mode": "min",
"nested_filter": {
"term": {
"customLists.id": {
"value": "26"
}
}
}
}
}
]
}

How to display "ALL" the nested documents in an object in separate rows from elasticsearch?

I have a nested object in the following form:
{
"name": "Multi G. Enre",
"books": [
{
"name": "Guns and lasers",
"genre": "scifi",
"publisher": "orbit"
},
{
"name": "Dead in the night",
"genre": "thriller",
"publisher": "penguin"
}
]
}
I tried the following JSON query for the above document:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "books",
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"term": {
"books.publisher": "penguin"
}
},
{
"term": {
"books.genre": "thriller"
}
}
]
}
}
}
}
}
}
}
}
So,I would like to see the second nested document i.e. "Dead in the night" as the result but, for anything I search only the first document i.e. "Guns and lasers" is displayed in the table in elasticsearch head plugin.
So, is there any way I can display the nested documents separately based on the search query and not just the first document?
I'm new to elasticsearch,so would appreciate any type of responses. THANKYOU!
You need to use inner_hits in your query.
Moreover, if you want to only retrieve the matching nested document and nothing else, you can add "_source":["books"] to your query and only the matching nested books will be returned, nothing else.
UPDATE
Sorry, I misunderstood your comment. You can add "_source": false and the top-level document will not be returned. Only the nested matching document.

Searching objects having all nested children matching a given query in Elasticsearch

Given an object with the following mapping:
{
"a": {
"properties": {
"id": {"type": "string"}
"b": {
"type": "nested",
"properties": {
"key": {"type": "string"}
}
}
}
}
}
I want to retrieve all the instances of this object having all nested children matching a given query.
For example, suppose I want to retrieve all the instances having all children with "key" = "yes".
Given the following instances:
{
"id": "1",
"b": [
{
"key": "yes"
},
{
"key": "yes"
}
]
},
{
"id": "2",
"b": [
{
"key": "yes"
},
{
"key": "yes"
},
{
"key": "no"
}
]
},
I want to retrieve only the first one (the one with "id" = "1").
Both using filters or queries is fine to me.
I already tried to use the "not filter" and the "must_not bool filter". The idea was to use a double negation to extract only objects that doesn't have fields that are different to the given one.
However, I was not able to write down this query correctly.
I realize that this is not a common query for a search engine, but, in my case, it can be useful.
Is it possible to write this query ("forall nested query") using nested objects?
In case it is not, would it be possible to write this query using parent-child?
Update
Andrei Stefan gave a good answer in case we know all the values of "key" that we want to avoid, ("no", in the example).
I am interested also in the case you don't know the values you want to avoid, and you just want to match nested object with "key"="yes".
You need a flattened data structure for this - an array of values. The simplest way and not to change the current mapping too much, is to use include_in_parent property and to query the field that's being included in the parent for this particular requirement:
{
"mappings": {
"a": {
"properties": {
"id": {
"type": "string"
},
"b": {
"type": "nested",
"include_in_parent": true,
"properties": {
"key": {
"type": "string"
}
}
}
}
}
}
}
And then your query would look like this:
{
"query": {
"filtered": {
"filter": {
"and": [
{
"query": {
"query_string": { "query": "b.key:(yes NOT no)"}
}
}
]
}
}
}
}
The alternative is to change the type of the field from nested to object but in this way you'll loose the advantages of using nested fields:
{
"mappings": {
"a": {
"properties": {
"id": {
"type": "string"
},
"b": {
"type": "object",
"properties": {
"key": {
"type": "string"
}
}
}
}
}
}
}
The query remains the same.
Encountered the same problem, though didn't have just yes/no variants.
As per Clinton Gormley's answer in https://github.com/elastic/elasticsearch/issues/19166:
"You can't do it any efficient way. You have to count all children and compare that to how many children match. The following will return all parents where all children match but it is a horrible inefficient solution and I would never recommend using it in practice":
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "b",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"match_all": {}
},
"functions": [
{
"weight": -1
},
{
"filter": {
"match": {
"b.key": "yes"
}
},
"weight": 1
}
],
"score_mode": "sum",
"boost_mode": "replace"
}
}
}
}
]
}
}
}

Resources