I would like to perform an AND operation in ElasticSearch using the URI Search (q=). How do I do it?
If I have document like:
[{ "name":"Test 1", "pub":"2"}, { "name":"Test 2", "pub":"1"}, { "name":"A", "pub":"1"}]
And I would like to query for documents containing with a name containing "Test" AND where pub equals "1". How do I do that?
Thanks!
Assuming your document looks like this:
{
"my_field": [
{ "name":"Test 1", "pub":"2"},
{ "name":"Test 2", "pub":"1"},
{ "name":"A", "pub":"1"}
]
}
And the mapping of my_field is of type nested similar to this:
{
"mappings": {
"doc_type": {
"properties": {
"my_field": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"pub": {"type": "integer" }
}
}
}
}
}
}
Then you can query your index and get the expected documents with the following nested query:
POST /_search
{
"query": {
"nested": {
"path": "my_field",
"query": {
"bool": {
"filter": [
{
"match": {
"name": "Test"
}
},
{
"match": {
"pub": 1
}
}
]
}
}
}
}
}
Actually you'd need nested fields. The following is a good resource.
https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html
Related
I am trying to implement a search-as-you-type query inside an array.
This is the structure of the documents:
{
"guid": "6f954d53-df57-47e3-ae9e-cb445bd566d3",
"labels":
[
{
"name": "London",
"lang": "en"
},
{
"name": "Llundain",
"lang": "cy"
},
{
"name": "Lunnainn",
"lang": "gd"
}
]
}
and up to now this is what I came with:
{
"query": {
"multi_match": {
"fields": ["labels.name"],
"query": name,
"type": "phrase_prefix"
}
}
which works exactly as requested.
The problem is that I would like to search also by language.
What I tried is:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
but these queries act on separate values of the array.
So, for example, I would like to search only Welsh language (cy). That means that my query that contains the city name should match only values that have "cy" on the "lang" tag.
How do I write this kind of query?
Internally, ElasticSearch flattens nested JSON objects, so it can't correlate the lang and name of a specific element in the labels array. If you want this kind of correlation, you'll need to index your documents differently.
The usual way to do this is to use the nested data type with a matching nested query.
The query would end up looking something like this:
{
"query": {
"nested": {
"path": "labels",
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
}
}
But note that you'll need to also specify nested mappings for your labels, e.g.:
"properties": {
"labels": {
"type": "nested",
"properties": {
"name": {
"type": "text"
/* you might want to add other mapping-related configuration here */
},
"lang": {
"type": "keyword"
}
}
}
}
Other ways to do this include:
Indexing each label as a separate document, repeating the guid field
Using parent/child documents
You should use Nested datatype in mapping instead of Object datatype. For detail explanation refer this:
https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html
So, you should define mapping of your field something like this:
{
"properties": {
"labels": {
"type": "nested",
"properties": {
"name": {
"type": "text"
},
"lang": {
"type": "keyword"
}
}
}
}
}
After this you could query using Nested Query as:
{
"query": {
"nested": {
"path": "labels",
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
}
}
I have the post_filter as below, Where I am trying to filter records where the school name is HILL SCHOOL AND containing a nested child object with name JOY AND section A.
school is present in the parent object, Which is holding children list of nested objects.
All of the above are AND conditions.
But the query doesn't seem to work. Any idea why ? And is there a way to combine the two nested queries?
GET /test_school/_search
{
"query": {
"match_all": {}
},
"post_filter": {
"bool": {
"must_not": [
{
"bool": {
"must": [
{
"term": {
"schoolname": {
"value": "HILL SCHOOL"
}
}
},
{
"nested": {
"path": "children",
"query": {
"bool": {
"must": [
{
"match": {
"name": "JACK"
}
}
]
}
}
}
},
{
"term": {
"children.section": {
"value": "A"
}
}
}
]
}
}
]
}
}
}
The schema is as below:
PUT /test_school
{
"mappings": {
"_doc": {
"properties": {
"schoolname": {
"type": "keyword"
},
"children": {
"type": "nested",
"properties": {
"name": {
"type": "keyword",
"index": true
},
"section": {
"type": "keyword",
"index": true
}
}
}
}
}
}
}
Sample data as below:
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JOY",
"section":"A"
}
}
second record
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JACK",
"section":"B"
}
}
https://stackoverflow.com/a/17543151/183217 suggests special mapping is needed to work with nested objects. You appear to be falling foul of the "cross object matching" problem.
I have seen similar questions posted, but of course, none are exactly what I am trying to do.
When I run the query below I get this error:
"reason": "[nested] failed to find nested object under path [contentGroup]"
I think the problem is contentGroup.name does not exist because contentGroup is an array not an object. It needs to be something like this:
contentGroup[0].name
and
contentGroup[1].name
But I can't figure out how to do that.
Another thing that might be wrong is that I have two items nested within each other, I don't know if that is right or not.
Any help would be great!
My mapping:
{
"mappings": {
"articles": {
"properties": {
"contentGroups": {
"type": "nested",
"properties": {
"contentGroup": {
"type": "nested",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
}
}
}
}
What gets created when I input in an article ( Note the array being created ):
"contentGroups": {
"contentGroup": [
{
"name": "Breaking",
"id": "104"
},
{
"name": "News",
"id": "22"
}
]
My query:
{
"query": {
"bool": {
"must": [
{ "match": { "headline": "whatever" }},
{
"nested": {
"path": "contentGroup",
"query": {
"bool": {
"must": [
{ "match": { "contentGroup.name": "Breaking" }}
]
}
}
}
}
]
}
}
You should use simpler mapping:
{
"mappings": {
"articles": {
"properties": {
"contentGroups": {
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
}
}
}
Each field in elasticsearch already supports multiple values, no need to specify this.
Here's the mapping:
PUT books-index
{
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested",
"fields": {
"name": {
"type": "string"
},
"weight": {
"type": "float"
}
}
}
}
}
}
}
Then doing a nested query using a field_value_factor fails with an error
GET books-index/books/_search
{
"query": {
"nested": {
"path": "tags",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"match": {
"tags.name": "world"
}
},
"field_value_factor": {
"field": "weight"
}
}
}
}
}
}
The error: "nested: ElasticsearchException[Unable to find a field mapper for field [weight]]"
Interestingly, if there's one book in the index with tags - there's no error and the query works well.
Why is this happening? how can I prevent the error when there are no books with tags in the index?
Any ideas?
Thank you!
P.S. There's also an issue on github for this: https://github.com/elastic/elasticsearch/issues/12871
it looks like your mapping is incorrect.
After PUTing the mapping you provided, try executing GET books-index/_mapping, It will show these results:
"books-index": {
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested"
}
}
}
}
}
It's missing name and weight! The problem with the mapping is that you used either you used fields instead of properties, or you forget to put in a second properties key.
I modified your mapping to reflect that you were looking for a nested name and type within tags, as it looks like that is what your query wants.
PUT books-index
{
"mappings": {
"books": {
"properties": {
"tags": {
"type": "nested",
"properties": { // <--- HERE!
"name": {
"type": "string"
},
"weight": {
"type": "float"
}
}
}
}
}
}
}
I've been trying to search on my document which contains a nested field. I created the nested mapping like this:
{
"message": {
"properties": {
"messages": {
"type": "nested",
"properties": {
"message_id": { "type": "string" },
"message_text": { "type": "string" },
"message_nick": { "type": "string" }
}
}
}
}
}
My search looks like this:
curl -XGET 'localhost:9200/thread_and_messages/thread/_search' \
-d '{"query": {"bool": {"must": [{"match": {"thread_name": "Banana"}}, {"nested": {"path": "messages", "query": {"bool": {"must": [{"match": {"messages.message_text": "Banana"}}]}}}]}}}}'
Yet I am receiving this error message:
QueryParsingException[[thread_and_messages] [nested] nested object under path [messages] is not of nested type]
EDIT
I am still receiving this error. I am doing this via Java so this is the document I am trying to create:
{
"_id": {
"path": "3",
"thread_id": "3",
"thread_name": "Banana",
"created": "Wed Mar 25 2015",
"first_nick": "AdminTech",
"messages": [
{
"message_id": "9",
"message_text": "Banana",
"message_nick": "AdminTech"
}
]
}
}
Creating the index like so:
CreateIndexRequestBuilder indexRequest = client.admin().indices().prepareCreate(INDEX).addMapping("message", mapping);
I think I am possibly indexing the document incorrectly.
TLDR: Put "type": "nested", in your nested type.
Say we have a normal type, and another type nested in it:
{
"some_index": {
"mappings": {
"normal_type": {
"properties": {
"nested_type": {
"type": "nested",
"properties": {
"address": {
"type": "string"
},
"country": {
"type": "string"
}
}
},
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
}
}
The "type": "nested", line is required for the nested queries to work which have "path": assigned to nested_type, like this:
GET /some_index/normal_type/_search
{
"query": {
"nested": {
"query": {
"bool": {}
},
"path": "nested_type"
}
}
}
The "type": "nested", line seems to be required in newer Elasticsearch versions only (since 1.1.1 ?).
Syntax error in query DSL. Incorrect closing for must block query->bool->must
{
"query": {
"bool": {
"must": [
}// Should be ]
}
}
}
Correct version query are :
curl -XGET 'localhost:9200/thread_and_messages/thread/_search' -d '{
"query": {
"bool": {
"must": [
{
"match": {
"thread_name": "Banana"
}
},
{
"nested": {
"path": "messages",
"query": {
"bool": {
"must": [
{
"match": {
"messages.message_text": "Banana"
}
}
]
}
}
}
}
]
}
}
}'
If your field is type object you will need to use the flatten name <object name>.<object key> has if it was a normal variable. For example:
Not this
{
"nested":
{
"path":"album",
"query":{
"bool":{
"boost":5,
"should":[{"match":{"album.name":"lady"}}]
}
}
}
},
Yes to this
{
"match":{
"genre":{
"query":"lady",
"boost":2
}
}
},
In meme form: