Elastic Search URI Search Filtering - elasticsearch

I have a movies ES index with the following mapping:
{
"doc": {
"properties": {
"director": {
"type": "text"
},
"genres": {
"type": "text",
"index": "false"
},
"title": {
"type": "text"
},
"year": {
"type": "long"
}
}
}
}
For genres, I'll be submitting an array of strings. Now, I want to allow my customers to submit a query against my endpoint, where I'd proxy q request parameter down to ES client.
But I am struggling to filter the data with the URI search the same way it can be achieved by submitting a query as a payload against my index. For instance:
GET movies/doc/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"genres": "War"
}
}
],
"must": [
{
"term": {
"genres": "Foo"
}
}
],
"must": [
{
"term": {
"genres": "Bar"
}
}
]
}
}
}
}
}
Can the same be achieved via URI search? If so, how?

From ES documentation
You can use query parameters to define your search criteria directly
in the request URI, rather than in the request body. Request URI
searches do not support the full Elasticsearch Query DSL, but are
handy for testing.
Based on our discussion, I suggest you build a DSL query in the backend after receiving request from the client. You need to expose an API which will send UI inputs to the backend in a structured format(JSON) which can help you build the DSL query easily.

Here is how I ended up doing it, without the q parameter. Version 6.0.1:
POST http://127.0.0.1:9200/movies/_mapping/doc
{
"doc": {
"properties": {
"director": {
"type": "text"
},
"title": {
"type": "text"
},
"year": {
"type": "long"
},
"genres": {
"type": "keyword"
}
}
}
}
Create documents:
POST http://127.0.0.1:9200/movies/doc/
{
"title": "title 1",
"director": "Igor M",
"year": 2011,
"genres": ["Bar", "Foo"]
}
Query for documents:
GET http://127.0.0.1:9200/movies/_search
{
"query": {
"bool": {
"must":[
{ "match": { "genres": "War" }},
{ "match": { "genres": "Bar" }}
]
}
}
}

Related

Search-as-you-type inside arrays

I am trying to implement a search-as-you-type query inside an array.
This is the structure of the documents:
{
"guid": "6f954d53-df57-47e3-ae9e-cb445bd566d3",
"labels":
[
{
"name": "London",
"lang": "en"
},
{
"name": "Llundain",
"lang": "cy"
},
{
"name": "Lunnainn",
"lang": "gd"
}
]
}
and up to now this is what I came with:
{
"query": {
"multi_match": {
"fields": ["labels.name"],
"query": name,
"type": "phrase_prefix"
}
}
which works exactly as requested.
The problem is that I would like to search also by language.
What I tried is:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
but these queries act on separate values of the array.
So, for example, I would like to search only Welsh language (cy). That means that my query that contains the city name should match only values that have "cy" on the "lang" tag.
How do I write this kind of query?
Internally, ElasticSearch flattens nested JSON objects, so it can't correlate the lang and name of a specific element in the labels array. If you want this kind of correlation, you'll need to index your documents differently.
The usual way to do this is to use the nested data type with a matching nested query.
The query would end up looking something like this:
{
"query": {
"nested": {
"path": "labels",
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
}
}
But note that you'll need to also specify nested mappings for your labels, e.g.:
"properties": {
"labels": {
"type": "nested",
"properties": {
"name": {
"type": "text"
/* you might want to add other mapping-related configuration here */
},
"lang": {
"type": "keyword"
}
}
}
}
Other ways to do this include:
Indexing each label as a separate document, repeating the guid field
Using parent/child documents
You should use Nested datatype in mapping instead of Object datatype. For detail explanation refer this:
https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html
So, you should define mapping of your field something like this:
{
"properties": {
"labels": {
"type": "nested",
"properties": {
"name": {
"type": "text"
},
"lang": {
"type": "keyword"
}
}
}
}
}
After this you could query using Nested Query as:
{
"query": {
"nested": {
"path": "labels",
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": ["labels.name"],
"query": "london",
"type": "phrase_prefix"
}
},
{
"term": {
"labels.lang": "gd"
}
}
]
}
}
}
}
}

Elastic Search query for an AND condition on two properties of a nested object

I have the post_filter as below, Where I am trying to filter records where the school name is HILL SCHOOL AND containing a nested child object with name JOY AND section A.
school is present in the parent object, Which is holding children list of nested objects.
All of the above are AND conditions.
But the query doesn't seem to work. Any idea why ? And is there a way to combine the two nested queries?
GET /test_school/_search
{
"query": {
"match_all": {}
},
"post_filter": {
"bool": {
"must_not": [
{
"bool": {
"must": [
{
"term": {
"schoolname": {
"value": "HILL SCHOOL"
}
}
},
{
"nested": {
"path": "children",
"query": {
"bool": {
"must": [
{
"match": {
"name": "JACK"
}
}
]
}
}
}
},
{
"term": {
"children.section": {
"value": "A"
}
}
}
]
}
}
]
}
}
}
The schema is as below:
PUT /test_school
{
"mappings": {
"_doc": {
"properties": {
"schoolname": {
"type": "keyword"
},
"children": {
"type": "nested",
"properties": {
"name": {
"type": "keyword",
"index": true
},
"section": {
"type": "keyword",
"index": true
}
}
}
}
}
}
}
Sample data as below:
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JOY",
"section":"A"
}
}
second record
POST /test_school/_doc
{
"schoolname":"HILL SCHOOL",
"children":{
"name":"JACK",
"section":"B"
}
}
https://stackoverflow.com/a/17543151/183217 suggests special mapping is needed to work with nested objects. You appear to be falling foul of the "cross object matching" problem.

Elasticsearch nested query and sorting

can somebody help me to understand what Elastic means by nested. In documentations https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_nested_sorting_examples is an example which does not show how the document object looks like. It look like I should imagine the mapping from the search query. Query looks like:
POST /_search
{
"query": {
"nested": {
"path": "parent",
"query": {
"bool": {
"must": {"range": {"parent.age": {"gte": 21}}},
"filter": {
"nested": {
"path": "parent.child",
"query": {"match": {"parent.child.name": "matt"}}
}
}
}
}
}
},
"sort" : [
{
"parent.child.age" : {
"mode" : "min",
"order" : "asc",
"nested": {
"path": "parent",
"filter": {
"range": {"parent.age": {"gte": 21}}
},
"nested": {
"path": "parent.child",
"filter": {
"match": {"parent.child.name": "matt"}
}
}
}
}
}
]
}
Can somebody write a document structure on which this query will work?
Something like this.
{
"parent": {
"name": "Elasti Sorch",
"age": 23,
"child": [
{
"name": "Kibana Lion",
"age": 12
},
{
"name": "Matt",
"age": 15
}
]
}
}
In Elastic nested means it's an array of objects. To store an array of objects into a field in elastic search you have to map the field to a nested while creating the index.
PUT parent
{
"mappings": {
"doc":{
"properties": {
"name":{
"type": "text"
},
"age":{
"type": "integer"
},
"child":{
"type": "nested",
"properties": {
"name":{
"type":"text"
},
"age":{
"type":"integer"
}
}
}
}
}
}
}
and a sample nested document cab be inserted like this
POST parent/doc
{
"name":"abc",
"age":50,
"child":[
{
"name":"son1",
"age":25
},
{
"name":"adughter1",
"age":20
}
]
}

Elastic Search nested multimatch query

So my problem is basically the same as described here, however it still remains unanswered on the group.
My mapping:
{
"abstract": {
"properties": {
"summary": {
"type": "string"
}
}
},
"authors": {
"type": "nested",
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
And I would like to perform a full-text search on both of these fields, probably unequally weighted. The query that comes to my mind, but unfortunately doesn't work, would be this:
{
"query": {
"bool": {
"should": [{
"multi_match": {
"query": "higgs boson",
"fields": ["abstract.summary^5", "author.last_name^2"]
}
}]
}
}
}
I don't get any results from the authors field, because of its nested mapping. I also can't get rid of the nested property - I use it for aggregations. Any elegant idea how to solve it?
The only solution that I managed to work out, which is not handy nor elegant but somehow works is such query:
"query": {
"bool": {
"should": [
{
"nested": {
"path": "authors",
"query": {
"multi_match": {
"query": "higgs",
"fields": ["last_name^2"]
}
}
}
},
{
"multi_match": {
"query": "higgs",
"fields": ["abstract.summary^5"]
}
}
]
}
}
I'm also not sure if the boosting will work as expected, providing it's set in different queries. Any suggestions appreciated.
Changing your mapping to the following one which uses include_in_root: true will allow you to use the query you original wrote:
{
"abstract": {
"properties": {
"summary": {
"type": "string"
}
}
},
"authors": {
"type": "nested",
"include_in_root": true,
"properties": {
"first_name": {
"type": "string"
},
"last_name": {
"type": "string"
}
}
}
}
You may want to index inner objects both as nested fields and as flattened object fields. This can be achieved by setting include_in_parent to true. - Link
Note: include_in_root may be deprecated in future versions of elasticsearch in favor of copy_to.

ElasticSearch: search inside the array of objects

I have a problem with querying objects in array.
Let's create very simple index, add a type with one field and add one document with array of objects (I use sense console):
PUT /test/
PUT /test/test/_mapping
{
"test": {
"properties": {
"parent": {"type": "object"}
}
}
}
POST /test/test
{
"parent": [
{
"name": "turkey",
"label": "Turkey"
},
{
"name": "turkey,mugla-province",
"label": "Mugla (province)"
}
]
}
Now I want to search by both names "turkey" and "turkey,mugla-province" . The first query works fine:
GET /test/test/_search {"query":{ "term": {"parent.name": "turkey"}}}
But the second one returns nothing:
GET /test/test/_search {"query":{ "term": {"parent.name": "turkey,mugla-province"}}}
I tried a lot of stuff including:
"parent": {
"type": "nested",
"include_in_parent": true,
"properties": {
"label": {
"type": "string",
"index": "not_analyzed"
},
"name": {
"type": "string",
"store": true
}
}
}
But nothing helps. What do I miss?
Here's one way you can do it, using nested docs:
I defined an index like this:
PUT /test_index
{
"mappings": {
"doc": {
"properties": {
"parent": {
"type": "nested",
"properties": {
"label": {
"type": "string"
},
"name": {
"type": "string"
}
}
}
}
}
}
}
Indexed your document:
PUT /test_index/doc/1
{
"parent": [
{
"name": "turkey",
"label": "Turkey"
},
{
"name": "turkey,mugla-province",
"label": "Mugla (province)"
}
]
}
Then either of these queries will return it:
POST /test_index/_search
{
"query": {
"nested": {
"path": "parent",
"query": {
"match": {
"parent.name": "turkey"
}
}
}
}
}
POST /test_index/_search
{
"query": {
"nested": {
"path": "parent",
"query": {
"match": {
"parent.name": "turkey,mugla-province"
}
}
}
}
}
Here's the code I used:
http://sense.qbox.io/gist/6258f8c9ee64878a1835b3e9ea2b54e5cf6b1d9e
For search multiple terms use the Terms query instead of Term query.
"terms" : {
"tags" : [ "turkey", "mugla-province" ],
"minimum_should_match" : 1
}
There are various ways to construct this query, but this is the simplest and most elegant in the current version of ElasticSearch (1.6)

Resources