How to pass array to ElasticSearch search template using mustache? - elasticsearch

This is a part of my search query template:
"query": {
"bool": {
"must": [
{
"terms": {
"platform_id": [
"{{#platform_ids}}",
"{{.}}",
"{{/platform_ids}}"
]
}
}
],
This solution was taken from the elasticsearch official website.
I have this query:
GET _render/template
{
"id": "fddf",
"params": {
"query_string": "tinders",
"platform_ids": [1, 2]
}
}
And this results with this:
"terms" : {
"platform_id" : [
"",
"1",
"",
"2",
""
]
}
But I need this:
"terms" : {
"platform_id" : [
1, 2
]
}
Couldn't find a solution for this.

You need to modify your template as shown below
GET _render/template
{
"source": "{\"query\":{\"bool\":{\"must\":[{\"terms\":{\"platform_id\":[{{#platform_id}}{{value}}{{#comma}},{{/comma}}{{/platform_id}}]}}]}}}",
"params": {
"platform_id": [
{
"value": 1,
"comma": true
},
{
"value": 2
}
]
}
}
Result would be
{
"template_output": {
"query": {
"bool": {
"must": [
{
"terms": {
"platform_id": [
1,
2
]
}
}
]
}
}
}
}
Refer to this SO answer to get a detailed explanation

Request:
GET _render/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}bids{{/toJson}} }}",
"params": {
"bids":[1,2]
}
}
Response:
{
"template_output" : {
"query" : {
"terms" : [
1,
2
]
}
}
}
Try below:
"query": {
"bool": {
"must": [
{
"terms": {
"platform_id": {{#toJson}}platform_id{{/toJson}}
}
}
],

Related

elastic - query multiple levels on nested object in inner_hits

i have a huge nested object which has lots of levels
i want to create a query which will return only the leaf / some object in the middle,
and the query is supposed to query multiple levels in the tree.
for example:
my DB is saving the whole company structure.
company -> wards -> employees -> working hours
i want to make a query that will return only the working hours of the employees in ward 2 which started later than 3pm this month
i tried to use inner_hits - but to no use.
as requested, sample document and expected result:
company:[{
properties:{companyId: 112}
ward:[{
properties: {wardId: 223}
employee:{
properties: {employeeId: 334},
workingHours: [
{ date: "1.1.2021", numOfHours: 4},
{ date: "1.2.2021", numOfHours: 7}
]
}]
}]
}]
the query:
I need to return the working hours of date "1.2.21" , of employee 334, of ward 223. and only the working hours, not the whole tree.
expected result:
4 or { date: "1.1.2021", numOfHours: 4} , whatever is simpler
hope its clear now
You need to add inner_hits to all nested queries
You can either parse entire result to get matched working hours(from inner hits) o can use response filtering to remove additional data
Mapping
PUT index123
{
"mappings": {
"properties": {
"company": {
"type": "nested",
"properties": {
"ward": {
"type": "nested",
"properties": {
"employee": {
"type": "nested",
"properties": {
"workingHours": {
"type": "nested",
"properties": {
"date": {
"type": "date"
}
}
}
}
}
}
}
}
}
}
}
}
Data
"_index" : "index123",
"_type" : "_doc",
"_id" : "9gGYI3oBt-MOenya6BcN",
"_score" : 1.0,
"_source" : {
"company" : [
{
"companyId" : 112,
"ward" : [
{
"wardId" : 223,
"employee" : {
"employeeId" : 334,
"workingHours" : [
{
"date" : "2021-01-01",
"numOfHours" : 4
},
{
"date" : "2021-01-02",
"numOfHours" : 7
}
]
}
}
]
}
]
}
}
Query
GET index123/_search?filter_path=hits.hits.inner_hits.ward.hits.hits.inner_hits.employee.hits.hits.inner_hits.workingHours.hits.hits._source
{
"query": {
"nested": {
"inner_hits": {
"name":"ward"
},
"path": "company.ward",
"query": {
"bool": {
"must": [
{
"term": {
"company.ward.wardId": {
"value": 223
}
}
},
{
"nested": {
"inner_hits": {
"name":"employee"
},
"path": "company.ward.employee",
"query": {
"bool": {
"must": [
{
"term": {
"company.ward.employee.employeeId": {
"value":334
}
}
},
{
"nested": {
"inner_hits": {
"name":"workingHours"
},
"path": "company.ward.employee.workingHours",
"query": {
"range": {
"company.ward.employee.workingHours.date": {
"gte": "2021-01-01",
"lte": "2021-01-01"
}
}
}
}
}
]
}
}
}
}
]
}
}
}
}
}
Result
{
"hits" : {
"hits" : [
{
"inner_hits" : {
"ward" : {
"hits" : {
"hits" : [
{
"inner_hits" : {
"employee" : {
"hits" : {
"hits" : [
{
"inner_hits" : {
"workingHours" : {
"hits" : {
"hits" : [
{
"_source" : {
"date" : "2021-01-01",
"numOfHours" : 4
}
}
]
}
}
}
}
]
}
}
}
}
]
}
}
}
}
]
}
}
Update:
Query with company ID
GET index123/_search?filter_path=hits.hits.inner_hits.company.hits.hits.inner_hits.ward.hits.hits.inner_hits.employee.hits.hits.inner_hits.workingHours.hits.hits._source
{
"query": {
"nested": {
"path": "company",
"inner_hits": {
"name": "company"
},
"query": {
"bool": {
"must": [
{
"term": {
"company.companyId": {
"value": 112
}
}
},
{
"nested": {
"inner_hits": {
"name": "ward"
},
"path": "company.ward",
"query": {
"bool": {
"must": [
{
"term": {
"company.ward.wardId": {
"value": 223
}
}
},
{
"nested": {
"inner_hits": {
"name": "employee"
},
"path": "company.ward.employee",
"query": {
"bool": {
"must": [
{
"term": {
"company.ward.employee.employeeId": {
"value": 334
}
}
},
{
"nested": {
"inner_hits": {
"name": "workingHours"
},
"path": "company.ward.employee.workingHours",
"query": {
"range": {
"company.ward.employee.workingHours.date": {
"gte": "2021-01-01",
"lte": "2021-01-01"
}
}
}
}
}
]
}
}
}
}
]
}
}
}
}
]
}
}
}
}
}

Using multiple Should queries

I want to get docs that are similar to multiple "groups" but separately. Each group has it's own rules (terms).
When I try to use more than one Should query inside a "bool" I get items that are a mix of both Should's terms.
I want to use 1 query total and not msearch for example.
Can someone please help me with that?
{
"explain": true,
"query": {
"filtered": {
"filter": {
"bool": {
"must_not": [
{
"term": {
"p_id": "123"
}
},
{
"term": {
"p_id": "124"
}
}
]
}
},
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"minimum_should_match": 1,
"should": [
{
"term": {
"cat": "1"
}
},
{
"term": {
"cat": "2"
}
},
{
"term": {
"keys": "a"
}
},
{
"term": {
"keys": "b"
}
}
]
}
},
{
"bool": {
"minimum_should_match": 1,
"should": [
{
"term": {
"cat": "6"
}
},
{
"term": {
"cat": "7"
}
},
{
"term": {
"keys": "r"
}
},
{
"term": {
"keys": "u"
}
}
]
}
}
]
}
}
}
},
"from": 0,
"size": 3
}
You can try using a terms aggregation on multiple fields with scripting and add a top hits aggregation as a sub-aggregation. Be warned this will be pretty slow. Add this after the query/filter and adjust the size parameter as needed
"aggs": {
"Cat_and_Keys": {
"terms": {
"script": "doc['cat'].values + doc['keys'].values"
},
"aggs":{ "separate_docs": {"top_hits":{"size":1 }} }
}
}

Elasticsearch - OR in term conditions

I need little help with transfering mysql query to ES. The query looks like this
SELECT * FROM `xyz` WHERE visibility IN (1,2) AND (active=0 OR (active=1 AND finished=1)
It's easy, to make only AND conditions, but how to mix AND with OR in term?
"query" : {
"bool" : {
"must" : [{
"terms" : { "visibility" : ["1", "2"] }
}, {
"term" : { "active" : "1" }
}, {
"term" : { "active" : "0", "finished" : "1" } // OR
},]
}
}
Try like this by nesting a bool/should and bool/filter query inside the main bool/filter query:
{
"query": {
"bool": {
"filter": [
{
"terms": {
"visibility": [
"1",
"2"
]
}
},
{
"bool": {
"should": [
{
"term": {
"active": "0"
}
},
{
"bool": {
"filter": [
{
"term": {
"active": "1"
}
},
{
"term": {
"finished": "1"
}
}
]
}
}
]
}
}
]
}
}
}

Search for records in elastic search with set of values for each field.

I have a set of records in elastic search
I want to compare 1 specific records (I have it with me in a variable) with rest of the records present in the index.
I want to compare values for each field individually with the values from the record that i have.
For eg.
If i have
{
"person" : [
{
"name" : "asd",
"awdawD" : "asdawd"
},
{
"name" : "adw ed",
"awdawD" : "asdawd"
},
{
"name" : "a wra",
"awdawD" : "asdawd"
},
{
"name" : "ca eyy",
"awdawD" : "asdawd"
}
]
}
and my record is
[
{
"name" : "asd",
"awdawD" : "asdawd"
},
{
"name" : "s tr",
"awdawD" : " tjy"
}
]
I want to compare name with name, awdawD with awdawD and get the result.
Since I have an array of Persons, will i have to create a loop for each element in the records I have, to compare it with the records in elastic search?
How can I do this in elastic search with out the loop ?
Comparing each name and awdawD individually.
I know it's probably not the elegant solution you're looking for (which I don't think exists), but you could always do something like this:
POST /test_index/_search
{
"query": {
"filtered": {
"filter": {
"or": {
"filters": [
{
"and": {
"filters": [
{
"term": {
"name": "asd"
}
},
{
"term": {
"awdawD": "asdawd"
}
}
]
}
},
{
"and": {
"filters": [
{
"term": {
"name": "s tr"
}
},
{
"term": {
"awdawD": " tjy"
}
}
]
}
}
]
}
}
}
}
}
or this
POST /test_index/_search
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"match": {
"name": "asd"
}
},
{
"match": {
"awdawD": "asdawd"
}
}
]
}
},
{
"bool": {
"must": [
{
"match": {
"name": "s tr"
}
},
{
"match": {
"awdawD": " tjy"
}
}
]
}
}
]
}
}
}
Both of these queries return only your first document (one or the other may be preferable depending on your analysis setup).
Here is some code I used to test it out:
http://sense.qbox.io/gist/52de7f75e685486a63d18bd387cbb02d03cc6feb

How to add filter to a more like this query in Elastic Search?

I want to use a More like this query in Elastic Search to find similar documents. However I need to filter the documents on which the query is executed.
Example below: I want to find blog items that are similar to blog with id 123456, but written by author 120 or author 123.
When executing this query, I get back similar blogs from ALL authors and thus not filtered ...
{
"query":{
"more_like_this" : {
"fields" : ["body" ],
"docs" : [
{
"_id" : "123456"
}
],
"percent_terms_to_match" : 0.4,
"min_term_freq" : 1
}
}
},
"filter":{
"and":[
{
"type":{ "value":"blog" }
},
{
"terms":{ "authorId": ["120", "123"] }
}
]
}
}
Try filtered query like this:
{
"query": {
"filtered": {
"query": {
"more_like_this": {
"fields": [
"body"
],
"docs": [
{
"_id": "123456"
}
],
"percent_terms_to_match": 0.4,
"min_term_freq": 1
}
},
"filter": {
"and": [
{
"type": {
"value": "blog"
}
},
{
"terms": {
"authorId": [
"120",
"123"
]
}
}
]
}
}
}
}
Hope it helps...!
The accepted answer is for earlier versions of ElasticSearch. This one works great on 2.x+ also not using any depreciated APIs
{
"query": {
"filtered": {
"query": {
"more_like_this": {
"fields": ["meta.keywords"],
"like": [{"_id": "5732759249d2b21f95641d50"}]
}
},
"filter" : {
"bool": {
"must": [
{"match": { "foo.bar": "A"}},
{"match": { "baz": "new"}}
]
}
}
}
}
}

Resources