Elastic Search search query for nested array - elasticsearch

{
"application": {
"package_name": "com.jackhenry.OregonFirstCU",
"countries": [
{
"short_name": "US"
}
]
},
"application": {
"package_name": "com.jackhenry.OregonFirstCU",
"countries": [
{
"short_name": "US"
}
]
},
"application": {
"package_name": "com.jackhenry.OregonFirstCU",
"countries": [
]
}
}
How can I get results for both empty array and US for application.countries and application.countries.short_name

In short you want to fetch results which either contain US or do not contain any country name. You can apply should between term and not exists
{
"query": {
"bool": {
"should": [
{
"bool": {
"must_not": {
"exists": {
"field": "application.countries.short_name"
}
}
}
},
{
"term": {
"application.countries.short_name": [
"US"
]
}
}
]
}
}

Related

Elasticsearch query to filter document by specific field value with conditions

I want to filter brand names if supplied but return all brands if the filter is empty.
here is my query.
"query": {
"bool": {
"must": [
{
"terms": {
"seller": [
"Seller1",
"Seller2"
]
}
},
{
"bool": {
"should": [
{
"terms": {
"brand": [
"brand1"
]
}
},
{
"bool": {
"must_not": [
{
"term": {
"brand": {
"value": ""
}
}
}
]
}
}
]
}
},
{
"nested": {
"path": "Stock",
"query": {
"bool": {
"must": {
"match": {
"region": "Regional"
}
},
"filter": [
{
"term": {
"pincode": "12345"
}
}
]
}
}
}
}
]
}
}
I want if (brand == ["brand1","brand2"]) return those brand products only. But if supplied empty return all. I tried to use it with should, still getting everything if brand is supplied.
There are multipul ways and some of are mentioned below:
Option 1:
You can handle this logic of include, exclude at the application level while creating queries. If you are using Java or python client then it is easily achievable. If you are calling direct search API of elasticsearch then you can not add brand term clause while calling api.
Option 2:
You can use the search template functionality of Elasticsearch.
First you need to create search template as shown in below example:
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": """{
"query": {
"bool": {
"must": [
{
"terms": {
"seller": [
"seller1",
"seller2"
]
}
}
],
"filter": [
{{#filter}}
{
"terms": {
"{{name}}":
{{#toJson}}value{{/toJson}}
}
}{{/filter}}
]
}
}
}"""
}
}
You can use below API to search using search template, here if you dont pass brand in filter it will not add in actual query:
POST querycheck/_search/template
{
"id": "my-search-template",
"params": {
"filter": [
{
"name": "brand",
"value": [
"brand1",
"brand2"
]
}
]
}
}
You can use render api as well to check if query template is generating query correct or not.
POST _render/template
{
"id": "my-search-template",
"params": {
"filter": [
{
"name": "brand",
"value": [
"brand1",
"brand2"
]
}
]
}
}

Multiple AND and OR condition in elasticsearch

I having trouble with elasticsearch query.
Data Structrue:
[{agent : "abc", origin: "US"}, {agent : "abc", origin: "US"}
I'm not able to find multiple agent name (OR condition) and (AND Condition) multiple Origin (OR condition)
You can use a combination of bool/must/should clause along with the terms query
{
"query": {
"bool": {
"must": [
{
"terms": {
"agent": [
"abc",
"abc"
]
}
},
{
"terms": {
"origin": [
"US",
"US"
]
}
}
]
}
}
}
Since terms already has OR semantics, you don't need to wrap them in bool/should queries. The following query should do what you expect:
{
"query": {
"bool": {
"filter": [
{
"terms": {
"agent": [
"agent1",
"agent2"
]
}
},
{
"terms": {
"origin": [
"US",
"CA"
]
}
}
]
}
}
}

ElasticSearch - Filtering data returned from nested query

I am have a set of data in the following structure:
[
{
"productId": "ProductId1",
"customerNumbers": [
"customer": {
"name": "Prod1Cust1",
"totalOrders": 23
},
"customer": {
"name": "Prod2Cust1",
"totalOrders": 5
},
"customer": {
"name": "Prod3Cust1",
"totalOrders": 5
}
]
},
{
"productId": "ProductId2",
"customerNumbers": [
"customer": {
"name": "Prod2Cust1",
"totalOrders": 23
},
"customer": {
"name": "Prod2Cust1",
"totalOrders": 5
}
]
}
]
and I need to fetch all the records which have a prefix of "Prod1 as in name field(in the example avoid, only first record should be returned i.e. ProductId1). Also, when the data is returned, I need to just fetch just the customer number whose prefix is Prod1 i.e:
Correct Output:
{
"productId": "ProductId1",
"customerNumbers": [
"customer": {
"name": "Prod1Cust1",
"totalOrders": 23
}
]
}
Instead of:
{
"productId": "ProductId1",
"customerNumbers": [
"customer": {
"name": "Prod1Cust1",
"totalOrders": 23
},
"customer": {
"name": "Prod2Cust1",
"totalOrders": 5
},
"customer": {
"name": "Prod3Cust1",
"totalOrders": 5
}
]
}
I'm able to fetch the records whose Name prefix is "Prod1" using nested query coupled with MatchPhrasePrefixQuery (this returns me result with all the customer numbers). How can I further filter the data to get customer numbers whose Name prefix is "Prod1".
Following is my current query:
{
"from": 0,
"size": 10,
"sort": [
{
"name.keyword": {
"missing": "_first",
"order": "asc"
}
}
],
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"nested": {
"query": {
"bool": {
"must": [
{
"match": {
"customerNumbers.name": {
"query": "Prod1",
"type": "phrase_prefix"
}
}
}
]
}
},
"path": "customerNumbers"
}
}
]
}
}
]
}
}
}
P.S: I'm using ElasticSearch 5.x with Nest.
Try using inner_hits:
PUT products
{"mappings":{"_doc":{"properties":{"customerNumbers":{"type":"nested"}}}}}
POST products/_doc
{"productId":"ProductId1","customerNumbers":[{"name":"Prod1Cust1","totalOrders":23},{"name":"Prod2Cust1","totalOrders":5},{"name":"Prod3Cust1","totalOrders":5}]}
POST products/_doc
{"productId":"ProductId2","customerNumbers":[{"name":"Prod2Cust1","totalOrders":23},{"name":"Prod2Cust1","totalOrders":5}]}
GET products/_search
{
"_source": "inner_hits",
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"nested": {
"path": "customerNumbers",
"query": {
"bool": {
"must": [
{
"match_phrase_prefix": {
"customerNumbers.name": {
"query": "Prod1"
}
}
}
]
}
},
"inner_hits": {}
}
}
]
}
}
]
}
}
}
yielding the following hits
[
{
"_index":"products",
"_type":"_doc",
"_id":"tyQGo3EBdiyDG0RsTa0N",
"_score":0.9808292,
"_source":{
},
"inner_hits":{
"customerNumbers":{
"hits":{
"total":1,
"max_score":0.9808292,
"hits":[
{
"_index":"products",
"_type":"_doc",
"_id":"tyQGo3EBdiyDG0RsTa0N",
"_nested":{
"field":"customerNumbers",
"offset":0
},
"_score":0.9808292,
"_source":{
"name":"Prod1Cust1", <-----
"totalOrders":23
}
}
]
}
}
}
}
]

Match multiple properties on the same nested document in ElasticSearch

I'm trying to accomplish what boils down to a boolean AND on nested documents in ElasticSearch. Let's say I have the following two documents.
{
"id": 1,
"secondLevels": [
{
"thirdLevels": [
{
"isActive": true,
"user": "anotheruser#domain.com"
}
]
},
{
"thirdLevels": [
{
"isActive": false,
"user": "user#domain.com"
}
]
}
]
}
{
"id": 2,
"secondLevels": [
{
"thirdLevels": [
{
"isActive": true,
"user": "user#domain.com"
}
]
}
]
}
In this case, I want to only match documents (in this case ID: 2) that have a nested document with both isActive: true AND user: user#domain.com.
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "secondLevels.thirdLevels",
"query": {
"bool": {
"must": [
{
"term": {
"secondLevels.thirdLevels.isActive": true
}
},
{
"term": {
"secondLevels.thirdLevels.user": "user#domain.com"
}
}
]
}
}
}
}
]
}
}
}
However, what seems to be happening is that my query turns up both documents because the first document has one thirdLevel that has isActive: true and another thirdLevel that has the appropriate user.
Is there any way to enforce this strictly at query/filter time or do I have to do this in a script?
With nested-objects and nested-query, you have made most of the way.
All you have to do now is to add the inner hits flag and also use source filtering for move entire secondLevels documents out of the way:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "secondLevels.thirdLevels",
"query": {
"bool": {
"must": [
{
"term": {
"secondLevels.thirdLevels.isActive": true
}
},
{
"term": {
"secondLevels.thirdLevels.user": "user#domain.com"
}
}
]
}
},
"inner_hits": {
"size": 100
}
}
}
]
}
}
}

How to query multiple parameters in a nested field in elasticsearch

I'm trying to search for keyword and then add nested queries for amenities which is a nested field of an array of objects.
With the query below I am able to search when I'm only matching one amenity id but when I have more than one it doesn't return anything.
Anyone have an idea what is wrong with my query ?
{
"sort": [
{
"_score": {
"order": "desc"
}
},
{
"_geo_distance": {
"geolocation": [
100,
10
],
"order": "asc",
"unit": "m",
"mode": "min",
"distance_type": "sloppy_arc"
}
}
],
"query": {
"bool": {
"must": [
{
"multi_match": {
"fields": [
"name^2",
"city",
"state",
"zip"
],
"fuzziness": 5,
"query": "complete"
}
},
{
"nested": {
"path": "amenities",
"query": {
"bool": {
"must": [
{
"term": {
"amenities.id": "1"
}
},
{
"term": {
"amenities.id": "2"
}
}
]
}
}
}
}
]
}
}
}
When you do:
"must": [
{
"term": {
"amenities.id": "1"
}
},
{
"term": {
"amenities.id": "2"
}
}]
What you're actually saying is find me any document where "amenities.id"="1" and "amenities.id"="2" which unless "amenities.id" is a list of values it won't work.
What you probably want to say is find me any document where "amenities.id"="1" or "amenities.id"="2"
To do that you should use should instead of must:
"should": [
{
"term": {
"amenities.id": "1"
}
},
{
"term": {
"amenities.id": "2"
}
}]

Resources