I have an index with nested objects houses.
My index contains these documents:
{
"_id": "hello",
"name": "pippos",
"houses": [
{
"address": "garden square",
"id1": 1,
"id2": 5
},
{
"address": "top square",
"id1": 1,
"id2": 5
}
]
},
{
"_id": "hellone",
"name": "pippoone",
"houses": [
{
"address": "central square",
"id1": 1,
"id2": 9
},
{
"address": "minimale square",
"id1": 1,
"id2": 5
}
]
}
Using this query I receive both documents:
GET /pippis/_search
{
"query": {
"nested": {
"path": "houses",
"query": {
"bool": {
"must": [
{ "match": { "houses.id1": 1 }},
{ "match": { "houses.id2": 5 }}
]
}
}
}
}
}
I want only documents having all houses with id1=1 and id2=5
Mapping:
PUT /user
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"houses": {
"type": "nested",
"properties": {
"address": {
"type": "text"
},
"id1": {
"type": "integer"
},
"id2": {
"type": "integer"
}
}
}
}
}
}
Data:
"hits" : [
{
"_index" : "user",
"_type" : "_doc",
"_id" : "5kQ6-2wBWSK8eKKSSozQ",
"_score" : 1.0,
"_source" : {
"name" : "pippos",
"houses" : [
{
"address" : "garden square",
"id1" : 1,
"id2" : 5
},
{
"address" : "top square",
"id1" : 1,
"id2" : 5
}
]
}
},
{
"_index" : "user",
"_type" : "_doc",
"_id" : "50Q9-2wBWSK8eKKStIzf",
"_score" : 1.0,
"_source" : {
"name" : "pippoone",
"houses" : [
{
"address" : "central square",
"id1" : 1,
"id2" : 9
},
{
"address" : "minimale square",
"id1" : 1,
"id2" : 5
}
]
}
},
{
"_index" : "user",
"_type" : "_doc",
"_id" : "6ERM-2wBWSK8eKKS3IzD",
"_score" : 1.0,
"_source" : {
"name" : "pippoone1",
"houses" : [
{
"address" : "central square",
"id1" : 2,
"id2" : 9
},
{
"address" : "minimale square",
"id1" : 2,
"id2" : 5
}
]
}
}
]
}
Query:
GET /user/_search
{
"query": {
"bool": {
"must_not": [ -----> Not of documents returned in nested query
{
"nested": {
"path": "houses",
"query": {
"bool": {
"should": [ -----> get documents where id1 is not 1 or id2 is not 5
{
"bool": {
"must_not": [
{
"match": {
"houses.id1": 1
}
}
]
}
},
{
"bool": {
"must_not": [
{
"match": {
"houses.id2": 5
}
}
]
}
}
]
}
}
}
}
]
}
}
}
Result:
[
{
"_index" : "user",
"_type" : "_doc",
"_id" : "5kQ6-2wBWSK8eKKSSozQ",
"_score" : 0.0,
"_source" : {
"name" : "pippos",
"houses" : [
{
"address" : "garden square",
"id1" : 1,
"id2" : 5
},
{
"address" : "top square",
"id1" : 1,
"id2" : 5
}
]
}
}
]
Related
I'm new to Elastic Search, I have document like below :
Mapping of same JSON index is like below :
Mapping
{
"mappings": {
"properties": {
"age": {
"type": "long"
},
"hobbiles": {
"type": "keyword"
}
}
}
}
Some sample documents are like below :
[{
"_id": "test#domain.com",
"age": 12,
"hobbiles": [{
"name": "Singing",
"level": "begineer"
},
{
"name": "Dancing",
"level": "begineer"
}
]
},
{
"_id": "test1#domain.com",
"age": 7,
"hobbiles": [{
"name": "Coding",
"level": "begineer"
},
{
"name": "Chess",
"level": "begineer"
}
]
},
{
"_id": "test2#domain.com",
"age": 20,
"hobbiles": [{
"name": "Singing",
"level": "begineer"
},
{
"name": "Dancing",
"level": "begineer"
}
]
},
{
"_id": "test3#domain.com",
"age": 21,
"hobbiles": [{
"name": "Coding",
"level": "begineer"
},
{
"name": "Dancing",
"level": "Football"
}
]
}
]
Now I want to fetch documents where id IN (test#domain.com, test1#domain.com) and age is greater than 5. [operationally] hobiiles Football.
My expectations from output is I should get three documents: and if hobbies is not matching then also it should be fine but if hobbies matches then that document should be on top. Basically I want to match hobbies but its optional if it doesn't match then also I should get data based on prior clauses.
[test3#domain.com, test#domain.com, test1#domain.com]
test3 on top because Football matches there, and test and test1 because age and id matches there.
Tldr;
It can be achieved via bool queries.
Solution
PUT /_bulk
{"index":{"_index":"73935795", "_id":"test#domain.com"}}
{"age":12,"hobbiles":[{"name":"Singing","level":"begineer"},{"name":"Dancing","level":"begineer"}]}
{"index":{"_index":"73935795", "_id":"test1#domain.com"}}
{"age":7,"hobbiles":[{"name":"Coding","level":"begineer"},{"name":"Chess","level":"begineer"}]}
{"index":{"_index":"73935795", "_id":"test2#domain.com"}}
{"age":20,"hobbiles":[{"name":"Singing","level":"begineer"},{"name":"Dancing","level":"begineer"}]}
{"index":{"_index":"73935795", "_id":"test3#domain.com"}}
{"age":21,"hobbiles":[{"name":"Coding","level":"begineer"},{"name":"Dancing","level":"Football"}]}
GET 73935795/_search
{
"query": {
"bool": {
"filter": [
{
"range": {
"age": {
"gt": 5
}
}
},
{
"terms": {
"_id": [
"test#domain.com",
"test1#domain.com",
"test3#domain.com"
]
}
}
],
"should": [
{
"query_string": {
"query": "(football) OR (begineer)",
"default_field": "hobbiles.level"
}
}
]
}
}
}
This requires using Should clause. Should is equivalent to "OR". So a document will be returned if it satisfies any one condition in should query.
For conditions on id and age I have used filter clause. It is equivalent to "AND" . Filter clause does not calculate score for matched documents so any document which matches "hobbiles.level" will be ranked higher.
Query
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"term": {
"hobbiles.level.keyword": {
"value": "Football"
}
}
},
{
"bool": {
"filter": [
{
"terms": {
"id.keyword": [
"test#domain.com",
"test1#domain.com"
]
}
},
{
"range": {
"age": {
"gt": 5
}
}
}
]
}
}
]
}
}
}
Result
"hits" : [
{
"_index" : "index8",
"_type" : "_doc",
"_id" : "qE06noMBfFiM6spcUTo4",
"_score" : 1.3112575,
"_source" : {
"id" : "test3#domain.com",
"age" : 21,
"hobbiles" : [
{
"name" : "Coding",
"level" : "begineer"
},
{
"name" : "Dancing",
"level" : "Football"
}
]
}
},
{
"_index" : "index8",
"_type" : "_doc",
"_id" : "pE03noMBfFiM6spc4jr2",
"_score" : 0.0,
"_source" : {
"id" : "test#domain.com",
"age" : 12,
"hobbiles" : [
{
"name" : "Singing",
"level" : "begineer"
},
{
"name" : "Dancing",
"level" : "begineer"
}
]
}
},
{
"_index" : "index8",
"_type" : "_doc",
"_id" : "pU03noMBfFiM6spc6DqZ",
"_score" : 0.0,
"_source" : {
"id" : "test1#domain.com",
"age" : 7,
"hobbiles" : [
{
"name" : "Coding",
"level" : "begineer"
},
{
"name" : "Chess",
"level" : "begineer"
}
]
}
}
]
I have the following data sample:
{
"_index" : "index-stats-202110",
"_type" : "_doc",
"_id" : "yT7vlHwBLghXjfKlKm7e",
"_score" : 9.583174,
"_source" : {
"client_id" : "111f34db-c88c-4675-8a69-f3028b3dfa18",
"campaign_id" : "2c3c62f7-9e77-48df-a211-108a2220a063",
"created_at" : "1634586272",
"date_start" : "1634580000",
"date_end" : "1634583600",
"specs" : [
"events",
"general"
],
"data" : {
"count" : 13
}
}
},
{
"_index" : "index-stats-202110",
"_type" : "_doc",
"_id" : "yj7vlHwBLghXjfKlKm7e",
"_score" : 9.583174,
"_source" : {
"client_id" : "111f34db-c88c-4675-8a69-f3028b3dfa18",
"campaign_id" : "2c3c62f7-9e77-48df-a211-108a2220a063",
"created_at" : "1634586272",
"date_start" : "1634580000",
"date_end" : "1634583600",
"specs" : [
"events",
"visit"
],
"data" : {
"label" : "visit",
"count" : 13
}
}
},
{
"_index" : "index-stats-202110",
"_type" : "_doc",
"_id" : "yz7vlHwBLghXjfKlKm7e",
"_score" : 9.583174,
"_source" : {
"client_id" : "111f34db-c88c-4675-8a69-f3028b3dfa18",
"campaign_id" : "2c3c62f7-9e77-48df-a211-108a2220a063",
"created_at" : "1634586272",
"date_start" : "1634580000",
"date_end" : "1634583600",
"specs" : [
"events"
],
"data" : {
"count" : 1
},
"geo" : {
"country" : "q",
"province" : "q",
"city" : "t"
}
}
},
{
"_index" : "index-stats-202110",
"_type" : "_doc",
"_id" : "zD7vlHwBLghXjfKlKm7e",
"_score" : 9.583174,
"_source" : {
"client_id" : "111f34db-c88c-4675-8a69-f3028b3dfa18",
"campaign_id" : "2c3c62f7-9e77-48df-a211-108a2220a063",
"created_at" : "1634586272",
"date_start" : "1634580000",
"date_end" : "1634583600",
"specs" : [
"events"
],
"data" : {
"count" : 1
},
"geo" : {
"country" : "j",
"province" : "q",
"city" : "d"
}
}
}
which I get from the query:
{
"query": {
"bool": {
"must": [
{
"term": {
"client_id": "111f34db-c88c-4675-8a69-f3028b3dfa18"
}
},
{
"term": {
"campaign_id": "2c3c62f7-9e77-48df-a211-108a2220a063"
}
},
{
"term": {
"specs": "events"
}
}
]
}
}
}
and this is my mapping:
{
"properties":{
"specs":{
"type":"keyword"
},
"campaign_id":{
"type":"keyword"
},
"client_id":{
"type":"keyword"
},
"created_at":{
"type":"date"
},
"date_start":{
"type":"date"
},
"date_end":{
"type":"date"
},
"geo":{
"type":"nested",
"properties":{
"country":{
"type":"keyword"
},
"province":{
"type":"keyword"
},
"city":{
"type":"keyword"
}
}
},
"data":{
"enabled":false
}
}
}
I want to get the geo.country = 'q' which I have in my sample, but when I try to execute the following query I get an empty response:
{
"query": {
"bool": {
"must": [
{
"term": {
"client_id": "111f34db-c88c-4675-8a69-f3028b3dfa18"
}
},
{
"term": {
"campaign_id": "2c3c62f7-9e77-48df-a211-108a2220a063"
}
},
{
"term": {
"specs": "events"
}
},
{
"term": {
"geo.country": "q"
}
}
]
}
}
}
Question: How can I separate the geo.country = 'q' from the list?
You can use nested query along with bool/must clause to achieve your required result
{
"query": {
"bool": {
"must": [
{
"term": {
"client_id": "111f34db-c88c-4675-8a69-f3028b3dfa18"
}
},
{
"term": {
"campaign_id": "2c3c62f7-9e77-48df-a211-108a2220a063"
}
},
{
"term": {
"specs": "events"
}
},
{
"nested": {
"path": "geo",
"query": {
"term": {
"geo.country": "q"
}
}
}
}
]
}
}
}
I have data:
[
{
"NAME": "John Doe",
"CLASS":[1,10,30]
},
{
"NAME": "Albert",
"CLASS": [1,10,40]
},
{
"NAME": "XINN",
"CLASS": [10,30]
},
{
"NAME": "UJANG",
"CLASS": [1,40]
},
{
"NAME": "BAMBANG",
"CLASS": [30,40]
}
]
I have the following query DSL:
{
query: {
terms: {
class: [1,10]
}
}
}
and I want what will appear is:
[{"NAME": "John Doe","CLASS":[1,10,30]},{"NAME": "Albert","CLASS": [1,10,40]}]
How do I change my search to match the result?
You need to combine multiple term queries in must clause.
Query
{
"query": {
"bool": {
"must": [
{
"term": {
"CLASS": {
"value": 1
}
}
},
{
"term": {
"CLASS": {
"value": 10
}
}
}
]
}
}
}
Result
"hits" : [
{
"_index" : "index34",
"_type" : "_doc",
"_id" : "2EdK6XsBP61bDf9bI3R1",
"_score" : 2.0,
"_source" : {
"NAME" : "John Doe",
"CLASS" : [
1,
10,
30
]
}
},
{
"_index" : "index34",
"_type" : "_doc",
"_id" : "2UdK6XsBP61bDf9bMHT5",
"_score" : 2.0,
"_source" : {
"NAME" : "Albert",
"CLASS" : [
1,
10,
40
]
}
}
]
Our Elastic Mapping
{"mappings": {
"products" : {
"properties":{
"name " : {
"type" : "keyword"
},
"resellers" : {
"type" : "nested",
"properties" : {
"name" : { "type" : "text" },
"price" : { "type" : "double" }
}
}
}
}
}}
In this mapping each product stores the list of resellers which are selling it at specific price.
We have requirement where we want to get count of products sell by specific resellers at specific price, I am able to get it for single reseller by using reverse nested agg and cardinality agg using following query DSL
. For ex:- Getting Total Product sell by Amazon at 2.
{
"query": {
"bool": {
"must": [
{
"match_all": {
"boost": 1.0
}
}
]
}
},
"aggs": {
"patchFilter": {
"nested": {
"path": "resellers"
},
"aggs": {
"nestedfilter": {
"filter": {
"bool": {
"must":[
{
"term" :{
"resellers.name.keyword": {
"value": "Amazon"
}
}
},{
"terms" :{
"resellers.price":[2]
}
}
]
}
},
"aggs": {
"resellerprice": {
"reverse_nested" :{},
"aggs": {
"resellers_price":{
"cardinality" : {
"field" : "name.keyword"
}
}
}
}
}
}
}
}
}
}
I want to fetch it for multiple resellers(Amazon,Flipkart, Walmart) which are selling at 2 in single query. Can somebody help me out in doing that?
Mapping:
PUT productreseller
{
"mappings": {
"properties": {
"name": {
"type": "keyword"
},
"resellers": {
"type": "nested",
"properties": {
"name": {
"type": "text",
"fields":{
"keyword":{
"type": "keyword"
}
}
},
"price": {
"type": "double"
}
}
}
}
}
}
Data:
[
{
"_index" : "productreseller",
"_type" : "_doc",
"_id" : "JNbCLm0B00idyGV0Pn1Z",
"_score" : 1.0,
"_source" : {
"name" : "P2",
"resellers" : [
{
"name" : "amazon",
"price" : 3
},
{
"name" : "abc",
"price" : 2
}
]
}
},
{
"_index" : "productreseller",
"_type" : "_doc",
"_id" : "JdbCLm0B00idyGV0Wn0y",
"_score" : 1.0,
"_source" : {
"name" : "P1",
"resellers" : [
{
"name" : "amazon",
"price" : 2
},
{
"name" : "abc",
"price" : 3
}
]
}
},
{
"_index" : "productreseller",
"_type" : "_doc",
"_id" : "JtbPLm0B00idyGV0D32Y",
"_score" : 1.0,
"_source" : {
"name" : "P4",
"resellers" : [
{
"name" : "xyz",
"price" : 2
},
{
"name" : "abc",
"price" : 3
}
]
}
}
]
Query:
GET productreseller/_search
{
"query": {
"bool": {
"must": [
{
"match_all": {
"boost": 1
}
}
]
}
},
"aggs": {
"patchFilter": {
"nested": {
"path": "resellers"
},
"aggs": {
"nestedfilter": {
"filter": {
"bool": {
"must": [
{
"terms": {
"resellers.price": [
2
]
}
}
]
}
},
"aggs": {
"NAME": {
"terms": {
--->terms aggregation to list resellers and reverse_nested as subaggregation
"field": "resellers.name.keyword",
"size": 10
},
"aggs": {
"resellerprice": {
"reverse_nested": {},
"aggs": {
"resellers_price": {
"cardinality": {
"field": "name"
}
}
}
}
}
}
}
}
}
}
}
}
Result:
"aggregations" : {
"patchFilter" : {
"doc_count" : 8,
"nestedfilter" : {
"doc_count" : 3,
"NAME" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "abc",
"doc_count" : 1,
"resellerprice" : {
"doc_count" : 1,
"resellers_price" : {
"value" : 1
}
}
},
{
"key" : "amazon",
"doc_count" : 1,
"resellerprice" : {
"doc_count" : 1,
"resellers_price" : {
"value" : 1
}
}
},
{
"key" : "xyz",
"doc_count" : 1,
"resellerprice" : {
"doc_count" : 1,
"resellers_price" : {
"value" : 1
}
}
}
]
}
}
}
}
If you want to display only certain resellers, you can add terms query in nested filter
My requirement is if user has not performed e2 then that user will not come in result. For example here user_id 15371082 has performed e2 ,e3 but because user has performed e2 so both record of user(e2,e3) for userid 15371082 will discard from result
{
"id": 1,
"name": "a",
"user_id": 15371080,
"event" : 'e1'
},
{
"id": 1,
"name": "a",
"user_id": 15371082,
"event" : 'e2'
},
{
"id": 1,
"name": "a",
"user_id": 15371081,
"event" : 'e3'
},
{
"id": 1,
"name": "a",
"user_id": 15371082,
"event" : 'e3'
}
Expected result
{
"id": 1,
"name": "a",
"user_id": 15371080,
"event" : 'e1'
},
{
"id": 1,
"name": "a",
"user_id": 15371081,
"event" : 'e3'
}
My result should like above
Right way would be to create unique document for each user_id and add event as nested document.
Mapping:
PUT eventindex/_mappings
{
"properties": {
"name":{
"type": "text"
},
"user_id":{
"type": "integer"
},
"event":{
"type": "nested",
"properties": {
"name":{
"type":"text"
}
}
}
}
}
Data:
[
{
"_index" : "eventindex",
"_type" : "_doc",
"_id" : "tT1IL20Bcyz1xvxninMr",
"_score" : 1.0,
"_source" : {
"name" : "b",
"user_id" : 2,
"event" : [
{
"name" : "e1"
},
{
"name" : "e3"
}
]
}
},
{
"_index" : "eventindex",
"_type" : "_doc",
"_id" : "tj1ML20Bcyz1xvxngXNn",
"_score" : 1.0,
"_source" : {
"name" : "a",
"user_id" : 1,
"event" : [
{
"name" : "e2"
},
{
"name" : "e3"
}
]
}
}
]
````
Query:
````
GET eventindex/_search
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "event",
"query": {
"term": {
"event.name": {
"value": "e2"
}
}
}
}
}
]
}
}
}
````
Result:
````
[
{
"_index" : "eventindex",
"_type" : "_doc",
"_id" : "tT1IL20Bcyz1xvxninMr",
"_score" : 0.0,
"_source" : {
"name" : "b",
"user_id" : 2,
"event" : [
{
"name" : "e1"
},
{
"name" : "e3"
}
]
}
}
]
````