I'm trying to learn Elasticsearch and I have an index with 8 documents. When I run the request (GET):
{
"query": {
"match_all" : {}
}
}
The response is:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 8,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "el1",
"_type": "_doc",
"_id": "X5jGjXgB0p8Gz5z3KdFE",
"_score": 1.0,
"_source": {
"itemId": "01",
"curriculumId": "curriculum01",
"disciplineId": "discipline01"
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "YJjGjXgB0p8Gz5z3eNH4",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "02",
"curriculumId": "curriculum01",
"disciplineId": "discipline02"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "YZjGjXgB0p8Gz5z3jdHN",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "03",
"curriculumId": "curriculum02",
"disciplineId": "discipline01"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "YpjGjXgB0p8Gz5z3tdHl",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "04",
"curriculumId": "curriculum02",
"disciplineId": "discipline02"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "Y5jGjXgB0p8Gz5z3w9Gx",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "01",
"curriculumId": "curriculum01",
"disciplineId": "discipline01"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "ZJjGjXgB0p8Gz5z30NF5",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "02",
"curriculumId": "curriculum01",
"disciplineId": "discipline02"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "ZZjGjXgB0p8Gz5z33NHI",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "03",
"curriculumId": "curriculum02",
"disciplineId": "discipline01"
}
}
},
{
"_index": "el1",
"_type": "_doc",
"_id": "ZpjGjXgB0p8Gz5z36dE1",
"_score": 1.0,
"_source": {
"doc": {
"itemId": "03",
"curriculumId": "curriculum02",
"disciplineId": "discipline02"
}
}
}
]
}
}
It has all my documents. But if I run this other request (GET):
{
"query": {
"match" : {"disciplineId": "discipline01"}
}
}
It returns just the first document, and if I change it do "discipline02" it returns nothing.
The same happens to the other fields. What I'm doing wrong? I already tried using size:"10" but it doesn't seems to work (I think the default size is already 10).
I'm running this docker image and sending the requests with Insomnia.
Only the first document is in this format
{
"itemId": "01",
"curriculumId": "curriculum01",
"disciplineId": "discipline01"
}
Rest all document are in this format
{
"doc": {
"itemId": "02",
"curriculumId": "curriculum01",
"disciplineId": "discipline02"
}
}
Modify your query as
{
"query": {
"match": {
"doc.disciplineId": "discipline02" // note this
}
}
}
Related
Not getting the correct score for the elastic search query result.
ES Query -
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "(emergency) OR (emergency*) OR (*emergency) OR (*emergency*)",
"fields": [
"MDMGlobalData.Name1"
]
}
}
]
}
}
}
ES result -
{
"took": 29,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 798,
"relation": "eq"
},
"max_score": 9.169065,
"hits": [
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551037160",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "PARAGON EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551040507",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551076447",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "COASTAL EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551100746",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551090880",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "PAFFORD EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551106787",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "CAPROCK EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551021568",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "WILTON EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551124137",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY ONE"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551125549",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY ONE"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551133066",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
}
]
}
}
Ideally, The first set in the result should be the Name1 which has value just "emergency" or start with the word "emergency"
And how could we have the same score for almost first 5 result sets? Being the Name1 value is different.
Due to wrong scoring, the results are messed up.
How to correct the score in the result?
No, That need not be the case. Because ES follows Lucene scoring function
Reason for the same score:
You have only two terms in each document - emergency and one more word
Emergency word matches as it is. Field Length is same
Number of occurrence is one. i.e Term frequencies are same.
Relevancy is same for all the terms. idf
Coord is same as your doc contains only one occurrence of Emergency
But if you have a document with Emergency X Y Z, then score of this will be lower than the other documents which you have. Because term frequency is higher for this one.
And if you have only Emergency, score of this document will be higher than all.
It is perfectly normal to have same score in your scenario as user doesn't know which emergency he/she meant.
Update:
{
"query":{
"bool":{
"must":{
"term":{
"MDMGlobalData.Name1":"emergency"
}
}
}
}
}
With the sample data, output:
"hits": [
{
"_index": "emerge",
"_type": "_doc",
"_id": "iN1hKnMBojxRtp6HNI7d",
"_score": 0.10938574,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "g91TKnMBojxRtp6Hto4q",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "PARAGON EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "hN1TKnMBojxRtp6H2I6A",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "hd1TKnMBojxRtp6H_I6_",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "COASTAL EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "h91VKnMBojxRtp6HYI4e",
"_score": 0.07223585,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD X"
}
}
}
]
Is there a way to query starting from a particular value and get the next n records in Elasticsearch?
For example, I want to get 10 records starting from employee id "ABC_123".
The below query gives an error saying
[terms] query does not support [empId]
GET /_search
{
"from": 0, "size": 10,
"query" : {
"terms" : {
"empId" : "ABC_123"
}
}
}
What can I do about this?
You can use the prefix query, Also you can read more about the autocomplete on my blog, which discussed 4 approaches to make it work and their trade-off.
I used prefix query on your sample data and got the expected output and below is the step by step guide.
Index mapping
{
"mappings": {
"properties": {
"empId": {
"type": "keyword" --> field type `keyword`
}
}
}
}
Index sample docs
{
"empId" : "ABC_1231"
}
{
"empId" : "ABC_1232"
}
{
"empId" : "ABC_1233"
}
{
"empId" : "ABC_1234"
}
and so on
Prefix Search query
{
"from": 0,
"size": 10,
"query": {
"prefix": {
"empId": "ABC_123"
}
}
}
Search result
"hits": [
{
"_index": "so_prefix",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"empId": "ABC_1231"
}
},
{
"_index": "so_prefix",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"empId": "ABC_1232"
}
},
{
"_index": "so_prefix",
"_type": "_doc",
"_id": "3",
"_score": 1.0,
"_source": {
"empId": "ABC_1233"
}
},
{
"_index": "so_prefix",
"_type": "_doc",
"_id": "4",
"_score": 1.0,
"_source": {
"empId": "ABC_1234"
}
}
]
I have written the following lucene query in elasticsearch for getting documents with Id field as mentioned:
GET requirements_v3/_search
{
"from": 0,
"size": 10,
"query": {
"bool": {
"filter": {
"bool": {
"should": [
{"match": {
"Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
}},
{
"match": {
"Id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69"
}
},
{
"match": {
"Id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa"
}
},
{
"match": {
"Id": "0aa1db52-c0fb-4bf6-9223-00edccc32703"
}
},
{
"match": {
"Id": "8c399993-f273-4ee0-a1ab-3a85c6848113"
}
},
{
"match": {
"Id": "4461eb37-487e-4899-a7be-914640fab0e0"
}
},
{
"match": {
"Id": "07052261-b904-4bfc-a6fd-3acd28114c6a"
}
},
{
"match": {
"Id": "95816ff0-9eae-4196-99fc-86c6f43395fd"
}
},
{
"match": {
"Id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a"
}
},
{
"match": {
"Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f"
}
}
]
}
}
}
}
The response for the above query is:
{
"took": 14,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 18,
"max_score": 0,
"hits": [
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
"_score": 0,
"_source": {
"Id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "4461eb37-487e-4899-a7be-914640fab0e0",
"_score": 0,
"_source": {
"Id": "4461eb37-487e-4899-a7be-914640fab0e0",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
"_score": 0,
"_source": {
"Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
"Name": "Create Configurator"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "d75d9a7c-e145-487e-922f-102c16d0026f",
"_score": 0,
"_source": {
"Id": "d75d9a7c-e145-487e-922f-102c16d0026f",
"Name": "Create Configurator"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
"_score": 0,
"_source": {
"Id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
"Name": "Detail Page - Build"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
"_score": 0,
"_source": {
"Id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
"_score": 0,
"_source": {
"Id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
"Name": "HUC"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
"_score": 0,
"_source": {
"Id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
"Name": "DAMS UpsertUnenrollPrice" }
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
"_score": 0,
"_source": {
"Id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
"Name": "Item Setup"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
"_score": 0,
"_source": {
"Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
"Name": "Installments"
}
}
]
}
}
This mentions totalHits as '18'. Why is it returning more items than 10? I believe match query should be used for 'exact' matches, so why more documents are returned here?
P.S.: I know I can use the Ids query for this, but I want to know why is this not returning the correct response
Update: Setting the size to 20 returns the following response:
{
"took": 195,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 18,
"max_score": 0,
"hits": [
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
"_score": 0,
"_source": {
"Id": "9d8060da-c3e2-4f6d-b4e2-17e65b266c76",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "4461eb37-487e-4899-a7be-914640fab0e0",
"_score": 0,
"_source": {
"Id": "4461eb37-487e-4899-a7be-914640fab0e0",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
"_score": 0,
"_source": {
"Id": "33f87d98-024f-4893-aa1c-8d438a98cd1f",
"Name": "Create Configurator"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "d75d9a7c-e145-487e-922f-102c16d0026f",
"_score": 0,
"_source": {
"Id": "d75d9a7c-e145-487e-922f-102c16d0026f",
"Name": "Create Configurator"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
"_score": 0,
"_source": {
"Id": "007eadb7-adda-487e-b7fe-6f6b5648de2e",
"Name": "Detail Page - Build"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
"_score": 0,
"_source": {
"Id": "95816ff0-9eae-4196-99fc-86c6f43395fd",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
"_score": 0,
"_source": {
"Id": "07052261-b904-4bfc-a6fd-3acd28114c6a",
"Name": "HUC"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
"_score": 0,
"_source": {
"Id": "d60daf3a-4681-4bfc-a3a9-b04b5b005f73",
"Name": "DAMS UpsertUnenrollPrice"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
"_score": 0,
"_source": {
"Id": "c1b367f2-a57a-487e-994c-84470e0f9db4",
"Name": "Item Setup"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
"_score": 0,
"_source": {
"Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b",
"Name": "Installments"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "b9437079-47c4-487e-abf0-1ff076f69e0f",
"_score": 0,
"_source": {
"Id": "b9437079-47c4-487e-abf0-1ff076f69e0f",
"Name": "Detail Page - Strings "
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "0aa1db52-c0fb-4bf6-9223-00edccc32703",
"_score": 0,
"_source": {
"Id": "0aa1db52-c0fb-4bf6-9223-00edccc32703",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a",
"_score": 0,
"_source": {
"Id": "ea8a59a6-2b2f-467a-9beb-e281b1581a0a",
"Name": "Create Configurator"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "fd259359-4f6d-4530-ac29-fcebe00d66a6",
"_score": 0,
"_source": {
"Id": "fd259359-4f6d-4530-ac29-fcebe00d66a6",
"Name": "Invite Platform"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "1b2ba0bb-3e7f-46fb-b904-07460b84848b",
"_score": 0,
"_source": {
"Id": "1b2ba0bb-3e7f-46fb-b904-07460b84848b",
"Name": "Training"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "8c399993-f273-4ee0-a1ab-3a85c6848113",
"_score": 0,
"_source": {
"Id": "8c399993-f273-4ee0-a1ab-3a85c6848113",
"Name": "Configure ASIN for Reporting"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa",
"_score": 0,
"_source": {
"Id": "3b385896-1207-4f6d-8ae9-f3ced84cf1fa",
"Name": "Create Extended/Limited Warranty Configuration"
}
},
{
"_index": "requirements_v3",
"_type": "_doc",
"_id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69",
"_score": 0,
"_source": {
"Id": "048b7907-2b5a-438a-ace9-f1e1fd67ca69",
"Name": "Invite Platform"
}
}
]
}
}
Lets understand this by the following mapping e.g:
{
"_doc": {
"properties": {
"Id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"Name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
The above mapping is created dynamically by elasticsearch. Lets us now focus on Id field. Its type is text. By default the analyzer for text datatype is standard analyzer. When this analyzer is applied on the input for this field it get tokenized into terms. So for example if you input value for Id is 33f87d98-024f-4893-aa1c-8d438a98cd1f following tokens get generated:
33f87d98
024f
4893
aa1c
8d438a98cd1f
As you can see the input value is splitted by - being used as delimiter. This is because standard analyzer is applied on it.
There is another sub-field under Id which is keyword and its type is keyword. For type keyword the input is indexed as it is without applying any modification.
Now lets understand why more documents get matched and result count is more than expected. In your query you used match query on Id field as below:
{
"match": {
"Id": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
}
}
By default match query uses the same analyzer that is applied on the field in mapping. So on the Id value in the query again the same analyzer is applied and the input is splitted into tokens in a similar way as above. The default operator that is applied between tokens of match query input string is OR and hence your query actually becomes:
b8bf49a4 OR 960b OR 4fa8 OR 8c5f OR a3fce4b4d07b
There if any of the above tokens match to any of the indexed terms stored in Id field, the document is considered a match.
Solution for the above based on above mapping:
Use the keyword field instead. So the query becomes:
{
"match": {
"Id.keyword": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
}
}
More on how match works see here.
Also as mention by #Curious_MInd in his answer its better to use terms than using multiple match in should.
As you said that your Id is text as well as keyword so you should use Id.keyword for matching exact values like
GET requirements_v3/_search
{
"from": 0,
"size": 10,
"query": {
"bool": {
"filter": {
"bool": {
"should": [
{"match": {
"Id.keyword": "b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b"
}},
{
"match": {
"Id.keyword": "048b7907-2b5a-438a-ace9-f1e1fd67ca69"
}
}
]
}
}
}
}
But I guess you should use terms if you wants to match multiple exact values. Have a look here. For an example:
{
"terms" : {
"Id" : ["b8bf49a4-960b-4fa8-8c5f-a3fce4b4d07b", "048b7907-2b5a-438a-ace9-f1e1fd67ca69"]
}
}
I currently have a dataset that features a nested datatype in products, these are all listed within different vendors. I have various queries that check for search terms within the nested products array, ideally I want to be able to combine all the inner hits so that I can sort on such things as score rankings and price. At the moment the search results come back on a per document basis. Is it possible to combine inner hits in elasticsearch so that I get just a list of all the matching products?
Example Query
{
"_source": {
"includes": [ "*" ],
"excludes": [ "products" ]
},
"query": {
"nested": {
"path": "products",
"inner_hits": {
"size": 10,
"_source": [
"title"
]
},
"query": {
"bool": {
"must": [
{
"match": {
"products.title" : {
"query": "Dress",
"fuzziness" : 0
}
}
}
]
}
}
}
}
}
Example output
{
"took": 477,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 2.9072125,
"hits": [
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a806c7af36d28314de953ff",
"_score": 2.9072125,
"_source": {
"name": "Argos",
"locations": [
{
"lon": -2.242797,
"lat": 53.482952
}
]
},
"inner_hits": {
"products": {
"hits": {
"total": 3,
"max_score": 3.0782251,
"hits": [
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a806c7af36d28314de953ff",
"_nested": {
"field": "products",
"offset": 3348
},
"_score": 3.0782251,
"_source": {
"title": "HOME Set of 2 Dress Covers - White"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a806c7af36d28314de953ff",
"_nested": {
"field": "products",
"offset": 2599
},
"_score": 3.0782251,
"_source": {
"title": "Chad Valley Designabear Spotty Dress Outfit"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a806c7af36d28314de953ff",
"_nested": {
"field": "products",
"offset": 771
},
"_score": 2.5651875,
"_source": {
"title": "Melissa and Doug Abby & Emma Magnetic Wooden Dress Up"
}
}
]
}
}
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_score": 2.3227787,
"_source": {
"name": "Superdry",
"locations": [
{
"lon": -2.241703,
"lat": 53.483469
}
]
},
"inner_hits": {
"products": {
"hits": {
"total": 186,
"max_score": 2.378731,
"hits": [
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6420
},
"_score": 2.378731,
"_source": {
"title": "Alexia Off Shoulder Dress"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6417
},
"_score": 2.378731,
"_source": {
"title": "Erin Festival Skater Dress "
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6416
},
"_score": 2.378731,
"_source": {
"title": "Erin Racer Dress "
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6415
},
"_score": 2.378731,
"_source": {
"title": "Alice Knot Dress"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6412
},
"_score": 2.378731,
"_source": {
"title": "Alice Knot Dress"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6389
},
"_score": 2.378731,
"_source": {
"title": "Lagoon Logo Midi Dress"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6388
},
"_score": 2.378731,
"_source": {
"title": "50's Boardwalk Dress "
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6386
},
"_score": 2.378731,
"_source": {
"title": "50's Boardwalk Dress "
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6385
},
"_score": 2.378731,
"_source": {
"title": "Graphic Sweat Dress"
}
},
{
"_index": "shopit",
"_type": "businesses",
"_id": "5a5c3beb734d1d3471839b1d",
"_nested": {
"field": "products",
"offset": 6382
},
"_score": 2.378731,
"_source": {
"title": "Breton Bardot Stripe Dress"
}
}
]
}
}
}
}
]
}
}
Nevermind, I should of paid better attention to the elasticsearch documentation which states:
Search requests return the whole document, not just the matching
nested documents. Although there are plans afoot to support returning
the best -matching nested documents with the root document, this is
not yet supported.
I think parent-child relationships are probably the way to go with this.
Using the following date range query includes results that are outside of the date range we wish to filter by.
{
"query" : {
"range" : {
"account.expiry" : {
"gte" : "2016-03-03T00:00:00.000Z",
"lt" : "2017-03-02T00:00:00.000Z"
}
}
},
"_source" : {
"includes" : [ "account.expiry"],
"excludes" : [ ]
}
}
I have tried various different variations (using 'to' and 'from' instead of 'gt' and 'lte').
I've checked the mappings of the index and they look OK to me (Elastic doesn't see the field as a string). Here's the relevant part:
{
"myindex.dev": {
"aliases": {},
"mappings": {
"account": {
"properties": {
"account": {
"properties": {
"expiry": {
"type": "date",
"format": "dateOptionalTime"
}
}
}
}
}
}
}
}
I've written a unit test that spins up an ES locally and populates it with data, and this doesn't seem to have the same problem. I've checked and they're both using the same version of ElasticSearch. Running the test from a different JRE doesn't seem to make a difference.
I'd appreciate any help you can give!
Edit:
These are the results I am getting from this query. I'm working on steps to reproduce but it seems to be behaving when running on my own machine
(
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 13234,
"max_score": 1,
"hits": [
{
"_index": "accounts.dev",
"_type": "account",
"_id": "4328398",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-09-30T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "7504549",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-09-30T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "8333546",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-09-30T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "12693814",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-09-30T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "32583326",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-09-30T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "66828717",
"_score": 1,
"_source": {
"account": {
"expiry": "2013-09-07T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "66905991",
"_score": 1,
"_source": {
"account": {
"expiry": "2013-11-25T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "12843517",
"_score": 1,
"_source": {
"account": {
"expiry": "2016-02-23T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "43597281",
"_score": 1,
"_source": {
"account": {
"expiry": "2013-06-12T00:00:00Z"
}
}
},
{
"_index": "accounts.dev",
"_type": "account",
"_id": "43597291",
"_score": 1,
"_source": {
"account": {
"expiry": "2012-12-31T00:00:00Z"
}
}
}
]
}
}
Edit 2:
In response to the comment, yes actually there is a nested expiry field in the source document too... it looks like this.
"account": {
"permissionSets": [
{
"expiry": "2013-10-29T00:00:00Z"
},
{
"expiry": "2016-09-30T00:00:00Z"
},
{
"expiry": "2012-09-30T00:00:00Z"
},
{
"expiry": "2012-09-30T00:00:00Z"
}
],
"expiry": "2012-09-30T00:00:00Z"