I have JSON Data
"hits": [
{
"_index": "Index",
"_type": "_doc",
"_id": "",
"_score": 1,
"_source": {
"supportdate": "20180631T000000Z",
}
},
{
"_index": "Index",
"_type": "_doc",
"_id": "",
"_score": 1,
"_source": {
"supportdate": "20190622T000000Z",
}
},
{
"_index": "Index",
"_type": "_doc",
"_id": "",
"_score": 1,
"_source": {
"supportdate": "20181231T000000Z",
}
}
]
Am trying to write aggregation query which will find records in particular "supportdate" range and categorise records as "Active and Expiring" buckets. Support date field has ISO date format(YYYYMMDDThhmmssZ).
As I have tried different date range queries, but no luck. As am new to elastic search database, bit struggling.
Appreciate any help!!
Related
my index key is stored as text but it is a nested json how can I update its values
Current Value of the key
{
"_index": "public_test",
"_type": "_doc",
"_id": "ec476fd0-c038-431a-be9a-4455cded91f8",
"_score": 1.0,
"_source": {
"user_id": "xxxxxxxxxx",
"lang_subscription": """[{"lang": "English", "is_subscribed": 1}]"""
}
}
But the value I want is
{
"_index": "public_test",
"_type": "_doc",
"_id": "ec476fd0-c038-431a-be9a-4455cded91f8",
"_score": 1.0,
"_source": {
"user_id": "xxxxxxxxxx",
"lang_subscription": [{ "lang": "English", "is_subscribed": 1 }]
}
}
I want to perform both exact word match and partial word/sub string match. For example, if I search for "test product" then I should be able to find "test" and "product" related text in the result. I'm searching Elasticsearch with the below match query, which is not giving me the exact match, instead its giving some more irrelevant match.
I'm using Elasticsearch 6.3
My query for GET /_search:
{
"must": {
"query_string": {
"query": "title:*test product*"
}
}
}
Search Result:
"hits": [
{
"_index": "67107104",
"_type": "_doc",
"_id": "1",
"_score": 0.6931471,
"_source": {
"title": "testing"
}
},
{
"_index": "67107104",
"_type": "_doc",
"_id": "2",
"_score": 0.6931471,
"_source": {
"title": "product good"
}
},
{
"_index": "67107104",
"_type": "_doc",
"_id": "3",
"_score": 0.6931471,
"_source": {
"title": "sample"
}
}
]
Expected Search Result:
"hits": [
{
"_index": "67107104",
"_type": "_doc",
"_id": "1",
"_score": 0.6931471,
"_source": {
"title": "testing"
}
},
{
"_index": "67107104",
"_type": "_doc",
"_id": "2",
"_score": 0.6931471,
"_source": {
"title": "product good"
}
}
]
In the search query above, you are searching in the review field, whereas in the search result you are getting data for title field
Adding a working example with index data, search query, and search result
Index Data:
{
"review": "testing"
}
{
"review": "product good"
}
{
"review": "sample"
}
Search Query:
{
"query": {
"match": {
"review": "test product"
}
}
}
Search Result:
"hits": [
{
"_index": "67119314",
"_type": "_doc",
"_id": "2",
"_score": 0.2876821,
"_source": {
"review": "product good"
}
}
]
Not getting the correct score for the elastic search query result.
ES Query -
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "(emergency) OR (emergency*) OR (*emergency) OR (*emergency*)",
"fields": [
"MDMGlobalData.Name1"
]
}
}
]
}
}
}
ES result -
{
"took": 29,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 798,
"relation": "eq"
},
"max_score": 9.169065,
"hits": [
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551037160",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "PARAGON EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551040507",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551076447",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "COASTAL EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551100746",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551090880",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "PAFFORD EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551106787",
"_score": 9.169065,
"_source": {
"MDMGlobalData": {
"Name1": "CAPROCK EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551021568",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "WILTON EMERGENCY"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551124137",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY ONE"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551125549",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY ONE"
}
}
},
{
"_index": "customermasterdata",
"_type": "_doc",
"_id": "MDMCM551133066",
"_score": 9.121077,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
}
]
}
}
Ideally, The first set in the result should be the Name1 which has value just "emergency" or start with the word "emergency"
And how could we have the same score for almost first 5 result sets? Being the Name1 value is different.
Due to wrong scoring, the results are messed up.
How to correct the score in the result?
No, That need not be the case. Because ES follows Lucene scoring function
Reason for the same score:
You have only two terms in each document - emergency and one more word
Emergency word matches as it is. Field Length is same
Number of occurrence is one. i.e Term frequencies are same.
Relevancy is same for all the terms. idf
Coord is same as your doc contains only one occurrence of Emergency
But if you have a document with Emergency X Y Z, then score of this will be lower than the other documents which you have. Because term frequency is higher for this one.
And if you have only Emergency, score of this document will be higher than all.
It is perfectly normal to have same score in your scenario as user doesn't know which emergency he/she meant.
Update:
{
"query":{
"bool":{
"must":{
"term":{
"MDMGlobalData.Name1":"emergency"
}
}
}
}
}
With the sample data, output:
"hits": [
{
"_index": "emerge",
"_type": "_doc",
"_id": "iN1hKnMBojxRtp6HNI7d",
"_score": 0.10938574,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "g91TKnMBojxRtp6Hto4q",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "PARAGON EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "hN1TKnMBojxRtp6H2I6A",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "hd1TKnMBojxRtp6H_I6_",
"_score": 0.08701137,
"_source": {
"MDMGlobalData": {
"Name1": "COASTAL EMERGENCY"
}
}
},
{
"_index": "emerge",
"_type": "_doc",
"_id": "h91VKnMBojxRtp6HYI4e",
"_score": 0.07223585,
"_source": {
"MDMGlobalData": {
"Name1": "EMERGENCY MD X"
}
}
}
]
In Elastic Search 5.1 I am making basic request with stored_fields body argument (new name for old fields argument) for retrieving the value of a specific field.
But my request give no field value in answer except _index, _type, _id and _score
I Give you sample for context:
I create index and mapping with:
PUT /base_well
{
"mappings": {
"person": {
"properties": {
"first_name":{
"type": "string"
},
"last_name":{
"type": "string"
},
"age":{
"type": "long"
}
}
}
}
}
I populate :
POST /base_well/person
{
"first_name":"James",
"last_name" : "Mopo",
"Age" : 21
}
POST /base_well/person
{
"first_name":"Polo",
"last_name" : "Rodriguez",
"Age" : 36
}
POST /base_well/person
{
"first_name":"Marc Aurelien",
"last_name" : "Poisson",
"Age" : 26
}
POST /base_well/person
{
"first_name":"Mustapha",
"last_name" : "Bulutu M'Bo",
"Age" : 47
}
I do my request with:
POST /base_well/person/_search
{
"stored_fields": ["first_name"]
}
And it give me an answere without the requested field fiest_person:
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 8,
"max_score": 1,
"hits": [
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFYzihcR_Z5VPUXUCL",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFiv3acR_Z5VPUXUCa",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFiwUKcR_Z5VPUXUCb",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFYx2LcR_Z5VPUXUCI",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFYyhScR_Z5VPUXUCJ",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFYzIJcR_Z5VPUXUCK",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFivgzcR_Z5VPUXUCZ",
"_score": 1
},
{
"_index": "base_well",
"_type": "person",
"_id": "AVlFiw2qcR_Z5VPUXUCc",
"_score": 1
}
]
}
}
Anybody could explain me to do it and how it works please?
By default, the document fields are not stored, i.e. in your mapping you don't specify store: true for each of them.
Hence, "stored_fields": ["first_name"] will not be able to return the first_name field since it's not stored.
You can use source filtering instead and specify "_source": ["first_name"] in your query, that will work.
Here is the query I am sending to ElasticSearch:
http://localhost:9200/user-index/user/_search/?queryb%5Bname%5D=Richard
The returned JSON is this:
{
"hits": [
{
"_index": "user-index",
"_type": "user",
"_id": "WgrvE-DzQJminNreBIsRNA",
"_score": 1.0,
"_source": {
"name": "Richard",
"db_id": "7"
}
},
{
"_index": "user-index",
"_type": "user",
"_id": "GwMOuYbUR8y48RrG4HgXdg",
"_score": 1.0,
"_source": {
"name": "John",
"db_id": "8"
}
},
{
"_index": "user-index",
"_type": "user",
"_id": "C-bgK3pNTNiX9Cz0x8EftA",
"_score": 1.0,
"_source": {
"name": "Harold",
"db_id": "2"
}
}
]
}
Only one of those actually matches. Why is it sending them all back?
Elasticsearch returns all records with the type user in the index user-index because it cannot find the search query. The search query should be specified as a query string in the parameter "q" or as a request body.
Try http://localhost:9200/user-index/user/_search?q=name%3ARichard