I've been facing some issues with multi field elasticsearch query. I am trying to query all the documents which matches the field called func_name to two hard coded strings, even though my index has documents with both these function names, but the query result is always fetching only one func_name. So far I have tried following queries.
1) Following returns only one function match, even though the documents have another function as well
GET /_search
{
"query": {
"multi_match": {
"query": "FEM_DS_GetTunerStatusInfo MDM_TunerStatusPrint",
"operator": "OR",
"fields": [
"func_name"
]
}
}
}
2) following intermittently gives me both the functions.
GET /_search
{
"query": {
"match": {
"func_name": {
"query": "MDM_TunerStatusPrint FEM_DS_GetTunerStatusInfo",
"operator": "or"
}
}
}
}
3) Following returns only one function match, even though the documents have another function as well
{
"query": {
"bool": {
"should": [
{ "match": { "func_name": "FEM_DS_GetTunerStatusInfo" }},
{ "match": { "func_name": "MDM_TunerStatusPrint" }}
]
}
}
}
Any help is much appreciated.
Thanks for your reply. Lets assume that I have following kind of documents in my elasticsearch. I want my search to return first two documents out of all as they matches my func_name.
{
"_index": "diag-178999",
"_source": {
"severity": "MIL",
"t_id": "03468500",
"p_id": "000007c6",
"func_name": "MDM_TunerStatusPrint",
"timestamp": "2017-06-01T02:04:51.000Z"
}
},
{
"_index": "diag-344563",
"_source": {
"t_id": "03468500",
"p_id": "000007c6",
"func_name": "FEM_DS_GetTunerStatusInfo",
"timestamp": "2017-07-20T02:04:51.000Z"
}
},
{
"_index": "diag-101010",
"_source": {
"severity": "MIL",
"t_id": "03468500",
"p_id": "000007c6",
"func_name": "some_func",
"timestamp": "2017-09-15T02:04:51.000Z"
}
The "two best ways" to request your ES is to filter by terms on a particular field or to aggregate your queries so that you can rename the field, apply multiple rules, and give a more understandable format to your response
See : https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html and the other doc page is here, very useful :
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
In your case, you should do :
{
"from" : 0, "size" : 2,
"query": {
"filter": {
"bool": {
"must": {
"term": {
"func_name" : "FEM_DS_GetTunerStatusInfo OR MDM_TunerStatusPrint",
}
}
}
}
}
}
OR
"aggs": {
"aggregationName": {
"terms": {
"func_name" : "FEM_DS_GetTunerStatusInfo OR MDM_TunerStatusPrint"
}
}
}
}
The aggregation at the end is just here to show you how to do the same thing as your query filter. Let me know if it's working :)
Best regards
As I understand, you should use filtered query to match any document with one of the values of func_name mentioned above:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"terms": {
"func_name": [
"FEM_DS_GetTunerStatusInfo",
"MDM_TunerStatusPrint"
]
}
}
]
}
}
}
}
}
See:
Filtered Query, Temrs Query
UPDATE in ES 5.0:
{
"query": {
"bool": {
"must": [
{
"terms": {
"func_name": [
"FEM_DS_GetTunerStatusInfo",
"MDM_TunerStatusPrint"
]
}
}
]
}
}
}
See: this answer
Related
I have following simple code snippet. I want to query the documents that contains value field and also the name is hello,
PUT /lib59/_doc/1
{
"name": "hello",
"value":1
}
GET /lib59/_search
{
"query":{
"term": {
"name":"hello"
},
"exists": {
"field": "value"
}
}
}
But when I run it, an error occurs, the error message is:
I would ask whether ES supports the query as I wrote above, if not, how could I express my query? Thanks.
You need to combine your query using bool/must clause
{
"query": {
"bool": {
"must": [
{
"term": {
"name": "hello"
}
},
{
"exists": {
"field": "value"
}
}
]
}
}
}
Using the following document, I'm trying to perform an Elasticsearch keyword query, conditionally excluding field data from the scope of the search. Is this possible?
{
"Name":"doc1",
"UserData":[
{
"EnteredBy":"Eric",
"Description":"Desc entered by Eric, abc"
},
{
"EnteredBy":"Alex",
"Description":"Desc entered by Alex, def"
}
]
}
The Elasticsearch query I need will allow me to search across the whole document, except it should exclude from the search UserData items where EnteredBy does not match the specified user.
The following queries would return results:
User:Eric doc1
User:Eric abc
User:Alex doc1
User:Fred doc1
The following queries would not return results:
User:Eric def
User:Fred def
Everything I've tried thus far, ends up filtering content based on the presence of UserData nodes which apply to the specified user. I can't think of a way to specify that a field should be searched, only if the EnteredBy field matches.
I could restructure the document, if that would solve the problem.
Edit 1
The index..
PUT index1
{
"settings": {
"number_of_shards": 2,
"number_of_replicas": 0
},
"mappings": {
"properties" : {
"UserData" : {
"type":"nested"
},
"Name": {
"type":"text"
}
}
}
}
Edit 2
The query below is providing the results that I need, except for the child entity, I have to search in a specific field. If I change the second condition of the nested search into a query_string search, then it no longer uses the EnteredBy condition.
GET index1/_search
{
"query": {
"bool": {
"should": [
{
"nested":
{
"path": "UserData",
"query": {
"bool": {
"must": [{
"match": {
"UserData.EnteredBy": "Eric"
}},
{
"match": {
"UserData.Description": "def"
}
}]
}
}
}
},
{
"query_string":
{
"query": "doc1x"
}
}
]
}
}
}
This query appears to be working. I think I answered my own question.
GET index1/_search
{
"query": {
"bool": {
"should": [
{
"nested":
{
"path": "UserData",
"query": {
"bool": {
"must": [{
"match": {
"UserData.EnteredBy": "Eric"
}},
{
"query_string": {
"query": "def"
}
}]
}
}
}
},
{
"query_string":
{
"query": "doc1"
}
}
]
}
}
}
How can I take the raw JSON String and query via the Java API? It should
work for any complex query that Curl accepts in Elastic.
For example:
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"name": "<name>"
}
},
{
"match": {
"address": {
"query": "<address>",
"fuzziness": 1,
"prefix_length": 1,
"operator": "or",
"minimum_should_match": "80%"
}
}
},
{
"match_phrase": {
"city_nm": "<city_nm>"
}
},
{
"term": {
"state_province_cd": "<state_province_cd>"
}
}
]
}
}
}
I tried Query Builders' simpleQueryStringQuery method to achieve this but it creates the query which doesn't give a correct result.
For Example, to search a record using just name, this is the query which simpleQueryStringQuery is generating but instead of returning just one record it returns multiple records.
{
"query": {
"simple_query_string": {
"query": """{"query":{"bool":{"must":[{"match_phrase":{"name":"Neeraj"}}]}}}"""
}
}
}
whereas if I run the same query like this in my Kibana console that returns the correct number of results.
{
"query": {
"bool": {
"must": [
{
"match_phrase": {
"name": "Jaas"
}
}
]
}
}
}
I have data in the following format:
{ "_id":1,
"s_id":121211,
"data_detail":{
"name":"John",
"phone_number":08089320xxx,
"city":"ABC"
}
}
I need to search data through elastic search which will query where s_id=? and any text which is available in data_detail object. Example s_id=121211 AND ABC. I need wildcard on data_detail object.
Keys for the data_detail object is not fixed.
Thanks in advance.
I would consider using a bool query with multi_match and term query like this. I haven't tested this, but something on these lines should work I guess.
GET test_index/_search
{
"query": {
"nested": {
"path": "data_detail",
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "ABC",
"fields": [
"data_detail.*"
]
}
},
{
"term": {
"s_id": {
"value": "121211"
}
}
}
]
}
}
}
}
}
Solved this by using the following query:
{
"query": {
"bool": {
"must": [
{
"query_string":{
"fields":["data_detail.*"],
"query": "*str*",
"analyze_wildcard":true
}
},
{
"term": {
"s_id": {
"value": "121211"
}
}
}
]
}
}
}
I am able to get data for the following elastic search query :
{
"query": {
"filtered": {
"query": [],
"filter": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"term": {
"gender": "malE"
}
},
{
"term": {
"sentiment": "positive"
}
}
]
}
}
]
}
}
}
}
}
However, If I query using "match" - I get error message with 400 status response
{
"query": {
"filtered": {
"query": [],
"filter": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"match": {
"gender": "malE"
}
},
{
"term": {
"sentiment": "positive"
}
}
]
}
}
]
}
}
}
}
}
Is match query not supported in nested bool filters ?
Since the term query looks for the exact term in the field’s inverted index and I want to query gender data as case_insensitive field - Which approach shall I try ?
Settings of the index :
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"analyzer_keyword": {
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}
}
Mapping for field Gender:
{"type":"string","analyzer":"analyzer_keyword"}
The reason you're getting an error 400 is because there is no match filter, only match queries, even though there are both term queries and term filters.
Your query can be as simple as this, i.e. no need for a filtered query, simply put your term and match queries into a bool/should:
{
"query": {
"bool": {
"should": [
{
"match": {
"gender": "male"
}
},
{
"term": {
"sentiment": "positive"
}
}
]
}
}
}
This answer is for ElasticSearch 7.x. As I understand from the question, you would like to use a match query for the gender field and a term query for the sentiment field. The mappings for each of these field should look like below:
"sentiment": {
"type": "keyword"
},
"gender": {
"type": "text"
}
The corresponding search API would be:
"query": {
"bool": {
"must": [
{
"terms": {
"sentiment": [
"very positive", "positive"
]
}
},
{
"match": {
"gender": "malE"
}
}
]
}
}
This search API returns all the documents where gender is "Male"/"MALE"/"mALe" etc. So, you may have indexed the gender field holding "mALe", but, the match query for "gender": "malE" will still be able to retrieve it. In the latest version of ElasticSearch, if the query is a match type, the value (which is "gender": "malE") will be automatically lower cased internally before search begins. But, it should not be that tough for a client of the API to pass a lowercase to the match query at the onset itself. Coming to the sentiment field, since, its a keyword field, you can search for values that contain spaces too like very positive.