suppose I have two indexed documents in my elasticsearch cluster --
{"APPLE": "TEST"}
{"TEST": "APPLE"}
Suppose I want to search on an APPLE term. What query would return to me both of these documents?
You can use match query to search for documents having APPLE as the field value, and exists query to search for documents having field APPLE. Try out this below query
{
"query": {
"bool": {
"should": [
{
"match": {
"TEST": "APPLE"
}
},
{
"exists": {
"field": "APPLE"
}
}
]
}
}
}
Search Result will be
"hits": [
{
"_index": "67048809",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"APPLE": "TEST"
}
},
{
"_index": "67048809",
"_type": "_doc",
"_id": "2",
"_score": 0.2876821,
"_source": {
"TEST": "APPLE"
}
}
]
Update 1:
You can query the field's existence with wildcard characters also.
{
"query": {
"bool": {
"should": [
{
"match": {
"TEST": "APPLE"
}
},
{
"exists": {
"field": "*APPLE*" // note this
}
}
]
}
}
}
Related
I have the following query over a string field:
const query = {
"query": {
"match_phrase": {
name: "ron"
}
},
"from": 0,
"size": 10
};
these are the names I have in the database
1. "ron"
2. "ron martin"
3. "ron ron"
4. "ron howard"
the result of this query is very fuzzy, all the of the rows are returned instead of row number 1 only.
It's like it is performing "contains" instead of "equals".
Thanks
In your case, all the documents are returning, because all the documents have ron in them.
If you want that only the exact field should match, then you need to add keyword subfield to the name field. This uses the keyword analyzer instead of the standard analyzer (notice the ".keyword" after name field). Try out this below query -
Index Mapping:
{
"mappings": {
"properties": {
"name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
Index Data:
{
"name":"ron"
}
{
"name":"ron martin"
}
{
"name":"ron ron"
}
{
"name":"ron howard"
}
{
"name": "john howard"
}
Search Query:
{
"query": {
"match_phrase": {
"name.keyword": "ron"
}
},
"from": 0,
"size": 10
}
Search Result:
"hits": [
{
"_index": "64982377",
"_type": "_doc",
"_id": "1",
"_score": 1.2039728,
"_source": {
"name": "ron"
}
}
]
Update 1:
Based on the comments below, if you want to search for both exact match and fuzzy match (according to your requirement), then you can use multi_match query.
Search Query:
{
"query": {
"multi_match": {
"query": "howard",
"fields": [
"name",
"name.keyword"
],
"type": "phrase"
}
}
}
Search Result:
"hits": [
{
"_index": "64982377",
"_type": "_doc",
"_id": "4",
"_score": 0.83740485,
"_source": {
"name": "ron howard"
}
},
{
"_index": "64982377",
"_type": "_doc",
"_id": "5",
"_score": 0.83740485,
"_source": {
"name": "john howard"
}
}
]
Is there a way in Elastic Search to weight results base on an attribute other than the one used for the search query. For example, we search the field 'name', but all documents that have 'with_pictures' attributed to true weighted higher.
You can use boost on individual fields, that will be boosted automatically — count more towards the relevance score — at query time, with the boost parameter
Adding working example with index data, mapping and search query
Index mapping:
{
"mappings": {
"properties": {
"with_pictures": {
"type": "boolean",
"boost": 2
},
"name": {
"type": "keyword"
}
}
}
}
Index data:
{
"name": "A",
"with_pictures": false
}
{
"name": "A",
"with_pictures": true
}
{
"name": "B",
"with_pictures": true
}
Search Query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"should": [
{
"term": {
"name": "A"
}
},
{
"term": {
"with_pictures": true
}
}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "fd_cb1",
"_type": "_doc",
"_id": "1",
"_score": 1.4100108,
"_source": {
"name": "A",
"with_pictures": true
}
},
{
"_index": "fd_cb1",
"_type": "_doc",
"_id": "3",
"_score": 0.9400072,
"_source": {
"name": "B",
"with_pictures": true
}
},
{
"_index": "fd_cb1",
"_type": "_doc",
"_id": "2",
"_score": 0.4700036,
"_source": {
"name": "A",
"with_pictures": false
}
}
]
Score of documents satisfying both the conditions of name and with_properties is having the highest score. But the document having name: B and with_pictures: true have higher score than name: A and with_pictures: false( because of the boost applied on the with_pictures
You can also refer function score query that allows you to modify the score of documents that are retrieved by a query.
I am trying to OR together two clauses in a single query but I am seeing some strange behavior and am looking for guidance
Given an index "index1" with two documents -
{"column1": "A"} with _id=1
{"column1": "B"} with _id=2
When run a POST index1/_search with following body -
{"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{"term": {"_id": "1"}},
{"term": {"column1": "A"}}
]
}
},
{
"bool": {
"must": [
{"term": {"_id": "2"}},
{"term": {"column1": "B"}}
]
}
}
]
}
}}
I get no results. What I was expecting was both documents to be returned. If I remove one of the column1 term matches, I will get the other document. If I remove both column1 term matches, I get both documents?
What am I doing wrong?
As #Val already suggested, you should have used column1.keword, if you are taking default mapping of column1
Search Query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"must": [
{
"term": {
"_id": "1"
}
},
{
"term": {
"column1.keyword": "A"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_id": "2"
}
},
{
"term": {
"column1.keyword": "B"
}
}
]
}
}
]
}
}
}
Search Result:
"hits": [
{
"_index": "fd_cb",
"_type": "_doc",
"_id": "1",
"_score": 1.6931472,
"_source": {
"column1": "A"
}
},
{
"_index": "fd_cb",
"_type": "_doc",
"_id": "2",
"_score": 1.6931472,
"_source": {
"column1": "B"
}
}
]
If you have already defined explicit mapping for column1, like the one shown below :
{
"mappings": {
"properties": {
"column1": {
"type": "keyword"
}
}
}
}
Then the search query, mentioned in your question works perfectly fine.
I want to query IP range from:172.16.0.0 to 172.31.0.0 in ELK
I try two query methods, but fail.
{
"query": {
"bool": {
"should": [
{
"regexp": {
"DstIP": "172.(3[0-1]|1[6-9]|2[0-9]).*"
}
}
],
"minimum_should_match": 1
}
}
}
{
"query": {
"range": {
"DstIP": {
"gte": "172.16.0.0",
"lte": "172.31.0.0"
}
}
}
}
How can query IP range in ELK?
For range queries to work correctly on IP values it is necessary to define the field data type as ip.
Below is the working example with mapping, sample docs, and search query.
Mapping:
{
"mappings": {
"properties": {
"dest": {
"type": "ip"
}
}
}
}
Index data:
Then I've taken a couple of sample documents like this:
{ "dest":"172.16.0.0"}
{ "dest":"172.31.0.0"}
{ "dest":"172.21.0.0"}
{ "dest":"172.1.0.0" }
{ "dest":"172.12.0.0"}
Search Query :
{
"query": {
"range": {
"dest": {
"gte": "172.16.0.0",
"lte": "172.31.0.0"
}
}
}
}
Search Result :
"hits": [
{
"_index": "foo4",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"dest": "172.16.0.0"
}
},
{
"_index": "foo4",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"dest": "172.31.0.0"
}
},
{
"_index": "foo4",
"_type": "_doc",
"_id": "3",
"_score": 1.0,
"_source": {
"dest": "172.21.0.0"
}
}
]
We have one document in elastic search with multiple sections of name/value pair and we want to fetch value's only based on name column value.
"envelopeData": {
"envelopeName": "Bills",
"details": {
"detail": [
{
"name": "UC_CORP",
"value": "76483"
},
{
"name": "UC_CYCLE",
"value": "V"
}
We are expecting only 76483 as result based on name equals to UC_CORP
If the field envelopeData.details.detail is nested type then you can perform a match query for the desired name on the nested path and can use inner_hits to get just the value.
Map the field envelopeData.details.detail as nested(if not nested):
PUT stackoverflow
{
"mappings": {
"_doc": {
"properties": {
"envelopeData.details.detail": {
"type": "nested"
}
}
}
}
}
then you can perform the following query to get value using inner_hits:
GET stackoverflow/_search
{
"_source": "false",
"query": {
"nested": {
"path": "envelopeData.details.detail",
"query": {
"match": {
"envelopeData.details.detail.name.keyword": "UC_CORP"
}
},
"inner_hits": {
"_source": "envelopeData.details.detail.value"
}
}
}
}
which outputs:
{
"_index": "stackoverflow",
"_type": "_doc",
"_id": "W5GUW2gB3GnGVyg-Sf4T",
"_score": 0.6931472,
"_source": {},
"inner_hits": {
"envelopeData.details.detail": {
"hits": {
"total": 1,
"max_score": 0.6931472,
"hits": [
{
"_index": "stackoverflow",
"_type": "_doc",
"_id": "W5GUW2gB3GnGVyg-Sf4T",
"_nested": {
"field": "envelopeData.details.detail",
"offset": 0
},
"_score": 0.6931472,
"_source": {
"value": "76483" -> Outputs value only
}
}
]
}
}
}
}