I have some documents in a index..
"hits" : [
{
"_index" : "siem-referencedata-table-table2d526444eff99b1706053853ef7",
"_type" : "_doc",
"_id" : "0table222cc244b04b59d9ecafb0476e6",
"_score" : 1.0,
"_source" : {
"column-name1" : "10.1.10.1",
"column-name2" : "range(100,200)",
"column-name3" : "nam3",
"create_time" : "2022-05-21 03:30:39",
"last_seen" : "2022-05-21 03:30:39",
"id" : "0table222cc244b04b59d9ecafb0476e6"
}
},...
I want to search documents with three fields column-name1, column-name2 and column-name3.
I use below query with term to search exact considered word:
{
"query": {
"bool": {
"must": [
{
"term": {
"column-name1": {"value":"10.1.10.1"}
}
},
{
"term": {
"column-name2": {"value":"range(100,200)"}
}
},
{
"term": {
"column-name3": {"value":"nam3"}
}
}
]
}
}
}
It works without "column-name2": {"value":"range(100,200)"}.. what should I do with range ability? Is there another way to handle this?
The query solved with adding keyword to filed as below:
{
"query": {
"bool": {
"must": [
{
"term": {
"column-name1.keyword": {"value":"10.1.10.1"}
}
},
{
"term": {
"column-name2.keyword": {"value":"range(100,200)"}
}
},
{
"term": {
"column-name3.keyword": {"value":"nam3"}
}
}
]
}
}
}
Thank from Barkha Jain!
Related
I have the below JSON object as _source in Elastic search. I need to filter the source object based on conditions. For example, I need only JSON with applied_as == "COMMISSION"
"_source": {
"factor" : [
{
"some_amount_usd" : [
{
"applied_as" : "TCKT_CNT",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-01-01"
},
{
"applied_as" : "TCKT_CNT",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-02-01"
},
{
"applied_as" : "COMMISSION",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-02-01"
},
]
}
]
}
I am using this documentation.
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/search-fields.html#source-filtering
I am currently using this query with no luck. What am I missing?
GET form_some_index/_search
{
"query": {
"match": {
"factor.some_amount_usd.applied_as": "COMMISSION"
}
}
}
You can start using Nested Query.
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "factor",
"query": {
"nested": {
"path": "factor.some_amount_usd",
"query": {
"bool": {
"must_not": [
{
"term": {
"factor.some_amount_usd.applied_as.keyword": {
"value": "COMMISSION"
}
}
}
]
}
}
}
}
}
}
]
}
}
}
I tried:
GET /_search
{
"query": {
"simple_query_string" : {
"query": "\"sometext*\"",
"fields": ["name"],
"default_operator": "and",
"quote_field_suffix": '.raw',
"analyze_wildcard": true
}
}
}
but the search result is incorrect.
for example, there is an index:
prefixsometext
sometextone
sometexttwo
I need to find only 2 and 3 when querying "sometext*"
Wildcard are slow, you can use match_phrase_prefix instead
{
"query": {
"match_phrase_prefix": {
"text": "sometext"
}
}
}
Result:
"hits" : [
{
"_index" : "index20",
"_type" : "_doc",
"_id" : "7tdM7XEBxsgtRl4gFK-i",
"_score" : 0.9808291,
"_source" : {
"text" : "sometextone"
}
},
{
"_index" : "index20",
"_type" : "_doc",
"_id" : "79dM7XEBxsgtRl4gJq8Z",
"_score" : 0.9808291,
"_source" : {
"text" : "sometexttwo"
}
}
]
In simple_query_string quotes denote exact phrase match so "\"sometextone\"" will return result but if you need to do prefix match , query has to be defined as "query": "sometext*"
{
"query": {
"simple_query_string" : {
"query": "sometext*",
"fields": ["name"],
"default_operator": "and",
"quote_field_suffix": '.raw',
"analyze_wildcard": true
}
}
}
To replicate query in your comment using bool query
{
"query": {
"bool": {
"should": [
{
"match_phrase_prefix": {
"FIELD": "PREFIX"
}
},
{
"match": {
"FIELD": "TEXT"
}
}
],
"minimum_should_match": 1,
"must_not": [
{
"match": {
"FIELD": "TEXT"
}
}
]
}
}
}
This is the structure of my documents:
{
"_index" : "index",
"_type" : "_doc",
"_id" : "4002809",
"_score" : 5.6219883,
"_source" : {
"manufacturer" : "manufacturer of the part",
"shortdesc" : "Description of the part",
"te_param" : "None",
"coverart" : "/partpics/placeholder.jpg",
"has_datasheet" : 0,
"id" : 4002809,
"part" : "437297OBD25"
}
},
I need to discard results with field "shortdesc" empty
This should work:
GET /_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "shortdesc"
}
}
}
}
}
Referenced from here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
I like AJ's suggestion:
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "shortdesc"
}
}
}
}
}
That will return all the docs that DON'T have that field on it.
Or you can use the following:
{
"query": {
"bool": {
"must_not": {
"term": {
"shortdesc": ""
}
}
}
}
}
This will return all the docs that contain the field but with an empty description. I assume that an empty description has an empty string (represented as " ").
I'm having trouble aggregating my nested data to include null values as well.
I'm using Elasticsearch version 6.8
I'll simplify the problem, I've a nested field that looks like:
PUT test/doc/_mapping
{
"properties": {
"fields": {
"type" : "nested",
"properties" : {
"name" : {
"type" : "keyword"
},
"value" : {
"type" : "long"
}
}
}
}
}
I created 3 documents:
PUT test/doc/1
{
"fields" : {
"name" : "aaa",
"value" : 1
}
}
PUT test/doc/2
{
"fields" : [{
"name" : "aaa",
"value" : 1
},
{
"name" : "bbb",
"value" : 2
}]
}
PUT test/doc/3
{
"fields" : [
{
"name" : "bbb",
"value" : 2
}]
}
Now I want to group my data to get how many documents there are where name="bbb" group by each value.
For the above data I want to get:
2 – 2 documents
N/A – 1 document (the first document where bbb is missing)
The problem is with the null values, I cannot find a way to match the documents where "bbb" is null and put them in a N/A bucket.
So far I wrote a query that match the values where "bbb" exist:
GET test/doc/_search
{
"size": 0,
"query": {
"match_all": {}
},
"aggs": {
"my_agg": {
"nested": {
"path": "fields"
},
"aggs": {
"my_filter": {
"filter": {
"term": {
"fields.name": "bbb"
}
},
"aggs": {
"my_term": {
"terms": {
"field": "fields.value"
}
}
}
}
}
}
}
}
And the response is:
"aggregations" : {
"my_agg" : {
"doc_count" : 4,
"my_filter" : {
"doc_count" : 2,
"my_term" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : 2,
"doc_count" : 2
}
]
}
}
}
}
I want to get also:
"key" : 0 (for N/A)
"doc_count" : 1
What am I missing?
If I understand this correctly, you want to know the buckets where there was zero/null/no matches. You can use min_doc_count
GET test/doc/_search
{
"size": ,
"query": {
"match_all": {}
},
"aggs": {
"my_agg": {
"nested": {
"path": "fields"
},
"aggs": {
"my_filter": {
"filter": {
"term": {
"fields.name": "bbb"
}
},
"aggs": {
"my_term": {
"terms": {
"field": "fields.value", --> you can also use "_id" to get count based on each document
"min_doc_count": 0 --> this will include all the buckets where count is zero/ or there is no match.
}
}
}
}
}
}
}
}
You could also use inner_hits to find a hit in each document or use _id in above aggregations query.
POST test/_search
{
"query": {
"bool": {
"should": [
{
"match_all": {}
},
{
"nested": {
"path": "fields",
"query": {
"match": {
"fields.name": "bbb"
}
},
"inner_hits": {}
}
}
]
}
}
}
I have a list, array or whichever language you are familiar. E.g. names : ["John","Bas","Peter"] and I want to query the name field if it matches one of those names.
One way is with OR Filter. e.g.
{
"filtered" : {
"query" : {
"match_all": {}
},
"filter" : {
"or" : [
{
"term" : { "name" : "John" }
},
{
"term" : { "name" : "Bas" }
},
{
"term" : { "name" : "Peter" }
}
]
}
}
}
Any fancier way? Better if it's a query than a filter.
{
"query": {
"filtered" : {
"filter" : {
"terms": {
"name": ["John","Bas","Peter"]
}
}
}
}
}
Which Elasticsearch rewrites as if you hat used this one
{
"query": {
"filtered" : {
"filter" : {
"bool": {
"should": [
{
"term": {
"name": "John"
}
},
{
"term": {
"name": "Bas"
}
},
{
"term": {
"name": "Peter"
}
}
]
}
}
}
}
}
When using a boolean filter, most of the time, it is better to use the bool filter than and or or. The reason is explained on the Elasticsearch blog: http://www.elasticsearch.org/blog/all-about-elasticsearch-filter-bitsets/
As I tried the filtered query I got no [query] registered for [filtered], based on answer here it seems the filtered query has been deprecated and removed in ES 5.0. So I provide using:
{
"query": {
"bool": {
"filter": {
"terms": {
"name": ["John","Bas","Peter"]
}
}
}
}
}
example query = filter by keyword and a list of values
{
"query": {
"bool": {
"must": [
{
"term": {
"fguid": "9bbfe844-44ad-4626-a6a5-ea4bad3a7bfb.pdf"
}
}
],
"filter": {
"terms": {
"page": [
"1",
"2",
"3"
]
}
}
}
}
}