This is the structure of my documents:
{
"_index" : "index",
"_type" : "_doc",
"_id" : "4002809",
"_score" : 5.6219883,
"_source" : {
"manufacturer" : "manufacturer of the part",
"shortdesc" : "Description of the part",
"te_param" : "None",
"coverart" : "/partpics/placeholder.jpg",
"has_datasheet" : 0,
"id" : 4002809,
"part" : "437297OBD25"
}
},
I need to discard results with field "shortdesc" empty
This should work:
GET /_search
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "shortdesc"
}
}
}
}
}
Referenced from here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
I like AJ's suggestion:
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "shortdesc"
}
}
}
}
}
That will return all the docs that DON'T have that field on it.
Or you can use the following:
{
"query": {
"bool": {
"must_not": {
"term": {
"shortdesc": ""
}
}
}
}
}
This will return all the docs that contain the field but with an empty description. I assume that an empty description has an empty string (represented as " ").
Related
I have the below JSON object as _source in Elastic search. I need to filter the source object based on conditions. For example, I need only JSON with applied_as == "COMMISSION"
"_source": {
"factor" : [
{
"some_amount_usd" : [
{
"applied_as" : "TCKT_CNT",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-01-01"
},
{
"applied_as" : "TCKT_CNT",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-02-01"
},
{
"applied_as" : "COMMISSION",
"version" : "8",
"factor_value" : "1.12",
"start_date" : "2022-02-01"
},
]
}
]
}
I am using this documentation.
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/search-fields.html#source-filtering
I am currently using this query with no luck. What am I missing?
GET form_some_index/_search
{
"query": {
"match": {
"factor.some_amount_usd.applied_as": "COMMISSION"
}
}
}
You can start using Nested Query.
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "factor",
"query": {
"nested": {
"path": "factor.some_amount_usd",
"query": {
"bool": {
"must_not": [
{
"term": {
"factor.some_amount_usd.applied_as.keyword": {
"value": "COMMISSION"
}
}
}
]
}
}
}
}
}
}
]
}
}
}
I have some documents in a index..
"hits" : [
{
"_index" : "siem-referencedata-table-table2d526444eff99b1706053853ef7",
"_type" : "_doc",
"_id" : "0table222cc244b04b59d9ecafb0476e6",
"_score" : 1.0,
"_source" : {
"column-name1" : "10.1.10.1",
"column-name2" : "range(100,200)",
"column-name3" : "nam3",
"create_time" : "2022-05-21 03:30:39",
"last_seen" : "2022-05-21 03:30:39",
"id" : "0table222cc244b04b59d9ecafb0476e6"
}
},...
I want to search documents with three fields column-name1, column-name2 and column-name3.
I use below query with term to search exact considered word:
{
"query": {
"bool": {
"must": [
{
"term": {
"column-name1": {"value":"10.1.10.1"}
}
},
{
"term": {
"column-name2": {"value":"range(100,200)"}
}
},
{
"term": {
"column-name3": {"value":"nam3"}
}
}
]
}
}
}
It works without "column-name2": {"value":"range(100,200)"}.. what should I do with range ability? Is there another way to handle this?
The query solved with adding keyword to filed as below:
{
"query": {
"bool": {
"must": [
{
"term": {
"column-name1.keyword": {"value":"10.1.10.1"}
}
},
{
"term": {
"column-name2.keyword": {"value":"range(100,200)"}
}
},
{
"term": {
"column-name3.keyword": {"value":"nam3"}
}
}
]
}
}
}
Thank from Barkha Jain!
I'm new to elasticsearch. I need to build the query dynamically, where for each field name the the corresponding file is fetched
I have the below query, can anyone say if its the right approach? Also with this query, the documents are just repeating for one particular file name
Please let me know how to go about it
GET index_name/_search
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "program"
}
},
{
"match_phrase": {
"field_value": "aaa-123"
}
}
]
}
},
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "species"
}
},
{
"match_phrase": {
"field_value": "mouse"
}
}
]
}
},
{
"bool": {
"must": [
{
"match_phrase": {
"field_name": "model name"
}
},
{
"match_phrase": {
"field_value": "b45"
}
}
]
}
}
]
}
},"aggs": {
"2": {
"terms": {
"field": "myfile_file_name.keyword",
"size": 1000,
"order": {
"_key": "asc"
}
},
"aggs": {
"3": {
"terms": {
"field": "field_name.keyword",
"size": 1000,
"order": {
"_key": "asc"
}
}
}
}
}
}
}
mapping and Output
{
"_index" : "test",
"_type" : "test_data",
"_id" : "123",
"_score" : 1.0,
"_source" : {
"document_id" : 123,
"m_id" : 1,
"source" : "ADDD",
"type" : "M",
"name" : "Animal",
"value" : "None",
"test_type" : "Test123",
"file_name" : "AA.zip",
"description" : "testing",
"program" : ["hello"],
"species" : ["mouse"],
"study" : ["Study1"],
"create_date" : "2020-08-20 11:51:21.152",
"update_date" : "2020-08-20 11:51:21.152",
"source_name" : "Anim",
"auth" : ["na"],
"treatment" : ["TR001", "TR002", "TR004"],
"timepoint" : ["72", "48"],
"findings_reports" : "na",
"model" : ["None",],
"additional" : "{'view': '', 'load': []}",
"data" : "Pre"
}
},
]
}
}
I tried:
GET /_search
{
"query": {
"simple_query_string" : {
"query": "\"sometext*\"",
"fields": ["name"],
"default_operator": "and",
"quote_field_suffix": '.raw',
"analyze_wildcard": true
}
}
}
but the search result is incorrect.
for example, there is an index:
prefixsometext
sometextone
sometexttwo
I need to find only 2 and 3 when querying "sometext*"
Wildcard are slow, you can use match_phrase_prefix instead
{
"query": {
"match_phrase_prefix": {
"text": "sometext"
}
}
}
Result:
"hits" : [
{
"_index" : "index20",
"_type" : "_doc",
"_id" : "7tdM7XEBxsgtRl4gFK-i",
"_score" : 0.9808291,
"_source" : {
"text" : "sometextone"
}
},
{
"_index" : "index20",
"_type" : "_doc",
"_id" : "79dM7XEBxsgtRl4gJq8Z",
"_score" : 0.9808291,
"_source" : {
"text" : "sometexttwo"
}
}
]
In simple_query_string quotes denote exact phrase match so "\"sometextone\"" will return result but if you need to do prefix match , query has to be defined as "query": "sometext*"
{
"query": {
"simple_query_string" : {
"query": "sometext*",
"fields": ["name"],
"default_operator": "and",
"quote_field_suffix": '.raw',
"analyze_wildcard": true
}
}
}
To replicate query in your comment using bool query
{
"query": {
"bool": {
"should": [
{
"match_phrase_prefix": {
"FIELD": "PREFIX"
}
},
{
"match": {
"FIELD": "TEXT"
}
}
],
"minimum_should_match": 1,
"must_not": [
{
"match": {
"FIELD": "TEXT"
}
}
]
}
}
}
I'm having trouble aggregating my nested data to include null values as well.
I'm using Elasticsearch version 6.8
I'll simplify the problem, I've a nested field that looks like:
PUT test/doc/_mapping
{
"properties": {
"fields": {
"type" : "nested",
"properties" : {
"name" : {
"type" : "keyword"
},
"value" : {
"type" : "long"
}
}
}
}
}
I created 3 documents:
PUT test/doc/1
{
"fields" : {
"name" : "aaa",
"value" : 1
}
}
PUT test/doc/2
{
"fields" : [{
"name" : "aaa",
"value" : 1
},
{
"name" : "bbb",
"value" : 2
}]
}
PUT test/doc/3
{
"fields" : [
{
"name" : "bbb",
"value" : 2
}]
}
Now I want to group my data to get how many documents there are where name="bbb" group by each value.
For the above data I want to get:
2 – 2 documents
N/A – 1 document (the first document where bbb is missing)
The problem is with the null values, I cannot find a way to match the documents where "bbb" is null and put them in a N/A bucket.
So far I wrote a query that match the values where "bbb" exist:
GET test/doc/_search
{
"size": 0,
"query": {
"match_all": {}
},
"aggs": {
"my_agg": {
"nested": {
"path": "fields"
},
"aggs": {
"my_filter": {
"filter": {
"term": {
"fields.name": "bbb"
}
},
"aggs": {
"my_term": {
"terms": {
"field": "fields.value"
}
}
}
}
}
}
}
}
And the response is:
"aggregations" : {
"my_agg" : {
"doc_count" : 4,
"my_filter" : {
"doc_count" : 2,
"my_term" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : 2,
"doc_count" : 2
}
]
}
}
}
}
I want to get also:
"key" : 0 (for N/A)
"doc_count" : 1
What am I missing?
If I understand this correctly, you want to know the buckets where there was zero/null/no matches. You can use min_doc_count
GET test/doc/_search
{
"size": ,
"query": {
"match_all": {}
},
"aggs": {
"my_agg": {
"nested": {
"path": "fields"
},
"aggs": {
"my_filter": {
"filter": {
"term": {
"fields.name": "bbb"
}
},
"aggs": {
"my_term": {
"terms": {
"field": "fields.value", --> you can also use "_id" to get count based on each document
"min_doc_count": 0 --> this will include all the buckets where count is zero/ or there is no match.
}
}
}
}
}
}
}
}
You could also use inner_hits to find a hit in each document or use _id in above aggregations query.
POST test/_search
{
"query": {
"bool": {
"should": [
{
"match_all": {}
},
{
"nested": {
"path": "fields",
"query": {
"match": {
"fields.name": "bbb"
}
},
"inner_hits": {}
}
}
]
}
}
}