how to ignore number_format_exception error in elasticsearch query - elasticsearch

Hii how to ignore datatype error in below query since it throws error when a string value is provided for a field that has non-numeric(long) datatype. I am aware of lenient parameter but it does not work with term query.
GET employee/_search
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"employee_id": [
"abcdef"
]
}
},
{
"terms": {
"employee_name": [
"abcdef"
]
}
}
]
}
}
]
}
}
}
Error Message
"caused_by": {
"type": "number_format_exception",
"reason": "For input string: \"abcdef\""
}
Elasticsearch details
"version" : {
"number" : "7.1.1",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "Unknown",
"build_date" : "2020-11-03T08:48:42.499923Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
}
So the expected behaviour is the datatype error should be ignored and the rest of query runs and produces the result since it is in a should condition and if there is a must condition then give no result
mapping of index
{
"employee" : {
"mappings" : {
"dynamic" : "true",
"properties" : {
"employee_id" : {
"type" : "long"
}
}
}
}
}

You can use the query_string_query instead:
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"query_string": {
"query": "employee_id:abcdef" <---
}
},
{
"terms": {
"employee_name": [
"abcdef"
]
}
}
]
}
}
]
}
}
Your original query was a terms query which is equivalent to a logical OR. As such, you can adapt the query string to be:
"employee_id:(abcdef OR xyz OR 123)"
where the value type won't play a role.

Related

Elasticsearch, can't remove a field inside nested field

I have mappings
{
"candidate-index" : {
"mappings" : {
"properties" : {
"provider_candidates" : {
"type" : "nested",
"properties" : {
"foo" : {
"type" : "object"
},
"group_key" : {
"type" : "keyword"
}
}
}
}
}
}
I want to delete foo field
POST /candidate-index/_update_by_query
{
"script" : "ctx._source.remove(\"provider_candidates.foo\")",
"query": {
"nested": {
"path": "provider_candidates",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "provider_candidates.foo"
}
}
]
}
}
}
}
}
It doesn't work. It doesn't generate an error, but the field is not removed.
I know the query part is correct, because if I turn it into _search it correctly finds documents
I also tried
POST /candidate-index/_update_by_query
{
"script" : "ctx._source.provider_candidates.remove(\"foo\")",
"query": {
"nested": {
"path": "provider_candidates",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "provider_candidates.foo"
}
}
]
}
}
}
}
}
it says
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"ctx._source.provider_candidates.remove(\"foo\")",
" ^---- HERE"
],
"script" : "ctx._source.provider_candidates.remove(\"foo\")",
"lang" : "painless"
}
],
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"ctx._source.provider_candidates.remove(\"foo\")",
" ^---- HERE"
],
"script" : "ctx._source.provider_candidates.remove(\"foo\")",
"lang" : "painless",
"caused_by" : {
"type" : "wrong_method_type_exception",
"reason" : "cannot convert MethodHandle(List,int)Object to (Object,String)Object"
}
},
"status" : 400
}
You need to loop provider_candidates field and then delete field inside it
POST /index51/_update_by_query
{
"script" : "for (int i = 0; i < ctx._source.provider_candidates.length; ++i) { ctx._source.provider_candidates[i].remove(\"foo\") }",
"query": {
"nested": {
"path": "provider_candidates",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "provider_candidates.foo"
}
}
]
}
}
}
}
}

Combining nested query get illegal_state_exception failed to find nested object under path

I'm creating a query on Elasticsearch, for find documents through all indices.
I need to combine should, must and nested query on Elasticsearch, i get the right result but i get an error inside the result.
This is the query I'm using
GET _all/_search
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{ "term": { "trimmed_final_url": "https://www.repubblica.it/t.../" } }
],
"must": [
{
"nested": {
"path": "entities",
"query": {
"bool": {
"must": [
{ "term": { "entities.id": "138511" } }
]
}
}
}
},
{
"term": {
"language": { "value": "it" }
}
}
]
}
}
And this is the result
{
"_shards" : {
"total" : 38,
"successful" : 14,
"skipped" : 0,
"failed" : 24,
"failures" : [
{
"shard" : 0,
"index" : ".kibana_1",
"node" : "7twsq85TSK60LkY0UiuWzA",
"reason" : {
"type" : "query_shard_exception",
"reason" : """
failed to create query: {
...
"index_uuid" : "HoHi97QFSaSCp09iSKY1DQ",
"index" : ".reporting-2019.06.02",
"caused_by" : {
"type" : "illegal_state_exception",
"reason" : "[nested] failed to find nested object under path [entities]"
}
}
},
...
"hits" : {
"total" : {
"value" : 50,
"relation" : "eq"
},
"max_score" : 16.90015,
"hits" : [
{
"_index" : "i_201906_v1",
"_type" : "_doc",
"_id" : "MugcbmsBAzi8a0oJt96Q",
"_score" : 16.90015,
"_source" : {
"language" : "it",
"entities" : [
{
"id" : 101580,
},
{
"id" : 156822,
},
...
I didn't write some fields because the code is too long
I am new to StackOverFlow (made this account to answer this question :D) so if this answer is out of line bear with me. I have been dabbling in nested fields in Elasticsearch recently so I have some ideas as to how this error could be appearing.
Have you defined a mapping for your document type? I don't believe Elasticsearch will recognize the field as nested if you do not tell it to do so in the mapping:
PUT INDEX_NAME
{
"mappings": {
"DOC_TYPE": {
"properties": {
"entities": {"type": "nested"}
}
}
}
}
You may have to specify this mapping for each index and document type. Not sure if there is a way to do that all with one request.
I also noticed you have a "should" clause with minimum matches set to 1. I believe this is exactly the same as a "must" clause so I am not sure what purpose this achieves (correct me if I'm wrong). If your mapping is specified, the query should look something like this:
GET /_all/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "entities",
"query": {
"term": {
"entities.id": {
"value": "138511"
}
}
}
}
},
{
"term": {
"language": {
"value": "it"
}
}
},
{
"term": {
"trimmed_final_url": {
"value": "https://www.repubblica.it/t.../"
}
}
}
]
}
}
}

Elasticsearch: How to get an exact match in a nested field

The mapping contains nested fields which shouldn't be analyzed (not sure if the 'not_analyzed' value is accurate). Is it possible to do an exact match on a nested field? In the query below the "metadata.value": "2014.NWJSD.47" still gets analyzed. Elasticsearch breaks up the string into several terms ("2014", "NWJSD", "47"). I tried to use "term" instead of "match" but this didn't return any result.
"mappings" : {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "text",
"index" : "not_analyzed"
},
"value" : {
"type" : "text",
"index" : "not_analyzed"
}
}
}
}
The Query:
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"match": {
"metadata.name": "number"
}
},
{
"match": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}
Try to use keyword instead of text in your mapping like:
{
"mappings": {
"your_type_name": {
"properties": {
"metadata" : {
"type" : "nested",
"properties" : {
"name" : {
"type" : "keyword"
},
"value" : {
"type" :"keyword"
}
}
}
}
}
}
}
These fields won't be analyzed. Then you should reindex your data and to query your data you should replace match (which is analyzed query) with term.
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "metadata",
"query": {
"bool": {
"must": [
{
"term": {
"metadata.name": "number"
}
},
{
"term": {
"metadata.value": "2014.NWJSD.47"
}
}
]
}
}
}
}
]
}
}
}
I think you are looking for a query string query.
You can freely disable "analyze" option for that field in mapping option and reindex everything again but you could also check this query out:
as written here:
GET /_search
{
"query": {
"query_string" : {
"query" : "your string"
}
}
}

failed to parse search source. expected field name but got [START_OBJECT]

I want to express this SQL in Elasticsearch:
select * from ticket where user_id = 1 and (class_a = 1000010 or class_b = 16);
I use a combining filter as below:
curl 'localhost:9200/ticket/_search?pretty' -d'
{
"query": {
"bool": {
"should": [
{"term": {"class_a": 1000010}},
{"term": {"class_b": 16}}
]
},
"filter": {
"term": {
"user_id": 1
}
}
}
}'
but got the error as below:
{
"error" : {
"root_cause" : [ {
"type" : "parse_exception",
"reason" : "failed to parse search source. expected field name but got [START_OBJECT]"
} ],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query_fetch",
"grouped" : true,
"failed_shards" : [ {
"shard" : 0,
"index" : "ticket",
"node" : "FO3-zhb1R1WCak381t88gQ",
"reason" : {
"type" : "parse_exception",
"reason" : "failed to parse search source. expected field name but got [START_OBJECT]"
}
} ]
},
"status" : 400
}
Anyone can help me? Thanks in advance!
You're almost there, you need to rewrite your query like this (i.e. move your filter inside the bool clause):
curl 'localhost:9200/ticket/_search?pretty' -d'{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"term": {
"class_a": 1000010
}
},
{
"term": {
"class_b": 16
}
}
],
"filter": {
"term": {
"user_id": 1
}
}
}
}
}'

elasticsearch searching array field inside nested type

i am trying to filter my result using nested filter but i am getting incorrect result
here is my mapping info
{
"stock" : {
"mappings" : {
"clip" : {
"properties" : {
"description" : {
"type" : "string"
},
"keywords" : {
"type" : "nested",
"properties" : {
"category" : {
"type" : "string"
},
"tags" : {
"type" : "string",
"index_name" : "tag"
}
}
},
"tags" : {
"type" : "string",
"index_name" : "tag"
},
"title" : {
"type" : "string"
}
}
}
}
}
}
clip document data
{
"_index" : "stock",
"_type" : "clip",
"_id" : "AUnsTOBBpafrKleQN284",
"_score" : 1.0,
"_source":{
"title": "journey to forest",
"description": "this clip contain information about the animals",
"tags": ["birls", "wild", "animals", "roar", "forest"],
"keywords": [
{
"tags": ["spring","summer","autumn"],
"category": "Weather"
},
{
"tags": ["Cloudy","Stormy"],
"category": "Season"
},
{
"tags": ["Exterior","Interior"],
"category": "Setting"
}
]
}
i am trying to filter tags inside nested field 'keywords'
here is my query
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "keywords",
"filter": {
"bool": {
"must": [
{
"terms": { "tags": ["autumn", "summer"] }
}
]
}
}
}
}
}
}
}
i am getting no result why ?
what's wrong with my query or schema please help
The above query is syntactically incorrect . You need to provide the full path to tags from root keywords in the term query i.e.keywords.tags
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "keywords",
"filter": {
"bool": {
"must": [
{
"terms": { "keywords.tags": ["autumn", "summer"] }
}
]
}
}
}
}
}
}
}

Resources