I'm having a problem in searching records with a field. Although, field exists with the value in the document in Elasticsearch but when I use this field to search as term, it does not fetch the record. Other fields are doing great.
JSON Request:
{
"query": {
"filtered": {
"query": {
"match_all": [
]
},
"filter": {
"and": [
{
"term": {
"sellerId": "6dd7035e-1d6f-4ddb-82f4-521902bfc29e"
}
}
]
}
}
}
}
It does not return any error, it just doesn't fetch the related document. I tried searching with other fields and they worked fine.
Is there anything I'm missing here ?
Elasticsearch version: 2.2.2
Arfeen
You need to reindex your data and change the mapping of that field to
"sellerId": {
"type": "string",
"index": "not_analyzed"
}
That way the UUID won't be analyzed and split into tokens and you'll be able to search it using a term query.
Related
I want to sort datas via elasticsearch rest client, below is my template in logstash
{
"index_patterns": ["index_name"],
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"_doc": {
"properties": {
"int_var": {
"type": "keyword"
}
}
}
}
}
}
When I try to reach, with the below code
{
"size": 100,
"query": {
"bool": {
"must": {
"match": {
"match_field": user_request
}
}
}
},
"sort": [
{"int_var": {"order": "asc"}}
]
}
I've got this error
Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true
How can i solve this ? Thanks for answering
Here's the documentation regarding field data and how to enable it as long as you are aware of the performance impacts.
When ingested into Elasticsearch, field values are tokenized based on their data type.
Text fields are broken into tokens delimited by whitespace. I.E. "quick brown fox" creates three tokens: 'quick', 'brown', and 'fox'. If you perform a search for any of these three words, you will generate matches.
Keyword fields, on the other hand, create a single token of the entire value. I.E. "quick brown fox" is a single token, 'quick brown fox'. Searching for anything that is not exactly 'quick brown fox' will generate no matches.
Unless you scrubbed your query before you posted it here, you need to modify the field name under match to be the actual field name, like below.
{
"size": 100,
"query": {
"bool": {
"must": {
"match": {
"int_var": "whatever value you are searching for"
}
}
}
},
"sort": [
{"int_var": {"order": "asc"}}
]
}
I have an index with optional Date/Time field called lastChackoutDate. Trying to filter rows by range or term query returns 0 rows but I know there are some documents where value for this field exists.
Mappings query returns me an expected answer with:
... ,
"lastCheckoutDate": {
"type": "date"
},
...
Trying to identify what query can return me result I'm waiting for eventually led me to an expression:
{
"from": 0,
"query": {
"bool": {
"filter": [
{
"bool": {
"must_not": [
{
"exists": {
"field": "lastCheckoutDate"
}
}
],
"must": [
{
"nested": {
"path": "nested_path",
"query": {
"term": {
"nested_path.id": {
"value": "some_unique_id"
}
}
}
}
}
]
}
}
]
}
},
"size": 50,
"sort": [
{
"displaySequence": {
"order": "asc"
}
}
]
}
which returned me a single row with existing path/value:
hits
[0]
_source
lastCheckoutDate: 2020-01-23T00:00:00
explain of this query didn't shed a light on "exists" response details: ConstantScore(+ToParentBlockJoinQuery (nested_path.id:some_unique_id) -ConstantScore(_field_names:lastCheckoutDate)), product of:
So are there any ways to determine why field is invisible for query?
This works fine for test database which is being created and dropped each time, but existing storage always gives me 0 hits for any valid (from my POV) query. Ofc I did a migration action for existing database (at least somehow mapping info appeared for a new field).
Elastic documentation shows some examples why "exists" query may fail:
- The field in the source JSON is null or []
- The field has "index" : false set in the mapping
- The length of the field value exceeded an ignore_above setting in the mapping
- The field value was malformed and ignore_malformed was defined in the mapping
But I'm not sure that any option is true for my case.
New documents were added before migration happened. So AFAIK Elastic won't re-index existing documents until they are updated in index.
So that's why on test database I had no issues.
try this
GET /index_name/_search
{
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "fieldname"
}
}
]
}
}
}
Upgrading from Elasticsearch 5.x to 6.x. We make extensive use of query string queries and commonly construct queries which used fields of different types.
In 5.x, the following query worked correctly and without error:
{
"query": {
"query_string": {
"query": "my_keyword_field:\"Exact Phrase Here\" my_text_field:(any words) my_other_text_field:\"Another phrase here\" date_field:[2018-01-01 TO 2018-05-01]",
"default_operator": "AND",
"analyzer": "custom_text"
}
}
}
In 6.x, this query will return the following error:
{
"type": "illegal_state_exception",
"reason": "field:[my_keyword_field] was indexed without position data; cannot run PhraseQuery"
}
If I wrap the phrase in parentheses instead of quotes, the search will return 0 results:
{
"query": {
"query_string": {
"query": "my_keyword_field:(Exact Phrase Here)",
"default_operator": "AND",
"analyzer": "custom_text"
}
}
}
I guess this is because there is a conflict between the way the analyzer stems the incoming query and how the data is stored in the keyword field, but the phrase approach (my_keyword_field:"Exact Phrase Here") did work in 5.x.
Is this no longer supported in 6.x? And if not, what is the migration path and/or a good workaround?
It would be better to rephrase the query by using different type of queries available for different use cases. For example use term query for exact search on keyword field. Use range query for ranges etc.
You can rephrase query as below:
{
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "my_text_field:(any words) my_other_text_field:\"Another phrase here\"",
"default_operator": "AND",
"analyzer": "custom_text"
}
},
{
"term": {
"my_keyword_field": "Exact Phrase Here"
}
},
{
"range": {
"date_field": {
"gte": "2018-01-01",
"lte": "2018-05-01"
}
}
}
]
}
}
}
elasticsearch version: 6.4
Here is my current data:
I want to search for products which has Xbox in name. I am using the match keyword but that is not working.
Below is my elastic search query:
{
"query": {
"bool": {
"must": [
{
"match": {
"name": {
"query": "xbox"
}
}
},
{
"terms": {
"deep_sub": [
"Konsol Oyunları",
"Konsol Aksesuarları"
]
}
}
]
}
},
"from": 0,
"size": 50
}
Whenever you face such kind of issues, try breaking down the query. You have Match Query and Term Query. Run both of them individually and see what's not working.
From what I understand, looks like your field deep_sub is of text type and this would mean Term Query is not returning results.
You would need to create its sibling equivalent using keyword type and then run Term Query on it for exact matches.
From the above link we have the below info:
Keyword fields are only searchable by their exact value.
If you do not control the mapping, meaning if your mapping if of dynamic type, then you must have its sibling equivalent keyword field available which would be deep_sub.keyword
You can check by running GET <your_index_name>/_mapping
Your query would then be as follows:
POST <your_index_name>/_search
{
"query":{
"bool":{
"must":[
{
"match":{
"name":{
"query":"xbox"
}
}
},
{
"terms":{
"deep_sub.keyword":[ <----- Change This
"Konsol Oyunları",
"Konsol Aksesuarları"
]
}
}
]
}
},
"from":0,
"size":50
}
Let me know if this helps!
I'm having issue while trying to query or filter by a field that stores a file Path like "c:\\change\\users\\89841b89-6529-43a6-9ca9-b2a851b4a7da\\example", I used to do a filter prefix without any issue in that field, but after updating ElasticSearch to version 1.4.0.
Any clue??? This is how my query looks like:
{
"query": {
"query_string": {
"query": "content.type:estimate"
}
},
"filter": {
"prefix": {
"storageDir": "c:\\change\\users\\89841b89-6529-43a6-9ca9-b2a851b4a7da\\example"
}
},
"sort": [
{
"modifiedWhen": {
"order": "desc"
}
}
]
}
Make sure that the field "storageDir" is marked as "not_analyzed" in the mapping. Otherwise it will split on slashes and dashes.