I am trying to perform the simplest filter for a specific property value, as a JSON input, in a Kibana visualization, thoroughly without success.
I can't, to my surprise, find concrete examples in doing that (have been searching for a couple of minutes now).
Say we have a document with the following structure:
{
a: true,
b: 10
}
How can I add a Filter aggregation for all documents with a = true ?
I tried using "script", "query", "filters" api, but all give me parse errors. My filter jsons are all valid, my problem is with the exact syntax elastic is expecting, but all examples I found out there and tried - give me parsing errors (after making the amendments to my index structure).
Kibana's version: 6.4.3
How is this accomplished ?
An example:
POST /sales/_search?size=0
{
"aggs" : {
"docs" : {
"filter" : { "term": { "a": "true" } },
}
}
}
Here is the link to the official documentation with example.
Related
I have field in index with mapping as :
"sequence_number" : {
"type" : "long",
"copy_to" : [
"_custom_all"
]
}
and using search query as
POST /my_index/_search
{
"query": {
"term": {
"sequence_number": {
"value": "we"
}
}
}
}
I am getting error message :
,"index_uuid":"FTAW8qoYTPeTj-cbC5iTRw","index":"my_index","caused_by":{"type":"number_format_exception","reason":"For input string: \"we\""}}}]},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:260) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:238) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212) ~[elasticsearch-rest-client-7.1.1.jar:7.1.1]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433) ~[elasticsearch-rest-high-level-client-7.1.1.jar:7.1.1]
at
How can i ignore number_format_exception errors, so the query just doesn't return anything or ignores this filter in particular - either is acceptable.
Thanks in advance.
What you are looking for is not possible, ideally, you should have coherce enabled on your numeric fields so that your index doesn't contain dirty data.
The best solution is that in your application which generated the Elasticsearch query(you should have a check for NumberFormatExcepton if you are searching for numeric fields as your index doesn't contain the dirty data in the first place and reject the query if you get an exception in your application).
Edit: Another interesting approach is to validate the data before inserting into ES, using the Validate API as suggested by #prakash, only thing is that it would add another network call but if your application is not latency-sensitive, it can be used as a workaround.
I am facing a strange issue in elasticsearch query. I don't know much about elasticsearch. My query is:
{
"query":
{
"bool":
{
"must":
[
{
"text":
{
"countryCode2":"DE"
}
}
],
"must_not":[],
"should":[]
}
},"from":0,"size":1,"sort":[],"facets":{}
}
The issues is for "DE". It is giving me results but for "BE" or "IN" it returns empty result.
You are indexing using the default mapping, which by default removes english stopwords. The country codes "IN", "BE", and many more are stopwords which don't even get indexed, therefore it's not possible to have matching documents, nor get back those country codes when faceting on that field.
The solution is to reindex after having submitted your own mapping for the country code field:
{
"your_type_name" : {
"country" : {
"type" : "string", "index" : "not_analyzed"
}
}
}
If you already tried to do this but nothing changed, the mapping didn't get submitted properly. I would suggest to double check that its json structure is correct and that you can actually get it back using the get mapping api.
As this is a common problem the defaults are probably going to change in the future to be less intrusive and avoid applying any language dependent text analysis.
I have a situation where I want to filter the results not when performing a search but rather a GET using elasticsearch. Basically I have a Document that has a status field indicating that the entity has a state of discarded. When performing the GET I need to check the value of this field thereby excluding it if the status is indeed one of "discarded".
I know i can do this using a search with a term query, but what about when using a GET against the index based on Document ID?
Update: Upon further investigation, it seems the only way to do this is to use percolation or a search. I hope I am wrong if anyone has any suggestions I am all ears.
Just to clarify I am using the Java API.
thanks
Try something like this:
curl http://domain/my_index/_search -d '{
"filter": {
"and": [
{
"ids" : {
"type" : "my_type",
"values" : ["123"]
}
},
{
"term" : {
"discarded" : "false"
}
}
]
}
}
NOTE: you can also use a missing filter if the discarded field does not exist on some docs.
NOTE 2: I don't think this will be markedly slower than a normal get request either...
I'm trying to use ElasticSearch to find all records containing a particular string. I'm using a match query for this, and it's working fine.
Now, I'm trying to sort the results based on a particular field. When I try this, I get some very unexpected output, and none of the records even contain my initial search query.
My request is structured as follows:
{
"query":
{
"match": {"_all": "some_search_string"}
},
"sort": [
{
"some_field": {
"order": "asc"
}
}
] }
Am I doing something wrong here?
In order to sort on a string field, your mapping must contain a non-analyzed version of this field. Here's a simple blog post I found that describes how you can do this using the multi_field mapping type.
I prepare query object and do search in elasticsearch.
For making query object, I give key and their value.
Problem is, when key and value is like "brand":"Men's Wear" then In this case elasticsearch is unable to give me related docs. I think problem is with comma or may be space. everything is fine if I use other json property for key and value (having no space and comma like "priority":"high")
Any help please!
Update:
no match query still not working! one more problem i found in creating search query. query i am using is:
var qryObj1 = {
"query" : {
"text" : {"name":"Tom"}
}
};
This will return all docs having name Tom. Now I want to get all docs having name Tom and profession is developer. So, here modified one:
qryObj1 = {
"query" : {
"text" : {"name":"Tom","profession":"developer"}
},"operator" : "and"
};
but search result is old one. any help!
Sounds like you are using TermQuery, aren't you?
TermQuery are not analyzed so they don't match with your analyzed content.
Try with a MatchQuery. It should work.
You need to use boolean query
http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html
Here you can ask ES to take AND or OR of various queries
"bool" : {
"must" : [
"text" : {"name":"Tom"},
"text" : {"profession":"developer"}
]
}