We have a table that shows warning counts in our services, but we aren't interested in those that are zero.
Because the warning count is a un-indexed number (integer) we can't include it as a filter criteria. How can these be removed from our dashboard?
I am not familliar with Grafana, but ES has an "exists" query which might be what you are looking for. So, if you can integrate this in your dashboard (or find that Grafana functionality that uses it) it might help you. For your information (because Grafana might use this as well like it is) you can use "exists" query in the Lucene query dsl as _exists_. A practical example - _exists_:warning_count_field. More about this here
The same question seems to have been asked before in a more or less similar form: https://community.grafana.com/t/how-to-query-for-null-values/799/5 and the answer is the same as the one I posted above: the _exists_ filter/query.
Related
I have a composite aggregation query doing exactly what I want (the details of said query should not matter). I would like very much to visualise the results in Vega as a nice time-based chart, but I've hit a very stupid roadblock: I cannot find how to ask Vega to fetch all results. Composite aggregation results are paged (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html#_pagination) and therefore, in order to get all results, multiple queries should be done. So I can display one page of data, which is not enough in my case.
Is there a way to fetch all pages with Vega or Vega-Lite? If not, perhaps in another graph module of Kibana? A quick search gave no definitive answers… And finally, I have the latest version of everything.
Thanks!
Yup, no. Dynamic elastic URLs are not doable (basing a query of another query), I think I put through a feature request for this a while back, but unfortunately Vega and Kibana integrations get pushed to the way side for improvements in Lens.
Hopefully in the future this is something they do because it would severely improve the Vega-Kibana capibilities. I guess it depends on what you are actually trying to do, and whether you can find a way to get the data through in one search - this would be my advice.
Once a query is executed on ElasticSearch, a relevance _score is calculated for each retrieved document.
Given a specific document (e.g. by doc ID) and a specific query, I would like to see what is its _score?
One way is perhaps to query ES, retrieve all the hit documents, and look up the desired document out of all the retrieved documents to see its score.
I assume there should be a more efficient way to do this. Given a query and a document ID, what is its _score?
I'm using ElasticSearch 7.x
PS: I need this for a learning-to-rank scenario (to create my judgment list). I have in fact a complex query that was created from various should and must over different fields. My major requirement was to get the score value for each individual sub-query, which seems there is no solution for it. I want to understand which part of this complex query is more useful and which one is less. The only way I've come up with is to execute each sub-query separately to get the score but I do not want to actually execute that query just asking for what is the score of a specific document for that sub-query.
Scoring of the document is not only related to just the document and all other documents in the index, but it also depends on various factor like:
_score is calculated per shard basis not on an index basis by default, although you can change this behavior by using DFS Query Then Fetch param in your query. More info on this official blog.
Is there is any boost applied at index or query time(index time is deprecated from 5.X).
Any custom scoring function is used in addition to the default ES scoring algorithm(tf/idf in old versions) and BM25 in the latest versions.
Edit: Based on the comments from the other respected community members, rephrasing the below statement:
To answer your question, Using the _explain API, you can understand how Elasticsearch computes a score explanation for a query and a specific document. This can give useful feedback on whether a document matches or didn’t match a specific query.
I'm attempting to filter search suggestions by multiple conditions in Elasticsearch 5.0+. For example, as a user types, we'd say "Here are suggestions that meet both criteria X AND criteria Y". My understanding is that the way to do that is with contexts and categories as defined here: https://www.elastic.co/guide/en/elasticsearch/reference/5.3/suggester-context.html
However, I also found this Github issue saying that these contexts were changed in 5.x to be OR'd together when they had previously been AND'd: https://github.com/elastic/elasticsearch/issues/21291
Does this mean this goal is impossible in Elasticsearch? The Github issue is from November, and it seems like this is a big loss/change in functionality.
I am pretty new to elasticsearch and already love it.
Right know I am interested in understanding on how I can let elasticsearch make suggestions for similar keywords.
I have already read this article: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-mlt-query.html.
The More Like This Query (MLT Query) finds documents that are "like" a given set of documents.
This is already more than I am looking for. I dont need similar documents but only related / similar keywords.
So lets say I have an index of documents about movies and I start a query about "godfather". Then elasticsearch should suggest related keywords - e.g. "al pacino" or "Marlon Brando" because they are likely to occur in the same documents.
any ideas how this can be done?
Unfortunately, there is no built-in way to do that in Elastic. What you could possibly do, is to write a program, that will query Elastic, return matched documents, then you will get the _source data, or just retrieve it from your original datasource (like DB or file), later you will need to calculate TF-IDF for each term in the retrieved ones and somehow combine everything all together to get top K terms out of all returned terms.
I am trying to implement a simple auto completion for query terms.
There are many different approaches but most of them do return documents instead of terms
- or the authors simply stopped explaining from that point and i am not able to adapt.
A user is typing in a query - e.g. phil
What i want is to provide a list of term completion suggestions like philipp, philius, philadelphia, ...
I am able to get document matches via (edge)ngrams, phrase_prefix and so on but i am am stuck at retrieving matching terms (completion suggestions).
Can someone give me a hint?
I have documents like this {"title":"...", "description":"...", "content":"..."}
All fields have larger string values but especially the field content contains fulltext content.
I do not want to suggest the whole title of a document containing e.g. Philadelphia. Just the word "Philadelphia".
Looking for something like that, myself.
In SOLR it was relatively simple to configure (although a pain to build and keep up-to-date) using solr.SpellCheckComponent. Somehow the same underlying Lucene functionality is used differently between SOLR and ElasticSearch, and in ElasticSearch it is geared towards finding whole documents (or whole field values, if you will) or so it seems...
Despite the profusion of "elasticsearch autocomplete" articles, none appears to deal with this particular issue. Like it doesn't exist. Maybe their use case is different and ElasticSearch works for them just fine, who knows?
At this point I think that preparing the exact field values to use with ElasticSearch autocomplete (yes, that's the input field values, not analyzer tokens) maybe the only way to solve the problem. Which is terrible, because the performance is going to be very low.
Try term suggester:
The term suggester suggests terms based on edit distance. The provided
suggest text is analyzed before terms are suggested. The suggested
terms are provided per analyzed suggest text token. The term suggester
doesn’t take the query into account that is part of request.