A somewhat similar question has been asked here but there's no answer for that yet. That question relates to an older version of Kibana so I hope you can help me.
I'm trying to setup some predefined queries in the Kibana dashboard. I'm using Kibana 5.1. The purpose of those queries is filtering some logs based on multiple different parameters.
Let's see a query I'd like to execute:
{
"index": "${index_name}",
"query": {
"query_string": {
"query": "message:(+\"${LOG_LEVEL}\")",
"analyze_wildcard": true
}
}
}
I know I can query directly in the dashboard something like "message:(+"ERROR")" and manually change the ERROR to WARN for example, but I don't want that - imagine that this query might be more complex and contain multiple fields.
Note that the data stored in the message is not structured - think of the message as a whole log line. This means I don't have fields like LOG_LEVEL which I could filter directly.
Is there any way I can set the index_name and LOG_LEVEL dynamically from the Kibana Discover dashboard?
You should go to discover, open one document and click over this button in any of the fields. After this, a filter will appear under the search bar and you can edit it and put any custom query. If you want add more filters with more custom queries you can repeat the same action with a different document or field or you can do to Settings (or Management), Saved Objects, go to the Search you saved and to the JSON representation and copy and paste the elements inside the filter array field as many times you want.
And remember that in order to apply one of the filters, you probably should disable the enabled ones (otherwise it will filter by all the enabled filters in your dashboard).
Related
I'd like to create a query variable in Grafana that will bring back a series a distinct Host Names that will be used to filter out results in my dashboard.
I am using ElasticSearch as the datasource and here is a sample of the fields in the log record(showing only the host:hostname for brevity):
"host": {
"hostname": "ip-xx-xx-xx-xx",
"architecture": "x86_64",
}
The term "host.hostname" is what I am trying to retrieve.
In the table view of ElasticSearch the field is displayed as follows:
Here is what I'm trying to do:
I want to write a Lucene query for a Grafana query variable.
For the time-range that the user selects I'd like the query to return all available hosts in the current ElasticSearch stream.
I'd like to return only unique entries. If the stream contains 3 unique hosts I'd like to return only 3.
This query variable will be used in the dashboard to filter out specific hosts (one, multiple or all) and then I'd like my panel queries to display results only for the hosts that the user has selected.
Here is the query I currently have, not surprisingly, it doesn't work:
{"find": "terms", "field": "host.hostname", "query": "ip-*"}
Here is what the Variable Edit screen looks like (Datasource name scrubbed):
Here is what the panel editor looks like. Not that ip addresses are shown in legend. That's fine. In this case I'm trying to display the "actual free" memory. Note that no data is displayed (another newbie issue).
I'm brand new to ElasticSearch with Grafana so my attempt I'm sure looks lame to more experienced folks. Any help would be greatly appreciated.
I am using the 7.5.0-beta Docker image.
While trying to change a Visualization in Kibana to use another property for the x-axis, that property doesn't appear there.
I changed recently nlog to target elastic search using the Elastic common schema.
After that change the property is not longer called ResolvedRoute but instead _metadata.resolved_route, the problem is that it doesn't appear on the field for x-axis, it says no matches found.
It is not on the available fields
I'm still new to elastic search and kibana, so it's possible i'm missing something simple.
Don't know if it's related, but when on Discovermenu, looking at the Available fields all of _metadata fields have a question mark
I'm already trying to map some of these fields in Index Management / Edit template
Also, if i go to the console and type
GET /logstash-2020.11.25/_search
{
"query": {
"match_all": {}
}
}
I can see the fields of _metadata that i want, inside _source which is inside of hits.
I think i already had a similar problem where i had to delete all indexes that match the pattern and then the field appeared, but that doesn't make much sense.
What could be the problem?
Chances are high that you haven't refreshed the corresponding index pattern in Kibana. Therefore the data might exist as documents in Elasticsearch but not yet as a field in the index pattern, which is a Kibana Saved Object.
Please go to Settings / Stack Management (depending on your Kibana version), click on the index pattern you expect the field to be in and refresh the fields list (icon is in the upper right corner).
Please let me know if that solved your problem.
The fields in question were not correctly mapped in the template.
since metadata is an object it needs to be mapped like that first,
then inside of it we can map it's own properties.
When I'm looking at my index in Kibana there are many representation in the index for the same field. Below is an example:
What I'm wondering about is, can I configure a way to hide the values that end users in Kibana don't need to see. I don't see anything in the "edit" section for each field that enables me to do so.
I'm trying to make kibana as user friendly as possible for end users and having 2 different representations is going to be confusing.
Is it something I need to configure in the Mapping? Sorry I'm just getting used to the new Kibana interface.
You can filter out a field by adding it to the source filters on the Index Pattern. Management > Index Patterns > Source Filters
I am new to Kibana and using it for visualising the data present in Elastic Search.
I am trying to create dynamic dashboard i.e. by using saved search indexes having field values as variable.
What I want
Want to use place holders in the query which can be populated from URL parameters and then search results rendered in dashboard.
So that user can search results by providing some input instead of fixed query.
Can it be done in Kibana? If not, is there any better visualisation tool other than Kibana to serve this purpose.
In the Dashboard View there is actually a searchbar where you can just fire normal Matchqueries and its easy to filter i.e:
Create a table with terms aggregation for one of the fields a user might be interested in.
Click on one of the Terms in the Dashboard
A filter can be seen under the searchbar and all elements in the dashboard will be filtered with it.
If you have line charts users can zoom into the charts to see only information of the zoomed in timeframe
Barcharts are interactive like tables
Play around a bit. Kibana is very powerful you just have to find the right visualizations.
I want to know if there is a way to update elasticsearch documents after filtering them out.
Let's say I have a user collection with following documents:
[
{ "name":"u1","age":23},
{ "name":"u2","age":31},
{ "name":"u3","age":27},
{ "name":"u4","age":33}
]
Now what I need to do is update the names of all the users who have ages above 30.
Looking at a lot of documentation and searching for hours on google, including the following document
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_updating_documents.html
I couldn't find a way to do it. So if we look into the docs, we are providing the id of the document, so it doesn't suite my need. Is there a way to do this sort do this sort of stuff in Elasticsearch?
From the link you provided:
Note that as of this writing, updates can only be performed on a
single document at a time. In the future, Elasticsearch will provide
the ability to update multiple documents given a query condition (like
an SQL UPDATE-WHERE statement).
So, this is not supported at the moment. But you can consider taking a look at this plugin: https://github.com/yakaz/elasticsearch-action-updatebyquery/.