I have set-up an elasticsearch server and I am feeding data to it with logstash. My data consists of various numeric fields which are logged through time. I want to create a chart in Kibana that will display the raw data as points, and the average of my data as a line chart on top of that. Pretty much something like the picture below... (This is from Kibana's homepage)
I have a real problem displaying the raw values of my data and I cannot find a way to do it in the documentation. Has anybody else encountered the same problem? Is this doable in Kibana?
Related
I created visualizations on fly through curl. However, it does not display data.
I have created visualizations by doing as follows
1. Exporting an already built visualization in JSON
2. Modifying the index_name and field names of JSON
3. Importing back the visualization
The visualizations are created in Kibana, they have the same visualization name and refer the index pattern and fields I want. However it does not display any data (graphs)
Can anyone explain what is the problem with the way I am doing ?
The problem was with the variable. I used for find and replacs. The variable I used was also being used to filter data and hence replacing it was not showing any data. My bad
I am new for developing the ELK. So, I have an idea to do with the Elasticsearch data with some dashboard design using Kibana. I have tried we are able to create dashboards like pie, graph etc. However, they are all based on the count and average related. So, I want data like table with whole data. But, a way of the form the query based on that it will generate table. I would like to know whether it is possible in Kibana?
Wondering if there is a way to get list of the messages related to a Kibana visualization. I understand if I apply the same filter on the "Discover", which is on "Visualization", I can filter the related messages. But I want to have more direct user experience like an user clicks on a region of a graph and can get the related messages which formed that region. Is there any way to do it?
This helped me:
https://discuss.elastic.co/t/can-i-get-the-related-messages-from-a-kibana-visualization/101692/2
It says:
Not directly, unfortunately. You can click on the visualization to create a filter, and you can pin that filter and take it to discover, which will do what you're asking, but isn't very obvious.
The reason is that visualizations are built using aggregate data, so they don't know what the underlying documents are, they only know the aggregate representation of the information. For example, if you have a bunch of traffic data, and you are looking at bytes over time, the records get bucketed by time and the aggregate of the bytes in that bucket are shown (average, sum, etc.).
In contrast, Discover only works with the raw documents, showing you exactly what you have stored in Elasticsearch. Both documents and aggregations can use filters and queries, which is why you can create a filter in one and use it in the other, but the underlying data is not the same.
I am trying to visualize time series data stored in elastic search using grafana.
I have the legend setup to show 2 decimal places but it does not reflect in the UI.
The decimal places show up for other dashboard panels with a tsdb datasource. So this issue is specific to using grafana with elasticsearch. Is there any other configuration setup I am missing here which will help me achieve this?
Just found out that elastic search does not allow displaying values without some sort of aggregation and in my case aggregation is resulting in values getting rounded.
There was a related request which seemed to not get much traction in kibana.
https://github.com/elastic/kibana/issues/3572
In short not feasible as of [2.x] elastic search.
How do Elasticsearch and Solr compare in respect to the following:
Indexing logs.
Indexing events.
Indexing PDF documents.
Ease of creating and distributing visualizations. Kibana vs Banana.
Support and documentation for developers.
Any help is appreciated.
EDIT
More specifically, i am trying to figure out how exactly a PDF document or an event can be indexed at all. I have worked a little bit on Elasticsearch and since i am a fan of JSON, i found it quite useful when i tried to index structured data.
For example logs are mostly structured and thus i guess easier to index and search. Now what if i want to index the whole log file itself?
Follow up
Is Kibana the only visualization tool available for Elasticsearch?
Is Banana the only visualization tool available for Solr?
Here is an answer to try to address just the Elasticsearch aspect of the post.
Take a look at https://github.com/elastic/elasticsearch-mapper-attachments for handling PDFs
For events/logs, you would need to transform those into structured data to index in Elasticsearch. You can have a field in there for the source (the log file the data came from and other information like that) - you will have all the data in the whole log file indexed in that fashion. You can take advantage of ES aggregations to group results based on log file, calculate statistics, etc.
The ELK stack is definitely worth a look.
I don't know if Kibana is the only visualization tool but it is probably the most popular and likely to offer more than something else.