I am new to logstash. I have set up my logstash to populate elastic search and have Kibana read out of it. The problem I am facing is that after the
number of records = results per page x page limit
the UI stops getting new results. Is there a way to set Kibana up such that it discards the old results instead of the latest after the limit is reached?
To have kibana read the latest results, reload the query.
To have more pages available (or more results per page), edit the panel.
Make sure the table is reverse sorted by #timestamp.
Related
I tried to enable logs in the elastic search server using the below link
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html
I verified my index setting using the url
http://localhost:9200/_all/_settings
The result is below
{"myindex":{"settings":{"index":{"search":{"slowlog":{"threshold":{"fetch":{"warn":"1ms","trace":"1ms","debug":"1ms","info":"1ms"},"query":{"warn":"1ms","trace":"1ms","debug":"1ms","info":"1ms"}}}},"number_of_shards":"3","provided_name":"occindex","creation_date":"1508319257925","number_of_replicas":"2","uuid":"dVAWgk62Sgivzr2B_OuCzA","version":{"created":"5040399"}}}}}
As per the document, I expect the logs to be populated when the threshold is breached.
I have set 1 ms as the threshold in order to log all queries that are hitting elastic search
I observed that under logs folder , the log files elasticsearch_index_search_slowlog.log and elasticsearch.log does not show the queries which are hitting elastic search.
Let me know if my configuration is correct.
The log worked after I inserted one record.
If you fire the query when there are no records in the index , the log was not updated
I have a problem with Kibana: Dashboard and visualizations don't show any results!
As you can see in this screenshot, in the discover tab I get some results! Which means data exists in my index "as-*". right? But I used a trick in order to display this data :
1) I changed the range to "Today" ==> it shows no result found !
2) I clicked "New" button ==> then i get my data displayed!
Is there an other way "more proper" to get data displayed?
Then in the my dashboard (or visualization) I can't get any results!! even if the range is the same as discover tab!
I restarted Kibana ==> no changes!
I deleted as-* then I created it => no changes!
I'm using Curator to create daily index and logstash to index the data into ES!
I'm stuck here ! I ll be glad if you can help me figure it out !
Thank you very much!
I am fairly new to elasticsearch and Kibana, but here are three mistakes that I made in the past:
Are you using the correct index? Make sure that the index you have chosen for the logs that are displayed on the visualise page and for the actual visualisations is the same.
Correct timeperiod: does the time period you have chosen contain the data you are looking for? Or did you happen to have zero logs during that time?
Correct filters and aggregations: when you were making the visualisations, did they show any results? Or were they empty from the beginning? Maybe one of your filters or aggregations is wrong and it's excluding the results you're expecting to see.
Not sure if this is any help, hope you've sold the problem by now :)
If you can see information in "discover", it means that kibana has connected to the database, and that the database has information. You shouldn't have to click the "New" button to see information in the discover view. I believe the "New" button in the "discover" page is used to create a new search.
Maybe try zooming on the time period of the data on the "discover" page, or
try checking the systems logs to see if logstash is successfully pushing information to Elasticsearch.
With Kibana, dashboards are made up of visualisations, and visulations are made up of searches.
The "No results found" on the dashboard page, shown in your second page is due to the visualisation having no results. I guess you imported visulations into Kibana.
I hope that helps.
I am very new to Elasticsearch and Kibana. I have setup an index with a number of documents that I want to monitor as a table on the dashboard.
The JSON documents stored in elastic search simply have four fields.
I created/saved a search on the discover tab with all four fields and then added that search to my dashboard.
I then set the auto refresh to every 5 seconds.
If I update an existing document the table does NOT change on the next refresh.
If I add/remove new documents to elasticsearch they appear/disappear in the table on the next refresh.
Table flickers every 5s so it is clearly re-rendering.
Examining the network traffic shows that the response to the query contains the new data.
Pressing the arrow drop-down next to the row in the table shows the correct data
Refreshing the entire page shows the new data
Am I missing something?
With Kibana 4.4.2 adding a Search to the Dashboard does not seem to update dynamically. This may or may not be by-design or a bug.
But thanks to the elasticsearch forums I did get this response.
Summary: Creating a table visualization and splitting the rows and adding each individual term as updates when the data changes.
The query is send to solr is below:
http://1.2.3.4:8983/solr/data.results/select?q=*:*&fq=date:[2016-01-01T00:00:00Z TO 2015-01-31T23:59:59Z]&sort=publishdate desc&wt=json&indent=true
This gives me 10 results. Now of those 10 results, I update the latest record as isdeleted=true and I pass a query to Solr &fq=-isdeleted:true. But it still gives me same 10 results as before. Now in the date range, I change start range to 2016-01-02T00:00:00Z instead of 2016-01-01T00:00:00Z (fq=date:[2016-01-02T00:00:00Z TO 2015-01-31T23:59:59Z]), then the latest record which is set to true does not shows up. I guess my output of [2016-01-01T00:00:00Z TO 2015-01-31T23:59:59Z] has got saved in the cache and hence it is showing the same results. This has been occuring repeadtedly, I have tried with many different queries. How can I refresh the cache so that I get the latest updated records? Is there any configuration change to make in solr so as to always get the latest updated records?
I have Kibana 4.0.1 running on top of elasticsearch 1.4.4. It was very smooth and virtually had no setup time. Suddenly I have run into a problem.
If I add a new field in my elasticsearch index, it's not visible in fields section. I can still query on that field in discover section. But, I can't make a graph based on the new field as it's not visible in fields list.
Kibana apparently fetches _mapping at the time of setup and stores it in elasticsearch index named .kibana. Once done, it never changes that. Deleting this index should load fresh _mapping from elasticsearch. But I don't want to lose all the saved dashboards and visualizations.
Is there a was to force Kibana to load fresh mapping at regular interval?
Yes in the settings tab you can refresh the index. Check the yellow refresh botton in the image below.