I am starting to use ES and Kibana, so apologies in advance if this question doesn't make sense!
I'd like to be able to see in Kibana a list of my current indices, similar to what you get with:
curl 'localhost:9200/_cat/indices?v'
I was expecting to be able to see in Kibana functionality partly like a DB client where you can connect to a DB server and see all the databases, then drill down in each of them to see tables and content. I'd love to have that kind of workflow in Kibana.
The closest I can find is in Management -> Index Pattern, but it'll display a list of all fields, which is too much information and I can't see any column in the table that points to which index each field belongs.
As I said I'm just starting so it might be I'm not looking in the right place!
I don't think we have any option to see the hierarchy like you see in traditional DB application.
If you are looking for something in kibana which can give you information similar to curl 'localhost:9200/_cat/indices?v', then you can go to "Monitoring-> Indices" which will list out all the index with there stat's(document count, data size, index rate etc).
If you don't have the x-pack installed then you have to use the "Discover" tab where you can see the list of all the index from the drop down, also the _type and all the available fields in the index.
Related
I have an ElasticSearch instance and it does one type of search - it takes a few parameters and returns the companies in its index that match the parameters given.
I'd like to be able to pull some stats that essentially says "This company has been returned from search queries X number of times in the past week".
Does ElasticSearch store metadata that will allow to pull this kind of info from it? If this kind of data isn't stored in ES out of the box, is there a way to enable it?
Elasticsearch (not ElasticSearch ;) ) does not do this natively, no. you can build something using the slow log, where you set the timing to 0 to get it to log everything, but that then logs everything which may not be useful/too noisy
things like https://www.elastic.co/enterprise-search, built on top of Elasticsearch, do provide this sort of insight
Can I put the response result that I query in Kibana dev tools into elasticsearch directly?
Or must I write a script to achieve it?
Any recommends?
Ok So here is one basic understanding after discussion.
Please observe carefully.
If you have head plugin installed for ES .
search for .kibana index .
open the .kibana index and you will have all the designed dashboards listed there with processd info.
Think ES as another Storage from where you can read the data and put that data into Another ES index.
Refer to this link :
https://www.elastic.co/blog/kibana-under-the-hood-object-persistence
Tools you can opt is Logstash for Reading and writing.
Grok pattern learning can give you good lead about that.
Tell me if need some real time pics for same problem.
Happy learning.
It is like you cook in kitchen and ask to put the cooked food in kitchen again.If you cooked food better consume it :)
See the visualization or processed data you see on kibana end is just for kibana.The algorithms or processing techniques for the data set residing at elastic search will be applied over the upcoming data set.
So offcourse you can put/consume your data in Elastic search back again.
It depends what sort of requirement you are facing.
Note : Data in elastic search(inverted index) after kibana processing not gonna change its architecture, due to which you are able to apply another processing techniques from kibana over the same index assuming that data is in it's earlier state.
I want to automate the creation of a set visualizations for new kibana/elasticsearch installations.
So I need to know if I can automate this, independent the programming language.
There are no APIs yet in Kibana to manage the searches, visualizations and dashboards. Some feature requests have been suggested (here and here) but they are still being discussed.
However, since Kibana visualizations are stored in the .kibana index with the visualization mapping type, you can definitely GET them, learn how they are built, modify them and PUT them again.
For a visualization named "Top consumers by country", you can get the visualization spec using
curl -XGET http://localhost:9200/.kibana/visualization/Top-consumers-by-country
You'll get a document containing the title of your visualization, another field called visState containing the specification of your visualization (obvisouly different for each visualization) and finally a field named kibanaSavedObjectMeta which contains the Elasticsearch query and index details.
You can also view/edit/export the same data in Settings > Objects > Visualizations
Let's say I have an index test and which already exists. I want to add a new field newfield1 with some data for all documents in the database. Currently I am simply deleting all everything and then reinserting the data with the newfield1 data added in. I understand this isn't the most efficient way, but that's not my question right now.
Sometimes the data in newfield1 does not get indexed and I can't visualize it in Kibana. It's pretty annoying. Is there something wrong with what I'm doing?
NOTE: I CAN query this field in ElasticSearch which makes me think there's a problem with Kibana
Kibana caches the field mapping. Go to Settings -> Indices, select your index, and click the orange "Refresh" button.
Not much to go on here but first make sure your cluster is Green.
$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
If you are still struggling to understand the state of you cluster then perhaps consider installing on of the plugins like HQ https://github.com/royrusso/elasticsearch-HQ
Our application exposes various metrics through an MBean, these are pulled by logstash and saved to ElasticSearch. Typical metrics are "number_of_connected_users", connection status for external dependencies etc. I'd like to visualize this, but I can't see that Kibana supports displaying only the last query result, discarding the older ones.
I figure I need to either 1) Limit a query to return only one result and then visualize it, or 2) Have the query return multiple results and selecting the last one in a visualization. Anyone done this?
[ Deleted old answer as your question makes more sense today! ]
Sadly, kibana doesn't support the idea of a "latest" record.
Here's their issue for it.