Running Kibana3, LogStash and ElasticSearch, all in one machine - elasticsearch

Kibana3 works successfully when ElasticSearch is in a different machine, by setting elasticsearch: "http://different_machine_ip:9200" in config.js of Kibana3.
Now , I want to run all three of them in my local machine for testing. I'm using Windows7 and using Chrome browser. I installed Kibana 3 on Tomcat7. I started the embedded ElasticSearch from LogStash jar file.
I set the ElasticSearch location to "localhost:9200" or "127.0.0.1:9200" or "computer_name:9200". When I check Kibana3 on the browser, the ElasticSearch query revealed via spying has no logstash index.
curl -XGET 'http://localhost:9200//_search?pretty' -d ''
As you can see, the index part is empty, showing // only. The expected query should look like this.
curl -XGET 'http://localhost:9200/logstash-2013.08.13/_search?pretty' -d 'Some JSON Data'
The browser is able to call ElasticSearch API successfully. Example, typing http://localhost:9200/logstash-2013.08.13/_mapping?pretty=true on the address bar returns the mapping of logstash index. This proves that there is no probelm in connecting to ElasticSearch.
The problem here is that the index is empty from Kibana query. Why is the index empty?


Kibana 3 works differently from Kibana 1 and 2. It runs entirely in the browser.
The config file is read by javascript and executed in your browser, so localhost:9200 tells Kibana to look for ElasticSearch running on the laptop in front of you, not the server.
BTW - Recent versions of LogStash has Kibana bundled, so you don't have to host it independently.

Related

In Elassandra not able to access elastic search url

Hi, I am new to Elassandra. I want to make setup(windows 10) and hit queries from elastic search url in stored documents. I have installed Elassandra and start it is working fine but i am not able to access elastic search url. I also tried to configure host and http.port in elasticsearch.yml but it did not work.
from bin i am running cassandra -f -e. here is no error in logs but still not able to access ES on localhost:9200
Please help me out on the steps.
Thanks in advance.

Where the elasticsearch data is stored?

I've installed filebeat in a server, collecting all the logs from all the containers i have. With filebeat i indicate to which elasticsearch and kibana hosts he must send them (both, elasticsearch and kibana are running as a service in another server). So now all the logs appear in kibana. My question is, all those logs that appear there, are stored somewhere? In elasticsearch or in kibana?
Thank you in advance

All the data is stored inside Elasticsearch.
Kibana is a visualization engine on top of Elasticsearch. Kibana itself also stores its configuration data inside an internal Elasticsearch index called .kibana.
Whatever you can see from Kibana always comes from Elasticsearch.
You can learn more about Elasticsearch here and Kibana here.

Running two different ES on a same machine and configuring kibana accordingly

I have installed two ES on my machine. One is 5 version(localhost:9200) and the other is 6 version(localhost:9500). Version 5 is used to index and store data alone while Version 6 is used to do some analytics using kibana dashboards.When i start kibana, the kibana automatically stops stating that all the ES should be on the same version. Is there any way, i can stop kibana from reading localhost:9200 ?

like #Abhijit Bashetti stated in the comment, you need to modify kibana.yml file in order to point kibana to the elasticsearch instance you wish.
you should change "localhost:9200" to "localhost:9500" in order for kibana to reach the ES v6.

Filebeat > is it possible to send data to Elasticsearch by means of Filebeat without Logstash

I am a newbie of ELK. I installed first Elasticsearch and Filebeat without Logstash, and I would like to send data from Filebeat to Elasticsearch. After I installed the Filebeat and configured the log files and Elasticsearch host, I started the Filebeat, but then nothing happened even though there are lots of rows in the log files, which Filebeats prospects.
So is it possible to forward log data directly to Elasticsearch host without Logstash at all? I

It looks like your ES 2.3.1 is only configured to be reachable from localhost (default since ES 2.0)
You need to modify your elasticsearch.yml file with this and restart ES:
network.host: 168.17.0.100
Then your filebeat output configuration needs to look like this:
output:
elasticsearch:
hosts: ["168.17.0.100:9200"]
Then you can check in your ES filebeat-* indices that you're getting the new log data (i.e. the hits.total count should increase over time):
curl -XGET 168.17.0.100:9200/filebeat-*/_search

Elasticsearch is still initializing the kibana index

When I am trying to start Kibana I am facing the following issue. I first restarted my elasticsearch server it was running successfully. After starting Elasticsearch I tried to start Kibana but no luck.
{"name":"Kibana","hostname":"ABCD","pid":3848,"level":30,"msg":"Elasticsearch is still initializing the kibana index... Trying again in 2.5 second.","time":"2015-07-03T07:35:34.936Z","v":0}
Thanks in advance

the curl -XDELETE http://localhost:9200/.kibana command works fine, however you lose all your Kibana settings (indexes, graphs, dashboards); by just querying the index I've solved the problem, without losing my data. For example:
curl -s http://localhost:9200/.kibana/_recovery?pretty
curl -XPUT 'localhost:9200/.kibana/_settings' -d '
{
"index" : {
"number_of_replicas" : 0
}
}'
Then start Kibana, it should work.
Gael Le Moellic

Warning: Removing .kibana index will make you lose all your kibana settings (indexes, graphs, dashboards)
This behavior is sometimes caused by an existing .kibana index.
Kindly delete the .kibana index in elasticsearch using following command:
curl -XDELETE http://localhost:9200/.kibana
After deleting the index, restart Kibana.
If the problem still persists, and you are willing to lose any existing data, you can try deleting all indexes using following command:
curl -XDELETE http://localhost:9200/*
Followed by restarting Kibana.
Note: localhost:9200 is the elasticsearch server's host:port, which may be different in your case.

Sometimes you need to wait few minutes after restarting ES.
That can be also connected with low disk space.
Observed on AWS t2.small machine with ELK stack.

Something is wrong with your Kibana index inside elasticsearch.
I had the same message and i just deleted my Kibana index and then,
when i restarted it, a new index of Kibana was created by the service.

Resources