Elasticsearch working with no issues on http://localhost:9200
And Operating system is Ubuntu 18.04
Here is the error log for Kibana
root#syed-MS-7B17:/var/log# journalctl -fu kibana.service
-- Logs begin at Sat 2020-01-04 18:30:58 IST. --
Apr 03 20:22:49 syed-MS-7B17 kibana[7165]: {"type":"log","#timestamp":"2020-04-03T14:52:49Z","tags":["fatal","root"],"pid":7165,"message":"{ Error: listen EADDRNOTAVAIL: address not available 7.0.0.1:5601\n at Server.setupListenHandle [as _listen2] (net.js:1263:19)\n at listenInCluster (net.js:1328:12)\n at GetAddrInfoReqWrap.doListen (net.js:1461:7)\n at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:61:10)\n code: 'EADDRNOTAVAIL',\n errno: 'EADDRNOTAVAIL',\n syscall: 'listen',\n address: '7.0.0.1',\n port: 5601 }"}
Apr 03 20:22:49 syed-MS-7B17 kibana[7165]: FATAL Error: listen EADDRNOTAVAIL: address not available 7.0.0.1:5601
Apr 03 20:22:50 syed-MS-7B17 systemd[1]: kibana.service: Main process exited, code=exited, status=1/FAILURE
Apr 03 20:22:50 syed-MS-7B17 systemd[1]: kibana.service: Failed with result 'exit-code'.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: kibana.service: Service hold-off time over, scheduling restart.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: kibana.service: Scheduled restart job, restart counter is at 2.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: Stopped Kibana.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: kibana.service: Start request repeated too quickly.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: kibana.service: Failed with result 'exit-code'.
Apr 03 20:22:53 syed-MS-7B17 systemd[1]: Failed to start Kibana.
I have resolved it myself after checking the /etc/hosts file
It was edited by mistake like below
7.0.0.1 localhost
Related
I am new to Kibana and how its setup.
We are testing setting up Kibana on an Azure VM with Ansible playbooks, all seems to be fine but unfortunately I think during our troubleshooting we made a mistake somewhere and now the Kibana service will not start. The VM is running CentOs, the error we get is
Dec 07 09:47:06 es-vm1 systemd[1]: Started Kibana.
Dec 07 09:47:06 es-vm1 systemd[1]: kibana.service: main process exited, code=exited, status=1/FAILURE
Dec 07 09:47:06 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:06 es-vm1 systemd[1]: kibana.service failed.
Dec 07 09:47:07 es-vm1 systemd[1]: Stopped Kibana.
Dec 07 09:47:07 es-vm1 systemd[1]: Started Kibana.
Dec 07 09:47:07 es-vm1 kibana[11134]: internal/fs/utils.js:332
Dec 07 09:47:07 es-vm1 kibana[11134]: throw err;
Dec 07 09:47:07 es-vm1 kibana[11134]: ^
Dec 07 09:47:07 es-vm1 kibana[11134]: Error: EACCES: permission denied, open '/etc/kibana/kibana.yml'
Dec 07 09:47:07 es-vm1 kibana[11134]: at Object.openSync (fs.js:497:3)
Dec 07 09:47:07 es-vm1 kibana[11134]: at readFileSync (fs.js:393:35)
Dec 07 09:47:07 es-vm1 kibana[11134]: at readYaml (/usr/share/kibana/node_modules/#kbn/apm-config-loader/target_node/utils/read_config.js:25:69)
Dec 07 09:47:07 es-vm1 kibana[11134]: at getConfigFromFiles (/usr/share/kibana/node_modules/#kbn/apm-config-loader/target_node/utils/read_config.js:57:18)
Dec 07 09:47:07 es-vm1 kibana[11134]: at loadConfiguration (/usr/share/kibana/node_modules/#kbn/apm-config-loader/target_node/config_loader.js:30:58)
Dec 07 09:47:07 es-vm1 kibana[11134]: at initApm (/usr/share/kibana/node_modules/#kbn/apm-config-loader/target_node/init_apm.js:18:64)
Dec 07 09:47:07 es-vm1 kibana[11134]: at module.exports (/usr/share/kibana/src/cli/apm.js:27:3)
Dec 07 09:47:07 es-vm1 kibana[11134]: at Object.<anonymous> (/usr/share/kibana/src/cli/dist.js:10:17)
Dec 07 09:47:07 es-vm1 kibana[11134]: at Module._compile (internal/modules/cjs/loader.js:1085:14)
Dec 07 09:47:07 es-vm1 kibana[11134]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10) {
Dec 07 09:47:07 es-vm1 kibana[11134]: errno: -13,
Dec 07 09:47:07 es-vm1 kibana[11134]: syscall: 'open',
Dec 07 09:47:07 es-vm1 kibana[11134]: code: 'EACCES',
Dec 07 09:47:07 es-vm1 kibana[11134]: path: '/etc/kibana/kibana.yml'
Dec 07 09:47:07 es-vm1 kibana[11134]: }
Dec 07 09:47:07 es-vm1 systemd[1]: kibana.service: main process exited, code=exited, status=1/FAILURE
Dec 07 09:47:07 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:07 es-vm1 systemd[1]: kibana.service failed.
Dec 07 09:47:10 es-vm1 systemd[1]: kibana.service holdoff time over, scheduling restart.
Dec 07 09:47:10 es-vm1 systemd[1]: Stopped Kibana.
Dec 07 09:47:10 es-vm1 systemd[1]: Started Kibana.
Dec 07 09:47:11 es-vm1 kibana[11149]: internal/fs/utils.js:332
Dec 07 09:47:11 es-vm1 kibana[11149]: throw err;
Dec 07 09:47:11 es-vm1 kibana[11149]: ^
Dec 07 09:47:11 es-vm1 kibana[11149]: Error: EACCES: permission denied, open '/etc/kibana/kibana.yml'
Dec 07 09:47:11 es-vm1 kibana[11149]: at Object.openSync (fs.js:497:3)
Dec 07 09:47:11 es-vm1 kibana[11149]: at readFileSync (fs.js:393:35)
Dec 07 09:47:11 es-vm1 kibana[11149]: at readYaml (/usr/share/kibana/node_modules/#kbn/apm-config-loader/target_node/utils/read_config.js:25:69)
Dec 07 09:47:11 es-vm1 systemd[1]: kibana.service: main process exited, code=exited, status=1/FAILURE
Dec 07 09:47:11 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:11 es-vm1 systemd[1]: kibana.service failed.
Dec 07 09:47:14 es-vm1 systemd[1]: kibana.service holdoff time over, scheduling restart.
Dec 07 09:47:14 es-vm1 systemd[1]: Stopped Kibana.
Dec 07 09:47:14 es-vm1 systemd[1]: start request repeated too quickly for kibana.service
Dec 07 09:47:14 es-vm1 systemd[1]: Failed to start Kibana.
Dec 07 09:47:14 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:14 es-vm1 systemd[1]: kibana.service failed.
The permissions on the yml file are as follows
-rwxrwx---. 1 root kibana 130 Dec 2 14:04 kibana.keystore
-rw-r--r--. 1 root root 5089 Dec 7 09:47 kibana.yml
-rw-r--r--. 1 root kibana 216 Nov 4 13:30 node.options
The systemctl status
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Tue 2021-12-07 09:47:14 UTC; 9min ago
Docs: https://www.elastic.co
Process: 11149 ExecStart=/usr/share/kibana/bin/kibana --logging.dest="/var/log/kibana/kibana.log" --pid.file="/run/kibana/kibana.pid" (code=exited, status=1/FAILURE)
Main PID: 11149 (code=exited, status=1/FAILURE)
Dec 07 09:47:11 es-vm1 systemd[1]: kibana.service: main process exited, code=exited, status=1/FAILURE
Dec 07 09:47:11 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:11 es-vm1 systemd[1]: kibana.service failed.
Dec 07 09:47:14 es-vm1 systemd[1]: kibana.service holdoff time over, scheduling restart.
Dec 07 09:47:14 es-vm1 systemd[1]: Stopped Kibana.
Dec 07 09:47:14 es-vm1 systemd[1]: start request repeated too quickly for kibana.service
Dec 07 09:47:14 es-vm1 systemd[1]: Failed to start Kibana.
Dec 07 09:47:14 es-vm1 systemd[1]: Unit kibana.service entered failed state.
Dec 07 09:47:14 es-vm1 systemd[1]: kibana.service failed.
Kibana user does exist also
uid=995(kibana) gid=991(kibana) groups=991(kibana)
Could anyone point me in the right direction? What should I do here? I've tried playing around with permissions on the file but the error always seems to be the same.
Solution Update: I was able to resolve this by doing the following chmod 2750 kibana from /etc/ directory it was an error on my part while troubleshooting on the /etc/kibana directory.
Working on a Kibana deployment, after installing Kibana & Elasticsearch i get the error 'Kibana server is not ready yet'.
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-centos-7
[opc#homer7 etc]$
[opc#homer7 etc]$ sudo systemctl status kibana
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2021-02-26 13:56:07 CET; 37s ago
Docs: https://www.elastic.co
Main PID: 18215 (node)
Memory: 208.3M
CGroup: /system.slice/kibana.service
└─18215 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist --logging.dest="/var/log/kibana/kibana.log" --pid.file="/run/kibana/kibana.pid"
Feb 26 13:56:07 homer7 systemd[1]: kibana.service failed.
Feb 26 13:56:07 homer7 systemd[1]: Started Kibana.
[opc#homer7 etc]$
[opc#homer7 etc]$
[opc#homer7 etc]$
[opc#homer7 etc]$ sudo journalctl --unit kibana
-- Logs begin at Fri 2021-02-26 11:31:02 CET, end at Fri 2021-02-26 13:56:57 CET. --
Feb 26 12:15:38 homer7 systemd[1]: Started Kibana.
Feb 26 13:21:25 homer7 systemd[1]: Stopping Kibana...
Feb 26 13:22:55 homer7 systemd[1]: kibana.service stop-sigterm timed out. Killing.
Feb 26 13:22:55 homer7 systemd[1]: kibana.service: main process exited, code=killed, status=9/KILL
Feb 26 13:22:55 homer7 systemd[1]: Stopped Kibana.
Feb 26 13:22:55 homer7 systemd[1]: Unit kibana.service entered failed state.
Feb 26 13:22:55 homer7 systemd[1]: kibana.service failed.
Feb 26 13:25:05 homer7 systemd[1]: Started Kibana.
Feb 26 13:25:29 homer7 systemd[1]: Stopping Kibana...
Feb 26 13:26:59 homer7 systemd[1]: kibana.service stop-sigterm timed out. Killing.
Feb 26 13:26:59 homer7 systemd[1]: kibana.service: main process exited, code=killed, status=9/KILL
Feb 26 13:26:59 homer7 systemd[1]: Stopped Kibana.
Feb 26 13:26:59 homer7 systemd[1]: Unit kibana.service entered failed state.
Feb 26 13:26:59 homer7 systemd[1]: kibana.service failed.
Feb 26 13:27:56 homer7 systemd[1]: Started Kibana.
Feb 26 13:40:53 homer7 systemd[1]: Stopping Kibana...
Feb 26 13:42:23 homer7 systemd[1]: kibana.service stop-sigterm timed out. Killing.
Feb 26 13:42:23 homer7 systemd[1]: kibana.service: main process exited, code=killed, status=9/KILL
Feb 26 13:42:23 homer7 systemd[1]: Stopped Kibana.
Feb 26 13:42:23 homer7 systemd[1]: Unit kibana.service entered failed state.
Feb 26 13:42:23 homer7 systemd[1]: kibana.service failed.
Feb 26 13:42:23 homer7 systemd[1]: Started Kibana.
Feb 26 13:44:09 homer7 systemd[1]: Stopping Kibana...
Feb 26 13:45:40 homer7 systemd[1]: kibana.service stop-sigterm timed out. Killing.
Feb 26 13:45:40 homer7 systemd[1]: kibana.service: main process exited, code=killed, status=9/KILL
Feb 26 13:45:40 homer7 systemd[1]: Stopped Kibana.
Feb 26 13:45:40 homer7 systemd[1]: Unit kibana.service entered failed state.
Feb 26 13:45:40 homer7 systemd[1]: kibana.service failed.
Feb 26 13:45:40 homer7 systemd[1]: Started Kibana.
Feb 26 13:54:37 homer7 systemd[1]: Stopping Kibana...
Feb 26 13:56:07 homer7 systemd[1]: kibana.service stop-sigterm timed out. Killing.
Feb 26 13:56:07 homer7 systemd[1]: kibana.service: main process exited, code=killed, status=9/KILL
Feb 26 13:56:07 homer7 systemd[1]: Stopped Kibana.
Feb 26 13:56:07 homer7 systemd[1]: Unit kibana.service entered failed state.
Feb 26 13:56:07 homer7 systemd[1]: kibana.service failed.
Feb 26 13:56:07 homer7 systemd[1]: Started Kibana.
[opc#homer7 etc]$
[opc#homer7 etc]$
check $systemctl status elasticsearch. I am guessing your elasticsearch service is not started yet.
I guess there are many factors that need to be checked, first of all please go to the config directory of where you installed Kibana and check the kibana.yml by sudo vi kibana.yml and check the port of elastic server that Kibana tries to connect(the default is 9200).
Here is an example of default configuration.
After matching this configuration with your need go to the script file that you save in for Kibana service and check the the [unix] part to if it needs activate elastic service first and if you didn't add "Required" part for Elasticserver make sure that the elastic server is up and run before running Kibana as service, you can also lunch Kibana as shell by going to the bin director of Kibana and lunching Kibana .
Maybe The issue happened due to kibana was unable to access elasticsearch locally.
I think that you have enabled xpack.security plugin for security purpose at elasticsearch.yml by adding a new line :
xpack.security.enabled : true
if so you need to uncomment the two lines on kibana.yml :
#elasticsearch.username: "kibana"
#elasticsearch.password: "pass"
elasticsearch.username = "kibana_system"
elasticsearch.password = "your-password"
after saving the changes, restart kibana service :
sudo sservice kibana restart
Hi after downloading the latest rpm for CentIS and and installing for the first time I am getting this error in the logs:
Jun 22 09:47:31 ssd316r.simpleservers.co.uk systemd[1]: Starting Elasticsearch...
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd-entrypoint[2501]: ERROR: Temporary file directory [/usr/share/elasticsearch/tmp] does not exist or is not accessible
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd[1]: elasticsearch.service: main process exited, code=exited, status=78/n/a
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd[1]: Failed to start Elasticsearch.
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd[1]: Unit elasticsearch.service entered failed state.
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd[1]: elasticsearch.service failed.
Error is due to below log:
Jun 22 09:47:32 ssd316r.simpleservers.co.uk systemd-entrypoint[2501]:
ERROR: Temporary file directory [/usr/share/elasticsearch/tmp] does
not exist or is not accessible
Can you check /usr/share/elasticsearch/tmp is present on your server or not, if not please create this folder at the same location and make sure your elasticsearch process has write access to it.
I want to configure Kibana, so, that I can access over https.
I did following changes in Kibana config file (/etc/kibana/kibana.yml):
server.host: 0.0.0.0
server.ssl.enabled: true
server.ssl.key: /etc/elasticsearch/privkey.pem // Using same SSL that I created for elasticsearch
server.ssl.certificate: /etc/elasticsearch/cert.pem // Using same SSL that I created for elasticsearch
elasticsearch.url: https://127.0.0.1:9200
elasticsearch.ssl.verificationMode: none
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opendistro_security.multitenancy.enabled: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.readonly_mode.roles: ["kibana_read_only"]
When I restart/start Kibana, it's giving me below error:
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; disabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Wed 2019-06-05 14:20:12 UTC; 382ms ago
Process: 32505 ExecStart=/usr/share/kibana/bin/kibana -c /etc/kibana/kibana.yml (code=exited, status=1/FAILURE)
Main PID: 32505 (code=exited, status=1/FAILURE)
Jun 05 14:20:11 mts-elk-test systemd[1]: kibana.service: Main process exited, code=exited, status=1/FAILURE
Jun 05 14:20:11 mts-elk-test systemd[1]: kibana.service: Unit entered failed state.
Jun 05 14:20:11 mts-elk-test systemd[1]: kibana.service: Failed with result 'exit-code'.
Jun 05 14:20:12 mts-elk-test systemd[1]: kibana.service: Service hold-off time over, scheduling restart.
Jun 05 14:20:12 mts-elk-test systemd[1]: Stopped Kibana.
Jun 05 14:20:12 mts-elk-test systemd[1]: kibana.service: Start request repeated too quickly.
Jun 05 14:20:12 mts-elk-test systemd[1]: Failed to start Kibana.
Jun 05 14:20:12 mts-elk-test systemd[1]: kibana.service: Unit entered failed state.
Jun 05 14:20:12 mts-elk-test systemd[1]: kibana.service: Failed with result 'start-limit-hit'.
root#mts-elk-test:/home/ronak# vi /etc/kibana/kibana.yml
I found the solution. There was a problem with file permission.
I copied cert.pem and privkey.pem files from elasticsearch directory to kibana and changed owner with kibana user:
chown kibana:kibana /etc/kibana/cert.pem
chown kibana:kibana /etc/kibana/privkey.pem
Changed path in kibana.yml file:
server.ssl.key: /etc/kibana/privkey.pem
server.ssl.certificate: /etc/kibana/cert.pem
Rstart kibana: service kibana restart
And it worked!
help my apacitor is not runnning, actually im running influxdb in the same server that kapacitor and telegraf, but my kapacitor don't work
kapacitor.service - Time series data processing engine.
Loaded: loaded (/lib/systemd/system/kapacitor.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2019-01-03 17:56:38 UTC; 3s ago
Docs: https://github.com/influxdb/kapacitor
Process: 2502 ExecStart=/usr/bin/kapacitord -config /etc/kapacitor/kapacitor.conf $KAPACITOR_OPTS (code=exited, status=1/FAILURE)
Main PID: 2502 (code=exited, status=1/FAILURE)
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: kapacitor.service: Service hold-off time over, scheduling restart.
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: kapacitor.service: Scheduled restart job, restart counter is at 5.
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: Stopped Time series data processing engine..
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: kapacitor.service: Start request repeated too quickly.
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: kapacitor.service: Failed with result 'exit-code'.
Jan 03 17:56:38 ip-172-31-43-67 systemd[1]: Failed to start Time series data processing engine..
i did find the solution for myself:
[[influxdb]]
enabled = true
name = "localhost"
default = true
urls = ["http://localhost:8086"]
username = "user"
password = "password"
you must take in count that you will need has an user create in influxdb before