Kibana deployment issue on server . . . client not able to access GUI - elasticsearch

I have configured Logstash + ES + kibana on 100.100.0.158 VM and Kibana is running under apache server. port 8080
Now what my need is . . I just have to give URL "100.100.0.158:8080/kibana" to client so client can see his data on web.
When when I put this URL on client browser I am getting this error
"can't contact elasticsearch at http://"127.0.0.1":9200 please ensure that elastic search is reachable from your system"
Do I need to configure ES with IP 100.100.0.158:9200 or 127.0.0.1:9200 is ok . . !
Help . . !
Thanks
Tushar

If your Kibana and ES are installed on the same box, you can have it auto-detect the the ES URL/IP by using this line in your Kibana's config.js file:
/** #scratch /configuration/config.js/5
* ==== elasticsearch
*
* The URL to your elasticsearch server. You almost certainly don't
* want +http://localhost:9200+ here. Even if Kibana and Elasticsearch are on
* the same host. By default this will attempt to reach ES at the same host you have
* elasticsearch installed on. You probably want to set it to the FQDN of your
* elasticsearch host
*/
elasticsearch: "http://"+window.location.hostname+":9200",
This is because the interface between Kibana and ES is via JavaScript, and so using 127.0.0.1 or localhost actually points to the client machine (that the browser is running on) rather than the server.

Modify elasticsearch configuration file elasticsearch.yml
Append or modify following configurations:
# Enable or disable cross-origin resource sharing.
http.cors.enabled: true
# Which origins to allow.
http.cors.allow-origin: /https?:\/\/<*your\.kibana\.host*>(:[0-9]+)?/
It is caused by kibana page trying to load jason data from elasticsearch which will be blocked for security reason.

It is about iptables rules. Kibana uses 9292 for web port, but for elasticsearch queries uses 9200. So you must add line to iptables for these ports.
netstat -napt | grep -i LISTEN
you will see these ports: 9200 9300 9301 9302 9292
iptables -I INPUT 4 -p tcp -m state --state NEW -m tcp --dport 9200 -j ACCEPT
see detail: http://logstash.net/docs/1.3.3/tutorials/getting-started-simple

Related

elasticsearch setup on Gcloud VM fails

I wish to run my elasticsearch remotely on gcloud VM, this is configured to run at 127.0.0.1 at a specific port 9200. How to access this from a website outside this vm? If I change the network host to 0.0.0.0 on the yml file, even 9200 port becomes inaccessible. How do I overcome this problem?
Changed network.host: [_site_ , _local_ , _global_ ]
_site_ = internal ip given by google cloud vm,
_local_ = 127.0.0.1,
_global_ = found using curl ifconfig.me,
Opened a specific port (9200) and tried to connect with global IP address.
curl to the global ip gives
>Output: Failed to connect to (_global_ ip) port 9200: connection refused.
So put network.host:0.0.0.0 and then try to allow 9200 and 9201 port and restart the elasticsearch service.If you are using ubuntu then sudo service elasticsearch restart then check by doing curl -XGET 'http://localhost:9200?pretty'.Let me know if you are still facing any issues.
Use following configurations for elasticsearch.yml
network.host: 0.0.0.0
action.auto_create_index: false
index.mapper.dynamic: false
Solved this problem by going through the logs and found out that the public ip address is re-mapped to the internal ip address, hence network.host can't be set to external ip directly. Elasticsearch yml config is as follows:
'network.host: xx.xx.xxx.xx' is set to the internal ip (given by google),
'http.cors.enabled: true',
'http.cors.allow-origin:"*", (Do not use * in production, its a security issue)
'discovery.type: single-node' in my case to make it work independently and not in a cluster
Now this sandboxed version can be accessed from outside the VM using the external IP address given by Google.

sgadmin.sh failed elasticsearch && searchguard

While trying to exucte
./sgadmin.sh -ts truststore.jks -tspass 90f3cbdb3eabe04f815b -ks CN=sgadmin-keystore.jks -kspass a65d2a4fa62d7ed7a4d5 -h host -p 9200 -nhnv -cn eslcl1 -cd ../sgconfig/
I am getting the following error:
Cannot retrieve cluster state due to: None of the configured nodes are available: [{#transport#-1}{6PPXnCNqTt-W5g-0fmeZuQ}{host}{host:9200}]. This is not an error, will keep on trying
error:
WARNING: JAVA_HOME not set, will use /usr/bin/java
Search Guard Admin v5
WARNING: Seems you want connect to the a HTTP port.
sgadmin connect through the transport port which is normally 9300.
Will connect to host:9200 ... done
### LICENSE NOTICE Search Guard ###
If you use one or more of the following features in production
make sure you have a valid Search Guard license
(See https://floragunn.com/searchguard-validate-license)
* Kibana Multitenancy
* LDAP authentication/authorization
* Active Directory authentication/authorization
* REST Management API
* JSON Web Token (JWT) authentication/authorization
* Kerberos authentication/authorization
* Document- and Fieldlevel Security (DLS/FLS)
* Auditlogging
In case of any doubt mail to <sales#floragunn.com>
###################################
Contacting elasticsearch cluster 'eslcl1' and wait for YELLOW clusterstate ...
Cannot retrieve cluster state due to: None of the configured nodes are available: [{#transport#-1}{6PPXnCNqTt-W5g-0fmeZuQ}{host}{host:9200}]. This is not an error, will keep on trying ...
Root cause: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{6PPXnCNqTt-W5g-0fmeZuQ}{host}{host:9200}]] (org.elasticsearch.client.transport.NoNodeAvailableException/org.elasticsearch.client.transport.NoNodeAvailableException)
* Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you need to check your clustername as well as hostnames in your SSL certificates)
* Make also sure that your keystore or cert is a client certificate (not a node certificate) and configured properly in elasticsearch.yml
* If this is not working, try running sgadmin.sh with --diagnose and see diagnose trace log file)
* Add --accept-red-cluster to allow sgadmin to operate on a red cluster.
My conf in elasticsearch.yml is
######## Start Search Guard Demo Configuration ########
searchguard.ssl.transport.keystore_filepath: CN=x.x.x.x-keystore.jks
searchguard.ssl.transport.keystore_password: 8a17368ff585a2c3afdc
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: 90f3cbdb3eabe04f815b
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: false
searchguard.ssl.http.keystore_filepath: CN=x.x.x.x-keystore.jks
searchguard.ssl.http.keystore_password: 8a17368ff585a2c3afdc
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_password: 90f3cbdb3eabe04f815b
searchguard.authcz.admin_dn:
- CN=sgadmin
cluster.name: eslcl1
network.host: x.x.x.x
Is there any configuration that I might need to look into ?
You need to connect to port 9300 (the transport protocol), not 9200 which is typically the HTTP/S port. On port 9300 the elasticsearch nodes talk to each other through a binary TCP protocol. On port 9200 the REST api is accessible via HTTP/S.
sgadmin connect through the binary TCP protocol to elasticsearch, so you need to use port 9300.
Therefore you get this warning
WARNING: Seems you want connect to the a HTTP port.
sgadmin connect through the transport port which is normally 9300.
So your command should look like
./sgadmin.sh -ts truststore.jks -tspass 90f3cbdb3eabe04f815b -ks CN=sgadmin-keystore.jks -kspass a65d2a4fa62d7ed7a4d5 -h host -p 9300 -nhnv -cn eslcl1 -cd ../sgconfig/ -nhnv
(Add -nhnv to disable hostname verification if your certificates are not matching your hostnames)

On iMac, access elasticsearch and neo4j ports on local ip address?

ifconfig shows
inet 192.168.10.1
I can access
http://localhost/
http://127.0.0.1/
http://192.168.10.1
They are all the same.
I also can access neo4j and elasticsearch ports on the following urls
Elasticsearch
http://127.0.0.1:9200/
http://localhost:9200/
Neo4j
http://127.0.0.1:7474/browser/
http://localhost:7474/browser/
But port 9200 and 7474 are not working for 192.168.10.1
http://192.168.10.1:9200
http://192.168.10.1:7474
Something I need to do make the port 7474 (neo4j) and 9200 (elasticsearch) working for 192.168.10.1, but I don't know how.
Please advise, thanks!
I figured it out.
Neo4j
Set up neo4j on the ip (except localhost), in my case
http://192.168.10.1:7474
In the neo4j.conf file
uncomment the following line.
# With default configuration Neo4j only accepts local connections.
# To accept non-local connections, uncomment this line:
dbms.connectors.default_listen_address=0.0.0.0
Elasticsearch
Modify elasticsearch.yml
add the following line
network.host: 0.0.0.0
Then start elasticsearch.

Elasticsearch: Failed to connect to localhost port 9200 - Connection refused

When I tried connecting to Elasticsearch using the
curl http://localhost:9200 it is working fine.
But when I run the curl http://IpAddress:9200 it is throwing an error saying
Failed to connect to localhost port 9200: Connection refused
How to resolve this error?
Edit /etc/elasticsearch/elasticsearch.yml and add the following line:
network.host: 0.0.0.0
This will "unset" this parameter and will allow connections from other IPs.
By default it should bind to all local addresses. So, assuming you don't have a network layer issue with firewalls, the only ES setting I can think to check is network.bind_host and make sure it is either not set or is set to 0.0.0.0 or ::0 or to the correct IP address for your network.
Update: per comments in ES 2.3 you should set network.host instead.
In my case elasticsearch was started.
But still had
curl: (7) Failed to connect to localhost port 9200: Connection refused
The following command was unsuccessful
sudo service elasticsearch restart
In order to make it work, I had to run instead
sudo systemctl restart elasticsearch
Then it went all fine.
Tried everything on this page, and only instructions from here helped.
in /etc/default/elasticsearch, make sure these are un-commented:
START_DAEMON=true
ES_USER=elasticsearch
ES_GROUP=elasticsearch
LOG_DIR=/var/log/elasticsearch
DATA_DIR=/var/lib/elasticsearch
WORK_DIR=/tmp/elasticsearch
CONF_DIR=/etc/elasticsearch
CONF_FILE=/etc/elasticsearch/elasticsearch.yml
RESTART_ON_UPGRADE=true
make sure /var/lib/elasticsearch is owned by elasticsearch user:
chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/
Why don't you start with this command-line:
$ sudo service elasticsearch status
I did it and get:
"There is insufficient memory for the Java Runtime..."
Then I edited /etc/elasticsearch/jvm.options file:
...
################################################################
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
#-Xms2g
#-Xms2g
-Xms512m
-Xmx512m
################################################################
...
This worked like a charm.
None of the proposed solutions here worked for me, but what eventually got it working was adding the following to elasticsearch.yml
network:
host: 0.0.0.0
http:
port: 9200
After that, I restarted the service and now I can curl it from both within the VM and externally. For some odd reason, I had to try a few different variants of a curl call inside the VM before it worked:
curl localhost:9200
curl http://localhost:9200
curl 127.0.0.1:9200
Note: I'm using Elasticsearch 5.5 on Ubuntu 14.04
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error='Cannot allocate memory' (errno=12)
be sure that the server is started. I've seen this problem when my virtual machine had too litle RAM and es could not start.
sudo systemctl status elasticsearch
the above will show you if es is indeed running.
Edit elasticsearch.yml and add the following line
http.host: 0.0.0.0
network.host: 0.0.0.0 didn't work for
For this problem, I had to use :
sudo /usr/share/elasticsearch/bin/elasticsearch start
to be able to get something on ports 9200/9300 (sudo netstat -ntlp) and a response to:
curl -XGET http://localhost:9200
I experienced a similar issue.
Here's how I solved it
Run the service command below to start ElasticSearch
sudo service elasticsearch start
OR
sudo systemctl start elasticsearch
If you still get the error
curl: (7) Failed to connect to localhost port 9200: Connection refused
Run the service command below to check the status of ElasticSearch
sudo service elasticsearch status
OR
sudo systemctl status elasticsearch
If you get a response (Active: active (running)) like the one below then you ElasticSearch is active and running
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: enabled)
Active: active (running) since Sat 2019-09-21 11:22:21 WAT; 3s ago
You can then test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost using the command below:
curl http://localhost:9200
Else, if you get a response a different response, you may have to debug further to fix it, but the running the command below, will help you detect what caveats are holding ElasticSearch service from starting.
sudo service elasticsearch status
OR
sudo systemctl status elasticsearch
If you want to stop the ElasticSearch service, simply run the service command below;
sudo service elasticsearch stop
OR
sudo systemctl stop elasticsearch
N/B: You may have to run the command sudo service elasticsearch status OR sudo systemctl status elasticsearch each time you encounter the error, in order to tell the state of the ElasticSearch service.
This also applies for Kibana, run the command sudo service kibana status OR sudo systemctl status kibana each time you encounter the error, in order to tell the state of the Kibana service.
That's all.
I hope this helps.
I had the same problem refusing connections on 9200 port.
Check elasticsearch service status with the command sudo service elasticsearch status. If it is presenting an error and you read anything related to Java, probably the problem is your jvm memory. You can edit it in /etc/elasticsearch/jvm.options. For a 1GB RAM memory machine on Amazon environment, I kept my configuration on:
-Xms128m
-Xmx128m
After setting that and restarting elasticsearch service, it worked like a charm. Nmap and UFW (if you use local firewall) checking should also be useful.
Open your Dockerfile under elasticsearch folder and update "network.host=0.0.0.0" with "network.host=127.0.0.1". Then restart the container. Check your connection with curl.
$ curl http://docker-machine-ip:9200
{
"name" : "vI6Zq_D",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "hhyB_Wa4QwSX6zZd1F894Q",
"version" : {
"number" : "5.2.0",
"build_hash" : "24e05b9",
"build_date" : "2017-01-24T19:52:35.800Z",
"build_snapshot" : false,
"lucene_version" : "6.4.0"
},
"tagline" : "You Know, for Search"
}
For versions higher than 6.8 (7.x) you need two things.
1. change the network host to listen on the public interface.
In the configuration file elasticsearch.yml (for debian and derivatives -> /etc/elasticsearch/elasticsearch.yml).
set the network.host or network.bind_host to:
...
network.host: 0.0.0.0
...
Or the interface that must be reached
2. Before going to production it's necessary to set important discovery and cluster formation settings.
According to elastic.co:
v6.8 -> discovery settings that should set.
by e.g
...
# roughly means the same as 1
discovery.zen.minimum_master_nodes: -1
...
v7.x -> discovery settings that should set.
by one single node
discovery.type: single-node
#OR set discovery.seed_hosts : 127.0.0.1:9200
at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured.
In this case, first of all you need to check the java version using below command:
java -version
after running this command you get something like this:
java version "1.7.0_51"
OpenJDK Runtime Environment (rhel-2.4.5.5.el7-x86_64 u51-b31)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)
then use this command:
update-alternatives --config java
and select the below version
*+ 1 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51-2.4.5.5.el7.x86_64/jre/bin/java
2 /usr/java/jdk1.8.0_73/jre/bin/java
Enter to keep the current selection[+], or type selection number: 2
curl -XGET http://127.0.0.1:9200
My 2 cents,
I just followed the install procedure on Digital Ocean, apparently the package available in the repos is not up to date, I deleted everything and followed the install procedure direct from Elastic Search and everything is working now, basically the out of the box behaviour is on a localhost pointing to 9200. Same thing/issue found with Kibana, the solution for me was too, to remove everything and just follow their procedure, Hope this saves someone two hours (the time I spent figuring out how to setup ELK!)
en
Update your jdk to latest minimum version for your elasticsearch.
Change the network.bind to 0.0.0.0 and http:port to 9200. The bind address 0.0.0.0 means all IPv4 addresses on the local machine. If a host has two IP addresses, 192.168.1.1 and 10.1.2.1, and a server running on the host listens on 0.0.0.0, it will be reachable at both of those IPs.
If you encounter the Connection refused error, simply run the command below to check the status of ElasticSearch service
sudo service elasticsearch status
This will help you decipher the state of ElasticSearch service and what to do about it.
For those of you installing ELK on virtual machine in GCP (Google Cloud Platform), make sure that you created firewall rule of Ingress type (i.e. for incoming to VM traffic). You can specify in the rule multiple ports at a time by separating them with comma: 5000,5044,5601,9200,9300,9600.
In that rule you may want to specify a tag (pick tag's name as you like, for example docker-elk that will target your VM (Targets column):
On VM's settings page assign that tag to your VM:
After doing that I was able to access Elasticsearch in my browser via port 9200. And I didn't have to edit elasticsearch.yml file whatsoever.
I have run across this problem every time I install or upgrade ES (7.0+). And the solution was ALWAYS just wait for ES to fully start. It takes about a minute for the REST API to be reponsive. No matter what service status says.
service elasticsearch start
*started
*wait for at least a minute
curl now works and returns responses on the port 9200
After utilizing some of the answers above, don't forget that after an apt install, a total reboot might be in order.
Just to add on this, I've came across many docs through google that said to set network.host to localhost.
Doing so gave me the infamous connection refused. You must use an IP address (127.0.0.1), not a FQDN.
Jeff
Make sure that port 9200 is open for my case it was an amazon instance so when i opened it in my security group the curl command worked.
Disabling SELinux worked for me, although I don't suggest it - I did that just for a PoC
My problem was I could not work with localhost I needed to set it to localhost's IP address
network.bind_host: 127.0.0.1
In my case, the problem is with java version, i installed open-jdk 11 previously. Thats creating the issue while starting the service. I changed it open-jdk 8 and it started working
I experienced this on CentOS 7, and the issue was that /etc/hosts had the following:
127.0.0.1 localhost.localdomain
which I updated to include localhost as follows:
127.0.0.1 localhost localhost.localdomain
after that, no issues.
you have to edit /etc/elasticsearch/elasticsearch.yml
by default all configurations will be commented ,add following configuration
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: [0.0.0.0]
then restart the service
I ran into a related situation recently.
Here's my take on the subject: Accessing Elastic 5.5 in vagrant guest from host through a private network
TL;DR
The settings:
network.host: 0.0.0.0
http.port: 9200
work fine. One just needs to wait enough time for ES to complete its initialization procedure, bind to the network iface and start listening on the port.
Now, from within the guest, curl http://localhost:9200 works and from the host, curl http://192.168.54.2:9200 works as well.
For Windows user try,
https://localhost:9200/
It worked for me.

ElasticSearch: Allow only local requests

How can allow only local requests for elasticsearch?
So command like:
curl -XGET 'http://localhost:9200/twitter/_settings'
can only be running on localhost and request like:
curl -XGET 'http://mydomain.com:9200/twitter/_settings'
would get rejected?
Because, from what i see, elasticsearch allows it by default.
EDIT:
According to http://www.elasticsearch.org/guide/reference/modules/network.html
you can manage bind_host parameter to allow hosts. And by default, it is set to anyLocalAddress
For elasticsearch prior to v2.0.0, if you want both http transport and internal elasticsearch transport to listen only on localhost simply add the following line to elasticsearch.yml file.
network.host: "127.0.0.1"
If you want only http transport to listen on localhost add the following line instead.
http.host: "127.0.0.1"
Starting from v2.0 elasticsearch is listening only on localhost by default. So, no additional configuration is needed.
If your final goal is to deny any requests from outside the host machine, the most reliable way would be to modify the host's iptables so that it denies any incoming requests to the service ports used by ElasticSearch (9200-9300).
If the end goal is to make sure that everyone refers to the service using an exclusive DNS, you're better off achieving this with an HTTP server that can proxy requests such as HTTPd or nginx.
I use this parameter:
http.host: "127.0.0.1"
This parameter not accept http requests for external request.

Resources