I followed this guide to install ELK on an EC2 instance: https://logz.io/learn/complete-guide-elk-stack/#installing-elk
Now what I'm trying to do is to view Kibana, for example, on my browser since EC2 doesn't have a GUI; it's all a CLI.
Now what should I change in the Kibana and/or Elasticsearch config files to allow this to happen? As Kibana is on port 5601, how can I see it from the browser like this?
ec2-x-xx-xx-xx.us-east-2.compute.amazonaws.com:5601
Whenever I put the above in the browser, nothing happens. I tried to change the network.host field to the public IP of the EC2 instance, but that didn't help. What am I missing here?
Updating answer based on comment thread
Looking through the guide you linked, a few things are missing:
Bind Kibana to an externally accessible port. In /etc/kibana/kibana.yml:
server.host: "<your private EC2 ip>"
Add port 5601 to your security group
The rule will look like this if you want it accessible to the world (though I'd recommend you select the "My IP" option from Source if you don't enable security):
Once you've done those things, then you, you can hit Kibana using the public IP/DNS for your EC2 instance like http://ec2-IP.aws.com:5601
Related
I am using elastic search in my ubuntu16.04 server. When i am trying to access elasticsearch from browser by using url ip:port/_cat/indices?v. I am getting site can't be reached. After that i am change the network.host value to network.host: 0.0.0.0. After change the network.host ip the search engine not started. How can i access the elasticsearch in my browser.I changed the port also.
Thank you..
There can be many reasons for ES not being reachable. I would start with the obvious and make sure that:
ES is listening on the port: on the ES instance when you run 'curl
ip:port' you should get an answer. if not the service didn't start
well.
make sure there are no firewall rules/security groups that prevent
access from remote network.
make sure network.publish_host is configure correctly:
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#advanced-network-settings
more info here: ElasticSearch instance not reachable from outside the server - Azure Windows 2012
I am having couple of the strange issues with the elasticsearch:
1) When I run my web app as localhost, I get the logs in ES. But when I run the app using my public IP/domain name, then I dont get any logs. I guess this is because ES needs some settings to be changed from localhost to some IP (public or private?)
I tried several options: network.bind_host, network.publish_host, network.host - setting them to public IP and then to the private IP - it did not help.
So I am a kinda stucked. Please advise.
2) I also tried to send the logs to the remote server and it did not work. I can't even access the remote server via: public_IP:9200
Neither I can't access kibana via: public_ip:5601
I tried several options:
I set up
network.bind_host, network.publish_host, network.host
to public ips, private ips, 0.0.0.0, 127.0.0.1, nothing helped.
Please advise.
Just in case, I am not sure if it's relative or not, I am using basic license for ES.
I have elasticsearch running on EC2,
I can hit form local IP address(ex. curl -XGET localhost:9200)
I cannot hit from public IP address, whether on the same machine, or from our network, it always times out,
IPtables are allowing
port is open(to itself as well as private network)
Elasticsearch http.cors is enabled and allows "*"
aside from Iptables, amazon security config, elasticsearch config could there be anything I am overlooking? (we can access 443 and get kibana up, it just times out on the elasticsearch ajax call or if I try to access 9200 directly)
been working on this for over a day so I humbly come to you all!
thank you
I had exactly the same issue.
I managed to solve it as follows:
Do what TJ said in his comment, + restart the instance. I wasn't sure if this was/is necessary, but I did it for good measure.
I made sure that the following is set in the elasticsearch.yml file:
a. http.enabled: true
b. http.cors.enabled: true
c. http.cors.allow-origin: "*"
Restarted elasticsearch (service elasticsearch restart)
Then when I tried to access elasticsearch from the public IP it worked - http://[PUBLIC IP OF INSTANCE]:9200
Hope this helps.
I just spent lots of time trying to get this working and just succeeded.
Setup: Elasticsearch 6.2.4, running on a Windows Server 2012, EC2 instance.
I also installed the discovery-ec2 plugin, not sure now if it is required, my assumption is, yes it is required although some of the settings it allows were not necessary to get it working.
Config (.yml). I tried tons of different .yml config settings which in the end did not help, in the end I think the main setting is:
network.host: 0.0.0.0
I tried setting the network.host to ec2:privateIpv4 and ec2:publicIpv4 (plugin settings) but they didn't help.
I had added the required Custom TCP Rules (allowing 9200 and 9300...not sure if 9300 is needed).
Either it failed to start (usually with a binding to 9300 error) or started but was not publicly accessible.
The Fix. What got it working in the end is you must also open the port in windows firewall. As soon as I added the inbound rule, boom it connected :)
I then stripped out all the extra configs I had been trying, restarted Elasticsearch... and it still worked!
I've spent some time now looking for information regarding elasticsearch.yml configurations that make my single instance Elasticsearch (on Windows 2012 Server EC2) accessible via public ip, but everytime I uncomment one or both of following settings the only thing that changes is, calling the private ip as well results in an error.
network.publish_host: <public ip>
network.bind_host: <private ip>
Is this correct and are there any other settings that have to be defined? Shouldn't it run with the default values?
This is more of a general answer as to how networking works within EC2 instead of a specific answer to your question. But it should help inform how to configure your application.
EC2 has 1:1 NAT between a public and private IP address. Because of this, only the private IP address is visible to the instance directly.
If you are binding a service to a network interface, it would be the one with the private IP.
Some services do require knowledge of the external IP address in order to function properly. The only one I have run into is FTP in a passive configuration, likely due to the fact that it needs to open a separate socket for data transfer.
In the case of elastic search, it appears that they have a special plugin that will help configure elastic search for the aws environment: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-network.html
I had the same problem.
Installed only one instance of ES on aws EC2 and wanted to grant it public access.
On ubuntu 16.04 this is what works for me:
in /etc/elasticsearch/elasticsearch.yml add this line:
network.host: <ec2 instance private ip>
The private ip should be something like 172.x.x.x
Also do not forget allow access in security group in your aws console for port 9200 (default) and ip address from which you will be sending requests.
So difference was setting not public but private ip address from aws console..
Also note that this can be dangerous as there is not any user/password or other access control
I'm struggling to get EC2 and ElasticSearch up and running. Specifically I'm trying to reach my node from outside Amazon's cloud for verification purposes. I've set up the security group so that I have a "Custom TCP" rule on port 9200 and ElasticSearch is listening on that port, which I can see with netstat -l. When I curl -XGET https://localhost:9200 I get the response expected from ElasticSearch. When I curl -XGET https://publicIP:9200 from WITHIN Amazon (ie. another node that I have running) I get the response expected from ElasticSearch. When I try to do the same request from my desktop I get "no response". I can not, for the life of me, figure out why this is happening.
There are several things to check:
Accessing the public URL of an instance from inside the amazon cloud will map to its private IP. In you test above, where you specify publicIP, did you use the public IP or public domain name? Make sure to test with the IP, not the domain name.
If access to the public IP works from the same machine, try the same thing from another EC2 instance.
Finally, you may have a firewall rule on your desktop, or your work network, preventing outgoing access on port 9200.
If you are running Elasticsearch as a service, then go to /etc/elasticsearch/elasticsearch.yml and make the
network.host: "0.0.0.0"
This solution worked for me.