I am having couple of the strange issues with the elasticsearch:
1) When I run my web app as localhost, I get the logs in ES. But when I run the app using my public IP/domain name, then I dont get any logs. I guess this is because ES needs some settings to be changed from localhost to some IP (public or private?)
I tried several options: network.bind_host, network.publish_host, network.host - setting them to public IP and then to the private IP - it did not help.
So I am a kinda stucked. Please advise.
2) I also tried to send the logs to the remote server and it did not work. I can't even access the remote server via: public_IP:9200
Neither I can't access kibana via: public_ip:5601
I tried several options:
I set up
network.bind_host, network.publish_host, network.host
to public ips, private ips, 0.0.0.0, 127.0.0.1, nothing helped.
Please advise.
Just in case, I am not sure if it's relative or not, I am using basic license for ES.
Related
I am just testing out Apis in my local and they all seem to work pretty good with localhost:8080. So i just uploaded the whole application to ec2 instance.
I uploaded my Local Springboot Application to the Ec2 instance, got the vpc, public & private subnets, got proper route tables, and when i try java -jar demoApplication-SNAPSHOT.jar from my SSH terminal, it seems to be running on port 8081 in the server. But when i try to click on the public ipv4 DNS address on ec2 instance, it says the site is not reachable.
I have allowed the chrome browser through firewall too but even that isn't working. Any help on this will be gladly appreciated!
below is the snippet from postman for the reference.
enter image description here
I followed this guide to install ELK on an EC2 instance: https://logz.io/learn/complete-guide-elk-stack/#installing-elk
Now what I'm trying to do is to view Kibana, for example, on my browser since EC2 doesn't have a GUI; it's all a CLI.
Now what should I change in the Kibana and/or Elasticsearch config files to allow this to happen? As Kibana is on port 5601, how can I see it from the browser like this?
ec2-x-xx-xx-xx.us-east-2.compute.amazonaws.com:5601
Whenever I put the above in the browser, nothing happens. I tried to change the network.host field to the public IP of the EC2 instance, but that didn't help. What am I missing here?
Updating answer based on comment thread
Looking through the guide you linked, a few things are missing:
Bind Kibana to an externally accessible port. In /etc/kibana/kibana.yml:
server.host: "<your private EC2 ip>"
Add port 5601 to your security group
The rule will look like this if you want it accessible to the world (though I'd recommend you select the "My IP" option from Source if you don't enable security):
Once you've done those things, then you, you can hit Kibana using the public IP/DNS for your EC2 instance like http://ec2-IP.aws.com:5601
Despite ElasticSearch and Kibana both running on my production server, I'm unable to visit the GUI over the public IP: http://52.4.153.19:5601/
Localhost curls return 200 but console errors on the browser report timeouts after a few images are retrieved.
I've successfully installed, run, and accessed Kibana on my local (Windows 10) and on my staging AWS EC2 Ubuntu 14.04 environment. I'm able to access both over port 5601 on localhost and the staging environment is accessible over the public IP address and all domains addressed accordingly. The reverse proxy also works and all status indicators are green on the dashboard.
I'm running Kibana 4.5, ElasticSearch 2.3.1, Apache 2.4.12
I've used the same exact volume from the working environment to attach to the production instance, so everything is identical on the two volumes, except that the staging environment's apache vhost uses a subdomain while the production environment's servername is the base domain. Both are configured for SSL wildcards. Both are in separate availability zones at Amazon. I've tried altering the server block to use a subdomain on the production server, just to see if the domain was impactful but the error remains.
I also tried running one instance individually, in case EC2 had some kind of networking error with 0.0.0.0 but I'm unable to come to a resolution. All logs and configurations are identical between the two servers for ElasticSearch and Kibana.
I've tried deleting and re-creating the kibana index, tried alternate settings inclusive of the host, elasticsearch url, extending the max ping and timeout, max retries, extended the apache limits, http.cors to allow different origins. I've tried other ports but both servers are indicating that 5601 is listening in the same way.
I also had the same problem on a completely different volume that was previously attached to this instance.
The only difference I can see is that the working version pings fine while the non-working version has a 100% packet loss when pinging the IP, although I can't imagine why that would be, as I'm able to reach the website on 80, just fine. I can also access various other tools running on other ports. I assume there might be some kind of networking conflict. Any ideas?
May be port 5601 is blocked by firewall
Allow incoming connections to port 5601 by:
sudo iptables -I INPUT -p tcp --dport 5601 -j ACCESS
For security:
Modify above mentioned command and accept connection only from specific address. (See man iptables)
or use Shield plugin for elasticseach
Sorry, forgot to update this question. The answer turned out being that I simply needed to deploy a new instance. Simply by creating a clone of the instance, I was able to resolve the issue. I've had networking problems at AWS, before, with their internal dns/ip conflicts, so I've had to do so, in the past and this turned out to be the quickest and cleanest solution, albeit not providing any definitive insight into the cause.
I have elasticsearch running on EC2,
I can hit form local IP address(ex. curl -XGET localhost:9200)
I cannot hit from public IP address, whether on the same machine, or from our network, it always times out,
IPtables are allowing
port is open(to itself as well as private network)
Elasticsearch http.cors is enabled and allows "*"
aside from Iptables, amazon security config, elasticsearch config could there be anything I am overlooking? (we can access 443 and get kibana up, it just times out on the elasticsearch ajax call or if I try to access 9200 directly)
been working on this for over a day so I humbly come to you all!
thank you
I had exactly the same issue.
I managed to solve it as follows:
Do what TJ said in his comment, + restart the instance. I wasn't sure if this was/is necessary, but I did it for good measure.
I made sure that the following is set in the elasticsearch.yml file:
a. http.enabled: true
b. http.cors.enabled: true
c. http.cors.allow-origin: "*"
Restarted elasticsearch (service elasticsearch restart)
Then when I tried to access elasticsearch from the public IP it worked - http://[PUBLIC IP OF INSTANCE]:9200
Hope this helps.
I just spent lots of time trying to get this working and just succeeded.
Setup: Elasticsearch 6.2.4, running on a Windows Server 2012, EC2 instance.
I also installed the discovery-ec2 plugin, not sure now if it is required, my assumption is, yes it is required although some of the settings it allows were not necessary to get it working.
Config (.yml). I tried tons of different .yml config settings which in the end did not help, in the end I think the main setting is:
network.host: 0.0.0.0
I tried setting the network.host to ec2:privateIpv4 and ec2:publicIpv4 (plugin settings) but they didn't help.
I had added the required Custom TCP Rules (allowing 9200 and 9300...not sure if 9300 is needed).
Either it failed to start (usually with a binding to 9300 error) or started but was not publicly accessible.
The Fix. What got it working in the end is you must also open the port in windows firewall. As soon as I added the inbound rule, boom it connected :)
I then stripped out all the extra configs I had been trying, restarted Elasticsearch... and it still worked!
I'm struggling to get EC2 and ElasticSearch up and running. Specifically I'm trying to reach my node from outside Amazon's cloud for verification purposes. I've set up the security group so that I have a "Custom TCP" rule on port 9200 and ElasticSearch is listening on that port, which I can see with netstat -l. When I curl -XGET https://localhost:9200 I get the response expected from ElasticSearch. When I curl -XGET https://publicIP:9200 from WITHIN Amazon (ie. another node that I have running) I get the response expected from ElasticSearch. When I try to do the same request from my desktop I get "no response". I can not, for the life of me, figure out why this is happening.
There are several things to check:
Accessing the public URL of an instance from inside the amazon cloud will map to its private IP. In you test above, where you specify publicIP, did you use the public IP or public domain name? Make sure to test with the IP, not the domain name.
If access to the public IP works from the same machine, try the same thing from another EC2 instance.
Finally, you may have a firewall rule on your desktop, or your work network, preventing outgoing access on port 9200.
If you are running Elasticsearch as a service, then go to /etc/elasticsearch/elasticsearch.yml and make the
network.host: "0.0.0.0"
This solution worked for me.