I have Windows EC2 instance I use for my public-facing C# API. The VPC(and related Internet Gateway, subnets, etc) are all default.
I've now setup an AWS ElasticSearch service using their more secure VPC Endpoint option (instead of public-facing) and I've associated it to the same subnet and vpc as my above Windows EC2 instance.
I'd like to get them to talk to each other.
Reading from https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html
It seems what you'd do is ssh tunnel / port forward traffic from localhost:9200 on the EC2 instance to the actual Elastic Search service (via that VPC endpoint).
It seems this command is where the magic happens:
ssh -i ~/.ssh/your-key.pem ec2-user#your-ec2-instance-public-ip -N -L 9200:vpc-your-amazon-es-domain.region.es.amazonaws.com:443
but that is for a Linux EC2 instance.
If I am Remote Desktopped into my Windows EC2 instance (the API), how can I make it so when I go to a browser, http://localhost:9200
will send traffic to my VPC Endpoint:
vpc-your-amazon-es-domain.region.es.amazonaws.com:443
Thanks!
Alright, so I'll answer my two questions:
First, it's actually quite easy, just RDP to your box and access the instance directly via the VPC endpoint. You don't need to do anything wacky like port forwarding using the netsh command or anything like that. Simply make sure the server (in my case my API) is on the same VPC and you're fine. I just had an error in my connection string that's why it didn't connect. To confirm, I RDP'D in and was able to hit the endpoint directly in a browser on port 80. While it's true the actual Elasticsearch runs on port 9200, you don't need to forward to localhost:9200 --> vpc:9200.
Now, regarding the second question, about hitting it locally, I think the problem is that because this service lacks a public IP address and you can't access it, that you can go through some complicated setup on AWS, or easier is just set it up to run locally for now until you are ready to use the VPC one (and thus your code will just run). Another option is to use security groups and make a publicly accessible cluster for now, and then when your code is done, search service/layer done, etc, you can start anew with a VPC/secure Elasticsearch service and that should be it.
Another thing that many mention is that it is cheaper/you have more control of things if you setup your own Elasticsearch on your local machine, and then set one up on EC2 (this is just reading blogs and seeing people mention how much frustration they had with it).
Related
I had EC2 server where I was running my existing application. The EC2 instance was on private subnet and ELB was created in public subnet with access to particular VPN IP. So whenever I was on VPN, I was able to access my application and if I am outside that VPN IP then I wasn't able to access the application.
Now I have created EKS cluster and I am deploying my application using kubectl with docker image of the application. Weird thing is the application works fine whenever I am NOT on VPN (I tweaked security group to allow all traffic from all IPs) and whenever I am on VPN, I receive "ERR_CONNECTION_RESET" in chrome and curl shows - empty response received from server.
Till now I have tried below things. As I am relatively new with EKS, I am not able to find much.
1. Same security group applied - Not resolving
2. Checked logs of all pods - whichever pods I received from "kubectl get po --all-namespaces" - No issues showing up
3. Checked /var/log/messages
4. Tried to change application port
5. kubectl get events not showing anything on why server is sending back empty response.
6. Tried to SSH to node and tried to curl localhost:30080 and it works fine, but when tried to curl from my machine (which is on VPN), it fails with empty response.
Please again note that, the application runs totally fine when I am outside VPN. Further my old application (that is on EC2) runs fine with VPN.
Thanks in advance!
Finally found the issue was with the corporate VPN which was blocking all ports other than 80 and 443. When I was creating the service, I wanted to have ELB to expose port 5000. So I was thinking elb-host:5000 will point to dev service nodeport which was 30080. This was perfectly working when I was NOT on the VPN. But when I was connecting the site using VPN, corporate traffic was blocking port 5000 on ELB. After I changed the port to 80, it started working as expected.
While using nginx, it was creating ELB with port 80 instead of my intended port 5000. I didn't notice that port change and thought that this is happening due to IP blocking.
I am trying to set up my JMeter master/slave set up on AWS EC2 instances using windows. Out of the box integration didn't work (connection refused errors) and after some investigation I got to know that RMI communication only works if the machines are on same subnet (is this true?). I found this great article and I tried to follow it as is on windows (tried running ssh port forward tunneling via cygwin) but not have any luck. So I was hoping if someone already done this with Windows and can share their experience.
Out of the box integration didn't work (connection refused errors) and
after some investigation I got to know that RMI communication only
works if the machines are on same subnet (is this true?)
It is not necessary. Even if the machines are in a different subnet and addressable via the network, RMI should work. Here make sure the Security Groups and NACLs are properly set so that both EC2 instances can communicate with each other via the network.
You can check whether you can ping(For this you need to enable ICMP in your Security Groups and also make sure other needed ports are opened for RMI) from one EC2 instance to another. If the servers can communicate with each other, then you need to troubleshoot internal firewall configuration in windows EC2 instances.
We have used https://testable.io/ AMI's from the marketplace. It takes care of all configuration and just needs to worry only about testing and the results.
https://aws.amazon.com/marketplace/pp/B073JNTVKP
All our endpoints are secured to certain IP-ranges from testable.io. If you have internal, still you need to setup those security settings.
Hope it helps.
I have installed rundeck in docker using ec2 instance.
When I run the image and start rundeck. It's fine.
Lynx http:localhost:4440
Us able to show rundeck dashboard.
But, how can I access this rundeck from Windows browser?
I tried using address but connection refused.
In order to access this from outside for your setup, you might have to ensure the following things:
Ensure that host server (ec2) is forwarding ports to the docker container. You should have used -p or -ports when launching the container for this.
Test: From your EC2 instance, you should be able to access: http://localhost:4440
Ensure you have a public IP assigned to your EC2. You should be able to see that from your aws ec2 console: http://console.aws.amazon.com/ec2
Ensure that your security group(s) for that instance has InBound connections to accept 4440 from your IP or rest of the world.
After this, your http://:4440 should work.
I hope I got your question correct.
Let me know how it goes,
Thanks,
Anoop
I had a running instance, and then I became unable to connect to it via http(80) and ssh(22). I tried to reboot the instance, but nothing went up. This has happened to me twice in the past month.
Why does it happen? Can I do anything to fix and/or prevent it from happening?
If I launch a new instance in same region, and it works.
Things to check when trying to connect to an Amazon EC2 instance:
Security Group: Make sure the security group allows inbound access on the desired ports (eg 80, 22) for the appropriate IP address range (eg 0.0.0.0/0). This solves the majority of problems.
Public IP Address: Check that you're using the correct Public IP address for the instance. If the instance is stopped and started, it might receive a new Public IP address (depending on how it has been configured).
VPC Configuration: Accessing an EC2 instance that is launched inside a Virtual Private Cloud (VPC) requires:
An Internet Gateway
A routing table connecting the subnet to the Internet Gateway
NACLs (Network ACLS) that permit through-traffic
If you are able to launch and connect to another instance in the same subnet, then the VPC configuration would appear to be correct.
The other thing to check would be the actual configuration of the operating system on the instance itself. Some software may be affecting the configuration so that the web server / ssh daemon is not working correctly. Of course, that is hard to determine without connecting to the instance.
If you are launching from a standard Amazon Linux AMI, ssh would work correctly anytime. The web server (port 80) would require installation and configuration of software on the instance, which is your responsibility to maintain.
I have a free Amazon EC2 instance. And I installed Apache web server on it. I have the DNS record for my domain point to the ip for the EC2 instance. I can not access to my website. Then I looked up and allow the http inbound. But I still failed to access my web? What might be the reason. Anybody gives me a clue?
Go to the AWS management console and look at the Security Group the instance is in. Then make sure you have the port open that you are trying to connect to (most likely 80). To open it to the world set the ip range to 0.0.0.0/0 and to open it to a specific ip (like only your house) set it to xxx.xxx.xxx.xxx/32.
That is almost always the reason people have problems connecting when they are new to AWS. I wrote this post, which should help get you setup.