Amazon EC2 instance can not be reached after reboot - amazon-ec2

I restarted my EC2 instance and now I am not able to ping it, ssh it or open it on my browser. However in my management console I could see the instance as up and having no problem.

Amazon instances change names sometimes between restarts, double check the public dns name that you are using, chances are that it has changed.
If it has not changed (double check, copy paste if possible) then double check the firewall settings, I've seen situations where the firewall settings get mucked up after a restart (very rare) - simply adding to the firewall configuration should do the trick.

Related

NLA error after updating AWS EC2 instance type

Has anyone encountered the NLA error after changing instance type for an EC2 instance?
Getting this error after upgrading a domain joined instance (t3.2xlarge) to the recommended instance type according to Computer Optimizer (m6i.2xlarge).
Cannot RDP using the local administrator account either, same NLA error.
Also made a re:post question but no answer yet.
Kind regards,
Ken
Things I have tried:
Changing back to t3.2xlarge, connected using domain credentials OK
Changing to m5.2xlarge, connected using domain crendentials OK
Added another NIC when it was on m6i.2xlarge, NLA error on the second interface.
(Don't think this matters, the instance is HVM) Upgraded to the latest PV driver, changing instance type to m6i.2xlarge, NLA error.
Launched a m6i.2xlarge instance in a different subnet(AZ), joined domain OK, connected using domain crendentials OK; changed to t3.2xlarge, NLA error; changed back to m6i.2xlarge, connected using domain crendentials OK
Launched another m6i.2xlarge instance in the same subnet as the t3.2xlarge, swapped the root volume, NLA error. Swapped back the volumes, connected OK.
All these tests leads me to think there is incompatibility b/w gen 5 Xeon processors and gen 6, which is strange, I thought at first it was a network card issue.
Copy pasted from my own answer on RE:POST
Managed to isolate the cause after performing some rescuing via SSM.
The issue seems to stem from the upgrade from the CPU generation leap.
I had always thought each component, Storage, Compute, and Networking are separate, but the ENI config was lost during the upgrade, so the server had trouble (i.e. did not know where the DNS server is) contacting the DCs for authentication.
Without this link to the DCs, NLA will never be met.
So if you are going to upgrade to the latest generation.
While on the current instance type (while you can RDP to the EC2 instance), navigate to System Properties and go to the Remote tab.
Untick the NLA option and apply and save the change.
Shutdown the instance and change to the desired instance type.
RDP to the instance using the Administrator account.
Here you will see that the network interface configuration is empty, so add your DNS server IP address back in here.
Confirm you have a connection to the DCs by pinging or something of the sort, then repeat step 1, but this time enable the NLA option and save.
Reboot and VoilĂ , you should now have access to the EC2 instance using your domain logins again.

firewall configuring on EC2 in AWS

Few days ago I configured firewall on EC2 in AWS. But, the problem was when I configure firewall in EC2, that server went down. Then when I remove this firewall, then the server went up again. but after restarting the server, it was down. then when I disabled firewall from the server it gone up, then remains ok also after restarting further. My question is should we avoid firewall installing on aws EC2?, as the firewall is automatically configured by aws instance. but won't it increase the security by configuring firewall. Did, I do something wrong like double layer protection by installing firewall?
Enabling the Firewall/Security Groups/ does not DOWN the server, it is still up and running but maybe inaccessible to you, as you might have locked yourself from it.
This is exactly the reason why the EC2 Security Groups are there, so you can block access to specific ports and allow access only from specific ports for management, and/or open public access to Web services for example.. if you are building a webserver.
Firewalls can be dangerous if you do not know what you are doing.
If you locked yourself out, then Yes you did something wrong. If you first allow your IP in the firewall, then you will still be able to manage or access it, once the Firewall is up.

AWS Cloud9: Environment Stops Responding

I trying to setup AWS Cloud9 and am running into a wall each time I try to setup my environments. Once I create the environment and start following this guide https://docs.aws.amazon.com/cloud9/latest/user-guide/sample-lamp.html to configure the LAMP server, through the Cloud9 IDE terminal, the environment will just stop responding. Once I try to reload the IDE I get the follow error;
Cannot connect to instance error message.
Rebooting the instance doesn't seem to resolve the error message. But any time I make a fresh instance it will let me work from anywhere to 30 seconds to 90 seconds before it stops responding.
I have looked through my VPC port settings, as well as security group settings, and they both appear to the correct.
VPC inbound rules VPC outbound rules Security Group inbound
rules Security Group outbound rules
Additionally, I was using the default t2.micro instance until I read this post AWS Cloud9: Cannot open environment and have tried with the t2.small but I am still getting the same results.
Any help with where else to look or what else to try would be much appreciated!
Edit: It appears to be random when it stops and freezes, for example when making a m4.large instance. It froze while I was setting up the sudo mysql_secure_installation.
Once I typed "Y" it wouldn't let me press enter. Reloading the IDE gave me the VPC error.
Welcome to SO! When I use cloud9 I tend to use m4.large for anything that's non-trivial. If you're running Apache and MySQL on the same host I would definitely try the m4.large instance. It's $0.10/hr (pricing) so you could try it out fairly cheaply. I'm guessing that's the root of the issue. If you're still having the issue please repost here and we can check further.
Just to confirm:
- You can connect to the instance at least once (even if for a few seconds)
- You see the IDE and can type for 30-60 seconds before it stops responding
If you can't connect that's likely a different issue.

AWS EC2 Instance Hacked

One of my EC2 instances was hacked a few days ago.
I tried logging in via SSH to the server, but I couldn't connect. I am the only one with access to the private key, and I keep it in a safe place.
Luckily, I had a backup of everything and was able to move the web app to a new instance quite fast.
My concern right now is that I don't know how my instance was hacked in the first place.
Why can't I log in via SSH using my private key? I would assume that the private key stored on the server can't be (easily) deleted.
Is there a way I can find out how the hacker gained access to the instance? Perhaps a log file that would point me in the right direction.
Should I attach the EBS volume in question to a new instance and see what's on it or what are my options in this case?
Right now, it seems I have to access at all to the hacked instance.
Thank you!
#Krishna Kumar R is correct about the hacker probably changing the ssh keys.
Next steps:
Security concerns (do these now!):
Stop the instance, but don't terminate yet
Revoke/expire any sensitive credentials that were stored on the instance, including passwords and keys for other sites and services. Everything stored on that instance should be considered compromised.
Post-mortem
Take an EBS snapshot of the instance's root volume (assuming that's where logs are stored)
Make a new volume from the snapshot and attach to a (non-production) instance
Mount and start reading logs. If this is a linux host and you have port 22 open in the firewall, I'd start with /<mount-point>/var/log/auth.log
They might have logged into your machine via password. In ssh config, check the value of: PasswordAuthentication. If it is set to yes, then users can login to the instance remotely via password. Check /var/log/secure for any remote logins. It will show up all logins (password or key based).
If someone logged in as 'root', they can modify the ssh keys.
The fact that you are unable to login to the machine does not mean that it has been "hacked". It could be due to a configuration change on the instance, or the instance might have changed IP address after a stop/start.
Do a search on StackOverflow for standard solutions to problems connecting to an instance and see if you can connect (eg recheck IP address, check security group, turn on ssh -v debugging, check network connectivity & VPC settings, view Get System Log, etc).
Worst case, yes, you could:
Stop the instance
Detach the EBS volume
Attach the EBS volume to another EC2 instance
Access the content of the EBS volume

EC2 Network Error : Connection timeout

I created an EC2 micro instance of Linux and launched it, created a keypair and all the beginning stuff specified in this video:
http://www.youtube.com/watch?v=hJRSti6DsJg
But when I connect to my instance with PUTTY terminal, it will not connect to my EC2 instance.
I have specified the correct instance Public DNS and private key which I created with the PUTTY key generator.
I get this error:
Network Error : Connection timeout
You simply need to add an ssh rule for inbound connections to you ec2 instance in the ec2 management console.
Go to ec2 console
Click Instances on Left
Select your instance
In the Description tab, locate Security Groups and click the available group link
Click edit button on Inbound tab
Click Add Rule and select SSH for type, Port Range 22, and Source Anywhere
Connect with putty :)
Are you sure you've enabled SSH access in the firewall settings of your instance?
Can you connect with PUTTY to other machines? Perhaps your local firewall is blocking SSH connections.
Also, you cannot simply use a private key you generated with PUTTY - you'll have to create one using the AWS web interface, assign it to your EC2 instance, download it to your local computer and instruct PUTTY to use this when connecting to your EC2 instance.
Having successfully connected in the past, I got this error after shutting down my instance and starting it again.
Apparently the Public DNS changes after you shut it down and start it again, so I had to replace the DNS string in PuTTY before it could find my instance to connect with it on port 22.
This error may occur when you enable ufw and reboot your instance. First you have to add 22/tcp before enabling ufw. Following is the command
$ ufw allow 22/tcp
If you already made the mistake. Then follow the following guide
Start a recovery instance.
Stop the blocked instance (DON'T TERMINATE)
Detach the blocked instance volume.
Attach Blocked volume to the recovery instance.
Log to the recovery instance(Newly Launched) via ssh/putty
Type sudo lsblk to display attached volumes
Verify the name of the Blocked volume. Generally start with /dev/xvdf.
Mount blocked volume.
$ sudo mount /dev/xvdf1 /mnt
$ cd /mnt/etc/ufw
Open ufw configuration file
$ sudo vi ufw.conf
Enable insert mode by pressing i in vi editor
Update ENABLED=yes to ENABLED=no
ClickESC and type :wq to update the file.
Verify the file contents. where update to ENABLED=yes -> ENABLED=no
$ sudo cat ufw.conf
Remove the mounted blocked volume from recovery instance
$ cd ~
$ sudo umount /mnt
Now detach blocked install volume from recovery instance and re-attach it to the original instance as /dev/sda1.
Finally, Start the blocked instance. Here's you will able to access your instance. If you enable ufw again don't forget to allow 22/tcp.
One more thing to remember when using putty to connect to, add security setting to accept ssh connection.
If you take default security group
I faced a similar issue.
Reason : Since in my Security Group, Inbound traffic to Port 22 was set to MyIP, now since every time I connected to Internet, I got assigned a different IP, while in our Security Group the Inbound Traffic to SSH was expected to come from previous IP only.
Solution : Edit the Security Group, and either make the Inbound Traffic to Port 22 as 'Anywhere'(not recommended) or again click the myIP(which will give the current IP assigned to you). Problem will be fixed.
I'm also facing the same Network Error : Connection timeout issue after keep all thing at correct place.
But in my case, internet gateway(igw) is not working so you guys also check default VPC configuration for troubleshooting.
This error is generally occurred because server not responding means source have not clear path to connect to you server even you put SSH 22 port in SG.
I faced this issue. Possible cases are
Make sure to open port number-22(ssh) in Inbound rule of Security group
Make sure to use correct .ppk file
Check Network settings. The VPC in which you have launched an instance, may not have
attached to Internet gateway. This happens when you accidentally delete the default
Internet-
gateway.
a) Create Internet gateway and attach to the mentioned VPC
b) In route table->route, add Internet gateway with (0.0.0.0/0)
This should fix the problem. :-)
You simply need to add an ssh rule for inbound connections to you ec2 instance.
Go to ec2 console Security Groups
Select your Security Groups
Click edit button on Inbound tab
Click Add Rule and select SSH for type, Port Range 22, and Source Anywhere or My IP
Click save rules button
Now connect it's working

Resources