I have an ec2 instance serving a webpage with apache. I created an autoscaling group using an AMI of this instance in the launch config. Once CPU went over 80% and the autoscale policy ran, a new instance was created. But the CPU of my original instance continued to rise and the CPU of my new instance remained at 0%.
The new instance was not serving the web page. I am guessing this is because apache was not started with the launch of the image. I tried to ssh into the new instance to run "service httpd start" but I got the following error:
ssh: Could not resolve hostname http://ec2-xxx-xx-xxx-xxx.compute-1.amazonaws.com:
nodename nor servname provided, or not known
Why could I not ssh in? How do I configure autoscaling to automatically start apache on launch?
It would appear that you are attempting to ssh to a host with http:// in the hostname. Remove that and ssh should work.
Assuming that you created an AMI to use in AutoScaling, you would need to ensure that you chkconfig httpd on in the source instance before creating a new AMI for AutoScaling.
In order for you to connect to an EC2 instance you need two things:
The Security Group associated with your instance has an inbound rule that allows SSH communication.
Make sure you have the private key generated for the instance. Note: This is only needed if you chose to use a key in the first place.
If those two things are correct, then you can connect to your instance like this:
ssh -i "PATH_TO_YOUR_KEY.pem" ec2-user#ec2-xxx-xx-xxx-xxx.compute-1.amazonaws.com
For the other point, that is, to make sure you can start apache on launch, you can do two things:
As #atbell mentioned on a previous answer, you can make sure that the chkconfig YOUR_SERVICE on is on the AMI used to start your instance.
You can add a command as user data to your LaunchConfiguration so it runs it as soon as the instance is started:
What this will do is run start YOUR_SERVICE start as soon as the instance can respond to commands. So, whenever your AutoScaling group creates another instance, your service will surely be started. Note that the commands added to the user data field of the LaunchConfiguration are, by default, going to be executed as sudo.
Related
One of my EC2 instances was hacked a few days ago.
I tried logging in via SSH to the server, but I couldn't connect. I am the only one with access to the private key, and I keep it in a safe place.
Luckily, I had a backup of everything and was able to move the web app to a new instance quite fast.
My concern right now is that I don't know how my instance was hacked in the first place.
Why can't I log in via SSH using my private key? I would assume that the private key stored on the server can't be (easily) deleted.
Is there a way I can find out how the hacker gained access to the instance? Perhaps a log file that would point me in the right direction.
Should I attach the EBS volume in question to a new instance and see what's on it or what are my options in this case?
Right now, it seems I have to access at all to the hacked instance.
Thank you!
#Krishna Kumar R is correct about the hacker probably changing the ssh keys.
Next steps:
Security concerns (do these now!):
Stop the instance, but don't terminate yet
Revoke/expire any sensitive credentials that were stored on the instance, including passwords and keys for other sites and services. Everything stored on that instance should be considered compromised.
Post-mortem
Take an EBS snapshot of the instance's root volume (assuming that's where logs are stored)
Make a new volume from the snapshot and attach to a (non-production) instance
Mount and start reading logs. If this is a linux host and you have port 22 open in the firewall, I'd start with /<mount-point>/var/log/auth.log
They might have logged into your machine via password. In ssh config, check the value of: PasswordAuthentication. If it is set to yes, then users can login to the instance remotely via password. Check /var/log/secure for any remote logins. It will show up all logins (password or key based).
If someone logged in as 'root', they can modify the ssh keys.
The fact that you are unable to login to the machine does not mean that it has been "hacked". It could be due to a configuration change on the instance, or the instance might have changed IP address after a stop/start.
Do a search on StackOverflow for standard solutions to problems connecting to an instance and see if you can connect (eg recheck IP address, check security group, turn on ssh -v debugging, check network connectivity & VPC settings, view Get System Log, etc).
Worst case, yes, you could:
Stop the instance
Detach the EBS volume
Attach the EBS volume to another EC2 instance
Access the content of the EBS volume
I booted the DataStax AMI for Amazon EC2, logged in via SSH, but the terminal hangs on "Installation Started":
Cluster started with these options:
--clustername CassandraDev --totalnodes 1 --version enterprise --username **** --password ****
Installation started.
"Installation started" keeps going through suffixes consisting of one, two, and three dots. But nothing happens, I can't quit the installation process, and I can't access any log files to see what might be going on (or I don't know how).
Tried on two separate m3.large instances operating in a VPC subnet, at the us-east-1 region. The exact AMI is datastax_clustering_ami_2.5.1_hvm.manifest.xml (ami-ada2b6c4). On the first instance, I waited about an hour and a half. The second instance I just left online all night, with the same results.
Because this is a VPC, all outbound traffic goes through a NAT server. Security groups allow outbound traffic only on ports 80, 443, and 123. Might there be another outbound port that needs to be opened? Inbound ports do not matter, as the server is not public-facing, but within the subnet I have allowed all traffic on all ports.
Someone else has had a similar issue, but without answers so far: DataStax AMI hangs on
Any help would be appreciated!
Since there were a few tickets that came up recently around the same issue, it seems as though something recently changed within the AMI provisioning side in EC2, or this specific configuration of VPCs had never been used before, which seems a bit unlikely.
The current fix is to add an additional entry into /etc/hostname to get rid of the stderr output that occurs after each sudo command. This in turn doesn't get flagged as an error on the provisioning side.
This has been fixed and patched as documented on this ticket:
https://github.com/riptano/ComboAMI/issues/51.
If you spot any additional issues, feel free to create another ticket there.
Going forward, just launch another set of instances using the same user-data and you should be up and running.
I am trying to setup an amazon ec2 instance for first time.
I've created one with ubuntu 10.4, managed to connect to ssh and installed mongodb, mysql, php and apache which need for my proyect(also python but it is already setup).
Then I associated an elastic ip to the instance, but when I try to open the IP, I can't. It gives timeout.
Could it be that the apache root is not where I think it is?(/var/www/)
You need to check the security group that is associated with the instance. Make sure that you open up port 80.
Also make sure that apache is started, and configured to start on boot.
If you're logged in, you should be able to use wget localhost to verify if apache is serving up pages.
I added a bad script in startup script, turned out this script is blocking and hence ubuntu instance can't boot up to ssh server and I can't ssh into it. Is there a way for me to go to the server console (like from VGA port)? like go to single-user mode or safe mode and fix it?
Thanks in advance.
I don't know of any way to obtain server console access or change boot modes on the EC2 instances...
I had the same issue sometime ago, and I ended up creating a temporary EC2 instance, mounting the root device from the original (failing) EC2 there, modifying the files, then reattaching the device and destroying the temporary instance. Note: you may end up paying more depending on which instance type you launch.
I'm totally new to AWS.
I managed to have an instance that runs PHPMyAdmin.
then I created an image (EBS AMI) for this instant and could not connect any more to my
phpmyadmin interface.
I know it's really stupid, but I don't know why it happens.
thanks
Make sure all needed services (e.g. ssh, Apache / nginx, MySQL) on your server get started when booting. If you create an AMI of your system AWS will shut down your server for the time the image creation takes place.
So ssh into your instance, take a look at the running processes and start the ones which you miss.
If you are taking an image from the AWS console, all services will be stopped and server will be restarted for the image to be created. However, you need to restart all the services ex: mysql, apache etc.