What's the recommended method to automatically run 'yum update' after a new EC2 Instance launch (Instance based on Amazon Linux AMI)
You could use a CloudInit User-Data Script.
I would personally configure a shell script to pass as user data, which would run all needed server set up tasks.
Configure an init task on a fresh EC2 instance to run yum update on some sort of schedule (you generally want to ensure that all of your instances are always running identical versions of software), then create a new AMI from the running instance you just modified.
From then-on, simply launch your new AMI instead of a fresh AMI. Over time, you'll want to do this same thing again so that instance launches won't take so long as more and more updates become available.
Once every 3-4 months seems to be a good delta.
Related
I have an Amazon EC2 instance which is registered to a cluster of Amazon ECS.
And I want to change this instance's type from c4.large to c4.8xlarge.
I'm able to change its type from c4.large to c4.8xlarge in AWS console. But after the change, I found
[ERROR] Could not register module="api client" err="ClientException: Container instance type changes are not supported. Container instance XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX was previously registered as c4.large.
being printed in /var/log/ecs/ecs-agent.log.20XX-XX-XX-XX file.
Is it possible to change ec2 instance type and re-register it to a cluster?
I think maybe deregister it first, then register it again should work. But I'm afraid this may cause something irreversible in my AWS working environment. So I haven't tried this method yet.
To solve this connection problem between the agent and cluster, just delete the file /var/lib/ecs/data/ecs_agent_data.json and restart docker and ECS.
After that, a new container instance will be created in your cluster with the new size.
sudo rm /var/lib/ecs/data/ecs_agent_data.json
sudo service docker restart
sudo start ecs
Then you can go to the ECS cluster console and deregister the old container instance
UPDATE:
According to #florins and #MBear commented below, AWS updated the data file on ECS instances.
sudo rm /var/lib/ecs/data/agent.db
sudo service docker restart
sudo start ecs
As of March 2021 / AMI image ami-0db98e57137013b2d, /var/lib/ecs/data/ecs_agent_data.json mentioned in the last useful answer does not exist. For me, the commands to execute on the changed instance were:
sudo rm /var/lib/ecs/data/agent.db
sudo service docker restart
After that, it was possible to deploy containers to the instance, without fresh registration (AWS automatically registered a second ECS container instance of the new type). I did have a leftover container instance with the resources of the old instance type to remove.
You can't do this. Per their docs:
The type of EC2 instance that you choose for your container instances determines the resources available in your cluster. Amazon EC2 provides different instance types, each with different CPU, memory, storage, and networking capacity that you can use to run your tasks. For more information, see Amazon EC2 Instances.
This means that when you launch a container on an instance, the agent gathers a bunch of metadata about the instance to run it. If you change it, all of that metadata (or a lot) has changed in a bad way. CPU units, memory, etc. The agent is aware of this and will report it as an error.
You should spin up a new instance of the new type and register it to the cluster and let the task run on it. If it's a service, just terminate the old instance and let it run it against the new one.
I can't think of any real reason why terminating your old instance would cause something irreversible unless it is misconfigured or fragile via user specific settings, by default this would not cause anything destructive.
As alternative approach if the EC2 instance does not store any valuable a new instance using the old instance as template could be started. This takes all existing values and can be achieved just with a few clicks in minutes.
Select the EC2 instance and then "Actions -> Images and templates -> Start more like this". Just change the instance type.
When the instance is running got the the ECS cluster to the tab "ECS instances" and activate the new created instance.
Shutdown the old instance
Update your task maybe taking more cpu and memory and update the service to take the new task revision
I've been playing around with EC2 Autoscaling but was wondering how do I make changes to these instances once launched? Do I need to create a new AMI then re-launch everything?
Use Puppet or Chef to rapidly change production settings and then rotate the AMIs from time to time to keep AMIs updated so your configuration management app does not need to apply too much changes on startup.
It depends what types of updates you are trying to do.
Setting up a new AMI is certainly one way to go. I do this when i need to make changes to the system configuration.
Use some type of automated release process or tools. There are a lot of ways to do do this and it will likely depend on your application. But you can set up a process which would update your application or other system configuration as needed without relaunching a new AMI.
I have a script that I need to run once a day that requires a lot of memory. I would like to run it on a dedicated amazon box.
Is there some automated way to build a box, download all required software (like ruby) and then run my script. After the script is ran, I would like to shutdown the box.
The two options I can think of are:
I am thinking about hacking EMR to do this. (My script is a mapper against an empty directory)
Chef - This seemed like too much for one simple script.
You can accomplish setting up a new EC2 instance on startup using the official Ubuntu AMIs, the official Amazon Linux AMIs, and any other AMI that supports the concept of a user-data script.
Create a script (bash, Perl, Python,
whatever) that starts with #!
Pass this script as the user-data when running the EC2 instance.
The script will automatically be run as root on the first boot.
Here's the article where I introduced the concept of a user-data script:
Automate EC2 Instance Setup with user-data Scripts
http://alestic.com/2009/06/ec2-user-data-scripts
Your user-data script can install the required software, configure it, install your work script, and set up a cron job that runs the work script once a day.
ENHANCEMENT:
If the installation script don't take a long time to run (e.g., under an hour or few) then you don't even have to run a single dedicated instance 24 hours a day. You can instead use an approach that lets AWS start an instance for you on a regular schedule.
Here's an article I wrote that provides details on this approach with sample commands:
Running EC2 Instances on a Recurring Schedule with Auto Scaling
http://alestic.com/2011/11/ec2-schedule-instance
The general approach is to use Auto Scaling to start an instance with your user-data script on a regular schedule. Your job will terminate the instance when it has completed. They key is to suspend Auto Scaling's normal desire to re-start instances that terminate so that you don't pay for a running instance until the next time your job starts.
I want to run some scripts to run at the boot of the instance (server by node, and a redis database).
Since Amazon AMI is based on debian, I thought I could use update-rc.d to manage scripts.
Hwoever, when I type update-rc.d it says the command is not found.
What is the correct way of adding a service to the init script?
I know about CloudFront, but that is for the case when one wants to start up the instance for the first time and install some basic programs, right? For my case, I just want my instance to run some programs when it starts running from the reboot.
The image that I am using is amzn-ami-2011.09.2.x86_64.ext4.
I was searching for EC2 EBS storage Centos 5.4 AMI in the community AMI and eventually I found Rightscale AMI (I think they called it RightImage).
Now I have created instance using that AMI, but I found out there is some Rightscale stuff inside which is worrying me about the safety on using it. I found out there are the following files in that AMI:
/etc/init.d/rightimage
/etc/init.d/rightlink
/etc/init.d/rightscale
/home/ec2
/home/s3sync
(may be more other files I haven't found out yet)
I know I can look into the script and folder and see what they do, but since a lot of user here recommended using Rightscale Centos AMI in EC2, I hope may be there is already some gurus here know what those mentioned script and folder doing and could advice me
i)whether is it safe to delete them. (I'm more concern on whether my data in the server will be safe by using this AMI)
ii)any installed apps in RightScale AMI that should be deleted
And if you think there is other free EC2 Centos AMI that is secure and solid, do suggest as well, thanks !
In order for RightScale to properly manage instances in ec2 they use a ruby based daemon called RightLink as a communication device between their core platform and each instance that is launched. The init scripts that you saw are required for the instance to self configure itself to the point where it can be managed by RightScale properly.
/etc/init.d/rightimage is the first script that is run. Essentially it just determines the OS, arch version, and installs the correct RightLink package from the S3 bucket. Afterwards it kicks off the /opt/rightscale/bin/post_install.sh script which uses the OS init control tools to register the startup scripts to be invoked on future boots of the OS; this ensures that RightLink will always be started.
/etc/init.d/rightscale is the next script that is run. It initializes RightScale-specific (but not RightLink-specific) system state. It is responsible for caching launch settings (aka userdata) and metadata in /var/spool and installing any available patches to the RightLink agent.
/etc/init.d/rightlink is the final script that is run. It configures and enrolls the RightLink agent idempotently. If configuration and enrollment succeed, rightlink starts the sandboxed monit which starts the persistent agent process. If you're not launching the AMI using the RightScale platform this will never properly enroll because they aren't expecting it to, as such RightScale will have no communication with the instance at all.
Removing all three of these from the image shouldn't in any way harm the overall stability of the image, but from a security standpoint they shouldn't cause any problems if they are present.
If you have any further specific questions about it I'd suggest hopping on their forums at https://forums.rightscale.com/
You could also try #rightscale on freenode.