I'm trying to share my AMI with a colleague, who is testing his change to my cluster computing application.
While it is easy to change the permission of my AMI to 'public', I can't find its manifest anywhere, including in my S3 buckets.
Is this feature missing for AMI created in EC2 console?
A manifest is only required for an Instance-Store Backed AMI. This is where the AMI is copied to Instance Store (ephemeral storage) rather than an EBS volume. This method is quite old and these days it is recommended to use a normal AMI that is used to create an EBS boot volume.
AMIs created in the EC2 Management Console can be shared either publicly or with with specific accounts.
Related
Had some data on ephemeral storage of a ec2 medium instance. They are lost between server reboot. I had a ami created just before the reboot. Is it supposed to contain that data? I had created a new instance from that ami but can't find the data on new ephemeral drive.
ok. I got it. Create Image only creates ebs backed ami. Creating instance storage backed ami is different. It is described here creating an instance store ami
Ephemeral drives on EC2 are, by definition, ephemeral. This means that their data in not kept between reboots, and not saved as part of the AMI image.
You can read more on what you can an cannot do with these kinds of disks here.
I know it is a newbie question but the documentation is really not clear about that.
I have successfully launched a Cluster Instances Amazon Linux AMI 2011.02.1 cc1.4xlarge instance. I need to upload and process a 15 GB text file and I don't know how to activate the "additional" storage.
By default, I have just 8 GB of storage. Can someone help me writing simple and clear instruction on how to activate all the storage (1690 GB) I should have?
Thanks a lot in advance.
Regards,
Eugenio
You should be able to mount your ephemeral drives by following this guide.
Cheers.
It's a very common question and there's a large confusion that everyone has around this when starting to work with AWS EC2.
There are 2 things to have in mind here :
Instance Type
AMI (Amazon Machine Image, basically a disk snapshot with some operating system)
Each AMI has some requirements that will dictate parts of the configuration of your created instance.
Here we're concerned with the Root Device type of an AMI :
EBS-backed AMI
These AMI's only work on instances that have an EBS volume attached. And therefore if you choose an EBS-backed AMI, the wizard will by default add an EBS volume (usually of size 8 GiB) without you noticing, and you'll be charged for this EBS separately in addition to the EC2 hourly charge. In this case when you ssh to your instance you will only find 8 GiBs of storage and not the large storage you're promised when you view the EC2 Pricing table at https://aws.amazon.com/ec2/pricing/ . And that is confusing.
If you also want to use your -usually large- instance store with these AMIs, then when you create that instance you have to explicitly add a volume and select Instance Store 0 from Volume Type (in Step 4 - Add Storage). You may have more than 1 drives assigned to your instance type, you need to add all of them in this case (Instance Store 0,1,..etc). These volumes can only be added at creation time. If the volumes were automatically mounted then you'll find them at /media/ephemral*, if not, you'll have to mount them manually.
Instance-Store-backed AMI
Those images use the Instance Storage as their root device (the OS will be installed on the instance store at / ). I think the confusion comes from the fact that these AMI's are not instantly viewable in the Quick Start tab of Step 1 at which you choose the AMI. All AMIs in the Quick Start tab are EBS-backed, and Instance Store AMIs are very uncommon these days, everyone uses EBS-backed AMIs because of their various speed and persistence benefits, and that's recommended by Amazon as well.
If you want an instance-store-backed AMI to avoid dealing with any EBS whatsoever, then in Step 1, choose the tab Community AMIs, and filter them by Root Device Type = Instance Store, then choose your AMI. Whether or not you add Instance Store 0 in Step 4 - Add Storage, the instance store will be available.
Check what Amazon Linux AMI type you're launching the image as. In my experience, if you launch with the "EBS-backed" AMI type, it won't mount the ephemeral storage. If you switch to the "Instance Store" type, it will mount it for you at /media/ephemeral0
http://aws.amazon.com/amazon-linux-ami/
Or is it absolutely necessary to have an EBS backed storage? Does anyone have manuals/Howtos?
I am following the manual given on this site.
You can create an AMI from either an EBS or S3-backed running instance. The simplest way is to use the AWS Management Console to select the instance and click 'Create Image' from the Instance Actions menu.
This will create either an EBS or S3-backed AMI, depending on the type of instance.
Be aware that creating an AMI from a Running instance is inherently potentially unsafe - the storage is not imaged as a hard 'point-in-time' snapshot, so changes to the filesystem whilst the image is being created may not be recorded, and may even result in a compromised image. You should quiesce the OS as far as possible before starting.
Note also that your instance will reboot during the image-creation process, so make sure you're prepared for any temporary loss of service from the VM.
If you bundle a Windows EBS instance while it is running, its Administrator password will be reset by Amazon's rebundling tools - so you'll have to use the EC2 "get Admin password" function with instances launched from the new AMI.
If you stop your Windows instance before rebundling, its Administrator password will remain intact through the rebundling process.
I don't know if this is documented anywhere, but it's certainly been my experience with Windows 2003 instances.
you will to refer the manual for the three commands:
ec2-bundle-vol, ec2-upload-bundle, ec2-register.
Keep at hand your private and certificate key (eg. pk.pem, cert.pem files) and your access and shared key. Download the ec2 ami tools and setup the environment variables for the ec2 ami tools. This site might give you the details on how to bundle the volume, create the images, upload them to S3 and finally register it as your own ami.
http://alestic.com/2009/06/ec2-ami-bundle
It is possible to create an AMI from an EBS running instance using the AWS management console. as mentioned by Eight-Bit Guru by clicking 'Create Image' from the Instance Actions menu.
For the S3 backed instance it is currently not possible without bundling the instance.
If you are using a linux ami then you have two ways as explained in this guide:
Creating an AMI from S3 backed instance - For Linux
For windows based S3 backed instance you can follow other guide
Creating an AMI from S3 backed instance - For Windows
Hope this will help for creating an AMI for S3 backed intances.
I am new to Amazon web services. I need to create a server image with a few software packages pre installed on the EC2 instances.
One option I am considering is to create an EBS volume with these packages and then use them to launch EC2 instances.
The other option I am thinking of is to create a private AMI and then use them to launch the EC2 instances.
I am not sure which option is better.
One other slightly related question I have is can I create a private EBS volume and share it with some other account.
You should create your own AMI, as it's probably more efficient. In order to create an instance, you must specify an AMI. Specifying an EBS snapshot as the root device in addition to specifying the AMI (if this even works, I haven't tested it) would just result in the EC2 cloud launching your instance with the AMI, followed by overwriting it with the EBS snapshot for the root volume.
It's easy enough to keep the AMIs laying around as prototypes.
I have 2 EC2 instances, each with their own EBS attached. Sitting infront of the EC2s is a load balancer.
These instances run CMS driven sites, where uses can upload files.
What would be the best solution to the problem of a file getting uploaded to one EBS and the load balancer sending a visitor to the EC2 instance whose EBS does not have the file? Some sort of cron which runs an rsync?
Suggestions very welcome!
Thanks
S
I believe the best solution would be to use single shared storage like Amazon S3. It's better to use some plugin for your CMS to store users' files on S3. But if there is no such plugin you can use Fuse s3fs adapter to mount the file system on both instances and configure your CMS to store those files in that specified directory.
there are several solutions to this problem from top of my head i think
nfs/samba shared dir between instances
svn deploy
cluster file systems - OCFS/GFS
cloud management such as capistrano and trriger a deploy when you need
and of course cron jobs when you can do ftp, scp, rsync, s3sync/copy etc
Or possibly, create one EC2 instance as NFS and share it's directories with your other instances.
There are multiple solutions to keep data in both EC2 in sync with or without using EBS volumes.
Can use AWS EFS service instead of using EBS volumes. EFS volume can be shared between EC2 instances within a VPC, and both instances will have data in sync on the mountpath where EFS is mounted on instances.
Another solution is using Gluster File Storage. This can also work between EBS volumes in different AWS region. Refer this link: http://sanketdangi.com/post/5601762671/gluster-config-aws-multi-az
Can mount S3 bucket on your EC2 instances using S3 Fuse. Refer this link: https://github.com/s3fs-fuse/s3fs-fuse/wiki/Fuse-Over-Amazon
May be you can also use "s3 sync" on both ebs volumes. This way both ebs will be in sync via S3. Refer this link: https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html