I want to deploy my task (which is implemented in Hive) on AWS EC2.
I just read about the AWS, I came to know that we can create our own environment on EC2.
I am completely new to AWS. So I don't know anything about it
How to create such environment there?
I also read that we can import our VM image?
But for that the connector needed for it having .ova extension. If I download it, How to install it? (having .ova extension)
All this things is possible on AWS Free instance (creating our own environment and also VM Import)
Please guide me through this if you know about this.
To setup an instance on Amazon EC2 you might want to go through following guide: http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/Welcome.html
Regarding importing your own VM image, you might want to go through following link: Amazon EC2 VM Import. They do support importing images of VMWare, Hyper V etc. In the link you will see 2 options how you can import your image:
Command line option
Amazon EC2 VM Import Connector for VMware vCenter
Personally I know that using command line option has option to provide image file in format specific to your virtual machine software. You can see details of command line options at ec2-import-instance API reference.
Importing an image is free as there is no charge for data coming in to AWS. But there is a change for storing an instance created using image import. Every service has some sort of charge associated with it like per hour charge for instance until instance is up. So those charges are applicable regardless you are importing your image or not. Please see Amazon EC2 pricing at EC2 pricing
Related
Im looking for some advice, this may seem like a silly question but I am having some issues with understanding how AWS EBS autoscaling works and its best practices.
I have a laravel application that is deployed to AWS EBS through bitbucket pipelines. This all works and deploys successfully.
My issue is when the autoscaling triggers it then brings up a new EC2 instance and then load balances the traffic. The problem is that the new EC2 instance in the fleet is a blank AWS Linux2 AMI so just shows the nginx welcome page.
I think the issue is that it's using a blank AMI and not getting my application. I am guessing i could create an image from the EC2 image running my application and then scale with that but i would have to do that every time i do a deployment.
Can you configure the auto scaling group to replicate the running EC2 instance?
Any help or advice as to the best way to accomplish autoscaling with my application would be great.
Its depend on the AMI selected in Launch Configuration..
You need to create AMI of your live EC2 instance after you updated your all required softwares, dbs, configurations and verified(tested) for proper work..
then add this AMI to Auto scale Launch Configuration..
you dont need to create AMI for each deployment..
Whenever you makes changes On Ec2 server , or updates your app source code, you need to create new AMI and need to specify that AMI in Autoscale launch configuration.
best practice is to config the auto scale with a user data script. So when the new AMI boots up during the auto scaling it reads the user data (cloud init/upstart). The user data script can pull the code from the git or what ever source control and run the necessary pre-deployment commands.
I have a question about the images to mount on openStack.
I can use any image of any operative system? I guess not... but why?
I found images already suitable for openStack, but what's the different between an image cloud-ready and a normal image?
For instance, I can create a virtual machine with windows desktop? If not, why?
thank you
Cloud-ready images have been customised by the distro maker to run well under a hypervisor such as OpenStack, EC2, kvm, and LXC (not strictly a hypervisor) instead of on physical hardware. This entails removing packages that are only need in physical environments like wireless drivers etc, and adding packages that are useful in a cloud environment. For example during the boot process, cloud-ready images download metadata from the environment such as hostname and networking information. This data is used to "personalise" a new instance when it boots up for the first time.
If you really want to get in to the nuts and bolts of things, the Ubuntu UEC Images page has lots of details about the composition of the Ubuntu cloud images and other information like how to build one yourself.
I'm sure you can create a virtual machine running Windows desktop, but I've never had occasion to do so. If you look at the Amazon page about Windows it's all about running server apps like SQL Server and ASP.NET apps.
As Everett Toews pointed out in a comment above, one of the main things for making an image cloud-ready is that it can retrieve data from the metadata server when it boots up. This is used for things like retrieving the private key and collecting user data.
In addition to CloudInit, there's also Condenser. Or, you can roll your own. OpenStack uses the same protocol as the Amazon EC2 metadata service, so the EC2 metadata docs explain how to access this data.
I am currently experimenting with Amazon EC2 and use standard ec2 console. The web app is ok but I want a better solution. I want to be able to ssh to the instances, monitor them, possibly attach with a debugger etc. Are there any better alternatives to the tool?
You should be able to login to any EC2 instance via SSH using key files and work with it like if it were an ordinary server. To do it you have to create a key pair, download public key to your local machine, and ensure you've selected that key-pair while launching new instance. You are free to install any software you like on the instance, so the way how you would monitor you instance is completely up to you (if you decide not to use AWS console).
Apart from the web console there are also Amazon EC2 API tools (a bunch of ec2 scripts to be run from Linux console) and the Query API. The later is considered to be the most flexible way to manage your cloud infrastructure. There are binding for EC2 in many scripting languages including Python (boto), Perl (Net::Amazon::EC2), Ruby (amazon-ec2 gem), node.js (aws2js).
Otherwise there's no better solution just because EC2 is IaaS service and it is meant to be equally good for almost any task. For your particular needs you'll have to develop or organize your own environment which will suite your unique needs.
Edit:
Since today it is possible to log in to running EC2 Linux instances from AWS web console:
Our third announcement today is about a new feature in the AWS console that makes it even easier for you to use Amazon EC2 Linux instances. Customers have been asking us to enable the ability to log into their instances directly from the AWS console. Starting today, you can log in to your Linux instances from the EC2 console without the need to install additional software clients. Please see the Amazon EC2 Getting Started Guide for details on how to use this new functionality.
Or is it absolutely necessary to have an EBS backed storage? Does anyone have manuals/Howtos?
I am following the manual given on this site.
You can create an AMI from either an EBS or S3-backed running instance. The simplest way is to use the AWS Management Console to select the instance and click 'Create Image' from the Instance Actions menu.
This will create either an EBS or S3-backed AMI, depending on the type of instance.
Be aware that creating an AMI from a Running instance is inherently potentially unsafe - the storage is not imaged as a hard 'point-in-time' snapshot, so changes to the filesystem whilst the image is being created may not be recorded, and may even result in a compromised image. You should quiesce the OS as far as possible before starting.
Note also that your instance will reboot during the image-creation process, so make sure you're prepared for any temporary loss of service from the VM.
If you bundle a Windows EBS instance while it is running, its Administrator password will be reset by Amazon's rebundling tools - so you'll have to use the EC2 "get Admin password" function with instances launched from the new AMI.
If you stop your Windows instance before rebundling, its Administrator password will remain intact through the rebundling process.
I don't know if this is documented anywhere, but it's certainly been my experience with Windows 2003 instances.
you will to refer the manual for the three commands:
ec2-bundle-vol, ec2-upload-bundle, ec2-register.
Keep at hand your private and certificate key (eg. pk.pem, cert.pem files) and your access and shared key. Download the ec2 ami tools and setup the environment variables for the ec2 ami tools. This site might give you the details on how to bundle the volume, create the images, upload them to S3 and finally register it as your own ami.
http://alestic.com/2009/06/ec2-ami-bundle
It is possible to create an AMI from an EBS running instance using the AWS management console. as mentioned by Eight-Bit Guru by clicking 'Create Image' from the Instance Actions menu.
For the S3 backed instance it is currently not possible without bundling the instance.
If you are using a linux ami then you have two ways as explained in this guide:
Creating an AMI from S3 backed instance - For Linux
For windows based S3 backed instance you can follow other guide
Creating an AMI from S3 backed instance - For Windows
Hope this will help for creating an AMI for S3 backed intances.
When autoscaling my EC2 instances for application, what is the best way to keep every instances in sync?
For example, there are custom settings and application files like below...
Apache httpd.conf
php.ini
PHP source for my application
To get my autoscaling working, all of these must be configured same in each EC2 instances, and I want to know the best practice to sync these elements.
You could use a private AMI which contains scripts that install software or checkout the code from SVN, etc.. The second possibility to use a deployment framework like chef or puppet.
The way this works with Amazon EC2 is that you can pass user-data to each instance -- generally a script of some sort to run commands, e.g. for bootstrapping. As far as I can see CreateLaunchConfiguration allows you to define that as well.
If running this yourself is too much of an obstacle, I'd recommend a service like:
scalarium
rightscale
scalr (also opensource)
They all offer some form of scaling.
HTH