Is it possible to deploy instances from AMI to multiple zones in Amazon EC2 using starcluster?
Can anyone give me your feedback on this please?
I need to deploy instances to various Zones in Amazon EC2.
Yes you can. In the addnode command, there is a flag just for that.
-z ZONE, --availability-zone=ZONE
In the load balancer however you can't. I have a fork of StarCluster that enables it though. When you launch it, you specify --ignore-grp. It's only useful when you work with spot instances as it will bid in the cheapest zone.
This fork also supports multiple zone into a VPC, which is also handy.
Related
In my project I have a constraint where all of the traffic received will go to a certain IP. The Elastic IP feature works well for this.
My question is, considering we are using Amazon's docker service (ECS) without autoscaling (so instances/tasks will be scaled manually), can we treat the instances created by the ECS service as we would treat normal, on-demand instances? As in they won't be terminated/stopped unless explicitly done by a user (or API call or whatever).
As is mentioned in the Scaling a Cluster documentation, if you created your cluster after November 24th, 2015 using either the first run wizard, or the Create Cluster wizard, then an Autoscaling group would have been created to manage the instances backing your cluster.
For the most part, the answer to your question is Yes. The instances wouldn't normally go about getting replaced. It is also important to note that because this is backed by an auto scaling group, AutoScaling might go about Replacing unhealthy instances for you. If an instance fails it EC2 Health Checks for some reason, it will be marked as unhealthy, and scheduled for replacement.
By default, my understanding is there are no CloudWatch Alarms or Scaling Policies effecting this AutoScaling group, so it should just be when an instance becomes healthy that it would get replaced.
I am very new to the Amazon AWS services. I was wondering if there is a way to run an instance of EC2 (say, Amazon Linux AMI) and then connect two environments to this instance.
Particularly, I'd like to run a PHP and a Tomcat environment on a single EC2 instance.
The problem is, every time I create a new environment in Elastic Beanstalk, it seems to create a new EC2 instance as well. Am I missing something here?
I'd appreciate any hint on this.
AWS Elastic Beanstalk is designed for deploying your running apps in a way that is designed for scalability from the ground-up. Because of this, Elastic Beanstalk will launch one or more EC2 instances, connect them to an Elastic Load Balancer instance, configure CloudWatch monitoring and Auto Scaling triggers.
Also, because of its fundamental design for scalability, Elastic Beanstalk is designed around a one-app-per-environment model (whereby "environment", I mean one of these EC2 + ELB + CloudWatch + AutoScaling clusters).
Since running two separate web servers with two separate apps (PHP & Java) is not a fundamentally scalable design, it's not a use-case that Elastic Beanstalk is optimized for.
You are free to spin-up a standalone EC2 instance and install whatever you'd like on it, but you're right — git aws.push support has not been made available for standalone EC2 instances. If the git support is important to you, you'll need to weigh the pros and cons of each approach.
I would also like to be able to do this, basically from a cost perspective for demos etc.
For example, a single instance with one PHP app and one Java app. Or, a single instance with two Java apps.
However, from what I have read so far in the Elastic Beanstalk developer guide, I have not found anything explicitly stating that multiple applications per environment is supported (or even, multiple environments per EC2 instance - if that even makes sense).
It makes me wonder if this is a feature that is often requested and planned for the future, or alternatively if the single-app-per-environment model is 'by design' for some reason.
I am working on a HDFS high availability project.
I have configured Hadoop on one Amazon EC2 instance. It is small instance (AMI: Ubuntu server)
I want to form a cluster of EC2 instances. So, i am thinking of replicating the same machine. Does anybody have a clue about how to duplicate this instance on another instance of EC2. If yes, please share.
Thanks!
If your instance is EBS backed, you can make a snapshot and then run as many instance as you want from it.
I usually use Munin as monitoring software, but this (as others software I presume) needs an IP to make the ICMP or whatever pings to collect data.
In Amazon EC2 instances are created on the fly, with IP's you don't know.
How can they be monitored ?
I was thinking about using amazon console commands to read the IP's of the instances up, and change the monit configuration file on the fly also , but it can be too complicated ... or not?
Any other solution / suggestion ?
Thank you
I use revealcloud to monitor my amazon instances. You can install it once and create an ami from that systen, or bootstrap the install command if that's your method. Since the install is just one command, it's easy enough to put into the rc.local (or similar). You can then see all the instances in the dashboard or topiew as soon as they boot up.
Our instances are bootstrapped using chef recipes, so it's easier for me to provide IPs/hosts as they (= all members of my cluster) get entered into /etc/hosts on start-up. Generally, it doesn't hurt to use elastic IPs for a master server and allow all connections (in /etc/munin/munin.conf by default).
I'd solve the security 'question' on the security groups level. E.g. allow only instances with a certain security group to connect to the munin-node process (on port 4949). The question which remains is.
E.g., using ec2-authorize you can achieve
ec2-authorize mygroup -o monitorgroup -u <AWS-USER-ID>
This means that all instances with group monitorgroup can access resources on instances with mygroup.
Let me know if this helps!
If your Munin master and nodes are all hosted on EC2 than it's better to use internal hosts like domU-00-00-00-00-00-00.compute-1.internal. because this way you don't have to deal with IP addresses and security groups.
You also have to set this in /etc/munin/munin-node.conf:
allow ^.*$
You can read more about it in Monitoring AWS Ubuntu Instances using Munin
But if your Munin master is not on EC2 your best bet is to attach Elastic IP to your EC2 instance.
When autoscaling my EC2 instances for application, what is the best way to keep every instances in sync?
For example, there are custom settings and application files like below...
Apache httpd.conf
php.ini
PHP source for my application
To get my autoscaling working, all of these must be configured same in each EC2 instances, and I want to know the best practice to sync these elements.
You could use a private AMI which contains scripts that install software or checkout the code from SVN, etc.. The second possibility to use a deployment framework like chef or puppet.
The way this works with Amazon EC2 is that you can pass user-data to each instance -- generally a script of some sort to run commands, e.g. for bootstrapping. As far as I can see CreateLaunchConfiguration allows you to define that as well.
If running this yourself is too much of an obstacle, I'd recommend a service like:
scalarium
rightscale
scalr (also opensource)
They all offer some form of scaling.
HTH