I am going to buy one AWS Reserved Instance and launch several EC2 instances. But these EC2 instances would not be running across time range. For example, instance 1 will be running from 00:00 to 11:59 and then be stopped. instance 2 will be running from 12:00 to 24:59 and then be stopped.
I'd like to know if only one Reserved Instance can apply to this two instances?
Thank you very much.
In short : Yes
Long: Yes, the same Reserved Instance (RI) would apply to both the instances because RI is just a billing concept. It is not specifically associated with any instance. The RI is automatically associated with any running instance. The instance chosen is such that your bill is the lowest.
So if you buy say 2 RIs and you run 4 instances at a time which match the RI criteria (instance-type, tenancy, region/availability zone and platform), the RI discount is automatically applied to any 2 of them.
Similarily, if you have 1 RI and 2 instances which match the criteria (the 4 attributes mentioned above) but only one instance is in running state at a time (others are in stopped state, not terminated) then yes, the RI discount will apply to both of them.
Just make sure that both instances are not running at the same time & both match the RI attributes.
Edit 1
AWS Billing automatically applies your RI’s discounted rate when attributes of EC2 instance usage match attributes of an active RI.
Refer this link. If you need further clarification/confirmation, you could contact AWS Customer Support by raising an Account & Billing Support case from your AWS Support center. You do not need to buy a support plan to raise billing cases & RIs are just a billing concept.
Related
Azure Function architecture:
So we are setting a pipeline in Azure Datafactory, it contains 4 triggers to trigging Function1 at the same time with 4 different parameters.
Pipeline -> Function1-param1 Function1-param2 Function1-param3 Function1-param4
Yesterday, I tried to trigger that pipeline 2 times in 5 minutes, e.g. 10:30 and 10:31. That means, that time I tiggered Function1 for 8 times in 5 minutes.
Pipeline ->
time 1 Function1-param1 Function1-param2 Function1-param3 Function1-param4 10:30
time 2 Function1-param1 Function1-param2 Function1-param3 Function1-param4 10:31
Strange thing is that, we expect 8 calls running parallel cause FUNCTION_WORKER_PROCESS_COUNT is setting to 10, but there are only 6 calls running parallel, 2 are running after that.
So the question is, what is the relationship between FUNCTION_WORKER_PROCESS_COUNT and the tasks that can running parallel.
Function is written in Powershell 7.
AFAIK, the maximum number of worker processes per Function host instance is limited by the FUNCTIONS WORKER PROCESS COUNT variable. These instances are treated as independent VMs, with the FUNCTIONS WORKER PROCESS COUNT limit applied to each one.
If the FUNCTIONS WORKER PROCESS COUNT is set to 10, each host instance will perform 10 individual functions at the same time.
Multiple Workers means Multiple Process Ids of the Same Function App which is a logical collection of Functions.
One Worker Process can host all functions of one Function App where the Single Host has the default value as 1 to the FUNCTIONS_WORKER_PROCESS_COUNT and the Function Host means it is the Physical/Virtual Host where Function App runs as Windows/Linux Process.
Refer here for more information on mechanism of FUNCTIONS_WORKER_PROCESS_COUNT.
Scaling out also allows you to adapt to changes in demand more quickly.
To meet resource requirements, services may usually be swiftly added or removed. This flexibility and quickness efficiently saves spending by only using (and paying for) the resources required at the moment.
Refer this article for more information on the benefits of Scale Up and Scale out.
References of more information on FUNCTIONS_WORKER_PROCESS_COUNT:
Azure Functions - Functions App Settings - functions_worker_process_count
Azure Functions - Best Practices - FUNCTIONS_WORKER_PROCESS_COUNT
I have one spring scheduler , which I will be deploying in 2 different data center.
My data centers will be in active and passive mode. I am looking for a mechanism where passive data center scheduler start working where that data center become active .
We can do it using manually changing some configurations to true/false but , I am looking for a automated process.
-Initial state:
Data center A active - Scheduler M is running.
Data center B passive - Scheduler M is turned off.
-May be after 3 days.
Data center A passive - Scheduler M turned off.
Data center B active - Scheduler M is starting
I don't know your business requirements but unless you want multiple instances running but only one active, the purpose you will have a load balancer would be to spread the load to multiple instances of the same application rather to stick with only one instance.
Anyway I think an easy way of doing this without using a very sophisticated mechanism (coming with a lot of complexity depending where you run your application) would be this:
Have shared location such as a semaphore table in your database storing the ID of the application instance owning the scheduler process
Have a timeout set for each task. Say if the scheduler is supposed to run every two minutes set the timeout to two minutes.
Have your schedulers always kick off on all application instances
Once the tasks kicks off first check if it is the one owning the processing. If yes do the work, if not go at point 7.
After doing the work record the time stamp of the task completion in the semaphore table
Wait for the time to pass for the next kick off
If not the one owning the processing check when the task last run in the semaphore table. If the time since last run is greater than the timeout set for that process take the ownership of the process (recording your application instance id in the semaphore table)
We applied this and it ran very well with one of our applications. In reality it was much more complex than explained above as we had a lot of application instances and we had to avoid starting an ownership battle between them. To address this we put in place a "Permission to process request" concept so no matter how many instances wanted to take control it was only one which was granted.
For another application with similar requirements we used a much much easier way to achieve this but the price we paid was some extra learning curve in using ILock from Hazelcast IMGB framework. That is really very easy but keep in mind the Hazelcat community edition comes with absolutely no security and paying for a Hazelcast license just to achieve this may be a bit of expense.
Again all depends on you use case, for us the semaphore table was good enough in first scenario but prove bad in the second one as the multiple processes trying to update the same table at the same time ended up with a lot of database contention which took us to Hazelcast.
Other ideas would be a custom health check implementation that could trigger activating one scheduler or the other depending of response received.
Hope that helps, just ideas from our experience. Good luck.
I want to stop EC2 instances after office hours to save costs. How can I do the same with ECS instances? Even if I stop all tasks/services, the instance is still there? Do I stop the EC2 instance directly?
From EC2 Management Console
Click Auto Scaling Groups from the left menu.
Select the group from the list.
Click edit on the details tab.
Set desired property to '0'.
After clicking save it is all done.
The Auto Scaling Group is smart enough to shut down all instances.
You can use the "Scheduled Actions" feature of Auto Scaling Groups.
Starts similar to Kerem Baydoğan's answer
From EC2 Management Console:
1 Click Auto Scaling Groups from the left menu.
2 Select the group from the list.
3 Select "Scheduled Actions" from the bar that appeared in the lower middle of the screen.
4 Click on create scheduled action
5 Fill the fields as you see fit and notice that under recurrence there is also a cron option for extra flexibility.
If you have the cluster set to a minimum number of nodes with an asg. If you turn off the nodes the asg will start another node to bring it up to three minimum number of nodes. You must set the asg to zero nodes.. Then turn off the current nodes.
Yes, just stop the EC2 instance directly. When you start the instance again during office hours, the ECS agent will make the services start according to their desired value.
We are doing the same thing and it works for us.
I have an EC2 instance that I want to scale based on the number of messages in a SQS queue. If there are many messages (for 5 minutes) I want to pop up a new EC2, for consuming faster the messages. Then if the messages are few (for 5 minutes), I want to pop down the oldest EC2. This way, if the service that consumes the messages stops for some reason, I will terminate the old EC2, and the service will run.
I have created an AutoScalling for this. I have set the TerminationPolicy to OldestInstance, but it works as I expect only if I set just one zone (eg: eu-west-1a): it creates a new instance and terminates the oldest each time. But if I have 3 regions (eu-west-1a, eu-west-1b, eu-west-1c), it just launches and terminates the instances not in the OldestInstance manner. Or, at least, not as I expect: delete the oldest every time. Is there something linked to different zones? On this pace I have not found anything about it, except for the default policy.
And even if the case linking to multiple zones from default policy is applied, I can have maximum only 2 instances that turn at the same time. And they are always launched in a new zone.
This is probably the key paragraph:
When you customize the termination policy, Auto Scaling first assesses the Availability Zones for any imbalance. If an Availability Zone has more instances than the other Availability Zones that are used by the group, then Auto Scaling applies your specified termination policy on the instances from the imbalanced Availability Zone. If the Availability Zones used by the group are balanced, then Auto Scaling selects an Availability Zone at random and applies the termination policy that you specified.
I interpret this to mean that if you have instances in multiple zones, and those zones are already balanced, then AWS will select a zone at random AND THEN pick the oldest instance, within the randomly selected zone - it won't pick the oldest instances across AZ's, it picks and random AZ and then the oldest instance is terminated within that AZ.
when you put your instances on autoscale & set the minimum & maximum limit , does it scale to the maximum anyway or does it check for the threshold for each instance created by as and then spawn?
As in , 1 instance triggers an alarm for 90% cpu usage -> AutoScale creates 1 instance more (total 2) -> does it check for 90% cpu usage on both the instances & THEN create new instances or does it create new instances anyway after the cooldown time ?
Auto scaling will observe the average CPU usage of every instance and then decide if a new instance is needed or an existing can be shut down. This will ensure that you only use (and pay for) what you need.