I am using an ami which has datadog enabled and configured to run. In my dev environment, I want to use the same ami, but stop the instance from registering to datadog. So, I was thinking about using a cronjob that triggers on reboot that would check to see if the instance is in the dev environment, and if so, disable the datadog agent, which will hopefully prevent the instance from registering on datadog. I know that the datadog agent is enabled and configured in the cloud-init, which also runs upon the first spinning up of the instance. So, is it possible to run the cronjob before cloud-init, or does it already do so, or does cloud-init run first no matter what? Thanks!!
edit: It turns out that the datadog agent is not part of cloud-init, but runs alongside it. So I suppose my question shifted to is it possible to confirm that I can run a cronjob before the datadog agent configures?
Related
I have scheduled EC2 instance to start every morning using Lambda. I have configured some task in task scheduler which I want it to run once EC2 is up however task wont get triggered until I login to EC2 instance. I want my task scheduler runs automatically at defined time without login to EC2 instance. Its a windows instance.
I tried configuring task without login but it doesn’t invoke.
Did you choose the Run whether user is logged on or not option of the task?
Fix Scheduled Task Won’t Run for .BAT File
Next, you have to choose the Run whether user is logged on or not option
To run a script every time an Amazon EC2 instance starts, simply add the script to this directory:
/var/lib/cloud/scripts/per-boot/
Most Amazon EC2 AMIs have cloud-init installed. It is responsible for running scripts from User Data. It will also check the above directory and run any scripts it finds there. This does not require a user to login.
Im trying to install Velero in a Kubernetes cluster on GCP, I have successfully installed it the first time but every time I logged back to the GCP shell the command is not available anymore, (as if the installation disappeared), I still can see my velero backups tho when I install it again
How can I persistently install any utility?
Bear in mind that Cloud Shell VM is ephemeral and is turned down when you are not using it interactively. See the documentation below:
The virtual machine instance that backs your Cloud Shell session is
not permanently allocated to a Cloud Shell session and terminates if
the session is inactive for an hour. After the instance is terminated,
any modifications that you made to it outside your $HOME are lost.
Also, I am not sure how exactly you have installed Valero in your GCP cluster but you might find this guide useful. It shows how to install Velero with HELM in GCP, step by step.
I have a .bat file on a windows ec2 instance I would like to run every day.
Is there any way to schedule the instance to run this file every day and then shut down the ec2 instance without manually going to the ec2 management console and launching the instance?
There are two requirements here:
Start the instance each day at a particular time (This is an assumption I made based on your desire to shutdown the instance each day, so something needs to turn it on)
Run the script and then shutdown
Option 1: Start & Stop
Amazon CloudWatch Events can perform a task on a given schedule, such as once-per-day. While it has many in-built capabilities, it cannot natively start an instance. Therefore, configure it to trigger an AWS Lambda function. The Lambda function can start the instance with a single API call.
When the instance starts up, use the normal Windows OS capabilities to run your desired program, eg: Automatically run program on Windows Server startup
When the program has finished running, it should issue a command to the Windows OS to shutdown Windows. The benefit of doing it this way (instead of trying to schedule a shutdown) is that the program will run to completion before any shutdown is activated. Just be sure to configure the EC2 instance to Stop on Shutdown (which is the default behaviour).
Option 2: Launch & Terminate
Instead of starting and stopping an instance, you could instead launch a new instance using an Amazon CloudWatch Events schedule.
Pass the desired PowerShell script to run in the instance's User Data. This script can install and run software.
When the script has finished, it should call the Windows OS command to shutdown Windows. However, this time configure Terminate on Shutdown so that the instance is terminated (deleted). This is fine because the above schedule will launch a new instance next time.
The benefit of this method is that the software configuration, and what should be run each time, can be fully configured via the User Data script, rather than having to start the instance, login, change the scripts, then shutdown. There is no need to keep an instance around just to be Stopped for most of the day.
Option 3: Rethink your plan and Go Serverless!
Instead of using an Amazon EC2 instance to run a script, investigate the ability to run an AWS Lambda function instead. The Lambda function might be able to do all the processing you desire, without having to launch/start/stop/terminate instances. It is also cheaper!
Some limitations might preclude this option (eg maximum 5 minutes run-time, limit of 500MB disk space) but it should be the first option you explore rather than starting/stopping an Amazon EC2 instance.
I have a script that I need to run once a day that requires a lot of memory. I would like to run it on a dedicated amazon box.
Is there some automated way to build a box, download all required software (like ruby) and then run my script. After the script is ran, I would like to shutdown the box.
The two options I can think of are:
I am thinking about hacking EMR to do this. (My script is a mapper against an empty directory)
Chef - This seemed like too much for one simple script.
You can accomplish setting up a new EC2 instance on startup using the official Ubuntu AMIs, the official Amazon Linux AMIs, and any other AMI that supports the concept of a user-data script.
Create a script (bash, Perl, Python,
whatever) that starts with #!
Pass this script as the user-data when running the EC2 instance.
The script will automatically be run as root on the first boot.
Here's the article where I introduced the concept of a user-data script:
Automate EC2 Instance Setup with user-data Scripts
http://alestic.com/2009/06/ec2-user-data-scripts
Your user-data script can install the required software, configure it, install your work script, and set up a cron job that runs the work script once a day.
ENHANCEMENT:
If the installation script don't take a long time to run (e.g., under an hour or few) then you don't even have to run a single dedicated instance 24 hours a day. You can instead use an approach that lets AWS start an instance for you on a regular schedule.
Here's an article I wrote that provides details on this approach with sample commands:
Running EC2 Instances on a Recurring Schedule with Auto Scaling
http://alestic.com/2011/11/ec2-schedule-instance
The general approach is to use Auto Scaling to start an instance with your user-data script on a regular schedule. Your job will terminate the instance when it has completed. They key is to suspend Auto Scaling's normal desire to re-start instances that terminate so that you don't pay for a running instance until the next time your job starts.
I want to run some scripts to run at the boot of the instance (server by node, and a redis database).
Since Amazon AMI is based on debian, I thought I could use update-rc.d to manage scripts.
Hwoever, when I type update-rc.d it says the command is not found.
What is the correct way of adding a service to the init script?
I know about CloudFront, but that is for the case when one wants to start up the instance for the first time and install some basic programs, right? For my case, I just want my instance to run some programs when it starts running from the reboot.
The image that I am using is amzn-ami-2011.09.2.x86_64.ext4.