need pretty trivial task
i have server, which in crontab every night will run "something" what will launch new EC2 instance, deploy there code (ruby script), run it, upon completion of the script shutdown the instance.
how to do it the best?
thanks.
Here's an approach that can accomplish this without any external computer/cron job:
EC2 AutoScaling supports schedules for running instances. You could use this to start an instance at a particular time each night.
The instance could be of an AMI that has a startup script that does the setup and running of the job. Or, you could specify a user-data script be passed to the instance that does this job for you.
The script could terminate the instance when it has completed running.
If you are running EBS boot instance, then shutdown -h now in your script will terminate the instance if you specify instance-initiated-shutdown-behavior of terminate.
Related
I have developed a shell script which connects to the AWS EKS Cluster, Execute a kubectl command, get the result as JSON string and insert into AWS RDS MySQL DB. Tested the script in AWS Cloud Shell which is fine there. Now, I want to schedule the script (using CRON Scheduler may be ) every 30 minutes. I am exploring options on where to schedule the Unix Shell Job. Below are the options I can see from web :-
From an AWS EC2 Instance
Using AWS Systems Manager
For #1, I can start an EC2 instance free tier and schedule the cron job.
For #2, not very sure on how AWS Systems Manager works.
Is there any other approach to schedule a shell job in AWS or GCP ?
I have a EC2 instance that's already launched using ansible ec2 module having user_data(say data1). Stopped the EC2 instance, now I want to modify the user_data(say data2) and start the instance. Giving modified user_data but its not getting reflected on aws.
To summarize, How to modify user_data of stopped aws EC2 using ansible script(with ec2 or ec2_instance) script.
By default, user data scripts and cloud-init directives run only during the boot cycle when you first launch an instance. You can update your configuration to ensure that your user data scripts and cloud-init directives run every time you restart your instance.
User Data for every restart
I am using user-data of ec2 instance to power up my auto scale instances and run the application. I am running node js application.
But it is not working properly. I have debugged and checked the instance cloud monitor output. So it says
pm2 command not found
After reading and investigating a lot I have found that the path for the command as root is not there.
As EC2 user-data when it tries to run it finds the path
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
After ssh as ec2-user it is
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin
After ssh as sudo su it is
/root/.nvm/versions/node/v10.15.3/bin:/sbin:/bin:/usr/sbin:/usr/bin
It works only for the last path.
So what is the way or script to run the command as root during launch of the instance provided by user-data?
All thought to start your application with userdata is not recommended, because as per AWS documentation they are not assuring that instance will only come up after successful execution of user data. Even if user data failed it will spin up your instance.
For your problem, I assume if you give the complete absolute path of the binary, It will work.
/root/.nvm/versions/node/v10.15.3/bin/pm2
Better solution for this approach, create a service file for your application startup and start application with systemd or service.
I have an Amazon EC2 instance running my website. I need to setup a Cron Job to run my file every 12hours.
if file setup via command line so please give a detail step wise.
Does anyone have any advise?
Thanks for your time.
I recently began using Amazon's linux distro on ec2 instances and after trying all kinds of things for cron all I needed was:
sudo service crond start
crontab -e
This allowed me to set a cron job as "ec2-user" without specifying the user. For example:
0 12 * * * python3 example.py
In fact, specifying a user here prevented it from running. I also posted this here.
It's just normal cron.
See: HowTo: Add Jobs To cron Under Linux or UNIX?
Short version:
How do I make init.d scripts run after cloud-init has run the userdata script on an EC2?
Long version:
Our deployment process is to construct AMIs with everything installed on them (tomcat, nginx, application etc), but with certain configuration values missed out. At boot time, the userdata script adds in the missing configuration values, and then the application stack can start up
Our current EC2s are based on an old version of the official Debian AMIs, which have the script ec2-run-user-data. This script runs at boot, and downloads and runs the EC2s userdata. When constructing the AMI, I simple edit the init.d scripts for tomcat, nginx etc to include ec2-run-user-data in their "Required-Start:" line, so they start up after the userdata has been run.
Unfortunately that approach is no longer viable, as we want to start using the hvm base AMIs, which have cloud-init installed rather than ec2-run-user-data. But I can't figure out how cloud-init works well enough to work out how to make the process work.
As far as I can tell, the userdata script is run by the cloud-final step, but cloud-final has $all in it's "Required-Start:" line. I could remove it, but I don't know what consequences that might have.
I've tried making tomcat etc run after cloud-init or cloud-config, but the userdata hasn't run by then. Also, it looks like cloud-init and cloud-config start processes then exit, which might explain why cloud-final needs to have $all in Required-Start
More Info:
We use the 'baked AMI' approach, where we create an AMI with all the packages/applications installed, then tell the existing Autoscaling Groups to replace their EC2s with new ones based on the new AMI (via CloudFormation). Some configuration information isn't known at baking time, but must be inserted via the userdata script.
When our tomcat app starts up it expects to read in the file /etc/appname/application.conf. That file has the text <<REPLACE_THIS>> in it. Tomcat will fail to start up if it tries to run before <<REPLACE_TIME>> has been replaced
The userdata script is something like:
#!/bin/bash
sed -i 's!<<REPLACE_TIME>>!{New value to use, determined at deploy time}!' /etc/appname/application.conf
The default Required-Start for tomcat is "$local_fs $remote_fs $network". At baking time, I change that to "$local_fs $remote_fs $network ec2-run-user-data"
By doing all that, the text in /etc/appname/application.conf gets replaced before tomcat runs. But as I said above, I want to change to using cloud-init, and I can't figure out what I need to do to make tomcat start after cloud-init has run the userdata. I get the impression that cloud-init doesn't run the userdata until very late in the process. I could change the userdata script to contain "/etc/init.d/tomcat restart" at the end, but it seems a bit dumb to have tomcat fail to start then get restarted.