Trying to run few steps of CI/CD in a EC2 instance. Please don't ask for reasons.
Need to:
1) Start an instance using AWS CLI. Set few environment variables.
2) Run few bash commands.
3) Stream the command from the above commands into the console of the caller script.
4) If any of the commands fail, need to fail the calling script as well.
5) Terminate the instance.
There is a SO thread which indicates that streaming the output is not as easy. [1]
What I would do, if I had to implement this task:
Start the instance using the cli command aws ec2 run-instances and using an AMI which has the AWS SSM agent preinstalled. [2]
Run your commands using AWS SSM. [3] This has the benefit that you can run any number of commands you want - whenever you want (i.e. the commands must not be specified at instance launch, but can be chosen afterwards). You also get the status code of each command.[4]
Use the CloudWatch integration in SSM to stream your command output to CloudWatch logs. [5]
Stream the logs from CloudWatch to your own instance. [6]
Note: Instead of streaming the command output via CloudWatch, you could also periodically poll the SSM API by using aws ssm get-command-invocation. [7]
Reference
[1] How to check whether my user data passing to EC2 instance working or not?
[2] Working with SSM Agent - AWS Systems Manager
[3] Walkthrough: Use the AWS CLI with Run Command - AWS Systems Manager
[4] Understanding Command Statuses - AWS Systems Manager
[5] Streaming AWS Systems Manager Run Command output to Amazon CloudWatch Logs | AWS Management Tools Blog
[6] how to view aws log real time (like tail -f)
[7] get-command-invocation — AWS CLI 1.16.200 Command Reference
Approach 1.
Start an instance using AWS CLI.
aws ec2 start-instances --instance-ids i-1234567890abcdef0
Set few environment variables.
Use user dat of ec2 to set env. & run commands
..
Run your other logic / scripts
To terminate the instance run below command in the same instance.
instanceid=`curl http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 terminate-instances --instance-ids $instanceid
Approach 2.
Use python boto3 or kitchen chef ci.
Related
I have developed a shell script which connects to the AWS EKS Cluster, Execute a kubectl command, get the result as JSON string and insert into AWS RDS MySQL DB. Tested the script in AWS Cloud Shell which is fine there. Now, I want to schedule the script (using CRON Scheduler may be ) every 30 minutes. I am exploring options on where to schedule the Unix Shell Job. Below are the options I can see from web :-
From an AWS EC2 Instance
Using AWS Systems Manager
For #1, I can start an EC2 instance free tier and schedule the cron job.
For #2, not very sure on how AWS Systems Manager works.
Is there any other approach to schedule a shell job in AWS or GCP ?
i have a shell script which need to be installed over 100 Ubuntu instances/servers.What is the best way to install the same script on all instance without logging into each one.
You can use AWS System Manager , according to AWS Documentation :
You can send commands to tens, hundreds, or thousands of instances by
using the targets parameter (the Select Targets by Specifying a
Tag option in the Amazon EC2 console). The targets parameter accepts
a Key,Value combination based on Amazon EC2 tags that you specified
for your instances. When you execute the command, the system locates
and attempts to run the command on all instances that match the
specified tags
You can Target Instance by tag :
aws ssm send-command --document-name name --targets Key=tag:tag_name,Values=tag_value [...]
or
Targeting Instance IDs:
aws ssm send-command --document-name name --targets Key=instanceids,Values=ID1,ID2,ID3 [...]
Read the AWS Documentation for Details.
Thanks
You have several different options when trying to accomplish this task.
Like Kush mentioned, AWS System manager is great, but is a tightly coupled AWS service.
Packer - You could use Packer to create an AMI of the servers, and have the script installed on them, or just executed whatever the script is doing.
Configuration Management.
Ansible/Puppet/Chef. - These tools allow you to manage thousands of servers with only a couple of commands. My preference would be for Ansible, it is light weight, the syntax is only yaml, connects over ssh, and still allows use of placing shell scripts, if need be.
No matter what instance name I choose, whenever I perform the following on an on prem instance:
aws deploy register --instance-name test --tags "Key=Name,Value=test" --region us-west-2 --debug
The following exception is thrown (always):
2016-04-12 11:02:52,625 - MainThread - awscli.errorhandler - DEBUG - HTTP Response Code: 400
ERROR
A client error (InvalidIamUserArnException) occurred when calling the RegisterOnPremisesInstance operation: Iam User ARN
arn:aws:iam::xxx:user/AWS/CodeDeploy/test is not in a valid format
Register the on-premises instance by following the instructions in "Configure Existing On-Premises Instances by Using AWS CodeDeploy" in the AWS CodeDeploy User Guide.
Despite this error, the user gets created on amazon, and I can continue to register the on prem instance with the following:
aws deploy register-on-premises-instance --instance-name test --iam-user-arn arn:aws:iam::xxx:user/test
aws deploy install --override-config --config-file codedeploy.onpremises.yml --region us-west-2 --agent-installer s3://aws-codedeploy-us-west-2/latest/codedeploy-agent.msi
The instance is registered and the user is created, but when deploying to it, I always get "No hosts succeeded". The logs for the codedeploy agent show no errors.
I am not sure whats happening here either since no logs on either end, in codedeploy console or on the on prem machine codedeploy agent. Any ideas?
Please note I am using Windows Embedded Standard 2010 (which is not in the supported list) with the latest version of aws cli but I have successfully deployed to it in the past (with previous version of aws cli).
Figured it out, seems to be broken* if you try and let 'aws deploy register' create IAM user for you. However, if you create the user first (via console or aws cli), then it will work.
You can pass in the option '--iam-user-arn arn:aws:iam::xxx:user/OnPremCodeDeploy' with the 'aws deploy register' command afterwards.
I created the on prem yml manually with the correct access keys from manually creating user and then finally ran:
aws deploy install --overide-config --config-file conf.onpremises.yml --region us-west-2 --agent-installer s3://aws-codedeploy-us-west-2/latest/codedeploy-agent.msi
* at least w/ codedeployagent OFFICIAL_1.0.1.950_msi and windows embedded
Could you check if the IAM user you registered the on-premises instance with CodeDeploy has proper permissions? Including the following.
"iam:CreateAccessKey",
"iam:CreateUser",
"iam:DeleteAccessKey",
"iam:DeleteUser",
"iam:DeleteUserPolicy",
"iam:ListAccessKeys",
"iam:ListUserPolicies",
"iam:PutUserPolicy",
"iam:GetUser"
This can also be referred here: http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-configure-on-premises-host.html#how-to-configure-on-premises-host-prerequisites
I am using AWS opsworks for statically configuring a simple stack, consisting of two layers (Rails app server and mySQL db).
After having successfully configured and started the stack and deploying my app, I would like to automate the start activity for the stack as part of my pipeline. AWS CLI provides the features to start stacks, retrieve the instance id's of the individual servers and then poll AWS for the completion status ("instance-running"), using the EC2 wait command.
The script below is what I am using (the first command starts the stack, the second command retrieves the instance id's for the two hosts, the third initiates the wait command for these two servers):
#!/bin/bash
aws opsworks --region us-east-1 start-stack --stack-id 9e1b0534-5b38-4fa5-b30c-f849dda8f46b
instance_id=$(aws opsworks --region us-east-1 describe-instances --stack-id 9e1b0534-5b38-4fa5-b30c-f849dda8f46b --query "Instances[].Ec2InstanceId" --output text)
aws ec2 wait --region ap-southeast-1 instance-running --instance-ids $instance_id
When running this script, I always get a "InvalidInstanceID" exception on one of the two id's even though it is definitely the right ID. Secondly, if running the last command in the shell directly while starting the stack in parallel via the AWS console, it turns out that the wait command returns BEFORE the servers are actually up and running (which is the whole point of the exercise).
Lastly, I couldn't find any information about time outs which seem to be quite essential for a blocking async operation. where can the wait timeout be defined ?
Any idea whether there is a glitch in my code, or some specific consideration that I need to take into account?
The aws opsworks describe-instances command is using the --region us-east-1, but the aws ec2 wait command is using --region ap-southeast-1. Are you sure that your instances you're waiting for are in ap-southeast-1 as opposed to us-east-1?
i wanted to automate the ec2 instance's start & stop and configured the crontab on an instance x. I followed these steps
1) Edited the crontab -e of instance X.
2) and added these lines
15 04 * * * username ec2-start-instances i-f1814c90
15 07 * * * username ec2-stop-instances i-f1814c90
10 10 * * * username ec2-start-instances i-f1814c90
3) and restarted the cron using sudo /etc/init.d/cron restart
But still am unable to either start or stop the ec2 instance using cronjob.
thanks,
Anand
Most likely the issue is that the AWS data need to run the ec2 start and stop commands are not in the cron environment.
Its better to write a separate script that does this, instead of of making the ec2 commands on the cron like that.
This is why AWS Data Pipeline is for (working fine):
https://aws.amazon.com/premiumsupport/knowledge-center/stop-start-ec2-instances/
Just mind the trap: --region eu-west-1 NOT --region eu-west-1a (which is an availability zone).
I'd suggest to Schedule EC2 Start / Stop using AWS Lambda
You don't need anything more than a small script or two that you schedule. No instance to launch, just a quick invocation of the script you have built. Pick the programming language of your choice and use the AWS SDK to perform instance operations. A quite lightweight solution,.