I need to trim a single " from a bash string both from starting and ending. I tried many things, but still didn't get the output.
Note: I tried $a{// \"}, but it didn't work.
The following code is what I have tried:
repoUri=$(aws ecr create-repository --repository-name $reponame | jq ".repository.repositoryUri")
$repoUri
You could use the -r jq option for "raw output" to suppress the double quotes:
aws ecr create-repository --repository-name "$reponame" \
| jq -r '.repository.repositoryUri'
But you don't actually need jq at all – you can use the --query option in the request, and suppress the double quotes with --output text:
aws ecr create-repository --repository-name "$reponame" \
--query 'repository.repositoryUri' --output text
Related
I can't get my escape characters correct for an AWS SSM command with double quotes inside. Here's the last attempt:
aws ssm send-command --instance-ids "i-012345678" --document-name "AWS-RunShellScript" --query "Command.CommandId" --output text --parameters commands='["sudo su","cd /opt/cassandra/bin","cqlsh -e \"select * from system_schema.keyspaces;"\"]'
Essentially it's the last command, the double quotes around the cqlsh command that I can't escape from erroring. Have tried to store it in variable and echo but neither works. Also looked at answers below.
aws ssm send-command not working with special characters
Send multiple lines of script to EC2 instance from PowerShell SSM cmdlets
Per the documentation,
v1 docs
v2 docs
quoting and escaping rules
You do not need to escape double quotation marks embedded in the JSON string, as they are being treated literally.
Could you instead try the below?
aws ssm send-command \
--instance-ids "i-012345678" \
--document-name "AWS-RunShellScript" \
--query "Command.CommandId" \
--output text \
--parameters commands='[{"sudo su","/opt/cassandra/bin/cqlsh -e \"SELECT * FROM system_schema.keyspaces;\""}]'
I am calling AWS CLI commands in a bash script. I have a need to add tags to files whose prefix is as follows:
/base/user1/foo/file1
/base/user2/foo/fileA
/base/user3/foo/fileX
I only want to delete those under "foo", but if user has files like:
/base/user1/bar/fileZ
, I don't want to delete those under "bar".
I have a script:
!/bin/bash
aws s3api list-objects --bucket myBucket --query 'Contents[].{Key: Key}' --prefix myPrefix --output text | xargs -n 1 aws s3api put-object-tagging --bucket myBucket --tagging "{\"TagSet\": [{ \"Key\": \"myKey\", \"Value\": \"myValue\" }]}" --key
This works fine as long as myPrefix is an absolute path like:
/base/user1/foo/
, but I have too many users to manual do this, so I wanted to do something like:
/base/*/foo/
for the prefix. However, that throws an error:
An error occurred (NoSuchKey) when calling the PutObjectTagging operation: The specified key does not exist
Is there a way in bash in a loop or something that traverses down to the "foo" level so that I can have the full path: prefix /base/user1/foo/, /base/user2/foo, /ase/user3/foo dynamically defined for the prefix? Thanks for any response.
I was able to find a decent solution on my own. Replacing this line from the original post:
aws s3api list-objects --bucket myBucket --query 'Contents[].{Key: Key}' --prefix myPrefix --output text | xargs -n 1 aws s3api put-object-tagging --bucket myBucket --tagging "{\"TagSet\": [{ \"Key\": \"myKey\", \"Value\": \"myValue\" }]}" --key
, with the following:
for BUCKET_PATH in $(aws s3 ls --recursive --summarize $BUCKET);
do
if [[ $BUCKET_PATH == *$PREFIX* ]]; then
echo $BUCKET_PATH
aws s3api put-object-tagging --bucket ${BUCKET} --key ${BUCKET_PATH} --tagging "{\"TagSet\": [{ \"Key\": \"${KEY}\", \"Value\": \"${VALUE}\" }]}"
fi
done
works great.
This commands returns all the AWS regions separated by whitespace:
aws ec2 describe-regions --query 'Regions[*].RegionName' --output text
eu-north-1 ap-south-1 eu-west-3 eu-west-2 eu-west-1 ap-northeast-2 ap-northeast-1 sa-east-1 ca-central-1 ap-southeast-1 ap-southeast-2 eu-central-1 us-east-1 us-east-2 us-west-1 us-west-2
I'm trying to pipe this to xargs but it's seeing it as a single string:
aws ec2 describe-regions --query 'Regions[*].RegionName' --output text | gxargs -I {} aws cloudformation list-stacks --region {}
Invalid endpoint: https://cloudformation.eu-north-1 ap-south-1 eu-west-3 eu-west-2 eu-west-1 ap-northeast-2 ap-northeast-1 sa-east-1 ca-central-1 ap-southeast-1 ap-southeast-2 eu-central-1 us-east-1 us-east-2 us-west-1 us-west-2.amazonaws.com
gxargs: aws: exited with status 255; aborting
gxargs is just gnu xargs (I'm on Mac).
Also, tried this to use jmespath to create a string from an array with a specific delimiter (which I could use with xargs):
aws ec2 describe-regions --query 'Regions[*].join(",",#.RegionName)'
In function join(), invalid type for value: None, expected one of: ['string'], received: "null"
EDIT: just following up, this is what I wound up with. It insists on throwing an error when it doesn't find a stack- probably the same for other aws cli commands
aws ec2 describe-regions --query 'Regions[*].RegionName' --output text | gxargs -n 1 sh -c 'aws cloudformation describe-stacks --stack-name findme --region $0 || true'
Here's man xargs for -I:
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks
do not terminate input items; instead the separator is the newline character. Implies -x and -L 1.
You can use xargs -n 1 aws cloudformation list-stacks --region instead
Trying to run below 2 commands in Jenkins file
NOTE: below commands are working fine locally where Jenkins is installed
sh ''' aws ecr start-image-scan --registry-id 123 \
--repository-name test1 \
--image-id imageTag=${BUILD_NUMBER} --output json | tee ecr_start_scan_${BUILD_NUMBER}.txt'''
sh ''' aws ecr describe-image-scan-findings --registry-id 123 \
--repository-name test \
--image-id imageTag=${BUILD_NUMBER} --output json | tee ecr_scanResult_${BUILD_NUMBER}.txt'''
Below is the output for both the commands:
+ aws ecr start-image-scan --repository-name valhalla --image-id imageTag=13 --region ap-southeast-1 --output json
+ tee ecr_start_scan_13.txt
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument operation: Invalid choice, valid choices are:
batch-check-layer-availability | batch-delete-image
batch-get-image | complete-layer-upload
create-repository | delete-lifecycle-policy
delete-repository | delete-repository-policy
describe-images | describe-repositories
get-authorization-token | get-download-url-for-layer
get-lifecycle-policy | get-lifecycle-policy-preview
get-repository-policy | initiate-layer-upload
list-images | put-image
put-lifecycle-policy | set-repository-policy
start-lifecycle-policy-preview | upload-layer-part
get-login | help
Update AWS CLI version. I had the same issue with aws-cli/1.11.13. But got the expected result in aws-cli/1.18.16
Yes updating the AWS CLI Version fixes the problem but I think there's a missing step in the middle which is aws ecr wait image-scan-complete because scan results don't show up instantaneously so this command waits till results are accessible.
I have a shell script as given below. This script actually add AWS instance in autoscalling scale in protection group. When I run individual commands that went fine. But when I created a shell file and tried to execute same there are error. See below script
set -x
INSTANCE_ID=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
ASG_NAME=$(aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" --region us-east-2 | jq '.Tags[] | select(.["Key"] | contains("a:autoscaling:groupName")) | .Value')
ASG_NAME=$(echo $ASG_NAME | tr -d '"')
aws autoscaling set-instance-protection --instance-ids $INSTANCE_ID --auto-scaling-group-name $ASG_NAME --protected-from-scale-in --region us-east-2
error is as given below. I think issue is with second line. It is not able to get ASG_NAME, I tried some of escape character but nothing is working.
+++ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id
++ INSTANCE_ID=i-----
+++ aws ec2 describe-tags --filters Name=resource-id,Values=i------ --region us-east-2
+++ jq '.Tags[] | select(.["Key"] | contains("a:autoscaling:groupName")) | .Value'
++ ASG_NAME=
+++ echo
+++ tr -d '"'
++ ASG_NAME=
++ aws autoscaling set-instance-protection --instance-ids i---- --auto-scaling-group-name --protected-from-scale-in --region us-east-2
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --auto-scaling-group-name: expected one argument
> Blockquote
Solved issue by recommendation of #chepner. Modified second line by
ASG_NAME=$(aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" --region us-east-2 --query 'Tags[1].Value')