I am using below command to list the UserPool Names.
aws cognito-idp list-user-pools --max-results 60 --region us-west-2 --query 'UserPools[*].{Names:Name}'
Now my all UserPool Names contains cust_ as prefix. And i want to remove that from whole list.
I know i can achieve this using jq.
But how?
Any help will be highly appreciated.
Thanks!
If you output with --output text, it will become a text list.
You could then use standard Linux tools such as piping it through | cut -c6-
This will provide character #6 onwards for each line.
Full command would be something like:
aws cognito-idp list-user-pools --max-results 60 --region us-west-2 --query 'UserPools[*].[{Names:Name}]' --output text | cut -c6-
Related
I am using this code to get my AWS region. I want working on them in a while loop.
awsRegionList=$(aws ec2 describe-regions | jq -r '.Regions[] | .RegionName')
while [I can't find the expression work with my variable]:
do
echo " working on : (I want here the regionName)"
done
In bash you need to use a for loop to iterate over a list, instead of a while loop:
awsRegionList=$(aws ec2 describe-regions | jq -r '.Regions[] | .RegionName')
for region in $awsRegionList
do
echo " working on : ${region}"
done
How do I pass the value of the variable to a command in a bash script?
Specifically, I want to create a AWS S3 bucket with (partially) random name. This is what I got so far:
#!/bin/bash
random_id=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)
bucket_name=s3://mybucket-$random_id
echo Bucket name: ${bucket_name}
aws s3 mb ${bucket_name}
The output I get:
Bucket name: s3://mybucket-z4nnli2k
Parameter validation failed:cket-z4nnli2k
": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
The bucket name is generated correctly, but aws s3 mb ${bucket_name} fails. If I just run aws s3 mb s3://mybucket-z4nnli2k then the bucket is created, so I assume that aws s3 mb ${bucket_name} is not the correct way to pass the value of the bucket_name to the aws s3 mb command.
It must be something obvious but I have almost zero experience with shell scripts and can't figure it out.
How do I pass the value of bucket_name to the aws s3 mb command?
Thanks to the comments above, this is what I got working:
#!/bin/bash
random_id=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)
bucket_name=s3://mybucket-$random_id
echo Bucket name: ${bucket_name}
aws s3 mb "${bucket_name}"
I also had to run dos2unix on the script, apparently there were bad line breaks.
I'm trying to write a scalable and reusable script to provision ec2s using ansible. As part of this, I would like to be able to determine which Route53 hosted zone my machine is a part of, so I can add it as a record set for a private zone. I don't want to have to enter the zone ... I want to be able to figure it out using the ec2.
For a given ec2, I can get the instance. From the instance, I get get VPC-ID. I know that VPC-IDs are associated with Route53 hosted zones, but I can't seem to find an AWS CLI command to figure out the hosted zone from the VPC-ID.
I've found the command'route53 list-vpc-association-authorizations --hosted-zone-id=' command, which has to be run on each individual zone, but the result is an empty array for a zone that I know for a fact is associated with a VPC.
Can anyone help me to derive the correct private hosted zone, given that I know the VPC ID and ec2 instance id?
Thanks
Maybe too simple for people, but this works:
aws route53 list-hosted-zones --output text | grep 'MYDOMAIN' | awk '{print $3}' | cut -c13-
...Just lists the domains in AWS in column format, searches for your domain and then cuts out the zone id with awk and cut.
Took me a while, but I figured it out:
getHostedZone(){
ZONE_IDS=$(aws route53 --region $2 list-hosted-zones | jq ".HostedZones | map(.Id)")
while IFS= read -r; do
ZONE=$(aws route53 --region $2 get-hosted-zone --id $REPLY)
hasVPCs=$(echo $ZONE | jq 'has("VPCs")')
VPCs=$(echo $ZONE | jq ".VPCs")
if [ "$hasVPCs" == true ]
then
VPC=$(echo $VPCs | jq ".[] | select(.VPCId == \"$1\")")
if [ -n "$VPC" ]
then
HOSTED_ZONE=$(echo $REPLY | sed 's/^\/hostedzone\///g')
fi
fi
done < <(echo $ZONE_IDS | jq -r '.[]')
echo $HOSTED_ZONE
}
Called with:
ZONE_ID=$(getHostedZone $VPC_ID $EC2_REGION)
I've got a group of AWS instances that I'm parsing via aws ec describe-instances. I'm looking to trim out all the records whose IP's do not start with '10.10'.
aws ec2 describe-instances --no-paginate --filter "Name=instance-state-name,Values=running" --query 'Reservations[].Instances[].{Private:PrivateIpAddress,PublicDNS:PublicDnsName,PublicIP:PublicIpAddress}' | jq '.[] | select( .Private | contains("10.10"))'
This gets me the exact opposite of what I want. It seems logical that I should be able to negate the contains in some way - but I've not been able to glean it from the documentation, nor through experimentation. My jq proficiency is middling, so perhaps I'm using the wrong operator or function here.
While i WOULD like an answer to this specific jq question - I'll accept an answer that utilizes JMESPath through the --query switch yield the same result.
Jeff Marcado's answer in the comments will be accepted if he writes it up as a full fledged answer. In the meantime, since I had hit a wall with trying to get JQ to do it, I experimented with the --query syntax for AWS to get this.
It might be a bit better, since this catches only objects that start with 10.10, whereas the jq from above will catch any object that contains 10.10, so things like 10.100. or 110.100, etc... will get through. That's assuming there is not a similar operator to "starts_with" in jq. Probably there is. Regardless, I'm putting this here because it worked for my end goal and may be helpful to someone else at some point.
aws ec2 describe-instances \
--no-paginate --filter "Name=instance-state-name,Values=running" \
--query 'Reservations[].Instances[?starts_with(PrivateIpAddress, `10.10.`) == `false`]' |
jq '.[] | .[] | {PrivateIpAddress, PublicIpAddress, PublicDnsName}'
I can't seem to get jq to behave "normally" in a shell pipeline. For example:
$ curl -s https://api.github.com/users/octocat/repos | jq | cat
results in jq simply printing out its help text*. The same thing happens if I try to redirect jq's output to a file:
$ curl -s https://api.github.com/users/octocat/repos | jq > /tmp/stuff.json
Is jq deliberately bailing out if it determines that it's not being run from a tty? How can I prevent this behavior so that I can use jq in a pipeline?
Edit: it looks like this is no longer an issue in recent versions of jq. I have jq-1.6 now and the examples above work as expected.
* (I realize this example contains a useless use of cat; it's for illustration purposes only)
You need to supply a filter as an argument. To pass the JSON through unmodified other than the pretty printing jq provides by default, use the identity filter .:
curl -s https://api.github.com/users/octocat/repos | jq '.' | cat
One use case I have found myself doing frequently as well is "How do I construct JSON data to supply into other shell commands, for example curl?" The way I do this is by using the --null-input/-n option:
Don’t read any input at all! Instead, the filter is run once using null as the input. This is useful when using jq as a simple calculator or to construct JSON data from scratch.
And an example passing it into curl:
jq -n '{key: "value"}' | curl -d #- \
--url 'https://some.url.com' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json'