I want to use the value of the DOMAIN_ID variable to filter the EFS to get a FileSystemId. I used the commands below. The first command works and it stores the domain ID. The second one returns an empty list, even though the DOMAIN_ID variable is present.
DOMAIN_ID=$(aws sagemaker list-domains --query 'Domains[0].DomainId')
aws efs describe-file-systems --query 'FileSystems[?CreationToken==`$DOMAIN_ID`].FileSystemId'
Output:
[]
Expected output:
<Some EFS identifier>
This works (escaping backticks) -
aws efs describe-file-systems --query "FileSystems[?CreationToken==\`$DOMAIN_ID\`].FileSystemId"
You can also use describe-domain command instead -
$ DOMAIN_ID=$(aws sagemaker list-domains --query 'Domains[0].DomainId' | tr -d '"')
$ aws sagemaker describe-domain --domain-id $DOMAIN_ID --query 'HomeEfsFileSystemId'
I'm trying to create a script to add my IP adress to AWS VPC security groups somthing like
> aws ec2 modify-security-group-rules --group-id GROUPID\
> --security-group-rules SecurityGroupRuleId= RULEID\
SecurityGroupRule={IpProtocol:'tcp',FromPort:433,ToPort:433,CidrIpv4:'MYIP'}
But I keep getting different errors like -
IpProtocol:tcp, type: <class 'str'>, valid types: <class 'dict'>
Can anyone please help figure out the correct syntax for this?
UPDATE:
I tried a new syntax that seems to work better
SecurityGroupRule={{IpProtocol=tcp},{FromPort=433},{ToPort=433},{CidrIpv4='IP'}}
But now I get a different error from AWS -
Invalid value for portRange. Must specify both from and to ports with TCP/UDP.
UPDATE: For reference - Here's the workaround I used- (based on John Rotenstein answer)
Instead of modifying the rule I create a new one each time and save the rule ID so I can delete it next time I run the script
IP=`curl -s http://whatismyip.akamai.com/`
aws ec2 revoke-security-group-ingress \
--group-id GROUP_ID \
--security-group-rule-ids $(cat ruleid_1.txt)
aws ec2 authorize-security-group-ingress --group-id GROUP_ID\
--ip-permissions "IpProtocol"="tcp","FromPort"=433,"ToPort"=443,"IpRanges"="[{CidrIp=$IP/32,Description=Shalev}]"|jq '.SecurityGroupRules[0].SecurityGroupRuleId' -r > ruleid_1.txt
Place value of parameter --security-group-rules inside quotes.
Both of the following seem to work for me (on Amazon Linux 2) -
Using double quotes for complete value, with description in single quotes-
aws ec2 modify-security-group-rules --group-id sg-xxx
--security-group-rules "SecurityGroupRuleId=sgr-xxx,SecurityGroupRule={Description='SSH
Test1',CidrIpv4=x.x.x.x/32,IpProtocol=tcp,FromPort=22,ToPort=22}"
Using single quotes for complete value, with description in double quotes-
aws ec2 modify-security-group-rules --group-id sg-xxx
--security-group-rules 'SecurityGroupRuleId=sgr-xxx,SecurityGroupRule={Description="SSH
Test2",CidrIpv4=x.x.x.x/32,IpProtocol=tcp,FromPort=22,ToPort=22}'
For reference - Here's the workaround I used- (based on John Rotenstein answer) Instead of modifying the rule I create a new one each time and save the rule ID so I can delete it next time I run the script
IP=`curl -s http://whatismyip.akamai.com/`
aws ec2 revoke-security-group-ingress \
--group-id GROUP_ID \
--security-group-rule-ids $(cat ruleid_1.txt)
aws ec2 authorize-security-group-ingress --group-id GROUP_ID\
--ip-permissions "IpProtocol"="tcp","FromPort"=433,"ToPort"=443,"IpRanges"="[{CidrIp=$IP/32,Description=Shalev}]"|jq '.SecurityGroupRules[0].SecurityGroupRuleId' -r > ruleid_1.txt
Here is a way to use the aws CLI to change a rule.
Requires "ec2:ModifySecurityGroupRules" permission.
aws ec2 describe-security-group-rules help
aws ec2 modify-security-group-rules --group-id sg--???????
--security-group-rules SecurityGroupRuleId=sgr---???????,SecurityGroupRule={IpProtocol=tcp,FromPort=22,ToPort=22,CidrIpv4=IP/32,Description="Regra
Alterada"}
Here's a script I use to add my current IP address to a Security Group:
IP=`curl -s http://whatismyip.akamai.com/`
aws ec2 authorize-security-group-ingress --group-name XXX --protocol tcp --port 22 --cidr $IP/32 --output text
It uses Akamai to retrieve my public IP address and then adds it to the desired Security Group.
Note that there is a limit to the number of rules in a Security Group, so eventually you will need to remove unused entries.
Use a variable for SecurityGroupRule.
Like:
IP=`curl -s http://whatismyip.akamai.com/`
security_group_rules={CidrIpv4=$IP/32,IpProtocol=tcp,FromPort=443,ToPort=443}
aws ec2 modify-security-group-rules \
--group-id sg-123 \
--security-group-rules SecurityGroupRuleId=sgr-123,SecurityGroupRule=$security_group_rules
It worked for me perfectly.
PS: This is my first answer here.
I seem to keep getting an error whenever I try to use bash to automate getting the status of a job.
My current bash script currently looks like this:
#!/bin/bash
aws ec2 start-instances --instance-ids=$1;
start=$(aws ec2 describe-instance-status --instance-id $1)
status=$(echo $start | jq '.InstanceStatuses[0].InstanceState.Name')
#wait for ec2 instance to start running before launching command
while [ "$status" != "\"running\"" ]
do
start=$(aws ec2 describe-instance-status --instance-id $1)
status=$(echo $star | jq '.InstanceStatueses[0].InstanceState.Name')
done
sh_command_id=$(aws ssm send-command --instance-ids=$1 --document-name "AWS-RunShellScript" --parameters 'commands=["echo Helloworld","sleep 60"]');
command_id=$(echo $sh_command_id | jq '.Command.CommandId')
full_status=$(aws ssm list-commands --command-id $command_id)
echo $command_id;
aws ec2 stop-instances --instance-ids=$1;
When the script gets to aws ssm list-commands --command-id $command_id I get this error.
An error occurred (ValidationException) when calling the ListCommands operation: 2
validation errors detected: Value '"67fb9aed-00bf-4741-ae1a-736ddbfba498"' at 'commandId'
failed to satisfy constraint: Member must satisfy regular expression pattern: ^[A-Fa-
f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}$.; Value
'"67fb9aed-00bf-4741-ae1a-736ddbfba498"' at 'commandId' failed to satisfy constraint:
Member must have length less than or equal to 36.
When running everything individually in terminal I get the same error. However, I do not get an error when I mannually type in the commandId as so: full_status=$(aws ssm list-commands --command-id 67fb9aed-00bf-4741-ae1a-736ddbfba498).
Is there some aws formatting I am missing here?
You might be able to avoid the use of jq by using the aws cli built in --query 'your.json.query' to specify your JSON query and then the --output text to return plain text. It has been a while since I checked so your mileage may vary.
I was able to verify that the following works for checking an ec2 is running:
check_instance() {
local instance_id="${1}"
local status="_"
while [ "${status}" != "running" ] ; do
status=$(aws ec2 describe-instance-status \
--instance-ids ${instance_id} \
--query "InstanceStatuses[*].InstanceState.Name" \
--output text)
done
echo "${instance_id} is running"
}
I am trying to do a filter using aws query to output 0.0.0.0/0 Ips and find which ports they are coming from however when I do
aws ec2 describe-security-groups --filters Name=ip-permission.cidr,Values='0.0.0.0/0' --query "SecurityGroups[*].{GN:GroupName,Ports: IpPermissions.FromPorts}" --output json
In the Ports column, it outputs None, since it is not finding that filter and when I do
aws ec2 describe-security-groups --filters Name=ip-permission.cidr,Values='0.0.0.0/0' --query "SecurityGroups[*].{GN:GroupName,Ports: IpPermissions}" --output json
I get all the Ip Ranges when I only want 0.0.0.0/0 and which Port that Open Ip is coming from (FromPort and ToPort). And I don't want to do a filter just for a certain Port because I want to see what port each open Ip is coming from.
I am not sure why this is not working since the IpPermissions.ToPort and IpPermissions.FromPort are both a filter of aws security groups.
You need two modification in the query
The key name is FromPort not FromPort
IpPermissions return array not object, so use IpPermissions[].FromPort
aws ec2 describe-security-groups --filters Name=ip-permission.cidr,Values='0.0.0.0/0' \
--query "SecurityGroups[*].{GN:GroupName,Ports: IpPermissions[].FromPort}" --output json
I'm writing a bash script that needs to get the private ips of all instances where describe-instances does not return platform=>windows, and store them in an array $ips
This script works so far (just getting the private ips)
ips=$(aws ec2 describe-instances --region $r --filters "Name=tag:Name,Values=*$c*" | jq '.Reservations[].Instances[].NetworkInterfaces[].PrivateIpAddress')
but what I can't figure out is how to exclude windows servers (or, conversely, only include non-windows servers)
Here's what I've tried
--filters "Name=tag:Name,Values=*$c*" "Name=platform,Values="
--filters "Name=tag:Name,Values=*$c*" "Name=platform,Values=null"
--filters "Name=tag:Name,Values=*$c*" "Name=platform,Values=NULL"
I've also tried doing it on the jq side of the pipe, but my attempts were ridiculous and I will not publish them here. I did find out you cannot do more than two filters on that side (unless I messed that up too)
Any ideas?
I don't use AWS so I don't have much to go on but based off of the describe-instances reference, I think this filter should work.
.Reservations[].Instances |
map(select(.Platform != "Windows") | .NetworkInterfaces[].PrivateIpAddress)