so I am running ansible against EC2. I need to filter the security groups by their tags. My approach was to get all the security groups and parse them for the required tags, but I am unable to get all the security groups that are present.
Is there a way to either get all the security groups or filter a security group based on a tag and get it's ID?
you could use aws command line interface to fetch security group info and add filters to choose specific security groups. for example, if i wanted to fetch all security groups that were tagged with a tag value of 'Production', this works:
aws ec2 describe-security-groups --filters Name=tag-value,Values=Production
using the command line like this requires that you've configured the command line with your specific AWS credentials (see http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html)
HTH
If you can use Ansible > 2.3 then use ec2_group_facts
You should be able to filter the sections you want either using the filters option in the module or by using a when statement in the Ansible.
Related
I am trying to set up a var template in grafana that would allow me to only show ec2 instances with specific ec2 tags. I did find
ec2_instance_attribute(us-east-1, InstanceId, {"tag:app": ["$application"]})
on a grafana community site and i changed it to
ec2_instance_attribute(us-west-2, InstanceId, {"tag:ENV": ["Prod"]})
The tags based off of my ec2 instance tags is and i keep getting a metric error. I tried removing the brackets and tweaking and still can't figure it out. Does anyone know how to create this using this method or another one i haven't thought of?
Your expression should be working. But I had this error also, due to a missing AWS Policy permission. This might be the case for you as well AWS has a default Policy to read CloudWatch data: "CloudWatchReadOnlyAccess ", but creating my own policy as a copy of "CloudWatchReadOnlyAccess" and adding "ec2:DescribeTags" and "ec2:DescribeInstances" made this work for me.
The above answer might be only applicable when you control your access via Roles with Policies.
I'm trying to setup terraform to handle creation of fine-grained user permissions, and have been able to create:
Cognito User Pools, Identity Pools
IAM Roles, Permissions
What I'm struggling with is how to link them together. I have two types of user:
Standard User
Manager
As such, I have found two ways that I could use to correctly hook up the correct IAM policy upon login:
Method 1 - Create a custom attribute, and Use the "Choose Role With Rules" to set a rule to set an IAM policy based on the attribute
Method 2 - Create Cognito Groups, and link users and the required IAM policy to each group.
The problem, as far as I can see, is that Terraform doesn't currently support either of those cases, so I need to find a work around. So, my question is essentially, how do I get around terraform's lack of support in some areas?
I've seen some projects that use [Ruby, Go, etc.] to make up for some of the limitations, but I don't quite understand where to start and what is the best option for my needs. I haven't been able to find much in Google yet (possibly https://github.com/infrablocks/ruby_terraform). Does anyone have a good guide or resource I could use to get started?
If terraform does not support something you can use the local-exec provisioner to execute commands after resource creation. For example you could use the aws cli to add a custom attribute:
resource "aws_cognito_identity_pool" "main" {
# ...
provisioner "local-exec" {
command = "aws cognito-idp add-custom-attributes --user-pool-id ${aws_cognito_identity_pool.main.id} --custom-attributes <your attributes>"
}
}
local-exec docs
I'd like to get EC2 instances metadata with Ansible and do something with those instances based on the metadata. However, ec2_facts wants to SSH into instances in order to get the metadata.
I believe it should be possible to obtain the instances metadata without SSH connections.
Could you help me with that please?
Thank you.
There is information you can retrieve about instances using the aws API but ec_facts does not use it. What that Ansible module does specifically is fetch metadata via http://169.254.169.254/latest/meta-data/ which can only be done from the instance itself.
Some more information about what instance data you wish to fetch would be helpful to know. At this time there is no aws cloud module in core that will retrieve general information about an instance but Ansible makes it easy to write one.
Here is an example of a module that returns information about instances that match a set of tags - https://github.com/edx/configuration/blob/master/playbooks/library/ec2_lookup
I've created new EC2 spot requests over the last weeks. A new security group was created for every request. When the spot requests were deleted the security groups were not deleted. I've hit the 100 groups limit and want to delete them. The EC2 interface apparently allows only one deletion at a time, which means I would have to make 300 clicks to delete these groups. Or is there a better way to delete multiple security groups with few clicks or lines of code?
THis would need some basic scripting and AWS SDK. you can do this with pretty much all the SDK provided by AWS.
I would prefer AWS-CLI as i already have it installed and configured. This is what I would do:
list all the SGs with describe-security-groups
Install jq (the Json parser for BASH)
Pull the SG IDs (check this for jq syntax)
Once you have the SG IDs, run delete-security-group by usig a for loop.
This is fairly simple and straight forward way of doing wat you want to do. THis can be done by any of the AWS SDKs.
These are just a couple of commands which can be constructed into a Bash script, provided:
You have aws-cli installed and configured
you have jq installed on your system.
If you already have some other AWS SDK installed, then you are better off with that as java/python/ruby...etc all have their own inbuilt way of parsing JSON/HASH/DataStructure.
Hope this helps.
I think you can do this by combining a command that lists all security groups and one other that deletes them.
If you are using the python boto API (for example) that would be:
import boto
conn = boto.connect_ec2(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
groups = conn.get_all_security_groups()
Which returns (as an example): [SecurityGroup:appserver, SecurityGroup:default, SecurityGroup:vnc, SecurityGroup:webserver]
And then you delete them all:
for group in groups:
conn.delete_security_group(group.split(":")[1])
Edit
You should run the commands on your shell.
These solutions only work if you don't have rules in other security groups that reference the security groups in question. I have a couple scripts that will delete a single security group, including the ingress rules in other security groups. I also handle the special case of ingress rules referencing the AWS ELB default security group. If you have this more complex situation, the solutions above won't delete your security group because of these other rules. My scripts are here (one for ec2-classic and one for VPC based security groups): https://gist.github.com/arpcefxl/2acd7d873b95dbebcd42
private static void delete(List<String> sgs) {
AmazonEC2Client ec2 = new AmazonEC2Client(Credentials.getCredentialsProvider());
ec2.setEndpoint("ec2.us-west-2.amazonaws.com"); // default
for(String sg:sgs) {
System.out.println("DELETING SECURITY GROUP " + sg);
DeleteSecurityGroupRequest delReq = new DeleteSecurityGroupRequest().withGroupName(sg);
try {
ec2.deleteSecurityGroup(delReq);
} catch (Exception e) {
// e.printStackTrace();
}
}
}
I've created an autoscaling group on EC2 and it's working just fine. Servers scale up and down depending on load. I'd like to have a little more info on the management side and am wondering if there's a way to get the autoscaling group to dynamically add names to the instances that it boots up. I'm referring to adding a Tag with key=Name and value=autogeneratedid.
For example, if I had an autoscaling group called test-group, servers would boot up with the following names:
test-group-1
test-group-2
test-group-3
...
I'd like to find them an enumerate them in the EC2 Management Console, but right now they're just showing up as "empty" names (the Tag key=Name isn't explicitly set on the instances).
Any ideas?
In order to get the tags to be set on the instances, make sure you are setting the PropagateAtLaunch flag ("p=1") for the tag in the Auto Scaling Group.
You'll want to read this section in Amazon's documentation:
http://docs.amazonwebservices.com/AutoScaling/latest/DeveloperGuide/ASTagging.html
As far as having Amazon dynamically adding parameters to the tag value, I'm not aware of any such feature.