I am using Enteprise Chef. There is just one validation key per organisation. That means that once I download it for my workstation, other devops people in team can't have it => they can't bootstrap nodes from workstation. If I automate process of bootstraping, for instance if I put the key on S3, then I also will have to think about keeping my workstation validation key and S3 key in sync (and all other devops people in team).
So question is:
What are the best practices for distributing this single validation key across nodes and workstations?
My best bet on this:
Use chef on your workstations to manage the distribution of the validation.pem file.
Another way is to set it on a shared place (cifs or nfs share) for all your team.
According to this blog post this will become unneeded with chef 12.2.
For the record, the validation key is only necessary for the node to register itself and create it's own client.pem file at first run, it should (must if you're security aware) be deleted from the node after this first run.
The chef_client cookbook can take care of cleaning the validator key and will help manage nodes configuration.
Related
while setting up a basic 1 x NGINX load-balancer in front of 2 backends, I ended up in what it is clear to me to be a bug: the cron of this Certified App cannot be edited:
As you can see, in this particular App the cron file is owned by root:root and doesn't have the extended attribute (the plus on the right of permissions) necessary for the file to be edited also by the logged in user (nginx in this case).
All other certified apps allow instead the main login user to have crontabs, even though I found the permissions of each file vary a lot.
I've stumbled on https://github.com/jelastic/jem/blob/master/etc/jelastic/export.conf and it seems the file to go for proposing a bugfix, but it's last update if Aug-2016, so I guess Jelastic had closed much of its source code.
How can we contribute to Certified App source code?
indeed it is a bug as cron file of nginx user isn't editable in a balancer template, by design in has to be.
As for exports.conf - this file left for backwards comparability, but no more used.
The problem will be definitely fixed in latest templates, as for existing containers - we would like to apply a patch to fix them, if you provide us more details about hosting service provider you are using - we will help with that.
As for contribution to certified templates, all the images are publicly available on Docker Hub, you can create your own version of template based on existing one if you build a docker image and in your Dockerfile you specify
"from jelastic/nginxbalancer" as a base, then you can do any modifications to the filesystem. Next step will be just to replace existing balancer with your custom one.
Anyway, let's start with fix of existing containers.
Many thanks for finding out the bug!
I would like have a setup where my ec2 instances are getting terminated sometimes and new nodes comes up with the same host name .My puppetserver supposed to have the old certificates with them and instantly push the configs via the required modules.
Is this a viable solution?In this case do I need to keep the ssl certs of clients and push them to the ec2instnces via user-data ? what would be the best alternatives?
I can't really think of too many arguments against certificate re-use, only that puppet cert clean "$certname" on the CA master is so simple that I can't really think of a reason to re-use certificate names.
That said, I'd recommend building some kind of pipeline that includes certificate cleaning upon destruction of your ec2 instances. This is fairly easy to do in a terraform workflow with the AWS Terraform Provider and local-exec provisioner. For each aws_instance you create, you'd also create a local-exec resource, and specify the local-execonly executes at destruction time: when = "destroy".
If you're re-using hostnames for convenience, maybe it would be wise to instead rely on dns to point to the new hosts instead of relying on the hostnames themselves and stop worrying about puppet cert clean.
I am trying to do the MSI web deployment with chef. I have about 400 web servers with same configuration. We will do deployment in two slots with 200 servers each.
I will follow below steps for new release,
1) Increase the cookbook version.
2) Upload the cookbook to server.
3) Update the cookbook version to role and run list.
I will do lot of steps from cookbook like install 7 msi, update IIS settings, update web.configure file and add registry entry. Once deployment is done we need to update testing team, so that they can start the testing. My question is how could I ensure deployment is done in all the machines successfully? How could I find if one MSI is not installed in one machine or one web.config file is not updated properly?
My understanding is chef client will run every 30 Mins default, so I have wait for next 30 mins to complete the deployment. Is there any other way with push (I can’t use push job, since chef is removed push job support from chef High Availability servers) like knife chef client from workstation?
It would be fine, If anyone share their experience who is using chef in large scale windows deployment.
Thanks in advance.
I personnaly use rundeck to trigger on demand chef runs.
According to your description, I would use 2 prod env, one for each group where you'll bump the cookbook version limitation for each group separately.
For the reporting, at this scale consider buying a license to get chef-manage and chef-reporting so you'll have a complete overview, next option is to use a handler to report the run status and send a mail if there was an error during the run.
Nothing in here is specific to Windows, so more you are asking how to use Chef in a high-churn environment. I would highly recommend checking out the new Policyfile workflow, we've had a lot of success with it though it has some sharp limitations. I've got a guide up at https://yolover.poise.io/. Another solution on the cookbook/data release side is to move a lot of your tunables (eg. versions of things to deploy) out of the cookbook and in to a little web service somewhere, than have your recipe code read from that to get their tuning data. As for the push vs. pull question, most people end up with a hybrid. As #Tensibai mentioned, RunDeck is a popular push-based option. Usually you still leave background interval runs on a longer cycle time (maybe 1 or 2 hours) to catch config drift and use the push system for more specific deploy tasks. Beyond RunDeck you can also check out Fabric, Capistrano, MCollective, and SaltStack (you can use its remote execution layer without the CM stuffs). Chef also has its own Push Jobs project but I think I can safely say you should avoid it at this point, it never got enough community momentum to really go anywhere.
I'm trying to run the S3->DynamoDB example and having some problems running the EMR cluster that is created for the MyImportJob activity.
We configured our IAM accounts such that every user can create EC2 machines with a specific 'team_id' tag (of his team). That helps us control the resources, prevent mistakes and monitor usage.
When Pipeline tries to launch the EMR cluster, it (probably) does it without the tags and therefore it fails with Terminated with errors: User account is not authorized to call EC2. I tried to find a configuration in the EMRCluster resource but couldn't find anything that will help me set that. I'm pretty sure that it fails because of the tags policy.
Any idea how I can overcome this?
Does it help if If create a CloudFormation template for that? Do I have more control there? (I'm going to create the pipeline as a part of the application template anyway, just wanted to experience the product before).
Thanks!
I could not find a solution for how to add tags to EMR(and how to set it to be visible to all users) so I have created a python script to run as bootstrap action. If its still relevant you can find it here
Our company provides some management on behalf of smaller businesses using Amazon EC2. Almost all our tasks are manual operations work, and therefore we are somewhat ignorant of the intricacies of EC2. However, I now want to build an ownership database, to show what attributes we manage for each customer. To build this database I will extract usage parameters from EC2. So I was wondering about the following mapping attributes: snapshot id, machine id and instance id... Are these 3 attributes safe to be considered GLOBALLY UNIQUE? What about DATACENTER UNIQUE? Or do I need to add our client's account as a prefix in my database to ensure that mapping of attributes remains unique within our system?
Snapshot Id is regional, Instance Id is tied to an availability zone. There is no such thing as a machine id.