I have an environment setup with Test-Kitchen v1.5.0, Vagrant v1.8.1. I have a recipe that uses chef vault to decrypt our encrypted passwords that our in our data_bags_path/passwords/pilot.json file.
I am using the solution here https://github.com/chef/chef-vault/issues/58 that daxgames provides towards the end of the page.
My .kitchen.yml:
---
driver:
name: vagrant
provisioner:
name: chef_zero
require_chef_omnibus: 12.14.77
roles_path: ../../roles
environments_path: ../../environments
data_bags_path: ../../data_bags
client_rb:
environment: lgrid2-dev
node_name: "ltylapp400a"
client_key: "/etc/chef/ltylapp400a.pem"
platforms:
- name: centos-6.8
driver:
synced_folders:
- ["/Users/212466756/.chef", "/etc/chef", "disabled:false"]
suites:
- name: ltylapp400a
run_list:
- role[lgrid-db]
attributes:
chef_client:
A snippet from my recipe that deals with chef-vault:
case node["customer_conf"]["status"]
when 'pilot'
passwords = ChefVault::Item.load('passwords', 'pilot')
when 'production'
passwords = ChefVault::Item.load('passwords', node[:hostname][1..3])
end
My directory structure for relevant data_bags:
data_bags
--passwords
--pilot.json
--pilot_keys.json
The error I am getting is that my client.pem that vagrant generates at /etc/chef/ltylapp400a.pem can not decrypt the contents of that databag. Chef suggest that I run knife vault refresh, I am not connected to my chef server on my local machine so if I run this it will give an error about not having a chef server to connect to. My question is how I can add my new key that vagrant generated to the pilot_keys.json so that it is able to decrypt that data_bag?
The more detailed answers are better I am still somewhat new to chef, test-kitchen, etc...
I was able to get this working, below are my results and conclusions. As I stated above my issue was I was unable to decrypt the data_bag since I could not add the new key that vagrant created to the pilot_key.json file since I was not connected to the chef server and could not run a knife vault refresh/update. What I had to do was get the client.pem key from a server that already had access to the pilot.json data_bag. I used our utility server key since it will not be destroyed in the near future.
So on my local PC I have a .chef/ directory under my home directory, I have the client.pem key I copied from the utility server and I sync this with the /tmp/kitchen/ which acts as the /etc/chef directory in the test-kitchen environment.
---
driver:
name: vagrant
provisioner:
name: chef_zero
require_chef_omnibus: 12.14.77
roles_path: ../../roles
environments_path: ../../environments
data_bags_path: ../../data_bags
client_rb:
node_name: "utilityServer"
client_key: "/tmp/kitchen/client.pem" #The Chef::Vault needs a client.pem file to authenticate back to the data_bag to decrypt it, this needs to be stored at /tmp/kitchen/client.pem
environment: dev
no_proxy: 10.0.2.2
platforms:
- name: centos-6.8
driver:
synced_folders:
- ["~/.chef","/tmp/kitchen/","disabled:false"] # Allows the vagrant box to have access to your .chef directory in your home directory. This is where you will store the client.pem for authentication.
suites:
- name: lzzzdbx400a
run_list:
- role[lgrid-db]
attributes:
The data_bags/passwords/pilot_key.json looks like this:
{
"id": "pilot_keys",
"admins": [
"utilityServer"
],
"clients": [
"webserver",
"database"
],
"search_query":"*:*"
"utilityServer":"key",
"webserver":"key",
"database": "key"
}
Since the utilityServer key was already able to decrypt the passwords/pilot data_bag it ran through fine during the next time I ran kitchen converge.
During previously struggles with Kitchen and chef-vault I used the synced_folders method to access key. Revisited this topic I found another solution.
Kitchen Support
To make this work in kitchen, just put a cleartext
data bag in the data_bags folder that your kitchen run refers to
(probably in test/integration/data_bags). Then the vault commands fall
back into using that dummy data when you use chef_vault_item to
retrieve it.
reference: http://hedge-ops.com/chef-vault-tutorial/
Related
We in the middle to developed molecule testing using Vagrant as driver and libvirt as provider.
However, the VM that Vagrant create from molecule use user 'vagrant' to perform validation or installation inside the VM. We plan to use 'root' user instead of default user as 'vagrant'
We try to include below option in molecule.yml,
---
dependency:
name: galaxy
driver:
name: vagrant
provider:
name: libvirt
type: libvirt
options:
memory: 192
cpus: 1
ssh_user: root
ssh_password: 'xxxxxxx'
platforms:
- name: test-vm
box: centos7
interfaces:
- auto_config: true
network_name: private_network
type: static
ip: 192.168.122.100
config_options:
ssh.remote_user: "'root'"
We try to search for details options that molecule.yml can use inside from molecule site but not able to found it.
Thus, the issue currently is:
Molecule use 'vagrant' user instead of root to perform any playbook tasks.
When login manually into the Vagrant VM using 'vagrant ssh' , it directly login as user 'vagrant' instead of 'root'
Need assistance from expertise for this kind of issue.
I am trying to learn how to do local development of chef recipes. I am following this guide https://gist.github.com/smford22/f00f46471047422bd8a7
I am prefixing all the kitchen commands with chef exec because if I try to run kitchen directly, I get all sorts of ruby/gem errors.
When I run chef exec kitchen converge it gets stuck on installing the Chef Omnibus, hanging on "Trying wget..."
If I login to the VM and try to run curl and wget commands like curl https://google.com it indeed cannot access the internet.
chef exec kitchen -v
Test Kitchen version 1.23.2
chef -v
Chef Development Kit Version: 3.5.13
chef-client version: 14.7.17
delivery version: master (6862f27aba89109a9630f0b6c6798efec56b4efe)
berks version: 7.0.6
kitchen version: 1.23.2
inspec version: 3.0.52
.kitchen.yml:
---
driver:
name: vagrant
## The private_network feature lets you setup a private network on the VM guest
## via localhost on the host.
## see also: https://www.vagrantup.com/docs/networking/private_network.html
# network:
# - ["private_network", {ip: "33.33.33.33"}]
provisioner:
name: chef_zero
## The verifier section determines which test platform you want to use.
verifier:
name: inspec
format: doc
platforms:
- name: centos-6.7
suites:
- name: default
run_list:
- recipe[chef_httpd::default]
attributes:
Are you accessing the internet through proxy?
If yes, you need to configure the same for your Vagrant VMs using vagrant-proxyconf plugin.
Documentation: http://tmatilai.github.io/vagrant-proxyconf/
I am running vagrant version,
vagrant -v
Vagrant 1.9.3
vagrant plugin list
vagrant-butcher (2.2.1)
vagrant-cachier (1.2.1)
vagrant-omnibus (1.5.0)
vagrant-share (1.1.7, system)
vagrant-vbguest (0.13.0)
When I start a vagrant VM, windows 2012r2,
I get "Renaming the Windows guest failed. Most often this is because you've specified a FQDN instead of just a host name."
It used to work before on the same host(centos7, with Virtualbox) with version 1.4.
If you are (like me) experiencing this with Kitchen, in your .kitchen.yml, in the platform section, you can't have "name: mwrock/Windows2012R2". Instead, name it something like "windows2012R2" and, in platform's "driver_config" section, specify "box: mwrock/Windows2012R2".
Another way you can resolve this issue is by setting the vm_hostname attribute to false like this:
platforms:
- name: BPA-TEST
driver_config:
username: Tester
password: [PASSWORD]
vm_hostname: false
driver:
port: 55985
customize:
memory: 4048
https://github.com/test-kitchen/kitchen-vagrant
vm_hostname Sets the internal hostname for the instance. This is not used when connecting to the Vagrant virtual machine.
To prevent this value from being rendered in the default Vagrantfile,
you can set this value to false.
The default will be computed from the name of the instance. For
example, the instance was called "default-fuzz-9" will produce a
default vm_hostname value of "default-fuzz-9". For Windows-based
platforms, a default of nil is used to save on boot time and potential
rebooting.
I have created my kitchen.yml in the following way:
---
driver:
name: vagrant
customize:
memory: 2048
driver_config:
require_chef_omnibus: true
use_vagrant_berkshelf_plugin: true
provisioner:
name: chef_zero
chef_omnibus_url: http://box-url/install.sh
platforms:
- name: prod.abc.com
driver:
box_url: http://abc.box
run_list:
- role[new_role]
suites:
- name: default
In the above kitchen.yml, I get the hostname of the machine as default-prodabccom. However, I want the hostname to be prod.abc.com
What changes should i made in my kitchen.yml to get the correct name?
Hostname of the Guest System
In order to define the hostname of the operating system running inside the VM (cf. /etc/hostname), use the vm_hostname option of the kitchen-vagrant driver:
platforms:
- name: prod.abc.com
driver_config:
vm_hostname: prod.abc.com
Name of the Test-Kitchen Suite/Platform
To rename the suite-platform combination shown in Converging <default-prodabccom>, you can only play with name of suite and platform, i.e., to get production-abccom. This name is computed here in test-kitchen and, e.g., all dots are stripped, which cannot simply be changed.
Nevertheless, if I understand it right that you want to change this name: it makes little sense to me. Don't change that.
Name of the VM in VirtualBox
The name of the VM (e.g. kitchen-default-prodabcom_..default_1234..) is derived here in kitchen-vagrant and cannot easily be changed.
I found this question because I was in the scenario where I am testing a number of kitchen enabled repos, each containing a number of platforms. e.g.
elasticsearch
centos-6
centos-7
java
centos-6
centos-7
and you can give those machines their own ip by virtualbox when they are spun up like so;
driver:
name: vagrant
network:
- ["private_network", { type: "dhcp" }]
This facilitates testing, if something has failed and then you can get to the box directly. And you can use the vagrant HostManager plugin to keep your /etc/hosts updated with the current ip address.
So you can go http://default-centos-74.vagrantup.com in a local browser to check that instance. You can also name your suites in such a way that it leads to unique names for each each, across repos, for example prefixing each like so;
suites:
- name: elasticsearch-default
and in other other .kitchen.yml
suites:
- name: java-default
which still leads to useful naming;
http://elasticsearch-default-centos-74.vagrantup.com
However whats happended recently is that chrome and firefox have started to enforce HSTS which makes trying to get to non-HTTPS local sites, mapped using /etc/hosts a PITA.
The main thing is to get rid of the vagrantup.com suffix. However that is hard coded in, and the only option for over writing it is in .kitchen.yml which is unfortunate, because that doesn't know the suite and platform at the point it generates the Vagrantfile, so it's not much use.
You can use chef/ansible to rename the box, but that is not very nice. The solution I came up with is like this;
you can set a custom Vagrantfile.erb in .kitchen.yml ;
---
driver:
name: vagrant
network:
- ["private_network", { type: "dhcp" }]
vagrantfile_erb: Vagrantfile.erb
Then copy that Vagrantfile.erb out of the gem on your local box into the root of your test-kitchen repo. Mine was at /home/user1/.gem/ruby/gems/kitchen-vagrant-1.3.0/templates/Vagrantfile.erb
And then you set arbitrary names to your boxes by changing it at line 36;
c.vm.hostname = "<%= #instance.name %>.<%=
config[:kitchen_root].split('/')[-1] %>.testbox"
or you can modify it like so, and allow over riding from the .kitchen.yml config
36c36
< c.vm.hostname = "<%= config[:vm_hostname] %>"
---
> c.vm.hostname = "<%= #instance.name %>.<%= config[:var_domain] ? config[:var_domain] : config[:kitchen_root].split('/')[-1] %>.<%= config[:var_suffix] ? config[:var_suffix] : "vagrantup.com" %>"
99d98
<
https://gist.github.com/tolland/fe01eb0f46d26850cc5c98e167578f7b
And then you set arbitrary names to your boxes by setting var_suffix and var_domain in .kitchen.yml
---
driver:
name: vagrant
network:
- ["private_network", { type: "dhcp" }]
vagrantfile_erb: Vagrantfile.erb
#var_domain: sometingsomething
var_suffix: testbox
When running commands with ansible on a vagrant box, it is using the identity file located here:
IdentityFile="/Users/me/.vagrant.d/boxes/ubuntu-VAGRANTSLASH-trusty32/0/virtualbox/vagrant_private_key"
Instead of the file on the box: ~/.ssh/id_rsa
What can I do to fix this? This is my task by the way:
---
- name: Fetch the Htt Database
run_once: true
delegate_to: 543.933.824.210
remote_user: ubuntu
become_user: root
fetch: src=/home/ubuntu/file.sql.gz dest=/tmp/file.sql.bz fail_on_missing=yes
By-default vagrant use insecure_private_key to login vagrant user and that is not secure because every one know that key so if you want to use your ssh key then you can modify your Vagrantfile by adding lines
config.ssh.username = "username"
config.ssh.private_key_path = "fullpath-of-ssh-private-key"
config.ssh.insert_key = false
According to vagrant Documentation
config.ssh.insert_key - If true, Vagrant will automatically insert an
keypair to use for SSH, replacing the default Vagrant's insecure key
inside the machine if detected. By default, this is true.
This only has an effect if you don't already use private keys for
authentication or if you are relying on the default insecure key. If
you don't have to take care about security in your project and want to
keep using the default insecure key, set this to false.
also make sure you have public-key of your private-key in guest vm at /home/username/.ssh
For info you can use vagrant Documentation