I use vagrant to test my salt configs. On one box I have a vagrant salt master and salt minion and on another box I have a salt minion.
I'm trying to switch over to use gitfs to fetch backend from my private repo.
This is my salt.master_config:
hash_type: sha256
auto_accept: True
roster_file: /srv/salt/roster
fileserver_backend:
- roots
- git
file_roots:
base:
- /srv/salt/environments/base/files
- /srv/salt/environments/base/states
- /srv/salt/users
gitfs_remotes:
- git#github.com:user/repo.git
- pubkey: /srv/salt/environments/base/files/.ssh/id_rsa.pub
- privkey: /srv/salt/environments/base/files/.ssh/id_rsa
- https://github.com/salt/users-formula.git
- https://github.com/salt/openssh-formula.git
The other gitfs remotes have worked in the past but the new one does not accept the ssh keys.
When running sudo salt '*' state.apply I get:
ERROR ] Error parsing configuration file: /etc/salt/master - mapping values are not allowed here
in "<string>", line 16, column 13:
- pubkey: /srv/salt/environments/base/fi
I have also tried using the Master Options from the vagrantup prosvisioning doc https://www.vagrantup.com/docs/provisioning/salt.html and added master_pub and master_key to my Vagrantfile:
master.vm.provision :salt do |salt|
salt.install_master = true
salt.master_pub = 'id_rsa.pub'
salt.master_key = 'id_rsa'
salt.install_type = 'stable'
salt.master_config = 'master'
salt.minion_config = 'salt-local'
end
But this is to ssh to the vagrant box and not actually used for gitfs.
How do you give ssh credentials for the vagrant config files?
Found the solution:
gitfs_provider: pygit2
gitfs_pubkey: /srv/salt/path/to/files/ssh/id_rsa.pub
gitfs_privkey: /srv/salt/path/to/files/ssh/id_rsa
gitfs_remotes:
- git#github.com:user/repo.git
Related
I'm writing a wrapper cookbook for nexus3 wherein I override the default attributes like so in the attributes/default.rb file of my cookbook
# Nexus Options
node.default['nexus3']['properties_variables'] = { port: '8383', host: '0.0.0.0', args: '${jetty.etc}/jetty.xml,${jetty.etc}/jetty-http.xml,${jetty.etc}/jetty-requestlog.xml', context_path: '/nexus/' }
node.default['nexus3']['api']['host'] = 'http://localhost:8383'
node.default['nexus3']['api']['username'] = 'admin'
node.default['nexus3']['api']['password'] = 'Ch5f#A4min'
While Chef does install nexus3 with the override properties, property values for the nexus3_api fail to take effect during cookbook run, as I see in the logs
==> provisioner: * execute[wait for http://localhost:8081/service/siesta/rest/v1/script to respond] action run
==> provisioner: [2018-06-11T05:58:17+00:00] INFO: Processing execute[wait for http://localhost:8081/service/siesta/rest/v1/script to respond] action run (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/resource.rb line 1285)
==> provisioner: [2018-06-11T05:58:17+00:00] INFO: Processing execute[wait for http://localhost:8081/service/siesta/rest/v1/script to respond] action run (/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.2.0/lib/chef/resource.rb line 1285)
I'm running this cookbook through vagrant chef provision and my Vagrant file is as follows
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
config.vm.define "provisioner" do |provisioner|
provisioner.vm.box = "ubuntu/xenial64"
provisioner.vm.box_version = "20180509.0.0"
provisioner.vm.box_check_update = false
provisioner.omnibus.chef_version = :latest
provisioner.vm.network "forwarded_port", guest: 8080, host: 8282
provisioner.vm.network "forwarded_port", guest: 8383, host: 8383
provisioner.vm.provider :virtualbox do |vbox|
vbox.name = "pipeline-jumpstart-chef"
vbox.memory = 2048
vbox.cpus = 2
end
provisioner.vm.provision "chef_solo" do |chef|
chef.node_name = "chef-provisioned"
chef.cookbooks_path = "../../cookbooks"
chef.verbose_logging = true
chef.add_recipe "pipeline-jumpstart-chef"
end
end
end
here's the source for cookbook on which I'm building wrapper
You mention your are overriding the attributes but your code indicates you are setting those attributes to the default level. You should review the Attribute Precedence in Chef to understand what default means exactly. In addition, inside the attributes file you don't need to prefix with node just use default::
default['nexus3']['properties_variables'] = { port: '8383', host: '0.0.0.0', args: '${jetty.etc}/jetty.xml,${jetty.etc}/jetty-http.xml,${jetty.etc}/jetty-requestlog.xml', context_path: '/nexus/' }
default['nexus3']['api']['host'] = 'http://localhost:8383'
default['nexus3']['api']['username'] = 'admin'
default['nexus3']['api']['password'] = 'Ch5f#A4min'
The node.default syntax is used inline, inside a recipe to set attributes. If you review the precedence chart you'll notice inline and default attributes are one level higher.
If you want to use override you can do this for each attribute:
override['nexus3']['properties_variables'] = { port: '8383', host: '0.0.0.0', args: '${jetty.etc}/jetty.xml,${jetty.etc}/jetty-http.xml,${jetty.etc}/jetty-requestlog.xml', context_path: '/nexus/' }
override['nexus3']['api']['host'] = 'http://localhost:8383'
override['nexus3']['api']['username'] = 'admin'
override['nexus3']['api']['password'] = 'Ch5f#A4min'
However, unless it's absolutely necessary to set these attributes in the wrapper cookbook you are likely better off setting this as a default attribute at a higher precedence, such as a role. See the quote below from the same document' Attribute Types section about override attributes:
An override attribute is automatically reset at the start of every
chef-client run and has a higher attribute precedence than default,
force_default, and normal attributes. An override attribute is most
often specified in a recipe, but can be specified in an attribute
file, for a role, and/or for an environment. A cookbook should be
authored so that it uses override attributes only when required.
If you simply set these as default inside your wrapper cookbook's attributes/default.rb file then both the source cookbook and your wrapper are trying to set the same attribute at the same level. This is likely going to lead to unexpected behavior or simply not work.
I am using encrypted data bags within Chef and I want to add a condition within my Chef recipe as follows:
If (test kitchen) then
encryptkey = data_bag_item("tokens", "encryptkey")
If ( not test kitchen ) then
secret = Chef::EncryptedDataBagItem.load_secret("/etc/chef/encrypted_data_bag_secret")
encryptkey = Chef::EncryptedDataBagItem.load("tokens", "encryptkey", secret)
I have added data_bags_path and encrypted_data_bag_secret_key_path within kitchen.yml as follows:
provisioner:
name: chef_zero
chef_omnibus_url: omni-url/chef/install.sh
roles_path: 'test/integration/default/roles'
data_bags_path: "test/integration/default/data_bags"
encrypted_data_bag_secret_key_path: "test/integration/default/encrypted_data_bag_secret"
Use the attributes in your kitchen.yaml.
suites:
- name: default
data_bags_path: 'databags'
run_list:
- recipe[x::y]
attributes: {'kitchen' : 'true' }
Inside your recipe put if condition using the value of node['chef-mode'].
if node['kitchen'] == 'true'
#something
else
#else
end
Just use data_bag_item("tokens", "encryptkey") for both. It will take care of decryption for you automatically.
Problem statement:
Given a folder name, Check if it exists on VCenter Server and if not create the same.
Clone a VM from template under this folder.
For example, I want to clone a VM under "Administrator Desktops" as shown in the following image:
Click here to see the image
The script below is cloning a VM from specified template and placing the VM into specified folder. But fails when that folder does not exist on VCenter server:
---
- hosts: localhost
connection: local
sudo: false
user: root
gather_facts: false
serial: 1
vars_files:
- createVmVars.yml
tasks:
- name: Deploying VM from template.
vsphere_guest:
vcenter_hostname: "{{vcenter_hostname}}"
username: "{{vcenter_username}}"
password: "{{vcenter_password}}"
guest: "{{guest_name}}"
from_template: yes
template_src: "{{template_src}}"
cluster: "{{cluster}}"
resource_pool: "{{resource_pool}}"
vm_extra_config:
folder: "{{folder_name}}"
Need help to make this script flexible so that when the folder does not exists, it should create the mentioned folder and then clone the VM under this folder.
You can use this Ansible module to create folder in vsphere:
#!/usr/bin/python
# -*- coding: utf-8 -*-
DOCUMENTATION = '''
---
author: "Shashank Awasthi"
module: vsphere_create_folder
short_description: Create a folder on VCenter if it does not exist
description:
- This module requires login to VCenter Server
- This module requires pysphere python module installed
- This module creates a folder on mentioned VCenter Server
- This module does not create any folder if the folder with the same name is already existing on the VCenter Server
- This module supports nesting upto only 2 levels
version_added: "1.2"
options:
host:
description:
- The vsphere Server on which the folder is to be created
required: true
login:
description:
- The login name to authenticate on VSphere
required: true
password:
description:
- The password to authenticate VSphere
required: true
folder_name:
description:
- The folder name which is to be created
required: true
parent_folder_name:
description:
The name of parent folder under which the folder_name is residing.
required: true
datacenter_name:
description:
- The name of the datacenter where the folder is to be created
required: true
examples:
- description: create the folder with name NewDeployments
code:
- local_action: vsphere_create_folder host=$eszserver login=$esxlogin password=$esxpassword folder_name=$folder_name
parent_folder_name=$parent_folder_name datacenter_name=$dc_name
notes:
- This module ought ot be run from a system which can access vsphere directly
'''
import sys
try:
import pysphere
from pysphere import *
from pysphere.resources import VimService_services as VI
except ImportError:
print "failed=true, msg=Pysphere Python module not available"
sys.exit(1)
def main():
module = AnsibleModule(
argument_spec = dict(
host = dict(requred = True),
login = dict(required = True),
password = dict(required = True),
folder_name = dict(required = True),
parent_folder_name = dict(required = True),
datacenter_name = dict(required = True)
)
)
host = module.params.get('host')
login = module.params.get('login')
password = module.params.get('password')
folder_name = module.params.get('folder_name')
parent_folder_name = module.params.get('parent_folder_name')
datacenter_name = module.params.get('datacenter_name')
server = pysphere.VIServer()
try:
server.connect(host,login,password)
except Exception, e:
module.fail_json(msg = 'Failed to connect to %s: %s' % (host, e))
def createFolder(vm_folder,folder_name):
try:
request = VI.CreateFolderRequestMsg()
_this = request.new__this(vm_folder)
_this.set_attribute_type(vm_folder.get_attribute_type())
request.set_element__this(_this)
request.set_element_name(folder_name)
server._proxy.CreateFolder(request)
except pysphere.ZSI.FaultException, e:
pass
try:
datacenters = server._get_datacenters()
dc = datacenters[datacenter_name]
dc_props = VIProperty(server,dc)
vm_folder = dc_props.vmFolder._obj
createFolder(vm_folder,parent_folder_name)
folders = server._retrieve_properties_traversal(property_names=['name'], from_node = dc, obj_type = 'Folder')
for f in folders:
if f.PropSet[0].Val == parent_folder_name:
vm_folder = f.Obj
break
createFolder(vm_folder,folder_name)
except Exception, e:
module.fail_json(msg = "failed to create folder: %s" % e)
module.exit_json(changed = True, folder = folder_name, parent_folder = parent_folder_name)
#<<INCLUDE_ANSIBLE_MODULE_COMMON>>
main()
I was happily using Vagrant and Chef Solo, but I read that using Chef Zero will lead to an easier transition to the full Chef Server (something which might happen in the near future of the project). When I started using vagrant plus chef-zero, I get the error during the Chef run, described below.
Vagrant: 1.7.4
Chef(on the guest machine) - 12.4.1
And when I do a $ vagrant up db, this is the output from Chef
==> db: [2015-07-28T09:14:13+00:00] INFO: Chef-client pid: 2525
==> db: [2015-07-28T09:14:14+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/db-machine
==> db: [2015-07-28T09:14:14+00:00] INFO: Setting the run_list to ["role[mysql]"] from CLI options
==> db: [2015-07-28T09:14:14+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/roles/mysql
==> db: [2015-07-28T09:14:14+00:00] ERROR: Role mysql (included by 'top level') is in the runlist but does not exist. Skipping expand.
==> db: Error expanding the run_list:
==> db:
==> db:
==> db: Missing Role(s) in Run List:
==> db: ----------------------------
==> db: * mysql included by 'top level'
==> db:
==> db: Original Run List
==> db: -----------------
==> db: * role[mysql]
This is the relevant bit of my Vagrantfile
machine.vm.provision "chef_zero" do |chef|
chef.node_name = #hostname
chef.cookbooks_path = "#{#chef_repo}/cookbooks"
chef.roles_path = "#{#chef_repo}/roles"
chef.environments_path = "#{#chef_repo}/environments"
chef.environment = #environment
roles.each do |role|
chef.add_role role
end
chef.json = {
"guest_folder_path" => guest_path,
"ubuntu_user" => #ubuntu_user
}
end
When I log in to the virtual machine(db) and go to /tmp/chef-vagrant
$ cd /tmp/vagrant-chef/
vagrant#db-machine:/tmp/vagrant-chef$ ls
2107f3d10c0db9887cdf980054c11e35/ client.rb dna.json
8a1f8968cff1cd79d46fc9ca5bd46931/ df5881620e1c72537e51bddcbc6ceb3a/
vagrant#db-machine:/tmp/vagrant-chef$ cat client.rb
node_name "db-machine"
file_cache_path "/var/chef/cache"
file_backup_path "/var/chef/backup"
cookbook_path ["/tmp/vagrant-chef/df5881620e1c72537e51bddcbc6ceb3a/cookbooks"]
role_path "/tmp/vagrant-chef/2107f3d10c0db9887cdf980054c11e35/roles"
log_level :info
verbose_logging false
enable_reporting false
encrypted_data_bag_secret nil
environment_path "/tmp/vagrant-chef/8a1f8968cff1cd79d46fc9ca5bd46931/environments"
environment "develop"
chef_zero.enabled true
local_mode true
add_formatter "null"
And this is the content of /roles
vagrant#db-machine:/tmp/vagrant-chef/2107f3d10c0db9887cdf980054c11e35/roles$ ls
mysql.rb
vagrant#db-machine:/tmp/vagrant-chef/2107f3d10c0db9887cdf980054c11e35/roles$ cat mysql.rb
name "mysql"
description "Role for a machine with a mysql db on it"
run_list "recipe[db]"
env_run_lists "_default"=>[],
"production" => ["recipe[git_deploy]", "recipe[db::aws]", "recipe[db]"],
"staging" =>["recipe[git_deploy]", "recipe[db::aws]", "recipe[db]"],
"develop" => ["recipe[db]"]
default_attributes "mysql_bind_address" =>"0.0.0.0",
"mysql_db_name"=> "some_db_name",
"mysql_password"=> "root",
"mysql_datadir" => "/var/lib/mysql",
"db_name" => "some_db_name",
"db_init_runlist" => [
"some_file.sql",
"some_other_file.sql"
]
Apparently rb role definitions are not supported in chef zero only JSON ones.
Chef Zero Bug reported
That one is closed but refers to other that is still open.
I have the same problem, so my solution it's going to be to use this gist to translate my roles to JSON and use them that way with vagrant.
I try to create a chef cookbook to launch multiple mpd instances in my vagrant virtual box (using chef-solo).
I want to configure each instance in my Vagrantfile like this:
mpd: {
channels: {
mix: {
name: 'mpd_mix',
bind: '0.0.0.0',
socket: '/home/vagrant/.mpd/socket/mix',
port: '6600'
},
tech: {
name: 'mpd_tech',
bind: '0.0.0.0',
socket: '/home/vagrant/.mpd/socket/tech',
port: '6601'
}
}
}
So the recipe should take these settings and loop through them (creating an mpd instance for each channel).
This is what I currently have as a recipe:
package "mpd"
node.normal[:mpd][:channels].each_value do |channel|
# create socket
file channel[:socket] do
action :touch
end
# trying to set the attributes for the config file and the service
node.set[:mpd][:port] = channel[:port]
node.set[:mpd][:db_file] = "/var/lib/mpd/tag_cache_" + channel[:name]
node.set[:mpd][:bind_2] = channel[:socket]
node.set[:mpd][:icecast_mountpoint] = "/" + channel[:name] + ".mp3"
node.set[:mpd][:channel_name] = channel[:name]
# create service
service channel[:name] do
service_name "mpd" # linux service command
action :enable
end
# create the corresponding config file
config_filename = "/etc/" + channel[:name] + ".conf"
template config_filename do
source "mpd.conf.erb"
mode "0644"
notifies :restart, resources(:service => channel[:name])
end
end
I have several Problems with this:
Ist does not create a system service for each mpd instance, so I can do sudo service mpd_mix start. Why?
It does not use the /etc/mpd_mix.conf config file when launching mpd, because it still calls /etc/init.d/mpd start which uses /etc/mpd.conf. How can I change that, so it uses the correct config file for each mpd instance?
Adjusting the attributes for the creation of the config files does not work as expected (see the node.set part in the code above). Both config files, /etc/mpd_tech.conf and /etc/mpd_mix.conf use the tech channel attributes. Looks like the mix settings get overwritten somehow? How can I fix that?
I'd really appreciate some help on this as I am quite new to chef cookbooks.
I figured out how to do it. Here is the relevant code part:
node[:mpd][:channels].each_value do |channel|
# create socket
file channel[:socket] do
action :touch
end
# create init file
init_filename = "/etc/init.d/" + channel[:name]
template init_filename do
variables :channel => channel
source "mpd.init.erb"
mode "0755"
end
# create service
service channel[:name] do
service_name channel[:name] # linux service command
action :enable
end
# create config file
config_filename = "/etc/" + channel[:name] + ".conf"
template config_filename do
variables :channel => channel
source "mpd.conf.erb"
mode "0644"
notifies :restart, resources(:service => channel[:name])
end
end
If you want to take a closer look, take a look at the complete cookbook repository on github: https://github.com/i42n/chef-cookbook-mpd/blob/master/recipes/default.rb