I'm using puppet as my provisioner in one of my vagrant project. I'm trying to add a module for a custom bash_profile.
The module_path for puppet is set to:
puppet.module_path = "puppet/modules"
The class for my bash_profile module looks like this:
class bash_profile
{
file
{
"/home/vagrant/bash_profile":
ensure => present,
source => "puppet:///modules/bash_profile/files/bash_profile"
}
}
Here's the file structure for my puppet structure:
puppet
| manifests
| | phpbase.pp // my main manifest file that has includes for modules
| modules
| | bash_profile
| | | files
| | | | bash_profile // the actual bash_profile file I want to ensure is present on my VM
| | | manifests
| | | | init.pp // the init file included for the bash_profile class
When I run the provisioning for vagrant, I get the error
err: /Stage[main]/Bash_profile/File[/home/vagrant/bash_profile]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/bash_profile/files/bash_profile at /tmp/vagrant-puppet-1/modules-0/bash_profile/manifests/init.pp:8
I'm not sure why it can't retrieve the information. The path seems to be correct. Can anyone see what I'm missing?
Yes, you are not supposed to include the literal files/ in the URL. Instead, it should just be
puppet:///modules/bash_profile/bash_profile
You may also receive this error with recurse => true if your module name is invalid. For instance, if you have this module structure:
modules
├── my-example
│ └── files
│ └── example
│ └── test.txt
and this resource:
file { "/tmp/example":
ensure => directory,
recurse => true,
source => "puppet:///modules/my-example/example",
}
you'll get this error:
==> default: Info: Could not find filesystem info for file 'my-example/example' in environment production
==> default: Error: /Stage[main]/Main/Node[default]/File[/tmp/example]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///my-example/example
The fix is to rename the module—for instance, naming it my_example fixes it. The rules for module names are documented but easy to miss.
Things to care about
The Puppet URI format puppet:///modules/name_of_module/filename
The fileserver directory to be present in the module directory
This video is an shows step-by-step guide to resolve the error
Related
I got collection that has structure
namespace/
── collectionA/
├── docs/
├── galaxy.yml
├── README.md
└── roles/
├── roleA/
| └── tasks/
| ├──taskA.yml
| ├──taskB.yml
└── roleB/
└── tasks/
├──taskA.yml
├──taskB.yml
according to using collections if I wan to use that roles all I have to do is include_role with fqdn
- hosts: all
collections:
- my_namespace.my_collection
tasks:
- import_role:
name: role1
but it seems not working. I still get error:
ERROR! the role 'manage_users' was not found in edaas.post_provisioning:ansible.legacy:/home/jenkins/agent/workspace/Create_Infra/playbooks/roles:/home/cirunner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/home/jenkins/agent/workspace/Create_Infra/playbooks
12:10:53
12:10:53 The error appears to be in '/home/jenkins/agent/workspace/Create_Infra/playbooks/ansible_main_initial.yml': line 24, column 15, but may
12:10:53 be elsewhere in the file depending on the exact syntax problem.
12:10:53
12:10:53 The offending line appears to be:
12:10:53
12:10:53 - ansible.builtin.import_role:
12:10:53 name: manage_users
12:10:53 ^ here
Collection is installed correctly - checked by ansible-galaxy collection list
Any idea what can be still wrong? Role names are aligned to rules (lowercase and only characters with _
Collection is installed in /home/cirunner/.ansible/collections
ansible [core 2.11.12] config file = None configured module
search path = ['/home/cirunner/.ansible/plugins/modules',
'/usr/share/ansible/plugins/modules'] ansible python module location
= /usr/local/lib/python3.8/dist-packages/ansible ansible collection location = /home/cirunner/.ansible/collections executable location =
/usr/local/bin/ansible python version = 3.8.0 (default, Dec 9 2021,
17:53:27) [GCC 8.4.0] jinja version = 3.1.2 libyaml = True
I came across this thread, and my issue was I had hyphens in my role name, which is not supported as per the documentation found here.
Changing to underscores in the role name resolved the issue.
I know this is old but I had a similar issue and it took me hours to find my problem. Maybe the way I figured it out will help someone else.
My problem was an empty collection directory left over from some tests in my project directory <project_dir>/collections/ansible_collection/my_namespace/my_collection directory. ansible-galaxy found the real installed collection containing the role in /usr/share/ansible/collections and reported it as installed and everything fine.
But ansible-playbook found the empty directory in my project directory first and interpreted this as the collection location and didn't look further to the actual installed collection in /usr/share/ansible/collections.
So, how I figured it out and how you might be able to figure out your problem:
Explicitly mention the collection in your playbook:
...
collections:
- my_namespace.my_collection
Add at least four '-v' arguments when running your playbook:
ansible-playbook ... -vvvv
Search for following line and see where your collection was found:
Loading collection my_namespace.my_collection from <project_dir>/collections/ansible_collections/my_namespace/my_collection
I hope it helps.
No one has been able to explain this inside my company so if you are able to solve this KUDOS to you!
Inside my puppet repo I have setup as follows:
environment/ops/modules/papertrail
├── files
│ ├── elasticsearch_log_files.yml
│ ├── log_files.yml
│ └── remote_syslog.conf
|
└── manifests
├── elasticsearch.pp
└──init.pp
└── templates
└── elasticsearch_log_files.yml.erb
MY elasticsearch.pp file contains the following:
class papertrail::elasticsearch inherits papertrail {
$source = "puppet:///modules/papertrail"
file { "/etc/log_files.yml" :
mode => 0644,
owner => root,
group => root,
ensure => present,
source => "$source/elasticsearch_log_files.yml",
}
}
Now when I try to change the last line to:
"$source/elasticsearch_log_files.yml.erb",
or
"$source/templates/elasticsearch_log_files.yml",
Puppet errors out and says that it can't locate the file:
Error: /Stage[main]/Papertrail::Elasticsearch/File[/etc/log_files.yml]: Could not evaluate: Could not retrieve information from environment ops source(s) puppet:///modules/papertrail/elasticsearch_log_files.yml.erb
What is strange is that when I use the following stanza to just include the yml file instead of erb it works fine and the file gets populated on the target:
"$source/elasticsearch_log_files.yml",
How can I include my erb? I have dynamic variables that I need to assign to the configuration file log_files.yml and I am so far unable to do so =(
This is solved. I didn't add the template directory to my git commit so once added with git add . it worked.
How can I quickly run a custom function on any node with puppet apply ?
Let's say I have this file (test.pp)
file { "/root/files/f":
content => test('test'),
}
And this ruby file (test.rb) which only log and return the first agument.
require 'logger'
module Puppet::Parser::Functions
newfunction(:test, :type => :rvalue
) do |args|
log = Logger.new(STDOUT)
log.level = Logger::INFO
log.info(args[0])
args[0]
end
end
How does one call the ruby function test with a puppet apply test.pp?
My problem is that is take like 5minutes to run my entire puppet agent -t and I need to add a function to puppet so I would like to test it quickly instead of waiting 5minutes each time.
This will certainly work, but it's hard to maintain.
A more maintainable way to do this is to add your function to a module, then install that module on your master.
For example, create a new with the name of your module (eg. test_function):
mkdir -p test_function/lib/puppet/parser/functions
You should have the following tree in your test_function module
├── lib
│ └── puppet
│ ├── parser
│ │ └── functions
│ │ ├── test.rb
Add your test.rb code to the test.rb file
Copy this module to your master's module path (this is probably /etc/puppet/modules depending on your Puppet version, use puppet module list to find out)
After that, the puppet master will read its module list and dynamically add-in all functions it finds.
More documentation about it here:
https://docs.puppetlabs.com/guides/plugins_in_modules.html
https://docs.puppetlabs.com/guides/custom_functions.html
I just found out.
All you need to do is to add the test.rb file directly into /var/lib/puppet/lib/puppet/parser/functions/test.rb and the puppet apply will "see" the function.
I am soooo confused when it comes to Chef / Berkshelf and need help and advice.
What I've found / read there's an underlining assumption with some things with Berkshelf and for the newbie there is a bit of a grey area that needs filling
Let me try to explain:
I followed the typical Chef path
Create Chef-repo in user directory
C:\Users\itsmeofcourse\chef-repo
then hooked that into an internal git-repo
and happily writing basic cookbooks for Windows and uploading everything into that git-repo
as it stands every cookbook exists under the "cookbook" folder in my chef-repo.
C:\Users\itsmeofcourse\chef-repo
/cookbook
I've then followed the path of writing wrapper cookbooks around community cookbooks, so it would look like
client_iis - depends upon
department_iis - depends upon
global_iis - depends upon
iis - community cookbook
this allows us make IIS changes at certain different levels within our infrastructure.
Now where documentation I feel falls down, is everyone is saying move your cookbooks out of the "cookbook" folder
so what I understand, "your" chef-repo will exist in a git-repo but just for changes to sub-folders like environments / data bags / roles / certificates etc ? and the cookbook are then separate projects is that correct or not ?
Where do you move your cookbooks to ? anywhere on you machine / within your user %home%?
How does Chef know where these are stored or do you have to amend your "knife.rb" and point to a certain directory ?
so it would look like
knife.rb
cookbook_path ["c:/cookbooks"]
C:\Users\itsmeofcourse\chef-repo :github => repo_1
c:/cooksbooks
/base :github => repo_2
/iis :github => repo_3
/sql :github => repo_4
/client_iis :github => repo_
/department_iis :github => repo_3
Can I ask what am I missing
or do you place a berksfile in the root of my chef-repo and then do what ? to manage everything in my cookbook folder ?
I have read through https://github.com/berkshelf/berkshelf/issues/535
please can someone help
Yes, it's correct and you would usually move your cookbooks to a separate repository.
One gotcha I had, was while reading this article: http://www.prashantrajan.com/posts/2013/06/leveling-up-chef-best-practices/ and the "Single Repo Per Cookbook" part. Read the whole thing, it's good!
It seems like you are not missing anything. Moving your cookbooks out of cookbooks directory, means creating separate repos per cookbook and depending on them using top level Berksfile (in the root of your chef-repo).
For a typical vagrant+chef repo for a web app (called coolwebapp), I would usually have:
.
+-- cmp-cookbooks
| +-- cmp-coolwebapp (this is only cookbook stored in this repo, and this repo exists because of this cookbook)
+-- data_bags
| +-- users
| | +-- mysql.json
| | +-- os.json
| | +-- admins.json
| +-- private_keys
| +-- deployment.json
+-- environments
| +-- production.rb
| +-- staging.rb
| +-- qa.rb
| +-- integration.rb
| +-- local.rb
+-- nodes (but this should not be stored in your repo I guess)
| +-- ip_here.json
| +-- other_ip_here.json
+-- Berksfile
+-- Vagrantfile
Berksfile would contain:
cookbook "cmp-coolwebapp", "~>0.3.0", path: "./cmp-cookbooks/cmp-coolwebapp"
cookbook "cmp-provisioning", "~>0.7.0", git: _priv_provisioning_cookbook_repo_
cookbook "cmp-role-db", "~>0.7.0", git: _priv_role1_cookbook_repo_
cookbook "cmp-role-www", "~>0.8.0", git: _priv_role2_cookbook_repo_
cookbook "cmp-role-devops", "~>0.7.0", git: _priv_role3_cookbook_repo_
"cmp" stands for our company name. Our cookbooks are stored in private repos, and are being maintained individually.
Cookbook cmp-role-www for example, would have mostly community cookbooks as dependencies in Berksfile, and our own cmp-apache2, cmp-nginx, cmp-varnish wrapper cookbooks stored in its repo.
Answering your last question" "How does Chef know where these are stored or do you have to amend your "knife.rb" and point to a certain directory ?"
If you manage your cookbook dependencies with Berkshelf, you can include cookbook from any location you prefer:
cookbook "artifact", path: "/Users/reset/code/artifact-cookbook"
cookbook "mysql", git: "https://github.com/opscode-cookbooks/mysql.git", branch: "foodcritic"
cookbook "rightscale", git: "https://github.com/rightscale/rightscale_cookbooks.git", rel: "cookbooks/rightscale"
The last one is useful when you store several company cookbook in one repository.
http://berkshelf.com/ "Source options" section.
I'm just getting started with puppet (and vagrant) to set up the development environment for our team, which consists of 8+ developers, each of which have their particular bash configuration, etc. I've got all the software installed on the system to quickly deploy new development virtual machines, but I'm not sure the best way to set up the development environment for each particular user in an automated way (we will end up having several development environments and it would be convenient to write this once and be done).
For example, I'd like to set up a user joe, clone Joe's configuration repo from github, and then run a script in that github repository to set up the environment for Joe. Any suggestions for how to do this for Joe as well as Jimmy, James, Julie, Jane, Jim, Jake, and Jimbo?
In case its any help, the development machines will almost certainly be ubuntu systems.
In addition to #Matt's suggestion, I created a custom puppet module that instantiates the configuration environment for each individual based on their github preferences. The resulting puppet module users looks something like this:
users/
├── manifests
│ ├── init.pp # base level configurations for all users
│ ├── jake.pp # custom setup for jake
│ ├── james.pp # custom setup for james
│ ├── jane.pp # custom setup for jane
│ ├── jim.pp # custom setup for jim
│ ├── jimbo.pp # custom setup for joe
│ ├── jimmy.pp # custom setup for jimmy
│ ├── joe.pp # custom setup for julie
│ └── julie.pp # custom setup for jimbo
└── templates
The relevant tidbit is in the custom setup files for each user. For example, here's what jim.pp might look like:
class users::jim {
# make sure that all base configuration in init.pp is set up first
require users
# add the user here
user { 'jim':
# comment => 'Dean Malmgren',
home => '/home/jim',
shell => '/bin/bash',
uid => 201,
managehome => 'true',
groups => ['sudo', 'vagrant'],
provider => 'useradd',
password => '$6$kxHLEuHW$78m3zHVLu0XUUDaU4bT.PEg./FfcloJiWml',
}
# clone the repository the first time
exec { 'jim-clone-dotfiles':
command => 'git clone git://github.com/jim/dotfiles.git && python dotfiles/create_softlinks.py',
cwd => '/home/jim',
creates => '/home/jim/dotfiles',
user => 'jim',
group => 'jim',
require => [ Package['git'] ],
}
# fetch and update if jim decides to update his dotfiles repo
exec { 'jim-update-dotfiles':
command => 'git merge --ff-only origin/master && python create_softlinks.py',
cwd => '/home/jim/dotfiles',
unless => 'git fetch && git diff --exit-code origin/master',
user => 'jim',
group => 'jim',
require => Exec['jim-clone-dotfiles'],
}
}
You could use a puppet fact in the vagrant file to set the username and pass this through to your puppet manifests. Something like the following:
Vagrant.configure("2") do |config|
config.vm.provision :puppet do |puppet|
puppet.facter = {
"user_name" => ENV['USER']
}
end
end
This would pass the current logged in username through to puppet and then within your manifest files you could use the variable "$user_name" within your git commands to checkout the correct users repo and do any other related tasks.