I have following code in my recipe, which runs bash script that returns users (except root). 1 user per line.
ruby_block "Delete users" do
action :run
block do
users = Mixlib::ShellOut.new("whatever command with lines containing users").run_command.stdout
users.each_line do |user|
ex = Chef::Resource::Execute.new("Removing User: #{user}", run_context)
ex.command "remove user"
ex.run_action(:run)
end
end
end
Now I managed to mock shell out Here, but I can't figure out why and if it is possible to expect such resources. For instance if command returns
user1
user2
Then following will work
allow(shellout).to receive("whatever command with lines containing users").and_return("user1")
expect(chef_run).to run_ruby_block('Delete users')
But if I add following
expect(chef_run).to run_execute('Removing User: user1')
It will fail, while there isn't any resources in context that I haven't cover. (I am using Chef::Coverage so I know).
Thanks
Related
I have the below hash in a chef recipe, that creates a directory/s
node['fnb_base_directory']['directory_name'].map do |directory_name, dir|
next if directory_name.empty?
directory directory_name do
owner dir['owner']
group dir['group']
mode dir['mode']
recursive dir['recursive']
action dir['action']
only_if "getent passwd #{dir['owner']}" && "getent group #{dir['group']}"
end
end
I ONLY want chef to try create the directory based on this guard:
only_if "getent passwd #{dir['owner']}" && "getent group #{dir['group']}"
So that basically means that BOTH the user and the group must exist before trying to create the directory.
The problem appears to be that when chef interprets this, I see it is only adhering to ONE of the statements i.e. ONLY checks that group exists and then proceeds to attempt to create the directory, but will fail because the user does not exist yet.
See below interpretation:
directory("/opt/test_dir_creation") do
action [:create]
default_guard_interpreter :default
declared_type :directory
cookbook_name "fnb_base_wrapper"
recipe_name "fnb_base_directory"
recursive false
owner "nonexistent_user"
group "opc"
mode "0755"
only_if "getent group opc"
end
Failure:
directory[/opt/test_dir_creation] action create
* cannot determine user id for 'nonexistent_user', does the user exist on this system?
================================================================================
Error executing action `create` on resource 'directory[/opt/test_dir_creation]'
================================================================================
Chef::Exceptions::UserIDNotFound
--------------------------------
cannot determine user id for 'nonexistent_user', does the user exist on this system?
The reason the user does not exist yet, is because that user is created in another cookbook whose priority is not as high as our base cook (which creates directories), hence why the directory creation is done before the user is created.
The directory will then be created on the 2nd converge, where the user will then exist, and proceed to create the directory ONLY then.
FYI.
I am using getent because we use AD on our servers, so it may not always be a static user/group, but one that resides in AD.
I have also checked this question:
Using multiple conditions in Chef only_if guard
It does not help me.
Your help, guidance and advice will be greatly appreciated.
Try removing the quotes in the middle so the whole expression including the && condition runs in the shell.
only_if "getent passwd #{dir['owner']} && getent group #{dir['group']}"
Chef accepts a string shell command or a ruby block
only_if "some shell commands which are possibly in a pipeline return 0"
only_if {
return true if <condition1> && <condition2>
return true if <condition3> || <condition4>
return false
}
I have the following code to figure out where Java is located on the box. Java comes with our application and what Java version that is included with the application differs.
def app_java_home
if Dir.exist?("#{app_home}/jre-server/linux")
Dir.chdir("#{app_home}/jre-server/linux") do
Dir.glob('jdk*').select { |f| File.directory? f }[0]
end
end
end
Then, in my cookbook I have
aws_s3_file "#{app_download_path}/#{app_s3['archive_file']}" do
bucket app_s3['bucket']
remote_path app_s3['remote_path']
region aws_region
not_if { ::Dir.exists?(app_bin_dir) }
not_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end
execute 'move' do
user 'root'
command "mv #{app_download_path}/ourapp/ #{app_install_path}"
not_if { ::Dir.exists?(app_home) }
end
cookbook_file "#{app_java_home}/jre/lib/security/local_policy.jar" do
source %W[#{app_release}/local_policy.jar default/local_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
cookbook_file "#{app_java_home}/jre/lib/security/US_export_policy.jar" do
source %W[#{app_release}/US_export_policy.jar default/US_export_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
However, the two cookbook_file resources fails because it can't find the directory:
No such file or directory # dir_chdir - /ourapp/jre-server/linux/
After a lot of googling, I've come to the conclusion that it's a .. "missmatch" (?) between compile time and run time of the recipes. Basically, if I understand it correctly, it tries to run the cookbook_file resource(s) first but fails. So never downloads, unpacks and installs the app artefact.
I've tried running app_java_home when the directory exists, and it does seem to work the way I want it..
I tried putting the cookbook_file resources in a ruby_block, but then I instead get:
undefined method `cookbook_file' for Chef::Resource::RubyBlock
The app_java_home .. function (?) used to look like this:
def app_java_home
"#{app_home}/jre-server/linux/#{jdk_version}"
end
Where jdk_version came from the databag. This worked fine, but we have a long standing bug/feature request in our system where it sometimes happens that "they" get the version they put in the databag wrong, causing all sorts of problems.. So they want a way to remove this dependency and instead "figure this out" dynamically.
Ruby and Chef isn't my forte, so I'm not sure what to try next. I have found references to Chef::Resources::CookbookFile (which, if I understand it, could/should be used inside ruby_blocks), but can't find any examples or documentation about it. The link on RubyDocs is broken.
Adding an answer here for a better explanation.
Any (Ruby) code that is not within any of the Chef resources, will run in Compile phase
All resource declarations will run in Convergence phase in the order they are defined
Thankfully, there is a way to make resources run in Compile phase if so required. Though IMHO it should be done sparingly and in exceptional cases.
As per your comment aws_s3_file and execute resources are the ones that unpack the app (and create the directory). In this case, it seems you want them to run in compile phase.
Prior to Chef client 16.0
Use the run_action option with the action that should be performed at the compile time. For example execute resource takes action :run:
# Note action ":nothing" and "run_action"
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
action :nothing
end.run_action(:run)
Chef client 16.0 onwards
We can add a common property to the resources. Example with execute resource:
# Note the extra property "compile_time"
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/ourapp") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
compile_time true
end
And finally to answer the subject of the question:
Chef use cookbook_file in ruby block
This is not possible. Refer to the first point on the top. If we want Ruby code to run during converge (instead of compile), we put it within the ruby_block resource. So it can contain code like (for example):
ruby_block 'get directory' do
block do
def app_java_home
"#{app_home}/jre-server/linux/#{jdk_version}"
end
end
end
With the help of #seshadri_c, I finally managed to solve the problem! It took some doing, because I kept misunderstanding the suggestions etc.
So this is what I came up with (for posterity):
def jdk_version(required = true)
base_dir = "#{app_home}/jre-server/linux"
if Dir.exist?("#{base_dir}")
Dir.chdir("#{app_home}/jre-server/linux") do
Dir.glob("jdk*").each do |f|
if File.directory?(f)
return "#{f}"
end
end
end
end
end
def app_java_home
return "#{app_home}/jre-server/linux/#{jdk_version}"
end
Turns out I need to get just the version, individually, as well, so I rearranged the functions a bit. I'm sure it could be written much cleaner, but here the trick was to use return instead of puts/print! Well, I'm a programmer, but not a Ruby programmer so didn't know that was an option..
Then, in the cookbook, I added the .run_action() where needed. I didn't need them for the cookbook_file, which simplified things a bit:
aws_s3_file "#{app_download_path}/#{app_s3['archive_file']}" do
bucket app_s3['bucket']
remote_path app_s3['remote_path']
region aws_region
not_if { ::Dir.exists?(app_bin_dir) }
not_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end.run_action(:create)
execute 'extract' do
user 'root'
command "unzip #{app_download_path}/#{app_s3['archive_file']} > /dev/null"
not_if { ::Dir.exists?("#{app_home}/app") }
only_if { ::File.exists?("#{app_download_path}/#{app_s3['archive_file']}") }
end.run_action(:run)
execute 'move' do
user 'root'
command "mv #{app_download_path}/app/ #{app_install_path}"
not_if { ::Dir.exists?(app_home) }
end.run_action(:run)
# JCE Unlimited Strength Jurisdiction Policy Files
cookbook_file "#{app_java_home}/jre/lib/security/local_policy.jar" do
source %W[#{app_release}/local_policy.jar default/local_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
cookbook_file "#{app_java_home}/jre/lib/security/US_export_policy.jar" do
source %W[#{app_release}/US_export_policy.jar default/US_export_policy.jar]
owner app_user_name
group app_group_name
mode 0755
end
With all that, everything is running exactly when they're supposed to and everything seems to be working.
Can we use a Chef resource to change the GID from 101 to 100(or any other number) in /etc/passwd file?libuuid:x:100:101::/var/lib/libuuid:
For all the GID's with 101 in /etc/passwd if one needs to change the GID, what is the way to do it using Chef resources?
Before suggesting a solution for this, I would like point out 2 things:
Hand/scripted editing of /etc/passwd file is best avoided as it can lead to issues.
Chef is not the tool for editing files. Chef resources are converged on the node they run, and bring the state of the resource to state defined in the recipe.
If you would still like to use Chef, you could use Ruby code inside ruby_block resource.
However, the cleanest way to handle this would be identify the users (separately) and use the user resource.
Example:
# This will set gid as 1001 for user1, user2, user3
%w(
user1
user2
user3
).each do |u|
user u do
# add any other properties as required
gid '1001'
end
end
Update:
A sample file /tmp/userlist with below contents:
john:x:100:101::/bin/bash:
david:x:207:100::/bin/bash:
joe:x:100:101::/bin/nologin:
mike:x:101:100::/bin/bash:
rich:x:103:207::/bin/bash:
fred:x:105:111::/bin/nologin:
Not an expert at Ruby, but here it goes. The following ruby_block will read lines from a file, and create a new file with the lines with GID 101 replaced:
ruby_block "Write file" do
block do
puts
userlist = File.readlines('/tmp/userlist')
fp = File.open('/tmp/newuserlist', mode='w')
userlist.each do |line|
gid = line.split(':')[3]
if gid == '101'
fp.write line.sub(/.*\K101/, '1001')
else
fp.write line
end
end
fp.close
end
end
Note: There may be a cleaner and easier way to do this with Ruby, or with some other language or even Shell script. Do consider the same.
utilize the user resource
user 'a user' do
comment 'A random user'
uid 1234
gid 'groupname'
home '/home/random'
shell '/bin/bash'
password '$1$JJsvHslasdfjVEroftprNn4JHtDi'
end
I'm having troubles trying to understand what's going on, why pure ruby code is executed first despite that the code is put last, this is a part of what the action :install contains:
action :install do
...
windows_package "#{installation_name}" do
source "#{Chef::Config[:file_cache_path]}\\#{installer_filename}"
options "INSTALLDIR=\"#{installation_path}\""
action :install
not_if {::Dir.exists?("#{installation_path}\\bin")}
end
env "MYSQL_PATH" do
value "#{installation_path}"
end
windows_path "#{installation_path}\\bin" do
action :add
end
windows_batch "Installing Service" do
code <<-EOH
set MYSQL_PATH="#{installation_path}"
call %MYSQL_PATH%\\bin\\mysqld-nt.exe --install MySQL
EOH
end
service "MySQL" do
action :start
end
service "MySQL" do
action :enable
end
change_pass_str = "call \"#{installation_path}\\bin\\mysql.exe\" -u root --execute \"UPDATE mysql.user SET Password=PASSWORD('#{root_password}') WHERE User='root';FLUSH PRIVILEGES;\""
puts change_pass_str
password_set_result = system(change_pass_str)
log !password_set_result ? "Password wasn't changed since root already have a password defined. Maybe there's still data from a previous installation." : "Password has been set!"
end
Please ignore the fact that i didn't put the variable definition, and know that they are well defined. The thing is that when this part of the lwrp is executed
change_pass_str = "call \"#{installation_path}\\bin\\mysql.exe\" -u root --execute \"UPDATE mysql.user SET Password=PASSWORD('#{root_password}') WHERE User='root';FLUSH PRIVILEGES;\""
puts change_pass_str
password_set_result = system(change_pass_str)
it can't find #{installation_path}\\bin\\mysql.exe since it is not yet installed, in spite of that the block is at the end of the action.
Can anyone point me what is my mistake? why the other (already defined in this case in windows LWRP) resources are executed at the end instead of the begining?
How can i fix it?
LWRPs are no different than recipes with regards to execution order. So, just like in a recipe, any ruby code that is not inside a resource will be executed during the 'resource gathering' phase. In your case, you need to wrap your code in a ruby_block or execute resource like this:
ruby_block 'change the password' do
block {
change_pass_str = "call \"#{installation_path}\\bin\\mysql.exe\" -u root --execute \"UPDATE mysql.user SET Password=PASSWORD('#{root_password}') WHERE User='root';FLUSH PRIVILEGES;\""
puts change_pass_str
password_set_result = system(change_pass_str)
end
end
OR
execute'change the password' do
command "call \"#{installation_path}\\bin\\mysql.exe\" -u root --execute \"UPDATE mysql.user SET Password=PASSWORD('#{root_password}') WHERE User='root';FLUSH PRIVILEGES;\""
end
That will cause this code to evaluate at run-time. You'll also want to add an only_if, not_if, or action :nothing to that ruby_block to ensure it only runs when needed. Basically, a not_if or only_if would run SQL code to check if the password was set or not, and if that block returned false, then the password change wouldn't run.
Alternately, you can use action :nothing to set the resource to not run, and then use a notification or subscription to trigger the resource to run only after mysql is installed.
Chef has a two pass system see THIS
first pass 'compile' the recipes and add the resources inside a collection
second pass 'converge' and for each resource check if it needs to be updated and act consequently.
In your lwrp, the ruby in it is run as soon as being processed, include your code into a ruby_block resource (DOC) for it to be executed at the converge time.
i would like to show you my use case and then discuss possible solutions:
Problem A:
i have 2 recipes, "a" and "b".. "a" installs some program on my file system (say at "/usr/local/bin/stuff.sh" and recipe "b" needs to run this and do something with the output.
so recipe "a" looks something like:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh"
(the script just echo(es) "stuff" to stdout)
and recipe "b" looks something like:
include_recipe "a"
var=`/usr/local/bin/stuff.sh`
(note the backquotes, var should contain stuff)
and now i need to do something with it, for instance create a user with this username. so at script "b" i add
user "#{node[:var]}"
As it happens, this doesn't work.. apparently chef runs everything that is not a resource and only then runs the resources so as soon as i run the script chef complains that it cannot compile because it first tries to run the "var=..." line at recipe "b" and fails because the "execute ..." at recipe a did not run yet and so the "stuff.sh" script does not exist yet.
Needless to say, this is extremely annoying as it breaks the "Chef runs everything in order from top to bottom" that i was promised when i started using it.
However, i am not very picky so i started looking for alternative solutions to this problem, so:
Problem B: i've run across the idea of "ruby_block". apparently, this is a resource so it will be evaluated along with the other resources. I said ok, then i'd like to create the script, get the output in a "ruby_block" and then pass it to "user". so recipe "b" now looks something like:
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`
end
end
user "#{node[:var]}"
However, as it turns out the variable (var) was not passed from "ruby_block" to "user" and it remains empty. No matter what juggling i've tried to do with it i failed (or maybe i just didn't find the correct juggling method)
To the chef/ruby masters around: How do i solve Problem A? How do i solve Problem B?
You have already solved problem A with the Ruby block.
Now you have to solve problem B with a similar approach:
ruby_block "create user" do
block do
user = Chef::Resource::User.new(node[:var], run_context)
user.shell '/bin/bash' # Set parameters using this syntax
user.run_action :create
user.run_action :manage # Run multiple actions (if needed) by declaring them sequentially
end
end
You could also solve problem A by creating the file during the compile phase:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh" do
action :nothing
end.run_action(:run)
If following this course of action, make sure that:
/usr/local/bin exist during Chef's compile phase;
Either:
stuff.sh is executable; OR
Execute it through a shell (e.g.: var=`sh /usr/local/bin/stuff.sh`
The modern way to do this is to use a custom resource:
in cookbooks/create_script/resources/create_script.rb
provides :create_script
unified_mode true
property :script_name, :name_property: true
action :run do
execute "creating #{script_name}" do
command "echo 'echo stuff' > #{script_name}"
not_if { File.exist?(script_name) }
end
end
Then in recipe code:
create_script "/usr/local/bin/stuff.sh"
For the second case as written I'd avoid the use of a node variable entirely:
script_location = "/usr/local/bin/stuff.sh"
create_script script_location
# note: the user resources takes a username not a file path so the example is a bit
# strange, but that is the way the question was asked.
user script_location
If you need to move it into an attribute and call it from different recipes then there's no need for ruby_blocks or lazy:
some cookbook's attributes/default.rb file (or a policyfile, etc):
default['script_location'] = "/usr/local/bin/stuff.sh"
in recipe code or other custom resources:
create_script node['script_location']
user node['script_location']
There's no need to lazy things or use ruby_block using this approach.
There are actually a few ways to solve the issue that you're having.
The first way is to avoid the scope issues you're having in the passed blocks and do something like ths.
include_recipe "a"
this = self
ruby_block "a_block" do
block do
this.user `/usr/local/bin/stuff.sh`
end
end
Assuming that you plan on only using this once, that would work great. But if you're legitimately needing to store a variable on the node for other uses you can rely on the lazy call inside ruby to do a little work around of the issue.
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`.strip
end
end
user do
username lazy { "#{node[:var]}" }
end
You'll quickly notice with Chef that it has an override for all default assumptions for cases just like this.