I have a following problem. The chef recipe code snippets below does not behave equally although they look the same to me in terms of pure logic.
template "Create a file if not exists" do
path "#{site_docroot}/somefile.php"
source 'somefile.php.erb'
action :create_if_missing
end
VS.
if !File.exists? "#{site_docroot}/somefile.php"
template "Create a file if not exists" do
path "#{site_docroot}/somefile.php"
source 'somefile.php.erb'
action :create
end
end
Both should create a file if it does not yet exist.
But in context of a custom recipe in Amazon OpsWorks "setup" stage, the first sollution works as expected.
But the second sollution delivers a "false positive" exactly every second time I run the recipe.
The "if" statement delivers false but the file does not exist at the end.
So I would like to know if there is any reason for that in chef or/and ruby with nesting "template" resource inside "if" block. Does "template" ressource run some kind of asynchronous?
the short answer is that chef actually runs in 2 phases. You have a compile phase and an execution (or sometimes called convergence) phase.
What this means is depending on the presence of the file the template will get inserted or not in the recipe at compile time.
Further reading:
https://docs.chef.io/chef_client.html
https://serverfault.com/questions/604719/chef-recipe-order-of-execution-redux
So what happens in your case:
first there is not file. The chef recipe (via compile phase) decides there should be a file and creates the file in the converge phase.
on the 2nd run, there is a file and the chef recipe decides (via compile again) that there should not be a file (i.e. missing template). When the node converges in the 2nd phase chef removes the file as it's trying to bring the node to the desired state.
That explains the flipping back and forth that you a see (every other run)
Related
My knowledge of Make is small. I have been told that everything you put after make (that does not contain "-") is a target.
Well a building process I have is failing.
First there is a line
make path/to/configuration_file
configuration_file is not a target. It is a autogenerated configuration file buried inside the directory structure ("path/to") that is of the form
#
# Boot Configuration
#
#
# DRAM Component
#
CONFIG_DRAM_TYPE_LPDDR4=y
# CONFIG_DRAM_TYPE_DDR4 is not set
CONFIG_DDR_SIZE=0x80000000
#
# Boot Device
#
# CONFIG_ENABLE_EMMC_BOOT is not set
# CONFIG_ENABLE_NAND_BOOT is not set
CONFIG_ENABLE_SPINAND_BOOT=y
# CONFIG_ENABLE_SPINOR_BOOT is not set
CONFIG_EMMC_ACCESS_8BIT=y
# CONFIG_EMMC_ACCESS_4BIT is not set
# CONFIG_EMMC_ACCESS_1BIT is not set
so I cannot understand how this is a target. For reference, when I run make there is a Makefile but this Makefile does not reference this file.
Still this line is going well.
The path where it fails says
make diags
and I have verified there is no "diags" target.
I will print here the error file that can give us more info of what is happening
GEN cortex_a/output/Makefile
Init diag test "orc_scheduler" ...
remoteconfig: Failed to generate configure in cortex_a/soc/visio/tests/orc_scheduler!
Makefile:11 recipe for target 'orc_scheduler-init' failed
make[10]: *** [orc_scheduler-init] Error 25
At least what I would like to know is how to interpret this error message. I don't know what the "11" or the "10" or the "25" refers to.
make is fundamentally a tool for automatically running commands in the right order so you don't have to type them in yourself. So all the commands make runs are commands that you could just type into your shell prompt. And all the errors that those commands generate are the same ones that you would see if you typed the command yourself. So, looking at make to try to understand those errors is looking in the wrong place: you have to look at the documentation for whatever command was invoked.
A "target" is just a file that make knows how to build. The fact that when you typed make <somefile> is didn't give you an error that it doesn't know how to build <somefile>, means that <somefile> is a target as far as your makefiles are concerned.
The error message Makefile:11: simply refers to the filename Makefile, line 11, which is where the command that make ran, that failed, can be found. But this likely won't help you solve the problem of why the command failed (unless the problem is you invoked it with the wrong arguments and you need to adjust the makefile to specify different arguments).
The command that failed generated the message:
remoteconfig: Failed to generate configure in cortex_a/soc/visio/tests/orc_scheduler!
I don't know what that means, but it's not related to make. You'll need to find out what this remoteconfig command is, what it does, and why it failed. It's unfortunate that it doesn't show any better error message as to why it failed to "generate configure", but again there's nothing make can do about that.
If you want to learn more about make you can look at the GNU make manual (note, GNU make is only one implementation of make; there are others and they are fundamentally the same but different in details).
The challenge prompt is above, and my latest attempt is below. The directories and files are created as expected, and the read-out after executing chef-apply multipleCopies.rb tells me the files are linked, but when I update any one of the files, the others do not follow suit. Any ideas? Here is my code:
for x in 1..3
directory "multipleCopy#{x}" do
mode '0755'
action :create
end
end
file "multipleCopy1/secret.txt" do
mode '0755'
action :create
end
for x in 2..3
link "multipleCopy#{x}/secret.txt" do
to "multipleCopy1/secret.txt"
link_type :hard
subscribes :reload, "multipleCopy1/secret.txt", :immediately
end
end
Note: For less headache, I am testing the recipe locally before uploading to the ubuntu server referenced in the prompt, which is why my file paths are different and why I have not yet included the ownership properties.
So a file hard link doesn't seem to be what the question is going for (though I would say your solution is maybe better since this is really not what Chef is for, more on that later). Instead they seem to want you to have three actually different files, but sync the contents.
So first the easy parts, creating the directories and the empty initial files. It's rare to see those for loops used in Ruby code, though it is syntactically valid:
3.times do |n|
directory "/var/save/multipleCopy#{n+1}" do
owner "ubuntu"
group "root"
mode "755"
end
file "/var/save/multipleCopy#{n+1}/secret.txt" do
owner "root
group "root"
mode "755"
end
end
But that doesn't implement the hard part of sync'ing the files. For that we need to first analyze the mtimes on the files and use the most recent as the file content to set.
latest_file = 3.times.sort_by { |n| ::File.mtime("/var/save/multipleCopy#{n+1}/secret.txt") rescue 0 }
latest_content = ::File.read("/var/save/multipleCopy#{latest_file+1}/secret.txt") rescue nil
and then in the file resource:
file "/var/save/multipleCopy#{n+1}/secret.txt" do
owner "root
group "root"
mode "755"
content latest_content
end
As for this not being a good use of Chef: Chef is about writing code which asserts the desired state of the machine. In the case of files like this, rather than doing this kind of funky stuff to check if a file has been edited, you would just say that Chef owns the file content for all three and if you want to update it, you do it via your cookbook (and then usually use a template or cookbook_file resource).
I have a recipe that looks similar to this:
...
custom_resource1 "example" do
writing stuff to a file
end
log 'File found!' do
message "Found it
level :info
notifies :run, 'custom_resource2[example]', :immediately
only_if { ::File.exists?(file) }
end
...
custom_resource1 is a big resource with other resources inside, and takes some time to complete (iterates over some data_bags and writes to a file).
Sometimes, I see that custom_resource1 fails during a chef run, but still custom_resource2 is triggered before the recipe fails.
Is there any way to ensure that custom_resource1 either failed or completed before moving on?
That isn't possible, Chef uses an entirely blocking execution model (other than the two-pass loading system). The full action method for each resource is run in order with no concurrency. You would have to post more code to isolate the actual problem.
I also thought it was strange, because the log statement was never printed out, even though custom_resource2 were triggered by the notify. The soloution was to remove the log statement and instead add:
custom_resource2 "example" do
do stuff
only_if { ::File.exists?(file) }
end
Guess it has something to do with the different chef phases
I just started using chef and don't know much about ruby.
I have problems understanding the language-syntax used in recipes.
Say, I create a directory in a cookbook in recipes/default.rb like:
directory "/home/test/mydir" do
owner "test"
mode "0755"
action :create
recursive true
end
I assume this is part of a valid ruby script. What do lines like owner "test" mean? Is this a function call, a variable assignment or something else entirely?
Chef is written in Ruby and makes an extensive use of Ruby ability to design custom DSL. Almost every chef configuration file is written with a Ruby-based DSL.
This means that in order to use chef effectively you should be familiar with the basic of Ruby syntax including
Grammar
Data types (the main difference compared to other languages are Symbols)
Blocks
You don't need to know a lot about metaprogramming in Ruby.
The case of the code you posted is an excellent example of a Ruby based DSL. Let me explain it a little bit.
# Call the method directory passing the path and a block
# containing some code to be evaluated
directory "/home/test/mydir" do
# chown the directory to the test user
owner "test"
# set the permissions to 0555
mode "0755"
# create the directory if it does not exists
action :create
# equivalent of -p flag in the mkdir
recursive true
end
Blocks are a convenient way to specify a group of operations (in this case create, set permissions, etc) to be evaluated in a single context (in this case in the context of that path).
Let's break it down.
directory "/home/test/mydir" do
...
end
You are just calling a global method defined by Chef called directory, passing one argument "/home/test/mydir", and a block (everything between the do and end).
This block is probably excecuted in a special scope created by Chef in which all of the options (owner, mode, action, etc.) are method.
i would like to show you my use case and then discuss possible solutions:
Problem A:
i have 2 recipes, "a" and "b".. "a" installs some program on my file system (say at "/usr/local/bin/stuff.sh" and recipe "b" needs to run this and do something with the output.
so recipe "a" looks something like:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh"
(the script just echo(es) "stuff" to stdout)
and recipe "b" looks something like:
include_recipe "a"
var=`/usr/local/bin/stuff.sh`
(note the backquotes, var should contain stuff)
and now i need to do something with it, for instance create a user with this username. so at script "b" i add
user "#{node[:var]}"
As it happens, this doesn't work.. apparently chef runs everything that is not a resource and only then runs the resources so as soon as i run the script chef complains that it cannot compile because it first tries to run the "var=..." line at recipe "b" and fails because the "execute ..." at recipe a did not run yet and so the "stuff.sh" script does not exist yet.
Needless to say, this is extremely annoying as it breaks the "Chef runs everything in order from top to bottom" that i was promised when i started using it.
However, i am not very picky so i started looking for alternative solutions to this problem, so:
Problem B: i've run across the idea of "ruby_block". apparently, this is a resource so it will be evaluated along with the other resources. I said ok, then i'd like to create the script, get the output in a "ruby_block" and then pass it to "user". so recipe "b" now looks something like:
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`
end
end
user "#{node[:var]}"
However, as it turns out the variable (var) was not passed from "ruby_block" to "user" and it remains empty. No matter what juggling i've tried to do with it i failed (or maybe i just didn't find the correct juggling method)
To the chef/ruby masters around: How do i solve Problem A? How do i solve Problem B?
You have already solved problem A with the Ruby block.
Now you have to solve problem B with a similar approach:
ruby_block "create user" do
block do
user = Chef::Resource::User.new(node[:var], run_context)
user.shell '/bin/bash' # Set parameters using this syntax
user.run_action :create
user.run_action :manage # Run multiple actions (if needed) by declaring them sequentially
end
end
You could also solve problem A by creating the file during the compile phase:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh" do
action :nothing
end.run_action(:run)
If following this course of action, make sure that:
/usr/local/bin exist during Chef's compile phase;
Either:
stuff.sh is executable; OR
Execute it through a shell (e.g.: var=`sh /usr/local/bin/stuff.sh`
The modern way to do this is to use a custom resource:
in cookbooks/create_script/resources/create_script.rb
provides :create_script
unified_mode true
property :script_name, :name_property: true
action :run do
execute "creating #{script_name}" do
command "echo 'echo stuff' > #{script_name}"
not_if { File.exist?(script_name) }
end
end
Then in recipe code:
create_script "/usr/local/bin/stuff.sh"
For the second case as written I'd avoid the use of a node variable entirely:
script_location = "/usr/local/bin/stuff.sh"
create_script script_location
# note: the user resources takes a username not a file path so the example is a bit
# strange, but that is the way the question was asked.
user script_location
If you need to move it into an attribute and call it from different recipes then there's no need for ruby_blocks or lazy:
some cookbook's attributes/default.rb file (or a policyfile, etc):
default['script_location'] = "/usr/local/bin/stuff.sh"
in recipe code or other custom resources:
create_script node['script_location']
user node['script_location']
There's no need to lazy things or use ruby_block using this approach.
There are actually a few ways to solve the issue that you're having.
The first way is to avoid the scope issues you're having in the passed blocks and do something like ths.
include_recipe "a"
this = self
ruby_block "a_block" do
block do
this.user `/usr/local/bin/stuff.sh`
end
end
Assuming that you plan on only using this once, that would work great. But if you're legitimately needing to store a variable on the node for other uses you can rely on the lazy call inside ruby to do a little work around of the issue.
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`.strip
end
end
user do
username lazy { "#{node[:var]}" }
end
You'll quickly notice with Chef that it has an override for all default assumptions for cases just like this.