I have a chef recipe that looks something like this:
package 'build-essential' do
action :install
end
cmd = Mixlib::ShellOut.new("gcc -dumpversion")
cmd.run_command
gcc_version = cmd.stdout.strip()
If I execute the recipe on a system where gcc is installed, the recipe runs fine without errors. However, if I run the recipe on a system which doesn't have gcc install I get the error 'no such file or directory - gcc'.
I came to know about the chef two-phases stuff when trying to find a solution to my problem. I was expecting the package installation to satisfy the gcc requirement. How can I tell chef that this requirement will be satisfied later and not throw an error at compile time?
I tried the following, but the attribute does not get updated.
Chef::Resource::RubyBlock.send(:include, Chef::Mixin::ShellOut)
ruby_block "gcc_version" do
block do
s = shell_out("gcc -dumpversion")
node.default['gcc_version'] = s.stdout.strip()
end
end
echo "echo #{node[:gcc_version]}" do
command "echo #{node[:gcc_version]}"
end
Any help is appreciated. Thanks.
So okay, a few issues here. First, forget that Chef::Resource::whatever.send(:include trick. Never do it, literally never. In this case, the ShellOut mixin is already available in all the places anyway.
Next, and more importantly, you've still got a two-pass confusion issue. See https://coderanger.net/two-pass/ for details but basically the strings in that echo resource (I assume that said execute originally and you messed up the coping?) get interpolated at compile time. You haven't said what you are trying to do, but you probably need to use the lazy{} helper method.
And last, don't store things in node attributes like that, it's super brittle and hard to work with.
Related
My knowledge of Make is small. I have been told that everything you put after make (that does not contain "-") is a target.
Well a building process I have is failing.
First there is a line
make path/to/configuration_file
configuration_file is not a target. It is a autogenerated configuration file buried inside the directory structure ("path/to") that is of the form
#
# Boot Configuration
#
#
# DRAM Component
#
CONFIG_DRAM_TYPE_LPDDR4=y
# CONFIG_DRAM_TYPE_DDR4 is not set
CONFIG_DDR_SIZE=0x80000000
#
# Boot Device
#
# CONFIG_ENABLE_EMMC_BOOT is not set
# CONFIG_ENABLE_NAND_BOOT is not set
CONFIG_ENABLE_SPINAND_BOOT=y
# CONFIG_ENABLE_SPINOR_BOOT is not set
CONFIG_EMMC_ACCESS_8BIT=y
# CONFIG_EMMC_ACCESS_4BIT is not set
# CONFIG_EMMC_ACCESS_1BIT is not set
so I cannot understand how this is a target. For reference, when I run make there is a Makefile but this Makefile does not reference this file.
Still this line is going well.
The path where it fails says
make diags
and I have verified there is no "diags" target.
I will print here the error file that can give us more info of what is happening
GEN cortex_a/output/Makefile
Init diag test "orc_scheduler" ...
remoteconfig: Failed to generate configure in cortex_a/soc/visio/tests/orc_scheduler!
Makefile:11 recipe for target 'orc_scheduler-init' failed
make[10]: *** [orc_scheduler-init] Error 25
At least what I would like to know is how to interpret this error message. I don't know what the "11" or the "10" or the "25" refers to.
make is fundamentally a tool for automatically running commands in the right order so you don't have to type them in yourself. So all the commands make runs are commands that you could just type into your shell prompt. And all the errors that those commands generate are the same ones that you would see if you typed the command yourself. So, looking at make to try to understand those errors is looking in the wrong place: you have to look at the documentation for whatever command was invoked.
A "target" is just a file that make knows how to build. The fact that when you typed make <somefile> is didn't give you an error that it doesn't know how to build <somefile>, means that <somefile> is a target as far as your makefiles are concerned.
The error message Makefile:11: simply refers to the filename Makefile, line 11, which is where the command that make ran, that failed, can be found. But this likely won't help you solve the problem of why the command failed (unless the problem is you invoked it with the wrong arguments and you need to adjust the makefile to specify different arguments).
The command that failed generated the message:
remoteconfig: Failed to generate configure in cortex_a/soc/visio/tests/orc_scheduler!
I don't know what that means, but it's not related to make. You'll need to find out what this remoteconfig command is, what it does, and why it failed. It's unfortunate that it doesn't show any better error message as to why it failed to "generate configure", but again there's nothing make can do about that.
If you want to learn more about make you can look at the GNU make manual (note, GNU make is only one implementation of make; there are others and they are fundamentally the same but different in details).
I have a simple chef cookbook and all it does is it sets the MOTD on a CentOS machine. It takes the content of the /tmp/mymotd.txt and turns it into the MOTD.
I also have a simple ruby script (a full-fledged ruby script) that simply reads the text from the web-server and puts in into the /tmp/mymotd.txt.
My questions are:
how do I run this ruby script from within the cookbook?
how do I pass some parameters to the script (e.g. the address of the web-server)
Thanks a lot beforehand.
Ad 1.
You can use libraries directory in scripts to place there your ruby script and declare it in a module. Example:
# includes
module MODULE_NAME
# here some code using your script
# Example function
def example_function (text)
# some code
end
end
You can use then
include MODULE_NAME
in your recipe to import those functions and just use it like
example_function(something)
What's good - you can use there also Chef functions and resources.
IMPORTANT INFO: Just remember that Chef has 2 compilation phases. First will be all of Ruby code, second all of Chef resources. This means, that you have to remember priority of code. I won't write here more info about it, since you haven't asked for this, but if you want, you can find it here.
Ad 2.
You can do this in several ways, but it seems to me, that the best option for you would be to use environments. You can find more info in here. Basically, you can set up environment for script before it will run - this way you can define some variables you would use later.
Hope this helps.
i would like to show you my use case and then discuss possible solutions:
Problem A:
i have 2 recipes, "a" and "b".. "a" installs some program on my file system (say at "/usr/local/bin/stuff.sh" and recipe "b" needs to run this and do something with the output.
so recipe "a" looks something like:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh"
(the script just echo(es) "stuff" to stdout)
and recipe "b" looks something like:
include_recipe "a"
var=`/usr/local/bin/stuff.sh`
(note the backquotes, var should contain stuff)
and now i need to do something with it, for instance create a user with this username. so at script "b" i add
user "#{node[:var]}"
As it happens, this doesn't work.. apparently chef runs everything that is not a resource and only then runs the resources so as soon as i run the script chef complains that it cannot compile because it first tries to run the "var=..." line at recipe "b" and fails because the "execute ..." at recipe a did not run yet and so the "stuff.sh" script does not exist yet.
Needless to say, this is extremely annoying as it breaks the "Chef runs everything in order from top to bottom" that i was promised when i started using it.
However, i am not very picky so i started looking for alternative solutions to this problem, so:
Problem B: i've run across the idea of "ruby_block". apparently, this is a resource so it will be evaluated along with the other resources. I said ok, then i'd like to create the script, get the output in a "ruby_block" and then pass it to "user". so recipe "b" now looks something like:
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`
end
end
user "#{node[:var]}"
However, as it turns out the variable (var) was not passed from "ruby_block" to "user" and it remains empty. No matter what juggling i've tried to do with it i failed (or maybe i just didn't find the correct juggling method)
To the chef/ruby masters around: How do i solve Problem A? How do i solve Problem B?
You have already solved problem A with the Ruby block.
Now you have to solve problem B with a similar approach:
ruby_block "create user" do
block do
user = Chef::Resource::User.new(node[:var], run_context)
user.shell '/bin/bash' # Set parameters using this syntax
user.run_action :create
user.run_action :manage # Run multiple actions (if needed) by declaring them sequentially
end
end
You could also solve problem A by creating the file during the compile phase:
execute "echo 'echo stuff' > /usr/local/bin/stuff.sh" do
action :nothing
end.run_action(:run)
If following this course of action, make sure that:
/usr/local/bin exist during Chef's compile phase;
Either:
stuff.sh is executable; OR
Execute it through a shell (e.g.: var=`sh /usr/local/bin/stuff.sh`
The modern way to do this is to use a custom resource:
in cookbooks/create_script/resources/create_script.rb
provides :create_script
unified_mode true
property :script_name, :name_property: true
action :run do
execute "creating #{script_name}" do
command "echo 'echo stuff' > #{script_name}"
not_if { File.exist?(script_name) }
end
end
Then in recipe code:
create_script "/usr/local/bin/stuff.sh"
For the second case as written I'd avoid the use of a node variable entirely:
script_location = "/usr/local/bin/stuff.sh"
create_script script_location
# note: the user resources takes a username not a file path so the example is a bit
# strange, but that is the way the question was asked.
user script_location
If you need to move it into an attribute and call it from different recipes then there's no need for ruby_blocks or lazy:
some cookbook's attributes/default.rb file (or a policyfile, etc):
default['script_location'] = "/usr/local/bin/stuff.sh"
in recipe code or other custom resources:
create_script node['script_location']
user node['script_location']
There's no need to lazy things or use ruby_block using this approach.
There are actually a few ways to solve the issue that you're having.
The first way is to avoid the scope issues you're having in the passed blocks and do something like ths.
include_recipe "a"
this = self
ruby_block "a_block" do
block do
this.user `/usr/local/bin/stuff.sh`
end
end
Assuming that you plan on only using this once, that would work great. But if you're legitimately needing to store a variable on the node for other uses you can rely on the lazy call inside ruby to do a little work around of the issue.
include_recipe "a"
ruby_block "a_block" do
block do
node.default[:var] = `/usr/local/bin/stuff.sh`.strip
end
end
user do
username lazy { "#{node[:var]}" }
end
You'll quickly notice with Chef that it has an override for all default assumptions for cases just like this.
I am very new to ruby and was trying to understand some code when I got stuck at this snippet:
directory "test_dir" do
action :create
recursive true
end
I tried googling directory class but was unsuccessful. I found a class Dir but its not the same. I see that intuitively this snippet should create a new directory and name it test_dir but I do not want to assume things and move forward.
EDIT
This was a part of a chef-recipe which is used to launch a particular task. For the purposes of launching, it needs to create a directory and download some jars to it. There is an execute method below which goes like this:
execute 'deploy' do
action :nothing
# ignore exit status of storm kill command
command <<-EOH
set -e
storm kill #{name} -w 1 || true
sleep 3
storm jar #{points to the jar}
EOH
end
Sorry I have to be a bit obfuscated as some of the things are not open sourced.
It is the Directory resource of the Chef framework. (DSL stands for domain-specific language. Ruby is well suited for them.)
It's the Chef internal DSL for Directory management. Read more here: http://wiki.opscode.com/display/chef/Resources#Resources-Directory
PS: The recursive true tells it to create the folder much like mkdir -p.
The snippet you pasted is not really enough information to go on (need context; where is the snippet from?)
That said directory looks more like a method than a class. First, it's lowercased and classes are CamelCased.
If it's a method, it's defined somewhere within the application. Have you tried something like this:
grep -r "def directory" ./ or
grep -r "directory" ./| grep "def"
If not in the application itself, it would be defined in one of the application's dependencies (grep -r "..." $GEM_HOME/gems instead)
directory is not a class, it is a method. I do not know what module it is a part of, but that snippet is about equivalent to this:
Kernel.directory.call("test_dir",lambda {action :create; recursive true})
That snippet uses some gem that adds a directory method to the Kernel object.
As others have mentioned, it is part of the directory management DSL Chef. A DSL is a set of methods integrated into the Kernel object; because of the very flexible method calling syntax of Ruby, method calls can look a lot like language keywords. That makes task specific commands (Domain Specific Languages: DSL) look very nice in Ruby; easy to use and flexible. Thus gems that add DSLs are very common.
Can I replace an executable (accessed via a system call from ruby) with an executable that expects certain input and supplies the expected output in a consistent amount of time? I'm mainly operating on Mac OSX 10.6 (Snow Leopard), but I also have access to Linux and Windows. I'm using MRI ruby 1.8.7.
Background: I'm looking at doing several DNA sequence alignments, one in each thread. When I try using BioRuby for this, either BioRuby or ruby's standard library's tempfile sometimes raise exceptions (which is better than failing silently!).
I set up a test that reproduces the problem, but only some of the time. I assume the main sources of variability between tests are the threading, the tempfile system, and the executable used for alignment (ClustalW). Since ClustalW probably isn't malfunctioning, but can be a source of variability, I'm thinking that eliminating it may aid reproducibility.
For those thinking select isn't broken - that's what I'm wondering too. However, according to the changelog, there was concern about tempfile's thread safety in August 2009. Also, I've checked on the BioRuby mailing list whether I'm calling the BioRuby code correctly, and that seems to be the case.
I really don't understand what the problem is or what exactly are you after, can't you just write something like
#!/bin/sh
#Test for input (syntax might be wrong, but you get the idea)
if [ $* ne "expected input" ]; then
echo "expected output for failure"
exit -1
fi
#have it work in a consistent amount of time
$CONSISTENT_AMOUNT_OF_TIME = 20
sleep $CONSISTENT_AMOUNT_OF_TIME
echo "expected output"
You can. In cases where I'm writing a functional test for program A, I may need to "mock" a program, B, that A runs via system. What I do then is to make program B's pathname configurable, with a default:
class ProgramA
def initialize(argv)
#args = ParseArgs(argv)
#config = Config.new(#args.config_path || default_config_path)
end
def run
command = [
program_b_path,
'--verbose',
'--do_something_wonderful',
].join(' ')
system(command)
...
end
def program_a_path
#config.fetch('program_b_path', default_program_b_path)
end
end
Program A takes a switch, "--config PATH", which can override the default config file path. The test sets up a configuration file in /tmp:
program_b_path: /home/wayne/project/tests/mock_program_b.rb
And passes to program A that configuration file:
program_a.rb --config /tmp/config.yaml
Now program A will run not the real program B, but the mock one.
Have you tried the Mocha gem? It's used a lot for testing, and you describe it perfectly. It "fakes" the method call of an object (which includes just about anything in ruby), and returns the result you want without actually running the method. Take this example file:
# test.rb
require 'rubygems'
require 'mocha'
self.stubs(:system).with('ls').returns('monkey')
puts system('ls')
Running this script outputs "monkey" because I stubbed out the system call. You can use this to bypass parts of an application you don't want test, to factor out irrelevant parts.