I have a custom puppet fact (written in ruby) which executes a simple bash script. The script checks all files in the file system for world-readable permissions and returns results accordingly. The puppet fact checks the output of the script and returns "fail" if the script output is not null.
The problem is that the script is resource intensive and I don't want it to be executed every 30 minutes (client' puppet.conf: runinternval=1800).
I've tried using the "schedule" resource but because of the way puppet works, it only affects the class I'm using to evaluate the fact's output. Puppet facts are evaluated on each puppet run, regardless of anything else. (thus making them available for resources)
I've also tried to move the code that executes the bash script out from the fact and into the puppet class but it appears you cannot evaluate the output of a bash script (using "exec" type) and store it in a variable (again, because of the way puppet works).
I am now reduced to try and apply some kind of scheduling mechanism using Ruby language, and implement it in the fact. I've read a bit on PStore and right now this seems like a good direction.
Does anyone know how I can make a ruby script execute only during night? I cannot use crontab because the code will run by puppet.
Code snippet from the puppet class:
if $::myfact == 'fail' {
notify { "warning blah blah...":
loglevel => err,
schedule => 'night',
}
}
Code snippet from the puppet fact:
require 'facter'
Facter.add(:myfact) do
setcode do
if File.exists? '/usr/local/puppet/scripts/myscript.sh'
result = Facter::Util::Resolution.exec("/usr/local/puppet/scripts/myscript.sh")
if result != ''
puts "Result of myfact is: \n #{result}"
'fail'
end
end
end
end
Your custom fact can memorialize its result in a file somewhere on the system. When it is evaluated, it can compare the timestamp on that file to the current system time, and either read back the value or compute it freshly, as appropriate. That also allows you to defer updates (for instance, by touching the file) or to manually request re-evaluation (by deleting the file).
Consider, however, whether it makes sense to decouple the costly evaluation from fact gathering. Even if Facter only occasionally has to re-evaluate the fact from scratch, Facter -- and therefore Puppet -- will take a long time to perform those runs. You could consider instead using a scheduled job to perform the evaluation at whatever intervals you like, and have the custom fact always rely on cached results from the latest of those evaluations.
Related
Since Ruby supports parallel assignments and automatic value return from functions, almost every assignment and method run ends up creating an output when working on REPLs like IRB and Pry.
Normally I prevent this echo effect by putting a semicolon at the end of each line. For instance:
JSON::parse(very_long_json_string);
This normally prevents REPL echo. But when working with very large enumerables even one mistake can generate enough output to make a mess on the screen and put all my useful command history out of memory before I have the reflex to hit the break.
Is there a way to turn this echo effect off by default in Pry? As mentioned in the comments below (#Stefan), the same can be achieved in IRB by setting conf.echo = false.
In IRB there is:
conf.echo = false
In Pry you could replace the print object with an empty proc:
_pry_.config.print = proc {}
You'll have to store the old print object in order to restore it.
In both cases, the result of the last expression is still available via _
I'm processing files and directories looking for the most recent modified file in each directory. The code I have works but, being new to Ruby, I'm having trouble handling errors correctly.
I use Find.find to get a recursive directory listing, calling my own function newestFile for each directory:
Find.find(ARGV[0]) { |f|
if File.directory?(f)
newestFile(f)
end
}
In the directory tree there are folders I do not have permission to access, so I want to ignore them and go on to the next, but I cannot see how to incorporate the exception handling in to the Find.find "loop".
I tried to put begin..rescue..end around the block but that does not allow me to continue processing the loop.
I also found this SO question: How to continue processing a block in Ruby after an exception? but that handles the error in the loop. I'm trying to recover from an errors occurring in Find.find which would be outside the exception block.
Here's the stack trace of the error:
PS D:\dev\ruby\> ruby .\findrecent.rb "M:/main/*"
C:/Ruby200/lib/ruby/2.0.0/find.rb:51:in `open': Invalid argument - M:/main/<A FOLDER I CAN'T ACCESS> (Errno::EINVAL)
from C:/Ruby200/lib/ruby/2.0.0/find.rb:51:in `entries'
from C:/Ruby200/lib/ruby/2.0.0/find.rb:51:in `block in find'
from C:/Ruby200/lib/ruby/2.0.0/find.rb:42:in `catch'
from C:/Ruby200/lib/ruby/2.0.0/find.rb:42:in `find'
from ./findrecent.rb:17:in `<main>'
How do I add exception handling to this code?
I had a look in the code where the exception is being generated and the method contains the following block:
if s.directory? then
begin
fs = Dir.entries(file)
rescue Errno::ENOENT, Errno::EACCES, Errno::ENOTDIR, Errno::ELOOP, Errno::ENAMETOOLONG
next
end
... more code
Performing a horrible hack I added Errno::EINVAL to the list of rescue errors. My code now executes and goes through all the folders but I can't leave that change in the Ruby library code.
Internally find is using Dir.entries, so maybe I need to rewrite my code to process the folders myself, and not rely on find.
I would still like to know if there is a way of handling errors in this sort of code construct as from reading other code this type of small/concise code is used a lot in Ruby.
Do you get this error on your newestFile function or when you try to run File#directory??
If this happens in newestFile you can do something like this:
Find.find(ARGV[0]) do |f|
if File.directory?(f)
newestFile(f) rescue nil
end
end
This just ignores any errors and punts until the next folder. You could also do some nicer output if desired:
Find.find(ARGV[0]) do |f|
if File.directory?(f)
begin
newestFile(f)
rescue
puts "error accessing: #{f}, you might now have permissions"
end
end
end
If the error happens in the File#directory? you need to wrap that section as well:
Find.find(ARGV[0]) do |f|
begin
if File.directory?(f)
newestFile(f)
end
rescue
puts "error accessing: #{f}, you might now have permissions"
end
end
Like you mentioned if the error is occurring in the Find#find itself then you can't catch that from the block. It would have to happen inside of that method.
Can you confirm that the exception is happening in that method and not the subsequent ones by pasting a stack trace of the exception?
Edit
I was going to suggest traversing the directories yourself with something like Dir#entries so you would have that capacity to catch the errors then. One thing I am interested in is if you leave of the * in the call from the command line. I am on MacOS so I can't duplicate 100% what you are seeing but If I allow it to traverse a directory that I don't have access to on my mac it prints debug info about what folders I can't access but continues on. If I give it the * on the other had it seems to do nothing except print the error of the first folder it can't access.
One difference in my experience on the MacOS is that it isn't actually throwing the exception, it is just printing that debug info to the console. But it was interesting that the inclusion of the * made mine stop completely if I didn't have access to a folder.
You can be reactive or proactive, either works, but by testing in advance, your code will run a little faster since you won't be triggering the exception mechanism.
Instead of waiting for a problem to happen then trying to handle the exception, you can find out whether you actually should try to change to a directory or access a file using the File class's owned? and grpowned? methods. From the File documentation:
grpowned?(file_name) → true or false
Returns true if the named file exists and the effective group id of the calling process is the owner of the file. Returns false on Windows.
owned?(file_name) → true or false
Returns true if the named file exists and the effective used id of the calling process is the owner of the file.
That means your code can look like:
Find.find(ARGV[0]) do |f|
if File.directory?(f) && %w[grpowned? owned?].any?{ |m| File.send(m.to_s, f) }
newestFile(f)
end
end
Ruby will check to see if the directory entry is a directory and whether it is owned or grpowned by the current process. Because && is short-circuiting, if it's not a directory the second set of tests won't be triggered.
On some systems the group permissions will give you a better chance of having access rights if there are lots of shared resources, so that gets tested first, and if it returns true, any? will return true and the code will progress. If false is returned because the group permissions don't allow access, then owned? will test the file and the code will skip or step into newestFile. Reverse those two tests for speed depending on the set-up of your system. Or, run the code one with using time ruby /path/to/your/code then twiddle the two and run it again. Compare the resulting times to know which is faster on your system.
There are different schools of thought about whether using exception handling to control program flow is good and different languages prefer different things in their programming styles. To me, it seems like code will always run faster and more safely if I know in advance whether I can do something, rather than try and have it blow up. If it blows up in an expected way, that's one thing, but if it blows up in ways I didn't expect, then I might not have exception handling in place to react correctly, or it might trigger other exceptions that mask the true cause. I'd rather see if I can work my way out of a situation by checking the state, and then if all my attempts failed, have an exception handler that lets me gracefully exit. YMMV.
Finally, in Ruby, we don't name methods using Camelcase, we use snake_case. snake_case_is_easier toReadThanCamelCase.
I would like to check for the value of a node attribute. This case statement is what I have so far, and it works:
case node[:languages][:ruby][:host_cpu]
when "x86_64"
...
when "i686"
...
end
What I would like to do is use an if statement instead. This is what I tried:
if node[:languages][:ruby][:host_cpu]?("X86_64")
...
end
This is based on the following, Which worked.
if platform?("ubuntu")
...
end
However, my try didn't work. it gave a syntax error on the if line saying that there was an unexpected \n and $end was expected.
I found that there are two kinds of ways of performing an if. The first being the one I demonstrated above, which (apparently) only works with resources, and if_only, which works with nodes. like so
if_only {node[:languages]}
which seems to work only for checking the presence of nodes, and within a do context.
How do I check the value of a node using an if statement? One method does check values, but only of resources, the other checks nodes, but only for their presence, and not their values.
You are mixing up way to many different variants for conditionals, most of which are part of Chef, not Ruby. Let me try to describe the different options one by one.
Generally, a case is roughly comparable to a series of if and elsif statements. Your case above
case node[:languages][:ruby][:host_cpu]
when "x86_64"
...
when "i686"
...
end
is thus roughly equivalent to
if node[:languages][:ruby][:host_cpu] == "x86_64"
...
elsif node[:languages][:ruby][:host_cpu] == "i686"
...
end
As a side remark, case actually uses the === operator which is often not commutative but more powerful. For simple comparisons it works the same as == though. Both these variants are part of the Ruby language, in which you write your cookbooks.
The other options you mentioned are actually part of the API which Chef defined on top of Ruby. This is often called the Chef DSL (which stands for Domain Specific Language, i.e. an extension or adaption of a language, in this case Ruby for a specific usage domain, in this case configuration management.)
The platform? method is a method defined by Chef that checks whether the curent platform is one of the passed values. You can read more about that (and similar methods, e.g. the now recommended platform_family? method at the Chef docs for recipes in general and some often used ruby idioms.
As a side-remark: you might be surprised by the fact that Ruby allows the ? and ! characters to appear at the end of method names, which makes Ruby rather unique among similar languages in this regard. These characters are simply part of the method name and have no special meaning to the language. They are only used by convention to programmers to better identify the purpose of a method. If a method has a ? at the end, it is to be used to check some condition and is expected to return either a truthy or falsy value. Methods with a ! at the end often perform some potentially dangerous operation, e.g. change object in place, delete stuff, ... Again, this is only a convention and is not interpreted by the language.
The last option you mentioned, the only_if and by extension not_if are used to define conditionals on Chef resources to make sure they are only executed when a certain condition is true (or when using not_if, if it is false). As these attributes are used on Chef resources only, they are naturally also defined by Chef.
To understand why they are useful it is necessary to understand how a Chef run works. The details can be found at the description of the Anatomy of a Chef Run. What is important there is, that you basically have two execution phases: Resource Compilation and Convergence. In the first step, the actual code to define the resources is executed. Here, also the code in your case statement would be run. After all the recipes have been loaded and all the resources have been defined, Chef enters the second phase, the Convergence phase. There, the actual implementation of the resources which performs the changes (create files and directories, in stall packages, ...) is run. Only in this phase, the only_if and not_if conditions are checked.
In fact, you can observe the difference between
file "/tmp/helloworld"
action :create
content "hello world"
end
if File.exist?("/tmp/helloworld")
file "/tmp/foobar"
action :create
content "foobar"
end
end
and
file "/tmp/helloworld"
action :create
content "hello world"
end
file "/tmp/foobar"
action :create
content "foobar"
only_if{ File.exist?("/tmp/helloworld") }
end
In the first variant, the condition whether /tmp/foobar exists is checked during resource compilation. At this time, the code to actually create the /tmp/helloworld file has not been run, as it does that only in the Conversion step. Thus, during your first run, the /tmp/foobar file would not be created.
In the second variant however, the check is done with only_if which is evaluated during conversion. Here you will notice that both files get created in the first run.
If you want to read a bit more on how the definition of the conditionals works in terms of Ruby (and you definitely should), you can read about Ruby Blocks which are more or less pieces of code that can be passed around for later execution.
I recently asked how to test in RSpec if a block was called and the answers to that question seem to work in a simple case. The problem is when the initialization with the block is more complex. Then it is done in before and reused by a number of different tests in the context, among them the one testing if the block was evaluated. See the example:
context "the node definition using block of code" do
before do
#n=node do
# this block should be called
end
# some more complex setup concerning #n
end
it "should call the block" do
# how to test it?
end
# here a bunch of other tests using #n
end
In this case the solution with side effect changing value of a local variable does not work. Raising an exception from the block is useless since the whole statement must be properly evaluated to be used by the other tests.
Obviously I could do the tests separately, but it seems to stink, since I must copy-paste the initialization part and since the was-the-block-called test inherently belongs to this very context.
How to test if the block was evaluated in such a case?
Explanation for question asked by #zetetic below.
The context is that I'm implementing a kind of DSL, with nodes defined by their parameters and blocks of code (that can define something else in the scope of node). Since the things defined by the node's block can be pretty generic, at least for the first attempt I just need to be sure the block is evaluated and that what a user provides there will be considered. For now does not matter what it is.
Probably I should refactor my tests now and using mocks make them test behaviors rather then implementation. However it will be a little bit tricky, for the sake of some mixins and dynamic handling of messages sent to objects. For now the cincept of such tests is a little bit fuzzy in my head ;-)
Anyway your answers and comments helped me to better understand how RSpec works and explained why what I'm trying to do looks as if it did not fit to the RSpec.
Try something like this (untested by me):
context "the node definition using block of code" do
let(:node){
node = Node.new "arg1", "arg2", node_block
# more complex stuff here
node
}
context "checking the block is called" do
let(:node_block) {
double = double("node_block")
double.should_receive("some kind of arg").and_return("something")
# this will now cause a fail if it isn't called
double
}
it "should call the block" do
node.blah()
end
end
let(:node_block) {
# some real code
}
subject { node.blah() }
it { should == 2 }
# ...
end
So that's a very shaky piece of code (you'll have to fill in the gaps as you didn't give very much to go on, and let is obviously a lambda too, which could mean you've got to play around with it a bit) that uses let and a double to check it's called, and avoids using before, which is really for side effects not setting up variables for use in the specs.
#zetetic makes a very insightful comment that you're not testing behaviour here. I'm not against using rspec for doing more unit test style stuff (guidelines are made to be broken), but you might ask how later tests will pass when using a real block of code if that block isn't being called? In a way, I'm not even sure you need to check the block is called, but only you know.
Is it possible to detect when the value of a variable has changed using the lua debug library.
Something like A callback function which would give details like the function in which the value was changed, previous value, etc.
Is such a thing possible?
I read about hooks, but I'm not sure hooks can be set to variables.
If you don't mind using a debugger, then some debuggers allow you to set Watch expressions, which will be triggered when the condition in the expression is true. I'll show how this can be done in MobDebug (it is using lua debug library, but there is no direct way to detect a variable change as far as I know).
Let say we have a script start.lua like the one below and want to detect where foo gets value 2:
print("Start")
local foo = 0
for i = 1, 3 do
local function bar()
print("In bar")
end
foo = i
print("Loop")
bar()
end
print("End")
Download mobdebug.lua and make it available to your scripts (the simplest way is to put it into the folder with your scripts).
Start the server using lua -e "require('mobdebug').listen()" command.
Start the client using lua -e "require('mobdebug').loop()" command.
You will see the prompt in the server window: '>'. Type load start.lua to load the script.
Type step and then step again. You will see "Paused at file start.lua line 3".
Let's see what the value of foo is. Type eval foo and you should see 0.
Now we can set up our watch. Type setw foo == 2. You can specify any Lua expression after setw command; the execution of your script will be stopped when the condition is evaluated as true.
Continue execution of the script using "run" command.
The watch now fires, which will show you the message like: "Paused at file start.lua line 8 (watch expression 1: [foo == 2])". This means that the previous expression changed the value of foo to 2 and the execution is stopped at line 8. You can then inspect your script and the current values (you can use "eval" and "exec" commands to run any Lua code to be evaluated in your script environment) to find what triggered the change.
The benefit of this approach is that you are not limited to monitoring table values and can specify any expression. The main disadvantage is that your script runs under a debugger and the expression is evaluated after each step, which may get really slow.
You can do this to a certain extent in Lua by using metatables and keeping a "proxy" table, and using the __newindex function call to detect attempts to add a variable.
This is covered here in the Programming in Lua book under the section "Tracking Table Accesses":
http://www.lua.org/pil/13.4.4.html
See Also
http://www.gammon.com.au/forum/?id=10887