I have a Rakefile I'm using to generate HTML from markdown (and do some other stuff that's irrelevant to the question.
I'm generating files from my source, .feature files (in the FileList DOCUMENTS), into my output directory OUTPUT as HTML. I have an htmlfile method to assemble and write my HTML file.
I'm trying two alternative options here:
File tasks:
DOCUMENTS.each do |doc|
file doc.pathmap("#{OUTPUT}/%X.html") => doc do |t|
htmlfile t.name, RDiscount.new(F.read doc).to_html, t.name.pathmap('%n')
end
end
Synthesized file tasks with a rule:
rule '.html' => proc {|html| html.pathmap("%{#{OUTPUT}/,}X.feature")} do |t|
htmlfile t.name, RDiscount.new(F.read t.source).to_html, t.name.pathmap('%n')
end
My understanding was that the latter option would synthesize file tasks, and have the same net effect. However I find that if I choose it, it does not cope with incremental building, whereas the first option does.
If I build, then modify one file, then run rake --trace I get the following:
With synthesized tasks:
** Invoke output/Module/Feature.html (first_time, not_needed)
** Invoke output/Module (not_needed)
And with the explicit file tasks:
** Invoke output/Module/Feature.html (first_time)
** Invoke output/Module (not_needed)
** Invoke Module/Feature.feature (first_time, not_needed)
** Execute output/Module/Feature.html
This option is clearly checking the source file. I thought linking output and source was exactly what rule
(I believe it's most helpful to put the answer as an actual answer, rather than a comment. See https://meta.stackexchange.com/questions/68507/what-to-do-if-you-find-the-answer-to-your-own-question)
It turns out that if you have file outdoc => something elsewhere in your Rakefile, it will mess with synthesized tasks. Whereas if you have file tasks for those output documents it adds to the pre-requisites and works fine. This sort of makes sense; synthesized tasks don't really exist.
I also found out that rules only work to one level of inference ( http://onestepback.org/articles/buildingwithrake/rulelimitations.html) though that didn't turn out to be the answer.
Fix: rearrange pre-requisites of tasks, or use the explicit file tasks.
Related
We have a rails app, and we use webpack which takes multiple javascript files and outputs one single javascript file. It takes a long time to run, and I'd like to create a rake task for this. But being new to rake I need some help.
I'd like to use rake's build system so that I can get automatic checking of the time stamps between the input and output .js files. So that if any of the input files are newer than the output file it will execute webpack. Otherwise if the none of the input files are newer than the output file, than the task does nothing.
In MSBuild, this is a cakewalk and lightning fast. But in Ruby I'm kind of lost.
I'm guessing it might consist of writing file tasks, and looping through and making the one output file depend on the inputs. Or should I use a rule, like this?
outputfile = "~/foo.js"
inputfiles = Dir["~/**/*.js"]
rule outputfile => inputfiles do
bin/webpack bla bla bla
end
You can use Rake::FileList to achieve this. Something like this:
file "foo.js" => Rake::FileList["**/*.js"] do
...
end
And, I'm not sure whether rake allows to use ~ in paths, I believe a full path is required. Or just use a "#{Dir.home}/foo.js" rule.
Then call it using:
rake ~/foo.js
And when you have multiple outputs:
task :build => Rake::FileList["config1.xml", "config2.xml"] do
# all that stuff is run only when the FileList above is changed
touch 'foo1.js'
touch 'foo2.js'
sh "compile foo3.js"
sh "do-anything-else foo4.js"
end
Run it using:
rake build
I have a following problem. The chef recipe code snippets below does not behave equally although they look the same to me in terms of pure logic.
template "Create a file if not exists" do
path "#{site_docroot}/somefile.php"
source 'somefile.php.erb'
action :create_if_missing
end
VS.
if !File.exists? "#{site_docroot}/somefile.php"
template "Create a file if not exists" do
path "#{site_docroot}/somefile.php"
source 'somefile.php.erb'
action :create
end
end
Both should create a file if it does not yet exist.
But in context of a custom recipe in Amazon OpsWorks "setup" stage, the first sollution works as expected.
But the second sollution delivers a "false positive" exactly every second time I run the recipe.
The "if" statement delivers false but the file does not exist at the end.
So I would like to know if there is any reason for that in chef or/and ruby with nesting "template" resource inside "if" block. Does "template" ressource run some kind of asynchronous?
the short answer is that chef actually runs in 2 phases. You have a compile phase and an execution (or sometimes called convergence) phase.
What this means is depending on the presence of the file the template will get inserted or not in the recipe at compile time.
Further reading:
https://docs.chef.io/chef_client.html
https://serverfault.com/questions/604719/chef-recipe-order-of-execution-redux
So what happens in your case:
first there is not file. The chef recipe (via compile phase) decides there should be a file and creates the file in the converge phase.
on the 2nd run, there is a file and the chef recipe decides (via compile again) that there should not be a file (i.e. missing template). When the node converges in the 2nd phase chef removes the file as it's trying to bring the node to the desired state.
That explains the flipping back and forth that you a see (every other run)
I just started using chef and don't know much about ruby.
I have problems understanding the language-syntax used in recipes.
Say, I create a directory in a cookbook in recipes/default.rb like:
directory "/home/test/mydir" do
owner "test"
mode "0755"
action :create
recursive true
end
I assume this is part of a valid ruby script. What do lines like owner "test" mean? Is this a function call, a variable assignment or something else entirely?
Chef is written in Ruby and makes an extensive use of Ruby ability to design custom DSL. Almost every chef configuration file is written with a Ruby-based DSL.
This means that in order to use chef effectively you should be familiar with the basic of Ruby syntax including
Grammar
Data types (the main difference compared to other languages are Symbols)
Blocks
You don't need to know a lot about metaprogramming in Ruby.
The case of the code you posted is an excellent example of a Ruby based DSL. Let me explain it a little bit.
# Call the method directory passing the path and a block
# containing some code to be evaluated
directory "/home/test/mydir" do
# chown the directory to the test user
owner "test"
# set the permissions to 0555
mode "0755"
# create the directory if it does not exists
action :create
# equivalent of -p flag in the mkdir
recursive true
end
Blocks are a convenient way to specify a group of operations (in this case create, set permissions, etc) to be evaluated in a single context (in this case in the context of that path).
Let's break it down.
directory "/home/test/mydir" do
...
end
You are just calling a global method defined by Chef called directory, passing one argument "/home/test/mydir", and a block (everything between the do and end).
This block is probably excecuted in a special scope created by Chef in which all of the options (owner, mode, action, etc.) are method.
I'm attempting to recursively copy directories with Ruby's cp_r method in fileutils. However, it crashes in the (silly but out of my control) case where in the directory I copy there is a file and a symbolic link to that file. I get the following error:
/usr/lib/ruby/1.8/fileutils.rb:1223:in `symlink': File exists - real_file_name or symbolic_link_filename (Errno::EEXIST)
Now, I might be wrong, but it seems like something that symlink's :force option should take care of; however, cp_r cannot take a :force option, and I see no way to make it pass that option to its internal calls to symlink. Also, catching the EEXISTS error doesn't seem to be a solution since the run of cp_r would still be interrupted.
Is there a clean way to get around this problem?
You can use FileUtils.cp_r and pass it the option remove_destination: true to accomplish this.
I'm using rake to help compile Coffeescript for a Chrome extension I'm writing.
My Rakefile looks like this:
COFFEE = FileList['src/*.coffee']
JS = COFFEE.ext 'js'
directory 'extension'
rule '.js' => ['.coffee', 'extension'] do |t|
`coffee -c -o extension #{t.source}`
end
desc "Build the extension in the 'extension' directory"
task :build => ['extension', JS] do
cp File.join('src', 'manifest.json'), 'extension'
end
When I only have one .coffee file in my src directory, there's no problem. But as soon as I have more than one .coffee files it errors:
$ rake build
> rake aborted!
> Don't know how to build task 'src/app.js src/background.js'
>
> Tasks: TOP => build
> (See full trace by running task with --trace)
Is it possible to specify a FileList as a dependency? How else would I tell rake that I want all my Coffeescript files compiled durring the build task?
Rake’s dependency list is an Array of task names. When you use a FileList as one of its elements, you nest arrays – effectively, this:
task :build => ['extension', ['src/app.js', 'src/background.js']] do
Rake just uses the String representation of all passed dependency Array elements, which is why it complains about being unable to build a 'src/app.js src/background.js' task (note how this is one string).
Splatting your FileList (or flattening the dependency Array) will solve the issue, i.e.:
task :build => ['extension', *JS] do
or
task :build => ['extension', JS].flatten do
Try this:
files = Dir.entries('path/to/scripts').select { |f| f.include? '.coffee' }
files.each do |file_path|
`coffee -c -o extension #{file_path}`
end
So far in my search it looks like the only way to accomplish what I want is to either have a task which loops through my FileList and compiles each one explicitly (like in the answer from #nicooga). Or, I can loop through everything in the FileList and add it as a dependency to the build task.
I don't like either of these because rake has FileLists for getting groups of files, rules for defining how to handle kinds of files, and a nice syntax for defining dependencies, but apparently no way to combine all three of those together.
So, my solution for now is to go with the second option, adding each file as a dependency. The shortest way I've found to do this is to concat the FileList onto the dependency array. So now my build task looks like this:
task :build => ['extension'].concat(JS) do
cp File.join('src', 'manifest.json'), 'extension'
end
And thanks to the comment by #kopischke, this can even be shortened to ['extension' *JS] using the splat operator.
Late to the game, but a FileList is lazy, and that is useful, especially if you have lots of file matching.
Think of C or C++ where you potentially can have many that require dependencies.
Most of the answers here require the FileList to be expanded/evaluated. Since Rake is Ruby and task is where the last argument is basically a Hash, you will be evaluating the array.
One of the answers suggests ['extension', JS].flatten. This approach will search the directory and collect all the .js files. This approach is fine when you have a task that will always get executed/invoked. However, you don't want to evaluate the FileList and invoke its search if the task is will not get executed/invoked.
The best way to use a FileList is the following
fl = FileList['extension', "src/*.coffee"] do |c|
c.ext('js')
end
file "somefile" => fl do
#some stuff
end
Then if "somefile" doesn't exist, or the "somefile" task is not even invoked, the read of your Rakefile doesn't invoke the FileList expansion, saving a bunch of time.