I'm using rake to help compile Coffeescript for a Chrome extension I'm writing.
My Rakefile looks like this:
COFFEE = FileList['src/*.coffee']
JS = COFFEE.ext 'js'
directory 'extension'
rule '.js' => ['.coffee', 'extension'] do |t|
`coffee -c -o extension #{t.source}`
end
desc "Build the extension in the 'extension' directory"
task :build => ['extension', JS] do
cp File.join('src', 'manifest.json'), 'extension'
end
When I only have one .coffee file in my src directory, there's no problem. But as soon as I have more than one .coffee files it errors:
$ rake build
> rake aborted!
> Don't know how to build task 'src/app.js src/background.js'
>
> Tasks: TOP => build
> (See full trace by running task with --trace)
Is it possible to specify a FileList as a dependency? How else would I tell rake that I want all my Coffeescript files compiled durring the build task?
Rake’s dependency list is an Array of task names. When you use a FileList as one of its elements, you nest arrays – effectively, this:
task :build => ['extension', ['src/app.js', 'src/background.js']] do
Rake just uses the String representation of all passed dependency Array elements, which is why it complains about being unable to build a 'src/app.js src/background.js' task (note how this is one string).
Splatting your FileList (or flattening the dependency Array) will solve the issue, i.e.:
task :build => ['extension', *JS] do
or
task :build => ['extension', JS].flatten do
Try this:
files = Dir.entries('path/to/scripts').select { |f| f.include? '.coffee' }
files.each do |file_path|
`coffee -c -o extension #{file_path}`
end
So far in my search it looks like the only way to accomplish what I want is to either have a task which loops through my FileList and compiles each one explicitly (like in the answer from #nicooga). Or, I can loop through everything in the FileList and add it as a dependency to the build task.
I don't like either of these because rake has FileLists for getting groups of files, rules for defining how to handle kinds of files, and a nice syntax for defining dependencies, but apparently no way to combine all three of those together.
So, my solution for now is to go with the second option, adding each file as a dependency. The shortest way I've found to do this is to concat the FileList onto the dependency array. So now my build task looks like this:
task :build => ['extension'].concat(JS) do
cp File.join('src', 'manifest.json'), 'extension'
end
And thanks to the comment by #kopischke, this can even be shortened to ['extension' *JS] using the splat operator.
Late to the game, but a FileList is lazy, and that is useful, especially if you have lots of file matching.
Think of C or C++ where you potentially can have many that require dependencies.
Most of the answers here require the FileList to be expanded/evaluated. Since Rake is Ruby and task is where the last argument is basically a Hash, you will be evaluating the array.
One of the answers suggests ['extension', JS].flatten. This approach will search the directory and collect all the .js files. This approach is fine when you have a task that will always get executed/invoked. However, you don't want to evaluate the FileList and invoke its search if the task is will not get executed/invoked.
The best way to use a FileList is the following
fl = FileList['extension', "src/*.coffee"] do |c|
c.ext('js')
end
file "somefile" => fl do
#some stuff
end
Then if "somefile" doesn't exist, or the "somefile" task is not even invoked, the read of your Rakefile doesn't invoke the FileList expansion, saving a bunch of time.
Related
We have a rails app, and we use webpack which takes multiple javascript files and outputs one single javascript file. It takes a long time to run, and I'd like to create a rake task for this. But being new to rake I need some help.
I'd like to use rake's build system so that I can get automatic checking of the time stamps between the input and output .js files. So that if any of the input files are newer than the output file it will execute webpack. Otherwise if the none of the input files are newer than the output file, than the task does nothing.
In MSBuild, this is a cakewalk and lightning fast. But in Ruby I'm kind of lost.
I'm guessing it might consist of writing file tasks, and looping through and making the one output file depend on the inputs. Or should I use a rule, like this?
outputfile = "~/foo.js"
inputfiles = Dir["~/**/*.js"]
rule outputfile => inputfiles do
bin/webpack bla bla bla
end
You can use Rake::FileList to achieve this. Something like this:
file "foo.js" => Rake::FileList["**/*.js"] do
...
end
And, I'm not sure whether rake allows to use ~ in paths, I believe a full path is required. Or just use a "#{Dir.home}/foo.js" rule.
Then call it using:
rake ~/foo.js
And when you have multiple outputs:
task :build => Rake::FileList["config1.xml", "config2.xml"] do
# all that stuff is run only when the FileList above is changed
touch 'foo1.js'
touch 'foo2.js'
sh "compile foo3.js"
sh "do-anything-else foo4.js"
end
Run it using:
rake build
I've got a helper class that scans my entire project directory and collects a list of source files and the corresponding (target) object files. The dependencies on the compile task is defined after scanning the source directory as shown below.
CLEAN.include(FileList[obj_dir + '**/*.o'])
CLOBBER.include(FileList[exe_dir + '**/*.exe'])
$proj = DirectoryParser.new(src_dir)
$proj.source_files.each do |source_file|
file source_file.obj_file do
sh "gcc -c ..."
end
end
$proj.obj_files.each do |obj_file|
task :compile => obj_file
end
task :compile do
end
Since $proj is global, the DirectoryParser.new() is invoked when any of the tasks are called including clean and clobber. This makes the clean and clobber tasks slow and that is not desirable.
To get around the problem I moved all the generation of File dependencies into the default task. This makes my clean and clobber tasks fast, however, I can't call my compile or link tasks independently now.
CLEAN.include(FileList[obj_dir + '**/*.o'])
CLOBBER.include(FileList[exe_dir + '**/*.exe'])
task :compile => $proj.source_files do # Throws error!
end
task :default => do
$proj = DirectoryParser.new(src_dir)
$proj.source_files.each do |source_file|
file source_file.obj_file do
sh "gcc -c ..."
end
end
$proj.obj_files.each do |obj_file|
task :compile => obj_file
end
... compile
... link
... execute
end
How do I get around this problem? I am sure someone has previously encountered a similar problem. I'd appreciate any help.
You could try a two step approach.
Create a new task generate_dependencies.
This task builds a (static) rake file with your dependencies and actions.
This generated rakefile can be loaded in your rake file.
Some sample code (untested):
GENERATED = 'generated_dependencies.rb'
task :generate_dependencies do
$proj = DirectoryParser.new(src_dir)
File.open(GENERATED, 'w') do |f|
$proj.source_files.each do |source_file|
f << <<-code
file #{source_file.obj_file} do
sh "gcc -c " #etc.
end
code
end
$proj.obj_files.each do |obj_file|
f << "task :compile => #{obj_file}"
end
#~ ... compile
#~ ... link
#~ ... execute
end
end
require GENERATED
Now you have two steps:
create an empty 'generated_dependencies.rb' (so you get no error when you call the script the first time)
call rake generate_dependencies
Check the generated file - if it's not good, change the generator ;)
call rake compile or rake link (or rake if you want to use the default task) ... - the dependencies are defined in the generated file.
When something changes (new files), continue from step 2.
If the structure stays the same (no new files, only code changes) you need only step 4.
I managed to get around this problem elegantly by using the Singleton design pattern and moving away from using Rake file/task dependencies completely. DirectoryParser is now a singleton class (by mixing in Ruby's built-in 'singleton' library)
CLEAN.include(FileList[obj_dir + '**/*.o'])
CLOBBER.include(FileList[exe_dir + '**/*.exe'])
task :compile do
$proj = DirectoryParser.instance
$proj.source_files.each do |source_file|
sh "gcc -c ..." unless uptodate?(obj_file, source_file)
end
end
task :link do
$proj = DirectoryParser.instance
...
end
Now my clean/clobber tasks are fast and I can still call compile/link tasks independently.
I have a Rakefile I'm using to generate HTML from markdown (and do some other stuff that's irrelevant to the question.
I'm generating files from my source, .feature files (in the FileList DOCUMENTS), into my output directory OUTPUT as HTML. I have an htmlfile method to assemble and write my HTML file.
I'm trying two alternative options here:
File tasks:
DOCUMENTS.each do |doc|
file doc.pathmap("#{OUTPUT}/%X.html") => doc do |t|
htmlfile t.name, RDiscount.new(F.read doc).to_html, t.name.pathmap('%n')
end
end
Synthesized file tasks with a rule:
rule '.html' => proc {|html| html.pathmap("%{#{OUTPUT}/,}X.feature")} do |t|
htmlfile t.name, RDiscount.new(F.read t.source).to_html, t.name.pathmap('%n')
end
My understanding was that the latter option would synthesize file tasks, and have the same net effect. However I find that if I choose it, it does not cope with incremental building, whereas the first option does.
If I build, then modify one file, then run rake --trace I get the following:
With synthesized tasks:
** Invoke output/Module/Feature.html (first_time, not_needed)
** Invoke output/Module (not_needed)
And with the explicit file tasks:
** Invoke output/Module/Feature.html (first_time)
** Invoke output/Module (not_needed)
** Invoke Module/Feature.feature (first_time, not_needed)
** Execute output/Module/Feature.html
This option is clearly checking the source file. I thought linking output and source was exactly what rule
(I believe it's most helpful to put the answer as an actual answer, rather than a comment. See https://meta.stackexchange.com/questions/68507/what-to-do-if-you-find-the-answer-to-your-own-question)
It turns out that if you have file outdoc => something elsewhere in your Rakefile, it will mess with synthesized tasks. Whereas if you have file tasks for those output documents it adds to the pre-requisites and works fine. This sort of makes sense; synthesized tasks don't really exist.
I also found out that rules only work to one level of inference ( http://onestepback.org/articles/buildingwithrake/rulelimitations.html) though that didn't turn out to be the answer.
Fix: rearrange pre-requisites of tasks, or use the explicit file tasks.
Let's say I have a target who needs to compile some files. That target has another target as a prerequisite, one that obtains the files.
Let's say this:
task :obtain do
# obtain files from somewhere
end
task :compile => :obtain do
# do compilation
end
Let's say that the :obtain target doesn't always places the files in the same folder. How would I pass :compile the path that :obtain found? Environment variables?
Using ENV['something'] is in my opinion preferable, because if you do it this way (as opposed to $global or #instance variables) you can treat those as task arguments, and use the sub task from commandline easily.
On the other hand if you keep your code in separate classes / modules / methods, you will not even have to deal with those sorts of hacks + your code will be more testable.
One way would be to store it in a global variable:
task :obtain do
$obtained_dir = "/tmp/obtained"
end
task :compile => :obtain do
puts "compiling files in #{$obtained_dir}"
end
Instance variables (i.e. #obtained_dir) should also work.
Another way would be to pull the "obtain" code into a method, as follows:
task :obtain do
obtain_files
end
task :compile do
obtained_dir = obtain_files
puts "compiling files in #{obtained_dir}"
end
def obtain_files
#obtain files from somewhere
"/tmp/obtained_files"
end
I have a Rakefile that compiles the project in two ways, according to the global variable $build_type, which can be :debug or :release (the results go in separate directories):
task :build => [:some_other_tasks] do
end
I wish to create a task that compiles the project with both configurations in turn, something like this:
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
# call task :build with all the tasks it depends on (?)
end
end
Is there a way to call a task as if it were a method? Or how can I achieve anything similar?
If you need the task to behave as a method, how about using an actual method?
task :build => [:some_other_tasks] do
build
end
task :build_all do
[:debug, :release].each { |t| build t }
end
def build(type = :debug)
# ...
end
If you'd rather stick to rake's idioms, here are your possibilities, compiled from past answers:
This always executes the task, but it doesn't execute its dependencies:
Rake::Task["build"].execute
This one executes the dependencies, but it only executes the task if
it has not already been invoked:
Rake::Task["build"].invoke
This first resets the task's already_invoked state, allowing the task to
then be executed again, dependencies and all:
Rake::Task["build"].reenable
Rake::Task["build"].invoke
Note that dependencies already invoked are not automatically re-executed unless they are re-enabled. In Rake >= 10.3.2, you can use the following to re-enable those as well:
Rake::Task["build"].all_prerequisite_tasks.each(&:reenable)
for example:
Rake::Task["db:migrate"].invoke
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
Rake::Task["build"].reenable
Rake::Task["build"].invoke
end
end
That should sort you out, just needed the same thing myself.
task :invoke_another_task do
# some code
Rake::Task["another:task"].invoke
end
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
Rake::Task["build"].execute
end
end
If you want each task to run regardless of any failures, you can do something like:
task :build_all do
[:debug, :release].each do |t|
ts = 0
begin
Rake::Task["build"].invoke(t)
rescue
ts = 1
next
ensure
Rake::Task["build"].reenable # If you need to reenable
end
return ts # Return exit code 1 if any failed, 0 if all success
end
end
I would suggest not to create general debug and release tasks if the project is really something that gets compiled and so results in files. You should go with file-tasks which is quite doable in your example, as you state, that your output goes into different directories.
Say your project just compiles a test.c file to out/debug/test.out and out/release/test.out with gcc you could setup your project like this:
WAYS = ['debug', 'release']
FLAGS = {}
FLAGS['debug'] = '-g'
FLAGS['release'] = '-O'
def out_dir(way)
File.join('out', way)
end
def out_file(way)
File.join(out_dir(way), 'test.out')
end
WAYS.each do |way|
desc "create output directory for #{way}"
directory out_dir(way)
desc "build in the #{way}-way"
file out_file(way) => [out_dir(way), 'test.c'] do |t|
sh "gcc #{FLAGS[way]} -c test.c -o #{t.name}"
end
end
desc 'build all ways'
task :all => WAYS.map{|way|out_file(way)}
task :default => [:all]
This setup can be used like:
rake all # (builds debug and release)
rake debug # (builds only debug)
rake release # (builds only release)
This does a little more as asked for, but shows my points:
output directories are created, as necessary.
the files are only recompiled if needed (this example is only correct for the simplest of test.c files).
you have all tasks readily at hand if you want to trigger the release build or the debug build.
this example includes a way to also define small differences between debug and release-builds.
no need to reenable a build-task that is parametrized with a global variable, because now the different builds have different tasks. the codereuse of the build-task is done by reusing the code to define the build-tasks. see how the loop does not execute the same task twice, but instead created tasks, that can later be triggered (either by the all-task or be choosing one of them on the rake commandline).