How do efficiently run multiple nested scripts from within a ruby script - ruby

Question

Update
I added an answer below that I think makes sense. Please feel free to answer or comment if you think I have this wrong.
I have an array of two strings, for example. ["report_a", "report_b"]
I need to generate a database from each of these report names, then once the database is created I need to generate two wave files using the database as a source.
The two databases can be generated independently of each other, and then the two wave files can be generated independently of each other from the same database.
So in the end I need to do something like this
Generate db_a from report_a
Generate wave_1a from db_a
Generate wave_1b from db_a
Generate db_b from report_b
Generate wave_2a from db_b
Generate wave_2b from db_b
I would like to generate db_a and db_b in parallel and then after a db is created generate corresponding wave files a and b in parallel.
Here is the code I am using, but I feel generating too many processes and running the same commands multiple times.
What's the correct way to maximize what gets done in parallel without having the some work done multiple times?
report_types.each do |report_type|
log.puts "starting build for #{report_type} at #{Time.now}"
Process.fork do
outfile_prefix = "#{cell}.#{report_type}"
log.puts "starting db generation for #{report_type} at #{Time.now}"
log.puts `generate_db.rb #{report_base_name}.#{report_type}.gz #{outfile_prefix}`
log.puts "finished db generation for #{report_type} at #{Time.now}"
Process.fork do
log.puts "starting generation of grf file for #{report_type} at #{Time.now}"
log.puts `generate_waves.rb #{outfile_prefix}.sqlite3.gz #{outfile_prefix}.grf`
log.puts "finished generation of grf file for #{report_type} at #{Time.now}"
end
Process.fork do
log.puts "starting generation of trn file for #{report_type} at #{Time.now}"
log.puts `generate_waves.rb #{outfile_prefix}.sqlite3.gz #{outfile_prefix}.trn`
log.puts "finished generation of trn file for #{report_type} at #{Time.now}"
end
end
log.puts "finished build for #{report_type} at #{Time.now}"
end

Answer 1

I now think this is doing what I want.
I based my original post on the observations of what I saw in the log file I created. For example I observed 11 copies of the line "starting build for ...", in the log. I think this is because I end up with 11 processes. The original plus 10 clones.
1 (original) + 2 (one clone for each report) + 2x2 (each report clone gets a clone for grf wave) + 2x2 (again each report cone gets cloned again for the trn wave ).
So in the end we end up with 11 processes = 1 + 2 + 2x2 + 2x2.
In each process there is open un-written buffer containing the string "starting build for ..". Eventually all these processes end and the buffers are flushed creating the 11 copies of the string in the output.
I updated my code to the following, adding a flush before each Process.fork statement, and now I get the expected number of log entires for each statement. For example I now observe only 2 copies of "starting build for ..." in the log.
report_types.each do |report_type|
log.puts "starting build for #{report_type} at #{Time.now}"
log.flush
Process.fork do
outfile_prefix = "#{cell}.#{report_type}"
log.puts "starting db generation for #{report_type} at #{Time.now}"
log.puts `generate_db.rb #{report_base_name}.#{report_type}.gz #{outfile_prefix}`
log.puts "finished db generation for #{report_type} at #{Time.now}"
log.flush
Process.fork do
log.puts "starting generation of grf file for #{report_type} at #{Time.now}"
log.puts `generate_waves.rb #{outfile_prefix}.sqlite3.gz #{outfile_prefix}.grf`
log.puts "finished generation of grf file for #{report_type} at #{Time.now}"
end
log.flush
Process.fork do
log.puts "starting generation of trn file for #{report_type} at #{Time.now}"
log.puts `generate_waves.rb #{outfile_prefix}.sqlite3.gz #{outfile_prefix}.trn`
log.puts "finished generation of trn file for #{report_type} at #{Time.now}"
end
end
log.puts "finished build for #{report_type} at #{Time.now}"
end