I am new to sidekiq, i just created sidekiq worker, when i run the sidekiq worker, it executes only the first rake task
I want to execute single rake with two different combinations of arguments
e.g
def perform
Rake::Task['test:fetch'].invoke("foo", "bar", 1, 1)
Rake::Task['test:fetch'].invoke("foo1", "bar1", 1, 1)
end
how to execute the above with interval of 5 minutes, should i run the rake 2 times or any other method to pass two diffrent arguments?
That's how Rake works. Read the rake documentation about tasks and invoke()
You need to re-enable rake tasks after calling the invoke method, or use execute to avoid re-enabling.
Using invoke, your code might look like this:
def perform
Rake::Task['test:fetch'].invoke("foo", "bar", 1, 1)
Rake::Task['test:fetch'].reenable
Rake::Task['test:fetch'].invoke("foo1", "bar1", 1, 1)
end
Using execute, your code might look like this:
def perform
task_args = {:arg1 => "foo", :arg2 => "bar", :arg3 => 1, :arg4 => 1 }
Rake::Task['test:fetch'].execute(Rake::TaskArguments.new(task_args.keys, task_args.values))
task_args = {:arg1 => "foo1", :arg2 => "bar1", :arg3 => 1, :arg4 => 1 }
Rake::Task['test:fetch'].execute(Rake::TaskArguments.new(task_args.keys, task_args.values)
end
For some more explanation see here: Why is Rake not able to invoke multiple tasks consecutively?
Related
I have an web based app that I built using Sinatra. From recent I was needing to collect data at regular intervals and store them in a database. For this I was told that I could use Resque and Clockwork gems in combine.
Every hour or so I need to do nearly 15 calculations based on the database and store the results in a database.
So this is the approach I took. I decided to make 15 classes that have the perform method ( the exact file I used for testing is below ). Then to do some thing similar to Resque.enqueue( GraphData ) for all 15 classes.
class GraphData
#queue = :graph_data
def self.init()
end
def self.perform()
File.open( '/home/ziyan/Desktop/resque.txt', 'a' ) { | file | file.write( "Resqueu - performed - #{Time.now}\n" ) }
end
end
To trigger the operation for testing purposes, I created a rake task.
desc "Start Resque Workers for Queue" # {{{
task :graph_data do |t|
ENV["QUEUE"] = "*"
ENV["VVERBOSE"] = "1"
ENV["INTERVAL"] = "5"
Resque.enqueue( GraphData )
#resque = Resque.new
#resque << AdminWorker.new
end # }}}
As you see, in the GraphData class, under self.perform method I am writing to a file.
My problem is it doesn't! Have I done some thing wrong?
rake graph_data will show no output. The web interface will show a job in the Queue.
Additional Information
I added another Rake task for running the web interface.
desc "Start Resque Web Frontend" # {{{
task :resque_web_frontend do |t|
sh "resque-web -p 8282"
puts "Resque Web Frontend is running on http://localhost:8282"
end # }}
Over their, some thing interesting is seen. As I run the rake task, under stats the pending value increases but not the processed.
Under queues -> graph_data I see some thing like this.
Class Args
GraphData []
GraphData []
GraphData []
What I ended up with:
desc "Start Resque Workers for Queue" # {{{
task :graph_data do |t|
ENV["QUEUE"] = "*"
ENV["VVERBOSE"] = "1"
ENV["INTERVAL"] = "5"
Rake::Task[ "resque:work" ].invoke
Resque.enqueue( GraphData )
#resque = Resque.new
#resque << AdminWorker.new
end # }}}
Is this in development or production? From what you describe, it looks like you are not lauching the Resque process. How are you running your app? I'm more familiar with using Resque with Rails, but you should have to run something like:
rake resque:work QUEUE='*'
To have the resque worker start.
I'm writing a Rake task and I want to pass an array as one of the arguments. Here's how I currently have it.
task :change_statuses, :ids, :current_status, :new_status do |task, args|
puts "args were #{args.inspect}"
end
I've tried running the task these ways:
# First argument as array
rake "change_statuses[[1,2,3], active, inactive]"
# First argument as string
rake "utility:change_account_statuses['1,2,3', foo, bar]"
My expected output would be:
args were {:ids=> [1,2,3], :current_status=> 2 , :new_status=> 3}
However, my actual output in each case is:
args were {:ids=>"[1", :current_status=>"2", :new_status=>"3]"}
How can I pass an array to a Rake task?
One of the soulutions is to avoid , symbol in the string, so your command line would look like:
$ rake change_statuses['1 2 3', foo, bar]
Then you can simply split the IDs:
# Rakefile
task :change_statuses, :ids, :current_status, :new_status do |task, args|
ids = args[:ids].split ' '
puts "args were #{args.inspect}"
puts "ids were #{ids.inspect}"
end
Or you can parse the ids value to get your expected output:
args[:ids] = args[:ids].split(' ').map{ |s| s.to_i }
I'm using rake 0.8.7
You can also try to change the approach a little, and try to fetch ids with the arg.extras method.
$ rake change_statuses[foo, bar, 1, 2, 3]
task :change_statuses, [:foo, :bar] do |_task, args|
puts args[:foo] # => foo
puts args[:bar] # => bar
puts args.extras # => ["1","2","3"]
end
You can find more information in this article -> https://blog.stevenocchipinti.com/2013/10/18/rake-task-with-an-arbitrary-number-of-arguments/
Another way to achieve this and also win the ability to pass a hash
namespace :stackoverflow do
desc 'How to pass an array and also a hash at the same time 8-D'
task :awesome_task, [:attributes] do |_task, args|
options = Rack::Utils.parse_nested_query(args[:attributes])
puts options
end
end
And just call the rake task like this:
bundle exec rake "stackoverflow:awesome_task[what_a_great_key[]=I know&what_a_great_key[]=Me too\!&i_am_a_hash[and_i_am_your_key]=what_an_idiot]"
And that will print
{"what_a_great_key"=>["I know", "Me too!"], "i_am_a_hash"=>{"and_i_am_your_key"=>"what_an_idiot"}}
rake "change_statuses[1 2 3, foo, bar]"
this works for me, you should not quota 1 2 3 with '
task :import_course, [:cids, :title] => :environment do |t, args|
puts args[:cids]
end
if you quota 1 2 3 as the correct answer, args[:cids] will be "'1 2 3'", which ' char included, you have to trim ' char, but if you use my answer, args[:cids] will be "1 2 3", which is more easy to use, you just need args[:cids].split(" ") to get [1, 2, 3]
You can do this as an alternative:
task :my_task, [:values] do |_task, args|
values = args.values.split(',')
puts values
end
and run the task using
rake my_task"[1\,2\,3]"
Lets say I have a rake task
task :aa, [:pp1, :pp2, :pp3] do |t, args|
args.with_defaults(:pp2 => "fff")
puts args.pp1
puts args.pp2
puts args.pp3
end
How do I pass to rake only 1st and 3rd param, taking the second one from defaults? Object of what type is a thing in square brackets in "rake aa[1,2,3]" call?
You could define an additional task (in my example aa2) with the parameters (and sequence) you want. The parameter are forwarded to the prerequisites, if they have the same names.
require 'rake'
task :aa, [:pp1, :pp2, :pp3] do |t, args|
args.with_defaults(:pp2 => "fff")
puts args.pp1
puts args.pp2
puts args.pp3
end
task :aa2, [:pp1, :pp3] => :aa
Call it with:
rake aa2[1,3]
So I ended up using old rake syntax for passing params
rake aa pp1=1 pp3=vvv
Thus, pp1 and pp3 are available with ENV. Then I replace defaults in args, if I find a param with the same name, which is passed via CLI.
args.to_hash.each_key { |k| args.to_hash[k] = ENV[k.to_s] if ENV.has_key?(k.to_s) }
After that, I can get any param with nice args.param syntax.
I have some jobs that run longer than 119 seconds during peak times and I keep getting the stalker error below when it does. I am using stalker, beanstalkd and clockwork for my background processing. Where/how can I change the timeout settings?
Exception Stalker::JobTimeout -> find.products hit 119s timeout
/home/blake/.rvm/gems/ruby-1.9.2-p180/gems/stalker-0.9.0/lib/stalker.rb:86:in
Stalker.enqueue takes 3 options: job name string, arguments hash, options hash.
The options hash can take the key/value pair :ttr => timeout_in_seconds, which, as you probably can guess, sets the number of seconds Stalker will allow before it raises the JobTimeout Exception.
E.g. if you have some crazy calculation you expect to take an hour:
#job code
job 'crazy.calculation' do |args|
args['x'].to_i + args['y'].to_i + args['z'].to_i
end
#queuing code
Stalker.enqueue 'crazy.calculation', {:x => 1, :y => 2, :z => 3}, {:ttr => 3600}
The same code if your calcuation doesn't take arguments
#job code
job 'crazy.calculation' do |args|
1 + 2 + 3
end
#queuing code
Stalker.enqueue 'crazy.calculation', {}, {:ttr => 3600}
Don't confused by the interchangeability of symbols and strings in the enqueue args hash and the job args hash, in the options hash :ttr MUST BE A SYMBOL.
I have been playing around with Rake and Albacore, to see if I can replace our existing MSBuild script that deploys software with something that isn't XML. I have a task that will change the debug value inside a web.config to false. The task takes the directory of the web.config as an argument, but I can't quite figure out the syntax needed to supply this argument in the default task.
require 'albacore'
require 'nokogiri'
deployment_path = 'c:/test-mars-deploy'
task :default => [ :build, :publish, :update_web_config['c:/test-mars-deploy'] ]
task :update_web_config, :deploy_path do |t, args|
deployment_path = #{args[:deploy_path]}
web_config_path = File.join deployment_path, 'Web.config'
File.open(web_config_path, 'r+') do |f|
doc = Nokogiri::XML(f)
puts 'finding attribute'
attribute = doc.xpath('/configuration/system.web/compilation')
attribute.attr('debug', 'false')
puts attribute.to_xml
end
File.delete(web_config_path)
File.new(web_config_path, 'w') do |f|
f.write(doc.to_s)
end
end
The task dependency notation doesn't support passing arguments. It only takes names or symbols referring to task names.
task :default => [ :build, :publish, :update_web_config['c:/test-mars-deploy'] ]
You'd need to do something like this.
task :default => [ :build, :publish ] do
Rake::Task[:update_web_config].invoke 'c:/test-mars-deploy'
end
Remember, though, invoke will only work once per task, even with different arguments. It's the real dependency chain invoke. But, it will call all dependent tasks. You can use execute if you need multiple executions, but that won't call dependent tasks.
Rake::Task[:update_web_config].invoke 'c:/test-mars-deploy'
Rake::Task[:update_web_config].execute 'c:/test-mars-deploy2'
Rake::Task[:update_web_config].execute 'c:/test-mars-deploy3'
In general, I don't recommend either of these approaches. Calling invoke or execute seems to me to indicate a poorly structured task. You simply don't have this problem if you don't prematurely parameterize.
web_config = 'c:/test-mars-deploy/Web.config'
task :update_web_config do
File.open(web_config, 'r+') do |file|
# ...
end
end
If you must parameterize, provide an array or FileList and generate the tasks per item.
web_configs = FileList['c:/test-*/Web.config']
web_configs.each do |config|
task config do
File.open(config, 'r+') do |file|
# ...
end
end
end
task :update_all_web_configs => web_configs
Better yet, I published a config update task that does all of this mess for you! Provide a FileList to update and a hash of xpath queries => replacements.
appconfig :update_web_configs do |x|
x.files = FileList['c:/test-*/Web.config']
x.replacements = {
"/configuration/system.web/compilation/#debug" => 'False' }
end
I think you might have to use the old style parameter passing, eg:
nicholas#hal:/tmp$ cat Rakefile
task :default => :all
deploy_path = ENV['deploy_path'] || "c:/some_path"
task :all do |t, args|
puts deploy_path.inspect
end
And invoke with:
nicholas#hal:/tmp$ rake
(in /tmp)
"c:/some_path"
Or, to override the path:
nicholas#hal:/tmp$ rake deploy_path=c:/other_path
(in /tmp)
"c:/other_path"
basically, you name your args as extra symbols after the name of the task. an args param will get passed into the block that responds to the name of your args, and you can invoke the task passing the args in square brackets ([])
ree-1.8.7-2010.02#rails3 matt#Zion:~/setup$ cat lib/tasks/blah.rake
task :blah, :n do |t, args|
puts args.n
end
ree-1.8.7-2010.02#rails3 matt#Zion:~/setup$ rake blah[20]
(in /home/matt/setup)
20
The task dependency notation does in fact support passing arguments. For example, say "version" is your argument:
task :default, [:version] => [:build]
task :build, :version do |t,args|
version = args[:version]
puts version ? "version is #{version}" : "no version passed"
end
Then you can call it like so:
$ rake
no version passed
or
$ rake default[3.2.1]
version is 3.2.1
or
$ rake build[3.2.1]
version is 3.2.1
However, I have not found a way to avoid specifying the task name (default or build) while passing in arguments. Would love to hear if anyone knows of a way.