I am trying to run the Kafka Consumer example found here:
http://www.rubydoc.info/gems/ruby-kafka/Kafka/Consumer
This is the program I am trying to run:
require "kafka"
kafka = Kafka.new(["kafka1:9092", "kafka2:9092"])
# Create a new Consumer instance in the group `my-group`:
consumer = kafka.consumer(group_id: "my-group")
# Subscribe to a Kafka topic:
consumer.subscribe("messages")
# Loop forever, reading in messages from all topics that have been
# subscribed to.
consumer.each_message do |message|
puts message.topic
puts message.partition
puts message.key
puts message.value
puts message.offset
end
But whenever I run this program I get the error:
example.rb:3:in '<main>': undefined method 'new' for Kafka:Module (NoMethodError)
I have seen this sort of error of Undefined method 'new' in a few questions relating to completely different programs but can't seem to figure out what is wrong as I haven't overwritten a kafka class or anything. I installed kafka on my machine using sudo gem install kafka-rb the version being kafka-rb-0.0.15.
Does anyone have any ideas why the problem is occurring and how to resolve this issue?
As was mentioned in the comments, you are using kafka-rb gem, which is no longer maintained. But if for some reason you want to continue using it, the code would probably look something like this:
require 'kafka'
consumer = Kafka::Consumer.new(topic: 'messages')
consumer.loop do |messages|
puts "Received"
puts messages
end
Related
I am running a simple Ruby program to produce and consume ActiveMQ Stomp messages. I keep on getting this error.
/root/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/stomp-1.3.4/lib/stomp/client.rb:132:in `join': AMQ339016: Error creating subscription f7b282396a9bef2da1ccd42d16dc511758234113 (Stomp::Error::BrokerException)
from /root/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/stomp-1.3.4/lib/stomp/client.rb:132:in `join'
from consumer.rb:10:in `<main>'
I made sure ActiveMQ is up and from I can tell it is not necessary to configure the xml config files since ActiveMQ is supposed to support Stomp out of the box. My program looks like this:
require 'rubygems'
require 'stomp'
client = Stomp::Client.open "stomp://localhost:61613"
client.subscribe "/queue/myqueue" do |message|
puts "received: #{message.body} on #{message.headers['destination']}"
end
client.join
client.close
puts "completed process"
The full example code can be found here:
http://arhipov.blogspot.com/2009/05/activemq-ruby-stomp-transport.html
I have this small ruby application, not Ruby on Rails - pure Ruby.
I've followed the instructions and I can queue stuff and see that everything is correctly queued using resque-web.
However, I have a problem to start a worker. The documentation indicates to run bin/resque work to launch a worker.
Doing so triggers the message -bash: bin/resque: No such file or directory
Everywhere on the Internet, people have the same problem, but for Rails app, not pure Ruby. The solution seems to include something in the rakefile, which I don't have.
How can I launch my worker? Thanks a lot!
The key to solving your problem is rake.
Resque includes three rake tasks. All you need is a rakefile that requires 'resque/tasks' to use them. When you ask Rake for its list of commands you'll see the three tasks that are included with Resque:
rake resque:failures:sort # Sort the 'failed' queue for the redis_multi_queue failure backend
rake resque:work # Start a Resque worker
rake resque:workers # Start multiple Resque workers
The one you're looking for (to start one worker) is resque:work. You tell it which queue to listen to using the QUEUE environment variable. So starting your worker would be something like:
QUEUE=your_queue_name rake resque:work.
Alternatively, you can listen to all queues using QUEUES=*.
EDIT:
Here's a more complete example. Create a file called rakefile:
require 'resque/tasks'
require 'resque'
class Worker
#queue = :default
def self.perform(my_arg)
puts my_arg
end
end
task :hello do
Resque.enqueue(Worker, 'hello')
end
Then in one terminal type TERM_CHILD=1 QUEUE=default rake resque:work. This will start the worker, watching the queue called default. It will print out any argument a job passes to its perform class method.
In a second terminal type rake hello. This will enqueue a job for the Worker class, passing the string hello as an argument (which will be passed to the perform method in the Worker class). It knows to push to the default queue by looking at the #queue property on Worker.
You'll see hello printed in the terminal where you started the worker.
This example doesn't do anything useful, and you wouldn't put all that in your rakefile, but I think it's a good starting point for you to start modifying it and build your own.
I am using the ruby sprinkle gem (using Capistrano as its deployment mechanism) to execute a command to start a simple ruby app on a remote Ubuntu server. Below is a small snippet of the ruby app code where I think the problem could lie.
...
require 'eventmachine'
...
fork { run }
def run
EM.run{
Signal.trap('INT') { #log.debug("trapped INT signal"); stop(true) }
Signal.trap('TERM'){ #log.debug("trapped TERM signal"); stop(true) }
begin
pulsate
#log.info "still ok here"
EM.add_periodic_timer 60 do # 1 minute
#log.info "doesn't get here"
pulsate
end
#log.info "doesn't get here"
rescue => exc
#never gets here
#log.error "Unable to add EM timer due to: #{exc}"
exit -1
end
}
end
def pulsate...
def stop...
etc
...
The strange thing is that it all runs without any problems when I ssh on to the server and run it there. However when using sprinkle/capistrano, as soon as the process hits the EM.add_periodic_timer it just disappears. No exception is thrown, no signals, no log output, it just seems to never get to the next line?
Also, I am using the latest version of the EventMachine gem: eventmachine (1.0.0.rc.4) and capistrano (2.12.0) (sprinkle is a red herring I think as it just falls back on capistrano)
Any ideas of why it could fail during the remote execution but not when executed in place on the server? Any ideas of things I can try to get more information?
Try setting the global EM error handler before calling EM.run. You can see the documentation at EM#error_handler which may give you a clue as to what's going on.
I was trying out this code (got from an online article here: http://www.randomhacks.net/articles/2009/05/08/chat-client-ruby-amqp-eventmachine-shoes)
require 'rubygems'
gem 'amqp'
require 'mq'
unless ARGV.length == 2
STDERR.puts "Usage: #{$0} "
exit 1
end
$channel, $nick = ARGV
AMQP.start(:host => 'localhost') do
$chat = MQ.topic('chat')
# Print any messages on our channel.
queue = MQ.queue($nick)
queue.bind('chat', :key => $channel)
queue.subscribe do |msg|
if msg.index("#{$nick}:") != 0
puts msg
end
end
# Forward console input to our channel.
module KeyboardInput
include EM::Protocols::LineText2
def receive_line data
$chat.publish("#{$nick}: #{data}",
:routing_key => $channel)
end
end
EM.open_keyboard(KeyboardInput)
end
But ended up the following error:
chat.rb:11:in `': uninitialized constant AMQP (NameError)
After that, I tried different example code with AMQP at my dev env but all shows me that error. So the problem is not in the code, the problem with my dev env. Can anybody point me out the issues with my dev env. Thanks in advance.
I have AMQP installed and integrated with Ruby (via the bunny gem). Maybe I can help?
Most likely the gem install failed to compile the amqp libs. Uninstall the gem and reinstall, taking a very close look at the messages produced. Possibly you're only missing some third-party libs.
Which platform are you on?
I'm writing a project at the moment in Ruby which makes use of the ActiveRecord gem for database interaction and I'm trying to log all the database activity using the ActiveRecord::Base.logger attribute with the following code
ActiveRecord::Base.logger = Logger.new(File.open('logs/database.log', 'a'))
This works fine for migrations etc (which for some reason seem to require that logging be enabled as it gives a NilClass error when it's disabled) but when I try to run the project which includes a threaded daemon calling the ActiveRecord object the script fails with the following error
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/logger.rb:504:in `write': closed stream (IOError)
Any ideas on how to solve this problem would be greatly appreciated. For the moment I've started to look through other code to see if people have other ways of implementing ActiveRecord logging in a more thread-safe manner
Thanks
Patrick
I ran into the same issue. You need to daemonize first, and then load the Rails environment.
the delayed_job have used daemons and activerecord,
before daemonize,get the files have opend,and then reopen in daemonize
#files_to_reopen = []
ObjectSpace.each_object(File) do |file|
#files_to_reopen << file unless file.closed?
end
Daemons.run_proc("xxxxx_name",:dir=>pid_file,:dir_mode=>:normal) do
Dir.chdir(Rails.root)
# Re-open file handles
#files_to_reopen.each do |file|
begin
file.reopen file.path
file.sync = true
rescue ::Exception
end
end
end
It turns out that for migrations to work the ActiveRecord::Base.logger variable cannot be nil, which explains the first half of the problem. I am as yet unable to fix the IOError though when a file is used instead of STDERR.