Some background: Gerrit exposes an event stream through SSH. It's a cute trick, but I need to convert those events into AMQP messages. I've tried to do this with ruby-amqp and Net::SSH but, well, it doesn't seem as if the AMQP sub-component is even being run at all.
I'm fairly new to EventMachine. Can someone point out what I am doing incorrectly? The answer to "Multiple servers in a single EventMachine reactor) didn't seem applicable. The program, also available in a gist for easier access, is:
#!/usr/bin/env ruby
require 'rubygems'
require 'optparse'
require 'net/ssh'
require 'json'
require 'yaml'
require 'amqp'
require 'logger'
trap(:INT) { puts; exit }
options = {
:logs => 'kili.log',
:amqp => {
:host => 'localhost',
:port => '5672',
},
:ssh => {
:host => 'localhost',
:port => '22',
:user => 'nobody',
:keys => '~/.ssh/id_rsa',
}
}
optparse = OptionParser.new do|opts|
opts.banner = "Usage: kili [options]"
opts.on( '--amqp_host HOST', 'The AMQP host kili will connect to.') do |a|
options[:amqp][:host] = a
end
opts.on( '--amqp_port PORT', 'The port for the AMQP host.') do |ap|
options[:amqp][:port] = ap
end
opts.on( '--ssh_host HOST', 'The SSH host kili will connect to.') do |s|
options[:ssh][:host] = s
end
opts.on( '--ssh_port PORT', 'The SSH port kili will connect on.') do |sp|
options[:ssh][:port] = sp
end
opts.on( '--ssh_keys KEYS', 'Comma delimeted SSH keys for user.') do |sk|
options[:ssh][:keys] = sk
end
opts.on( '--ssh_user USER', 'SSH user for host.') do |su|
options[:ssh][:user] = su
end
opts.on( '-l', '--log LOG', 'The log location of Kili') do |log|
options[:logs] = log
end
opts.on( '-h', '--help', 'Display this screen' ) do
puts opts
exit
end
end
optparse.parse!
log = Logger.new(options[:logs])
log.level = Logger::INFO
amqp = options[:amqp]
sshd = options[:ssh]
queue= EM::Queue.new
EventMachine.run do
AMQP.connect(:host => amqp[:host], :port => amqp[:port]) do |connection|
log.info "Connected to AMQP at #{amqp[:host]}:#{amqp[:port]}"
channel = AMQP::Channel.new(connection)
exchange = channel.topic("traut", :auto_delete => true)
queue.pop do |msg|
log.info("Pulled #{msg} out of queue.")
exchange.publish(msg[:data], :routing_key => msg[:route]) do
log.info("On route #{msg[:route]} published:\n#{msg[:data]}")
end
end
end
Net::SSH.start(sshd[:host], sshd[:user],
:port => sshd[:port], :keys => sshd[:keys].split(',')) do |ssh|
log.info "SSH connection to #{sshd[:host]}:#{sshd[:port]} as #{sshd[:user]} made."
channel = ssh.open_channel do |ch|
ch.exec "gerrit stream-events" do |ch, success|
abort "could not stream gerrit events" unless success
# "on_data" is called when the process writes something to
# stdout
ch.on_data do |c, data|
json = JSON.parse(data)
if json['type'] == 'change-merged'
project = json['change']['project']
route = "com.carepilot.event.code.review.#{project}"
msg = {:data => data, :route => route}
queue.push(msg)
log.info("Pushed #{msg} into queue.")
else
log.info("Ignoring event of type #{json['type']}")
end
end
# "on_extended_data" is called when the process writes
# something to stderr
ch.on_extended_data do |c, type, data|
log.error(data)
end
ch.on_close { log.info('Connection closed') }
end
end
end
end
Net::SSH is not asynchronous, so your EventMachine.run() is never reaching the end of the block, thus never resuming the reactor thread. This causes the AMQP code to never start. I would suggest running your SSH code within another thread.
If you go back to EventMachine, give em-ssh https://github.com/simulacre/em-ssh a try.
Related
I connect using ruby to the remote, pass the command, but channel.send_data ("passwd\n") dont work , I cant catch in than the reason. The password doesn't pass.
require 'rubygems'
require 'net/ssh'
...
Net::SSH.start(#hostname, #username, :password => #password) do |session|
stdout = ""
session.exec!(#cmd) do |channel, stream, data|
if data =~ /Password/
channel.send_data("passw\n")
end
stdout << data
end
puts stdout
session.close
end
This is the code I currently have:
#!/usr/bin/ruby
require 'em-websocket'
$cs = []
EM.run do
EM::WebSocket.run(:host => "::", :port => 8085) do |ws|
ws.onopen do |handshake|
$cs << ws
end
ws.onclose do
$cs.delete ws
end
end
end
I would like to watch a file with rb-inotify and send a message to all connected clients ($cs.each {|c| c.send "File changed"}) when a file changes. The problem is, I do not understand EventMachine, and I can't seem to find a good tutorial.
So if anyone could explain to me where to put the rb-inotify-related code, I would really appreciate it.
Of course! As soon as I post the question, I figure it out!
#!/usr/bin/ruby
require 'em-websocket'
$cs = []
module Handler
def file_modified
$cs.each {|c| c.send "File was modified!" }
end
end
EM.run do
EM.watch_file("/tmp/foo", Handler)
EM::WebSocket.run(:host => "::", :port => 8085) do |ws|
ws.onopen do |handshake|
$cs << ws
end
ws.onclose do
$cs.delete ws
end
end
end
I'm building a Ruby app that runs both an EM::WebSocket server as well as a Sinatra server. Individually, I believe both of these are equipped to handle a SIGINT. However, when running both in the same app, the app continues when I press Ctrl+C. My assumption is that one of them is capturing the SIGINT, preventing the other from capturing it as well. I'm not sure how to go about fixing it, though.
Here's the code in a nutshell:
require 'thin'
require 'sinatra/base'
require 'em-websocket'
EventMachine.run do
class Web::Server < Sinatra::Base
get('/') { erb :index }
run!(port: 3000)
end
EM::WebSocket.start(port: 3001) do |ws|
# connect/disconnect handlers
end
end
I had the same issue. The key for me seemed to be to start Thin in the reactor loop with signals: false:
Thin::Server.start(
App, '0.0.0.0', 3000,
signals: false
)
This is complete code for a simple chat server:
require 'thin'
require 'sinatra/base'
require 'em-websocket'
class App < Sinatra::Base
# threaded - False: Will take requests on the reactor thread
# True: Will queue request for background thread
configure do
set :threaded, false
end
get '/' do
erb :index
end
end
EventMachine.run do
# hit Control + C to stop
Signal.trap("INT") {
puts "Shutting down"
EventMachine.stop
}
Signal.trap("TERM") {
puts "Shutting down"
EventMachine.stop
}
#clients = []
EM::WebSocket.start(:host => '0.0.0.0', :port => '3001') do |ws|
ws.onopen do |handshake|
#clients << ws
ws.send "Connected to #{handshake.path}."
end
ws.onclose do
ws.send "Closed."
#clients.delete ws
end
ws.onmessage do |msg|
puts "Received message: #{msg}"
#clients.each do |socket|
socket.send msg
end
end
end
Thin::Server.start(
App, '0.0.0.0', 3000,
signals: false
)
end
I downgrade thin to version 1.5.1 and it just works. Wired.
I was wondering if I could stop execution of an operation that has been deffered.
require 'rubygems'
require 'em-websocket'
EM.run do
EM::WebSocket.start(:host => '0.0.0.0', :port => 8080) do |ws|
ws.onmessage do |msg|
op = proc do
sleep 5 # Thread safe IO here that is safely killed
true
end
callback = proc do |result|
puts "Done!"
end
EM.defer(op, callback)
end
end
end
This is an example web socket server. Sometimes when I get a message I want to do some IO, later on another message might come in that needs to read the same thing, the next thing always has precedence over the previous thing. So I want to cancel the first op and do the second.
Here is my solution. It is similar to the EM.queue solution, but just uses a hash.
require 'rubygems'
require 'em-websocket'
require 'json'
EM.run do
EM::WebSocket.start(:host => '0.0.0.0', :port => 3333) do |ws|
mutex = Mutex.new # to make thread safe. See https://github.com/eventmachine/eventmachine/blob/master/lib/eventmachine.rb#L981
queue = EM::Queue.new
ws.onmessage do |msg|
message_type = JSON.parse(msg)["type"]
op = proc do
mutex.synchronize do
if message_type == "preferred"
puts "killing non preferred\n"
queue.size.times { queue.pop {|thread| thread.kill } }
end
queue << Thread.current
end
puts "doing the long running process"
sleep 15 # Thread safe IO here that is safely killed
true
end
callback = proc do |result|
puts "Finished #{message_type} #{msg}"
end
EM.defer(op, callback)
end
end
end
I'm trying to send and "message" object via TCP on Ruby and my client class simply don't see any thing comming. What am I doing wrong?
My message class (that I'm trying to send)
class Message
attr_reader :host_type, :command, :params
attr_accessor :host_type, :command, :params
def initialize(host_type, command, params)
#host_type = host_type
#command = command
#params = params
end
end
My "Server" class
require 'socket'
require_relative 'message'
class TCP_connection
def start_listening
puts "listening"
socket = TCPServer.open(2000)
loop {
Thread.start(socket.accept) do |message|
puts message.command
end
}
end
def send_message
hostname = 'localhost'
port = 2000
s = TCPSocket.open(hostname, port)
message = Message.new("PARAM A", "PARAM B", "PARAM C")
s.print(message)
s.close
end
end
Below an example of client server comunication via json. You will expect something like this RuntimeError: {"method1"=>"param1"}. Instead of raising errors, process this json with the server logic.
Server
require 'socket'
require 'json'
server = TCPServer.open(2000)
loop {
client = server.accept
params = JSON.parse(client.gets)
raise params.inspect
}
Client
require 'socket'
require 'json'
host = 'localhost'
port = 2000
s = TCPSocket.open(host, port)
request = { 'method1' => 'param1' }.to_json
s.print(request)
s.close
You need to serialize your object. The most portable way to do this is using YAML. First, require the yaml library:
require "yaml"
Than, replace s.print(message) with s.print(YAML.dump(message)).