I wish to create a TCPServer which listens in the background while the script continues it's normal execution.
For example-
server=TCPServer.open(srvhost,srvport)
client=server.accept
res=send_request_cgi({
'uri' => "/",
'method' => 'GET',
'vars_get' =>
'cmd' => rand_text_alphanumeric(10)
}
})
print "#{res.headers}"
data=client.recv(1024)
puts data
client.close
This pauses the script at the listener.
Objective-The server will always receive a response on srvport which it should print.
Edit-
webserv=Thread.new do
server=TCPServer.new(srvhost,srvport)
client=server.accept
data=client.recv(1024)
if(data.empty?)
puts 'nope'
client.close
server.close
webserv.exit
end
puts data
client.write(cmd)
client.close
server.close
webserv.exit
end
You can put your server in a new Thread with
websrv = Thread.new do
<your server logic> # Main server loop, handle data, etc.
ensure
<your cleanup code> # Close connections, files, etc.
end
<application code>
websrv.join # wait for web server to do its thing
Needless to say, in ruby you'd usually wrap that in some sort of BackgroundServer class, but the basic idea is the same.
Note that rubys threads don't really execute in parallel; the interpreter just takes care of switching back and forth between them.
Related
I'm new both to IMAP and multi-thread programming, and I'd like to write a script to fetch incoming mails and process them in parallel.
Thread, Queue, Mutex and Monitor are new concepts to me, I may miss-use them in the following question and example.
The script's goal is to:
receive IDLE notifications from the IMAP server when a new mail arrives
fetch the corresponding mail
parse its body
generate a PDF from parsed data
send the report through SMTP
Another aproach could be to fetch and identify mails based on their UID, thought learning IMAP & multi-thread programing is another goal.
Here's what I've got so far (IMAP part):
def self.idle(imap)
#imap = Net::IMAP.new 'mail.company.org', 143, false # Not using TLS for test purposes
#imap.login 'username', 'password'
#idler_listener = Thread.new do
loop do
begin
imap.examine 'INBOX'
imap.idle do |res|
if res.kind_of?(Net::IMAP::UntaggedResponse) and res.name == 'EXISTS'
imap.idle_done
Thread.new { process_email(imap) }
end
end
rescue => e
puts e.inspect
end
end
end.join
end
def self.process_email(imap)
begin
uid = imap.uid_search(['SUBJECT', 'MyFavoriteSubject']).last
mail = imap.uid_fetch(uid, 'BODY[TEXT]')[0].attr['BODY[TEXT]']
puts mail
rescue => e
puts e.inspect
end
end
This example successfully prints out the body of an incoming mail. However, if several mail arrives at the same time, only one will be treated.
Q:
Is this behavior due to the fact that the loop is executed in a single thread ?
Does it illustrate the need for a queue, a thread pool, or a producer / consumer pattern ?
Chances that this script will attempt to access the same resource at the same time are low, however, would Mutex.new.synchronize protect me against race conditions & deadlocks ?
(Eventhough I've not fully understand it, I think it's worth sharing this great resource from Masatoshi Seki: https://www.druby.org/sidruby/)
I've written little UDP server in Ruby:
def listen
puts "Started UDP server on #{#port}..."
Socket.udp_server_loop(#port) do |message, message_source|
puts "Got \"#{message}\" from #{message_source}"
handle_incoming_message(message)
end
end
I start it in a separate thread:
thread = Thread.new { listen }
Is there a way to gracefully stop the udp_server_loop from outside the thread without just killing it (thread.kill)? I also dont't want to stop it from the inside by receiving any UDP message. Is udp_server_loop maybe not the right tool for me?
I don’t think you can do this with udp_server_loop (although you might be able to use some of the methods it uses). You are going to have to call IO::select in a loop of your own with some way of signalling it to exit, and some way of waking the thread so you don’t have to send a packet to stop it.
A simple way would be to use the timeout option to select with a variable to set to indicate you want the thread to end, something like:
#halt_loop = false
def listen
puts "Started UDP server on #{#port}..."
sockets = Socket.udp_server_sockets(#port)
loop do
readable, _, _ = IO.select(sockets, nil, nil, 1) # timeout 1 sec
break if #halt_loop
next unless readable # select returns nil on timeout
Socket.udp_server_recv(readable) do |message, message_source|
puts "Got \"#{message}\" from #{message_source}"
handle_incoming_message(message)
end
end
end
You then set #halt_loop to true when you want to stop the thread.
The downside to this is that it is effectively polling. If you decrease the timeout then you potentially do more work on an empty loop, and if you increase it you have to wait longer when stopping the thread.
Another, slightly more complex solution would be to use a pipe and have the select listen on it along with the sockets. You could then signal directly to finish the select and exit the thread.
#read, #write = IO.pipe
#halt_loop = false
def listen
puts "Started UDP server on #{#port}..."
sockets = Socket.udp_server_sockets(#port)
sockets << #read
loop do
readable, _, _ = IO.select(sockets)
break if #halt_loop
readable.delete #read
Socket.udp_server_recv(readable) do |message, message_source|
puts "Got \"#{message}\" from #{message_source}"
handle_incoming_message(message)
end
end
end
def end_loop
#halt_loop = true
#write.puts "STOP!"
end
To exit the thread you just call end_loop which sets the #halt_loop flag then writes to the pipe, making the other end readable and causing the other thread to return from select.
You could have this code check the readable IOs and exit if one of them is the read end of the pipe instead of using the variable, but at least on Linux there is a potential bug where a call to select might return a file descriptor as readable when it actuallt isn’t. I don’t know if Ruby deals with this, so better safe than sorry.
Also be sure to remove the pipe from the readable array before passing it to udp_server_recv. It’s not a socket so will cause an exception if you don’t.
A downside to this technique is that pipes are “[n]ot available on all platforms".
Although I doubt I understand what would be wrong with Thread::kill and/or Thread#exit, you might use the thread local variable for that.
def listen
Socket.udp_server_loop(#port) do |message, message_source|
break :interrupted if Thread.current[:break]
handle_incoming_message(message)
end
end
and do
thread[:break] = true
from the outside.
I have a TCPserver that I made in ruby, the server seems to work, I can see that two or more clients can connect and be served by the server, but, they sometime get stuck (as in need to wait for the other client to disconnect or just get unresponsive), usually after the "pass_ok" bit, When connecting only with one client I don't see this issue.
Here is my code:
def self.main_server
begin
server = TCPServer.open(#port)
rescue Exception => e
CoreLogging.syslog_error("Cant start server: #{e}")
end
#main_pid = Process.pid
# Main Loop
Thread.abort_on_exception = true
while true
Thread.fork(server.accept) do |client|
#client = client
sock_domain, remote_port, remote_hostname, remote_ip = #client.peeraddr # Get some info on the incoming connection
CoreLogging.syslog_error("Got new connection from #{#client.peeraddr[3]} Handeled by Thread: #{Thread.current}") # Log incoming connection
#client.puts "Please enter password: " # Password testing (later will be from a config file or DB)
action = #client.gets(4096).chomp # get client password response 'chomp' is super important
if action == #password
# what to do when password is right
pass_ok
Thread.exit
else
# what to do when password is wrong
pass_fail
Thread.exit
end
end
begin
CoreLogging.syslog_error("Thread Ended (SOFT)")
rescue Exception => e
CoreLogging.syslog_error("Thread was killed (HARD)")
end
end
end
I'll leave it here for future reference and hope someone in a close situation will find it useful.
The issue was the global #client variable, which got overwritten every new thread and then inherited to the subclasses inside the thread.
using a local client variable (without the '#') got it to work as supposed.
That's my first time with EM so I really need some help here
so here's the code:
EM.run do
queue = EM::Queue.new
EM.start_server('0.0.0.0', '9000', RequestHandler, queue)
puts 'Server started on localhost:9000' # Any interface, actually
process_queue = proc do |url|
request = EM::HttpRequest.new(url, connect_timeout: 1).get # No time to wait, sorry
request.callback do |http| # deferrable
puts http.response_header.status
end
queue.pop(&process_queue)
end
EM.next_tick { queue.pop(&process_queue) }
end
I've read a couple of articles about EM, now my understanding of above code is the following:
EM::HttpRequest is deferrable, which means it won't block a reactor.
But when I try running 50 concurrent connections with ab, it only serves ~20 concurrently ( according to ab report ).
But if I place the process_queue execution inside EM.defer( which means it will run in a separate thread? ) it performs just fine.
Why is it so? process_queue just inits a deferrable object and assigns a callback, how does running it inside EM.defer makes a difference?
One thing you may want to do is put the queue.pop(&process_queue) in the process_queue callback inside an EM.next_tick. Currently you're going to process all of the queued connections before you allow anything new to connect. If you put the queue.pop into a next_tick call you'll let the reactor do some work before you process the next item.
I'm using the following code to connect to a network service i'm writing (thats backed by EventMachine) and I'm having a bit of trouble getting into a situation allowing me to use one socket connection to execute multiple commands.
#!/usr/bin/env ruby
require 'socket'
opts = {
:address => "0.0.0.0",
:port => 2478
}
connection = TCPSocket.open opts[:address], opts[:port]
# Get ID
connection.print "ID something"
puts connection.read
# Status
connection.print "STATUS"
puts connection.read
# Close the connection
connection.close
Here's what my EventMachine server hander looks like...
module ConnectionHandler
def receive_data data
send_data "Some output #{data}"
end
end
However, my first ruby script hangs when it executes connection.read as I presume its waiting for the connection to close so it knows its got all of the data? This is not what I want to happen.
My socket server will just take one command (on one line) and return one line of output.
Any ideas how I can do this? Thanks.
It turns out the connection.gets method will return a line of data received if the server sends a response ending in a \n character. So I just added \n to the end of my send_data call and switch to using puts connection.gets and it worked great!