How can I kill running processes in EventMachine?
Below is an example, I'm starting 10 processes and then I'm trying to erase them all (but it doesn't work). My goal is to not have the "Finished" output.
require "rubygems"
require "eventmachine"
class Event
def start
sleep(5)
puts Time.now.to_s + ": Finished!"
end
end
EventMachine.run do
events = []
10.times {
handle = Event.new
events << handle
EventMachine.defer(proc {
handle.start
})
}
# Terminate all events!
events.each do |handle|
handle = nil
ObjectSpace.garbage_collect
end
end
I'm aware that I could set a variable and check whether it's set when doing the output, but I feel like this isn't the "real" thing, or is this really the only solution there is?
Try EventMachine.stop_event_loop, it will “cause all open connections and accepting servers to be run down and closed”.
Related
I have 3 methods that do something on the file system but each method changes current directory and I can't use Threading for those since it uses same ruby process and I have racing condition. I found Process.spawn (can't use fork on windows) might be solution but I don't know how to spawn 3 process and call method in each of those.
I tried following as a test, but didn't work
#!/usr/bin/env ruby
def something
puts 'printaj'
end
def nextsomething
puts 'vici'
end
pid1 = Process.spawn(something)
pid2 = Process.spawn(nextsomething)
Process.wait pid1
Process.wait pid2
puts 'both finished'
How about threads and mutex?
semaphore = Mutex.new
a = Thread.new {
semaphore.synchronize {
# do something on the file system
}
}
b = Thread.new {
semaphore.synchronize {
# do something on the file system
}
}
With this code you will run only one operation on file system in the same time.
https://ruby-doc.org/core-2.5.0/Mutex.html
I am encountering an interesting issue with Ruby TCPServer, where once a client connects, it continually uses more and more CPU processing power until it hits 100% and then the entire system starts to bog down and can't process incoming data.
The processing class that is having an issue is designed to be a TCP Client that receives data from an embedded system, processes it, then returns the processed data to be further used (either by other similar data processors, or output to a user).
In this particular case, there is an external piece of code that would like this processed data, but cannot access it from the main parent code (the thing that the original process class is returning it's data to). This external piece may or may not be connected at any point while it is running.
To solve this, I set up a Thread with a TCPServer, and the processing class continually adds to a queue, and the Thread pulls from the queue and sends it to the client.
It works great, except for the performance issues. I am curious if I have something funky going on in my code, or if it's just the nature of this methodology and it will never be performant enough to work.
Thanks in advance for any insight/suggestions with this problem!
Here is my code/setup, with some test helpers:
process_data.rb
require 'socket'
class ProcessData
def initialize
super
#queue = Queue.new
#client_active = false
Thread.new do
# Waiting for connection
#server = TCPServer.open('localhost', 5000)
loop do
Thread.start(#server.accept) do |client|
puts 'Client connected'
# Connection established
#client_active = true
begin
# Continually attempt to send data to client
loop do
unless #queue.empty?
# If data exists, send it to client
begin
until #queue.empty?
client.puts(#queue.pop)
end
rescue Errno::EPIPE => error
# Client disconnected
client.close
end
end
sleep(1)
end
rescue IOError => error
# Client disconnected
#client_active = false
end
end # Thread.start(#server.accept)
end # loop do
end # Thread.new do
end
def read(data)
# Data comes in from embedded system on this method
# Do some processing
processed_data = data.to_i + 5678
# Ready to send data to external client
if #client_active
#queue << processed_data
end
return processed_data
end
end
test_embedded_system.rb (source of the original data)
require 'socket'
#data = '1234'*100000 # Simulate lots of data coming ing
embedded_system = TCPServer.open('localhost', 5555)
client_connection = embedded_system.accept
loop do
client_connection.puts(#data)
sleep(0.1)
end
parent.rb (this is what will create/call the ProcessData class)
require_relative 'process_data'
processor = ProcessData.new
loop do
begin
s = TCPSocket.new('localhost', 5555)
while data = s.gets
processor.read(data)
end
rescue => e
sleep(1)
end
end
random_client.rb (wants data from ProcessData)
require 'socket'
loop do
begin
s = TCPSocket.new('localhost', 5000)
while processed_data = s.gets
puts processed_data
end
rescue => e
sleep(1)
end
end
To run the test in linux, open 3 terminal windows:
Window 1: ./test_embedded_system.rb
Window 2: ./parent.rb
\CPU usage is stable
Window 3: ./random_client.rb
\CPU usage continually grows
I ended up figuring out what the issue was, and unfortunately I lead folks astray with my example.
It turns out my example didn't quite have the issue I was having, and the main difference was the sleep(1) was not in my version of process_data.rb.
That sleep is actually incredibly important, because it is inside of a loop do, and without the sleep, the Thread won't yield the GVL, and will continually eat up CPU resources.
Essentially, it was unrelated to TCP stuff, and more related to Threads and loops.
If you stumble on this question later on, you can put a sleep(0) in your loops if you don't want it to wait, but you want it to yield the GVL.
Check out these answers as well for more info:
Ruby infinite loop causes 100% cpu load
sleep 0 has special meaning?
I'm a DBA, and I stumbled upon such case: developers run irb sessions (from Ruby on Rails app). This irb keeps database connection open. Sometimes - they forget about it, and it keeps on "running" - not doign anything, but still using one db connection.
I'd like to add some kind of "idle timeout" to their irb config. Is it possible? How to do it?
Here's a quick hack how you might implement this.
Note that this does not take into account that the user might be executing some long-running task inside the irb session. It simply looks at the time stamp of the last input; if it has not changed then it just flat out kills the process:
Update: it now checks if irb is currently running a command and ignores any timeouts if that is the case.
# Add some methods to IRB::Context and IRB::Irb
# for easier timeout implementation.
class IRB::Irb
def status
#signal_status
end
end
class IRB::Context
attr_reader :irb
attr_reader :line_no
def is_evaluating?
self.irb.status == :IN_EVAL
end
end
# Implement an auto exit timer thread. Timeout is given in seconds.
module IRB
def self.auto_exit_after(timeout = 60)
Thread.new {
context = IRB.conf[:MAIN_CONTEXT]
last_input = Time.now
last_line = context.line_no
loop {
sleep 10
# Check if irb is running a command
if context.is_evaluating?
# Reset the input time and ignore any timeouts
last_input = Time.now
next
end
# Check for new input
if last_line != context.line_no
# Got new input
last_line = context.line_no
last_input = Time.now
next
end
# No new input, check if idle time exceeded
if Time.now - last_input > timeout
$stderr.puts "\n** IRB exiting due to idle timeout. Goodbye..."
Process.kill("KILL", Process.pid)
end
}
}
end
end
To use it add the code to .irbrc, or some other place that auto-loads when irb is started, and then just start the timer:
IRB.auto_exit_after(60)
I'm using RSpec to test the behavior of a simple REPL. The REPL just echoes back whatever the input was, unless the input was "exit", in which case it terminates the loop.
To avoid hanging the test runner, I'm running the REPL method inside a separate thread. To make sure that the code in the thread has executed before I write expectations about it, I've found it necessary to include a brief sleep call. If I remove it, the tests fail intermittently because the expectations are sometimes made before the code in the thread has run.
What is a good way to structure the code and spec such that I can make expectations about the REPL's behavior deterministically, without the need for the sleep hack?
Here is the REPL class and the spec:
class REPL
def initialize(stdin = $stdin, stdout = $stdout)
#stdin = stdin
#stdout = stdout
end
def run
#stdout.puts "Type exit to end the session."
loop do
#stdout.print "$ "
input = #stdin.gets.to_s.chomp.strip
break if input == "exit"
#stdout.puts(input)
end
end
end
describe REPL do
let(:stdin) { StringIO.new }
let(:stdout) { StringIO.new }
let!(:thread) { Thread.new { subject.run } }
subject { described_class.new(stdin, stdout) }
# Removing this before hook causes the examples to fail intermittently
before { sleep 0.01 }
after { thread.kill if thread.alive? }
it "prints a message on how to end the session" do
expect(stdout.string).to match(/end the session/)
end
it "prints a prompt for user input" do
expect(stdout.string).to match(/\$ /)
end
it "echoes input" do
stdin.puts("foo")
stdin.rewind
expect(stdout.string).to match(/foo/)
end
end
Instead of letting :stdout be a StringIO, you could back it by a Queue. Then when you try to read from the queue, your tests will just wait until the REPL pushes something into the queue (aka. writes to stdout).
require 'thread'
class QueueIO
def initialize
#queue = Queue.new
end
def write(str)
#queue.push(str)
end
def puts(str)
write(str + "\n")
end
def read
#queue.pop
end
end
let(:stdout) { QueueIO.new }
I just wrote this up without trying it out, and it may not be robust enough for your needs, but it gets the point across. If you use a data structure to synchronize the two threads like this, then you don't need to sleep at all. Since this removes the non-determinism, you shouldn't see the intermittent failures.
I've used a running? guard for situations like this. You probably can't avoid the sleep entirely, but you can avoid unnecessary sleeps.
First, add a running? method to your REPL class.
class REPL
...
def running?
!!#running
end
def run
#running=true
loop do
...
if input == 'exit
#running = false
break
end
...
end
end
end
Then, in your specs, sleep until the REPL is running:
describe REPL do
...
before { sleep 0.01 until REPL.running? }
...
end
I'm trying to do something like this, where I have two loops going in seperate threads. The problem I am having is that in the main thread, when I use gets and the script is waiting for user input, the other thread is stopped to wait as well.
class Server
def initialize
#server = TCPServer.new(8080)
run
end
def run
#thread = Thread.new(#server) { |server|
while true
newsock = server.accept
puts "some stuff after accept!"
next if !newsock
# some other stuff
end
}
end
end
def processCommand
# some user commands here
end
test = Server.new
while true do
processCommand(STDIN.gets)
end
The above is just a sample of what I want to do.
Is there a way to make the main thread block while waiting for user input?
You might want to take a look at using the select method of the IO class. Take a look at
good select example for handling select with asynchronous input. Depending upon what version of ruby you're using you might have issues with STDIN though, I'm pretty sure it always triggers the select in 1.8.6.
I'm not sure if this is what you are looking for, but I was looking for something similar and this example does exactly what I wanted. The thread will continue processing until the user hits enter, and then the thread will be able to handle your user input as desired.
user_input = nil
t1 = Thread.new do
while !user_input
puts "Running"
end
puts "Stopping per user input: #{user_input}"
end
user_input = STDIN.gets
t1.join