For the purpose of a challenge over on PPCG.SE I want to run programs (submissions) with a strict time limit. In order for those submissions not to waste their precious time with I/O, I want to run the following scheme:
Submissions do their computation in an infinite loop (as far as they get) and register a signal handler.
After 10 minutes SIGINT is sent, and they have one second to produce whatever output is necessary.
SIGKILL is sent to terminate the submission for good.
However, submissions should have the option not to register a signal handler and just produce output in their own time.
I will run the submissions on Windows 8 and wanted to use Ruby to orchestrate this. However, I'm running into some trouble. I thought I'd be able to do this:
solver = IO.popen(solver_command, 'r+')
sleep(10*60)
Process.kill('INT', solver.pid)
sleep(1)
Process.kill('KILL', solver.pid)
puts solver.read
solver.close
However, the moment I send SIGINT, not only the submission process but the controlling Ruby process immediately aborts, too! Even worse, if I do this in PowerShell, PowerShell also shuts down.
A simple Ruby "submission" to reproduce this:
puts "My result"
$stdout.flush
next while true
What am I doing wrong here? I feel that it's very likely I'm misunderstanding something about processes and signals in general and not Ruby in particular, but I'm definitely missing something.
Related
I have a thread in some console process, whose code is not available to me. Is there a way to get notified when its state changes (e.g. becomes idle)?
Maybe hooks API or even kernel-mode API (if no other way...)?
Expansion:
I have legacy console app I have to operate in my program. The only way to interact with it, obviously, is via stdin. So I run it as new Process, and send commands to its stdin. I take some text from its stdout, but it is not entirely predictable, so I cannot rely on it in order to know when it finished its current work and ready to receive the next commands. The flow is something like this: I run it, send (e.g.) command1, wait for it to finish its work (some CPU load and some IO operations), and then issue command2, and so on, until last command. When it finished working on last command, I can close it gracefully (send exit command). The thing is that I have several processes of this console exe that are working simultaneously.
I can:
use timeouts since last stdout received - not good, because it can be a millisecond and an hour
parse stdout using (e.g.) regex to wait for expected outputs - not good... the output is wholly unexpected. It's almost random
using timers, poll its threads state and only when all of them are in wait state (and not for IO), and at least one is waiting for user input (see this) - not good either: in case I use many processes simultaneously it can create unnecessary, non-proportional burden on the system.
So I like the last option, just instead polling, I rather events fired when these threads become idle.
I hope it explains the issue well...
I am using Ruby 1.9.2.
I have a thread running which makes periodic calls to a database. The calls can be quite long, and sometimes (for various reasons) the DB connection disappears. If it does disappear, the thread just silently hangs there forever.
So, I want to wrap it all in a timeout to handle this. The problem is, on the second time through when a timeout should be called (always second), it still simply hangs. The timeout never takes effect. I know this problem existed in 1.8, but I was lead to believe timeout.rb worked in 1.9.
t = Thread.new do
while true do
sleep SLEEPTIME
begin
Timeout::timeout(TIMEOUTTIME) do
puts "About to do DB stuff, it will hang here on the second timeout"
db.do_db_stuff()
process_db_stuff()
end
rescue Timeout::Error
puts "Timed out"
#handle stuff here
end
end
end
Any idea why this is happening and what I can do about it?
One possibility is that your thread does not hang, it actually dies. Here's what you should do to figure out what's going on. Add this before you create your worker thread:
Thread.abort_on_exception = true
When an exception is raised inside your thread that is never caught, your whole process is terminated, and you can see which exception was raised. Otherwise (and this is the default), your thread is killed.
If this turns out not to be the problem, read on...
Ruby's implementation of timeouts is pretty naive. It sets up a separate thread that sleeps for n seconds, then blindly raises a Timeout exception inside the original thread.
Now, the original code might actually be in the middle of a rescue or ensure block. Raising an exception in such a block will silently abort any kind of cleanup code. This might leave the code that times out in an improper state.
It's quite difficult to tell if this is your problem exactly, but seeing how database handlers might do a fair bit of locking and exception handling, it might be very likely. Here's an article that explains the issue in more depth.
Is there any way you can use your database library's built-in timeout handling? It might be implemented on a lower level, not using Ruby's timeout implementation.
A simple alternative is to schedule the database calls in a separate process. You can fork the main process each time you do the heavy database-lifting. Or you could set up a simple cronjob to execute a script that executes it. This will be slightly more difficult if you need to communicate with your main thread. Please leave some more details if you want any advice on which option might suit your needs.
Based on your comments, the thread is dying. This might be a fault in libraries or application code that you may or may not be able to fix. If you wish to trap any arbitrary error that is generated by the database handling code and subsequently retry, you can try something like the following:
t = Thread.new do
loop do
sleep INTERVAL
begin
# Execute database queries and process data
rescue StandardError
# Log error or recover from error situation before retrying
end
end
end
You can also use the retry keyword in the rescue block to retry immediately, but you probably should keep a counter to make sure you're not accidentally retrying indefinitely when an unrecoverable error keeps occurring.
What's the best way to synchronize two ruby scripts running on the same computer?
The scripts will be started as separate processes from the shell.
I want the scripts to take turns running. That is, I want script A to run, send a signal to script B, then wait for a signal from script B. When script B gets the signal from script A, it starts running, signals script A when it is finished, and then waits for a signal from A. Basically, I want the two scripts to run interleaved.
What's the best way to implement this synchronization?
Right now all I can come up with is the creation of a file as the signal (each script busy loops waiting for a file to be created). Are there other implementations that are faster/easier/safer?
In case it affects the answer, I'm on OSX.
Probably the easiest way of doing IPC in ruby is via drb and using Queue, which is located in thread:
require 'thread'
queue = Queue.new # provides blocking read
Note, when using drb, you'll want to have the following line near the top of your program:
Socket.do_not_reverse_lookup=true;
Without it, things just run extremely slowly (source).
To solve the specific problem described in the question, you can create a Pipe class, which essentially is just two Queue objects, one for the inbox and one for the outbox. The blocking read behavior of the Queue makes it easy to have the processes wait for each other. The Pipe is shared between the two processes via drb.
The server startup code might look like this:
require 'drb'
Socket.do_not_reverse_lookup=true;
uri = "druby://localhost:2250" # you can pick a port to communicate on
pipe = Pipe.new
DRb.start_service uri, pipe
The client startup code would look like:
require 'drb'
Socket.do_not_reverse_lookup=true;
uri = "druby://localhost:2250"
DRb.start_service
pipe = DRbObject.new nil, uri
Now the client and server can communicate via the Pipe object.
I want to write a simple program that does nothing but does not easily terminate when asked to. I want to see the Windows dialog box which says, "this program is not responsive, do you want to wait to let it finish what it's doing, or terminate it now?". After the user chooses "terminate it now", it should, of course, exit.
The reason I want this is for a testing environment. I want to test a scenario in which the user is trying to log out, but the system doesn't log them out right away, because of an unresponsive program.
I tried responding to WM_DESTROY by calling Sleep(), but that doesn't seem to do it. The program still terminates immediately when killed from the Task Manager. Again, I'm not trying to write something truly "unkillable", just a simple program which makes that dialog box come up asking if the user wants to wait for the program to finish.
Thanks very much for any help.
You can try modifying your main-message loop . Use PeekMessage(...) with NO_REMOVE and ignore WM_QUIT messages
EDIT: Remove every message (except WM_QUIT) before processing it (GetMessage( &msg ,msg.hWnd ,msg.message ,msg.message ))
I'm writing a job-scheduling app in Ruby for my work (primarily to move files using various protocol at a given frequency)
My main loop looks like this :
while true do
# some code to launch the proper job
sleep CONFIG["interval"]
end
It's working like a charm, but I'm not really sure if it is safe enough as the application might run on a server with cpu-intensive software running.
Is there another way to do the same thing, or is sleep() safe enough in my case ?
Any time I feel the need to block, I use an event loop; usually libev. Here is a Ruby binding:
http://rev.rubyforge.org/rdoc/
Basically, sleep is perfectly fine if you want your process to go to sleep without having anything else going on in the background. If you ever want to do other things, though, like sleep and also wait for TCP connections or a filehandle to become readable, then you're going to have to use an event loop. So, why not just use one at the beginning?
The flow of your application will be:
main {
Timer->new( after => 0, every => 60 seconds, run => { <do your work> } )
loop();
}
When you want to do other stuff, you just create the watcher, and it happens for you. (The jobs that you are running can also create watchers.)
Using sleep is likely OK for quick and dirty things. But for things that need a bit more robustness or reliability I suggest that sleep is evil :) The problem with sleeping is that the thread is (I'm assuming Windows here...) is truly asleep - the scheduler will not run the thread until some time after sleep interval has passed.
During this time, the thread will not wake up for anything. This means it cannot be canceled, or wake up to process some kind of event. Of course, the process can be killed, but that doesn't give the sleeping thread an opportunity to wake up and clean anything up.
I'm not familiar with Ruby, but I assume it has some kind of facility for waiting on multiple things. If you can, I suggest that instead of using sleep, you waint on two things\
A timer that wakes the thread periodically to do its work.
An event that is set when he process needs to cancel or quite (trapping control-C for example).
It would be even better if there is some kind of event that can be used to signal the need to do work. This would avoid polling on a timer. This generally leads to lower resource utilization and a more responsive system.
If you don't need an exact interval, then it makes sense to me. If you need to be awoken at regular times without drift, you probably want to use some kind of external timer. But when you're asleep, you're not using CPU resources. It's the task switch that's expensive.
While sleep(timeout) is perfectly appropriate for some designs, there's one important caveat to bear in mind.
Ruby installs signal handlers with SA_RESTART (see here), meaning that your sleep (or equivalent select(nil, nil, nil, timeout)) cannot easily be interrupted. Your signal handler will fire, but the program will go right back to sleep. This may be inconvenient if you wished to react timely to, say, a SIGTERM.
Consider that ...
#! /usr/bin/ruby
Signal.trap("USR1") { puts "Hey, wake up!" }
Process.fork() { sleep 2 and Process.kill("USR1", Process.ppid) }
sleep 30
puts "Zzz. I enjoyed my nap."
... will take about 30 seconds to execute, rather than 2.
As a workaround, you might instead throw an exception in your signal handler, which would interrupt the sleep (or anything else!) above. You might also switch to a select-based loop and use a variant of the self-pipe trick to wake up "early" upon receipt of a signal. As others have pointed out, fully-featured event libraries are available, too.
It wont use CPU while it is sleeping but if you are sleeping for a long time I would be more concerned of the running ruby interpreter holding up memory while it wasn't doing anything. This is not that big of a deal tho.