What is active_record doing to the signal processes under windows (I don't see this with the same versions on the mac) that causes it to behave so strangely? For instance:
require 'rubygems'
trap("INT"){puts "interrupted"}
puts __LINE__
sleep 5
require 'active_record'
trap("INT"){puts "interrupted again"}
puts __LINE__
sleep 5
When I run the above code (ruby 1.8.6, gem 1.3.1, activerecord 2.2.2,) I can hit ^C as many times as I like during the first sleep, but the first interrupt after the require of activerecord causes the script to terminate. In the above case, the trap still executes, it only fails to allow the program to continue. Usually.
Removing the second call to trap does not have any effect upon the behaviors.
The real annoyance is that in some conditions, the trap fails to execute at all. Considering that the whole point of doing this is to get my code to clean up after itself (remove its footprint in the database so the next guy sees a sane state,) this is a real problem. For instance:
require 'rubygems'
require 'active_record'
trap("INT"){puts "interrupted"}
puts __LINE__
gets
Pressing ^C after seeing the puts will not execute the trap at all.
I only see this problem after requiring active_record. Is there a workaround? I'd be curious to know if this is a bug or if there is an explanation of some sort. As I said, I have no issue with this on the mac - repeated ^Cs result in multiple executions of the trap proc.
thanks...
Considering that the whole point of doing this is to get my code to clean up after itself (remove its footprint in the database ...
Have you considered just using a database transaction? It seems like it would be a much easier way to solve the problem.
I saw a different pattern when trying to duplicate this problem:
puts "start"
trap("INT") { puts "interrupted" }
sleep 5
puts "end"
On Ubuntu (Ruby 1.8.6) this produces
start
interrupted
interrupted
(etc)
interrupted
end
So "interrupted" prints each time Crtl-C is pressed, until the 5 seconds are up. Under Windows (also Ruby 1.8.6), this produces:
start
interrupted
end
i.e. it prints "interrupted" once and then exits.
So it appears that while handling SIGINT Ruby exits the sleep routine and continues on to the next statement. My guess (hand-waving) is that this is somehow due to Ruby using green threads instead of native threads on Windows. Any experts please chime in here.
You could emulate the Unix-y behavior by restarting sleep in the handler:
puts "start"
trap("INT") do
puts "interrupted"
sleep 5
end
sleep 5
puts "end"
Unfortunately this resets the timer each time SIGINT is trapped, so it needs some hacking:
$interval = 5
def go_to_sleep(secs)
$started = Time.now
sleep secs
end
trap("INT") do
puts "interrupted"
time_to_sleep = [0,$interval - (Time.now - $started)].max
if time_to_sleep > 0
sleep time_to_sleep
end
end
puts "one"
go_to_sleep($interval)
puts "two"
go_to_sleep($interval)
puts "three"
go_to_sleep($interval)
Related
I am developing a long-running program in Ruby. I am writing some integration tests for this. These tests need to kill or stop the program after starting it; otherwise the tests hang.
For example, with a file bin/runner
#!/usr/bin/env ruby
while true do
puts "Hello World"
sleep 10
end
The (integration) test would be:
class RunReflectorTest < TestCase
test "it prints a welcome message over and over" do
out, err = capture_subprocess_io do
system "bin/runner"
end
assert_empty err
assert_includes out, "Hello World"
end
end
Only, obviously, this will not work; the test starts and never stops, because the system call never ends.
How should I tackle this? Is the problem in system itself, and would Kernel#spawn provide a solution? If so, how? Somehow the following keeps the out empty:
class RunReflectorTest < TestCase
test "it prints a welcome message over and over" do
out, err = capture_subprocess_io do
pid = spawn "bin/runner"
sleep 2
Process.kill pid
end
assert_empty err
assert_includes out, "Hello World"
end
end
. This direction also seems like it will cause a lot of timing-issues (and slow tests). Ideally, a reader would follow the stream of STDOUT and let the test pass as soon as the string is encountered and then immediately kill the subprocess. I cannot find how to do this with Process.
Test Behavior, Not Language Features
First, what you're doing is a TDD anti-pattern. Tests should focus on behaviors of methods or objects, not on language features like loops. If you must test a loop, construct a test that checks for a useful behavior like "entering an invalid response results in a re-prompt." There's almost no utility in checking that a loop loops forever.
However, you might decide to test a long-running process by checking to see:
If it's still running after t time.
If it's performed at least i iterations.
If a loop exits properly given certain input or upon reaching a boundary condition.
Use Timeouts or Signals to End Testing
Second, if you decide to do it anyway, you can just escape the block with Timeout::timeout. For example:
require 'timeout'
# Terminates block
Timeout::timeout(3) { `sleep 300` }
This is quick and easy. However, note that using timeout doesn't actually signal the process. If you run this a few times, you'll notice that sleep is still running multiple times as a system process.
It's better is to signal the process when you want to exit with Process::kill, ensuring that you clean up after yourself. For example:
pid = spawn 'sleep 300'
Process::kill 'TERM', pid
sleep 3
Process::wait pid
Aside from resource issues, this is a better approach when you're spawning something stateful and don't want to pollute the independence of your tests. You should almost always kill long-running (or infinite) processes in your test teardown whenever you can.
Ideally, a reader would follow the stream of STDOUT and let the test pass as soon as the string is encountered and then immediately kill the subprocess. I cannot find how to do this with Process.
You can redirect stdout of spawned process to any file descriptor by specifying out option
pid = spawn(command, :out=>"/dev/null") # write mode
Documentation
Example of redirection
With the answer from CodeGnome on how to use Timeout::timeout and the answer from andyconhin on how to redirect Process::spawn IO, I came up with two Minitest helpers that can be used as follows:
it "runs a deamon" do
wait_for(timeout: 2) do
wait_for_spawned_io(regexp: /Hello World/, command: ["bin/runner"])
end
end
The helpers are:
def wait_for(timeout: 1, &block)
Timeout::timeout(timeout) do
yield block
end
rescue Timeout::Error
flunk "Test did not pass within #{timeout} seconds"
end
def wait_for_spawned_io(regexp: //, command: [])
buffer = ""
begin
read_pipe, write_pipe = IO.pipe
pid = Process.spawn(command.shelljoin, out: write_pipe, err: write_pipe)
loop do
buffer << read_pipe.readpartial(1000)
break if regexp =~ buffer
end
ensure
read_pipe.close
write_pipe.close
Process.kill("INT", pid)
end
buffer
end
These can be used in a test which allows me to start a subprocess, capture the STDOUT and as soon as it matches the test Regular Expression, it passes, else it will wait 'till timeout and flunk (fail the test).
The loop will capture output and pass the test once it sees matching output. It uses a IO.pipe because that is most transparant for subprocesses (and their children) to write to.
I doubt this will work on Windows. And it needs some cleaning up of the wait_for_spawned_io which is doing slightly too much IMO. Antoher problem is that the Process.kill('INT') might not reach the children which are orphaned but still running after this test has ran. I need to find a way to ensure the entire subtree of processes is killed.
I'm trying to create an app in ruby which can be started from command line and it does two things: runs a continous job (loop with sleep which runs some action [remote feed parsing]) with one thread and sinatra in a second thread. My code (simplified) looks like that:
require 'sinatra'
class MyApp < Sinatra::Base
get '/' do
"Hello!"
end
end
threads = []
threads << Thread.new do
loop do
# do something heavy
sleep 10
end
end
threads << Thread.new do
MyApp.run!
end
threads.each { |t| t.join }
The above code actually does it's job very well - the sinatra app is started an available under 4567 port and the do something heavy task is beeing fired each 10 seconds. However, i'm not able to kill that script.
I'm running it with ruby app.rb but killing it with ctrl + c is not working. It kills just the sinatra thread but the second one is still running and, to stop the script, i need to close the terminal window.
I was trying to kill all the threads on SIGNINT but it's also not working as expected
trap "SIGINT" do
puts "Exiting"
threads.each { |t| Thread.kill t }
exit 130
end
Can you help me with this? Thanks in advance.
To trap ctrl-c, change "SIGINT" to "INT".
trap("INT") {
puts "trapping"
threads.each{|t|
puts "killing"
Thread.kill t
}
}
To configure Sinatra to skip catching traps:
class MyApp < Sinatra::Base
configure do
set :traps, false
end
...
Reference: Ruby Signal module
To list the available Ruby signals: Signal.list.keys
Reference: Sinatra Intro
(When I run your code and trap INT, I do get a Sinatra socket warning "Already in use". I presume that's fine for your purposes, or you can solve that by doing a Sinatra graceful shutdown. See Sinatra - terminate server from request)
Late to the party, but Trap has one big disadvantage - it gets overriden by the webserver. For example, Puma sets several traps which basically makes your one never to be called.
The best workaround is to use at_exit which can be defined multiple times and Ruby makes sure all blocks are called. I haven't tested this if it would work for your case tho.
I have a ruby script on a remote server that I'm running via Net:SSH on my local pc.
The remote script takes a few minutes to run and outputs it's progress to stdout.
The problem I have is the block in my exec command only gets called when the packet/chunk is full.
So I get the progress all in one hit about each minute.
Here is some cut down examples that illustrate my problem:
Server Script:
(0.999).each do |i|
puts i
sleep 1
end
puts 1000
Local Script:
Net::SSH.start('ip.v.4.addr', 'user', :keys => ['my_key']) do |ssh|
ssh.exec("ruby count_to_1000.rb") do |ch, stream, data|
puts data if stream == :stdout
end
ssh.loop(1)
end
Is there any way from the remote script to force the sending of the packet/chunk?
Or is there a way to set a limit of say a second (or n bits) before it's flushed? (within Net:SSH)
Thanks for all your help!
Try flush:
http://www.ruby-doc.org/core-2.1.5/IO.html#method-i-flush
(0.999).each do |i|
puts i
STDOUT.flush
sleep 1
end
Or sync:
http://www.ruby-doc.org/core-2.1.5/IO.html#method-i-sync
STDOUT.sync = true
(0.999).each do |i|
puts i
sleep 1
end
(Untested, btw. Maybe they need to be used on the client-side instead, or on some other IO stream. But those are the two methods that immediately come to mind.)
In my test setup this works as expected (tested with localhost). However, there might be some issues with the STDOUT flush.
You can try to to write to STDOUT in stead of using puts (I have heard that there is some difference that I don't really understand).
Thus, you can on your server use:
(0.999).each do |i|
STDOUT.puts i
sleep 1
end
STDOUT.puts 1000
#You could possibly also use "STDOUT.write 1000", but it will not append a newline, like puts does.
If that does not work, then you can try to force-flush the STDOUT by using STDOUT.flush(). I believe the same can be achieved by writing an empty string to STDOUT, but I am not 1000% sure.
It might also happen that the exec command actually waits for the entire process to terminate for some reason(I was not able to figure out from the docs). In which case, you won't be able to achieve what you want. Then you can consider setting up websockets, use DRB, or some other means to pass the data.
I'm trying this example:
10.times do
Thread.new do
greeting_message = "Hello World ruby !"
puts "#{greeting_message}
end
end
I tried running this multiple times, and sometimes it puts once:
Hello World ruby ! ruby basic_threadding_ruby.rb 0.05s user 0.04s system 97% cpu 0.096 total
other times its twice, sometimes its the full 10 times.
This inconsistency is making me confused. Is there a reason why Hello World ruby ! is printed only once? I thought when you run a ruby script, it waits until all threads/processes are done before terminating and returning.
I thought when you run a ruby script, it waits until all threads/processes are done before terminating and returning?
Nope! From the documentation for Thread:
If we don't call thr.join before the main thread terminates, then all other threads including thr will be killed.
So you’ll need to join all of them:
threads = 10.times.map do
Thread.new do
puts 'Hello, Ruby!'
end
end
threads.each &:join
Here's the code:
while 1
input = gets
puts input
end
Here's what I want to do but I have no idea how to do it:
I want to create multiple instances of the code to run in the background and be able to pass input to a specific instance.
Q1: How do I run multiple instances of the script in the background?
Q2: How do I refer to an individual instance of the script so I can pass input to the instance (Q3)?
Q3: The script is using the cmd "gets" to take input, how would I pass input into an indivdual's script's gets?
e.g
Let's say I'm running threes instances of the code in the background and I refer to the instance as #1, #2, and #3 respectively.
I pass "hello" to #1, #1 puts "hello" to the screen.
Then I pass "world" to #3 and #3 puts "hello" to the screen.
Thanks!
UPDATE:
Answered my own question. Found this awesome tut: http://rubylearning.com/satishtalim/ruby_threads.html and resource here: http://www.ruby-doc.org/core/classes/Thread.html#M000826.
puts Thread.main
x = Thread.new{loop{puts 'x'; puts gets; Thread.stop}}
y = Thread.new{loop{puts 'y'; puts gets; Thread.stop}}
z = Thread.new{loop{puts 'z'; puts gets; Thread.stop}}
while x.status != "sleep" and y.status != "sleep" and z.status !="sleep"
sleep(1)
end
Thread.list.each {|thr| p thr }
x.run
x.join
Thank you for all the help guys! Help clarified my thinking.
I assume that you mean that you want multiple bits of Ruby code running concurrently. You can do it the hard way using Ruby threads (which have their own gotchas) or you can use the job control facilities of your OS. If you're using something UNIX-y, you can just put the code for each daemon in separate .rb files and run them at the same time.
E.g.,
# ruby daemon1.rb &
# ruby daemon2.rb &
There are many ways to "handle input and output" in a Ruby program. Pipes, sockets, etc. Since you asked about daemons, I assume that you mean network I/O. See Net::HTTP.
Ignoring what you think will happen with multiple daemons all fighting over STDIN at the same time:
(1..3).map{ Thread.new{ loop{ puts gets } } }.each(&:join)
This will create three threads that loop indefinitely, asking for input and then outputting it. Each thread is "joined", preventing the main program from exiting until each thread is complete (which it never will be).
You could try using multi_daemons gem which has capability to run multiple daemons and control them.
# this is server.rb
proc_code = Proc do
loop do
sleep 5
end
end
scheduler = MultiDaemons::Daemon.new('scripts/scheduler', name: 'scheduler', type: :script, options: {})
looper = MultiDaemons::Daemon.new(proc_code, name: 'looper', type: :proc, options: {})
MultiDaemons.runner([scheduler, looper], { force_kill_timeout: 60 })
To start and stop
ruby server.rb start
ruby server.rb stop