I am trying to run multiple ruby scripts simultaneously on my mac, and I'm not having any luck. I can see the the ruby processes start up, but then they immediately stop. The script works fine as a single process, no errors. Here are some examples of things I've tried.
10.times do
system "nohup ruby program.rb \"arg1 arg2\" &"
end
10.times do
`nohup ruby program.rb \"arg1 arg2\" &`
end
10.times do
system "ruby program.rb \"arg1 arg2\""
end
Do you need to start it from ruby for any specific reason? Why don't you start it 10 times directly from bash? Like:
$ for i in seq 1 10; do nohup ruby foo.rb \&; done
Let me know..
nohup redirects its output to a file $HOME/nohup.out, unless it is explicitly redirected. You should redirect the output of each invocation to a different file.
Also, for the safe side, I would redirect stdin to /dev/null - just in case the called program reads from stdin.
10.times do |i|
system "nohup ruby program.rb 'arg1 arg2' </dev/null >#{ENV['HOME']}/nohup#{i}.out &"
end
BTW (and off topic): Are you sure, that you want to pass arg1 arg2 as a SINGLE argument to program.rb?
You can build a solution with fork, exec and wait of the module Process.
# start child processes
10.times { fork { exec(cmd) } }
# wait for child processes
10.times { |pid| Process.wait }
Or a bit longer to play around with (Tested with Ruby 1.8.7 on Ubuntu). Added rescue nil to suppress error when waiting.
10.times do |i|
fork do
ruby_cmd = "sleep(#{10-i});puts #{i}"
exec("ruby -e \"#{ruby_cmd}\"")
end
end
10.times { Process.wait rescue nil }
puts "Finished!"
Related
I want to launch and manage long running (several hours) shell commands on Linux, from a ruby script.
I need to be able to parse the output line by line, as they come. Not when the process is done.
I need to be able to kill the command and restart it if I don't like the output.
I also need to know if the process dies.
I found the right_popen gem but it hasn't been updated for 2 years and has no documentation. What would be the cleanest way to do all this?
I found a solution that allows me to do all the above, so I thought I'd share it with people who come after me :-)
We do it through pseudo-terminals, which are in stdlib (no gem required).
Here is an example of code that does what I need:
require 'pty'
cmd = 'for i in 1 2 3 4 5; do echo $i; sleep 1; done'
PTY.spawn cmd do |r, w, pid|
begin
r.sync
r.each_line do |l|
# Process each line immediately
line = l.strip
puts line
if line == '3'
# We kill the command on 3, because 3s are evil...
::Process.kill('SIGINT', pid)
end
end
rescue Errno::EIO => e
# Linux raises this error when the command ends
puts "COMMAND DIED"
ensure
# Wait for the process to end before we go on
::Process.wait pid
puts "WE ARE CLEAR"
end
end
Output:
1
2
3
COMMAND DIED
WE ARE CLEAR
I am trying to learn how to use the net-ssh gem for ruby. I want to execute the commands below, after I login to the directory - /home/james.
cd /
pwd
ls
When I do this with putty, it works and i can see a list of directories. But, when I do it with ruby code, it does not give me the the same output.
require 'rubygems'
require 'net/ssh'
host = 'server'
user = 'james'
pass = 'password123'
def get_ssh(host, user, pass)
ssh = nil
begin
ssh = Net::SSH.start(host, user, :password => pass)
puts "conn successful!"
rescue
puts "error - cannot connect to host"
end
return ssh
end
conn = get_ssh(host, user, pass)
def exec(linux_code, conn)
puts linux_code
result = conn.exec!(linux_code)
puts result
end
exec('cd /', conn)
exec('pwd', conn)
exec('ls', conn)
conn.close
Output -
conn successful!
cd /
nil
pwd
/home/james
ls
nil
I was expecting pwd to give me / instead of /home/james. That is how it works in putty. What is the mistake in the ruby code?
It seems like every command runs on it's own environment, so the current directory is not carried over exec to exec. You can verify this if you do:
exec('cd / && pwd', conn)
It will print /. It is not clear from the documentation how to make all the commands execute on the same environment or if this is even possible at all.
This is because net/ssh is stateless, so it opens a new connection with each command execution.
You can use the rye gem that implements a work around for this. but I do not know if it works with ruby > 2, since its development is not that active.
Another way is to use a pty process, in which you'll open a pseudo terminal with the ssh command, than use the input and output files to write commands for the terminal and read the results. To read the results you need to use the select method of the IO class. But you need to learn how to use those utilities since its not that obvious for a non experienced programmer.
And, Yey, I found how to do that, and in fact it is so simple. I think I did not get to this solution last time, because I was a little new to this thing of net-ssh, pty terminal. But yey, I found it finally, and here and example.
require 'net/ssh'
shell = {} #this will save the open channel so that we can use it accross threads
threads = []
# the shell thread
threads << Thread.new do
# Connect to the server
Net::SSH.start('localhost', 'your_user_name', password: 'your_password') do |session|
# Open an ssh channel
session.open_channel do |channel|
# send a shell request, this will open an interactive shell to the server
channel.send_channel_request "shell" do |ch, success|
if success
# Save the channel to be used in the other thread to send commands
shell[:ch] = ch
# Register a data event
# this will be triggered whenever there is data(output) from the server
ch.on_data do |ch, data|
puts data
end
end
end
end
end
end
# the commands thread
threads << Thread.new do
loop do
# This will prompt for a command in the terminal
print ">"
cmd = gets
# Here you've to make sure that cmd ends with '\n'
# since in this example the cmd is got from the user it ends with
#a trailing eol
shell[:ch].send_data cmd
# exit if the user enters the exit command
break if cmd == "exit\n"
end
end
threads.each(&:join)
and here we are, an interactive terminal using net-ssh ruby gem.
For more info look here its for the previous version 1, but it is so useful for you to understand how every piece works. And here
I have been trying to create a Ruby gem that simply exits my terminal whenever "x" is entered. Here is my main project file:
module SwissKnife
VERSION = '0.0.1'
class ConsoleUtility
def exit
`exit`
end
end
end
and my executable:
#!/usr/bin/env ruby
require 'swissknife'
util = SwissKnife::ConsoleUtility.new
util.exit
For some reason whenever I run this nothing appears to happen. I debugged it by adding in a simple puts 'Hello World!' in there, and it would print "Hello World!" but not exit. What am I doing wrong? Any help is greatly appreciated!
Backticks execute code in a new shell
exit isn't an executable that your shell runs, it's a special command understood by your shell - telling it to exit
So when you do
`exit`
it starts a shell which immediately exits. Not very useful. To exit the shell, you can instead kill Ruby's parent process.
Process.kill 'HUP', Process.ppid
I'm using:
- Ruby 1.9.3-p448
- Windows Server 2008
I have a file that contains commands that is used by a program, I'm using it in this way
C:\> PATH_TO_FOLDER/program.exe file.txt
File.txt have some commands so program.exe will do the following:
- Execute commands
- Reads from a DB using an ODBC method used by program
- Outputs result in a txt file
Using powershell this command works fine and as expected.
Now I have this in a file (app.rb):
require 'sinatra'
require 'open3'
get '/process' do
program_path = "path to program.exe"
file_name = "file.txt"
Open3.popen3(program_path, file_name) do |i, o, e, w|
# I have some commands here to execute but just as an example I'm using o.read
puts o.read
end
end
Now when using this by accessing http://localhost/process, Open3 works by doing this (I'm not 100% sure but after trying several times I think is the only option)
Reads commands and executes them (this is ok)
Tries to read from DB by using ODBC method (Here is my problem. I
need to receive some output from Open3 so I can show it in a browser, but I guess when it tries to read it starts another process that Open3 is not aware of, so Open3 goes on and finish without waiting for it)
Exits
Exits
I've found about following:
Use Thread.join (in this case, w.join) in order to wait for process to finish, but it doesn't work
Open4 seems to handle child process but doesn't work on Windows
Process.wait(pid), in this case pid = w.pid, but also doesn't work
Timeout.timeout(n), the problem here is that I'm not sure how long
will it take.
Is there any way of handling this? (waiting for Open3 subprocess so I get proper output).
We had a similar problem getting the exit status and this is what we did
Open3.popen3(*cmd) do |stdin, stdout, stderr, wait_thr|
# print stdout and stderr as it comes in
threads = [stdout, stderr].collect do |output|
Thread.new do
while ((line = output.gets rescue '') != nil) do
unless line.blank?
puts line
end
end
end
end
# get exit code as a Process::Status object
process_status = wait_thr.value #.exitstatus
# wait for logging threads to finish before continuing
# so we don't lose any logging output
threads.each(&:join)
# wait up to 5 minutes to make sure the process has really exited
Timeout::timeout(300) do
while !process_status.exited?
sleep(1)
end
end rescue nil
process_status.exitstatus.to_i
end
Using Open3.popen3 is easy only for trivial cases. I do not know the real code for handling the input, output and error channels of your subprocess. Neither do I know the exact behaviour of your subprocesses: Does it write on stdout? Does it write on stderr? Does it try to read from stdin?
This is why I assume that there are problems in the code that you replaced by puts o.read.
A good summary about the problems you can run into is on http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/63.
Though I disagree with the author of the article, Pavel Shved, when it comes to finding a solution. He recommends his own solution. I just use one of the wrapper functions for popen3 in my projects: Open3.capture*. They do all the difficult things like waiting for stdout and stderr at the same time.
I have one simple script:
fork do
STDOUT.reopen(File.open('/tmp/log', 'w+'))
STDOUT.sync = true
exec 'bundle exec ruby script.rb'
end
script.rb:
loop do
sleep 1
puts "MESSAGE"
end
When it work, all outputs is buffering(?) and writes to /tmp/log by big pices.
It works only if I modify script:
$stdout.puts "MESSAGE"
$stdout.flush
How can I do the same without modifying script.rb ?
Thanks.
When you call exec, you create a new process, and although this process inherits the file you set as standard out, it doesn't inherit the other settings, in particular the sync setting.
In order to get unbuffered output in the new process, you need to set it in that process. If you don't want to modify script.rb one workaround could be to create another file, named somethig like sync.rb containing just:
STDOUT.sync = true
which you can then require when running your command:
exec 'bundle exec ruby -r./sync script.rb'
The new Ruby process will now require sync.rb, which simply sets sync mode on STDOUT to true before executing your script.