This code is not working properly:
my_command = %x{java -version &>/dev/null}
if $? != 0
How do I see what the STDERR/STDOUT status code is?
I tried using puts:
puts "values ===> $? or or '$?' my_command or #{my_command} %x{echo $?}"
but it doesn't work.
Using %x or something equivalent is an easy way of doing this but doesn't give you a lot of control. Instead use the Open3 library and popen3:
require 'open3'
Open3.popen3("java", "-version") do |stdin, stdout, stderr, wait_thr|
version = stdout.chomp
status = wait_thr.value
end
To get the status code :
stdout = %x(ls -l)
exit_code = $?.exitstatus
and then
if exit_code == 0 then
puts "Success"
else
puts "Problem"
end
Related
I'm writing a Thor script to run some tests from a different tool i.e. running a shell command. I'd like the stdout and stderr from the command to continuously stream out into my console.
First attempt was to just use backticks, but naturally the stdout/stderr are not printed (rather, stdout is captured in the return value).
desc "mytask", "my description"
def mytask
`run-my-tests.sh`
end
My next approach was to use Open3 as in:
require "open3"
desc "mytask", "my description"
def mytask
Open3.popen3("run-my-tests.sh") do |stdin, stdout, stderr|
STDOUT.puts(stdout.read())
STDERR.puts(stderr.read())
end
end
However, the above approach will get the whole output from both stdout and stderr and only print at the end. Un my use case, I'd rather see the output of failing and passing tests as it becomes available.
From http://blog.bigbinary.com/2012/10/18/backtick-system-exec-in-ruby.html, I saw that we can read the streams by chunks i.e. with gets() instead of read(). For example:
require "open3"
desc "mytask", "my description"
def mytask
Open3.popen3(command) do |stdin, stdout, stderr|
while (out = stdout.gets()) || err = (stderr.gets())
STDOUT.print(out) if out
STDERR.print(err) if err
end
exit_code = wait_thr.value
unless exit_code.success?
raise "Failure"
end
end
end
Does it look like the best and cleanest approach? Is it an issue that I have to manually try to print stdout before stderr?
I'm using IO.popen for similar task, like so:
IO.popen([env, *command]) do |io|
io.each { |line| puts ">>> #{line}" }
end
To capture stderr I'd just redirect it to stdout command = %w(run-my-tests.sh 2>&1)
Update
I've constructed a script using Open3::popen3 to capture stdout and stderr separately. It obviously has a lot of room form improvement, but basic idea hopefully is clear.
require 'open3'
command = 'for i in {1..5}; do echo $i; echo "$i"err >&2; sleep 0.5; done'
stdin, stdout, stderr, _command_thread = Open3.popen3(command)
reading_thread = Thread.new do
kilobyte = 1024
loop do
begin
stdout.read_nonblock(kilobyte).lines { |line| puts "stdout >>> #{line}" }
stderr.read_nonblock(kilobyte).lines { |line| puts "stderr >>> #{line}" }
rescue IO::EAGAINWaitReadable
next
rescue EOFError
break
end
sleep 1
end
end
reading_thread.join
stdin.close
stdout.close
stderr.close
Seems to me like the simplest way to run a shell command and not try to capture the stdout or stderr (instead, let them bubble up as they come) was something like:
def run *args, **options
pid = spawn(*args, options)
pid, status = Process.wait2(pid)
exit(status.exitstatus) unless status.success?
end
The problem with backticks or system() is that the former captures the stdout and the latter only returns whether the command succeeded or not. spawn() is a more informative alternative to system(). I'd rather have my Thor script tool fail as if it was merely a wrapper for those shell commands.
I'm writing a function to execute a shell command, and returns its exit code, STDOUT and STDERR.
The problem is, this function cannot capture STDOUT and STDERR output properly.
def sh(*args)
options = args[-1].respond_to?(:to_hash) ? args.pop.to_hash: {}
options = { :timeout => 0, :sudo => false }.merge(options)
cmd = options[:sudo] == false ? args[0] : "sudo " << args[0]
begin
stdin, stdout, stderr, wait_thr = Open3.popen3(cmd)
pid = wait_thr[:pid]
out_buf = ""
err_buf = ""
start = Time.now
# Manually ping the process per 0.2 second to check whether the process is alive or not
begin
out_buf << stdout.read_nonblock(4096)
err_buf << stderr.read_nonblock(4096)
# kill the process if it timeouts
if options[:timeout] != 0 && (Time.now - start) > options[:timeout]
Process.kill("KILL", pid)
Process.detach(pid)
raise RuntimeError, "process with pid #{pid} timed out with #{options[:timeout]} seconds."
end
sleep 0.2
rescue IO::WaitReadable, EOFError
end while wait_thr.alive?
rescue => e
NtfLogger.warn("sh '#{args}' executed with failure: #{e}")
ensure
if wait_thr.nil?
return 1, out_buf, err_buf
else
return wait_thr.value.exitstatus, out_buf, err_buf
end
end
end # end of sh
Could anybody please help me figure out what the problem is?
My understanding of popen3's docs are that it's better to do the processing within a block:
Open3.popen3([env,] cmd... [, opts]) do |stdin, stdout, stderr, wait_thr|
pid = wait_thr.pid # pid of the started process.
...
exit_status = wait_thr.value # Process::Status object returned.
end
In non-block form, the docs note that the streams must be closed:
stdin, stdout, stderr, wait_thr = Open3.popen3([env,] cmd... [, opts])
pid = wait_thr[:pid] # pid of the started process.
...
stdin.close # stdin, stdout and stderr should be closed explicitly in this form.
stdout.close
stderr.close
exit_status = wait_thr.value # Process::Status object returned.
http://www.ruby-doc.org/stdlib-2.0.0/libdoc/open3/rdoc/Open3.html#method-c-popen3
Lastly, FWIW, here's a wrapper around capture3 which I use on my end. You could easily extend it to add a sudo option, in case the thread-related parts of your sh utility aren't critical:
#
# Identical to Open3.capture3, except that it rescues runtime errors
#
# #param env optional (as `Kernel.system')
# #param *cmd the command and its (auto-escaped) arguments
# #param opts optional a hash of options (as `Kernel.system')
#
# #return [stdout, stderr, success] | [$/, $/, nil] on error
#
def system3(*cmd)
begin
stdout, stderr, status = Open3.capture3(*cmd)
[stdout, stderr, status.success?]
rescue
[$/, $/, nil]
end
end
I'm looking for something equivalent of the backticks operator (``) with the capability to display output during shell command execution.
I saw a solution in another post:
(Running a command from Ruby displaying and capturing the output)
output = []
IO.popen("ruby -e '3.times{|i| p i; sleep 1}'").each do |line|
p line.chomp
output << line.chomp
end
p output
This solution doesn't fit my needs since $? remains nil after the shell command execution. The solution I'm looking for should also set $? (returning the value of $?.exitstatus in another way is also sufficient)
Thanks!
First, I'd recommend using one of the methods in Open3.
I use capture3 for one of my systems where we need to grab the output of STDOUT and STDERR of a lot of command-line applications.
If you need a piped sub-process, try popen3 or one of the other "pipeline" commands.
Here's some code to illustrate how to use popen2, which ignores the STDERR channel. If you want to track that also use popen3:
require 'open3'
output = []
exit_status = Open3.popen2(ENV, "ruby -e '3.times{|i| p i; sleep 1}'") { |stdin, stdout, thr|
stdin.close
stdout.each_line do |o|
o.chomp!
output << o
puts %Q(Read from pipe: "#{ o }")
end
thr.value
}
puts "Output array: #{ output.join(', ') }"
puts "Exit status: #{ exit_status }"
Running that outputs:
Read from pipe: "0"
Read from pipe: "1"
Read from pipe: "2"
Output array: 0, 1, 2
Exit status: pid 43413 exit 0
The example code shows one way to do it.
It's not necessary to use each_line, but that demonstrates how you can read line-by-line until the sub-process closes its STDOUT.
capture3 doesn't accept a block; It waits until the child has closed its output and exits, then it returns the content, which is great when you want a blocking process. popen2 and popen3 have blocking and non-blocking versions, but I show only the non-blocking version here to demonstrate how to read and output the content as it comes in from the sub-process.
Try following:
output = []
IO.popen("ruby -e '3.times{|i| p i; sleep 1 }'") do |f|
f.each do |line|
p line.chomp
output << line.chomp
end
end
p $?
prints
"0"
"1"
"2"
#<Process::Status: pid 2501 exit 0>
Using open3
require 'open3'
output = []
Open3.popen2("ruby -e '3.times{|i| p i; sleep 1}'") do |stdin,stdout,wait_thr|
stdout.each do |line|
p line.chomp
output << line.chomp
end
p wait_thr.value
end
I'm using IO.popen to start a subprocess, but I only get the result of everything that happened in the time it took for the subprocess to run (sometimes 5 minutes or whatever) when the subprocess exits. I really need to be able to see everything the subprocess writes to stderr and stdout as-and-when it happens.
So far I could not find anything that works like this, but I'm sure it's possible.
if you need to get output in real time i would recommend to use stdlib PTY instead of popen
something like this:
require 'pty'
cmd = 'echo a; sleep 1; cat /some/file; sleep 1; echo b'
PTY.spawn cmd do |r, w, pid|
begin
r.sync
r.each_line { |l| puts "#{Time.now.strftime('%M:%S')} - #{l.strip}" }
rescue Errno::EIO => e
# simply ignoring this
ensure
::Process.wait pid
end
end
exit "#{cmd} failed" unless $? && $?.exitstatus == 0
> 33:36 - a
> 33:37 - cat: /some/file: No such file or directory
> 33:38 - b
this way you get output instantly, just as in terminal
You might want to use Open3.popen3 from standard library, it gives access to stdin, stdout, and stderr as streams.
When I do something like the following:
output = `identify some_file`
output == "Output of identify"
But when...
output = `identify non_existant_file`
output != "Error output of identify"
How can I get the error output of system calls?
I found out the answer. The output is being sent to stderr. So I can just add the following at the end of the command to redirect stderr to stdout:
output = `identify any_file 2>&1`
output == "Error or output of identify"
Here is the explanation of this witchcraft
You may use Open3.popen3.
http://www.ruby-doc.org/stdlib-1.9.3/libdoc/open3/rdoc/Open3.html#method-c-popen3
popen3(*cmd, &block) click to toggle source
Open stdin, stdout, and stderr streams and start external executable.
Open3.popen3([env,] cmd... [, opts]) {|stdin, stdout, stderr, wait_thr|
pid = wait_thr.pid # pid of the started process.
...
exit_status = wait_thr.value # Process::Status object returned.
}