I have an IO object (like $stdout, $stderr). I can do read.empty? on it to see if it is empty, but is it possible to know if it is empty without reading it? I know that File has size, but IO does not.
Edit
Sorry if my question was not clear enough. I get these objects as return values from Open3.popen3. I want to see if anything was written to the standard error, for example. And I want to do it without reading it if possible.
That's a purpose of IO::select method:
Updated example after question edited:
require 'open3'
select_timeout = Rational(1,10) # optional
Open3.popen3('ls /') do
|stdin, stdout, stderr, w_thread|
rdin, rdout, rderr = IO.select([stdin], [stdout], [stderr], select_timeout)
if rdin
p 'no data on standard input' unless rdin.member? stdin
p 'no data on standard output' unless rdout.member? stdout
p 'no data on standard error' unless rderr.member? stderr
else
p 'none of streams has data available'
end
retval = w_thread.value
end
# "no data on standard output"
# "no data on standard error"
Whereas only standard input contains data from spawned thread command ls /
Related
I have a basic understanding of popen before now but it seems it has changed completely.
Please refer the example to know why?
# process.rb
IO.popen("ruby test_ex.rb","w") do |io|
io.write("#{Process.pid} hello")
io.close_write
## this does not work.
##io.readlines
end
## text_ex.rb
def readWrite
#string = gets()
puts "#{Process.pid} -- #{#string}"
end
readWrite
Now I understand in write mode the STDOUT(of popen.rb) will be writable end of the pipe and STDIN (of text_ex.rb) will be the readable end of the pipe.
All is good here.
But let see the other example
my_text = IO.popen("ssh user#host 'bash'", "w+")
my_text.write("hostname")
my_text.close_write
my_rtn = my_text.readlines.join('\n')
my_text.close
puts my_rtn
Ok, now what is different over here?
The popen start a child process(i.e ssh) send the hostname.
Now, I fail to understand how does the STDOUT of the child process(i.e ssh) is available to the parent process i.e how does the readlines work over here and does not work in my earlier example.
Thanks
The difference is in the second argument to popen: "w" versus "w+". You can read more here in the docs:
"w" Write-only, truncates existing file
to zero length or creates a new file for writing.
"w+" Read-write, truncates existing file to zero length
or creates a new file for reading and writing.
The notion of "truncating" doesn't really apply to pipes, but the fact that you need read-write mode does.
Suppose I have a method which computes hard problems (maybe a depth-first search in a huge graph), we can call this method dfs(graph).
This method also output to stdout each result reached, using puts result.
def dfs(graph)
while true
# lots of computation
result = something_reached
puts result
end
end
I want to display a progressbar in shell to show that computation is running, so I add an instance like this:
pbar = ProgressBar.create(title: "Computing", starting_at: 1, total: nil)
and progressbar status must be updated (pbar.increment) as computation is running.
In a shell, I execute my program like this:
ruby dfs.rb > dfs_results.txt
Issues:
With pbar is flushed to stdout so prograss-bar is redirected to dfs_results.txt and computation results are not store in this file.
Without pbar result data is stored in file as is expected, but obviously without progress-bar.
I know that results could be stored with File.open usage, but it is desirable done with shell redirections.
Question:
How should I be implemented to flush computation resulta to dfs_results.txt file and show a progress-bar to keep a executing progress for user?
Just open a file to log into:
log = File.open('dfs_results.txt', 'w')
# write into log file
log.write(result)
And your progressbar still writes to stdout. When you open a second screen you can follow the output with:
tails -f dfs_results.txt
Update: Or you can use stdout for the progressbar and stderr for the output of the script. Instead of puts result write:
$stderr.puts results
and start the script with:
ruby dfs.rb 2> dfs_results.txt
I still think the first version is better because results are not errors...
I'm trying to open a Ruby script that prompts for stuff with popen3, for testing purposes.
I need to be able to type into the STDIN of the proccess and inspect the outputs. So far I have this:
require 'open3'
def run(executable, opts = {})
Signal.trap('CLD') do
puts 'STDIN:', #stdout.readlines
# #stderr.rewind
puts 'STDERR:', #stderr.readlines
# got EOF, trying to rewind gives me 'Errno::ESPIPE: Illegal seek'
[#stdin, #stdout, #stderr].each &:close
end
#stdin, #stdout, #stderr, #thread = Open3.popen3(executable)
#status = #thread.value
end
Have been trying for days, my brain is melting.
The things you are allowed to do in a signal handler are very limited. For example, you are not allowed to do I/O in a signal handler. Signal handlers are notoriously easy to do wrong.
What do you think about using Open3.capture*? It avoids all blocks/race conditions when reading from stdout and stderr at the same time. Besides, it allows you to pass data for stdin. Use the keyword argument :stdin_data!
I'm building a daemon that will help me manage my server(s). Webmin works fine, as does just opening a shell to the server, but I'd prefer to be able to control server operations from a UI I design, and also expose some functionality to end users.
The daemon will pick up actions from a queue and execute them. However, since I'll be accepting input from users, I want to make sure they're not permitted to inject something dangerous into a privileged shell command.
Here's a fragment that exemplifies my problem:
def perform
system "usermod -p #{#options['shadow']} #{#options['username']}"
end
A gist that explains more: https://gist.github.com/773292
I'm not positive if typical escaping and sanitizing of inputs is enough for this case, and being a designer, I don't have a ton of security-related experience. I know that this is something that should probably be obvious to me, but its not!
How can I ensure that the web application that will create and serialize the actions can't pass dangerous text into the privileged process that receives the actions?
Thanks for the help
arb
It doesn't look like you need a shell for what you're doing. See the documentation for system here: http://ruby-doc.org/core/classes/Kernel.html#M001441
You should use the second form of system. Your example above would become:
system 'usermod', '-p', #options['shadow'], #options['username']
A nicer (IMO) way to write this is:
system *%W(usermod -p #{#options['shadow']} #{#options['username']})
The arguments this way are passed directly into the execve call, so you don't have to worry about sneaky shell tricks.
If you need not just the exit status but also the result you probably want to use Open3.popen3:
require 'open3'
stdin, stdout, stderr = Open3.popen3('usermod', '-p', #options['shadow'], #options['username'])
stdout.gets
sterr.gets
More information here: Getting output of system() calls in Ruby
I'd suggest looking into the 'shellwords' module. This script:
require 'shellwords'
parts = ['echo', "'hello world'; !%& some stuff", 'and another argument']
command = Shellwords.shelljoin( parts )
puts command
output = `#{ command }`
puts output
outputs the escaped text and the expected output:
echo \'hello\ world\'\;\ \!\%\&\ some\ stuff and\ another\ argument
'hello world'; !%& some stuff and another argument
This is an old question, but since it's pretty much the only real answer you'll find when googling I thought I'd add a caveat. The multi argument version of system seems reasonably safe on Linux, but it is NOT on Windows.
Try system "dir", "&", "echo", "hi!"
on a Windows system. Both dir and echo will be run. Echo could of course just as well be something far less innocuous.
I know this is an old thread, but there is another option that was lightly touched on by Simon Hürlimann.
There is not a lot of information about this topic and I think this might help others in need.
For this example we'll use Open3 which gives you the ability to run commands synchronously or asynchronously, and provides stdout, stderr, exit codes, and PID.
Open3 grants you access to stdout, stderr, exit codes and a thread to wait for the child process when running another program. You can specify various attributes, redirections, current directory, etc., of the program in the same way as for Process.spawn. (Source: Open3 Docs)
I chose to format the output as a CommandStatus object. This contains our stdout, stderr, pid (Of the worker thread) and exitstatus.
class Command
require 'open3'
class CommandStatus
#stdout = nil
#stderr = nil
#pid = nil
#exitstatus = nil
def initialize(stdout, stderr, process)
#stdout = stdout
#stderr = stderr
#pid = process.pid
#exitstatus = process.exitstatus
end
def stdout
#stdout
end
def stderr
#stderr
end
def exit_status
#exitstatus
end
def pid
#pid
end
end
def self.execute(command)
command_stdout = nil
command_stderr = nil
process = Open3.popen3(ENV, command + ';') do |stdin, stdout, stderr, thread|
stdin.close
stdout_buffer = stdout.read
stderr_buffer = stderr.read
command_stdout = stdout_buffer if stdout_buffer.length > 0
command_stderr = stderr_buffer if stderr_buffer.length > 0
thread.value # Wait for Process::Status object to be returned
end
return CommandStatus.new(command_stdout, command_stderr, process)
end
end
cmd = Command::execute("echo {1..10}")
puts "STDOUT: #{cmd.stdout}"
puts "STDERR: #{cmd.stderr}"
puts "EXIT: #{cmd.exit_status}"
While reading the STDOUT/ERR buffers, I use command_stdout = stdout_buffer if stdout_buffer.length > 0 to control whether the command_stdout variable is assigned or not. You should pass nil instead of "" when no data is present. It's more clear when handing data later on.
You probably noticed me using command + ';'. The reason for this is based on the documentation from Kernel.exec (Which is what popen3 uses):
If the string from the first form (exec("command")) follows these
simple rules:
no meta characters
no shell reserved word and no special built-in
Ruby invokes the command directly without shell
You can force shell invocation by adding ";" to the string (because
";" is a meta character)
This simply prevents a Ruby from throwing a 'spawn': No such file or directory error if you pass a malformed command. Instead it will pass it straight to the kernel where the error will be resolved gracefully and appear as STDERR instead of an uncaught exception.
Modern, secure and simple solution (popen will escape arguments for you):
IO.popen(['usermod', '-p', #options['shadow'], #options['username']]).read
(#read will close the IO before returning)
I'm looking to expose an interactive command line program via JSON or another RPC style service using Ruby. I've found a couple tricks to do this, but im missing something when redirecting the output and input.
One method at least on linux is to redirect the stdin and stdout to a file then read and write to that file asynchronously with file reads and writes. Another method ive been trying after googling around was to use open4. Here is the code I wrote so far, but its getting stuck after reading a few lines from standard output.
require "open4"
include Open4
status = popen4("./srcds_run -console -game tf +map ctf_2fort -maxplayers 6") do |pid, stdin, stdout, stderr|
puts "PID #{pid}"
lines=""
while (line=stdout.gets)
lines+=line
puts line
end
while (line=stderr.gets)
lines+=line
puts line
end
end
Any help on this or some insight would be appreciated!
What I would recommend is using Xinetd (or similar) to run the command on some socket and then using the ruby network code. One of the problems you've already run into in your code here is that your two while loops are sequential, which can cause problems.
Another trick you might try is to re-direct stderr to stdout in your command, so that your program only has to read the stdout. Something like this:
popen4("./srcds_run -console -game tf +map ctf_2fort -maxplayers 6 2>&1")
The other benefit of this is that you get all the messages/errors in the order they happen during the program run.
EDIT
Your should consider integrating with AnyTerm. You can then either expose AnyTerm directly e.g. via Apache mod_proxy, or have your Rails controller act as the reverse proxy (handling authentication/session validation, then playing back controller.request minus any cookies to localhost:<AnyTerm-daemon-port>, and sending back as a response whatever AnyTerm replies with.)
class ConsoleController < ApplicationController
# AnyTerm speaks via HTTP POST only
def update
# validate session
...
# forward request to AnyTerm
response = Net::HTTP.post_form(URI.parse('http://localhost:#{AnyTermPort}/', request.params))
headers['Content-Type'] = response['Content-Type']
render_text response.body, response.status
end
Otherwise, you'd need to use IO::Select or IO::read_noblock to know when data is available to be read (from either network or subprocess) so you don't deadlock. See this too. Also check that either your Rails is used in a multi-threaded environment or that your Ruby version is not affected by this IO::Select bug.
You can start with something along these lines:
status = POpen4::popen4("ping localhost") do |stdout, stderr, stdin, pid|
puts "PID #{pid}"
# our buffers
stdout_lines=""
stderr_lines=""
begin
loop do
# check whether stdout, stderr or both are
# ready to be read from without blocking
IO.select([stdout,stderr]).flatten.compact.each { |io|
# stdout, if ready, goes to stdout_lines
stdout_lines += io.readpartial(1024) if io.fileno == stdout.fileno
# stderr, if ready, goes to stdout_lines
stderr_lines += io.readpartial(1024) if io.fileno == stderr.fileno
}
break if stdout.closed? && stderr.closed?
# if we acumulated any complete lines (\n-terminated)
# in either stdout/err_lines, output them now
stdout_lines.sub!(/.*\n/m) { puts $& ; '' }
stderr_lines.sub!(/.*\n/m) { puts $& ; '' }
end
rescue EOFError
puts "Done"
end
end
To also handle stdin, change to:
IO.select([stdout,stderr],[stdin]).flatten.compact.each { |io|
# program ready to get stdin? do we have anything for it?
if io.fileno == stdin.fileno && <got data from client?>
<write a small chunk from client to stdin>
end
# stdout, if ready, goes to stdout_lines