I'm trying to write a little lib which generates a keypair using open4, is this the best way of dealing with system calls with multiple prompts?
require 'rubygems'
require 'open4'
Open4::popen4("sh") do |pid, stdin, stdout, stderr|
stdin.puts "openssl genrsa -des3 -out tmp_priv.pem 2048"
stdin.puts "1234"
stdin.puts "1234"
stdin.close
end
Open4::popen4("sh") do |pid, stdin, stdout, stderr|
stdin.puts "openssl rsa -in tmp_priv.pem -out tmp_public.pem -outform PEM -pubout"
stdin.puts "1234"
stdin.close
end
Open4::popen4("sh") do |pid, stdin, stdout, stderr|
stdin.puts "cat tmp_priv.pem tmp_public.pem >> tmp_keypair.pem"
stdin.close
end
I'm not sure your example is going to do what you want. If run as in your question openssl is going to open /dev/tty and it will end up prompting the user despite the pipe. It won't see the 1234.
If instead you run:
openssl genrsa -passout stdin ...
then in that case it will read stdin but it will only need the output file password once. And to answer the question you asked, yes, that's a good way, though it's not a system call.
It's also quite rare on Unix-like systems to need to fake up program input in the first place. You might want to reread the openssl(1ssl) and genrsa(1ssl) man pages; they will note various different password source options.
I've found that calling popen4 with the "block" syntax just doesnt work.
But what i've found works is to do this:
harp: > cat sample/simple.rb
require "open4"
pid, stdin, stdout, stderr = Open4:open4 "sh"
stdin.puts "echo 42.out"
stdin.puts "echo 42.err 1>&2"
stdin.close
ignored, status = Process::waitpid2 pid
puts "pid : #{ pid }"
puts "stdout : #{ stdout.read.strip }"
puts "stderr : #{ stderr.read.strip }"
puts "status : #{ status.inspect }"
puts "exitstatus : #{ status.exitstatus }"
harp: > ruby sample/simple.rb
pid : 17273
stdout : 42.out
stderr : 42.err
status : #<Process::Status: pid=17273,exited(0)>
exitstatus : 0
Which will work with passing the stdin. But at the same time, also giving back the stdout and stderr. And it avoided the exception:
in 'write': closed stream (IOError)
too. So this seems the best way to use popen4.
For other examples, see the README:
http://github.com/ahoward/open4
Related
So I'm having some trouble running my script.
My script has two commands : -e which is for encryption, and -d for decryption. The second ARGV is the key for the asymmetric cryptography and the string that it encrypt/decrypts is just static you will see it says 'Words and Stuff' in the code.
When I run the script it just pops up blank and the command is not ran, and when I do try to run -e for example ruby encryptor.rb -e sup3rS3cretKey it just says Invalid command '-e'; type "help" for a list. error in -e. So it seems its running openssl for some reason because of my require `openssl` statement and it does not operate my commands its seems as my script is not being ran from the terminal. So how do I fix this, and what is the openssl thing that it is doing called?
-Script
require `openssl`
if ARGV[0] == '-e' #Encrypt
if ARGV.length != 2
puts "Please input a key."
exit
end
puts "Encrypting"
key = ARGV[1]
cipher = OpenSSL::Cipher.new('Words and Stuff').encrypt
cipher.key = Digest::SHA1.hexdigest key
s = cipher.update(self) + cipher.final
s.unpack('H*')[0].upcase
puts "Encrypted"
elsif ARGV[0] == '-d' #Decrypt
if ARGV.length != 2
puts "Please input a key."
exit
end
puts "Decrypting"
key = ARGV[1]
cipher = OpenSSL::Cipher.new('Words and Stuff').decrypt
cipher.key = Digest::SHA1.hexdigest key
s = [self]/pack("H*").unpack("C*").pack("c*")
cipher.update(s) + cipher.final
puts "String decrypted."
end
Your initial problem is that you are requiring 'openssl' with backticks so it is trying to call it in the shell. Swap the backticks for normal quotes and you will make it past that line.
irb is very handy for these situations. Just type irb and enter require "openssl" at the prompt. Then you can test things out a line at a time.
Here are a few issues:
Backticks around openssl is hanging the script(try single or double quotes as noted).
"Words and Stuff" is not a supported algorithm(try AES-128-CBC).
cipher.key wants a 16 byte string(I just stuck "sup3rS3cretKeyyz" in there)
I'm writing a Thor script to run some tests from a different tool i.e. running a shell command. I'd like the stdout and stderr from the command to continuously stream out into my console.
First attempt was to just use backticks, but naturally the stdout/stderr are not printed (rather, stdout is captured in the return value).
desc "mytask", "my description"
def mytask
`run-my-tests.sh`
end
My next approach was to use Open3 as in:
require "open3"
desc "mytask", "my description"
def mytask
Open3.popen3("run-my-tests.sh") do |stdin, stdout, stderr|
STDOUT.puts(stdout.read())
STDERR.puts(stderr.read())
end
end
However, the above approach will get the whole output from both stdout and stderr and only print at the end. Un my use case, I'd rather see the output of failing and passing tests as it becomes available.
From http://blog.bigbinary.com/2012/10/18/backtick-system-exec-in-ruby.html, I saw that we can read the streams by chunks i.e. with gets() instead of read(). For example:
require "open3"
desc "mytask", "my description"
def mytask
Open3.popen3(command) do |stdin, stdout, stderr|
while (out = stdout.gets()) || err = (stderr.gets())
STDOUT.print(out) if out
STDERR.print(err) if err
end
exit_code = wait_thr.value
unless exit_code.success?
raise "Failure"
end
end
end
Does it look like the best and cleanest approach? Is it an issue that I have to manually try to print stdout before stderr?
I'm using IO.popen for similar task, like so:
IO.popen([env, *command]) do |io|
io.each { |line| puts ">>> #{line}" }
end
To capture stderr I'd just redirect it to stdout command = %w(run-my-tests.sh 2>&1)
Update
I've constructed a script using Open3::popen3 to capture stdout and stderr separately. It obviously has a lot of room form improvement, but basic idea hopefully is clear.
require 'open3'
command = 'for i in {1..5}; do echo $i; echo "$i"err >&2; sleep 0.5; done'
stdin, stdout, stderr, _command_thread = Open3.popen3(command)
reading_thread = Thread.new do
kilobyte = 1024
loop do
begin
stdout.read_nonblock(kilobyte).lines { |line| puts "stdout >>> #{line}" }
stderr.read_nonblock(kilobyte).lines { |line| puts "stderr >>> #{line}" }
rescue IO::EAGAINWaitReadable
next
rescue EOFError
break
end
sleep 1
end
end
reading_thread.join
stdin.close
stdout.close
stderr.close
Seems to me like the simplest way to run a shell command and not try to capture the stdout or stderr (instead, let them bubble up as they come) was something like:
def run *args, **options
pid = spawn(*args, options)
pid, status = Process.wait2(pid)
exit(status.exitstatus) unless status.success?
end
The problem with backticks or system() is that the former captures the stdout and the latter only returns whether the command succeeded or not. spawn() is a more informative alternative to system(). I'd rather have my Thor script tool fail as if it was merely a wrapper for those shell commands.
This code is not working properly:
my_command = %x{java -version &>/dev/null}
if $? != 0
How do I see what the STDERR/STDOUT status code is?
I tried using puts:
puts "values ===> $? or or '$?' my_command or #{my_command} %x{echo $?}"
but it doesn't work.
Using %x or something equivalent is an easy way of doing this but doesn't give you a lot of control. Instead use the Open3 library and popen3:
require 'open3'
Open3.popen3("java", "-version") do |stdin, stdout, stderr, wait_thr|
version = stdout.chomp
status = wait_thr.value
end
To get the status code :
stdout = %x(ls -l)
exit_code = $?.exitstatus
and then
if exit_code == 0 then
puts "Success"
else
puts "Problem"
end
Goal: I am writing a workflow command-line program in ruby that sequentially executes other programs on the UNIX shell, some of which require the user to enter input.
Problem: Although I can successfully handle the stdout and stderr thanks to this helpful blog post by Nick Charlton, I am however stuck on capturing user input and passing it into the sub-processes via the command line. The code is as follows:
Method
module CMD
def run(cmd, &block)
Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
Thread.new do # STDOUT
until (line = stdout.gets).nil? do
yield nil, line, nil, thread if block_given?
end
end
Thread.new do # STDERR
until (line = stderr.gets).nil? do
yield nil, nil, line, thread if block_given?
end
end
Thread.new do # STDIN
# ????? How to handle
end
thread.join
end
end
end
Calling the method
This example calls the shell command units which prompts the user to enter a unit of measurement and then prompts for a unit to convert to. This is how it would look in the shell
> units
586 units, 56 prefixes # stdout
You have: 1 litre # user input
You want: gallons # user input
* 0.26417205 # stdout
/ 3.7854118 # stdout
When I run this from my program I expect to be able to interact with it in exactly the same way.
unix_cmd = 'units'
run unix_cmd do | stdin, stdout, stderr, thread|
puts "stdout #{stdout.strip}" if stdout
puts "stderr #{stderr.strip}" if stderr
# I'm unsure how I would allow the user to
# interact with STDIN here?
end
Note: Calling the run method this way allows the user to be able to parse the output, control process flow and add custom logging.
From what I've gathered about STDIN, the snippet below is as close as I've come in understanding how to handle STDIN, there are clearly some gaps in my knowledge because I'm still unsure how to integrate this into my run method above and pass the input into the child process.
# STDIN: Constant declared in ruby
# stdin: Parameter declared in Open3.popen3
Thread.new do
# Read each line from the console
STDIN.each_line do |line|
puts "STDIN: #{line}" # print captured input
stdin.write line # write input into stdin
stdin.sync # sync the input into the sub process
break if line == "\n"
end
end
Summary: I wish to understand how to handle user input from the command-line via the Open3.popen3 method so that I can allow users to enter data into various sequence of sub-commands called from my program.
Here's something that should work:
module CMD
def run(cmd, &block)
Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
Thread.new do # STDOUT
until (line = stdout.gets).nil? do
yield nil, line, nil, thread if block_given?
end
end
Thread.new do # STDERR
until (line = stderr.gets).nil? do
yield nil, nil, line, thread if block_given?
end
end
t = Thread.new { loop { stdin.puts gets } }
thread.join
t.kill
end
end
end
I've just added two lines to your original run method: t = Thread.new { loop { stdin.puts gets } }, and t.kill.
After a lot of reading about STDIN as well as some good old trial and error, I discovered an implementation not to dissimilar to Charles Finkel's answer but with some subtle differences.
require "open3"
module Cmd
def run(cmd, &block)
Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
# We only need to check if the block is provided once
# rather than every cycle of the loop as we were doing
# in the original question.
if block_given?
Thread.new do
until (line = stdout.gets).nil? do
yield line, nil, thread
end
end
Thread.new do
until (line = stderr.gets).nil? do
yield nil, line, thread
end
end
end
# $stdin.gets reads from the console
#
# stdin.puts writes to child process
#
# while thread.alive? means that we keep on
# reading input until the child process ends
Thread.new do
stdin.puts $stdin.gets while thread.alive?
end
thread.join
end
end
end
include Cmd
Calling the method like so:
run './test_script.sh' do | stdout, stderr, thread|
puts "#{thread.pid} stdout: #{stdout}" if stdout
puts "#{thread.pid} stderr: #{stderr}" if stderr
end
Where test_script.sh is as follows:
echo "Message to STDOUT"
>&2 echo "Message to STDERR"
echo "enter username: "
read username
echo "enter a greeting"
read greeting
echo "$greeting $username"
exit 0
Produces the following successful output:
25380 stdout: Message to STDOUT
25380 stdout: enter username:
25380 stderr: Message to STDERR
> Wayne
25380 stdout: enter a greeting
> Hello
25380 stdout: Hello Wayne
Note: You will notice the stdout and stderr don't appear in order, this is a limitation I'm yet to solve.
If you're interested in knowing more about stdin it's worth reading the following answer to the question - What is the difference between STDIN and $stdin in Ruby?
I'd like to programmatically check if someone has their SSH keys set up correctly for GitHub. I understand that I can use `ssh -T git#github.com` in Ruby. However, I'd like to keep the ssh output in a variable.
My current code is:
github_response = `ssh -T git#github.com`
unless github_response.start_with?('Hi')
puts 'Please set up your GitHub ssh keys'
end
`ssh -T git#github.com` outputs the response (starting with "Hi"). However the github_response variable is nil.
How can I assign the output of `ssh -T git#github.com` to github_response?
Your example failed because the Hi xxx! You've successfully authenticated.... message is not from stdout, but stderr.
> require 'open3'
=> true
> stdin, stdout, stderr, wait_thr = Open3.popen3('ssh -T git#github.com')
=> [#<IO:fd 8>, #<IO:fd 9>, #<IO:fd 11>, #<Thread:0x007f89ee1149a8 sleep>]
> stdout.gets
=> nil
> stderr.gets
=> "Hi halfelf! You've successfully authenticated, but GitHub does not provide shell access.\n"
You could add -v for verbose output, it will then dump much of the connection info to stdout. From that log you can scrape to find whether the server accepted any of the keys the ssh client offered