Is there an Expect equivalent gem for Ruby?
I tried searching on code.google and rubygems.org, but sadly it did not show up.
FYI: Expect is a Unix automation and testing tool, written by Don Libes as an extension to the Tcl scripting language, for interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, ssh, and others.
Ruby comes with the PTY module for setting up pseudoterminals to drive interactive command line applications. With it comes an expect method that allows you to interact with an application kinda like Expect. For learning how to use expect, I found "What to expect from the Ruby expect library?" helpful.
As far as gems go, maybe checkout greenletters which is supposed to improve upon PTY + expect (although I haven't tried it myself).
I recently spent quite a bit of time struggling with this issue (I am stuck with 1.8.7). I found this question, this blog post and this forum thread really useful.
At the end this is my application code if anyone is interested in a little example (pass the password to rpm when signing packages):
def run_interactive command, password, promt
output = ''
begin
r, w, pid = PTY.spawn(command)
puts r.expect(promt)
sleep(0.5)
w.puts(password)
begin
r.each { |l| output += l }
rescue Errno::EIO
end
$?.exitstatus
Process.wait(pid)
rescue PTY::ChildExited => e
$stderr.puts "The child process #{e} exited! #{$!.status.exitstatus}"
end
output
end
password = "mypassword"
command = "rpm --define '_signature gpg' --define '_gpg_name #{key_id}' --addsign #{package}"
promt = %r{.*: }
expected = %r{good}
output = run_interactive(command, password, promt)
if output.match(expected)
puts output
else
abort "Error: expected: '#{expected}' got '#{output}'"
end
It has little error checking but it was all I needed.
Edit: Update the code with Process.wait(pid) to make sure it finishes before continuing and add comment about this being for 1.8.7.
checkout this rubygem: https://github.com/abates/ruby_expect. It could handle some small task for you. from its official example, it's enough to 'enter password' and login and interactive with local script.
here is an example that update the git code (which is authenticated with password):
require 'rubygems'
require 'ruby_expect'
def update_code
password = 'your password here'
exp = RubyExpect::Expect.spawn('git pull', :debug => true)
exp.procedure do
each do
expect /password: / do
send password
end
end
end
end
update_code
just run the code above, and your will see like this:
$ ruby update_code.rb
shensiwei#gforge.1ver??.net's password:
remote: Counting objects: 133, done.
remote: Compressing objects: 100% (84/84), done.
remote: Total 85 (delta 62), reused 0 (delta 0)
Unpacking objects: 100% (85/85), done.
for more example and details, please dive into its source code.
expect4r seems to do what you are asking for, though it is made specific for connections to Cisco and Juniper devices.
Perhaps even better is yax as this is "yet another expect".
RExpect
From the project's website:
RExpect is a drop in replacement for the expect.rb module in the
standard library that is faster and more robust, cabable of driving
many devices simultaneously.
parley is another one you can try, (written by me). It is inspired by Perl expect.
Related
What I'm trying to achieve:
From a Ruby process, spawning a subprocess
The subprocess should print as normal back to the terminal. By "normal", I mean the process shouldn't miss out color output, or ignore user input (STDIN).
For that subprocess, capturing STDOUT/STDERR (jointly) e.g. into a String variable that can be accessed after the subprocess is dead. Escape characters and all.
Capturing STDOUT/STDERR is possible by passing a different IO pipe, however the subprocess can then detect that it's not in a tty. For example git log will not print characters that influence text color, nor use it's pager.
Using a pty to launch the process essentially "tricks" the subprocess into thinking it's being launched by a user. As far as I can tell, this is exactly what I want, and the result of this essentially ticks all the boxes.
My general tests to test if a solution fits my needs is:
Does it run ls -al normally?
Does it run vim normally?
Does it run irb normally?
The following Ruby code is able to check all the above:
to_execute = "vim"
output = ""
require 'pty'
require 'io/console'
master, slave = PTY.open
slave.raw!
pid = ::Process.spawn(to_execute, :in => STDIN, [:out, :err] => slave)
slave.close
master.winsize = $stdout.winsize
Signal.trap(:WINCH) { master.winsize = $stdout.winsize }
Signal.trap(:SIGINT) { ::Process.kill("INT", pid) }
master.each_char do |char|
STDOUT.print char
output.concat(char)
end
::Process.wait(pid)
master.close
This works for the most part but it turns out it's not perfect. For some reason, certain applications seem to fail to switch into a raw state. Even though vim works perfectly fine, it turned out neovim did not. At first I thought it was a bug in neovim but I have since been able to reproduce the problem using the Termion crate for the Rust language.
By setting to raw manually (IO.console.raw!) before executing, applications like neovim behave as expected, but then applications like irb do not.
Oddly spawning another pty in Python, within this pty, allows the application to work as expected (using python -c 'import pty; pty.spawn("/usr/local/bin/nvim")'). This obviously isn't a real solution, but interesting nonetheless.
For my actual question I guess I'm looking towards any help to resolving the weird raw issue or, say if I've completely misunderstood tty/pty, any different direction to where/how I should look at the problem.
[edited: see the bottom for the amended update]
Figured it out :)
To really understand the problem I read up a lot on how a PTY works. I don't think I really understood it properly until I drew it out. Basically PTY could be used for a Terminal emulator, and that was the simplest way to think of the data flow for it:
keyboard -> OS -> terminal -> master pty -> termios -> slave pty -> shell
|
v
monitor <- OS <- terminal <- master pty <- termios
(note: this might not be 100% correct, I'm definitely no expert on the subject, just posting it incase it helps anybody else understand it)
So the important bit in the diagram that I hadn't really realised was that when you type, the only reason you see your input on screen is because it's passed back (left-wards) to the master.
So first thing's first - this ruby script should first set the tty to raw (IO.console.raw!), it can restore it after execution is finished (IO.console.cooked!). This'll make sure the keyboard inputs aren't printed by this parent Ruby script.
Second thing is the slave itself should not be raw, so the slave.raw! call is removed. To explain this, I originally added this because it removes extra return carriages from the output: running echo hello results in "hello\r\n". What I missed was that this return carriage is a key instruction to the terminal emulator (whoops).
Third thing, the process should only be talking to the slave. Passing STDIN felt convenient, but it upsets the flow shown in the diagram.
This brings up a new problem on how to pass user input through, so I tried this. So we basically pass STDIN to the master:
input_thread = Thread.new do
STDIN.each_char do |char|
master.putc(char) rescue nil
end
end
that kind of worked, but it has its own issues in terms of some interactive processes weren't receiving a key some of the time. Time will tell, but using IO.copy_stream instead appears to solve that issue (and reads much nicer of course).
input_thread = Thread.new { IO.copy_stream(STDIN, master) }
update 21st Aug:
So the above example mostly worked, but for some reason keys like CTRL+c still wouldn't behave correctly. I even looked up other people's approach to see what I could be doing wrong, and effectively it seemed the same approach - as IO.copy_stream(STDIN, master) was successfully sending 3 to the master. None of the following seemed to help at all:
master.putc 3
master.putc "\x03"
master.putc "\003"
Before I went and delved into trying to achieve this in a lower level language I tried out 1 more thing - the block syntax. Apparently the block syntax magically fixes this problem.
To prevent this answer getting a bit too verbose, the following appears to work:
require 'pty'
require 'io/console'
def run
output = ""
IO.console.raw!
input_thread = nil
PTY.spawn('bash') do |read, write, pid|
Signal.trap(:WINCH) { write.winsize = STDOUT.winsize }
input_thread = Thread.new { IO.copy_stream(STDIN, write) }
read.each_char do |char|
STDOUT.print char
output.concat(char)
end
Process.wait(pid)
end
input_thread.kill if input_thread
IO.console.cooked!
end
Bundler.send(:with_env, Bundler.clean_env) do
run
end
I was reading the documentation (which is pretty outdated). Net-ssh does not have .shell method anymore? how can i achieve signing into the shell, run command and get stdout until its done?
Can anyone point me to a good documentation or advice on how i can do it?
Thank you
Net::SSH has been updated a few days ago, and using the first example of the README on the first page of the project did exactly what you wanted to do :
require 'net/ssh'
Net::SSH.start('my_server', 'my_user') do |ssh|
output = ssh.exec!("ls")
puts output
end
#=>
# 20130402_083136_DSCF0923.jpg
# 20160715_113357_DSC_6050.jpg
# 20160715_121646_DSC_2.jpg
...
...
I'm trying to run commands remotely with ruby's net ssh. I need the output as well as the exit code. There are other stackoverflow threads on this topic but the accepted solutions do not work. Some people suggested using net-ssh-shell gem but as I tried it, I got an error saying that this package is in conflict with the version of my net-ssh package...
Here is what I have:
Net::SSH.start(remote_ip, username,:keys => [ssh_key_path], :verbose => :debug) do |ssh|
cmd = "blah bla"
result = ssh.exec!(cmd)
puts result
end
It works but it does not raise an exception if it fails. I've also tried using the channel to retrieve the exit code but it always returns 0:
channel.on_request("exit-status") do |ch,data|
exit_code = data.read_long
end
Please help me out. I've already tried several things based on wrong info on the internet.
If you are using Ruby 1.9+ I would suggest Open3.popen3:
i, o, e, t = Open3.popen3("ssh ... remote_user#remote_host remote_command")
i.write ... # if your command needs input
output = o.read
error = e.read
status = t.value.exitstatus
I'm writing some code which takes a file, passes that file to one of several binaries for processing, and monitors the conversion process for errors. I've written and tested the following routine on OSX but linux fails for reasons about which I'm not clear.
#run the command, capture the output so it doesn't display
PTY.spawn(command) {|r,w,pid|
until r.eof? do
##mark
puts r.readline
end
}
The command that runs varies quite a lot and the code at the ##mark has been simplified into a local echo in an attempt to debug the problem. The command executes and the script prints the expected output in the terminal and then throws an exception.
The error it produces on Debian systems is: Errno::EIO (Input/output error - /dev/pts/0):
All of the command strings I can come up with produce that error, and when I run the code without the local echo block it runs just fine:
PTY.spawn(command) {|r,w,pid|}
In either case the command itself executes fine, but it seems like debian linux isn't sending eof up the pty. The doc pages for PTY, and IO on ruby-doc don't seem to lend any aid here.
Any suggestions? Thanks.
-vox-
So I had to go as far as reading the C source for the PTY library to get really satisfied with what is going on here.
The Ruby PTY doc doesn't really say what the comments in the source code say.
My solution was to put together a wrapper method and to call that from my script where needed. I've also boxed into the method waiting on the process to for sure exit and the accessing of the exit status from $?:
# file: lib/safe_pty.rb
require 'pty'
module SafePty
def self.spawn command, &block
PTY.spawn(command) do |r,w,p|
begin
yield r,w,p
rescue Errno::EIO
ensure
Process.wait p
end
end
$?.exitstatus
end
end
This is used basically the same as PTY.spawn:
require 'safe_pty'
exit_status = SafePty.spawn(command) do |r,w,pid|
until r.eof? do
logger.debug r.readline
end
end
#test exit_status for zeroness
I was more than a little frustrated to find out that this is a valid response, as it was completely undocumented on ruby-doc.
It seems valid for Errno::EIO to be raised here (it simply means the child process has finished and closed the stream), so you should expect that and catch it.
For example, see the selected answer in Continuously read from STDOUT of external process in Ruby and http://www.shanison.com/2010/09/11/ptychildexited-exception-and-ptys-exit-status/
BTW, I did some testing. On Ruby 1.8.7 on Ubuntu 10.04, I don't get a error. With Ruby 1.9.3, I do. With JRuby 1.6.4 on Ubuntu in both 1.8 and 1.9 modes, I don't get an error. On OS X, with 1.8.7, 1.9.2 and 1.9.3, I don't get an error. The behavior is obviously dependent on your Ruby version and platform.
As answered here and here, EIO can be avoided by keeping a file descriptor to the pty slave device open in the parent process.
Since PTY.spawn closes the slave file descriptor passed to the child process, a simple workaround is to open a new one. For example:
PTY.spawn("ls") do |r, w, pid|
r2 = File.open(r.path)
while IO.select([r], [], [], 1)
puts r.gets
end
r2.close
end
ruby-doc.org says this since ruby 1.9:
# The result of read operation when pty slave is closed is platform
# dependent.
ret = begin
m.gets # FreeBSD returns nil.
rescue Errno::EIO # GNU/Linux raises EIO.
nil
end
Ok, so now I get this behavior is "normal" on Linux, but that means it's a little tricky to get the output of a PTY. If you do m.read it reads everything and then throws it away and raises Errno::EIO. You really need to read the content chunk by chunk with m.readline. And even then you risk losing the last line if it doesn't end with "\n" for whatever reason. To be extra safe you need to read the content byte by byte with m.read(1)
Additional note about the effect of tty and pty on buffering: it's not the same as STDOUT.sync = true (unbuffered output) in the child process, but rather it triggers line buffering, where output is flushed on "\n"
We have an automatic build system that spits out packages, regression-tested & wrapped into a neat installer, ready for end-users to d/l & deploy.
We do tracking of end user support requests/bug reports via redmine. So far we uploaded the packages manually to the resp. 'Files' section of the redmine project, via the web interface.
What I'd like to do is to automate this step.
I imagine this would requires a few lines of Ruby to interface with redmine's db. I have zero knowledge about redmine's internals. :)
Basically I want the equivalent of a
mv package-x.y.z.tbz /usr/local/redmine/files/
as a Ruby (or whatever language suits the need) script that creates the right filename and registers the file in redmine's db so it shows up as if it had been uploaded through the Web interface, manually.
Cheers!
I've been frustrated with Redmine about things like this before. But before I go much further: is there a specific reason why you're using the Files section for this? It seems another tool (such as SSH/SFTP for uploading to someplace accessible to HTTP) might be a better fit for your needs. It would also be easily scriptable. Just point people to some constant URL like http://yourcompany.com/productname-current.zip.
If you really need to use Redmine for managing this, you might check out Mechanize: http://mechanize.rubyforge.org/. They should have a RESTful API also, but I've never used it.
I found this post, hope it can help you
Automating packaging and RedMine
I'm a bit late, but I've wrote a Redmine upload tool in Perl, using the WWW::Mechanize module.
Please find it on http://github.com/subogero/redgit
As already stated, you can use Mechanize for that.
There's a Python script written by Gert van Dijk's: https://github.com/gertvdijk/redmine-file-uploader
To use it you'll have to install Python Mechanize package first:
easy_install mechanize
If you prefer Ruby, you can use:
require 'mechanize'
# Replaces \ with / and removes "
ARGV.map!{|a|a.gsub('\\','/').gsub(/^"(.+)"$/,'\\1')}
filename = ARGV[0] || abort('Filename must be specified')
puts "File: #{filename}"
url = ARGV[1] || abort('Redmine URL must be specified')
puts "Redmine URL: #{url}"
username = ARGV[2] || abort('Redmine username must be specified')
puts "Username: #{username}"
password = ARGV[3] || abort('Redmine password must be specified')
puts "Password: #{'*' * password.length}"
project = ARGV[4] || abort('Redmine project must be specified')
puts "Project: #{project}"
login_page_path = '/login'
files_page_path = '/projects/' + project + '/files'
agent = Mechanize.new
# No certificate verification (I had to use this hack because our server is bound to custom port)
# agent.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
agent.get(URI.join(url, login_page_path)) do |login_page|
login_page.form_with(:action => login_page_path) do |login_form|
login_form.username = username
login_form.password = password
end.submit
end
agent.get(URI.join(url, files_page_path + '/new')) do |upload_page|
upload_page.form_with(:action => files_page_path) do |upload_form|
upload_form.file_uploads.first.file_name = filename
end.submit
end
And don't forget to install gem first:
gem install mechanize