I have an app that runs continuously, dumping output from a server and sending strings to stdout. I want to process this output with a Ruby script. The strings are \n-terminated.
For example, I'm trying to run this on the command line:
myapp.exe | my_script.rb
...with my_script.rb defined as:
while $stdin.gets
puts $_
end
I ultimately am going to process the strings using regexes and display some summary data, but for now I'm just trying to get the basic functionality hooked up. When I run the above, I get the following error:
my_script.rb:1:in `gets': Bad file descriptor (Errno::EBADF)
from my_script.rb:1
I am running this on Windows Server 2003 R2 SP2 and Ruby 1.8.6.
How do I continuously process stdin in a Ruby script? (Continuously as in not processing a file, but running until I kill it.)
EDIT:
I was able to make this work, sort of. There were several problems standing in my way. For one thing, it may be that using Ruby to process the piped-in stdin from another process doesn't work on Windows 2003R2. Another direction, suggested by Adrian below, was to run my script as the parent process and use popen to connect to myapp.exe as a forked child process. Unfortunately, fork isn't implemented in Windows, so this didn't work either.
Finally I was able to download POpen4, a RubyGem that does implement popen on Windows. Using this in combination with Adrian's suggestion, I was able to write this script which does what I really want -- processes the output from myapp.exe:
file: my_script.rb
require 'rubygems'
require 'popen4'
status =
POpen4::popen4("myapp.exe") do |stdout, stderr, stdin, pid|
puts pid
while s = stdout.gets
puts s
end
end
This script echoes the output from myapp.exe, which is exactly what I want.
Try just plain gets, without the $stdin. If that doesn't work, you might have to examine the output of myapp.exe for non-printable characters with another ruby script, using IO.popen.
gets doesn't always use stdin but instead tries to open a file.
See SO.
Try executing your Ruby script by explicitly calling ruby:
myapp.exe | ruby my_script.rb
I've experienced some odd behavior using stdin in Ruby when relying on Windows to invoke the correct program based on the file associations.
Related
I was writing a Slack bot in Ruby under Windows and everything worked just fine until I decided to run it on a Linux server. When I access my shell and run the script it works correctly in the foreground, but once I move it to the background it stops working. I'm getting a timeout error on a HTTP request with Net::HTTP, or an EOFError on the socket read.
I'm using Ruby 2.3 on Debian 7.
I think that the Ruby process stops on its own, because I only get the errors once I return the process to the foreground, and, if I run ps aux when the process is in the background it has the "T" (stopped) flag listed.
Since I want to become more familiar with Linux, I'd like to know what is causing the issue, rather than how to solve it.
EDIT: I found that my user input handler is causing the problem. Here is the problematic bit:
def input_handler
return Thread.new {
loop do
user_input = gets.chomp
end
}
end
The problem looks like it's gets.
By default gets reads from STDIN. The documentation says:
Returns (and assigns to $_) the next line from the list of files in ARGV (or $*), or from standard input if no files are present on the command line.
The code/thread will stop and wait for a prompt from the keyboard, or read from the piped input if STDIN is redirected or from a file given as a parameter to the script on the command-line.
I'm using a helper script to execute rspec tests.
command = "rake spec #{path} #{scope}"
output = `#{command}`
puts output
This works fine, except that I lose all the colors from the rake rspec output. The appropriate ANSI codes do not appear to be contained within the output string.
How can I execute a process so that it returns output which includes the text color?
Kernel.exec() gets me the solution I want (colored rspec output), but it does so by replacing my ruby script process with the rspec process. That means I can't do anything with the output or run anything after the rspec call.
That's acceptable in my particular situation, but less than ideal as a general solution. So I'd like a better answer if available.
RSpec will disable colour if it is not writing to a tty (i.e. the console).
In case of RSpec you can force colouring by rspec --tty or via rake by rake spec SPEC_OPTS=' --tty'.
See also Ruby popen3 and ANSI colour
However this solution is still specific to Rspec. I'd be interested to hear a general one.
Turns out it's possible to run commands in a pseudo terminal via the PTY module in order to preserve a user facing terminal-like behaviour. Credits go to the creator of the tty-command gem (see this issue) who implemented this behaviour in his gem:
require 'tty-command'
cmd = TTY::Command.new(pty: true)
cmd.run('rake', 'rspec')
Keep in mind that using a pseudo terminal may have unwanted side effects, such as certain git commands using a pager which will essentially cause commands to hang. So introducing the functionality might be a breaking change.
If you don't want to replace your ruby process with that command, use Kernel.system() or Kernel.spawn() instead of a Kernel.exec(). Both of them execute your command in a subshell, system waits for the subprocess to finish, spawn returns its pid and you have to wait by yourself using Process.wait pid.
command = "rake spec #{path} #{scope}"
system(command)
or
command = "rake spec #{path} #{scope}"
pid = spawn(command)
# Some other stuff here
Process.wait pid
What I would like to do is:
run a ruby script...
that executes a shell command
and redirects it to a named pipe accessible outside the script
from the system shell, read from that pipe
That is, have the Ruby script capture some command output and redirect it in such a way that it's connectable to from outside the script?
I want to mention that the script cannot simply start and exit, since it's a REPL. The idea is that using the REPL you would be able to run a command and redirect its output elsewhere to consume it.
Using abort and an exit message, will pass the message to STDERR (and the script will fail with exit code 1). You can pass this shell command output in this way.
This is possibly not the only (or best) way, but it has worked for me in the past.
[edit]
You can also redirect the output to a file (using standard methods), and read that file outside the ruby script.
require 'open3'
stdin, stderr, status = Open3.capture3(commandline)
stdin.chomp #Here, you should ge
Incase, if someone wanted to use you can get the output via stdin.chomp
I have a problem at hand which requires me to spawn a command prompt as a different process and send some commands to it and capture/parse the command output. This interaction needs to be in the form of a parent-child process where say all the commands can be put in a ruby file and upon running the ruby file, the commands are sent to the console(command prompt) and output is received from it and processed in the ruby script.
The general logic which I would follow is:
Spawn a different process by using a fork and get a process id
Obtain streams for the process
Write to the input stream of the process and read from the output stream.
The environment which I am using is Windows XP machine with Ruby 1.9.2 installed on it. I downloaded the win32-process library found over here. By using that library, I could do step 1 as follows
require 'win32/process'
APP_NAME = "C:\\Windows\\system32\\cmd.exe"
process_info = Process.create(:app_name => APP_NAME,
:creation_flags => Windows::Process::CREATE_NEW_CONSOLE,
:process_inherit => false,
:thread_inherit => true,
:cwd => "C:\\"
)
Since the win32-process library is based on using processes and threads on windows, I tried to go through the MSDN help for it. While reading the Creation of a Console article, I found that the GetStdHandle method could be used to get the handles to the input and output streams. But, i could not find this method implemented anywhere in win32-process.
Can someone provide me with some guidance on how to proceed with steps 2 and 3?
Also, is there any other way which can be used to solve the problem at hand?
Also, I would like to learn more about inter-process communication or in general spawning and forking of processes, so can somebody please tell me some good references where I could study them?
Thanks in advance
Here an example using IO.popen in windows, imo if it works with the stdlib don't use gems
IO.popen("other_program", "w+") do |pipe|
pipe.puts "here, have some input"
pipe.close_write # If other_program process doesn't flush its output, you probably need to use this to send an end-of-file, which tells other_program to give us its output. If you don't do this, the program may hang/block, because other_program is waiting for more input.
output = pipe.read
end
# You can also use the return value from your block. (exit code stored in $? as usual)
output = IO.popen("other_program", "w+") do |pipe|
pipe.puts "here, have some input"
pipe.close_write
pipe.read
end
My Ruby script is running a shell command and parsing the output from it. However, it seems the command is first executed and output saved in an array. I would like to be able to access the output lines in real time just as they are printed. I've played around with threads, but haven't got it to work. Any suggestions?
You are looking for pipes. Here is an example:
# This example runs the netstat command via a pipe
# and processes the data in Ruby as it come back
pipe = IO.popen("netstat 3")
while (line = pipe.gets)
print line
print "and"
end
When call methods/functions to run system/shell commands, your interpreter spawns another process to run it and waits for it to finish, then gives you the output.
Even if you use threads, the only thing that you would accomplish is not letting your program to hang while the command is run, but you still won't get the output till its done.
I think you can accomplish that with pipes, but I am not sure how.
#Marcel got it.