I'm working on a ruby script that ultimately starts up a system process that takes quite a while. I need to read from the stderr of this process and react to it depending on what is output.
I'm currently doing it as such:
Open3.popen3(cmd_to_run) do |stdin, stdout, stderr, waitthread|
stderr.each_line do |line|
# look out for specific lines and react to them accordingly
end
end
But I've also seen implementations to achieve something similar but doing it with kernel#select:
Open3.popen3(cmd_to_run) do |stdin, stdout, stderr, waitthread|
io = select([stderr], nil, nil, 30)
if io.nil?
log("Command timed out during Kernel#select")
return
end
io[0][0].each_line do |line|
# look out for specific lines and react to them accordingly
end
end
I've read the pickaxe description of what select does, but I'm confused as to why I should (of if I should) use it? The first method works just the same.
Probably two reasons:
You can use timeout, which you can't with each_line
You can wait for more than one IO object, e. g. io = select([stdout, stderr]) and more than one event (e.g. write event or exception too)
Related
Here is a simple ruby script, that takes input from a user and provides an output(yes, it will be refactored). I wanted this script to provide output into a text file, rather than console window. That was accomplished, by simply adding $stdout = File.new('out.txt', 'w'), but I thought that this line will just describe a variable which I will use later to tell script to use it to write output into created file.
I cant find much documentation about this method and wondering how does this program knows how to write generated output into that file?
$stdout is a global variable. By default it stores an object of type IO associated with the standard output of the program (which is, by default, the console).
puts is a method of the Kernel module that actually calls $stdout.send() and pass it the list of arguments it receives. As the documentation explains, puts(obj, ...) is equivalent to $stdout.puts(obj, ...).
Your code replaces $stdout with an object of type File that extends class IO. When it is created, your object opens the file out.txt for writing and together with its inheritance from IO it is fully compatible with the default behaviour of $stdout.
Since by default, all the output goes to $stdout, your new definition of $stdout ensures the output is written to the file out.txt without other changes in the code.
$stdout is a global variable (as indicated by $) and according to the documentation, puts is:
Equivalent to
$stdout.puts(obj, ...)
If you assign another object to $stdout, then Kernel#puts will simply send puts to that object. Likewise, print will send write:
class Foo < BasicObject
def puts(*args)
::STDOUT.puts "Foo#puts called with #{args.inspect}"
end
def write(*args)
::STDOUT.puts "Foo#write called with #{args.inspect}"
end
end
$stdout = Foo.new
puts 'hello', 'world'
# Foo#puts called with ["hello", "world"]
print "\n"
# Foo#write called with ["\n"]
Note that if you assign to $stdout, Ruby checks whether the object responds to write. If not, a TypeError will be raised.
It's probably worth highlighting the fact, mentioned by the other posts, that this is a global variable, and modifying it is not thread safe. For instance, if you point $stdout to a new IO stream, anything across the app (on that process thread) that would be logged to stdout will now be logged to your new IO stream. This can lead to multiple streams of unexpected and possibly sensitive input landing in your new IO stream.
In a blog post about unconditional programming Michael Feathers shows how limiting if statements can be used as a tool for reducing code complexity.
He uses a specific example to illustrate his point. Now, I've been thinking about other specific examples that could help me learn more about unconditional/ifless/forless programming.
For example in this cat clone there is an if..else block:
#!/usr/bin/env ruby
if ARGV.length > 0
ARGV.each do |f|
puts File.read(f)
end
else
puts STDIN.read
end
It turns out ruby has ARGF which makes this program much simpler:
#!/usr/bin/env ruby
puts ARGF.read
I'm wondering if ARGF didn't exist how could the above example be refactored so there is no if..else block?
Also interested in links to other illustrative specific examples.
Technically you can,
inputs = { ARGV => ARGV.map { |f| File.open(f) }, [] => [STDIN] }[ARGV]
inputs.map(&:read).map(&method(:puts))
Though that's code golf and too clever for its own good.
Still, how does it work?
It uses a hash to store two alternatives.
Map ARGV to an array of open files
Map [] to an array with STDIN, effectively overwriting the ARGV entry if it is empty
Access ARGV in the hash, which returns [STDIN] if it is empty
Read all open inputs and print them
Don't write that code though.
As mentioned in my answer to your other question, unconditional programming is not about avoiding if expressions at all costs but about striving for readable and intention revealing code. And sometimes that just means using an if expression.
You can't always get rid of a conditional (maybe with an insane number of classes) and Michael Feathers isn't advocating that. Instead it's sort of a backlash against overuse of conditionals. We've all seen nightmare code that's endless chains of nested if/elsif/else and so has he.
Moreover, people do routinely nest conditionals inside of conditionals. Some of the worst code I've ever seen is a cavernous nightmare of nested conditions with odd bits of work interspersed within them. I suppose that the real problem with control structures is that they are often mixed with the work. I'm sure there's some way that we can see this as a form of single responsibility violation.
Rather than slavishly try to eliminate the condition, you could simplify your code by first creating an array of IO objects from ARGV, and use STDIN if that list is empty.
io = ARGV.map { |f| File.new(f) };
io = [STDIN] if !io.length;
Then your code can do what it likes with io.
While this has strictly the same number of conditionals, it eliminates the if/else block and thus a branch: the code is linear. More importantly, since it separates gathering data from using it, you can put it in a function and reuse it further reducing complexity. Once it's in a function, we can take advantage of early return.
# I don't have a really good name for this, but it's a
# common enough idiom. Perl provides the same feature as <>
def arg_files
return ARGV.map { |f| File.new(f) } if ARGV.length;
return [STDIN];
end
Now that it's in a function, your code to cat all the files or stdin becomes very simple.
arg_files.each { |f| puts f.read }
First, although the principle is good, you have to consider other things that are more importants such as readability and perhaps speed of execution.
That said, you could monkeypatch the String class to add a read method and put STDIN and the arguments in an array and start reading from the beginning until the end of the array minus 1, so stopping before STDIN if there are arguments and go on until -1 (the end) if there are no arguments.
class String
def read
File.read self if File.exist? self
end
end
puts [*ARGV, STDIN][0..ARGV.length-1].map{|a| a.read}
Before someone notices that I still use an if to check if a File exists, you should have used two if's in your example to check this also and if you don't, use a rescue to properly inform the user.
EDIT: if you would use the patch, read about the possible problems at these links
http://blog.jayfields.com/2008/04/alternatives-for-redefining-methods.html
http://www.justinweiss.com/articles/3-ways-to-monkey-patch-without-making-a-mess/
Since the read method isn't part of String the solutions using alias and super are not necessary, if you plan to use a Module, here is how to do that
module ReadString
def read
File.read self if File.exist? self
end
end
class String
include ReadString
end
EDIT: just read about a safe way to monkey patch, for your documentation see https://solidfoundationwebdev.com/blog/posts/writing-clean-monkey-patches-fixing-kaminari-1-0-0-argumenterror-comparison-of-fixnum-with-string-failed?utm_source=rubyweekly&utm_medium=email
I would like to control a separate process with a classic 1), 2), 3), etc. menu system. Similar to piping an input file in to control the process, I would like to use Ruby to control the process' $stdin and $stdout. I've experimented with IO.popen and Open3.popen3, but cannot seem to get it to work. The documentation examples are not clear (but I'm also quite new to this sort of programming).
The basic idea is:
Open3.popen3("./server") do |stdin,stdout,stderr|
stdout.gets
stdin.puts "1"
stdout.gets
stdin.puts "2"
stdout.gets
end
Currently, the first stdout.gets gets the correct header, but then the program seems to hang. Can anybody offer any advice? I've been googling for a while now but haven't found anything.
Thanks!
IPC (Inter Process Communication) is very complicated. When working with IPC you need to flush the buffer after every write operation. It would look like this:
Open3.popen3("./server") do |stdin,stdout,stderr|
stdout.gets
stdin.puts "1"
stdin.flush
stdout.gets
stdin.puts "2"
stdin.flush
stdout.gets
end
It's ugly, but you can redefine puts as a singleton method on stdin (you can do the same with print and write, if needed):
class <<stdin
old_puts=method(puts)
def puts(str)
old_puts[str]
flush
end
end
This will save allot of trouble if you are doing a lot of IPC.
I could use:
File.open('/dev/null', 'w')
on Unix systems, but if there is a Ruby way to achieve this, I'd like to use it. I am just looking for an I/O stream, that immediately "trashes" all writes, kind of like a null-object.
If you want the full behavior of streams, the best is probably to use:
File.open(File::NULL, "w")
Note that File::NULL is new to Ruby 1.9.3; you can use my backports gem:
require 'backports/1.9.3/file/null' # => Won't do anything in 1.9.3+
File.open(File::NULL, "w") # => works even in Ruby 1.8.6
You could also copy the relevant code if you prefer.
There's stringIO, which I find useful when I want to introduce a dummy filestream:
require "stringio"
f = StringIO.new
f.gets # => nil
And here's some code from heckle that finds the bit bucket for both Unix and Windows, slightly modified:
# Is this platform MS Windows-like?
# Actually, I suspect the following line is not very reliable.
WINDOWS = RUBY_PLATFORM =~ /mswin/
# Path to the bit bucket.
NULL_PATH = WINDOWS ? 'NUL:' : '/dev/null'
No, I don't believe there is anything like a null stream in Ruby, at least in earlier versions. In that case, you must make one yourself. Depending on the methods that it will call, you will need to write
stub methods on the null stream class, like this:
class NullStream
def <<(o); self; end
end
The above example is by no means complete. For example, some streams may require calling the write, puts or other methods. Moreover, some methods should be implemented by returning self in their methods, like <<, others not.
Logger.new("/dev/null") does the trick
There's a gem called devnull
Ruby implementation of null file (like /dev/null on Un*x, NUL on
Windows)
It doesn't interact with the null file, but instead has dummy methods for all the methods that IO objects implement.
Why does Kernel#p print to standard out? Isn't printf debugging supposed to output to standard error?
You can define a global function "q", which works just like "p" except it prints to $stderr.
#!/usr/bin/ruby1.8
module Kernel
def q(*stuff)
stuff.each { |thing| $stderr.print(thing.inspect + "\n")}
end
end
q 'foo' # => "foo"
You may be tempted to use puts instead of print ... + "\n". This code uses print to make it thread-safe: puts can be interrupted between the time it prints its arguments and the time it prints the new-line, causing output from two threads to appear on one line. It's seldom that you have code from multiple threads writing to $stdout/$stderr at the same time, so it's not usually an issue. But this being a debugging tool, you will certainly end up using it to find out what is going on in threads.
Why are you assuming Kernel#p is intended for debugging? It writes to stdout just like Kernel#print or printf in C.
If you want to write to standard error you could do:
$stderr.puts(x.inspect)
By the way, if you really want to use printf debugging I suggest you read this article about debugging techniques