Why can I capture rb_warn messages on $stderr only once? - ruby

Question

Due to a bad decision on OpenSSL's part that affected the Ruby bindings, the only way to check if an OCSP request is signed is by parsing the warning from OpenSSL::OCSP::Request#verify. I intercept $stderr and read the warning, but if I repeat this process in multiple unit tests, the first error message gets captured every time, even though each intercept uses a new buffer.
As an example, I created this script: test.rb
#!/usr/bin/env ruby
require 'openssl'
require 'stringio'
def main
if ARGV[0]
puts "signed: \n#{signed}"
puts "unsigned: \n#{unsigned}"
else
puts "unsigned: \n#{unsigned}"
puts "signed: \n#{signed}"
end
end
def unsigned
cert = OpenSSL::X509::Certificate.new
certid = OpenSSL::OCSP::CertificateId.new cert, cert
request = OpenSSL::OCSP::Request.new.add_certid certid
store = OpenSSL::X509::Store.new
capture_stderr { request.verify([], store) }
end
def signed
key = OpenSSL::PKey::RSA.generate(2048)
cert = OpenSSL::X509::Certificate.new
cert.public_key = key.public_key
cert.sign(key, OpenSSL::Digest::SHA1.new)
certid = OpenSSL::OCSP::CertificateId.new OpenSSL::X509::Certificate.new, cert
store = OpenSSL::X509::Store.new
store.add_cert cert
request = OpenSSL::OCSP::Request.new.add_certid certid
request.sign(cert, key)
capture_stderr { request.verify([], store) }
end
def capture_stderr
$stderr = StringIO.new
result = yield
[result, $stderr.string]
ensure
$stderr = STDERR
end
# try `./test.rb` and `./test.rb 1`
main
By flipping the order of function calls, I get different results.
$ ./test.rb
unsigned:
[false, "./test.rb:22: warning: error:27074080:OCSP routines:OCSP_request_verify:request not signed\n"]
signed:
[false, "./test.rb:38: warning: error:27074080:OCSP routines:OCSP_request_verify:request not signed\n"]
and
$ ./test.rb 1
signed:
[false, "./test.rb:38: warning: error:27074065:OCSP routines:OCSP_request_verify:certificate verify error\n"]
unsigned:
[false, "./test.rb:22: warning: error:27074065:OCSP routines:OCSP_request_verify:certificate verify error\n"]
I imagine that the explanation for this weird behavior probably goes into the C language part of the Ruby stdlib codebase.

Answer 1

It turned out to be a bug in Ruby's OpenSSL library. It was fixed in the Gem version 2.0.0-beta2: https://github.com/ruby/openssl/commit/9af69abcec15b114d9a0ec3811983fc1d7b5a1dc
Now OpenSSL messages aren't repeatedly leaked to stderr. However, it is now impossible to differentiate between untrusted signatures and missing signatures. Thankfully, I was wrong before and verification does indeed fail for missing signatures.