How to lock IO shared by fork in ruby - ruby

Question

How can we lock an IO that has been shared by multiple ruby process?
Consider this script:
#!/usr/bin/ruby -w
# vim: ts=2 sw=2 et
if ARGV.length != 2
$stderr.puts "Usage: test-io-fork.rb num_child num_iteration"
exit 1
end
CHILD = ARGV[0].to_i
ITERATION = ARGV[1].to_i
def now
t = Time.now
"#{t.strftime('%H:%M:%S')}.#{t.usec}"
end
MAP = %w(nol satu dua tiga empat lima enam tujuh delapan sembilan)
IO.popen('-', 'w') {|pipe|
unless pipe
# Logger child
File.open('test-io-fork.log', 'w') {|log|
log.puts "#{now} Program start"
$stdin.each {|line|
log.puts "#{now} #{line}"
}
log.puts "#{now} Program end"
}
exit!
end
pipe.sync = true
pipe.puts "Before fork"
CHILD.times {|c|
fork {
pid = Process.pid
srand
ITERATION.times {|i|
n = rand(9)
sleep(n / 100000.0)
pipe.puts "##{c}:#{i} #{MAP[n]} => #{n}, #{n} => #{MAP[n]} ##{c}:#{i}"
}
}
}
}
And try it like this:
./test-io-fork.rb 200 50
Like expected, the test-io-fork.log files would contains sign of IO race condition.
What I want to achieve is to make a TCP server for custom GPS protocol that will save the GPS points to database. Because this server would handle 1000 concurrent clients, I would like to restrict database connection to only one child instead opening 1000 database connection simultaneously. This server would run on linux.

Answer 1

UPDATE
It may be bad form to update after the answer was accepted, but the original is a bit misleading. Whether or not ruby makes a separate write(2) call for the automatically-appended newline is dependent upon the buffering state of the output IO object.
$stdout (when connected to a tty) is generally line-buffered, so the effect of a puts() -- given reasonably sized string -- with implicitly added newline is a single call to write(2). Not so, however, with IO.pipe and $stderr, as the OP discovered.
ORIGINAL ANSWER
Change your chief pipe.puts() argument to be a newline terminated string:
pipe.puts "##{c} ... #{i}\n" # <-- note the newline
Why? You set pipe.sync hoping that the pipe writes would be atomic and non-interleaved, since they are (presumably) less than PIPE_BUF bytes. But it didn't work, because ruby's pipe puts() implementation makes a separate call to write(2) to append the trailing newline, and that's why your writes are sometimes interleaved where you expected a newline.
Here's a corroborating excerpt from a fork-following strace of your script:
$ strace -s 2048 -fe trace=write ./so-1326067.rb
....
4574 write(4, "#0:12 tiga => 3, 3 => tiga #0:12", 32) = 32
4574 write(4, "\n", 1)
....
But putting in your own newline solves the problem, making sure that your entire record is transmitted in one syscall:
....
5190 write(4, "#194:41 tujuh => 7, 7 => tujuh #194:41\n", 39 <unfinished ...>
5179 write(4, "#183:38 enam => 6, 6 => enam #183:38\n", 37 <unfinished ...>
....
If for some reason that cannot work for you, you'll have to coordinate an interprocess mutex (like File.flock()).