Reading / Writing from a Unix Socket in Ruby - ruby
I'm trying to connect, read and write from a UNIX socket in Ruby. It is a stats socket used by haproxy.
My code is the following:
require 'socket'
socket = UNIXSocket.new("/tmp/haproxy.stats.socket")
# First attempt: works
socket.puts("show stat")
while(line = socket.gets) do
puts line
end
# Second attemp: fails
socket.puts("show stat")
while(line = socket.gets) do
puts line
end
It succeeds the first time, but on the second attempt fails. I'm not sure why.
# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,
stats,FRONTEND,,,0,0,2000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,1,0,,,,0,0,0,0,,,,0,0,0,0,0,0,,0,0,0,,,
stats,BACKEND,0,0,0,0,2000,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,22,0,,1,1,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,
legacy_socket,FRONTEND,,,0,0,1000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,0,,,,0,0,0,0,0,0,,0,0,0,,,
all,FRONTEND,,,0,0,10000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,3,0,,,,0,0,0,0,,,,0,0,0,0,0,0,,0,0,0,,,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,1,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,2,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,3,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,4,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,5,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,6,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,22,22,,1,4,7,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,21,21,,1,4,8,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,21,21,,1,4,9,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,socket,0,0,0,0,200,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,21,21,,1,4,10,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
socket_backend,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,0,0,DOWN,0,0,0,,1,21,21,,1,4,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,
api_backend,api,0,0,0,0,200,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,22,0,,1,5,1,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,0,,,,0,0,
api_backend,api,0,0,0,0,1,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,22,0,,1,5,2,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,0,,,,0,0,
api_backend,api,0,0,0,0,1,0,0,0,,0,,0,0,0,0,DOWN,1,1,0,0,1,21,21,,1,5,3,,0,,2,0,,0,L4CON,,0,0,0,0,0,0,0,0,,,,0,0,
api_backend,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,0,0,UP,2,2,0,,0,22,0,,1,5,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,
www_backend,ruby-www,0,0,0,0,10000,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,22,0,,1,6,1,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,0,,,,0,0,
www_backend,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,0,0,UP,1,1,0,,0,22,0,,1,6,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,
/Users/Olly/Desktop/haproxy_stats.rb:14:in `write': Broken pipe (Errno::EPIPE)
from /Users/Olly/Desktop/haproxy_stats.rb:14:in `puts'
from /Users/Olly/Desktop/haproxy_stats.rb:14
What is the problem? Is there a good reference to using UNIX sockets and Ruby?
Olly,
HAproxy closes connection after the first request unless you use the "prompt" command (see http://haproxy.1wt.eu/download/1.4/doc/configuration.txt section 9.2) :
#!/usr/bin/env ruby
require 'socket'
socket = UNIXSocket.new("/tmp/haproxy.stats.socket")
# Goes interactive mode
socket.puts("prompt")
# Ask statistics every second
while true
socket.puts("show stat")
socket.each_char do |c|
# We had the prompt, break out
break if c == '>'
print c
end
sleep 1
end
Looks like the connection has been closed after the first request. I don't think you are doing anything wrong. The HAProxy stats socket is probably designed so that it responds to a single command and then closes the connection.
I think you need to reconnect for each request.
If you look at this blog post which is about using HAProxy stats socket with socat then this makes sense because you pipe the show stat command into socat and socat reads from the socket until it closes.
I also encountered the same problem when use socket.puts, you can use socket.write instead of socket.puts to fix it.
#!/usr/bin/evn ruby
# -*- coding: UTF-8 -*-
require 'rubygems'
require 'uri'
require 'socket'
require 'yaml'
SOCKET = URI.parse("/var/run/haproxy/haproxy.sock")
def get_info
UNIXSocket.open(SOCKET.path) do |socket|
socket.write("show info;")
info = YAML::load socket
#info.each {|key, value| puts "#{key} ➤ #{value}"}
end
end
puts get_info["Uptime_sec"]
Look this gem for more information, source code is here.
You can use man socket. You can use the socket-class just like you would if it was a C-function.
I found man-pages to be very useful.
Related
Interactive SSH session using Net::SSH or creating a STDIN Socket
This is not a duplicate of How to SSH interactive Session or Net::SSH interactive terminal scripting. How? or Ruby net-ssh interactive response/reply
I'm trying to write an interactive SSH client using Net:SSH, because I'm already using it for run non-interactive commands on the target hosts.
I could just shell out to system "ssh" but it would require converting the connection settings, proxying, etc to ssh params.
The problem is streaming the data from STDIN to the shell channel. The Net::SSH documentation for listen_to shows how to do it when the input is from a socket instead of STDIN. $stdin or IO.console are not sockets and thus not compatible with Net::SSH::BufferedIo.
Is there a way to create a socket from STDIN that can be used for this? Or is there a better way to send everything from the STDIN to the Net::SSH::Channel until the channel closes?
Here's code that works, but is way too slow to be usable:
require 'net/ssh'
require 'io/console'
require 'io/wait'
Net::SSH.start('localhost', 'root') do |session|
session.open_channel do |channel|
channel.on_data do |_, data|
$stdout.write(data)
end
channel.on_extended_data do |_, data|
$stdout.write(data)
end
channel.request_pty do
channel.send_channel_request "shell"
end
channel.connection.loop do
$stdin.raw do |io|
input = io.readpartial(1024)
channel.send_data(input) unless input.empty?
end
channel.active?
end
end.wait
end
Sockets are really nothing more than file descriptors, and since STDIN is a file descriptor too, it doesn't hurt to try.
What you want however, is to put the TTY into raw mode first to get interactivity.
This code seems to work fine:
begin
system('stty cbreak -echo')
Net::SSH.start(...) do |session|
session.open_channel do |...|
...
session.listen_to(STDIN) { |stdin|
input = stdin.readpartial(1024)
channel.send_data(input) unless input.empty?
}
end.wait
end
ensure
system('stty sane')
end
Goliath not being asynchronous
I am running a simple goliath server on my localhost with Ruby 1.9.3, and it's not running http requests asynchronously. Here's the code:
require 'goliath'
require 'em-synchrony'
require 'em-synchrony/em-http'
class Server < Goliath::API
use Goliath::Rack::Validation::RequestMethod, %w(GET PUT POST)
def initialize
super
puts "Started up Bookcover server... Let 'em come!"
end
def response(env)
thumbnail_cover_url, large_book_cover_url = ["http://riffle-bookcovers.s3.amazonaws.com/B00GJYXA5I-thumbnail.jpg", "http://riffle-bookcovers.s3.amazonaws.com/B00GJYXA5I-original.jpg"]
puts "start"
a = EM::HttpRequest.new(thumbnail_cover_url).get
b = EM::HttpRequest.new(large_book_cover_url).get
puts "done"
[200, {}, "Hello World"]
end
end
When I run ab -n 100 http://127.0.0.1:9000/ I can see it waits for each request to be done, which means that the calls are blocking.
However, according to the documentation Goliath uses Em-synchrony to let me write "synchronous-looking" code, which is not the case here.
I would appreciate any hints and comments!
Credits go to igrigorik for the answer. I quote from his answer here: You also need to specify the concurrency level when you run ab... e.g. ab -c 10 :) Also, make sure you run it in production mode (-e prod).
Is it okay to connect raw ruby sockets to event machine UNIX server?
I have a UNIX server started and the code goes like:
module UNIX_Server
def receive_data(data)
send_data "testing"
end
def unbind
puts "[server] client disconnected."
end
end
EM::run {
EM::start_unix_domain_server('/tmp/file.sock', UNIX_Server)
}
This works fine, and I am trying to connect to this using a Ruby 1.8.7
UNIX Socket:
s = UNIXSocket.new
s.puts "test"
s.gets
The problem here is that my gets method seems to hang and the client
only gets data when I do a Ctrl-C and terminate the server. What am I
missing here?
IO#gets reads a whole line at a time. Your client is waiting for the newline char which your server never sends. Using send_data "testing\n" # note the newline character in your server should work, or you could use IO#getc in a loop.
How To Display Characters Received Via A Socket?
I have a very simple Ruby program that acts as an "echo server". When you connect to it via telnet any text you type is echoed back. That part is working. If I add a 'putc' statement to also print each received character on the console running the program only the very first character displayed is printed. After that it continues to echo things back to the telnet client but there is nothing printed on the console.
The following is a small, stripped down program that exhibits the problem.
I am very new to Ruby and have probably made a typical rookie mistake. What did I do wrong?
require 'socket'
puts "Simple Echo Server V1.0"
server = TCPServer.new('127.0.0.1', '2150')
cbuf = ""
while socket = server.accept
cbuf = socket.readchar
socket.putc cbuf
putc cbuf
end
The problem is that your code is only running the while loop once for every time somebody connects (TCPServer#accept accepts a connection). Try something more like:
require 'socket'
puts "Simple Echo Server V1.0"
server = TCPServer.new('127.0.0.1', '2150')
socket = server.accept
while line = socket.readline
socket.puts line
puts line
end
Exposing console apps to the web with Ruby
I'm looking to expose an interactive command line program via JSON or another RPC style service using Ruby. I've found a couple tricks to do this, but im missing something when redirecting the output and input.
One method at least on linux is to redirect the stdin and stdout to a file then read and write to that file asynchronously with file reads and writes. Another method ive been trying after googling around was to use open4. Here is the code I wrote so far, but its getting stuck after reading a few lines from standard output.
require "open4"
include Open4
status = popen4("./srcds_run -console -game tf +map ctf_2fort -maxplayers 6") do |pid, stdin, stdout, stderr|
puts "PID #{pid}"
lines=""
while (line=stdout.gets)
lines+=line
puts line
end
while (line=stderr.gets)
lines+=line
puts line
end
end
Any help on this or some insight would be appreciated!
What I would recommend is using Xinetd (or similar) to run the command on some socket and then using the ruby network code. One of the problems you've already run into in your code here is that your two while loops are sequential, which can cause problems.
Another trick you might try is to re-direct stderr to stdout in your command, so that your program only has to read the stdout. Something like this:
popen4("./srcds_run -console -game tf +map ctf_2fort -maxplayers 6 2>&1")
The other benefit of this is that you get all the messages/errors in the order they happen during the program run.
EDIT
Your should consider integrating with AnyTerm. You can then either expose AnyTerm directly e.g. via Apache mod_proxy, or have your Rails controller act as the reverse proxy (handling authentication/session validation, then playing back controller.request minus any cookies to localhost:<AnyTerm-daemon-port>, and sending back as a response whatever AnyTerm replies with.)
class ConsoleController < ApplicationController
# AnyTerm speaks via HTTP POST only
def update
# validate session
...
# forward request to AnyTerm
response = Net::HTTP.post_form(URI.parse('http://localhost:#{AnyTermPort}/', request.params))
headers['Content-Type'] = response['Content-Type']
render_text response.body, response.status
end
Otherwise, you'd need to use IO::Select or IO::read_noblock to know when data is available to be read (from either network or subprocess) so you don't deadlock. See this too. Also check that either your Rails is used in a multi-threaded environment or that your Ruby version is not affected by this IO::Select bug.
You can start with something along these lines:
status = POpen4::popen4("ping localhost") do |stdout, stderr, stdin, pid|
puts "PID #{pid}"
# our buffers
stdout_lines=""
stderr_lines=""
begin
loop do
# check whether stdout, stderr or both are
# ready to be read from without blocking
IO.select([stdout,stderr]).flatten.compact.each { |io|
# stdout, if ready, goes to stdout_lines
stdout_lines += io.readpartial(1024) if io.fileno == stdout.fileno
# stderr, if ready, goes to stdout_lines
stderr_lines += io.readpartial(1024) if io.fileno == stderr.fileno
}
break if stdout.closed? && stderr.closed?
# if we acumulated any complete lines (\n-terminated)
# in either stdout/err_lines, output them now
stdout_lines.sub!(/.*\n/m) { puts $& ; '' }
stderr_lines.sub!(/.*\n/m) { puts $& ; '' }
end
rescue EOFError
puts "Done"
end
end
To also handle stdin, change to:
IO.select([stdout,stderr],[stdin]).flatten.compact.each { |io|
# program ready to get stdin? do we have anything for it?
if io.fileno == stdin.fileno && <got data from client?>
<write a small chunk from client to stdin>
end
# stdout, if ready, goes to stdout_lines