Sharing DB connections across objects using class methods in ruby? - ruby

I am writing a ruby script to be used as Postfix SMTP access policy delegation. The script needs to access a Tokyo Tyrant database. I am using EventMachine to take care of network connections. EventMachine needs a EventMachine::Connection class that is instantiated by EventMachineā€˜s processing loop whenever a new connection is created. so for each connection a class is instantiated and destroyed.
I am creating a connection to Tokyo Tyrant from the post_init of the EventMachine::Connection (ie right after connection is setup) and tearing it down after connection is terminated.
My question is if this is the proper way to connect to db? ie making a connection every yime I need it and tearing it down after I am finished? Wouldn't be better to connect to DB once (when program is started) tear it down during program shutdown? If that is so how should I code that ?
My code is:
require 'rubygems'
require 'eventmachine'
require 'rufus/tokyo/tyrant'
class LineCounter < EM::Connection
ActionAllow = "action=dunno\n\n"
def post_init
puts "Received a new connection"
#tokyo = Rufus::Tokyo::Tyrant.new('server', 1978)
#data_received = ""
end
def receive_data data
#data_received << data
#data_received.lines do |line|
key = line.split('=')[0]
value = line.split('=')[1]
#reverse_client_name = value.strip() if key == 'reverse_client_name'
#client_address = value.strip() if key == 'client_address'
#tokyo[#client_address] = #reverse_client_name
end
puts #client_address, #reverse_client_name
send_data ActionAllow
end
def unbind
#tokyo.close
end
end
EventMachine::run {
host,port = "127.0.0.1", 9997
EventMachine::start_server host, port, LineCounter
puts "Now accepting connections on address #{host}, port #{port}..."
EventMachine::add_periodic_timer( 10 ) { $stderr.write "*" }
}
with regards,
raj

Surprising there's no answers to this question.
What you probably need is a connection pool where you can fetch, use, and return connections as they are required.
class ConnectionPool
def initialize(&block)
#pool = [ ]
#generator = block
end
def fetch
#pool.shift or #generator and #generator.call
end
def release(handle)
#pool.push(handle)
end
def use
if (block_given?)
handle = fetch
yield(handle)
release(handle)
end
end
end
# Declare a pool with an appropriate connection generator
tokyo_pool = ConnectionPool.new do
Rufus::Tokyo::Tyrant.new('server', 1978)
end
# Fetch/Release cycle
tokyo = tokyo_pool.fetch
tokyo[#client_address] = #reverse_client_name
tokyo_pool.release(tokyo)
# Simple block-method for use
tokyo_pool.use do |tokyo|
tokyo[#client_address] = #reverse_client_name
end

Related

Server should reply as binary packet to the client

I have a module that should send GPS location to my server. Below I describe how communication happens between module and server:
When the module connects to the server, module sends its IMEI.
If server accepts data, then should reply to module 01. Note that confirmation should be sent as binary packet. I.e. 1 byte 0x01.
I'm struggling at second step. Have tried various combination, but neither worked.
client.puts('\x01')
client.send([0x01].pack("C"), 0)
client.write["01"].pack('H*')
Below is a full code example:
require 'socket'
class ClientThread
def initialize(port)
#server = TCPServer.open(port)
end
def run
puts "Started TCP Server"
loop do
Thread.start(#server.accept) do |client|
2.times do |index|
data = client.recv(8192)
if index == 0
client.send('\x01'.encode('utf-8'), 0) # RESPONSE TO DEVISE THAT SERVER IS READY TO ACCEPT DATA
elsif index == 1
puts self.log("Done! Closing Connection")
client.close
else
client.send('\x00'.encode('utf-8'), 0) # RESPONSE TO DEVISE THAT SERVER IS NOT READY TO ACCEPT DATA
end
end # end of loop twice
end # end of Thread
end # end of infinite loop
end # run method
end # end of class
new_thread = ClientThread.new(65432)
p new_thread.run
Device does not understand that server is ready to accept data. Most likely, because binary packet is formed incorrectly. How do you form response, so device would understand to send GSP data?
Ask me please if any questions. Thanks.
I have managed to make it work.
Instead of client.send('\x01'.encode('utf-8'), 0) I have used client.puts [0x01].pack("C") and module started to send data.
Below is a full working example for Teltonika FMT100.
Module sends its IMEI to the server.
Server accepts data and replies as binary packet 0x01.
Module send GPS data to the server. At this point you will need to decode data.
require 'socket'
class ClientThread
def initialize(port)
#server = TCPServer.open(port)
#imei = "unknown"
end
def log(msg)
"#{Time.now.utc} #{msg}"
end
def run
puts self.log("Started TCP Server")
loop do
Thread.start(#server.accept) do |client|
if client
2.times do |index|
begin
data = client.recv(8192)
if index == 0
#imei = data
p self.log("Device Authenticated | IMEI: #{#imei}")
client.puts [0x01].pack("C")
elsif index == 1
p data.unpack('H*').first
p self.log("Done! Closing Connection")
client.close
else
client.puts [0x00].pack("C")
end
rescue IOError
p self.log("Stream is already closed")
end
end # end of loop twice
else
p self.log('Socket is null')
end # if conditional
end # end of Thread
end # end of infinite loop
end # run method
end # end of class
new_thread = ClientThread.new(65432)
p new_thread.run
You should get something similar to:
"2021-12-30 11:12:22 UTC Device Authenticated | IMEI: \u0000\u000F357544374597827"
"00000000000004d608130000017dfcc4f8f8000f1753862097342800000000000000f00c05ef00f0011505c800450205b50000b60000422fbb430fc944000002f100006019100009d77a000000017dfcc5f2f8000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fbe430fcb44000002f100006019100009d77a000000017dfcce8a30000f1753862097342800000000000000000c05ef00f0001505c800450205b50000b60000422f6a430fbd44000002f100006019100009d77a000000017dfcce8e18000f1753862097342800000000000000f00c05ef00f0011505c800450205b50000b60000422f78430fc244000002f100006019100009d77a000000017dfccf8430000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fbd430fcb44000002f100006019100009d77a000000017dfcdbd488000f1753862097342800000000000000000c05ef00f0001504c800450205b50000b60000422fc0430fcb44000002f100006019100009d77a000000017dfcdbd870000f1753862097342800000000000000f00c05ef00f0011504c800450205b50000b60000422fac430fc744000002f100006019100009d77a000000017dfcdcbee8000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fb8430fcb44000002f100006019100009d77a000000017dfcddb500000f1753862097342800000000000000f00c05ef00f0011504c800450205b50000b60000422fb5430fcb44000002f100006019100009d77a000000017dfcdebab8000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fbe430fca44000002f100006019100009d77a000000017dfced4948000f1753862097342800000000000000000c05ef00f0001504c800450205b50000b60000422fbe430fc644000002f100006019100009d77a000000017dfced4d30000f1753862097342800000000000000f00c05ef00f0011504c800450205b50000b60000422fbf430fc644000002f100006019100009d77a000000017dfcee3790000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fc2430fc644000002f100006019100009d77a000000017dfcefd5a0000f1753862097342800000000000000f00c05ef00f0011504c800450205b50000b60000422fbe430fc644000002f100006019100009d77a000000017dfcf13918000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fc1430fc644000002f100006019100009d77a000000017dfcf2b7e8000f1753862097342800000000000000f00c05ef00f0011504c800450205b50000b60000422fb7430fc644000002f100006019100009d77a000000017dfcf3a630000f1753862097342800000000000000f00c05ef00f0001504c800450205b50000b60000422fb8430fc644000002f100006019100009d77a000000017dfd2aac20000f1753862097342800000000000000000c05ef00f0001504c800450205b50000b60000422fa0430fcb44000002f100006019100009d77a000000017dfd61b210000f1753862097342800000000000000000c05ef00f0001504c800450205b50000b60000422f9b430fcb44000002f100006019100009d77a00130000b8e0"
"2021-12-30 11:12:24 UTC Done! Closing Connection"

Ruby Thrift::TransportException End of File Reached

I made a Ruby web server based on Apache Thrift, but the client (also in ruby, for unit tests) refuses to work and keeps telling me either Thrift::TransportException: Could not connect to 127.0.0.1:8001: Connection refused - connect(2) for 127.0.0.1:8001, or Thrift::TransportException: end of file reached. Tried a bunch of different server implementations and transports, and that doesn't seem to work.
When the server is running, lsof -i :8001 shows
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ruby 19073 lafickens 9u IPv4 0x878e6fd36b981a71 0t0 TCP *:vcom-tunnel (LISTEN)
So I think the server is functioning.
Here's the server code:
class Server
attr_reader :name
def initialize(name)
#LOGGER = Logger.new $stdout
#name = name
#started = false
#processor = ::Thrift::MultiplexedProcessor.new
#processor.register_processor 'User Service', Thrift::UserService::Processor.new(Handlers::UserServiceHandler.new)
#processor.register_processor 'Sync Service', Thrift::SyncService::Processor.new(Handlers::SyncServiceHandler.new)
end
def start
#transport = ::Thrift::ServerSocket.new(Options.get('port'))
#transport_factory = ::Thrift::BufferedTransportFactory.new
#protocol_factory = ::Thrift::CompactProtocolFactory.new
#server = ::Thrift::ThreadPoolServer.new #processor, #transport, #transport_factory, #protocol_factory
#server_thread = Thread.new {
#server.serve
}
#started = true
#LOGGER.info('Server started successfully')
end
def stop
return if #server_thread.nil?
#server_thread.exit
#transport.close
#started = false
#LOGGER.info('Server stopped successfully')
end
def restart
stop
start
end
def started?
#started
end
end
Client code (actually unit tests)
class TestUserServiceHandler < Test::Unit::TestCase
def setup
#server = Billboard::Server.new 'test handler'
#server.start
#port = Billboard::Options.get 'port'
#transport = ::Thrift::BufferedTransport.new(::Thrift::Socket.new('127.0.0.1', #port))
#binary_protocol = ::Thrift::BinaryProtocol.new #transport
#multiplexed_protocol = ::Thrift::MultiplexedProtocol.new #binary_protocol, 'mprotocol'
#client = Billboard::Thrift::UserService::Client.new #multiplexed_protocol
#transport.open
end
def teardown
#server.stop
#transport.close
end
def test_authenticate
#client.authenticate('test', 'test')
end
# And other tests...
end
Thanks in advance.

Uninitialized Constant Error from Ruby EventMachine Chat Server

I'm trying to build a chat server in ruby using EventManager. Needless to day, I'm new to Ruby and feeling a little over my head with the current error I am getting, as I have no clue what it means and a search doesn't return anything valuable. Here's some of the logistics-
(ive only implemented LOGIN and REGISTER so I'll only include those..)
user can enter-
REGISTER username password - registers user
LOGIN username password - logins user
I'm taking in the string of data the user sends, splitting it into an array called msg, and then acting on the data based on msg[0] (as its the command, like REGISTER, LOGIN, etc)
Here is my code, all contained in a single file- chatserver.rb (explanation follows):
require 'rubygems'
require 'eventmachine'
class Server
attr_accessor :clients, :channels, :userCreds, :userChannels
def initialize
#clients = [] #list of clients connected e.g. [192.168.1.2, 192.168.1.3]
#users = {} #list of users 'logged in' e.g. [tom, sam, jerry]
#channels = [] #list of channels e.g. [a, b, c]
#userCreds = {} #user credentials hash e.g. { tom: password1, sam: password2, etc }
#userChanels = {} #users and their channels e.g. { tom: a, sam: a, jerry: b }
end
def start
#signature = EventMachine.start_server("127.0.0.1", 3200, Client) do |con|
con.server = self
end
end
def stop
EventMachine.stop_server(#signature)
unless wait_for_connections_and_stop
EventMachine.add_periodic.timer(1) { wait_for_connections_and_stop }
end
end
# Does the username already exist?
def has_username?(name)
#userCreds.has_key?(name)
end
# Is the user already logged in?
def logged_in?(name)
if #users[name] == 1
true
else
false
end
end
# Did the user enter the correct pwd?
def correct_pass?(pass)
if #userCreds[name] == pass
true
else
false
end
end
private
def wait_for_connections_and_stop
if #clients.empty?
EventMachine.stop
true
else
puts "Waiting for #{#clients.size} client(s) to stop"
false
end
end
end
class Connection < EventMachine::Connection
attr_accessor :server, :name, :msg
def initialize
#name = nil
#msg = []
end
# First thing the user sees when they connect to the server.
def post_init
send_data("Welcome to the lobby.\nRegister or Login with REGISTER/LOGIN username password\nOr try HELP if you get stuck!")
end
# Start parsing incoming data
def receive_data(data)
data.strip!
msg = data.split("") #split data by spaces and throw it in array msg[]
if data.empty? #the user entered nothing?
send_data("You didn't type anything! Try HELP.")
return
elsif msg[0] == "REGISTER"
handle_register(msg) #send msg to handle_register method
else
hanlde_login(msg) #send msg to handle_login method
end
end
def unbind
#server.clients.each { |client| client.send_data("#{#name} has just left") }
puts("#{#name} has just left")
#server.clients.delete(self)
end
private
def handle_register(msg)
if #server.has_username? msg[1] #user trying to register with a name that already exists?
send_data("That username is already taken! Choose another or login.")
return
else
#name = msg[1] #set name to username
#userCreds[name] = msg[2] #add username and password to user credentials hash
send_data("OK") #send user OK message
end
end
end
EventMachine::run do
s = Server.new
s.start #start server
puts "Server listening"
end
Whew, okay, it's only the beginning, so not that complicated. Since I'm new to Ruby I have a feeling I'm just not declaring variable or using scope correctly. Here's the error output:
chatserver.rb:16:in start': uninitialized constant Server::Client
(NameError) from chatserver.rb:110:inblock in ' from
/Users/meth/.rvm/gems/ruby-1.9.3-p392#rails3tutorial2ndEd/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in
call' from
/Users/meth/.rvm/gems/ruby-1.9.3-p392#rails3tutorial2ndEd/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in
run_machine' from
/Users/meth/.rvm/gems/ruby-1.9.3-p392#rails3tutorial2ndEd/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in
run' from chatserver.rb:108:in<\main>'
ignore the slash in main in that last line.
line 108 is the last function- EventMachine::run do etc.
Any help would be appreciated, if I didn't provide enough info just let me know.
I would think that when you call EventMachine::start_server you need to give it your Connection class as the handler. Client is not defined anywhere.

EventMachine server and serial-port using SQL

I'm new to Ruby.
I'm trying to make an app that reads from a serial-port and puts values into a sqlite3 database. When a client connects via TCP socket he should recieve values from the db. Values written by the client should be sent via serial-port.
I have two questions regarding my app.
This would open one connection to the db on the main thread(?) and one for each client..
Is there a better way to use sqlite3?
I think i figured this out. sqlite3 is not thread safe by defaul,t so this seems like the way to do it..
How do i write to the serialport in the recieve_data method? Is it okay to make serial a global variable?
#!/usr/bin/env ruby
#
# server_1
require 'rubygems'
require 'eventmachine'
require 'sqlite3'
require 'em-serialport'
require 'json'
module SocketClient
def self.list
#list ||= []
end
def post_init
SocketClient.list << self
#db = SQLite3::Database.new( "data.db" )
values = []
#db.execute("SELECT * FROM values") do |row|
values << {row[0] => row[1]} #id => value
end
self.send_data "#{values.to_json}\n"
p "Client connected"
end
def unbind
SocketClient.list.delete self
#db.close
end
def receive_data data
p data
#How do i send via serialport from here??? serial.send_data data
end
end
db = SQLite3::Database.new( "data.db" )
EM.run{
EM.start_server '0.0.0.0', 8081, SocketClient
serial = EM.open_serial '/dev/tty.usbserial-xxxxxxxx', 9600, 8, 1, 0
serial.on_data do |data|
#Parse data into an array called values
db.execute("UPDATE values SET value = ? WHERE id = ?", values["value"], values["id"])
SocketClient.list.each{ |c| c.send_data "#{values.to_json}\n" }
end
}
db.close
Setup the constructor for your Socket client so that it will receive the shared serial connection.
module SocketClient
def initialize serial
#serial = serial
end
def receive_data data
p data
#serial.send_data data
end
Then pass it when you call EM.start_server
EM.run{
serial = EM.open_serial '/dev/tty.usbserial-xxxxxxxx', 9600, 8, 1, 0
EM.start_server '0.0.0.0', 8081, SocketClient, serial

How to disconnect redis client in websocket eventmachine

I'm trying to build a websocket server where each client establish its own redis connections used for publish and subscribe.
When the redis server is running I can see the two new connections being established when a client connects to the websocket server and I can also publish data to the client, but when the client drops the connection to the websocket server I also want to disconnect from Redis . How can I do this?
Maybe I'm doing it wrong, but this is my code.
#require 'redis'
require 'em-websocket'
require 'em-hiredis'
require 'json'
CLIENTS = Hash.new
class PubSub
def initialize(client)
#socket = client.ws
# These clients can only be used for pub sub commands
#publisher = EM::Hiredis.connect #Later I will like to disconnect this
#subscriber = EM::Hiredis.connect #Later I will like to disconnect this
client.connections << #publisher << #subscriber
end
def subscribe(channel)
#channel = channel
#subscriber.subscribe(channel)
#subscriber.on(:message) { |chan, message|
#socket.send message
}
end
def publish(channel,msg)
#publisher.publish(channel, msg).errback { |e|
puts [:publisherror, e]
}
end
def unsubscribe()
#subscriber.unsubscribe(#channel)
end
end
class Client
attr_accessor :connections, :ws
def initialize(ws)
#connections = []
#ws = ws
end
end
EventMachine.run do
# Creates a websocket listener
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8081) do |ws|
ws.onopen do
# I instantiated above
puts 'CLient connected. Creating socket'
#client = Client.new(ws)
CLIENTS[ws] = #client
end
ws.onclose do
# Upon the close of the connection I remove it from my list of running sockets
puts 'Client disconnected. Closing socket'
#client.connections.each do |con|
#do something to disconnect from redis
end
CLIENTS.delete ws
end
ws.onmessage { |msg|
puts "Received message: #{msg}"
result = JSON.parse(msg)
if result.has_key? 'channel'
ps = PubSub.new(#client)
ps.subscribe(result['channel'])
elsif result.has_key? 'publish'
ps = PubSub.new(ws)
ps.publish(result['publish']['channel'],result['publish']['msg']);
end
}
end
end
This version of em-hiredis supports close connection: https://github.com/whatupdave/em-hiredis
Here is how I would (and did many times) this:
instead of always opening and closing connections for each client you can keep 1 connection open per Thread/Fiber dependeing on what you are basing your concurrency on, that way if you are using a poll of Thread/Fibers once each one of them have its connections they will keep it and reuse them.
I did not worked much with websocket until now (I was waiting for a standard implementation) but I am sure you can apply that thinking to it too.
You can also do what rails/activerecord: keeo a pool of redis connection, each time you need to use a connection you request one, use it and realease it, it could look like this:
def handle_request(request)
#redis_pool.get_connection do |c|
# [...]
end
end
before yielding the block a connection is taken from the available ones and after it the connection is marked as free.
This was added to em-hiredis: https://github.com/mloughran/em-hiredis/pull/6

Resources