WARN TCPServer Error: Address already in use - bind(2) in linux EC2 and Heroku servers - ruby

[2013-01-29 09:17:50] INFO WEBrick 1.3.1
[2013-01-29 09:17:50] INFO ruby 1.8.7 (2012-10-12) [i386-linux]
[2013-01-29 09:17:50] WARN TCPServer Error: Address already in use - bind(2)
[2013-01-29 09:17:50] INFO WEBrick::HTTPServer#start: pid=4107 port=8080
When I run the file attached below in linux I get the error described. I tried all possible command and strategies online to listen to processes (including rogue) and kill them. I did this in lots of ports. No luck.
As soon as I run the script in Mac OS and it works. Nevertheless I have to mount it on a server and clients have to communicate with it. It happens on every instance of amazon ec2 and on heroku. I have seen this error one too many times and spend many hours trying to fix it. I configured the security group of ec2 instances and still did not work. I am beyond desperate. At this point I have to think that the problem must be WEBrick itself or something in my code.
require 'webrick'
require 'uri'
require 'net/http'
$own_address = 8080
class AuctionInfo
# The representation is a hash mapping item names to [highest_bidder, highest_bid, end_time]
def initialize
#data = {}
end
def new_item(item, endTime)
#data[item] = ["UNKNOWN", 0, endTime]
end
def bid(item, bid, client)
if #data.has_key?(item)
endTime = #data[item][2]
if #data[item][1].to_i < bid.to_i and Time.new.to_i < endTime.to_i
#data[item] = [client, bid, endTime]
end
end
end
def get_status(item)
if #data.has_key?(item)
return #data[item][0]
end
end
def winner(item)
if #data.has_key?(item)
if #data[item][2].to_i + 1 <= Time.new.to_i
return #data[item][0]
else return "UNKNOWN"
end
end
end
def reset
#data = {}
end
def has_item(item)
return #data.has_key?(item)
end
def get_data
return {}.replace(#data)
end
end
class StartAuctionServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
if request.query['name'] and request.query['end_time']
#data.new_item(request.query['name'], request.query['end_time'].to_i)
end
response.status = 200
end
alias_method :do_GET, :do_POST
end
class BidServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
if request.query['name'] and request.query['client'] and request.query['bid']
#data.bid(request.query['name'], request.query['bid'].to_i, request.query['client'])
end
response.status = 200
end
alias_method :do_GET, :do_POST
end
class StatusServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
if request.query['name']
response.body = #data.get_status(request.query['name'])
end
response.status = 200
end
alias_method :do_POST, :do_GET
end
class WinnerServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
if request.query['name']
response.body = #data.winner(request.query['name'])
end
response.status = 200
end
alias_method :do_POST, :do_GET
end
class ResetServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_POST(request, response)
#data.reset
response.status = 200
end
alias_method :do_GET, :do_POST
end
class RandomServlet < WEBrick::HTTPServlet::AbstractServlet
def initialize(server, data)
#data = data
end
def do_GET(request, response)
response.status = 200
response.body = #data.get_data.to_s
end
alias_method :do_POST, :do_GET
end
data = AuctionInfo.new
server = WEBrick::HTTPServer.new(:Port => $own_address)
server.mount '/start_auction', StartAuctionServlet, data
server.mount '/bid', BidServlet, data
server.mount '/status', StatusServlet, data
server.mount '/winner', WinnerServlet, data
server.mount '/rst', ResetServlet, data
server.mount '/', RandomServlet, data
trap("INT") { server.shutdown }
server.start

Have you checked whether the linux server is running apache, tomcat, trinidad or any other web server? Odds are one of them is already running on port 8080 on the server.
lsof is a useful command. Try lsof | grep 8080 and see whether anything shows up

Related

What is wrong with my Celluloid actors

I'm playing with celluloid gem. The example works well, but when I press Ctrl-C I get the unexpected message:
^CD, [2015-10-07T09:53:19.784411 #16326] DEBUG -- : Terminating 8 actors...
and after few seconds, I get the error:
E, [2015-10-07T09:53:29.785162 #16326] ERROR -- : Couldn't cleanly terminate all actors in 10 seconds!
/usr/local/rvm/gems/ruby-2.0.0-p353/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run_machine': Interrupt
from /usr/local/rvm/gems/ruby-2.0.0-p353/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run'
Strange that I create only 4 actors, not 8, and my TERM, INT signals handler isn't be called.
#!/usr/bin/env ruby
require './config/environment'
opts = CommandlineOptions.new.to_h
iface = opts[:iface] || '0.0.0.0'
port = opts[:port] || 3000
App.logger.info('Starting communication server')
connections = Connections.new
local_inbox = LocalQueue.new
auth_server = AuthServer.new(connections, local_inbox)
inbox_service = InboxService.new('inbox', iface, port)
inbox_service.async.process_inbox(local_inbox) # <--------
remote_outbox_name = "outbox_#{iface}:#{port}"
outbox_service = OutboxService.new(connections)
outbox_service.async.subscribe(remote_outbox_name) # <--------
conn_server_opts = { host: iface, port: port }
conn_server_opts.merge!(auth_server.callbacks)
conn_server = ConnServer.new(conn_server_opts)
%W(INT TERM).each do |signal|
trap(signal) do
info("Shutting down...")
conn_server.stop
end
end
conn_server.start
Here InboxService is an actor which creates another actor - there are 2 actors, then OutboxService also creates one actor, so I got created 4 actors.
require 'redis'
require 'celluloid/current'
class InboxServiceActor
include Celluloid
def initialize(remote_inbox_name)
#remote_inbox_name = remote_inbox_name
create_redis_connection
end
def publish(full_msg)
#redis.publish(#remote_inbox_name, full_msg)
end
private
def create_redis_connection
#redis = Redis.new
end
end
require 'json'
require 'redis'
require 'celluloid/current'
class OutboxServiceActor
include Celluloid
include HasLoggerMethods
def initialize
create_redis_connection
end
def subscribe(remote_outbox_name, &block)
#redis.subscribe(remote_outbox_name) do |on|
on.message do |_channel, full_msg|
debug("Outbox message received: '#{full_msg}'")
hash = parse_msg(full_msg)
block.call(hash['signature'], hash['msg']) if message_valid?(hash)
end
end
end
private
def create_redis_connection
#redis = Redis.new
end
def parse_msg(full_msg)
JSON.parse(full_msg)
rescue JSON::ParserError
error('Outbox message JSON parse error')
nil
end
def message_valid?(msg)
msg.is_a?(Hash) && msg.key?('signature') && msg.key?('msg') ||
error('Invalid outbox message. Should '\
'contain "signature" and "msg" keys') && false
end
end

API integration error HTTParty

I'm learning how to work with HTTParty and API and I'm having an issue with my code.
Users/admin/.rbenv/versions/2.0.0-p481/lib/ruby/2.0.0/uri/generic.rb:214:in `initialize': the scheme http does not accept registry part: :80 (or bad hostname?)
I've tried using debug_output STDOUT both as an argument to my method and after including HTTParty to have a clue but with no success. Nothing gets displayed:
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
self.class.base_uri "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
self.class.base_uri "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end
I verified my URL which is fine so I'm lost as to where the problem is coming from.
I tested with a static base_uri and it doesn't change anything.
The odd thing is when I do:
HTTParty.get("https://euw.api.pvp.net/observer-mode/rest/consumer/getSpectatorGameInfo/EUW1/randomid?api_key=myapikey")
Everything is working fine and I'm getting a response.
HTTParty doesn't seem to like the way you set your base_uri.
Unless you need it to be like that just add another attr_reader called domain and it will work.
require 'httparty'
class LolObserver
include HTTParty
default_timeout(1) #timeout after 1 second
attr_reader :api_key, :playerid, :domain
attr_accessor :region
def initialize(region,playerid,apikey)
#region = region_server(region)
#playerid = playerid
#api_key = apikey
end
def region_server(region)
case region
when "euw"
#domain = "https://euw.api.pvp.net"
self.region = "EUW1"
when "na"
#domain = "https://na.api.pvp.net"
self.region = "NA1"
end
end
def handle_timeouts
begin
yield
#Timeout::Error, is raised if a chunk of the response cannot be read within the read_timeout.
#Timeout::Error, is raised if a connection cannot be created within the open_timeout.
rescue Net::OpenTimeout, Net::ReadTimeout
#todo
end
end
def base_path
"/observer-mode/rest/consumer/getSpectatorGameInfo"
end
def current_game_info
handle_timeouts do
url = "#{domain}/#{ base_path }/#{region}/#{playerid}?api_key=#{api_key}"
puts '------------------------------'
puts url
HTTParty.get(url,:debug_output => $stdout)
end
end
end

Ruby Thrift::TransportException End of File Reached

I made a Ruby web server based on Apache Thrift, but the client (also in ruby, for unit tests) refuses to work and keeps telling me either Thrift::TransportException: Could not connect to 127.0.0.1:8001: Connection refused - connect(2) for 127.0.0.1:8001, or Thrift::TransportException: end of file reached. Tried a bunch of different server implementations and transports, and that doesn't seem to work.
When the server is running, lsof -i :8001 shows
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ruby 19073 lafickens 9u IPv4 0x878e6fd36b981a71 0t0 TCP *:vcom-tunnel (LISTEN)
So I think the server is functioning.
Here's the server code:
class Server
attr_reader :name
def initialize(name)
#LOGGER = Logger.new $stdout
#name = name
#started = false
#processor = ::Thrift::MultiplexedProcessor.new
#processor.register_processor 'User Service', Thrift::UserService::Processor.new(Handlers::UserServiceHandler.new)
#processor.register_processor 'Sync Service', Thrift::SyncService::Processor.new(Handlers::SyncServiceHandler.new)
end
def start
#transport = ::Thrift::ServerSocket.new(Options.get('port'))
#transport_factory = ::Thrift::BufferedTransportFactory.new
#protocol_factory = ::Thrift::CompactProtocolFactory.new
#server = ::Thrift::ThreadPoolServer.new #processor, #transport, #transport_factory, #protocol_factory
#server_thread = Thread.new {
#server.serve
}
#started = true
#LOGGER.info('Server started successfully')
end
def stop
return if #server_thread.nil?
#server_thread.exit
#transport.close
#started = false
#LOGGER.info('Server stopped successfully')
end
def restart
stop
start
end
def started?
#started
end
end
Client code (actually unit tests)
class TestUserServiceHandler < Test::Unit::TestCase
def setup
#server = Billboard::Server.new 'test handler'
#server.start
#port = Billboard::Options.get 'port'
#transport = ::Thrift::BufferedTransport.new(::Thrift::Socket.new('127.0.0.1', #port))
#binary_protocol = ::Thrift::BinaryProtocol.new #transport
#multiplexed_protocol = ::Thrift::MultiplexedProtocol.new #binary_protocol, 'mprotocol'
#client = Billboard::Thrift::UserService::Client.new #multiplexed_protocol
#transport.open
end
def teardown
#server.stop
#transport.close
end
def test_authenticate
#client.authenticate('test', 'test')
end
# And other tests...
end
Thanks in advance.

rails sidekiq background process

i'm having an issue configuring the sidekiq server, the process seems to be running in the foreground as soon as i refresh my page. /consumers/fetch i need to put it in the background permanently.
consumers_controller.rb
require 'kafka'
class ConsumersController < ApplicationController
def fetch
#consumer = Kafka::Consumer.new( { :host => ENV["host"],
:port => ENV["port"],
:topic => ENV["topic"]})
#consumer.loop do |message|
logger.info "-------------#{message.inspect}--------------"
logger.info "-------------#{message.first.payload.inspect}--------------"
unless message.blank?
ConsumerWorker.perform_async(message.first.payload)
end
end
end
end
consumer_worker.rb
class ConsumerWorker
include Sidekiq::Worker
def perform(message)
payload = message.first["payload"]
hash = JSON.parse(payload)
return #message = Message.new(hash) if hash["concern"] == 'order_create' or hash["concern"] == 'first_payment'
end
end
message.rb
class Message
attr_reader :bundle_id, :order_id, :order_number, :event
def initialize(message)
#payload = message["payload"]
#bundle_id = #payload["bundle_id"]
#order_id = #payload["order_id"]
#order_number = #payload["order_number"]
#event = message["concern"]
end
end
I think you need to move this block
#consumer.loop do |message|
end
inside your worker somehow, as I think the consumption is done after block execution.

Pipe data from HTTP GET to HTTP POST/PUT

I'd like to stream data from an HTTP GET request to an HTTP POST or PUT request. I'd prefer to use Ruby and have already made an attempt using EventMachine and EM-HTTP-Request.
Here's my attempt, to be called using:
HttpToS3Stream.new(src_url, dest_bucket, dest_key, aws_access_key_id, aws_secret_access_key)
http_to_s3_stream.rb
require 'em-http-request'
class HttpToS3Stream
def initialize(http_url, s3_bucket, s3_key, s3_access_key_id, s3_secret_access_key)
#http_url = http_url
#s3_bucket = s3_bucket
#s3_key = s3_key
#s3_access_key_id = s3_access_key_id
#s3_secret_access_key = s3_secret_access_key
go
end
private
def go
EM.run {
# initialize get stream, without listener does not start request
#get_stream = HttpGetStream.new(#http_url)
# initialize put stream, send content length, request starts
#put_stream = S3PutStream.new(#s3_bucket, #s3_key, #s3_access_key_id, #s3_secret_access_key, #get_stream.content_length)
# set listener on get stream, starts request, pipes data to put stream
#get_stream.listener = #put_stream
}
end
end
http_get_stream.rb
require 'httparty'
require 'em-http-request'
class HttpGetStream
def initialize(http_url, listener = nil)
#http_url = http_url
self.listener = listener
end
def listener=(listener)
#listener = listener
listen unless #listener.nil?
end
def content_length
response = HTTParty.head(#http_url)
response['Content-length']
end
private
def listen
http = EventMachine::HttpRequest.new(#http_url).get
http.stream do |chunk|
#listener.send_data chunk
end
http.callback do |chunk|
EventMachine.stop
end
end
end
s3_put_stream.rb
require 'em-http-request'
class S3PutStream
def initialize(s3_bucket, s3_key, s3_access_key_id, s3_secret_access_key, content_length = nil)
#s3_bucket = s3_bucket
#s3_key = s3_key
#s3_access_key_id = s3_access_key_id
#s3_secret_access_key = s3_secret_access_key
#content_length = content_length
#bytes_sent = 0
listen
end
def send_data(data)
#bytes_sent += data.length
#http.on_body_data data
end
private
def listen
raise 'ContentLengthRequired' if #content_length.nil?
#http = EventMachine::HttpRequest.new(put_url).put(
:head => {
'Content-Length' => #content_length,
'Date' => Time.now.getutc,
'Authorization' => auth_key
}
)
#http.errback { |error| puts "error: #{error}" }
end
def put_url
"http://#{#s3_bucket}.s3.amazonaws.com/#{#s3_key}"
end
def auth_key
"#{#s3_access_key_id}:#{#s3_secret_access_key}"
end
end
HttpToS3Stream.new(src_url, dest_bucket, dest_key, aws_access_key_id, aws_secret_access_key)
It seems to be working but always stops at 33468 bytes. Not sure what that's about. Now, by passing chunks directly to #listener.send_data, it is processing the entire GET body. However, the upload is not occurring successfully.
How can I get this to work? And is there a name for what I'm trying to do? I'm having trouble searching for more information.
Any help is appreciated.

Resources