If I were using ActiveRecord, my puma config would establish the database connection as described in Heroku's guide Deploying Rails Applications with the Puma Web Server.
on_worker_boot do
ActiveRecord::Base.establish_connection
end
However, I am using ROM (Ruby Object Mapper).
I tried omitting the on_worker_boot block, but (predictably) the database connection is either not established, or is not established correctly, and the following error is raised.
PG::ConnectionBad: PQconsumeInput() SSL error: decryption failed or bad record mac
I've read the ROM Setup Guide but did not see anything relevant.
How do I establish the ROM connection in puma?
The solution seems to be to disconnect the gateway connection.
on_worker_boot do
ROM.env.gateways[:default].connection.disconnect
end
Related
I have a rack puma server hosted, and there are multiple HTTP request-response cycles. I'm querying the data from MongoDB. Quite often the server hangs, and if I press Ctrl+C it resumes , if i don't then after a couple of minutes I get the following error:
C:/Rubies/Ruby193/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/client.rb:127:in 'readpartial': An existing connection was forcibly closed by the remote host. (E rrno::ECONNRESET)
I have pretty good exception handling and loggers, but this is not caught anywhere. I have to restart the server again to proceed.
Please suggest.
Using activerecord outside rails, may I be confident all the connection drudgework is performed behind the curtains the same as inside rails?
In rails, activerecord does a great job establishing a connection pool and activating or closing connection as needed.
If I have a ruby daemon which calls a class file with:
ActiveRecord::Base.establish_connection(:production)
# more active_record tasks
can I assume on the following calls a connection from a pool is used?
Yup, calling ActiveRecord::Base.establish_connection will create a connection pool according to: https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb#L842
UPDATE: I tried changing the storage of the PostgreSQL connection object from a constant POSTGRES to a class variable ##pg. That didn't fix the issue. Then, I tried changing it to a global variable $pg. That seems to have worked! I still would like to eventually implement database connection pooling, however, so that I can have a pool of up to 5 (or so) database connections handling requests instead of just one global connection for all requests. END UPDATE
I'm hosting the Acani Chats REST Server on Heroku.
The first request works OK, but subsequent requests fail to connect to the PostgreSQL database.
I get the following errors:
Rack app error: #<PG::UnableToSend: SSL error: decryption failed or bad record mac>
Rack app error: #<PG::UnableToSend: no connection to the server>
What's going on?
In /config/application.rb, I define the constant POSTGRES to be the PostgreSQL connection object.
Should I be using a global or class variable instead of a constant to hold onto the connection instance?
In /config/routes.rb, I define the Rack call method.
I want to learn how to implement database connection pooling in Ruby for Rack with Puma and PostgreSQL.
Puma is threaded so you need a thread safe pool of connections to PostgreSQL, otherwise concurrent requests will all use the same connection concurrently, which is unexpected.
Please have a look to the connection_pool gem. It should help.
I run several sql statements in a transaction using Ruby pg gem. The problem that I bumped in is that connection times out on these queries due to firewall setup. Solution proposed here does not work, because it requires jdbc connections string, and I'm in Ruby (jRuby is not an option). Moving driver program to AWS to remove firewall is not an option either.
The code that I have is along the following lines:
conn = RedshiftHelper.get_redshift_connection
begin
conn.transaction do
# run my queries
end
ensure
conn.flush
conn.finish
end
I'm now looking into PG asynchronous API. I'm wondering if I can use is_busy to prevent firewall from timing out, or something to that effect. I can't find good documentation on the topic though. Appreciate any hints on that.
PS: I have solved this problem for a single query - I can trigger it asynchronously and track its completion using system STV_INFLIGHT Redshift table.Transaction does not work this way as I have to keep connection open.
Ok, I nailed it down. Here are the facts:
Redshift is based on Postgres 8.0. To check that, connect to Redshift instance using psql and see that it says "server version 8.0"
Keepalive requests are specified on the level of tcp socket (link).
Postgres 8.0 does not support keepalive option when specifying a connection string (link to 9.0 release changes, section E.19.3.9.1 on libpq)
PG gem in Ruby is a wrapper about libpq
Based on the facts above, tcp keepalive is not supported by Redshift. However, PG allows you to retrieve a socket that is used in the established connection. This means that even though libpq does not set keepalive feature, we still can use it manually. The solution thus:
class Connection
attr_accessor :socket, :pg_connection
def initialize(conn, socket)
#socket = socket
#pg_connection = conn
end
def method_missing(m, *args, &block)
#pg_connection.send(m, *args, &block)
end
def close
#socket.close
#pg_connection.close
end
def finish
#socket.close
#pg_connection.close
end
end
def get_connection
conn = PGconn.open(...)
socket_descriptor = conn.socket
socket = Socket.for_fd(socket_descriptor)
# Use TCP keep-alive feature
socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1)
# Maximum keep-alive probes before asuming the connection is lost
socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_KEEPCNT, 5)
# Interval (in seconds) between keep-alive probes
socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_KEEPINTVL, 2)
# Maximum idle time (in seconds) before start sending keep-alive probes
socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_KEEPIDLE, 2)
socket.autoclose = true
return Connection.new(conn, socket)
end
The reason why I introduce a proxy Connection class is because of Ruby tendency to garbage-collect IO objects (like sockets) when they get out of scope. This means that we now need connection and socket to be in the same scope, which is achieved through this proxy class. My Ruby knowledge is not deep, so there may be a better way to handle the socket object.
This approach works, but I would be happy to learn if there are better/cleaner solutions.
The link you provided has the answer. I think you just want to follow the section at the top, which has settings for 3 different OS'es, pick the one you are running the code on (the client to the Amazon service).
Look in this section: To change TCP/IP timeout settings - this is the OS that your code is running on (i.e. The client for the Amazon Service is your Server probably)
Linux — If your client is running on Linux, run the following command as the root user.
-- details omitted --
Windows — If your client runs on Windows, edit the values for the following registry settings under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters:
-- details omitted --
Mac — If your client is a Mac, create or modify the /etc/sysctl.conf file with the following values:
-- Details omitted --
I run redis2.4.16 on ec2 medium instance, the persistent is standard ebs, and i checked the redis log , found there is some log report "Reading from client: Connection reset " occurs every few hours， all my clients and server are in the same zone:ap-northeast-1a, and the operation system is ubuntu server 12.04. The client is the jredis + spring data redis 1.0.0.M4,Anyone can figure this out or give some advice, thanks!
below is the redis info command result:
redis_version:2.4.16
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:64
multiplexing_api:epoll
gcc_version:4.5.2
process_id:3265
uptime_in_seconds:2658600
uptime_in_days:30
lru_clock:561139
used_cpu_sys:29421.34
used_cpu_user:10731.37
used_cpu_sys_children:20022.24
used_cpu_user_children:75702.79
connected_clients:44
connected_slaves:1
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
used_memory:1111572800
used_memory_human:1.04G
used_memory_rss:1133101056
used_memory_peak:1112071512
used_memory_peak_human:1.04G
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.0.0
loading:0
aof_enabled:0
changes_since_last_save:1343
bgsave_in_progress:0
last_save_time:1368760178
bgrewriteaof_in_progress:0
total_connections_received:904643
total_commands_processed:592333133
expired_keys:0
evicted_keys:0
keyspace_hits:443393839
keyspace_misses:30383206
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:359082
vm_enabled:0
role:master
slave0:xxx,online
db0:keys=364558,expires=0
As you can see from logs, redis tries to communicate with a client that has closed its connection.
Thats probably because some of your client are not closing the connection with redis after they are done with it.
This can eventually lead redis to run out of connections (depending on your connection limits and the amount of traffic you have)
An easy solution for this is to set a connection timeout (0 as 'no timeout' by default) in redis.conf so that redis will close opened connection after X seconds.
Note: you should include the output of redis config get * when asking this kind of questions ;)