What are the consequences, (if any) not calling the conn.logoff() method after the following script when connecting to an Oracle database using the Ruby OCI8 library.
conn = OCI8.new('scott', 'tiger')
num_rows = conn.exec('SELECT * FROM emp') do |r|
puts r.join(',')
end
puts num_rows.to_s + ' rows were processed.'
The reason I'm asking because we're experiencing slow downs with other applications that connect to this same Oracle db.
Thanks
I would imagine that when the Ruby process exits, the session will be killed automatically.
You could check by querying v$session to see if the ruby process is still connected to Oracle after Ruby exits.
Given only the information in your question, its really impossible to say what could be causing slowdowns - there are so many variable.
If you don't call conn.logoff(), the connection is alive even though it is garbage-collected until the ruby process exits.
The problem is fixed in ruby-oci8 2.1, which have not been released yet though.
Related
I am using Ruby SQLite3 (1.13.11) on macos (Ruby 2.0.0-p247) to create a few databases for my application. I need to set some pragmas, but I am not sure I am doing the right thing. This is what I do to set PRAGMA synchronous = OFF
db = SQLite3::Database.new("test.db")
db.synchronous
2
db.synchronous = 0
db.synchronous
0
This seems to work, but when I open my test.db with DB Browser for SQLite, synchronous is still set to Full.
I also tried
db.execute("PRAGMA synchronous = OFF")
with the same result.
Is synchronous associated with the connection? Is this the case for all PRAGMAS?
Some PRAGMAs are associated with the current database connection so they are not persisted between sessions. For example: journal_mode
For a list of all PRAGMAS refer to this link
I think the question does not pin-point to the real problem, I have difficulties to nail it down precisely and concisely.
I have a gem that implements i.e. MySQL-database "queries" (also inserts, updates...)
module DBGEM::Query
def self.client settings=DBGEM.settings
##client ||= Mysql2::Client.new settings
end
def query_this
client.query(...)
end
def process_insert_that list_of_things
list_of_things.each do |thing|
# process
client.query(...)
end
end
Furthermore, this gem is used by a sinatra app sitting on a forking webserver like puma.
Within the sinatra-app i can now
get '/path' do
happy = DBGEM::Query.query_this
# process happy
great = DBGEM::Query.process_insert_that 1..20
# go on
end
I like that API and this code should open only one database connection.
But as far as I understood, because the code within the 'get' definition is not guaranteed to be the only one accessing the DBGEM::Query stuff at that time, weird things could happen (through race-conditions, shared internal state?).
Is there a clever way to keep the nice syntax and the connection sharing without boilerplate object creation (query = DBGEM::Query.new() #...) wrapping the stuff in a block (DBGEM::Query.process do |query| #...)?
The example above is obviously simplified. The sinatra handling might be more involved, the Queries actually done in a Service object etc.pp. Also, afaiu in a forking webserver environment, the GC would destroy the client (closing the connection - thats how mysql2 is implemented).
I think that the connection will not be closed every time.
##client is shared between DBGEM::Query object itself (in Ruby modules and classes are also objects) and all the instances of that object (to be precise: all the instances of classes to which that object is mixed in).
So, this variable will live as long as the DBGEM::Query object will live.
You can check out when DBGEM::Query object will be garbage collected, by defining finalizer logging a text and observe the server console.
module DBGEM::Query
ObjectSpace.define_finalizer(self, proc { print 'garbage collected' })
..
end
Im not sure, however I guess that DBGEM::Query object will be garbage collected only when you stop the server.
As it goes for weird "things could happen", I believe you mean potential conflicts, race conditions, situations where you create double records, or update the same record nearly at the same time overwriting something, etc. And when that happen you lose data integrity.
IMHO you can't prevent it by allowing only one client instance. I'd suggest aiming for solid database design (unique constrains, indexes, foreign keys, validations) which can raise errors when race condition occure and then handling that errors in your application.
I'm using ActiveRecord in a Ruby script, without using rails, just running the Ruby script. The script kicks off threads which access the database. Eventually, I get:
could not obtain a database connection within 20.000 seconds
database configuration:
The pool is set for 10 connection. The timeout is set for 20 seconds.
I tried without using connection poll calls directly, but I'm not clear on how that is supposed to be managed. So I put cxn = ActiveRecord::Base.connection_pool.checkout and ActiveRecord::Base.connection_pool.checkin(cxn) around the database access portions of the code. I still get the error.
I put some debug in the script, and I can see the checkout/checkin calls happening. There are 7 successful checkout/checkins. There are 13 total checkouts, so 6 left open.
I'm also seeing:
undefined method `run_callbacks'
when the timeout occurs.
Thank you,
Jerome
You need to explicitly release connections back to the ActiveRecord connection pool. You either need to explicitly call ActiveRecord::Base.clear_active_connections! or else define
a helper method which does the following:
def with_connection(&block)
ActiveRecord::Base.connection_pool.with_connection do
yield block
end
end
which you would use as follows:
def my_method
with_connection do
User.where(:id => 1)
end
end
which should release your connection when the block exits.
Be warned that ActiveRecord uses the current Thread ID to manage connections, so if you are spawning off threads each thread needs to release connections once done with them. Your Threads are most likely not clearing connections properly, or you are spawning more threads than you have connections available in your connection pool.
I have a ruby script that needs to run continually on the server. I've daemonized it using the daemon gem, and in my script I have it running in an infinite loop, since the daemon gem handles starting and stopping of the process that kicks off my script. In my script, I start out by setting up my DB instance using the Sequel gem and tiny_tds. Like so:
DB = Sequel.connect(adapter: 'tinytds', host: MSSQLHost, database: MSSQLDatabase, user: MSSQLUser, password: MSSQLPassword)
Then I have a loop do that is my infinite loop. Inside that, I test to see if I have a connection using DB.test_connection and then I query the DB every second or so to check if there is new content using a query such as:
DB['SELECT * FROM dbo.[MyTable]'].all do |row|
# MY logic here
# As part of my logic I test to see if I need to delete this row in the table and if so I use
DB.run('DELETE FROM dbo.[MyTable] WHERE some condition')
end
Then at the end of my logic, just before I loop again, I do:
sleep 1
DB.disconnect
All of this works great for about an hour to an hour and a half with everything checking the table, doing the logic, deleting rows, etc., then it dies and gives me this error message TinyTds::Error: Adaptive Server connection timed out
My question, why is that happening? Do I need to reformat my code in a different way? Why doesn't the DB.test_connection do what it is advertised to do? The documentation on that says it checks for a connection in the connection pool, and uses it if it finds it, and creates a new one otherwise.
Any help would be much appreciated
DB.test_connection just acquires a connection from the connection pool, it doesn't check that the connection is still valid (it must have been valid at one point or it wouldn't be in the pool). There's no way that a connection is still valid without actually sending a query. You can use the connection_validator extension that ships with Sequel if you want to do that automatically.
If you are loading Sequel before forking, you need to make sure you call DB.disconnect before forking, otherwise you can end up with multiple forked processes sharing the same connection, which can cause many different issues.
I finally ended up just putting a rescue statement in there that caught this, and re-ran my line of code to create the DB instance, yes, it puts a warning in my log about already setting that instance, but I guess I could just make that not a contstant an that would go away. Anyway, it appears to be working now, and the times it does timeout, I'm recovering gracefully from those. I just wish I could have figured out why it was/is disconnecting like it is.
I have an application that essentially does this:
queue = SizedQueue.new(2)
Thread.new do
conn = OCI8.new(DSN)
cursor = conn.exec([a query])
cursor.fetch { |rec| queue << rec }
queue << :queue_ended
cursor.close
conn.logoff
end.tap { |t| t.abort_on_exception = true }
until a = queue.pop == :queue_ended
do_things(a)
end
The issue comes when do_things raises an error. The thread seems not to abort. The reason I believe it's the oci8 is because we had this code running with a different set of functions/libraries and everything was fine. The other reason i think it is OCI8 because the connection remains open, surely if the thread had aborted the connection would close?
I occasionally have an issue raising a KeyBoardInterrupt in irb with long to generate queries as well.
Apologies if this isn't much to go on, I don't have much to go on - I can't recreate the issue without using thread and as we all know it's fairly difficult to debug.
N.B I am running with ruby OCI8 (most recent version (2.1.5) on rb 1.9.3).
Thanks for your help in advance
Turn out this isn't an issue, or at least doesn't occur in Ruby 2.0.0; I don't know why though.