How to print postgres "raise notice" output from Sequel? - ruby

Raise notice is typically used to debug PSQL scripts in postgres (link).
The docs say that there's some kind of support for printing notices when using the pg gem, but there's no info on how to use this proc, what it yields, possible (probable?) caveats etc.
Does anyone have a working code example for production and/or development? Ideally, I'm looking for a solution that allows PG notices to be printed out in development when Sequel logging is enabled.
When I do:
DB = Sequel.connect(
ENV['DATABASE_URL'],
notice_receiver: lambda{ |x| binding.pry }
)
the notice_receiver lambda never gets called once I execute a function that raises a notice. I.e
[1] pry(#<Psql::CalculateMasterBalancesTest>)> DB.select{ |o| Sequel.function(:emit_notice) }.first
I, [2017-05-17T16:51:56.746003 #23139] INFO -- : (0.000335s) SELECT emit_notice() LIMIT 1
=> {:emit_notice=>""}
where emit notice is:
CREATE OR REPLACE FUNCTION emit_notice()
RETURNS VOID AS $$
BEGIN
RAISE NOTICE 'NOTICE ME!!!';
END;
$$ LANGUAGE plpgsql;
and it works from PgAdmin:
NOTICE: NOTICE ME!!!
Total query runtime: 21 ms.
1 row retrieved.
UPDATE
Alejandro C gave a good working example, and it seems that notices don't get distributed with the notice_receiver hook. For example:
Sequel.connect(DB.opts.merge(:notice_receiver=>proc{|r| puts r.result_error_message})){ |db|
db.do("BEGIN\nRAISE NOTICE 'foo';\nEND;")
}
prints nothing, and:
Sequel.connect(DB.opts.merge(:notice_receiver=>proc{|r| puts r.result_error_message})){ |db|
db.do("BEGIN\nRAISE WARNING 'foo';\nEND;")
}
Prints
WARNING: foo
Since Sequel just calls set_notice_receiver from PG, I guess I should file a bug report with PG.
EDIT 2
Yet when I try things just with the PG gem I get
conn = PG.connect( :dbname => 'db_test', user: 'test', password: 'test', host: '127.0.0.1' )
conn.set_notice_receiver{|r| puts r.result_error_message }
conn.exec("SELECT emit_notice()")
NOTICE: NOTICE ME!!!
=> #<PG::Result:0x0000000405ac18 status=PGRES_TUPLES_OK ntuples=1 nfields=1 cmd_tuples=1>
So at this point I'm a bit confused...
EDIT 3
Posted an issue GitHub...
EDIT 4
Ah, apparently there's another options you need to use, client_min_messages needs to be set to :notice as so:
DB = Sequel.connect(
ENV['DATABASE_URL'],
notice_receiver: proc{|r| puts r.result_error_message},
client_min_messages: :notice
)
and this works

You pass in your own proc which gets the notice as a string. To have it trigger on notices and not just warnings and above, use client_min_messages. For example:
a = nil
Sequel.connect(
DB.opts.merge(
notice_receiver: proc{|r| a = r.result_error_message},
client_min_messages: :notice)) { |db|
db.do("BEGIN\nRAISE WARNING 'foo';\nEND;")
}
a == "WARNING: foo\n" # true

Related

Threading sqlite connections in Ruby

I've been trying to get my ruby script threaded since yesterday. I've since opted for SQLite to save data, with the parallel gem to manage concurrency.
I've built a quick script for testing, but I'm having trouble getting the threading working; the database is locked. I've added db.close to the end, which doesn't help, and I've tried adding sleep until db.closed?, but that just sleeps indefinitely. What am I doing wrong?
The error is "database is locked (SQLite3::BusyException)".
Here's my code:
require 'sqlite3'
require 'pry'
require 'parallel'
STDOUT.sync = true
db = SQLite3::Database.new "test.db"
arr = [1,2,3,4,5,6,7,8,9,10]
rows = db.execute <<-SQL
create table test_table (
original string,
conversion string
);
SQL
def test(num)
db = SQLite3::Database.new "test.db"
puts "the num: #{num}"
sleep 4
{ num => num + 10}.each do |pair|
db.execute "insert into test_table values (?, ?)", pair
end
db.close
end
Parallel.each( -> { arr.pop || Parallel::Stop}, in_processes: 3) { |number| test(number) }
SQLite is threadsafe by default (using its "serialized" mode) and the ruby wrapper apparently supports this to whatever extent it needs to. However, it's not safe across processes, which makes a certain sense since the adapter or engine probably has to negotiate some state in the process to prevent locks.
To fix your example change in_processes to in_threads

How to access to various databases with Ruby?

I am building a module in Ruby to read metadata from source tables in various databases.
I wrote a small program to test with PostgreSQL:
#!/usr/bin/ruby
require 'pg'
begin
puts "start"
puts 'Version of libpg: ' + PG.library_version.to_s
con = PG.connect(host: 'localhost', dbname: 'rdv_app_dev', user: 'rdv_app', password: 'rdv_app')
puts con.server_version
pst = con.exec "SELECT * FROM users"
pst.each do |row|
puts "%s %s " % [ row['id'], row['email'] ]
end
puts 'There are %d columns ' % pst.nfields
puts 'The column names are:'
pst.fields.each do |f|
puts pst.fnumber(f).to_s + ' ' + f + ' ' + pst.ftype(pst.fnumber(f)).to_s
end
rescue PG::Error => e
puts e.message
ensure
pst.clear if pst
con.close if con
puts "stop"
end
It works fine, but it uses functions that are specific to Postgres. I need to have it working for any database without re-coding it for each one of them.
I read about Ruby-DBI, but it looks to be out of date, since it did not evolve for 7 years.
Is there a generic solution for accessing a database with Ruby ?
ActiveRecord is by far the most popular (see RubyGems stats). DataMapper is very similar, but more lightweight and makes changing the database quicker. I'm not familiar with sequel.
These gems introduce their own syntax for communicating with the database that is intended to abstact away the database-specific implementation details. I.e. a query like User.where(verified: true).includes(:posts).order(created_at: :desc) will list users ordered by most-recent creation date and include their' posts (performing a join behind the scenes). This Ruby syntax will compile to db-specific code based on what adapter and configuration you've specified.
Look into Sinatra and DataMapper; there are many tutorials.
Also look into Rails and how to configure Rails to use MySQL or Postgres instead of (its default) SQLite. You will find that the ORM (ActiveRecord) code doesn't change regardless of which you use.

What is "return can't jump across threads" error mean?

I have a pair of Puppet custom functions, one of which, namely am_func_cluster, returns a hash of array of currently running instances (reading a file as input) and the second one call that function, iterate over the array and returns the first successful one that listening to port 22. Here is the 2nd function:
module Puppet::Parser::Functions
newfunction(:am_func_head, :type => :rvalue ) do |args|
Puppet::Parser::Functions.function('am_func_cluster')
mCls = function_am_func_cluster(['/opt/running-inst.txt'])
cls = args[0].to_sym if args[0].is_a? String
require 'socket'
require 'timeout'
mCls[cls].each do |dns|
begin
Timeout::timeout(1) { TCPSocket.new(dns, 22)
return (dns if mCls.key?(cls)) || 'undefined'
}
break
rescue SocketError
rescue Timeout::Error
end
end
end
end
upon running, it returns this error:
Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: return can't jump across threads at
/etc/puppet/manifests/nodes.pp:19 on node ip-10-0-9-130.xxx
It works just fine, if I comment out the begin ... end bit in the script. Google didn't favor much in this case, so far. Does anyone one know what that error means or what am I doing wrong. Still don't find myself very efficient understanding the errors returned by Ruby. Any pointer much appreciated. Cheers!!
As it says, you cannot use return there. Use break to escape from the timeout block:
require "timeout"
Timeout.timeout(3){break "foo"}
# => "foo"
and you should not use break outside of it.
As a general tip, it is easy to confuse return, break, and next. If one of them does not work, try another.

ODBC on Mac Lion with Ruby Sequel

I'm having issues getting Sequel to connect to a MS SQL Database.
I have installed the following:
UnixODBC
FreeTDS
I have configured both software packages, and the following commands allow me to connect to my database without a problem:
isql
tsql
osql
However, when I try it from Ruby code using the Sequel.odbc command, I receive the following error:
ODBC::Error: IM003 (0) [iODBC][Driver Manager]Specified driver could not be loaded.
This is as far as I can get. I used to receive another error first, but managed to solve that by redoing the configuration part. Guess I missed something there.
EDIT
This is the code for my base talker class. It basically loads a YAML file like rails does, holding the database settings and establishes a connection to the database.
This seems to be working, trying it manually returns me a DB object from sequel:
module Talkers
require 'yaml'
require 'sequel'
class BaseTalker
# This function will load the desired settings as a hash from the database.yml file
# in the config folder. The data will be returned as a hash following the standard
# YAML structure.
def self.load_config(name)
cfg = YAML::load(File.open(File.join(ENV['API_ROOT'], 'config', 'database.yml')))
cfg.key?(name) ? cfg[name] : nil
end
# This function will establish a connection with the Florensia User database and return
# the Sequel database object, so that queries can be executed against the database.
def self.connect_to_user_db
settings = self.load_config("florensia_user_#{ENV['RACK_ENV']}")
Sequel.odbc settings['dsn'], :db_type => settings['adapter'], :user => settings['user'], :password => settings['password']
end
end
end
The class below inherits from the talker and performs certain actions for a User. It contains the DB logic specific to the game. When I call this logic, I receive the errors:
module Talkers
require 'yaml'
require 'sequel'
class LoginDbTalker < BaseTalker
#
# Bans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly set:
# - id : The identifier of the account.
# - gm_name : The name of the GM setting the ban.
# - punish_code : The punishment code being applied on the account.
# - days : The duration of the ban in days, starting from today.
#
# The function will return true if the ban has been properly set; otherwise the function
# will return false.
def self.ban_user(options = {})
return false if options.empty?
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :apply_ban)
ps.call(
:punishcode => options[:punish_code],
:punishstory => "Banned by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => (options[:days].to_i == -1) ? (Date.today + (30 * 265)) : (Date.today + options[:days].to_i))
true
rescue Exception => e
puts "Exception caught in ban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}, punish_code=#{options[:punish_code]}, days=#{options[:days]}"
false
end
#
# Unbans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly lifted:
# - id : The identifier of the account.
# - gm_name : The name of the GM removing the ban.
#
# The function will return true if the ban has been properly lifted; otherwise the function
# will return false.
def self.unban_user(options = {})
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :lift_ban)
ps.call(
:punishcode => '0',
:punishstory => "Ban removed by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => Date.today
)
true
rescue Exception => e
puts "Exception caught in unban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}"
false
end
#
# Kicks the specified User from the game, regardless on the server or character he currently is on.
# This requires a direct connection to the game servers so a specialized command can be sent that
# instruct the server to close the connection with the offending player.
# The function returns true if the kick succeeded; otherwise false.
def self.kick_player(id)
false
end
end
end
Calling any of the ban/unban functions results in the error message.
EDIT2
I've added the folder /Library/ODBC and linked all config files to there for iODBC. This removes the error I had before and now brings me this error:
ODBC::Error: 01000 (20002) [FreeTDS][SQL Server]Adaptive Server connection failed
So it seems I made some progress again
I recommend you use the tinytds adapter instead of the odbc adapter if you are connecting to Microsoft SQL Server from OS X. I know there are people who have got Sequel/ODBC running on non-Windows boxes, but I only have experience with Sequel/ODBC on Windows.

Repeated last row of query

I am using ruby-dbi to access a MS SQL database. The problem is that whenever I select more than one row from the DB, the result contains correct number of items, but all of them are the same, when they shouldn't be:
irb(main):001:0> require 'dbi'
=> true
irb(main):010:0> db=DBI.connect('dbi:ODBC:dataSource', 'userName', '****')
=> #<DBI::DatabaseHandle:0xff3df8 #handle=#<DBI::DBD::ODBC::Database:0xff3e88 #h
andle=#<ODBC::Database:0xff3f30>, #attr={}>, #trace_output=nil, #trace_mode=nil,
#convert_types=true, #driver_name="odbc">
irb(main):009:0> db.select_all('select distinct top 10 id from rawdata')
=> [[308], [308], [308], [308], [308], [308], [308], [308], [308], [308]]
The problem seems to be the as the one discussed here, but the solution proposed there (using alias) didn't work for me (or maybe I misunderstood it).
How can I fix this?
I'm using DBI 0.4.5, and Ruby 1.9.2 on Windows.
That looks kind of strange because select_all are supposed to return DBI::Row objects. Try
rows = db.select_all('select distinct top 10 id from rawdata')
rows.each do |row|
printf "ID: %d\n", row["id"]
end
I can only recommend: Go for TinyTds
https://github.com/rails-sqlserver/tiny_tds
Its
- easier to install and configure
- faster
- more stable
In the end, after realizing (at least partially) what was the post I linked in the question talking about, I modified the file row.rb from the source code of DBI:
I removed the code
if RUBY_VERSION =~ /^1\.9/
def __getobj__
#arr
end
def __setobj__(obj)
#delegate_dc_obj = #arr = obj
end
else
and the acommpanying end and I also removed the inheritance: < DelegateClass(Array).
I had the same problem on a MS-SQL database with ruby 1.9.2p180 (2011-02-18)
This is how I solved it:
def myDBIexecute(dbhash,query)
begin
# open the connection
conn = DBI.connect('DBI:ODBC:'+dbhash['datasource'].to_s,dbhash['username'].to_s,dbhash['password'].to_s)
sth = conn.prepare(query)
sth.execute()
outputme=[]
while row = sth.fetch
mrow={}
sth.column_names.each{|aname|
mrow[aname]=row[aname].to_s
}
outputme << mrow
end
sth.finish
return outputme
rescue DBI::DatabaseError => e
puts "Error code: #{e.err}"
puts "Error message: #{e.errstr}"
ensure
# disconnect from server
conn.disconnect if conn
end
end

Resources