I'm currently hosting a simple Ruby script that stores URLs and Scores and saving them to YAML. However, I'd like to save to a Postgresql database instead since the yaml file is deleted every time I restart the app. Here's the error I'm getting in Heroku:
could not connect to server: No such file or directory (PG::ConnectionBad)
Here's an example script that works locally, but throws me the above error in Heroku:
require 'pg'
conn = PG.connect( dbname: 'template1' )
res1 = conn.exec('SELECT * from pg_database where datname = $1', ['words'])
if res1.ntuples == 1 # db exists
# do nothing
else
conn.exec('CREATE DATABASE words')
words_conn = PGconn.connect( :dbname => 'words')
words_conn.exec("create table top (url varchar, score integer);")
words_conn.exec("INSERT INTO top (url, score) VALUES ('http://apple.com', 1);")
end
Thanks in advance for any help or suggestions!
Assuming you have created a Postgres database using the Heroku toolchain via heroku addons:add heroku-postgresql:dev (or the plan of your choice) you should have a DATABASE_URL environmental variable that contains your connection string. You can check that locally through heroku pg:config.
Using the pg gem (docs: http://deveiate.org/code/pg/PG/Connection.html) - and modifying the example from there to suit -
require 'pg'
# source the connection string from the DATABASE_URL environmental variable
conn = PG::Connection.new(ENV['DATABASE_URL'])
res = conn.exec_params('create table top (url varchar, score integer;")
Update: A slightly more complete example for the purposes of error handling:
conn = PG::Connection.new(ENV['TEST_DATABASE_URL'])
begin
# Ensures the table is created if it doesn't exist
res = conn.exec("CREATE TABLE IF NOT EXISTS top (url varchar, score integer);")
res.result_status
rescue PG::Error => pg_error
puts "Table creation failed: #{pg_error.message}"
end
Related
we have a database legacy from a SQL SERVER, and now the new database is a postgres, I would like to do some testing to see if the content is the same from both tables, sinnce I still don't have acess to the legacy database (SQL server), I said let play with the new postgres database by polling two table and try to compare the data, in the iteration loop I got confused, any idea will be helpful :
require 'pg'
pg_conn = PGconn.connect("localhost", 5432, '', '', "myDB", "userxx", "Zazzz")
all_children = pg_conn.exec("SELECT * from COMPANY;")
all_children2 = pg_conn.exec("SELECT * from COMPANY2;")
all_children.each do |child|
??????
end
Thanks,
The quick and dirty way would be to do a fast checking on one column, for example the name of the companies and compare the results to see the differences by substracting the resulting arrays :
require 'pg'
pg_conn = PGconn.connect("localhost", 5432, '', '', "myDB", "userxx", "Zazzz")
all_children = pg_conn.exec("SELECT name from COMPANY;")
all_children2 = pg_conn.exec("SELECT name from COMPANY2;")
# first check the count, if it's the same it's probably a good sign
puts all_children.length
puts all_children2.length
# check differences
results = all_children.uniq - all_children2.uniq
results2 = all_children2.uniq - all_children.uniq
For more complex tests you can maybe use the scientist gem from github : https://github.com/github/scientist
Edit:
sql version ?
select * from company left outer join company2 on company.name = company.name2 where company2.name is null;
I have a webserver that uses Sinatra and the Sequel gem. I would like to know if it is possible to print every query executed into the console.
I found in the Sequel documentation that I can setup a log file path.
You can also specify optional parameters, such as the connection pool size, or loggers for logging SQL queries:
DB = Sequel.connect("postgres://user:password#host:port/database_name",
:max_connections => 10, :logger => Logger.new('log/db.log'))
However I was unable to find anything about printing the queries into the console rather than a log.
You can, and you can log to multiple loggers too, see example below
db_location_test = "/db/reservation.accdb"
log_file_path = "#{__FILE__}_#{Time.now.strftime("%Y%m%d")}.txt"
log_file = File.open(log_file_path, "a")
$filelog = Logger.new log_file
$console = Logger.new STDOUT
$console.info "connecting to access database" #only logged to console
sConnectionStringAccess = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=#{db_location_test}"
#sql will be logged to both file and console
DBA = Sequel.ado(:conn_string=>sConnectionStringAccess, :loggers=>[$filelog,$console])
class Reservations < Sequel::Model(:TABLE_RESERVATIONS);end
Reservations.all.each do |record|
$console.info Hash[record]
end
How do I create prepared statements for insert and select queries in MySQL? I am using the MySQL2 gem, and my connection object looks like this:
con = Mysql2::Client.new(:host => "#{ENV['DB_HOST']}", :port => '3306', :username => "#{ENV['DB_UNAME']}", :password => "#{ENV['DB_PWD']}", :database => 'dbname')
Unfortunately, mysql2 gem does not have prepared statement support yet. The contributors are planning to add such a feature in a near future, as we can see by this Pull Request discussion:
https://github.com/brianmario/mysql2/pull/289
If you must have prepared statements in your application, I would recommend you to read about Sequel, which has a very nice support for prepared statements and bound variables:
https://github.com/jeremyevans/sequel
http://sequel.jeremyevans.net/rdoc/files/doc/prepared_statements_rdoc.html
UPDATE
As mentioned by #lulalala starting on version 0.4.0 MySQL2 gem supports prepared statements:
statement = #client.prepare("SELECT * FROM users WHERE login_count = ?")
result1 = statement.execute(1) # Binds the value 1 to the placeholder
result2 = statement.execute(2) # Binds the value 2 to the placeholder
statement = #client.prepare("SELECT * FROM users WHERE last_login >= ? AND location LIKE ?")
result = statement.execute(1, "CA") # Binds 1 and 'CA' to the placeholders, respectively
I hope that helps.
You may also use mysql2-cs-bind gem, a thin wrapper of mysql2 which adds prepared statement feature:
https://github.com/tagomoris/mysql2-cs-bind
It provides xquery method that accepts arguments for the statement:
require 'mysql2-cs-bind'
client = Mysql2::Client.new(...)
client.xquery('SELECT x,y,z FROM tbl WHERE x=? AND y=?', val1, val2) #=> Mysql2::Result
I have an importer which takes a list of emails and saves them into a postgres database. Here is a snippet of code within a tableless importer class:
query_temporary_table = "CREATE TEMPORARY TABLE subscriber_imports (email CHARACTER VARYING(255)) ON COMMIT DROP;"
query_copy = "COPY subscriber_imports(email) FROM STDIN WITH CSV;"
query_delete = "DELETE FROM subscriber_imports WHERE email IN (SELECT email FROM subscribers WHERE suppressed_at IS NOT NULL OR list_id = #{list.id}) RETURNING email;"
query_insert = "INSERT INTO subscribers(email, list_id, created_at, updated_at) SELECT email, #{list.id}, NOW(), NOW() FROM subscriber_imports RETURNING id;"
conn = ActiveRecord::Base.connection_pool.checkout
conn.transaction do
raw = conn.raw_connection
raw.exec(query_temporary_table)
raw.exec(query_copy)
CSV.read(csv.path, headers: true).each do |row|
raw.put_copy_data row['email']+"\n" unless row.nil?
end
raw.put_copy_end
while res = raw.get_result do; end # very important to do this after a copy
result_delete = raw.exec(query_delete)
result_insert = raw.exec(query_insert)
ActiveRecord::Base.connection_pool.checkin(conn)
{
deleted: result_delete.count,
inserted: result_insert.count,
updated: 0
}
end
The issue I am having is that when I try to upload I get an exception:
PG::ERROR: another command is already in progress: ROLLBACK
This is all done in one action, the only other queries I am making are user validation and I have a DB mutex preventing overlapping imports. This query worked fine up until my latest push which included updating my pg gem to 0.14.1 from 0.13.2 (along with other "unrelated" code).
The error initially started on our staging server, but I was then able to reproduce it locally and am out of ideas.
If I need to be more clear with my question, let me know.
Thanks
Found my own answer, and this might be useful if anyone finds the same issue when importing loads of data using "COPY"
An exception is being thrown within the CSV.read() block, and I do catch it, but I was not ending the process correctly.
begin
CSV.read(csv.path, headers: true).each do |row|
raw.put_copy_data row['email']+"\n" unless row.nil?
end
ensure
raw.put_copy_end
while res = raw.get_result do; end # very important to do this after a copy
end
This block ensures that the COPY command is completed. I also added this at the end to release the connection back into the pool, without disrupting the flow in the case of a successful import:
rescue
ActiveRecord::Base.connection_pool.checkin(conn)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am trying to write a ruby script that interacts with a PostgreSQL database. I am trying to piece together how to do this from the documentation, but a nice tutorial or sample code would work wonders to decrease the amount of time to get this working. If anyone has a link, some tips or has some code they could share I would be most grateful.
Edit, made this note more clear:
Note: this isn't to do with rails ActiveRecord, I am writing a Ruby script that will be involved in a program that is completely independent from Rails.
Please be more specific about what postgresql library you're using.
I'm going to assume the 'pg' gem, apart from ActiveRecord.
The project source has an html file that might be helpful.
Go to https://bitbucket.org/ged/ruby-pg/src/b477174160c8/doc/postgres.html
Then click "raw" at the upper right side of the html. Open the file in your web browser.
This sample code helps you connect (copied from the html file):
require "postgres"
conn = PGconn.connect("localhost", 5432, "", "", "test1")
# or: conn = PGconn.open('dbname=test1')
res = conn.exec("select * from a;")
The res object is a PGResult. Scroll down to that section in the html to see what methods you can call.
This link has a PGResult example:
http://rubydoc.info/gems/pg/0.10.0/PGresult
Excerpt:
require 'pg'
conn = PGconn.open(:dbname => 'test')
res = conn.exec('SELECT 1 AS a, 2 AS b, NULL AS c')
res.getvalue(0,0) # '1'
res[0]['b'] # '2'
res[0]['c'] # nil
I confirm, "postgres" package is outdated, you need "pg".
It tooks me lot of time just to get a basic select * from films working with ruby and postgres. As I am kind, here is my code:
postgres preparation (database=megatest user=roger pass=123456 table=films)
$ su postgres
psql
CREATE USER roger WITH PASSWORD '123456';
GRANT ALL PRIVILEGES ON DATABASE megatest to roger;
megatest=# GRANT SELECT ON films TO PUBLIC;
PG package preparation
sudo gem install pg
Ruby Code
require 'pg'
conn=PGconn.connect( :hostaddr=>"127.0.0.1", :port=>5432, :dbname=>"megatest", :user=>"roger", :password=>'123456')
# or for a non IP address :host => 'my.host.name.com' instead of hostaddr
# run the query
res = conn.exec("SELECT * FROM films")
# Ran only once in order to get field Name
fieldArray=res.fields()
fieldArray.each do |elem|
print "elem="+elem+"\n"
end
# print data from the query
res.each{ |row|
puts "Code="+row["code"] +" title="+row["title"] +" did="+row["did"] +" date_prod="+row["date_prod"] +" kind="+row["kind"] +" len="+row["len"]
}
Results
root#eblain-VirtualBox:/home/eblain/ruby# ruby postgresTest.rb
Code=UA502 title=Bananas did=105 date_prod=1971-07-13 kind=Comedy len=01:22:00
Code=UA503 title=Cowboy did=105 date_prod=1979-07-13 kind=Horror len=01:32:00
Code=UA544 title=YoBro did=105 date_prod=1981-07-13 kind=Action len=01:42:00
You only need to require the pg gem and establish the connection to the DB:
require 'pg'
# require 'active_record' # uncomment for not Rails environment
ActiveRecord::Base.establish_connection(:adapter => "postgresql",
:username => "username",
:password => "password",
:database => "database")
When you define models to inherit from ActiveRecord::Base they will use this database connection. Everything else should work like it does in Rails.
For parametrized SQL statements, you should use PGconn#exec_params, e.g.
conn = PGconn.new(:dbname => 'test')
conn.exec_params(
'INSERT INTO comedians (first_name, last_name) VALUES ($1, $2)',
['Louis', 'CK'])
conn.close
Source: http://deveiate.org/code/pg/PGconn.html
Look here for a complete list of parameters that can be passed to the PGconn constructor.