we have a database legacy from a SQL SERVER, and now the new database is a postgres, I would like to do some testing to see if the content is the same from both tables, sinnce I still don't have acess to the legacy database (SQL server), I said let play with the new postgres database by polling two table and try to compare the data, in the iteration loop I got confused, any idea will be helpful :
require 'pg'
pg_conn = PGconn.connect("localhost", 5432, '', '', "myDB", "userxx", "Zazzz")
all_children = pg_conn.exec("SELECT * from COMPANY;")
all_children2 = pg_conn.exec("SELECT * from COMPANY2;")
all_children.each do |child|
??????
end
Thanks,
The quick and dirty way would be to do a fast checking on one column, for example the name of the companies and compare the results to see the differences by substracting the resulting arrays :
require 'pg'
pg_conn = PGconn.connect("localhost", 5432, '', '', "myDB", "userxx", "Zazzz")
all_children = pg_conn.exec("SELECT name from COMPANY;")
all_children2 = pg_conn.exec("SELECT name from COMPANY2;")
# first check the count, if it's the same it's probably a good sign
puts all_children.length
puts all_children2.length
# check differences
results = all_children.uniq - all_children2.uniq
results2 = all_children2.uniq - all_children.uniq
For more complex tests you can maybe use the scientist gem from github : https://github.com/github/scientist
Edit:
sql version ?
select * from company left outer join company2 on company.name = company.name2 where company2.name is null;
Related
I'm trying to make a softlayer API call using Ruby to see upcoming maintenance and the machines that may be effected by the maintenance. I have a few questions but I'm running into an issue seeing many of the relational properties documented here:
http://sldn.softlayer.com/reference/datatypes/SoftLayer_Notification_Occurrence_Event
Here is my simple program:
require 'rubygems'
require 'softlayer_api'
require 'pp'
client = SoftLayer::Client.new(:username => user, :api_key => api_key, :timeout => 99999999)
account = client['Account'].object_mask("mask[pendingEventCount]").getObject()
pending_event_count = account["pendingEventCount"]
current_time = Time.now.to_i
for i in 0..(pending_event_count/30.0).ceil - 1
list_of_pending_events = client['Account'].result_limit(i*30,30).object_mask("mask[id, startDate, endDate, recoveryTime, subject, summary]").getPendingEvents
for x in 0..list_of_pending_events.length - 1
start_time = DateTime.parse(list_of_pending_events[x]['startDate']).to_time.to_i
if start_time > current_time
pp list_of_pending_events[x]
end
end
end
The above works, but if I try to add a relational property to the mask, such as "impactedResources" it will fail saying that property does not belong to SoftLayer_Notification_Occurrence_Event. Can someone help explain why this, and many other, relational properties are not valid in the above call?
Also, two quick other questions on this topic:
1) Why do some of the results in getPendingEvents have start AND end times in the past? And why do some have a missing end time altogether? Notice I'm checking if the start time is greater than the current time as there seems to be old maintenance data in the results.
2) Am I taking the right approach for getting upcoming maintenance and figuring out machines that will be impacted? (Using getPendingEvents and then the 'impactedResources' property)
I used your code and I added the "impactedResources" to the object mask and it worked fine. You may be getting the issue, because your API user does not have enough permissions, I recomend you try with the master user.
regarding to your other questions:
1.- When I ran your code I got only 3 events whose startDate is greater than the current time, so it worked fine for me. If you still are getting that issue you can try using objectFilters.
require 'rubygems'
require 'softlayer_api'
require 'pp'
client = SoftLayer::Client.new(:username => user, :api_key => apikey, :timeout => 99999999)
object_filter = SoftLayer::ObjectFilter.new
object_filter.set_criteria_for_key_path('pendingEvents.startDate', 'operation' => 'greaterThanDate',
'options' => [{
'name' => 'date',
'value' => ['9/8/2016']
}])
list_of_pending_events = client['Account'].object_filter(object_filter).object_mask("mask[id, startDate, endDate, recoveryTime, subject, summary]").getPendingEvents
pp list_of_pending_events
2.- Yes, that approach wil work.
Regards
When performing a COPY command, a few informations are printed, like :
INFO: Load into table '<table>' completed, 22666 record(s) loaded successfully.
INFO: Load into table '<table>' completed, 1 record(s) could not be loaded. Check 'stl_load_errors' system table for details.
And I need to identify failing records.
Thus I need 2 things :
Determine when there are failing rows: now, it's only printed on screen and I don't know how to get the message in code.
Determine the failing rows.
One way to do that would be to access to the query identifier that is visible in the table stl_load_errors, but I have no clue how to access it by code.
(I currently use the pg gem to connect to redshift)
stl_load_errors is a table in Redshift that (as you may have guessed already) includes all the errors that happen when loading into Redshift. So you can query it by doing something like:
SELECT * FROM stl_load_errors
Now, to answer your questions use the following snippet:
database = PG.connect(redshift)
begin
query = "COPY %s (%s) FROM 's3://%s/%s' CREDENTIALS 'aws_access_key_id=%s;aws_secret_access_key=%s' CSV GZIP" %
[ table, columns, s3_bucket, s3_key, access_key_id, secret_access_key ]
database.exec(query)
puts 'File succesfully imported'
rescue PG::InternalError
res = database.exec("SELECT line_number, colname, err_reason FROM pg_catalog.stl_load_errors WHERE filename = 's3://#{s3_bucket}/#{s3_key}'")
res.each do |row|
puts "Importing failed:\n> Line %s\n> Column: %s\n> Reason: %s" % row.values_at('line_number', 'colname', 'err_reason')
end
end
That should output all the information you need, recall variables like redshift, table, columns, s3_bucket, s3_key, access_key_id, and secret_access_key depend on your configuration.
UPDATE:
To answer your comment below, more specifically, you could use a query like this:
"SELECT lines_scanned FROM pg_catalog.stl_load_commits WHERE filename = 's3://#{s3_bucket}/#{s3_key}' AND errors = -1"
I'm currently hosting a simple Ruby script that stores URLs and Scores and saving them to YAML. However, I'd like to save to a Postgresql database instead since the yaml file is deleted every time I restart the app. Here's the error I'm getting in Heroku:
could not connect to server: No such file or directory (PG::ConnectionBad)
Here's an example script that works locally, but throws me the above error in Heroku:
require 'pg'
conn = PG.connect( dbname: 'template1' )
res1 = conn.exec('SELECT * from pg_database where datname = $1', ['words'])
if res1.ntuples == 1 # db exists
# do nothing
else
conn.exec('CREATE DATABASE words')
words_conn = PGconn.connect( :dbname => 'words')
words_conn.exec("create table top (url varchar, score integer);")
words_conn.exec("INSERT INTO top (url, score) VALUES ('http://apple.com', 1);")
end
Thanks in advance for any help or suggestions!
Assuming you have created a Postgres database using the Heroku toolchain via heroku addons:add heroku-postgresql:dev (or the plan of your choice) you should have a DATABASE_URL environmental variable that contains your connection string. You can check that locally through heroku pg:config.
Using the pg gem (docs: http://deveiate.org/code/pg/PG/Connection.html) - and modifying the example from there to suit -
require 'pg'
# source the connection string from the DATABASE_URL environmental variable
conn = PG::Connection.new(ENV['DATABASE_URL'])
res = conn.exec_params('create table top (url varchar, score integer;")
Update: A slightly more complete example for the purposes of error handling:
conn = PG::Connection.new(ENV['TEST_DATABASE_URL'])
begin
# Ensures the table is created if it doesn't exist
res = conn.exec("CREATE TABLE IF NOT EXISTS top (url varchar, score integer);")
res.result_status
rescue PG::Error => pg_error
puts "Table creation failed: #{pg_error.message}"
end
How do I create prepared statements for insert and select queries in MySQL? I am using the MySQL2 gem, and my connection object looks like this:
con = Mysql2::Client.new(:host => "#{ENV['DB_HOST']}", :port => '3306', :username => "#{ENV['DB_UNAME']}", :password => "#{ENV['DB_PWD']}", :database => 'dbname')
Unfortunately, mysql2 gem does not have prepared statement support yet. The contributors are planning to add such a feature in a near future, as we can see by this Pull Request discussion:
https://github.com/brianmario/mysql2/pull/289
If you must have prepared statements in your application, I would recommend you to read about Sequel, which has a very nice support for prepared statements and bound variables:
https://github.com/jeremyevans/sequel
http://sequel.jeremyevans.net/rdoc/files/doc/prepared_statements_rdoc.html
UPDATE
As mentioned by #lulalala starting on version 0.4.0 MySQL2 gem supports prepared statements:
statement = #client.prepare("SELECT * FROM users WHERE login_count = ?")
result1 = statement.execute(1) # Binds the value 1 to the placeholder
result2 = statement.execute(2) # Binds the value 2 to the placeholder
statement = #client.prepare("SELECT * FROM users WHERE last_login >= ? AND location LIKE ?")
result = statement.execute(1, "CA") # Binds 1 and 'CA' to the placeholders, respectively
I hope that helps.
You may also use mysql2-cs-bind gem, a thin wrapper of mysql2 which adds prepared statement feature:
https://github.com/tagomoris/mysql2-cs-bind
It provides xquery method that accepts arguments for the statement:
require 'mysql2-cs-bind'
client = Mysql2::Client.new(...)
client.xquery('SELECT x,y,z FROM tbl WHERE x=? AND y=?', val1, val2) #=> Mysql2::Result
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am trying to write a ruby script that interacts with a PostgreSQL database. I am trying to piece together how to do this from the documentation, but a nice tutorial or sample code would work wonders to decrease the amount of time to get this working. If anyone has a link, some tips or has some code they could share I would be most grateful.
Edit, made this note more clear:
Note: this isn't to do with rails ActiveRecord, I am writing a Ruby script that will be involved in a program that is completely independent from Rails.
Please be more specific about what postgresql library you're using.
I'm going to assume the 'pg' gem, apart from ActiveRecord.
The project source has an html file that might be helpful.
Go to https://bitbucket.org/ged/ruby-pg/src/b477174160c8/doc/postgres.html
Then click "raw" at the upper right side of the html. Open the file in your web browser.
This sample code helps you connect (copied from the html file):
require "postgres"
conn = PGconn.connect("localhost", 5432, "", "", "test1")
# or: conn = PGconn.open('dbname=test1')
res = conn.exec("select * from a;")
The res object is a PGResult. Scroll down to that section in the html to see what methods you can call.
This link has a PGResult example:
http://rubydoc.info/gems/pg/0.10.0/PGresult
Excerpt:
require 'pg'
conn = PGconn.open(:dbname => 'test')
res = conn.exec('SELECT 1 AS a, 2 AS b, NULL AS c')
res.getvalue(0,0) # '1'
res[0]['b'] # '2'
res[0]['c'] # nil
I confirm, "postgres" package is outdated, you need "pg".
It tooks me lot of time just to get a basic select * from films working with ruby and postgres. As I am kind, here is my code:
postgres preparation (database=megatest user=roger pass=123456 table=films)
$ su postgres
psql
CREATE USER roger WITH PASSWORD '123456';
GRANT ALL PRIVILEGES ON DATABASE megatest to roger;
megatest=# GRANT SELECT ON films TO PUBLIC;
PG package preparation
sudo gem install pg
Ruby Code
require 'pg'
conn=PGconn.connect( :hostaddr=>"127.0.0.1", :port=>5432, :dbname=>"megatest", :user=>"roger", :password=>'123456')
# or for a non IP address :host => 'my.host.name.com' instead of hostaddr
# run the query
res = conn.exec("SELECT * FROM films")
# Ran only once in order to get field Name
fieldArray=res.fields()
fieldArray.each do |elem|
print "elem="+elem+"\n"
end
# print data from the query
res.each{ |row|
puts "Code="+row["code"] +" title="+row["title"] +" did="+row["did"] +" date_prod="+row["date_prod"] +" kind="+row["kind"] +" len="+row["len"]
}
Results
root#eblain-VirtualBox:/home/eblain/ruby# ruby postgresTest.rb
Code=UA502 title=Bananas did=105 date_prod=1971-07-13 kind=Comedy len=01:22:00
Code=UA503 title=Cowboy did=105 date_prod=1979-07-13 kind=Horror len=01:32:00
Code=UA544 title=YoBro did=105 date_prod=1981-07-13 kind=Action len=01:42:00
You only need to require the pg gem and establish the connection to the DB:
require 'pg'
# require 'active_record' # uncomment for not Rails environment
ActiveRecord::Base.establish_connection(:adapter => "postgresql",
:username => "username",
:password => "password",
:database => "database")
When you define models to inherit from ActiveRecord::Base they will use this database connection. Everything else should work like it does in Rails.
For parametrized SQL statements, you should use PGconn#exec_params, e.g.
conn = PGconn.new(:dbname => 'test')
conn.exec_params(
'INSERT INTO comedians (first_name, last_name) VALUES ($1, $2)',
['Louis', 'CK'])
conn.close
Source: http://deveiate.org/code/pg/PGconn.html
Look here for a complete list of parameters that can be passed to the PGconn constructor.