Sequel Model is not mapping all columns by default.
I'm using:
Sequel version 4.24.0
Ruby version 1.9.3p194
Windows Server 2008 Enterprise
SQLServer 2008 R2 Enterprise
Below is a brief script that shows the problem:
require "sequel"
DB = Sequel.ado(:conn_string=>CONNECTION_STRING)
print "Loading LnProfile... "
class LnProfile < Sequel::Model(:lnprofile)
# attr_accessor :crawl_start_time
end
puts "Done!"
id = "808d8421-6c05-471e-addc-6165de044ad3"
# crawl_start_time field shouldn't be empty
p1 = LnProfile.where(:id=>id).first
puts ".id:#{p1.id.to_s}:." # => .id:808d8421-6c05-471e-addc-6165de044ad3:.
puts ".crawl_start_time:#{p1.crawl_start_time.to_s}:." # => .crawl_start_time::.
# same record, different data
p2 = LnProfile.select_all.select_append(:crawl_start_time).where(:id=>id).first
puts ".id:#{p2.id.to_s}:." # => .id:808d8421-6c05-471e-addc-6165de044ad3:.
puts ".crawl_start_time:#{p2.crawl_start_time.to_s}:." # => .crawl_start_time:2016-01-10 12:02:29 -0300:.
Below is the output of the script above:
Loading LnProfile... Done!
.id:{808D8421-6C05-471E-ADDC-6165DE044AD3}:.
.crawl_start_time::.
.id:{808D8421-6C05-471E-ADDC-6165DE044AD3}:.
.crawl_start_time:2016-01-10 12:02:29 -0300:.
Please advice. This issue is dirtying the database when the code loads an object and saves it again few lines above.
Thanks in advance.
Try using the tinytds adapter, it is much more robust than the ado adapter.
Related
I am in the process of upgrading a rails 4.2 application to rails 5. One of the models uses an ActiveRecord::Store. When retrieving records after the upgrade, said store only returns an empty Hash. I have no idea why that's happening and wasn't able to find anything in the changelog. So any help and explanation would be appreciated.
Here's the output of the rails 4 console:
=> #<StoredDataQuery ..., config: {"start_date"=>6, "end_date"=>0, ...>
and rails 5:
=> #<StoredDataQuery ..., config: {}, ...
psql output:
development=# SELECT config FROM stored_data_queries WHERE id=1;
config
---------------------------------------------
--- !ruby/hash:ActionController::Parameters+
start_date: 6 +
end_date: 0 +
interval: day +
(1 row)
Looking at the SQL output, I'm suspecting it has something to do with the data being serialized as ActionController::Parameters.
Thanks for your help!
Here's how to fix it in sql (postgres):
UPDATE stored_data_queries SET config = replace(config, 'ActionController::Parameters', 'ActiveSupport::HashWithIndifferentAccess');
Same here, after Upgrading Zammad from Rails 4.2 to Rails 5.0. After some research I found out that active record is only reading ActiveSupport::HashWithIndifferentAccess from store anymore (other classes are skipped).
So for migration you can overwrite ActiveRecord::Store::IndifferentCoder.as_indifferent_hash in a migration, read all related records from database and just save them back (then all ActionController::Parameters are converted to ActiveSupport::HashWithIndifferentAccess).
For me the following migration (under Rails 5.0) has worked to convert all ActionController::Parameters to ActiveSupport::HashWithIndifferentAccess:
require 'active_record/store'
module ActiveRecord
module Store
class IndifferentCoder
def self.as_indifferent_hash(obj)
case obj
# re-enable using ActionController::Parameters in stores,
# convert them to ActiveSupport::HashWithIndifferentAccess
# at reading from db
when ActionController::Parameters
obj.permit!.to_h
# /re-enable
when ActiveSupport::HashWithIndifferentAccess
obj
when Hash
obj.with_indifferent_access
else
ActiveSupport::HashWithIndifferentAccess.new
end
end
end
end
end
class FixedStoreUpgrade45 < ActiveRecord::Migration[5.0]
def up
[Macro, Taskbar, Calendar, Trigger, Channel, Job, PostmasterFilter, Report::Profile, Setting, Sla, Template].each do |class_name|
class_name.all.each do |record|
begin
record.save!
rescue => e
Rails.logger.error "Unable to save/update #{class_name}.find(#{record.id}): #{e.message}"
end
end
end
end
end
I'm having trouble getting an object_mask applied to the data I'm retrieving. Here is a snippet of what I'm doing:
client = SoftLayer::Client.new(<...hidden...>)
<BREAK>
if (item["hostName"])
machines = SoftLayer::BareMetalServer.find_servers({ :client => client, :hostname => item["hostName"], :object_mask => "[id,hostname,tagReferences]"})
machines.each do |machine|
pp machine
end
end
When I print "machine" it is still printing all the fields. Thank you in advance for any help.
$ ruby -v
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
I was not able to get specific items using masks with “BareMetalServer.find_servers”, but below is another ruby example that may help you:
require 'rubygems'
require 'softlayer_api'
# Your SoftLayer API username.
SL_API_USERNAME = 'set me'
# Your SoftLayer API key.
SL_API_KEY = 'set me'
softlayer_client = SoftLayer::Client.new(:username => SL_API_USERNAME,
:api_key => SL_API_KEY)
account_service = softlayer_client.service_named('SoftLayer_Account')
# Create an object mask to get more information than by default
mask = 'mask[id,hostname]'
begin
result = account_service.object_mask(mask).getHardware
puts 'Process finished successfully'
p result
rescue Exception => e
raise e
end
References:
http://sldn.softlayer.com/reference/services/SoftLayer_Account/getHardware
https://softlayer.github.io/ruby/token_auth/
https://softlayer.github.io/ruby/find_my_vms/
https://softlayer.github.io/ruby/
https://www.omniref.com/ruby/gems/softlayer_api/2.1.0
https://github.com/softlayer/softlayer-ruby/blob/master/lib/softlayer/BareMetalServer.rb
Currently the ojectMask for the method find_servers does not limit the fields, it adds the fields of your object mask to the result.
If you need to limit the fields, you could use the use "map" to create an array with just the fields you are interested in.
I am building a module in Ruby to read metadata from source tables in various databases.
I wrote a small program to test with PostgreSQL:
#!/usr/bin/ruby
require 'pg'
begin
puts "start"
puts 'Version of libpg: ' + PG.library_version.to_s
con = PG.connect(host: 'localhost', dbname: 'rdv_app_dev', user: 'rdv_app', password: 'rdv_app')
puts con.server_version
pst = con.exec "SELECT * FROM users"
pst.each do |row|
puts "%s %s " % [ row['id'], row['email'] ]
end
puts 'There are %d columns ' % pst.nfields
puts 'The column names are:'
pst.fields.each do |f|
puts pst.fnumber(f).to_s + ' ' + f + ' ' + pst.ftype(pst.fnumber(f)).to_s
end
rescue PG::Error => e
puts e.message
ensure
pst.clear if pst
con.close if con
puts "stop"
end
It works fine, but it uses functions that are specific to Postgres. I need to have it working for any database without re-coding it for each one of them.
I read about Ruby-DBI, but it looks to be out of date, since it did not evolve for 7 years.
Is there a generic solution for accessing a database with Ruby ?
ActiveRecord is by far the most popular (see RubyGems stats). DataMapper is very similar, but more lightweight and makes changing the database quicker. I'm not familiar with sequel.
These gems introduce their own syntax for communicating with the database that is intended to abstact away the database-specific implementation details. I.e. a query like User.where(verified: true).includes(:posts).order(created_at: :desc) will list users ordered by most-recent creation date and include their' posts (performing a join behind the scenes). This Ruby syntax will compile to db-specific code based on what adapter and configuration you've specified.
Look into Sinatra and DataMapper; there are many tutorials.
Also look into Rails and how to configure Rails to use MySQL or Postgres instead of (its default) SQLite. You will find that the ORM (ActiveRecord) code doesn't change regardless of which you use.
My simple Sqlite3 database is as follows:
CREATE TABLE balances(
balance zilch
);
My Ruby is as follows:
require('active_record')
ActiveRecord::Base.establish_connection(:database => "testbalance.db", :adapter => "sqlite3")
class Balance < ActiveRecord::Base
end
x = Balance.new
x.balance = 50
x.save
When I exit, and come back, and enter in the same Ruby again, at first, (before I runx.balance = 50) balance is nil. Why is this? Why isn't my DB saving?
If you enter the same code, then you're creating a new object again. No wonder its balance is nil.
To check that your object is saved, you can (for example) check Balance.count before and after record creation.
This is an old demo way of using Active Record and not very useful for production. It will get you started though. My code will make connections without sqlite3 gem required. I think that Active Record will include it if you use the :adapter hash entry. Of course you need it installed but it's not really needed in your code for Active Record. Just try it without that require to see. Then if you're still in doubt, un-install the gem just for fun. There are more Active Record namespaces and methods you should try especially ones that check to see if the database already exists. Then by pass the creation of one.
Here's some sample code from the book Metaprogramming Ruby.
#---
# Excerpted from "Metaprogramming Ruby",
# published by The Pragmatic Bookshelf.
# Copyrights apply to this code. It may not be used to create training material,
# courses, books, articles, and the like. Contact us if you are in doubt.
# We make no guarantees that this code is fit for any purpose.
# Visit http://www.pragmaticprogrammer.com/titles/ppmetr2 for more book information.
#---
# Create a new database each time
File.delete 'dbfile' if File.exist? 'dbfile'
require 'active_record'
ActiveRecord::Base.establish_connection :adapter => "sqlite3",
:database => "dbfile.sqlite3"
# Initialize the database schema
ActiveRecord::Base.connection.create_table :ducks do |t|
t.string :name
end
class Duck < ActiveRecord::Base
validate do
errors.add(:base, "Illegal duck name.") unless name[0] == 'D'
end
end
my_duck = Duck.new
my_duck.name = "Donald"
my_duck.valid? # => true
my_duck.save!
require_relative '../test/assertions'
assert my_duck.valid?
bad_duck = Duck.new(:name => "Ronald")
assert !bad_duck.valid?
duck_from_database = Duck.first
duck_from_database.name # => "Donald"
assert_equals "Donald", duck_from_database.name
duck_from_database.delete
File.delete 'dbfile' if File.exist? 'dbfile'
This code deletes the db file after usage and that's not very good persistence either. But you get the idea as it's just for testing assertions. You could try that to be sure as you change balances.
Do you want the rest of the code? https://pragprog.com/book/ppmetr/metaprogramming-ruby
Am I training you or am I the like? Moderators delete this if I'm wrong here please. I don't want to set a bad example.
I'm having issues getting Sequel to connect to a MS SQL Database.
I have installed the following:
UnixODBC
FreeTDS
I have configured both software packages, and the following commands allow me to connect to my database without a problem:
isql
tsql
osql
However, when I try it from Ruby code using the Sequel.odbc command, I receive the following error:
ODBC::Error: IM003 (0) [iODBC][Driver Manager]Specified driver could not be loaded.
This is as far as I can get. I used to receive another error first, but managed to solve that by redoing the configuration part. Guess I missed something there.
EDIT
This is the code for my base talker class. It basically loads a YAML file like rails does, holding the database settings and establishes a connection to the database.
This seems to be working, trying it manually returns me a DB object from sequel:
module Talkers
require 'yaml'
require 'sequel'
class BaseTalker
# This function will load the desired settings as a hash from the database.yml file
# in the config folder. The data will be returned as a hash following the standard
# YAML structure.
def self.load_config(name)
cfg = YAML::load(File.open(File.join(ENV['API_ROOT'], 'config', 'database.yml')))
cfg.key?(name) ? cfg[name] : nil
end
# This function will establish a connection with the Florensia User database and return
# the Sequel database object, so that queries can be executed against the database.
def self.connect_to_user_db
settings = self.load_config("florensia_user_#{ENV['RACK_ENV']}")
Sequel.odbc settings['dsn'], :db_type => settings['adapter'], :user => settings['user'], :password => settings['password']
end
end
end
The class below inherits from the talker and performs certain actions for a User. It contains the DB logic specific to the game. When I call this logic, I receive the errors:
module Talkers
require 'yaml'
require 'sequel'
class LoginDbTalker < BaseTalker
#
# Bans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly set:
# - id : The identifier of the account.
# - gm_name : The name of the GM setting the ban.
# - punish_code : The punishment code being applied on the account.
# - days : The duration of the ban in days, starting from today.
#
# The function will return true if the ban has been properly set; otherwise the function
# will return false.
def self.ban_user(options = {})
return false if options.empty?
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :apply_ban)
ps.call(
:punishcode => options[:punish_code],
:punishstory => "Banned by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => (options[:days].to_i == -1) ? (Date.today + (30 * 265)) : (Date.today + options[:days].to_i))
true
rescue Exception => e
puts "Exception caught in ban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}, punish_code=#{options[:punish_code]}, days=#{options[:days]}"
false
end
#
# Unbans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly lifted:
# - id : The identifier of the account.
# - gm_name : The name of the GM removing the ban.
#
# The function will return true if the ban has been properly lifted; otherwise the function
# will return false.
def self.unban_user(options = {})
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :lift_ban)
ps.call(
:punishcode => '0',
:punishstory => "Ban removed by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => Date.today
)
true
rescue Exception => e
puts "Exception caught in unban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}"
false
end
#
# Kicks the specified User from the game, regardless on the server or character he currently is on.
# This requires a direct connection to the game servers so a specialized command can be sent that
# instruct the server to close the connection with the offending player.
# The function returns true if the kick succeeded; otherwise false.
def self.kick_player(id)
false
end
end
end
Calling any of the ban/unban functions results in the error message.
EDIT2
I've added the folder /Library/ODBC and linked all config files to there for iODBC. This removes the error I had before and now brings me this error:
ODBC::Error: 01000 (20002) [FreeTDS][SQL Server]Adaptive Server connection failed
So it seems I made some progress again
I recommend you use the tinytds adapter instead of the odbc adapter if you are connecting to Microsoft SQL Server from OS X. I know there are people who have got Sequel/ODBC running on non-Windows boxes, but I only have experience with Sequel/ODBC on Windows.