Repeated last row of query - ruby

I am using ruby-dbi to access a MS SQL database. The problem is that whenever I select more than one row from the DB, the result contains correct number of items, but all of them are the same, when they shouldn't be:
irb(main):001:0> require 'dbi'
=> true
irb(main):010:0> db=DBI.connect('dbi:ODBC:dataSource', 'userName', '****')
=> #<DBI::DatabaseHandle:0xff3df8 #handle=#<DBI::DBD::ODBC::Database:0xff3e88 #h
andle=#<ODBC::Database:0xff3f30>, #attr={}>, #trace_output=nil, #trace_mode=nil,
#convert_types=true, #driver_name="odbc">
irb(main):009:0> db.select_all('select distinct top 10 id from rawdata')
=> [[308], [308], [308], [308], [308], [308], [308], [308], [308], [308]]
The problem seems to be the as the one discussed here, but the solution proposed there (using alias) didn't work for me (or maybe I misunderstood it).
How can I fix this?
I'm using DBI 0.4.5, and Ruby 1.9.2 on Windows.

That looks kind of strange because select_all are supposed to return DBI::Row objects. Try
rows = db.select_all('select distinct top 10 id from rawdata')
rows.each do |row|
printf "ID: %d\n", row["id"]
end

I can only recommend: Go for TinyTds
https://github.com/rails-sqlserver/tiny_tds
Its
- easier to install and configure
- faster
- more stable

In the end, after realizing (at least partially) what was the post I linked in the question talking about, I modified the file row.rb from the source code of DBI:
I removed the code
if RUBY_VERSION =~ /^1\.9/
def __getobj__
#arr
end
def __setobj__(obj)
#delegate_dc_obj = #arr = obj
end
else
and the acommpanying end and I also removed the inheritance: < DelegateClass(Array).

I had the same problem on a MS-SQL database with ruby 1.9.2p180 (2011-02-18)
This is how I solved it:
def myDBIexecute(dbhash,query)
begin
# open the connection
conn = DBI.connect('DBI:ODBC:'+dbhash['datasource'].to_s,dbhash['username'].to_s,dbhash['password'].to_s)
sth = conn.prepare(query)
sth.execute()
outputme=[]
while row = sth.fetch
mrow={}
sth.column_names.each{|aname|
mrow[aname]=row[aname].to_s
}
outputme << mrow
end
sth.finish
return outputme
rescue DBI::DatabaseError => e
puts "Error code: #{e.err}"
puts "Error message: #{e.errstr}"
ensure
# disconnect from server
conn.disconnect if conn
end
end

Related

Why am I unable to write a value into a blank CSV file? Nil error

I have a total of 7 columns with 6 columns initially filled out in a CSV file that I'm writing. When I try to populate the 7th column with data, I keep running into this error:
NoMethodError: You have a nil object when you didn't expect it!
You might have expected an instance of Array.
The error occurred while evaluating nil.<<
Why do I keep running into this error? I should be able to write values into a 'nil'/blank space on a CSV file. Below is my code:
#Finds two records in the database
accounts = Account.find(1,2)
spammer_status = []
#Makes a call into the akismet API and populates spammer_status array with
#true or false values if the person is a spammer or not.
accounts.each do |accounts|
spammer_status << client.comment_check(accounts.last_seen_ip, nil,
:comment_author => accounts.name,
:comment_author_email => accounts.email,
:comment_author_url => accounts.url,
:comment_content => accounts.about)
end
#Changes the values from booleans to strings
spammer_status.map! { |value| value.to_s }
#Populates the initial 6 columns from the database values
CSV.open("/var/local/openhub/current/script/akismet_results.csv","w") do |row|
row << ["id",
"name",
"email",
"url",
"description",
"last seen ip",
"spammer status"]
accounts.each do |accounts|
row << [accounts.id,
accounts.name,
accounts.email,
accounts.url,
accounts.about,
accounts.last_seen_ip]
end
end
#Attempts to populate the 7th column, nil error
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv", headers:true) do |row|
# binding.pry
row[6] << spammer_status.shift
end
What am I doing wrong here? The error is on the foreach part of the program. All I want to do is to iterate a row at a time and then add the string converted booleans to the correct column. Any help would be appreciated?
You are trying to << to a nil object. row[6] is nil. I believe you just want to assign a value to row[6] or if you want to use <<, just do it to row itself.
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv", headers:true) do |row|
# binding.pry
row[6] = spammer_status.shift
# or you could do row << spammer_status.shift
end
I eventually figured this issue out by refactoring my code. I have to admit that the above variation was not written very well. By extracting the spam status as a method and then creating a method call when the CSV is written, I was able to make the functionality work.
def is_spam?(account)
spammer_status = client.comment_check(account.last_seen_ip, nil,
:comment_author => account.name,
:comment_author_email => account.email,
:comment_author_url => account.url,
:comment_content => account.about)
spammer_status.to_s
end
CSV.foreach("/var/local/openhub/current/script/akismet_results.csv","a", headers: true) do |row|
row << [account.id,
account.name,
account.email,
account.url,
account.about,
account.last_seen_ip,
is_spam?(account)]
end

Preparing and executing SQLite Statements in Ruby

I have been trying to puts some executed statements after I prepare them. The purpose of this is to sanitize my data inputs, which I have never done before. I followed the steps here, but I am not getting the result I want.
Here's what I have:
require 'sqlite3'
$db = SQLite3::Database.open "congress_poll_results.db"
def rep_pull(state)
pull = $db.prepare("SELECT name, location FROM congress_members WHERE location = ?")
pull.bind_param 1, state
puts pull.execute
end
rep_pull("MN")
=> #<SQLite3::ResultSet:0x2e69e00>
What I am expecting is a list of reps in MN, but instead I just get "SQLite3::ResultSet:0x2e69e00" thing.
What am I missing here? Thanks very much.
Try this
def rep_pull(state)
pull = $db.prepare("SELECT name, location FROM congress_members WHERE location = ?")
pull.bind_param 1, state
pull.execute do |row|
p row
end
end

Structuring Nokogiri output without HTML tags

I got Ruby to travel to a web site, iterate through a list of campaigns and scrape the pages for specific data. The problem I have now is getting it from the structure Nokogiri gives me, and outputting it into a readable form.
campaign_list = Array.new
campaign_list.push(1042360, 1042386, 1042365, 992307)
browser = Watir::Browser.new :chrome
browser.goto '<redacted>'
browser.text_field(:id => 'email').set '<redacted>'
browser.text_field(:id => 'password').set '<redacted>'
browser.send_keys :enter
file = File.new('hourlysales.csv', 'w')
data = {}
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
puts "Error loading page, I recommend restarting script"
# Possibly automatic restart of script
else
hourly_data = Nokogiri::HTML.parse(browser.html).text
# file.write data
puts hourly_data
end
This is the output I get:
{"views":[[17,145],[18,165],[19,99],[20,71],[21,31],[22,26],[23,10],[0,15],[1,1], [2,18],[3,19],[4,35],[5,47],[6,44],[7,67],[8,179],[9,141],[10,112],[11,95],[12,46],[13,82],[14,79],[15,70],[16,103]],"orders":[[17,10],[18,9],[19,5],[20,1],[21,1],[22,0],[23,0],[0,1],[1,0],[2,1],[3,0],[4,1],[5,2],[6,1],[7,5],[8,11],[9,6],[10,5],[11,3],[12,1],[13,2],[14,4],[15,6],[16,7]],"conversion_rates":[0.06870229007633588,0.05442176870748299,0.050505050505050504,0.014084507042253521,0.03225806451612903,0.0,0.0,0.06666666666666667,0.0,0.05555555555555555,0.0,0.02857142857142857,0.0425531914893617,0.022727272727272728,0.07462686567164178,0.06134969325153374,0.0425531914893617,0.044642857142857144,0.031578947368421054,0.021739130434782608,0.024390243902439025,0.05063291139240506,0.08571428571428572,0.06741573033707865]}
The arrays stand for { views [[hour, # of views], [hour, # of views], etc. }. Same with orders. I don't need conversion rates.
I also need to add the values up for each key, so after doing this for 5 pages, I have one key for each hour of the day, and the total number of views for that hour. I tried a couple each loops, but couldn't make any progress.
I appreciate any help you guys can give me.
It looks like the output (which from your code I assume is the content of hourly_data) is JSON. In that case, it's easy to parse and add up the numbers. Something like this:
require "json" # at the top of your script
# ...
def sum_hours_values(data, hours_values=nil)
# Start with an empty hash that automatically initializes missing keys to `0`
hours_values ||= Hash.new {|hsh,hour| hsh[hour] = 0 }
# Iterate through the [hour, value] arrays, adding `value` to the running
# count for that `hour`, and return `hours_values`
data.each_with_object(hours_values) do |(hour, value), hsh|
hsh[hour] += value
end
end
# ... Watir/Nokogiri stuff here...
# Initialize these so they persist outside the loop
hours_views, orders_views = nil
campaign_list.each do |campaign|
browser.goto "<redacted>"
if browser.text.include? "Application Error"
# ...
else
# ...
hourly_data_parsed = JSON.parse(hourly_data)
hours_views = sum_hours_values(hourly_data_parsed["views"], hours_views)
hours_orders = sum_hours_values(hourly_data_parsed["orders"], orders_views)
end
end
puts "Views by hour:"
puts hours_views.sort.map {|hour_views| "%2i\t%4i" % hour_views }
puts "Orders by hour:"
puts hours_orders.sort.map {|hour_orders| "%2i\t%4i" % hour_orders }
P.S. There's a really nice recursive version of sum_hours_values I didn't include since the iterative version is clearer to most Ruby programmers. If you're into recursion I leave it as an exercise for you. ;)

nil class when searching tags

I had a method with the following line
#noticias = Noticia.where(:tags.all => array).paginate(:page => params[:page])
it happens that brakeman says that it has a possible sql injection.
I tried the following instead:
array = params[:query].split(' ')
array.each_with_index do |query, index|
array[index] = array[index].gsub(/<\/?[^>]*>/, "").downcase
end
array.each do |tag|
#noticias << Noticia.where(:tags => tag)
end
but i got something like `undefined << for nil:NilClass
what am i missing?
If you're using Mongodb, you can sure that your code is SQL Injection free.
Although MongoDB isn't vulnerable to anything like SQL-injection, it may be worth checking the search string for anything malicious.
mongodb tutorial

trying to find the 1st instance of a string in a CSV using fastercsv

I'm trying to open a CSV file, look up a string, and then return the 2nd column of the csv file, but only the the first instance of it. I've gotten as far as the following, but unfortunately, it returns every instance. I'm a bit flummoxed.
Can the gods of Ruby help? Thanks much in advance.
M
for the purpose of this example, let's say names.csv is a file with the following:
foo, happy
foo, sad
bar, tired
foo, hungry
foo, bad
#!/usr/local/bin/ruby -w
require 'rubygems'
require 'fastercsv'
require 'pp'
FasterCSV.open('newfile.csv', 'w') do |output|
FasterCSV.foreach('names.csv') do |lookup|
index_PL = lookup.index('foo')
if index_PL
output << lookup[2]
end
end
end
ok, so, if I want to return all instances of foo, but in a csv, then how does that work?
so what I'd like as an outcome is happy, sad, hungry, bad. I thought it would be:
FasterCSV.open('newfile.csv', 'w') do |output|
FasterCSV.foreach('names.csv') do |lookup|
index_PL = lookup.index('foo')
if index_PL
build_str << "," << lookup[2]
end
output << build_str
end
end
but it does not seem to work
Replace foreach with open (to get an Enumerable) and find:
FasterCSV.open('newfile.csv', 'w') do |output|
output << FasterCSV.open('names.csv').find { |r| r.index('foo') }[2]
end
The index call will return nil if it doesn't find anything; that means that the find will give you the first row that has 'foo' and you can pull out the column at index 2 from the result.
If you're not certain that names.csv will have what you're looking for then a bit of error checking would be advisable:
FasterCSV.open('newfile.csv', 'w') do |output|
foos_row = FasterCSV.open('names.csv').find { |r| r.index('foo') }
if(foos_row)
output << foos_row[2]
else
# complain or something
end
end
Or, if you want to silently ignore the lack of 'foo' and use an empty string instead, you could do something like this:
FasterCSV.open('newfile.csv', 'w') do |output|
output << (FasterCSV.open('names.csv').find { |r| r.index('foo') } || ['','',''])[2]
end
I'd probably go with the "complain if it isn't found" version though.

Resources