I'm trying to read and parse an sqlite3 query in Ruby using the sqlite3 gem. This db already exists on my machine.
I'm opening the db with
db = SQLite3::Database.new "/path to/database.sqlite"
The I'm executing my particular query with
db.execute( "SELECT * FROM `ZSFNOTE` WHERE `ZTRASHED` LIKE '0'" ) do |row|
Now, based on my (limited) experience, I was hoping that this could be parsed like a JSON response, where I could call something like row["ZTITLE"]. However, those headers aren't available in my response, I can only get at what I'm looking for by guessing an integer, like row[19].
I know I'm not even scratching the surface of the sqlite3 gem, but couldn't find the answer to this in the docs. Any help would be much appreciated.
You can use #execute2 to get the headers.
require 'sqlite3'
db = SQLite3::Database.new(':memory:')
db.execute 'CREATE TABLE "examples" ("header" varchar(20), "value" integer(8))'
db.execute 'INSERT INTO examples(header, value) VALUES("example", 1)'
db.execute2('select * from examples')
# => [["header", "value"], ["example", 1]]
You can map the headers to the columns like so:
headers, *rows = db.execute2('select * from examples')
rows.map! do |row|
row.each_with_index.with_object({}) do |(col, i), o|
o[headers[i]] = col
end
end
rows.each do |row|
p row['header']
end
# => "example"
Related
I'm new to MongoDB and databases in general. I'm using Ruby and I would like to query against a specific UUID in the database.
The ID is stored as _id and the value is '101b437a-be16-44f6-b0b0-0201cdee6510'
I have the following that usually queries my database:
field = '_id:'
value = 101b437a-be16-44f6-b0b0-0201cdee6510
def query_field(field,value)
query = {#{field}: value}
#result = #mongo_interface.get(query)
expect(#result.count).to be >= 1
puts "Number of matched values: #{#result.count}"
end
def get(param_hash, collection_name = nil)
col_name = (collection_name.nil? || collection_name.empty?) ? #collection : collection_name
#docs = #db[col_name].find(param_hash)
end
When I look within the _id field, I'm assuming it's stored as some sort of binary key and thus isn't found using my search.
Is there some conversion I could/should do to make the query above work?
Thank you.
Using an ODM like Mongoid will ease your pain. Add it to your Gemfile:
gem 'mongoid'
and run bundle install. Make sure you skimmed through the installation guide to add all the necessary configs.
Then include the following line to your model/class, say:
class Product
include Mongoid::Document
...
end
You'll be able to query the records like Product.find(id) right after.
I've been trying to get my ruby script threaded since yesterday. I've since opted for SQLite to save data, with the parallel gem to manage concurrency.
I've built a quick script for testing, but I'm having trouble getting the threading working; the database is locked. I've added db.close to the end, which doesn't help, and I've tried adding sleep until db.closed?, but that just sleeps indefinitely. What am I doing wrong?
The error is "database is locked (SQLite3::BusyException)".
Here's my code:
require 'sqlite3'
require 'pry'
require 'parallel'
STDOUT.sync = true
db = SQLite3::Database.new "test.db"
arr = [1,2,3,4,5,6,7,8,9,10]
rows = db.execute <<-SQL
create table test_table (
original string,
conversion string
);
SQL
def test(num)
db = SQLite3::Database.new "test.db"
puts "the num: #{num}"
sleep 4
{ num => num + 10}.each do |pair|
db.execute "insert into test_table values (?, ?)", pair
end
db.close
end
Parallel.each( -> { arr.pop || Parallel::Stop}, in_processes: 3) { |number| test(number) }
SQLite is threadsafe by default (using its "serialized" mode) and the ruby wrapper apparently supports this to whatever extent it needs to. However, it's not safe across processes, which makes a certain sense since the adapter or engine probably has to negotiate some state in the process to prevent locks.
To fix your example change in_processes to in_threads
I'm trying to transfer all the information from my Ruby file into a Postgres database. I am able to transfer the information when I do not have an array column, so I am assuming the error message I am getting is because of the array column I am trying to add. The error message I am getting is:
in `exec_prepared': ERROR: missing dimension value (PG::InvalidTextRepresentation)
Here is the code I used to connect my Ruby file to my Postgres database:
require 'pg'
class Postgres
# Create the connection instance. Scraping is the name of the database I am adding this information to
def connect
#conn = PG.connect(:dbname => 'scraping')
end
# Create our venue table
def createVenueTable
#conn.exec("CREATE TABLE venues (venue_number varchar(15) UNIQUE,...,img_array varchar[]);")
end
...
def prepareInsertVenueStatement
#conn.prepare("insert_venue", "insert into venues(venue_number,...,img_array) values ($1,...,$24)")
end
# Add a venue with the prepared statement.
def addVenue(venue_number,...,img_array)
#conn.exec_prepared("insert_venue", [venue_number,...,img_array])
end
end
When I check my Postgres database, the img_array column is made, however, I am unable to populate it. Please help! Thank you.
I would suggest using serialization to handle this so that you are actually just writing a string rather than an actual array.
require 'pg'
require 'yaml'
class Postgres
# Create the connection instance. Scraping is the name of the database I am adding this information to
def connect
#conn = PG.connect(:dbname => 'scraping')
end
# Create our venue table
def createVenueTable
#changed img_array to a varchar(8000) for storing serialized Array
#conn.exec("CREATE TABLE venues (venue_number varchar(15) UNIQUE,...,img_array varchar(8000));")
end
...
def prepareInsertVenueStatement
#conn.prepare("insert_venue", "insert into venues(venue_number,...,img_array) values ($1,...,$24)")
end
# Add a venue with the prepared statement.
def addVenue(venue_number,...,img_array)
#conn.exec_prepared("insert_venue", [venue_number,...,serialized(img_array)])
end
#serialize the Object
def serialized(obj)
YAML.dump(obj)
end
#deserialize the Object
def deserialized(obj)
YAML.load(obj)
end
end
Abstracted Usage Example just to show serialization
a = [1,2,4,5]
serialized = YAML.dump(a)
#=> "---\n- 1\n- 2\n- 3\n- 4\n- 5\n"
YAML.load(serialized)
#=> [1,2,3,4,5]
#Also works on Hash Objects
h = {name: "Image", type: "jpeg", data:[1,2,3,4,5]}
serial = YAML.dump(h)
#=> "---\n:name: Image\n:type: jpeg\n:data:\n- 1\n- 2\n- 3\n- 4\n- 5\n"
YAML.load(serial)
#=> {:name=>"Image", :type=>"jpeg", :data=>[1, 2, 3, 4, 5]}
Hope this helps you out with handling this issue.
If you need to store more than 8000 characters you can switch to varchar(MAX) or text column definitions. I would recommend varchar(MAX) because the data will be stored as a standard varchar until it exceeds 8000 character at which point the db basically converts it to a text column under the hood.
How do I display the result set of a postgres query using the pg gem only ? I want it to be in tabular form like it is in pgAdmin GUI tool. The code below does not help. The documentation is not clear, so I am unable to figure another way out. Please help !
require 'pg'
conn = PGconn.connect("db.corp.com", 5432, '', '', "schema", "user", "pass")
sql = 'select * from tbl limit 2'
res = conn.exec(sql)
res.each do |row|
row.each do |column|
end
end
gem list -
pg (0.9.0.pre156 x86-mswin32)
ruby - 1.8.7
Steps -
1. Get list of column names in result set (type PGResult).
2. Iterate each row(hash) of result set.
3. For each row (hash key)/columns found in step 1, find the column values (hash value).
Then print results as csv. I dont think this is efficient, but it gets the job done.
require 'pg'
conn = PGconn.connect("db.corp.com", 5432, '', '', "schema", "user", "pass")
sql = 'select * from tbl limit 2'
res = conn.exec(sql)
rows_count = res.num_tuples
column_names = res.fields
col_header = column_names.join(', ')
puts col_header
for i in 0..rows_count-1
row_hash = res[i]
row_arr = []
column_names.each do |col|
row_arr << row_hash[col]
end
row = row_arr.join(', ')
puts row
end
I have been trying to puts some executed statements after I prepare them. The purpose of this is to sanitize my data inputs, which I have never done before. I followed the steps here, but I am not getting the result I want.
Here's what I have:
require 'sqlite3'
$db = SQLite3::Database.open "congress_poll_results.db"
def rep_pull(state)
pull = $db.prepare("SELECT name, location FROM congress_members WHERE location = ?")
pull.bind_param 1, state
puts pull.execute
end
rep_pull("MN")
=> #<SQLite3::ResultSet:0x2e69e00>
What I am expecting is a list of reps in MN, but instead I just get "SQLite3::ResultSet:0x2e69e00" thing.
What am I missing here? Thanks very much.
Try this
def rep_pull(state)
pull = $db.prepare("SELECT name, location FROM congress_members WHERE location = ?")
pull.bind_param 1, state
pull.execute do |row|
p row
end
end