Export content of a SQLite3 table in CSV - ruby

Question

I have a Ruby script that generates a SQLite3 database.
I want to be able to generate an "output.csv" file containing one of the database tables.
Is there a way to handle that in Ruby?

Answer 1

It is easy with Sequel and to_csv:
require 'sequel'
DB = Sequel.sqlite
# since Sequel 3.48.0 to_csv is deprecated,
# we must load the to_csv feature via a extension
DB.extension(:sequel_3_dataset_methods) #define to_csv
DB.create_table(:test){
Fixnum :one
Fixnum :two
Fixnum :three
}
#Prepare some test data
5.times{|i|
DB[:test].insert(i,i*2,i*3)
}
File.open('test.csv', 'w'){|f|
f << DB[:test].to_csv
}
The result is:
one, two, three
0, 0, 0
1, 2, 3
2, 4, 6
3, 6, 9
4, 8, 12
In my test I had problems with line ends, so I needed an additional gsub:
File.open('test.csv', 'w'){|f|
f << DB[:test].to_csv.gsub("\r\n","\n")
}
If you want the export without the header line, use to_csv(false)
Remarks:
.to_csv is deprecated since Sequel 3.48.0 (2013-06-01).
You may use an old version with gem 'sequel', '< 3.48.0' or load the extension sequel_3_dataset_methods).
To get support for other seperators and other CSV-features you may use a combination of Sequel and CSV:
require 'sequel'
require 'csv'
#Build test data
DB = Sequel.sqlite
DB.create_table(:test){
Fixnum :one
Fixnum :two
Fixnum :three
String :four
}
#Prepare some test data
5.times{|i|
DB[:test].insert(i,i*2,i*3, 'test, no %i' % i)
}
#Build csv-file
File.open('test.csv', 'w'){|f|
DB[:test].each{|data|
f << data.values.to_csv(:col_sep=>';')
}
}
Result:
0;0;0;"test, no 0"
1;2;3;"test, no 1"
2;4;6;"test, no 2"
3;6;9;"test, no 3"
4;8;12;"test, no 4"
As an alternative you may patch Sequel::Dataset (modified code from a post of marcalc at Github):
class Sequel::Dataset
require 'csv'
#
#Options:
#* include_column_titles: true/false. default true
#* Other options are forwarded to CSV.generate
def to_csv(options={})
include_column_titles = options.delete(:include_column_titles){true} #default: true
n = naked
cols = n.columns
csv_string = CSV.generate(options) do |csv|
csv << cols if include_column_titles
n.each{|r| csv << cols.collect{|c| r[c] } }
end
csv_string
end
end

Answer 2

# Assume that model is an activerecord model
#secrets = Model.all
#csv = CSV.generate do |csv|
#secrets.each { |secret|
csv << ["#{secret.attr1.to_s}", "#{secret.attr2.to_s"] # and so on till your row is finished
}
end
render :text => #csv, :content_type => 'application/csv'
If you have further problems, leave a comment.

Answer 3

Adding an update for 2020. Since Sequel v5, sequel_3_dataset_methods has been completely removed and is unavailable. As such, generating a CSV as a Database extension has also been completely removed.
It appears the current "best practice" is to add the csv_serializer plugin to a Sequel::Model class. There is a catch here though, that the Sequel::Model class you define must be defined after the call to Sequel.connect. The act of subclassing Sequel::Model invokes a read from the database.
This prevents a typical workflow of pre-defining your classes as part of any generic Gem.
According to the Sequel author, the preferred way to do this is through MyClass = Class.new(Sequel::Model(:tablename)) in-line, or otherwise only calling require within your method definitions.
Making no promises about efficiency, here is a code sample that defines 'best practice'
require 'sequel'
require 'csv'
module SequelTsv
class One
def self.main
db = Sequel.connect('sqlite://blog.db') # requires sqlite3
db.create_table :items do
primary_key :id
String :name
Float :price
end
items = db[:items] # Create a dataset
items.insert(:name => 'abc', :price => rand * 100)
items.insert(:name => 'def', :price => rand * 100)
items.insert(:name => 'ghi', :price => rand * 100)
item_class = Class.new(Sequel::Model(:items))
item_class.class_eval do
plugin :csv_serializer
end
tsv = item_class.to_csv(write_headers: true, col_sep:"\t")
CSV.open('output.tsv', 'w') do |csv|
CSV.parse(tsv) do | c |
csv << c
end
end
end
end
end
SequelTsv::One.main
output:
id name price
1 abc 39.307899453608364
2 def 99.28471503410731
3 ghi 58.0295131255661