How to bundle an SQLite database in a Go binary? - go

Question

I am try to use go-bindata and packr, but those packages do not show how to pack an SQLite database file in to a binary file.
I don't need to update the database in any way, I just want to read the data from it on startup.
How can I embed an SQLite database file in a Go binary file?

Answer 1

The SQLite driver can't read a database file from memory (e.g. from a byte slice). But you can write the data to a temporary file, and open that:
//go:generate go run gen.go
package main
import (
"database/sql"
"fmt"
"io/ioutil"
"log"
"os"
_ "github.com/mattn/go-sqlite3"
)
func main() {
// Create temporary file for database.
tmpDB, err := ioutil.TempFile("", "db*.sqlite3")
if err != nil {
log.Fatal(err)
}
// Remove this file after on exit.
defer func() {
err := os.Remove(tmpDB.Name())
if err != nil {
log.Print(err)
}
}()
// Write database to file.
_, err = tmpDB.Write(sqlDB)
if err != nil {
log.Print(err)
}
err = tmpDB.Close()
if err != nil {
log.Print(err)
}
// Open DB.
db, err := sql.Open("sqlite3", tmpDB.Name()+"?mode=ro")
if err != nil {
log.Fatal(err)
}
// Make sure it's loaded correct.
rows, err := db.Query("select * from test")
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var c string
err := rows.Scan(&c)
if err != nil {
log.Fatal(err)
}
fmt.Println(c)
}
}
And you can write the database to db.go with something like:
// +build generate
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
// Read source database file.
d, err := ioutil.ReadFile("source.sqlite3")
if err != nil {
log.Fatal(err)
}
fp, err := os.Create("db.go")
if err != nil {
log.Fatal(err)
}
_, err = fmt.Fprintf(fp, "// Code generated by gen.go; DO NOT EDIT.\n\n"+
"package main\n\n"+
"var sqlDB = %s\n", asbyte(d))
if err != nil {
log.Fatal(err)
}
}
// Write any data as byte array.
func asbyte(s []byte) string {
var b strings.Builder
for i, c := range s {
if i%19 == 0 {
b.WriteString("\n\t\t")
}
b.WriteString(fmt.Sprintf("%#x, ", c))
}
return "[]byte{" + b.String() + "}"
}
You can also use go-bindata or packr for that if you prefer, but I don't really see an advantage.
An alternative way is to use a memory database, which may be faster depending on what you want to do.
Embed the SQL schema and rows you want in your Go binary as strings.
Open a new memory database when your program starts (sql.Open("sqlite3",:memory:`) and create the schema and insert the rows.
There is no disk access with this method, so querying it will probably be a bit faster at the expensive of slower startup times (benchmark to be sure!)