I'm doing an online course on Golang. The following piece of code is presented in the course material as an example of misuse of sync.Once:
var (
once sync.Once
db *sql.DB
)
func DbOnce() (*sql.DB, error) {
var err error
once.Do(func() {
fmt.Println("Am called")
db, err = sql.Open("mysql", "root:test#tcp(127.0.0.1:3306)/test")
if err != nil {
return
}
err = db.Ping()
})
if err != nil {
return nil, err
}
return db, nil
}
Supposedly, the above is a faulty implementation of an SQL connection manager. We, the students, are to find the error ourselves, which I struggle with. The code runs fine even in parallel. This is how I used it:
func main() {
wg := sync.WaitGroup{}
wg.Add(10)
for i := 0; i < 10; i++ {
go (func() {
db, err := DbOnce()
if err != nil {
panic(err)
}
var v int
r := db.QueryRow("SELECT 1")
err = r.Scan(&v)
fmt.Println(v, err)
wg.Done()
})()
}
wg.Wait()
}
I understand that homework questions are discouraged here, so I'm not asking for a complete solution, just a hint would be fine. Is the error related to concurrency (i.e. I need to run it in a specific concurrent context)? Is it usage of sql.Open specifically?
Initialization of the db variable is OK. The problem is with the returned error.
If you call DbOnce() for the first time and opening a DB connection fails, that error will be returned properly. But what about subsequent calls? The db initialization code will not be run again, so nil db may be returned, and since the initialization code is not run, the default value of the err variable is returned, which will be nil. To sum it up, the initialization error is lost and will not be reported anymore.
One solution is to stop the app if connection fails (at the first call). Another option is to store the initialization error too in a package level variable along with db, and return that from DbOnce() (and not use a local variable for that). The former has the advantage that you don't have to handle errors returned from DbOnce(), as it doesn't even have to return an error (if there's an error, your app will terminate).
The latter could look like this:
var (
once sync.Once
db *sql.DB
dbErr error
)
func DbOnce() (*sql.DB, error) {
once.Do(func() {
fmt.Println("Am called")
db, dbErr = sql.Open("mysql", "root:test#tcp(127.0.0.1:3306)/test")
if dbErr != nil {
return
}
dbErr = db.Ping()
})
return db, dbErr
}
Related
Playground link: https://go.dev/play/p/laQo-BfF7sK
It's subtle, but this InTx "context manager" (in transaction) has at least one bug. If there is a panic during the "Fun" call:
type Fun func(context.Context, *sql.Tx) error
func InTx(db *sql.DB, fn Fun) error {
ctx := context.Background()
t, err := db.BeginTx(ctx, nil)
if err != nil {
log.Panicln(err)
return err
}
return safe(ctx, t, fn)
}
// safe should run the provided function in the context of a SQL transaction
// expect a nil error if (and only if) everything worked w/o incident
func safe(ctx context.Context, t *sql.Tx, fn Fun) (err error) {
defer func() {
if err == nil {
err = t.Commit()
return
}
if bad := t.Rollback(); bad != nil && bad != sql.ErrTxDone {
err = fmt.Errorf("during rollback, panic(%v); err=%w", bad, err)
// log error
return
}
}()
err = fn(ctx, t)
return
}
Here is an example to demonstrate:
func main() {
var db *sql.DB;
// ...
_ = InTx(db, func(ctx context.Context, t *sql.Tx) error {
// ... lots more SQL executed here ...
if _, err := t.Exec("DELETE FROM products"); err != nil {
return err
}
// ...
panic("will cause Commit")
// should expect Rollback() instead, as if we:
//return nil
})
}
Related: Would it be inappropriate to panic during another panic, e.g. if Rollback fails? If so, why? (or when not)
Adding recover in another defer (after the first one in the safe function, since they unwind in stack order) would guard against an "inner" panic from the callback, but that may be sub-optimal or less idiomatic that other approaches.
defer func() {
if veryBad := recover(); veryBad != nil {
bad := t.Rollback()
err = fmt.Errorf("aborted SQL due to panic: %v; err=%w", veryBad, bad)
// log error, should re-panic here?
return
}
}()
I'd be very happy to accept someone else's Go wisdom in lieu of my potentially-flawed approach.
How to handle error of singleton only once?
I have a singleton service which could generate error only at first call and then it returns already created instance.
Service looks like below:
package data
import (
"sync"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
var (
databaseSingleton *gorm.DB
once sync.Once
)
func NewDatabase() (*gorm.DB, error) {
once.Do(func() {
// ...
databaseSingleton, err = gorm.Open(postgres.Open(connectionString), config)
if err != nil {
return nil, err
}
})
return databaseSingleton, nil
}
The problem is multiple services which uses databaseSingleton above handle error which can occurs only once.
Services which uses databaseSingleton looks like below:
func NewServiceOne() (ServiceOne, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceOne{database}, nil
}
func NewServiceTwo() (ServiceTwo, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceTwo{database}, nil
}
func NewServiceThree() (ServiceThree, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceThree{database}, nil
}
If there any way to omit this error handling because err could be generated only once?
If the error occurs (only once), your databaseSingleton will not be setup. You should return the error in all cases.
Although this isn't something you can do anything about (since the attempt to initialize databaseSingleton will not be repeated due to the use of sync.Once), you could as well halt the app.
In fact, there is no point deferring this initialization, you could just do it during package init, and terminate if it fails. And if it succeeds, you could use databaseSingleton without having to check error of the initialization.
So simply do it like this:
var databaseSingleton *gorm.DB
func init() {
var err error
databaseSingleton, err = gorm.Open(postgres.Open(connectionString), config)
if err != nil {
log.Fatalf("Failed to connect to DB: %v", err)
}
}
NewServiceOne() could look like this:
func NewServiceOne() ServiceOne {
return &serviceOne{databaseSingleton}
}
This question already has answers here:
Type converting slices of interfaces
(9 answers)
Closed 3 years ago.
my golang sqlite insert function. i'm using this package "github.com/mattn/go-sqlite3"
func Insert(args ...string)(err error){
db, err:=sql.Open("sqlite3","sqlite.db")
if err !=nil {
return
}
q, err := db.Prepare(args[0])
if err !=nil{
return
}
_,err = q.Exec(args[1:]...)
return
}
main (){
err := Insert("INSERT INTO table(first,last) VALUES(?,?)","Nantha","nk")
if err !=nil{
fmt.Println(err.Error())
return
}
}
i'm getting this error
cannot use args (type []string) as type []interface {} in argument to
q.Exec
The error is pretty clear, the function expects type []interface{} but you're passing in a value of type []string. You have to first convert []string to []interface{} before passing it to Exec. And the way to do that is to loop over the strings and add each one to a new slice of interface{}.
https://golang.org/doc/faq#convert_slice_of_interface
As an alternative approach, you can change the Insert argument types.
func Insert(query string, args ...interface{}) (err error) {
db, err := sql.Open("sqlite3", "sqlite.db")
if err != nil {
return err
}
q, err := db.Prepare(query)
if err != nil {
return err
}
_, err = q.Exec(args...)
return err
}
func main() {
err := Insert("INSERT INTO table(first,last) VALUES(?,?)", "Nantha", "nk")
if err !=nil{
fmt.Println(err.Error())
return
}
}
Please note that you're using the database/sql package incorrectly. Many of the objects returned from that package's functions/methods need to be closed to release the underlying resources.
This is true for *sql.DB returned by Open, *sql.Stmt returned by Prepare, *sql.Rows returned by Query, etc.
So your function should look closer to something like this:
func Insert(query string, args ...interface{}) (err error) {
db, err := sql.Open("sqlite3", "sqlite.db")
if err != nil {
return err
}
defer db.Close()
q, err := db.Prepare(query)
if err != nil {
return err
}
defer q.Close()
_, err = q.Exec(args...)
return err
}
Also note that sql.DB is reusable, that means that you don't have to sql.Open a new instance every time you need to talk to the database.
From the docs on Open:
The returned DB is safe for concurrent use by multiple goroutines and
maintains its own pool of idle connections. Thus, the Open function
should be called just once. It is rarely necessary to close a DB.
If you keep doing it the way you're doing it, openning a new DB every time you call Insert or any other function that needs to talk to the DB, your program will perform worse than if you had a single DB and have your functions reuse that.
I am new to golang and trying to get a better understanding of context.
In the below snippet, it appears to me that I've instantiated my computeService with a context. why do I have to pass it again to the .Context() function when calling Stop()?
package main
func stopTaggedMachines(ctx context.Context, svc *compute.Service, project, zone, tag string) ([]string, error) {
var instances []string
f := func(page *compute.InstanceList) error {
for _, v := range page.Items {
if v.Labels["gcp-idler-managed"] == "true" {
result, err := svc.Instances.Stop(project, zone, v.Name).Context(ctx).Do()
if err != nil {
log.Fatal(err)
}
fmt.Printf("[INFO] gcp-machine-idler: Instance in state %v, Stopping %v... Response: %v \n", v.Status, v.Name, result.HTTPStatusCode)
}
}
return nil
}
call := svc.Instances.List("my-project", "us-west1-b")
if err := call.Pages(oauth2.NoContext, f); err != nil {
return instances, nil
}
return instances, nil
}
func main() {
// Use oauth2.NoContext if there isn't a good context to pass in.
ctx := context.Background()
computeService, err := compute.NewService(ctx)
if err != nil {
log.Fatal(err)
}
stopTaggedMachines(ctx, computeService, "my-project", "us-west1-b", "gcp-idler-managed")
return
}
It seems redundant to me that I pass ctx into compute.NewService(), then again into stopTaggedMachines()
Is this really the correct convention or usage of context? Why does my call to svc.Instances.Stop(project, zone, v.Name).Context(ctx).Do() need to be passed ctx yet again as a parameter?
svc.Instances.Stop(project, zone, v.Name) returns InstanceStopCall
By calling Context(ctx) you are setting the context to be used in this call's Do method. This allows the HTTP request to be aborted if the context is canceled.
The Stop method can take a long time (as in minutes). This allows a user to cancel waiting for a VM to shutdown.
I thought I'd try to be a little bit "clever" and abstract some of my boilerplate SQL code (using sqlx -- https://github.com/jmoiron/sqlx). The idea is to feed a code a function pointer to process the result, along with the sql string and args that produce the rows. As it happens, the code works fine provided I strip out the "sqlArgs interface" stuff, but in the "cleverer" format errors with the statement
sql: converting Exec argument $1 type: unsupported type []interface {}, a slice of interface
Here are two versions, the first one that errors, the second that works but without parameterization:
//GetRows (doesn't work)
func GetRows(parseRows func(*sqlx.Rows), sql string, sqlArgs ...interface{}) {
db := sqlx.MustConnect("mysql", ConnString)
defer db.Close()
rows, err := db.Queryx(sql, sqlArgs)
defer rows.Close()
if err != nil {
panic(err)
}
parseRows(rows)
}
//GetRows ... (works, but doesn't allow parameterization)
func GetRows(fp func(*sqlx.Rows), sql string) {
db := sqlx.MustConnect("mysql", ConnString)
defer db.Close()
rows, err := db.Queryx(sql)
defer rows.Close()
if err != nil {
panic(err)
}
fp(rows)
}
The idea is to call the code something like this:
func getUser(userID string) User {
var users []*User
parseRows := func(rows *sqlx.Rows) {
for rows.Next() {
var u User
err := rows.StructScan(&u)
if err != nil {
panic(err)
}
users = append(users, u)
}
}
sql := "SELECT * FROM users WHERE userid = ?;"
sqlutils.GetRows(parseRows, sql, userID)
if len(users) == 1{
return users[0]
}
return User{}
}
I guess my code doesn't actually pass through the userID from call to call, but instead it passes an []interface{}, which the sql package can't handle. I'm not sure about that, however. In any case, is there any way to accomplish this idea? Thanks.