golang memory reference rules for go-routines [duplicate] - go

I have the following file structure:
models/db.go
type DB struct {
*sql.DB
}
var db *DB
func init() {
dbinfo := fmt.Sprintf("user=%s password=%s dbname=%s sslmode=disable",
DB_USER, DB_PASSWORD, DB_NAME)
db, err := NewDB(dbinfo)
checkErr(err)
rows, err := db.Query("SELECT * FROM profile")
checkErr(err)
fmt.Println(rows)
}
func NewDB(dataSourceName string) (*DB, error) {
db, err := sql.Open("postgres", dataSourceName)
if err != nil {
return nil, err
}
if err = db.Ping(); err != nil {
return nil, err
}
return &DB{db}, nil
}
models/db_util.go
func (p *Profile) InsertProfile() {
if db != nil {
_, err := db.Exec(...)
checkErr(err)
} else {
fmt.Println("DB object is NULL")
}
}
When I try to access db in InsertProfile function, it says NULL ptr exception. How do I access the db in db_utils.go?
I would not like to capitalize db (as it would give access to all the packages).
I am getting the QUERY returned from the db in init() correctly.

Edit: The problem is that you used Short variable declaration := and you just stored the created *DB value in a local variable and not in the global one.
This line:
db, err := NewDB(dbinfo)
Creates 2 local variables: db and err, and this local db has nothing to do with your global db variable. Your global variable will remain nil. You have to assign the created *DB to the global variable. Do not use short variable declaration but simple assignment, e.g:
var err error
db, err = NewDB(dbinfo)
if err != nil {
log.Fatal(err)
}
Original answer follows.
It's a pointer type, you have to initialize it before you use it. The zero value for pointer types is nil.
You don't have to export it (that's what starting it with a capital letter does). Note that it doesn't matter that you have multiple files as long as they are part of the same package, they can access identifiers defined in one another.
A good solution would be to do it in the package init() function which is called automatically.
Note that sql.Open() may just validate its arguments without creating a connection to the database. To verify that the data source name is valid, call DB.Ping().
For example:
var db *sql.DB
func init() {
var err error
db, err = sql.Open("yourdrivername", "somesource")
if err != nil {
log.Fatal(err)
}
if err = db.Ping(); err != nil {
log.Fatal(err)
}
}

icza has already correctly answered your specific problem but it's worth adding some additional explanation on what you're doing wrong so you understand how not to make the mistake in the future. In Go, the syntax := for assignment creates new variables with the names to the left of the :=, possibly shadowing package, or even parent scope function/method variables. As an example:
package main
import "fmt"
var foo string = "global"
func main() {
fmt.Println(foo) // prints "global"
// using := creates a new function scope variable
// named foo that shadows the package scope foo
foo := "function scope"
fmt.Println(foo) // prints "function scope"
printGlobalFoo() // prints "global"
if true {
foo := "nested scope"
fmt.Println(foo) // prints "nested scope"
printGlobalFoo() // prints "global"
}
// the foo created inside the if goes out of scope when
// the code block is exited
fmt.Println(foo) // prints "function scope"
printGlobalFoo() // prints "global"
if true {
foo = "nested scope" // note just = not :=
}
fmt.Println(foo) // prints "nested scope"
printGlobalFoo() // prints "global"
setGlobalFoo()
printGlobalFoo() // prints "new value"
}
func printGlobalFoo() {
fmt.Println(foo)
}
func setGlobalFoo() {
foo = "new value" // note just = not :=
}
Note Go has no way to delete or unset a variable, so once you have shadowed a higher scope variables (such as by creating a function scope variable of the same name as a package scope variable), there is no way to access the higher scope variable within that code block.
Also be aware that := is a shorthand for var foo =. Both act in exactly the same way, however := is only valid syntax within a function or method, while the var syntax is valid everywhere.

For who came here and wants a fast answer.
in db.go file:
package db
var db *DB
type DB struct {
*gorm.DB // or what database you want like *mongo.Client
}
func GetDB() *DB {
if db == nil{
db = ConnectToYourDbFunc("connection_string")
}
return db
}
then in your other packages you can get it just with this:
db := db.GetDB()
thats all.

Related

Golang - create an object of the same type as passed

I'm trying to build a generic function which will parse input (in JSON) into a specified structure. The structure may vary at run-time, based on parameters which are passed to the function. I'm currently trying to achieve this by passing an object of the right type and using reflect.New() to create a new output object of the same type.
I'm then parsing the JSON into this object, and scanning the fields.
If I create the object and specify the type in code, everything works. If I pass an object and try to create a replica, I get an "invalid indirect" error a few steps down (see code).
import (
"fmt"
"reflect"
"encoding/json"
"strings"
)
type Test struct {
FirstName *string `json:"FirstName"`
LastName *string `json:"LastName"`
}
func genericParser(incomingData *strings.Reader, inputStructure interface{}) (interface{}, error) {
//******* Use the line below and things work *******
//parsedInput := new(Test)
//******* Use vvv the line below and things don't work *******
parsedInput := reflect.New(reflect.TypeOf(inputStructure))
decoder := json.NewDecoder(incomingData)
err := decoder.Decode(&parsedInput)
if err != nil {
//parsing error
return nil, err
}
//******* This is the line that generates the error "invalid indirect of parsedInput (type reflect.Value)" *******
contentValues := reflect.ValueOf(*parsedInput)
for i := 0; i < contentValues.NumField(); i++ {
//do stuff with each field
fmt.Printf("Field name was: %s\n", reflect.TypeOf(parsedInput).Elem().Field(i).Name)
}
return parsedInput, nil
}
func main() {
inputData := strings.NewReader("{\"FirstName\":\"John\", \"LastName\":\"Smith\"}")
exampleObject := new(Test)
processedData, err := genericParser(inputData, exampleObject)
if err != nil {
fmt.Println("Parsing error")
} else {
fmt.Printf("Success: %v", processedData)
}
}
If I can't create a replica of the object, then a way of updating / returning the one supplied would be feasible. The key thing is that this function must be completely agnostic to the different structures available.
reflect.New isn't a direct analog to new, as it can't return a specific type, it only can return a reflect.Value. This means that you are attempting to unmarshal into a *reflect.Value, which obviously isn't going to work (even if it did, your code would have passed in **Type, which isn't what you want either).
Use parsedInput.Interface() to get the underlying value after creating the new value to unmarshal into. You then don't need to reflect on the same value a second time, as that would be a reflect.Value of a reflect.Value, which again isn't going to do anything useful.
Finally, you need to use parsedInput.Interface() before you return, otherwise you are returning the reflect.Value rather than the value of the input type.
For example:
func genericParser(incomingData io.Reader, inputStructure interface{}) (interface{}, error) {
parsedInput := reflect.New(reflect.TypeOf(inputStructure).Elem())
decoder := json.NewDecoder(incomingData)
err := decoder.Decode(parsedInput.Interface())
if err != nil {
return nil, err
}
for i := 0; i < parsedInput.Elem().NumField(); i++ {
fmt.Printf("Field name was: %s\n", parsedInput.Type().Elem().Field(i).Name)
}
return parsedInput.Interface(), nil
}
https://play.golang.org/p/CzDrj6sgQNt

Golang Standard Package Structure

I am faily new to Go and I am trying to create a structured application using guidance from Ben Johnson's webpage. Unfortunately, his example is not a complete working application.
His webpage is https://medium.com/#benbjohnson/standard-package-layout-7cdbc8391fc1
I have tried to use his methods and I keep getting "Undefined: db" error. It doesn't tell me what line is causing the error, just the file "MSSQL.go"
Could someone help with guidance to help me fix this error?
Edited code with accepted solution.
StatementPrinter.go
package statementprinter
type Statement struct {
CustomerId string
CustomerName string
}
type StatementService interface {
Statement(id string) (*Statement, error)
}
main.go
package main
import (
"fmt"
"log"
"github.com/ybenjolin/StatementPrinter"
"github.com/ybenjolin/StatementPrinter/mssql"
"database/sql"
_ "github.com/alexbrainman/odbc"
)
const DB_INFO = "Driver={SQL Server};Server=cdc-edb2;Database=CostarReports;Trusted_Connection=yes;"
var db *sql.DB
func init() {
var err error
db, err = sql.Open("odbc", DB_INFO)
if err != nil {
log.Fatal("Error opening database connection.\n", err.Error())
}
err = db.Ping()
if err != nil {
log.Fatal("Error pinging database server.\n", err.Error())
}
fmt.Println("Database connection established.")
}
func main () {
var err error
defer db.Close()
// Create services
// Changes required here. Was ss := &statementprinter.Stat..
ss := &mssql.StatementService{DB: db}
// Use service
var s *statementprinter.Statement
s, err = ss.Statement("101583")
if err != nil {
log.Fatal("Query failed:", err.Error())
}
fmt.Printf("Statement: %+v\n", s)
}
mssql.go
package mssql
import (
_ "github.com/alexbrainman/odbc"
"database/sql"
"github.com/ybenjolin/StatementPrinter"
)
// StatementService represents a MSSQL implementation of statemenetprinter.StatementService.
type StatementService struct {
DB *sql.DB
}
// Statement returns a statement for a given customer.
func (s *StatementService) Statement(customer string) (*statementprinter.Statement, error) {
var err error
var t statementprinter.Statement
// Changes required here. Was row := db.Query......
row := s.DB.QueryRow(`Select Customer, CustomerName From AccountsReceivable.rptfARStatementHeader(?)`, customer)
if row.Scan(&t.CustomerId, &t.CustomerName); err != nil {
return nil, err
}
return &t, nil
This seems like it's just a typo. It seems like the problematic line is in the method
func (s *StatementService) Statement(customer string)
in mssql.go,
row := db.QueryRow(`Select Customer, CustomerName From AccountsReceivable.rptfARStatementHeader(?)`, customer)
QueryRow is supposed to be a method of db, but db is not defined. However, in the struct
type StatementService struct {
DB *sql.DB
}
there's a *sql.DB instance. The method you're using has a *StatementService parameter, s. So, my guess is the intention would be to access the sql.DB field in s like so
func (s *StatementService) Statement(customer string) (*statementprinter.Statement, error) {
var err error
var t statementprinter.Statement
//CHANGED LINE:
row := s.DB.QueryRow(`Select Customer, CustomerName From AccountsReceivable.rptfARStatementHeader(?)`, customer)
if row.Scan(&t.CustomerId, &t.CustomerName); err != nil {
return nil, err
}
return &t, nil
Then, the method is called in main.go, and is passed a StatementService instance that contains a database:
ss := &statementprinter.StatementService{DB: db}
I believe you need to change this line to
ss := &mssql.StatementService{DB: db}
becuase that's the actual interface implementation. The line you have now treats the StatementService interface like a struct which will not compile.
The global db in main.go lives for the lifetime of the application. It's just a pointer which is copied around for use.

Always a new variable when multiple returns

I'm facing a wierd problem with Golang.
On init() function, i want to assign a value to my variable that was declared outside this function.
But to assign the value to this var, i need to get
error
to check if everything is ok.
Here is the code:
var retryValue time.Duration
func init() {
retryValue, err := time.ParseDuration(retries)
if err != nil {
log.Fatal("retries value is invalid", err)
}
}
func a(){
fmt.Println(retryValue)
}
And i get the compiling error:
retryValue declared and not used
I need to change init() to this:
func init() {
var err error
retryValue, err = time.ParseDuration(retries)
if err != nil {
log.Fatal("retries value is invalid", err)
}
}
There is another way to solve this problem?
:=
always create a new variable if one of the variables are already declared? It's about variable golang's sope?
Thanks!
There are two basic ways to do this. You found one:
var retryValue time.Duration
func init() {
var err error
retryValue, err = time.ParseDuration(retries)
if err != nil {
log.Fatal("retries value is invalid", err)
}
}
A slightly shorter method would be:
var retryValue = func() time.Duration {
rv, err := time.ParseDuration(retries)
if err != nil {
log.Fatal("retries value is invalid", err)
}
return rv
}()
But better than either of these, would be to just specify the value directly, rather than parsing it:
var retryValue = 15 * time.Minute // or whatever value you want
Your fix is indeed the shortest way to do this, and no there is not another way to solve the issue.
This problem is rare enough that it is not really an issue in practice, after all it is only one more (short) line.

Wrapped sql.DB by struct can't invoke methods in instance

My goal is using a struct to wrap sql.DB to do something more than sql.DB.
The instance I create can't work and give me this error when I run it.
DbConn.db.prepare undefined (cannot refer to unexported field or method sql.(*DB)."".prepare)
My code is:
type DatabaseConn struct {
driverName string
databaseName string
userName string
password string
dataSourceName string
db *sql.DB
}
func (d DatabaseConn)Open() error {
d.driverName = DB_DRNAME
d.userName = DB_UNAME
d.password = DB_PWD
d.databaseName = DB_DBNAME
d.dataSourceName = fmt.Sprintf("%s:%s#/%s?charset=utf8",d.userName, d.password, d.databaseName)
db, err := sql.Open(d.driverName, d.dataSourceName)
return err
}
func (d *DatabaseConn)Close() error {
defer func() {
if err := recover(); err != nil {
fmt.Println("Trying to handle error in DatabaseConn.Close(): ", err)
}
}()
err := d.db.Close()
return err
}
I am trying to create an instance and invoke sql.DB method.
var dbConn DatabaseConn
dbConn.Open()
defer dbConn.Close()
dbQuery := fmt.Sprintf("SELECT *, FROM ms_node WHERE node_id = ?")
getNodeRecord, err := dbConn.db.prepare(dbQuery)
The error message is pointed here.
./server.go:343: dbConn.db.prepare undefined (cannot refer to unexported field or method sql.(*DB)."".prepare)
(Personal background: I am newbie for 2 weeks)
The function Prepare starts with a capital letter ;)
So you should have:
getNodeRecord, err := dbConn.db.Prepare(dbQuery)

How to dry up database code

I have a database package that contains the following code.
package database
import (
"log"
"github.com/jinzhu/gorm"
// required by gorm
_ "github.com/mattn/go-sqlite3"
)
type Podcast struct {
ID int `sql:"index"`
Title string
RssURL string `sql:"unique_index"`
Paused bool
Episodes []Episode
}
type Episode struct {
ID int `sql:"index"`
PodcastID int
Title string
EnclosureURL string `sql:"unique_index"`
Downloaded bool
GUID string `sql:"unique_index"`
PubDate string
}
func DBSession() (db gorm.DB) {
sqliteSession, err := gorm.Open("sqlite3", cache.db)
if err != nil {
log.Fatal(err)
}
return sqliteSession
}
Followed by a bunch of methods that all start with the following code.
FindSomethingByID(id int) {
db := DBSession()
db.LogMode(false)
// code
}
FindSomethingElse {
db := DBSession()
db.LogMode(false)
// code
}
Calling DBSession and setting LogMode in each func seems bad. I just don't know how to do it better. Could someone help?
Calling gorm.Open inside every function isn't very efficient: Open opens a new connection pool, and should be called just once (see the database/sql docs, which gorm wraps).
A simple improvement is to establish a global gorm.DB, initialise it in init() it from all of your functions - e.g.
package database
var db gorm.DB
func init() {
var err error
// Note we use an = and not a := as our variables
// are already initialised
db, err = gorm.Open("sqlite3", "cache.db")
if err != nil {
log.Fatal(err)
}
// Turn off logging globally
db.LogMode(false)
}
FindSomethingByID(id int) {
err := db.Query("...")
// code
}
This is a quick win and reduces the repetition.
In a larger application it typically makes sense to pass dependencies (like DB pools, config params, etc.) more explicitly by wrapping them in types and creating custom handlers.
You also might initialise the connection in your package main and pass the *gorm.DB to your database package via a func New(db *gorm.DB) function that sets a private, package-level variable.
The most obvious simplification would be to move the db.LogMode(false) call into the DBSession() function, and give DBSession() a shorter name like DB():
func DB() (db gorm.DB) {
sqliteSession, err := gorm.Open("sqlite3", cache.db)
if err != nil {
log.Fatal(err)
}
sqliteSession.LogMode(false)
return sqliteSession
}
And using it:
FindSomethingByID(id int) {
db := DB()
// code
}
Now there's only 1 line in each of your functions using the db session, one simple function call. You can't really make it any shorter if you always need a new db session.

Resources