Let's assume that I want to make a client, for example, MySQL client (but my question is generic and not just for MySQL client, any client), this is a sample code
func main() {
db, err := sql.Open("mysql", "root:<yourMySQLdatabasepassword>#tcp(127.0.0.1:3306)/test")
if err != nil {
panic(err.Error())
}
defer db.Close()
}
I want to move it into a function so I don't have to initialize it all over the place and here my question begins I can do it in various ways and I want to know which one is the recommended way
Here is functional way
func client() (client *sql.DB, err error) {
client, err = sql.Open("mysql", "root:<yourMySQLdatabasepassword>#tcp(127.0.0.1:3306)/test")
if err != nil {
panic(err.Error())
}
return
}
Global variable way
var client, err = sql.Open("mysql", "root:<yourMySQLdatabasepassword>#tcp(127.0.0.1:3306)/test")
init func way
var Client *sql.DB
func init() {
Client, err := sql.Open("mysql", "root:root#tcp(localhost:3306)/otp")
if err != nil {
panic(err.Error())
}
}
and finally struct way
type MSQL struct {
// Fields...
}
func (m *MSQL) Client() *sql.DB {
client, err := sql.Open("mysql", "root:<yourMySQLdatabasepassword>#tcp(127.0.0.1:3306)/test")
if err != nil {
panic(err.Error())
}
return client
}
// NewMysql().where(...)
I'm really confused here to select which one to follow
Exposing a public variable var Client *sql.DB is error prone as I can do <package>.Client = null from anywhere.
struct way is the way to go, As you can work with multiple connection each to a specific database at any time.
You can also use New() to pass in a config object or other props to customise other db props. like Max Connection.
package database;
type Db struct{
*db sql.DB
}
func New(driver, connection string) *Db {
db, err := sql.Open(driver, connection)
if err != nil {
panic(err)
}
if err = db.Ping(); err != nil {
panic(err)
}
/* configure details
db.SetConnMaxLifetime(conLifeTime)
db.SetMaxOpenConns(maxOpenConns)
db.SetMaxIdleConns(maxIdleConns)
*/
return &Db{db}
}
func (d *Db) Close() error {
return d.db.Close();
}
Its generic, you can pass any driver & connection string for creating a connection to any database.
connection database.New("mysql", "root:<yourMySQLdatabasepassword>#tcp(127.0.0.1:3306)/test")
usage database.Db.Where(...)
close connection database.Db.Close()
Related
I use Gorm for Golang like this code,
func Connect() (*gorm.DB, error) {
var (
err error
db *gorm.DB
)
dsn := "sqlserver://User:12345#127.0.0.1:1433?database=gorm"
db, err = gorm.Open(sqlserver.Open(dsn), &gorm.Config{})
if err != nil {
return nil, err
}
err = db.Debug().AutoMigrate(&models.CompanyName{}, &models.CarModel{}, &models.CreateYear{}, &models.Diversity{})
if err != nil {
return nil, err
}
return db, nil
}
and in main.go
func main() {
db, err := Connect()
if err != nil {
panic("database connection failed")
}
ech := echo.New()
}
Ok in the first time when we run code, gorm create tables in database,but in the second time I got an error thet
mssql: There is already an object named 'table_names' in the database.
Do I have to delete db.Debug().AutoMigrate)(?
I have a gorm connection that I initially create by passing an AWS authentication token that expires every 15 minutes. The service will be able to connect to the DB for 15 minute. I have some functions for connecting to the db, for creating a new token, and for using a cron library to "refresh" the connection. I'm stuck in knowing how to actually provide the new token to GORM. I though of just doing gorm.Open() from within the cron job, but was told this would be bad because it'd not be thread safe, and would leave the old connections open. I was directed to this but I'm not sure what to do with it:
https://golang.org/pkg/database/sql/driver/#Connector
There is an example here:
https://github.com/aws/aws-sdk-go/issues/3043#issuecomment-581931580
But I'm not sure what I'd replace the driver and connector code in that example for GORM. Anyhow.
Here's the code I have so far. First this connection function that wraps around gorm.Open:
func Connect(conf *config.DbConfig) (*gorm.DB, error) {
host, port, err := net.SplitHostPort(conf.Host)
if err != nil {
return nil, err
}
sslMode := "require"
if conf.DisableTLS {
sslMode = "disable"
}
return gorm.Open(conf.Dialect,
fmt.Sprintf("host=%s port=%s user=%s password=%s dbname=%s sslmode=%s",
host, port, conf.Username, conf.Password, conf.Name, sslMode))
}
This is called in main like so:
//IF WE WANT TO USE AWS TOKEN, REPLACE PW IN CONFIG WITH TOKEN
if conf.DB.UseAwsToken {
authToken, err := field.CreateAuthToken(&conf.DB)
if err != nil {
logger.Fatal(err)
}
conf.DB.Password = authToken
}
//CONNECT TO DB
db, err := field.Connect(&conf.DB)
if err != nil {
logger.Fatal(err)
}
defer db.Close()
//INITIALIZE CRON JOB THAT RUNS IN SEPARATE THREAD TO REFRESH CONNECTION
if conf.DB.UseAwsToken {
processorCron := cron.New()
cronInterval := "#every 10m"
if conf.DB.CronInterval > 0 {
cronInterval = fmt.Sprintf("#every %s", conf.DB.CronInterval)
}
processorCron.AddFunc(cronInterval, func() {
repo.RenewDB(&conf.DB)
})
processorCron.Start()
}
The part I'm stuck with the actual code for the RenewDB function, in the spot with the TODO:
type Repo struct {
db *gorm.DB
}
// FUNCTION
func (r *Repo) RenewDB(dbConf *config.DbConfig) {
var err error
if dbConf.Password, err = CreateAuthToken(dbConf); err != nil {
panic(fmt.Sprintf("failed at createAuthToken: %s\n", err.Error()))
}
//TODO: HOW DO I UPDATE GORM TO USE THE NEW AUTH TOKEN THAT I INSERTED
// INTO dbConf?
}
//AUTH TOKEN GENERATOR
func CreateAuthToken(dbConf *config.DbConfig) (string, error) {
awsSession, err := session.NewSession()
if err != nil {
return "", err
}
authToken, errToken := rdsutils.BuildAuthToken(dbConf.Host, dbConf.AwsRegion, dbConf.Username, awsSession.Config.Credentials)
if errToken != nil {
return "", errToken
}
return authToken, nil
}
Here's a little golang program that connects to localhost via ssh and does some sftp-like operations. It works fine but I would like to make the doTheWork() function more testable.
I've read up on using gomock to create mocks for interfaces. So I've already created two interfaces, Sftper and Walker, and I'm using those instead of calling sftp's methods directly.
What I want to do is mock out all the stuff from stfp inside doTheWork(). Close() is easy because it only returns an error (or nil). But Create() returns *sftp.File and that is a struct, not an interface. Same with Walk() which returns another struct. I would have to create one of these structs in my test code, and that's not really possible without actually calling the real sftp library and making a remote connection and creating a remote file, and this is exactly what I am trying to mock out.
What am I missing?
package main
import (
"log"
"os"
"github.com/kr/fs"
"github.com/pkg/sftp"
"golang.org/x/crypto/ssh"
)
// Sftper helps make things testable
type Sftper interface {
Close() error
Create(path string) (*sftp.File, error)
Lstat(p string) (os.FileInfo, error)
Walk(root string) *fs.Walker
}
// Walker helps make things testable
type Walker interface {
Step() bool
Err() error
Path() string
}
var sftpclient Sftper
var w Walker
func doTheWork(sftpclient Sftper) {
defer sftpclient.Close()
// walk a directory
w = sftpclient.Walk("/tmp/")
for w.Step() {
if w.Err() != nil {
continue
}
log.Println(w.Path())
}
// leave your mark
f, err := sftpclient.Create("hello.txt")
if err != nil {
log.Fatal(err)
}
if _, err1 := f.Write([]byte("Hello world!")); err1 != nil {
log.Fatal(err1)
}
// check it's there
fi, err := sftpclient.Lstat("hello.txt")
if err != nil {
log.Fatal(err)
}
log.Println(fi)
}
func main() {
config := &ssh.ClientConfig{
User: "myusername",
Auth: []ssh.AuthMethod{
ssh.Password("mysupersecretpassword"),
},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}
conn, err0 := ssh.Dial("tcp", "localhost:22", config)
if err0 != nil {
log.Fatal("Failed to dial: ", err0)
}
// open an SFTP session over an existing ssh connection.
var err error
sftpclient, err = sftp.NewClient(conn)
if err != nil {
log.Fatal(err)
}
doTheWork(sftpclient)
}
I am beginner at Go, I had wrote small server to testing and deploy it on heroku platform. I have /logout request, which almost works, but sometimes I see something like this:
PANIC: write tcp 172.17.110.94:36641->10.11.189.195:9951: use of closed network connection
I don't know why it happens, and why sometimes it works perfectly.
My steps:
I send 1st POST request to /token-auth with body then generate token and send as response.
At 2nd I do /logout GET request with that token, and set token to Redis store
Here is full code of my redil_cli.go
package store
import (
"github.com/garyburd/redigo/redis"
)
type RedisCli struct {
conn redis.Conn
}
var instanceRedisCli *RedisCli = nil
func Connect() (conn *RedisCli) {
if instanceRedisCli == nil {
instanceRedisCli = new(RedisCli)
var err error
//this is works!!!
instanceRedisCli.conn, err = redis.Dial("tcp", "lab.redistogo.com:9951")
if err != nil {
panic(err)
}
if _, err := instanceRedisCli.conn.Do("AUTH", "password"); err != nil {
//instanceRedisCli.conn.Close()
panic(err)
}
}
return instanceRedisCli
}
func (redisCli *RedisCli) SetValue(key, value string, expiration ...interface{}) error {
_, err := redisCli.conn.Do("SET", key, value)
if err == nil && expiration != nil {
redisCli.conn.Do("EXPIRE", key, expiration[0])
}
return err
}
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
After that my function that checks Authorization header will panic while trying to do GetValue(key string) method
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
Can anyone point me, what I doing wrong?
I've an app that uses net/http. I register some handlers with http that need to fetch some stuff from a database before we can proceed to writing the response and be done with the request.
My question is in about which the best pratice is to connect to this database. I want this to work at one request per minute or 10 request per second.
I could connect to database within each handler every time a request comes in. (This would spawn a connection to mysql for each request?)
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
)
func main() {
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
db, err := sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
defer db.Close()
row := db.QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
I could connect to database at app start. Whenever I need to use the database I Ping it and if it's closed I reconnect to it. If it's not closed I continue and use it.
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
"sync"
)
var db *sql.DB
var mutex sync.RWMutex
func GetDb() *sql.DB {
mutex.Lock()
defer mutex.Unlock()
err := db.Ping()
if err != nil {
db, err = sql.Open("mysql","dsn...")
if err != nil {
panic(err)
}
}
return db
}
func main() {
var err error
db, err = sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
row := GetDb().QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
Which of these ways are the best or is there another way which is better. Is it a bad idea to have multiple request use the same database connection?
It's unlikly I will create an app that runs into mysql connection limit, but I don't want to ignore the fact that there's a limit.
The best way is to create the database once at app start-up, and use this handle afterwards. Additionnaly, the sql.DB type is safe for concurrent use, so you don't even need mutexes to lock their use. And to finish, depending on your driver, the database handle will automatically reconnect, so you don't need to do that yourself.
var db *sql.DB
var Database *Database
func init(){
hostName := os.Getenv("DB_HOST")
port := os.Getenv("DB_PORT")
username := os.Getenv("DB_USER")
password := os.Getenv("DB_PASS")
database := os.Getenv("DB_NAME")
var err error
db, err = sql.Open("mysql", fmt.Sprintf("%s:%s#tcp(%s:%d)/%s", username, password, hostName, port, database))
defer db.Close()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
Database := &Database{conn: db}
}
type Database struct {
conn *sql.DB
}
func (d *Database) GetConn() *sql.DB {
return d.conn
}
func main() {
row := Database.GetConn().QueryRow("select * from")
}
I'd recommend make the connection to your database on init().
Why? cause init() is guaranteed to run before main() and you definitely want to make sure you have your db conf set up right before the real work begins.
var db *sql.DB
func GetDb() (*sql.DB, error) {
db, err = sql.Open("mysql","dsn...")
if err != nil {
return nil, err
}
return db, nil
}
func init() {
db, err := GetDb()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
}
I did not test the code above but it should technically look like this.