Connection DB in golang - model-view-controller

Where i can put initialize files like languages, connection db etc. in mvc structur golang (beego,revel)?
I tried to use in controller but it isn't good.
Is a good solution would be create base controller and put here all init connection, languages etc? or is there some other way (better)?

You can use global variables, but I don't suggest to do it. What happens in more complicated applications where database logic is spread over multiple packages? It's better to use dependency injection:
File: main.go
package main
import (
"bookstore/models"
"database/sql"
"fmt"
"log"
"net/http"
)
type Env struct {
db *sql.DB
}
func main() {
db, err := models.NewDB("postgres://user:pass#localhost/bookstore")
if err != nil {
log.Panic(err)
}
env := &Env{db: db}
http.HandleFunc("/books", env.booksIndex)
http.ListenAndServe(":3000", nil)
}
func (env *Env) booksIndex(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
http.Error(w, http.StatusText(405), 405)
return
}
bks, err := models.AllBooks(env.db)
if err != nil {
http.Error(w, http.StatusText(500), 500)
return
}
for _, bk := range bks {
fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price)
}
}
File: models/db.go
package models
import (
"database/sql"
_ "github.com/lib/pq"
)
func NewDB(dataSourceName string) (*sql.DB, error) {
db, err := sql.Open("postgres", dataSourceName)
if err != nil {
return nil, err
}
if err = db.Ping(); err != nil {
return nil, err
}
return db, nil
}

I always do some packega where i keep my enviroment variables.
For example main.go
package main
import (
"net/http"
env "github.com/vardius/example/enviroment"
)
func main() {
//some extra code here, http srever or something
defer env.DB.Close()
}
end inside enviroment dir env.go
package env
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
var (
DB *sql.DB
)
func connectToDB(dbURL string) *sql.DB {
conn, err := sql.Open("mysql", dbURL)
//check for err
return conn
}
func init() {
DB = connectToDB("root:password#tcp(127.0.0.1:3306)/test")
}
this way you initialize once your DB and can use it in all parts of your app by injecting env
Ofcourse this solution has some downsides. First, code is harder to
ponder because the dependencies of a component are unclear. Second,
testing these components is made more difficult, and running tests in
parallel is near impossible. With global connections, tests that hit
the same data in a backend service could not be run in parallel.
There is a great article about a Dependency Injection with Go
I hope you will find this helpfull

Related

go build doesn't recognise methods

I try to setup a small Golang Microservice for users with Gin and Mongodb.
package main
import (
"context"
"fmt"
"github.com/wzslr321/artiver/entity"
"github.com/wzslr321/artiver/settings"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"log"
"os"
"os/signal"
"syscall"
"time"
)
type application struct {
users *entity.UserCollection
}
var app *application
func init() {
initMongo()
}
func initMongo() {
oc := options.Client().ApplyURI(settings.MongodbSettings.Uri)
client, err := mongo.NewClient(oc)
if err != nil {
log.Fatalf("Error occured while initializing a new mongo client: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
log.Fatalf("Errorr occurred while connecting to a client: %v", err)
}
defer func() {
if err = client.Disconnect(ctx); err != nil {
panic(err)
}
}()
log.Println("Successfully connected to the database!")
app = &application{
users: &entity.UserCollection{
C: client.Database("artiver").Collection("users"),
},
}
}
func main() {
router := app.InitRouter()
It doesn't show any errors in my IDE ( GoLand ), but when I try to build it I get an error:
# command-line-arguments
users/cmd/app/main.go:67:15: app.InitRouter undefined (type *application has no field or method InitRouter)
It it easily visible on the image above, that I do have access to such a method. It is defined in the same package.
package main
import (
"github.com/gin-gonic/gin"
cors "github.com/rs/cors/wrapper/gin"
"net/http"
)
func (app *application) InitRouter() *gin.Engine {
r := gin.New()
r.Use(gin.Recovery())
r.Use(cors.Default())
r.GET("/", func(ctx *gin.Context) {
ctx.String(http.StatusOK, "Hello World")
})
user := r.Group("/api/user")
{
user.POST("/add", app.CreateUser)
}
return r
}
I have no idea how am I supposed to fix it and what is done wrong. I'd appreciate any hint about what isn't done correctly.
Answer based on #mkopriva help in comments.
The issue was related to not running all needed .go files.
In my case, the solution was to build it this way in my Makefile:
go build -o $(path)users cmd/app/*
In similar cases, go run . most likely will do the job.

How correctly to make controllers to routes in Golang?

I am new in Golang and need some help. I am tring to create REST API web service without ORM.
Right now I am successfully connected to PostgreSQL database. In database I have table which called factors. I want to create CRUD operations. The problem is with controllers logic.
main.go:
package main
import (
"github.com/gorilla/mux"
"log"
"net/http"
"rest_api/configurations"
"rest_api/controllers"
)
func main() {
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
router := mux.NewRouter()
router.StrictSlash(true)
subrouter := router.PathPrefix("/api").Subrouter()
subrouter.HandleFunc("/factors", controllers.GetFactors(db)).Methods("GET")
log.Fatal(http.ListenAndServe(":8000", router))
}
models/factors.go:
package models
type Factor struct {
ID int `json:"id"`
Name string `json:"name"`
}
How correctly looks like the GetFactors controller? Can someone show me please. For instance I pass db object to GetFactors controller as in the example below. Unfortunately it seems like it's incorrect.
controllers/factors.go:
func GetFactors(db *sql.DB, w http.ResponseWriter, req *http.Request) {
// some code
}
configurations/PostgreSQL.go:
func PostgreSQLDatabase() (*sql.DB, error) {
// Load environment variables from ".env" file.
err := godotenv.Load(".env")
if err != nil {
log.Fatal(err)
}
// Initialize database-related variables.
dbUser := os.Getenv("PostgreSQL_USER")
dbPassword := os.Getenv("PostgreSQL_PASSWORD")
dbHost := os.Getenv("PostgreSQL_HOST")
dbName := os.Getenv("PostgreSQL_DB_NAME")
dbURL := fmt.Sprintf("user=%s password=%s host=%s dbname=%s sslmode=disable", dbUser, dbPassword, dbHost, dbName)
// Create PostgreSQL database connection pool.
db, err := sql.Open("postgres", dbURL)
if err != nil {
return nil, err
}
// Ping PostgreSQL database to make sure it's alive.
err = db.Ping()
if err != nil {
log.Fatal(err)
} else {
log.Println("Web service successfully connected to remote PostgreSQL database.")
}
return db, nil
}
A pattern I like to use is to define your own Router struct that has a mux.Router as a field as well as encapsulates things like your database connection, application config and etc.
I find doing it this way makes it easily update your routes when they require different resources and development proceeds.
First create a router object that takes in the database connection on creation and makes it available to all routes you wish to use.
router.go
package main
import (
"net/http"
"database/sql"
"github.com/gorilla/mux"
)
type Router struct {
router *mux.Router
db *sql.DB
}
func NewRouter(db *sql.DB) (*Router, error) {
router := mux.NewRouter()
router.StrictSlash(true)
subrouter := router.PathPrefix("/api").Subrouter()
r := &Router{
router: router,
db: db,
}
subrouter.HandleFunc("/factors", r.GetFactors).Methods(http.MethodGet)
return r, nil
}
func (r *Router) GetFactors(w http.ResponseWriter, req *http.Request) {
// Now you can access your database via `r.db`
}
// Needed so we can pass our custom router to ListenAndServe.
func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
r.router.ServeHTTP(w, req)
}
Then in main.go you can simply create your custom router, passing it your database connection. Then the custom router can be passed directly to ListenAndServe.
main.go
package main
import (
"log"
"net/http"
"rest_api/configurations"
"rest_api/controllers"
)
func main() {
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
router, err := NewRouter(db)
if err != nil {
log.Fatalf("error initializing router: %v", err)
}
log.Fatal(http.ListenAndServe(":8000", router))
}
Hopefully this helps.
Your func GetFactors must looks like:
func GetFactors(w http.ResponseWriter, r *http.Request) {}
and in main file you must have:
subrouter.HandleFunc("/factors", controllers.GetFactors).Methods("GET")
and with purpose to get DB connection you must have func like GetDB in your package "rest_api/configurations".
In "rest_api/configurations" you must have something like:
var db *PostgreSQLDatabase
func init() {
var err error
db, err = configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
}
func GetDB() *PostgreSQLDatabase {
return db
}
There is no correct way, it mostly opinion-based.
The semantic of HandlerFunc function should be like func(w http.ResponseWriter, r *http.Request), in order to pass database you can use closures, here is an example.
main.go
// ... some code here
subrouter.HandleFunc("/factors", controllers.GetFactors(db)).Methods("GET")
// ... some code here
controllers/factors.go
func GetFactors(db *sql.DB) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
// some code
})
}
Another option:
I'm not quite sure about this, but you can adjucts it to your needs. Initialize a Controller struct and pass db to it:
main.go
// ... some code here
db, err := configurations.PostgreSQLDatabase()
if err != nil {
log.Fatal(err)
}
ctrl := controllers.Controller{DB: db}
subrouter.HandleFunc("/factors", ctrl.GetFactors).Methods("GET")
// ... some code here
Denote a method on the Controller struct.
Define a struct in the controllers
controllers/factors.go
type Controller struct {
DB *PostgreSQLDatabase
}
func (c Controller) GetFactors(w http.ResponseWriter, req *http.Request) {
// some code
// c.DB.MySqlMethod()
}

How to separate DB connection initialization as a package in Go?

I have two packages, main and db.
However, I get "DB declared and not used" error.
db.go
package db
import (
"database/sql"
)
var DB *sql.DB
func Connect() {
DB, err := sql.Open("mysql", "root:xxxx#/xxxx")
if err != nil {
panic(err.Error())
}
}
func Close() {
DB.Close()
}
main.go
package main
import (
"database/sql"
// "fmt"
_ "github.com/go-sql-driver/mysql"
"html/template"
"net/http"
"github.com/****/****/config"
"github.com/****/****/db"
)
var tpl *template.Template
func init() {
tpl = template.Must(template.ParseGlob("templates/*.gohtml"))
}
func main() {
Connect()
defer Close()
loadRoutes()
http.ListenAndServe(":8080", nil)
}
Golang is strict about variable declaration, it is also mentioned in the Golang FAQs:
The presence of an unused variable may indicate a bug, while unused
imports just slow down compilation, an effect that can become
substantial as a program accumulates code and programmers over time.
For these reasons, Go refuses to compile programs with unused
variables or imports, trading short-term convenience for long-term
build speed and program clarity.
It's easy to address the situation, though. Use the blank identifier to let unused things persist while you're developing.
_, err := sql.Open("mysql", "root:Berlin2018#/jplatform")
But since you want db instance by creating a connection. I suggest to use it either by returning from the function OR You can check for the connection if it is working or not by sending ping to the database server as:
var DB *sql.DB
func Connect() {
DB, err := sql.Open("mysql", "root:Berlin2018#/jplatform")
if err = DB.Ping(); err != nil {
log.Panic(err)
}
}
Or you can create a struct which you can use anywhere you want including use of method receiver for every function which require db connection for querying the database as
type Env struct {
db *sql.DB
}
func Connect() {
db, err := sql.Open("mysql", "root:Berlin2018#/jplatform")
_ = &Env{db: db}
}
func(env *Env) getDataFromDatabase(){}
You are not using your DB variable on db.go:
package db
import (
"database/sql"
)
var DB *sql.DB
func Connect() {
con, err := sql.Open("mysql", "root:Berlin2018#/jplatform")
if err != nil {
panic(err.Error())
}
DB = con
}
func Close() {
DB.Close()
}

GORM DB Connection on other package

I start learning Go, reading about pointers, and want to split my database connection , and handler function for API. Already tried myself, by following this solution , but when i trying to read data, i am having this error
[2018-06-26 21:59:45] sql: database is closed
this is my source code.
db.go
package db
import (
"fmt"
"github.com/jinzhu/gorm"
"github.com/joho/godotenv"
"os"
)
var Db *gorm.DB
func Open() error {
var err error
_ = godotenv.Load(".env")
dbType := os.Getenv("DB_TYPE")
dbConnString := os.Getenv("DB_CONN_STRING")
Db, err = gorm.Open(dbType, dbConnString)
if err != nil {
fmt.Println(err)
}
Db.LogMode(true)
defer Db.Close()
return err
}
func Close() error {
return Db.Close()
}
person.go
package model
import (
"github.com/jinzhu/gorm"
"github.com/gin-gonic/gin"
"fmt"
"namastra/gin/result"
"namastra/gin/db"
)
type Person struct {
gorm.Model
FirstName string `json:”firstname”`
LastName string `json:”lastname”`
}
/*var db *gorm.DB
var err error*/
func GetPeople(c *gin.Context) {
var people []result.Person
if err := db.Db.Select("ID,first_name,last_name").Find(&people).Error; err != nil {
c.AbortWithStatus(404)
fmt.Println(err)
} else {
c.JSON(200, people)
}
}
main.go
package main
import (
"log"
"namastra/gin/handler"
"namastra/gin/model"
"net/http"
"time"
"github.com/adam-hanna/jwt-auth/jwt"
"github.com/gin-gonic/gin"
_ "github.com/jinzhu/gorm/dialects/postgres"
"namastra/gin/db"
)
func main() {
if err := db.Open(); err != nil {
// handle error
panic(err)
}
defer db.Close()
router := gin.Default()
router.Use(gin.Recovery())
private := router.Group("/auth")
....(ommited)
router.GET("/", gin.WrapH(regularHandler))
router.GET("/people/", model.GetPeople)
router.Run("127.0.0.1:3000")
}
Sorry for my bad english, any kind of help is appreciated.
thank you.
edit1: case closed.
solution is by removing
defer Db.Close()
from db.go.
edi2: update some knowledge i learn by working in go project
As start learning GO, usually we put everything on single main.go file, and we think to split the code to multiple files.
That is the time Dependency Injection comes to play.
we can create something like this Env to store the handler.
type Env struct {
db *sql.DB
logger *log.Logger
templates *template.Template
}
and create something like this in models/db.go
package models
import (
"database/sql"
_ "github.com/lib/pq"
)
func NewDB(dataSourceName string) (*sql.DB, error) {
db, err := sql.Open("postgres", dataSourceName)
if err != nil {
return nil, err
}
if err = db.Ping(); err != nil {
return nil, err
}
return db, nil
}
main.go files
package main
import (
"namastra/gin/models"
"database/sql"
"fmt"
"log"
"net/http"
)
type Env struct {
db *sql.DB
}
func main() {
db, err := models.NewDB("postgres://user:pass#localhost/bookstore")
if err != nil {
log.Panic(err)
}
env := &Env{db: db}
http.HandleFunc("/peoples", env.peoplesIndex)
http.ListenAndServe(":3000", nil)
}
func (env *Env) peoplesIndex(w http.ResponseWriter, r *http.Request) {
# ...
}
and in models/people.go
package models
import "database/sql"
type Book struct {
Isbn string
Title string
Author string
Price float32
}
func AllPeoples(db *sql.DB) ([]*People, error) {
rows, err := db.Query("SELECT * FROM peoples")
if err != nil {
return nil, err
}
defer rows.Close()
# ... ommited for simplicity
}
you can read the full code & explanation in Alex Edwards post

Database connection best practice

I've an app that uses net/http. I register some handlers with http that need to fetch some stuff from a database before we can proceed to writing the response and be done with the request.
My question is in about which the best pratice is to connect to this database. I want this to work at one request per minute or 10 request per second.
I could connect to database within each handler every time a request comes in. (This would spawn a connection to mysql for each request?)
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
)
func main() {
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
db, err := sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
defer db.Close()
row := db.QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
I could connect to database at app start. Whenever I need to use the database I Ping it and if it's closed I reconnect to it. If it's not closed I continue and use it.
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
"sync"
)
var db *sql.DB
var mutex sync.RWMutex
func GetDb() *sql.DB {
mutex.Lock()
defer mutex.Unlock()
err := db.Ping()
if err != nil {
db, err = sql.Open("mysql","dsn...")
if err != nil {
panic(err)
}
}
return db
}
func main() {
var err error
db, err = sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
row := GetDb().QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
Which of these ways are the best or is there another way which is better. Is it a bad idea to have multiple request use the same database connection?
It's unlikly I will create an app that runs into mysql connection limit, but I don't want to ignore the fact that there's a limit.
The best way is to create the database once at app start-up, and use this handle afterwards. Additionnaly, the sql.DB type is safe for concurrent use, so you don't even need mutexes to lock their use. And to finish, depending on your driver, the database handle will automatically reconnect, so you don't need to do that yourself.
var db *sql.DB
var Database *Database
func init(){
hostName := os.Getenv("DB_HOST")
port := os.Getenv("DB_PORT")
username := os.Getenv("DB_USER")
password := os.Getenv("DB_PASS")
database := os.Getenv("DB_NAME")
var err error
db, err = sql.Open("mysql", fmt.Sprintf("%s:%s#tcp(%s:%d)/%s", username, password, hostName, port, database))
defer db.Close()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
Database := &Database{conn: db}
}
type Database struct {
conn *sql.DB
}
func (d *Database) GetConn() *sql.DB {
return d.conn
}
func main() {
row := Database.GetConn().QueryRow("select * from")
}
I'd recommend make the connection to your database on init().
Why? cause init() is guaranteed to run before main() and you definitely want to make sure you have your db conf set up right before the real work begins.
var db *sql.DB
func GetDb() (*sql.DB, error) {
db, err = sql.Open("mysql","dsn...")
if err != nil {
return nil, err
}
return db, nil
}
func init() {
db, err := GetDb()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
}
I did not test the code above but it should technically look like this.

Resources