Could you please give me some explanations and some code examples on how it would be done (ex: creating tables and inserting data) ?
Which library would you advise me to use ?
Thanks !
Please note the side-effect import of github.com/lib/pq
After this queries can be run by db.Query() or db.Exec()
https://golang.org/pkg/database/sql/#example_DB_Query
https://golang.org/pkg/database/sql/#pkg-examples
import (
_ "github.com/lib/pq"
"database/sql"
"fmt"
)
func MakeRedshfitConnection(username, password, host, port, dbName string) (*sql.DB, error) {
url := fmt.Sprintf("sslmode=require user=%v password=%v host=%v port=%v dbname=%v",
username,
password,
host,
port,
dbName)
var err error
var db *sql.DB
if db, err = sql.Open("postgres", url); err != nil {
return nil, fmt.Errorf("redshift connect error : (%v)"), err
}
if err = db.Ping(); err != nil {
return nil, fmt.Errorf("redshift ping error : (%v)", err)
}
return db, nil
}
Related
I use xorm to connect to my mysql database,but when my mysql doesn't start,xorm cann't print error message
package main
import (
"fmt"
_ "github.com/go-sql-driver/mysql"
"xorm.io/xorm"
)
var engine *xorm.Engine
func main() {
var err error
engine, err = xorm.NewEngine("mysql", "root:1234567#/blog?charset=utf8mb4")
if err != nil {
fmt.Println(err.Error())//can't print
return
}
}
Use the Ping method to check whether the database is alive
if err := engine.Ping(); err != nil {
panic(err)
}
Ping method
I am new to Golang and I am trying to connect to Dremio using ODBC in Golang with Dremio host, port, username and password. The following code gives me error
2022/04/19 17:36:42 missing username and password
exit status 1
import (
"database/sql"
"fmt"
"gographqlservice/graph/model"
"log"
_ "github.com/pinpt/go-dremio/driver"
)
const (
host = "dremio.xyz.abc.com"
user = "user1"
password = "user_password"
port = 32010
)
func GetAsset(id string) (*model.Asset, error) {
drminfo := fmt.Sprintf("host=%s port=%d user=%s password=%s sslmode=disable",
host, port, user, password)
db, err := sql.Open("dremio", drminfo)
if err != nil {
log.Fatal("error occurred")
log.Fatal(err)
}
rows, err := db.Query("SELECT * FROM space.xyz.\"assets\" WHERE ASSET_ID = ?", id)
if err != nil {
log.Fatal(err)
}
var asset model.Asset
defer rows.Close()
for rows.Next() {
// some code goes here
}
err = rows.Err()
if err != nil {
log.Fatal(err)
}
defer db.Close()
return &asset, err
}
It looks like you are using driver code from "Pinpoint" that is documented here:
https://pkg.go.dev/github.com/pinpt/go-dremio#section-readme
You can see their connection string should be in the following form:
db, err := sql.Open("dremio", "https://user:pass#dremio.example.com")
Your code appears to be in the form:
drminfo := fmt.Sprintf("host=%s port=%d user=%s password=%s sslmode=disable",
host, port, user, password)
To allow the Pinpoint code to parse this connection string and pull out the user and password, try the following instead:
drminfo := fmt.Sprintf("http://%s:%s#%s",
user, password, host)
The error message you're seeing comes from the following line in the Pinpoint code. In this case, it means they were not able to figure out the user name from the connection string provided.
https://github.com/pinpt/go-dremio/blob/master/driver/driver.go#L187
if u.User == nil {
return nil, fmt.Errorf("missing username and password")
}
This is primarily focused towards having setup() and teardown() methods for a test suite that I'm planning on writing that involves creation of a DB.
I've figured out how to create a DB using GORM. However, I'm not sure if this is the best approach.
package main
import (
"fmt"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/postgres"
"log"
)
func main() {
db, err := gorm.Open("postgres", "host=127.0.0.1 port=5432 user=superuser dbname=postgres password='' sslmode=disable")
capture(err)
db = db.Exec("CREATE DATABASE test_db;")
if db.Error != nil {
fmt.Println("Unable to create DB test_db, attempting to connect assuming it exists...")
db, err = gorm.Open("postgres", "host=127.0.0.1 port=5432 user=superuser dbname=test_db password='' sslmode=disable")
if err != nil {
fmt.Println("Unable to connect to test_db")
capture(err)
}
}
defer db.Close()
}
func capture(err error) {
if err != nil {
log.Fatalf("%s", err)
}
}
I'm connecting to the default postgres DB first, after which I'm creating a second test DB which I'm planning on using.
Is this the best approach ? Or is there a way to connect to Postgres without having a pre-existing DB.
NOTE: I've already looked up answers where people have used SQL driver to connect to a DB using only the connection string user:password#/. That has not worked in my case.(like here)
I've alse tried a connection string without having a DB name, that results in the driver trying to connect to a DB with the same name as the user. Which fails since such a DB does not exist.
The way I went about it was to avoid creating the db first with the expectation of an error and then using that as an indication db already exists. I find below to be more graceful IMO. This is with using GORM btw
connStr := fmt.Sprintf("user=%s password=%s host=%s port=%s dbname=%s sslmode=disable",
"user",
"password",
"host",
"port",
"postgres")
// connect to the postgres db just to be able to run the create db statement
db, err := gorm.Open(postgres.Open(connStr), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
return err
}
// check if db exists
stmt := fmt.Sprintf("SELECT * FROM pg_database WHERE datname = '%s';", client.Name)
rs := db.Raw(stmt)
if rs.Error != nil {
return rs.Error
}
// if not create it
var rec = make(map[string]interface{})
if rs.Find(rec); len(rec) == 0 {
stmt := fmt.Sprintf("CREATE DATABASE %s;", dbName)
if rs := db.Exec(stmt); rs.Error != nil {
return rs.Error
}
// close db connection
sql, err := db.DB()
defer func() {
_ = sql.Close()
}()
if err != nil {
return err
}
}
Here is how I achieved creating a postgreSQL database using Gorm, the key is to connect to postgreSQL only, a connection to "database" is not required to create a database, only connecting to database engine is enough. Just don't pass the database base in connection string.
In main.go
package main
import (
"gorm.io/driver/postgres"
"gorm.io/gorm"
"fmt"
"github.com/joho/godotenv"
)
func createDatabase() {
dsn := fmt.Sprintf("host=%s port=%s user=%s password=%s sslmode=disable TimeZone=%s", Config("DB_HOST"), Config("DB_PORT"), Config("DB_USER"), Config("DB_PASSWORD"), Config("DB_TIMEZONE"))
DB, _ := gorm.Open(postgres.Open(dsn), &gorm.Config{})
createDatabaseCommand := fmt.Sprintf("CREATE DATABASE %s", Config("DB_NAME"))
DB.Exec(createDatabaseCommand)
}
func main() {
createDatabase()
}
Now simply just run go get -d ./... && go run main.go
Your method seems valid enough. You could also use postgres' createdb utility to create the DB before you connect to it. For example:
import (
"log"
"os/exec"
"bytes"
)
func createPgDb() {
cmd := exec.Command("createdb", "-p", "5432", "-h", "127.0.0.1", "-U", "superuser", "-e", "test_db")
var out bytes.Buffer
cmd.Stdout = &out
if err := cmd.Run(); err != nil {
log.Printf("Error: %v", err)
}
log.Printf("Output: %q\n", out.String())
}
This example is paraphrased from the Command / Run examples in the Go manual https://golang.org/pkg/os/exec/#Command
I suggest using database/psql package for creating the database , not gorm
be sure you imported these packages
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
and use below codes to create database
url := fmt.Sprintf("host=%s port=%s user=%s password=%s sslmode=disable",
DBHost, DBPort, DBUsername, DBPassword)
db, err := sql.Open("postgres", url)
if err != nil {
panic(err)
}
defer db.Close()
_, err = db.Exec(fmt.Sprintf("CREATE DATABASE %s;", DBName))
if err != nil {
panic(err)
}
I am attempting to create a basic connection to a database. The problem happens when I try to test the connection with db.Ping(); everything works until I get to this line. The Ping sends the program into an infinite loop (the function call never returns), and I'm not sure how to go about fixing this.
package main
import (
"database/sql"
"fmt"
"html/template"
"net/http"
_ "github.com/lib/pq"
}
type Page struct {
Name string
DBStatus bool
}
const (
host = "localhost"
port = 8080
user = "username"
password = "password"
dbname = "GoTest"
)
func main() {
templates := template.Must(template.ParseFiles("templates/index.html"))
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
db, err := sql.Open("postgres", psqlInfo)
if err != nil {
panic(err)
}
defer db.Close()
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
p := Page{Name: "Gopher"}
if name := r.FormValue("name"); name != "" {
p.Name = name
}
p.DBStatus = db.Ping() == nil //this point is reached but never returned
if err := templates.ExecuteTemplate(w, "index.html", p); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
})
fmt.Println(http.ListenAndServe(":8080", nil))
}
It seems I can connect to the database fine, as the sql.Open call doesn't return an error, and if I called the Ping outside of the http server handle function, it also returns just fine.
Any help would be greatly appreciated!
Your database configurations are wrong. It's pointing to the Golang server port 8080. It should point to the pgsql port(default 5432)
I've an app that uses net/http. I register some handlers with http that need to fetch some stuff from a database before we can proceed to writing the response and be done with the request.
My question is in about which the best pratice is to connect to this database. I want this to work at one request per minute or 10 request per second.
I could connect to database within each handler every time a request comes in. (This would spawn a connection to mysql for each request?)
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
)
func main() {
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
db, err := sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
defer db.Close()
row := db.QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
I could connect to database at app start. Whenever I need to use the database I Ping it and if it's closed I reconnect to it. If it's not closed I continue and use it.
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"net/http"
"fmt"
"sync"
)
var db *sql.DB
var mutex sync.RWMutex
func GetDb() *sql.DB {
mutex.Lock()
defer mutex.Unlock()
err := db.Ping()
if err != nil {
db, err = sql.Open("mysql","dsn...")
if err != nil {
panic(err)
}
}
return db
}
func main() {
var err error
db, err = sql.Open("mysql","dsn....")
if err != nil {
panic(err)
}
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
row := GetDb().QueryRow("select...")
// scan row
fmt.Fprintf(w,"text from database")
})
http.ListenAndServe(":8080",nil)
}
Which of these ways are the best or is there another way which is better. Is it a bad idea to have multiple request use the same database connection?
It's unlikly I will create an app that runs into mysql connection limit, but I don't want to ignore the fact that there's a limit.
The best way is to create the database once at app start-up, and use this handle afterwards. Additionnaly, the sql.DB type is safe for concurrent use, so you don't even need mutexes to lock their use. And to finish, depending on your driver, the database handle will automatically reconnect, so you don't need to do that yourself.
var db *sql.DB
var Database *Database
func init(){
hostName := os.Getenv("DB_HOST")
port := os.Getenv("DB_PORT")
username := os.Getenv("DB_USER")
password := os.Getenv("DB_PASS")
database := os.Getenv("DB_NAME")
var err error
db, err = sql.Open("mysql", fmt.Sprintf("%s:%s#tcp(%s:%d)/%s", username, password, hostName, port, database))
defer db.Close()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
Database := &Database{conn: db}
}
type Database struct {
conn *sql.DB
}
func (d *Database) GetConn() *sql.DB {
return d.conn
}
func main() {
row := Database.GetConn().QueryRow("select * from")
}
I'd recommend make the connection to your database on init().
Why? cause init() is guaranteed to run before main() and you definitely want to make sure you have your db conf set up right before the real work begins.
var db *sql.DB
func GetDb() (*sql.DB, error) {
db, err = sql.Open("mysql","dsn...")
if err != nil {
return nil, err
}
return db, nil
}
func init() {
db, err := GetDb()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
}
I did not test the code above but it should technically look like this.