I currently develop golang projects with xorm.
I want to use a cache to manage the result of sql queries called once. I expected that sql requests would be called once, and would not be called again, but they do get called again.
Also, Redis keys that get created do not appear in redis-cli(keys *).
Why are my sql queries getting called more than once?
package main
import (
"github.com/go-xorm/xorm"
_ "github.com/go-sql-driver/mysql"
xrc "github.com/go-xorm/xorm-redis-cache"
)
type User struct {
Id int
Name string
}
func main() {
engine, err := xorm.NewEngine("mysql", "root:#/xorm_test_db")
if nil != err {
log.Fatal(err)
}
engine.ShowSQL(true)
cacher := xrc.NewRedisCacher("localhost:6379", "", xrc.DEFAULT_EXPIRATION, engine.Logger())
engine.SetDefaultCacher(cacher)
engine.Get(User{Id: 1})
engine.Get(User{Id: 1})
}
Two things has to be addressed for the caching to work properly:
Table must have a Primary Key for it to be cached. So Id can be
made as Primary Key as follows:
type User struct {
Id int `xorm:"pk"`
Name string
}
The type User must be registered using Golang's encoding/gob package:
gob.Register(new(User))
Don't forget to drop the existing table and sync the new User structure.
// Drop the existing table
DROP TABLE user;
// Sync the User struct to table
engine.Sync(new(User))
// Create a sample user
engine.Insert(&User{Id: 1, Name: "user1"})
The corrected code would look something as follows:
package main
import (
"github.com/go-xorm/xorm"
_ "github.com/go-sql-driver/mysql"
xrc "github.com/go-xorm/xorm-redis-cache"
"encoding/gob"
)
type User struct {
Id int `xorm:"pk"`
Name string
}
func main() {
gob.Register(new(User))
engine, err := xorm.NewEngine("mysql", "root:#/xorm_test_db")
if nil != err {
log.Fatal(err)
}
engine.ShowSQL(true)
cacher := xrc.NewRedisCacher("localhost:6379", "", xrc.DEFAULT_EXPIRATION, engine.Logger())
engine.SetDefaultCacher(cacher)
engine.Get(User{Id: 1})
engine.Get(User{Id: 1})
}
I resolved that!
package main
import (
"github.com/go-xorm/xorm"
_ "github.com/go-sql-driver/mysql"
xrc "github.com/go-xorm/xorm-redis-cache"
"encoding/gob"
"log"
)
type User struct {
Id int `xorm:"pk"`
Name string `xorm:"'name'"`
}
func main() {
gob.Register(new(User))
engine, err := xorm.NewEngine("mysql", "root:#/xorm_test2?charset=utf8")
if nil != err {
log.Fatal(err)
}
engine.Sync(new(User))
engine.Insert(&User{Id: 1, Name: "user1"})
engine.ShowSQL(true)
cacher := xrc.NewRedisCacher("localhost:6379", "", xrc.DEFAULT_EXPIRATION, engine.Logger())
engine.SetDefaultCacher(cacher)
engine.MapCacher(&User{}, cacher)
engine.Get(&User{Id: 1})
engine.Get(&User{Id: 1})
}
Related
Have this struct of User and Post and I try to make Name from User to be included within Post Struct when a user create a new post.
type User struct {
ID int
Name string
Created time.Time
}
type Post struct {
ID int
PostTitle string
PostDesc string
Created time.Time
}
How can I create something connected between this two struct such as Author of the Post ?
The goal is try to get the name of the post author which from User struct with the code below:
post, err := app.Models.Posts.GetPost(id)
GetPost() just run SELECT query and scan row
This approach is without any ORM.
It's a simple query that can return multiple rows. You've to scan the whole resultset and map each column on the struct's fields.
Keep in mind to always check for errors.
Below you can find the solution:
package main
import (
"database/sql"
"fmt"
"time"
_ "github.com/lib/pq"
)
type Post struct {
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
func main() {
conn, err := sql.Open("postgres", "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable")
if err != nil {
panic(err)
}
defer conn.Close()
// get MAX id
var id int
conn.QueryRow(`SELECT MAX(id) FROM posts`).Scan(&id)
// insert
sqlInsertStmt := `INSERT INTO posts (id, post_title, post_desc, created, user_id) VALUES ($1,$2,$3,$4,$5)`
if _, err = conn.Exec(sqlInsertStmt, id+1, "TDD", "Introduction to Test-Driven-Development", time.Now(), 1); err != nil {
panic(err)
}
// read
rows, err := conn.Query(`SELECT posts.id, post_title, post_desc, posts.created, users.id, users.name, users.created FROM posts INNER JOIN users ON posts.user_id=users.id`)
if err != nil {
panic(err)
}
var posts []Post
for rows.Next() {
var post Post
if err = rows.Scan(&post.ID, &post.PostTitle, &post.PostDesc, &post.Created, &post.User.ID, &post.User.Name, &post.User.Created); err != nil {
panic(err)
}
posts = append(posts, post)
}
if err = rows.Err(); err != nil {
panic(err)
}
for _, v := range posts {
fmt.Printf("author name: %q\n", v.User.Name)
}
}
Let me know if this helps.
Edit
I've also included an example of INSERT in Postgres. To achieve it, we've to use the db.Exec() function, and provide the parameters.
Pay attention to how you construct the query as you can get a SQL-Injection vulnerability.
Lastly, in a real-world scenario, you shouldn't lookup for the MAX id in the posts table but should be automatically generated.
Give it a try to this solution, maybe it resolves your issue.
First, I defined the structs in this way:
// "Post" belongs to "User", "UserID" is the foreign key
type Post struct {
gorm.Model
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
In this way, you can say that Post belongs to User and access the User's information within the Post struct.
To query the records, you've to use Preload("User") to be sure to eager load the User records from the separate table.
Keep attention to the name you pass in as the argument in Preload, as it can be tricky.
Lastly, you'll be able to access data in the embedded struct (with the dot notation).
Below, you can find a complete working example (implemented with the use of Docker):
package main
import (
"fmt"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
// "Post" belongs to "User", "UserID" is the foreign key
type Post struct {
gorm.Model
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic(err)
}
db.AutoMigrate(&Post{})
newPost := &Post{ID: 1, PostTitle: "Golang", PostDesc: "Introduction to Golang", Created: time.Now(), UserID: 1, User: User{ID: 1, Name: "John Doe", Created: time.Now()}}
db.Create(newPost)
var post Post
db.Preload("User").Find(&post, 1)
fmt.Printf("author name: %q\n", post.User.Name)
}
Let me know if I answered your question!
I'm attempting to query a database given a model, and I'm getting a blank model as a response. I'm using Revel framework and GORM to create the model and query the database.
app.go
package controllers
import (
"github.com/revel/revel"
"route/to/models"
)
type App struct {
*revel.Controller
GormController
}
func (c App) Index() revel.Result {
accounts := models.Account{}
account := c.DB.Find(accounts)
return c.RenderJSON(account)
}
gorm.go
package controllers
import (
_ "github.com/jinzhu/gorm/dialects/postgres"
"github.com/jinzhu/gorm"
r "github.com/revel/revel"
)
type GormController struct {
*r.Controller
DB *gorm.DB
}
// it can be used for jobs
var Gdb *gorm.DB
// init db
func InitDB() {
var err error
// open db
Gdb, err = gorm.Open("postgres", "host=hostname port=5432 user=postgres password=password dbname=some_db_name sslmode=disable")
Gdb.LogMode(true) // Print SQL statements
if err != nil {
r.ERROR.Println("FATAL", err)
panic(err)
}
}
func (c *GormController) SetDB() r.Result {
c.DB = Gdb
return nil
}
Not sure where I'm going wrong, ultimately I want to do a select * from accounts;
Note: I can run a migration and create in the Index() function and the data is handled correctly.
When populating a struct from the database, you need to pass a pointer for GORM to be able to populate it. Otherwise it's pass-by-value and GORM populates a copy and you never see it.
And if you want to get all accounts you should pass a pointer to a slice of accounts. Regarding pointers versus values, a slice is different from a struct in that the underlying array can still be modified even in pass-by-value, but outside the function the slice length (and capacity) won't change even as the array is populated within the function, so it still won't behave as expected without a pointer.
And you should check Error afterward:
accounts := make([]models.Account, 0)
if err := c.DB.Find(&accounts).Error; err != nil {
if gorm.IsRecordNotFoundError(err) {
// handle not found
} else {
// print/log/return error
}
return
}
// do something with accounts
I'm trying to write my first beego web app and I'm trying to register some models, i register them in models/model.go's init() function, but when I run the command bee run, I get the following error:
no Model found, need register your model
main.go:
package main
import (
"fmt"
_ "test_blog/routers"
"time"
_ "github.com/lib/pq"
"github.com/astaxie/beego"
"github.com/astaxie/beego/orm"
)
func init() {
orm.RegisterDriver("postgres", orm.DRPostgres)
maxIdle := 30
maxConn := 30
orm.RegisterDataBase(
"default",
"postgres",
"postgres://user:password#localhost/test_db", maxIdle, maxConn
)
orm.DefaultTimeLoc = time.UTC
}
func main() {
// Database alias.
name := "default"
// Drop table and re-create.
force := false
// Print log.
verbose := false
// Error.
err := orm.RunSyncdb(name, force, verbose)
if err != nil {
fmt.Println(err)
}
beego.Run()
}
Note: force & verbose both were set to true before running bee run for the first time.
models.go:
package main
import "github.com/astaxie/beego/orm"
type User struct {
Name string
Posts []*Post `orm:"reverse(many)"`
}
type Post struct {
Title string `orm:"size(50)"`
Text string `orm:"size(4000)"`
Created time.Time `orm:"auto_now_add;type(datetime)"`
Updated time.Time `orm:"auto_now;type(datetime)"`
Author *User `orm:"rel(fk)"`
}
func init() {
orm.RegisterModel(new(User), new(Post))
}
try this:
in models.go, change package main to package models
in main.go, add import _ "test_blog/models"
I've been trying to use the Associations feature in golang orm (https://github.com/jinzhu/gorm/), and am unable to create a pretty simple association.
In the example below, the user table contains data, but email table does not.
I've tried a bunch of things and I'm probably missing something basic, but have been unable to find the right answer in github/stackoverflow.
Code :
package main
import (
"database/sql"
"log"
"github.com/jinzhu/gorm"
"github.com/mattn/go-sqlite3"
)
var db gorm.DB
type User struct {
Name string
Mail Email
}
type Email struct {
Address string
}
//Initialize DB .
func InitDB() {
var DB_DRIVER string
sql.Register(DB_DRIVER, &sqlite3.SQLiteDriver{})
log.Printf("Initializing Database with ", DB_DRIVER)
dbSql, _ := sql.Open(DB_DRIVER, "simple-sqlite")
var err error
db, err = gorm.Open("sqlite3", dbSql)
if err != nil {
log.Fatalf("Got error when connecting to the database, the error is '%v'", err)
}
db.LogMode(true)
// Then you could invoke `*sql.DB`'s functions with it
db.DB().Ping()
db.DB().SetMaxIdleConns(10)
db.DB().SetMaxOpenConns(100)
// Disable table name's pluralization
db.SingularTable(true)
}
func InitSchema() {
db.CreateTable(&User{}, &Email{})
}
func DoStuff() {
user := User{Name: "Jinzhu", Mail: Email{Address: "hello#hello.com"}}
db.Create(&user)
}
func main() {
InitDB()
InitSchema()
DoStuff()
}
go run main.go prints the following output
2015/09/30 17:25:04 Initializing Database with %!(EXTRA string=)
[2015-09-30 17:25:04] [3.21ms] CREATE TABLE "user" ("name" varchar(255))
[2015-09-30 17:25:04] [4.01ms] CREATE TABLE "email" ("address" varchar(255) )
[2015-09-30 17:25:04] [0.54ms] INSERT INTO "user" ("name") VALUES ('Jinzhu')
Not sure what I'm missing here - appreciate any response!
You're missing your primary/foreign key references for each of your models, here's the updated code:
package main
import (
"database/sql"
"log"
"github.com/jinzhu/gorm"
"github.com/mattn/go-sqlite3"
)
var db gorm.DB
type User struct {
ID uint `gorm:"primary_key"`
Name string
Mail Email
MailID sql.NullInt64
}
type Email struct {
ID uint `gorm:"primary_key"`
Address string
}
//Initialize DB .
func InitDB() {
var DB_DRIVER string
sql.Register(DB_DRIVER, &sqlite3.SQLiteDriver{})
log.Printf("Initializing Database with ", DB_DRIVER)
dbSql, _ := sql.Open(DB_DRIVER, "simple-sqlite")
var err error
db, err = gorm.Open("sqlite3", dbSql)
if err != nil {
log.Fatalf("Got error when connecting to the database, the error is '%v'", err)
}
db.LogMode(true)
// Then you could invoke `*sql.DB`'s functions with it
db.DB().Ping()
db.DB().SetMaxIdleConns(10)
db.DB().SetMaxOpenConns(100)
// Disable table name's pluralization
db.SingularTable(true)
}
func InitSchema() {
db.CreateTable(&User{}, &Email{})
}
func DoStuff() {
user := User{Name: "Jinzhu", Mail: Email{Address: "hello#hello.com"}}
db.Create(&user)
}
func main() {
InitDB()
InitSchema()
DoStuff()
}
notice the primary keys on the User and Email structs as well as the foreign key reference on User
I have a database package that contains the following code.
package database
import (
"log"
"github.com/jinzhu/gorm"
// required by gorm
_ "github.com/mattn/go-sqlite3"
)
type Podcast struct {
ID int `sql:"index"`
Title string
RssURL string `sql:"unique_index"`
Paused bool
Episodes []Episode
}
type Episode struct {
ID int `sql:"index"`
PodcastID int
Title string
EnclosureURL string `sql:"unique_index"`
Downloaded bool
GUID string `sql:"unique_index"`
PubDate string
}
func DBSession() (db gorm.DB) {
sqliteSession, err := gorm.Open("sqlite3", cache.db)
if err != nil {
log.Fatal(err)
}
return sqliteSession
}
Followed by a bunch of methods that all start with the following code.
FindSomethingByID(id int) {
db := DBSession()
db.LogMode(false)
// code
}
FindSomethingElse {
db := DBSession()
db.LogMode(false)
// code
}
Calling DBSession and setting LogMode in each func seems bad. I just don't know how to do it better. Could someone help?
Calling gorm.Open inside every function isn't very efficient: Open opens a new connection pool, and should be called just once (see the database/sql docs, which gorm wraps).
A simple improvement is to establish a global gorm.DB, initialise it in init() it from all of your functions - e.g.
package database
var db gorm.DB
func init() {
var err error
// Note we use an = and not a := as our variables
// are already initialised
db, err = gorm.Open("sqlite3", "cache.db")
if err != nil {
log.Fatal(err)
}
// Turn off logging globally
db.LogMode(false)
}
FindSomethingByID(id int) {
err := db.Query("...")
// code
}
This is a quick win and reduces the repetition.
In a larger application it typically makes sense to pass dependencies (like DB pools, config params, etc.) more explicitly by wrapping them in types and creating custom handlers.
You also might initialise the connection in your package main and pass the *gorm.DB to your database package via a func New(db *gorm.DB) function that sets a private, package-level variable.
The most obvious simplification would be to move the db.LogMode(false) call into the DBSession() function, and give DBSession() a shorter name like DB():
func DB() (db gorm.DB) {
sqliteSession, err := gorm.Open("sqlite3", cache.db)
if err != nil {
log.Fatal(err)
}
sqliteSession.LogMode(false)
return sqliteSession
}
And using it:
FindSomethingByID(id int) {
db := DB()
// code
}
Now there's only 1 line in each of your functions using the db session, one simple function call. You can't really make it any shorter if you always need a new db session.