db.First() not using primary key name - go

I am writing some sample code to understand gorm but appear to be having an issue with setting up the primary_key value. As a result, the resulting SQL query is broken.
Please see the sample code:
package main
import (
"os"
"fmt"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type Post struct {
id int `json:"id" gorm:"primary_key:id"`
url string `gorm:"url"`
content string `gorm:"content"`
created_at time.Time `gorm:"created_at"`
normalized string `gorm:"normalized"`
account_id int `gorm:"account_id"`
posthash []byte `gorm:"posthash"`
received_at time.Time `gorm:"received_at"`
}
func main() {
dsn := "user=postgres dbname=localtest"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println("An error occurred", err)
os.Exit(1)
}
db.AutoMigrate(&Post{})
var post Post
db.First(&post, 1)
fmt.Println(post)
}
When I run this code, I receive the following error:
$ go run gorm_test.go
2020/12/01 00:34:06 /home/farhan/gorm_test.go:32 ERROR: syntax error at or near "=" (SQLSTATE 42601)
[0.128ms] [rows:0] SELECT * FROM "posts" WHERE "posts". = 1 ORDER BY "posts". LIMIT 1
{0 {0 0 <nil>} 0 [] {0 0 <nil>}}
The nature of this error suggests to me that the primary_key value is not set. I tried primaryKey and primarykey, but none appeared to work. Any ideas?

#Shubham Srivastava - Your structure declaration is bad; you need to export fields so gorm can use them Declaring Models
❗️Use exported ID field
In Go, when a field name starts with a lowercase letter that means the field is private (called unexported in Go parlance). Gorm, being a third party package, cannot see inside your struct to know there is an id field, or any other unexported field for that matter.
The solution is to make sure all the fields that need to come from the DB are exported:
type Post struct {
ID uint `json:"id" gorm:"primaryKey"`
...
}

Related

How to use pgtype.Numeric with gorm and sqlite3?

I need to store very large and high precision numbers with gORM, and using a pgtype.Numeric seems like the best bet. However, I cannot because I get an error: sql: Scan error on column index 4, name "foo": cannot scan int64
My model looks something like this:
type Model struct {
gorm.Model
Foo *pgtype.Numeric `gorm:"not null"`
}
Not sure if using pgtype.Numeric is the best (that's what i've seen everyone else use), or I'm doing something wrong. Thanks!
The code that caused the error:
package main
import (
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"math/big"
"github.com/jackc/pgtype"
)
type Model struct {
gorm.Model
Foo *pgtype.Numeric `gorm:"not null"`
}
func main() {
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
db.AutoMigrate(&Model{})
// Create
db.Create(&Model{Foo: &pgtype.Numeric{Int: big.NewInt(10000000), Status: pgtype.Present}})
var m Model
db.First(&m) // this line causes the error
}
Sqlite3 does not support big integer so there is no way you can accomplish that directly. I run the code and foo column is create as:
`foo` numeric NOT NULL
Which in sqlite https://www.sqlite.org/datatype3.html means
A column with NUMERIC affinity may contain values using all five storage classes... If the TEXT value is a well-formed integer literal that is too large to fit in a 64-bit signed integer, it is converted to REAL.
So your big int will turn into float64. Good thing it paniced instead of losing accuracy silently.
What you can do is convert the big int to string or bytes first and store that.
When debugging the sql.Scanner interface for database deserialization, it is noticeable that the value from the database arrives either as int64 or float64. This then leads to the corresponding error message.
A possible solution is to use a text data type in the database, by adding the type text to the field tag:
`gorm: "type:text;"`
Using the github.com/shopspring/decimal package, you can conveniently create a decimal number using the NewString function.
The adapted code to insert the data:
num, err := decimal.NewFromString("123456789012345.12345678901")
if err != nil {
panic(err)
}
db.Create(&Model{Foo: &num})
The model structure might then look something like this:
type Model struct {
gorm.Model
Foo *decimal.Decimal `gorm: "not null;type:text;"`
}
This would result in the following schema:
Test
If one inserts a breakpoint in decimal.Scan, one can see that the value comes from the database as expected as a string, resulting in the creation of a decimal with NewFromString (see Decimal's scan method).
If you add this line of code to the end of the main function
fmt.Println(m.Foo)
it would result in the following output in the debug console:
123456789012345.12345678901
Complete Program
Your complete program, slightly adapted to the above points, would then look something like this:
package main
import (
"fmt"
"github.com/shopspring/decimal"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
type Model struct {
gorm.Model
Foo *decimal.Decimal `gorm:"not null;type:text;"`
}
func main() {
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
db.AutoMigrate(&Model{})
// Create
num, err := decimal.NewFromString("123456789012345.12345678901")
if err != nil {
panic(err)
}
db.Create(&Model{Foo: &num})
var m Model
db.First(&m)
fmt.Println(m.Foo)
}
pgtype.Numeric and SQLite
If a PostgreSQL database is used, gorm can be used together with pgtype.Numeric to handle decimal numbers like 123456789012345.12345678901. You just need to use the numeric data type on the Postgres side with the appropriate desired precision (e.g. numeric(50,15)).
After all, this is exactly what pgtype is for, see the pgtype readme where it says:
pgtype is the type system underlying the https://github.com/jackc/pgx PostgreSQL driver.
However, if you use a text data type in SQLite for the reasons mentioned above, pgtype.Numeric will not work with SQLite. An attempt with the above number writes 12345678901234512345678901e-11 to the DB and when reading it out the following error occurs:
sql: Scan error on column index 4, name "foo": 12345678901234512345678901e-11 is not a number

Difference in string and *string in Gorm model declaration

The docs of gorm https://gorm.io/docs/models.html present an example below.
The field Name and Email are described with string and *string.
What is the main difference here?
Also how to provide the datatype for the images field storing a list of images link?
Should it be []string or []*string?
type User struct {
ID uint
Name string
Email *string
Images []string
Age uint8
Birthday *time.Time
MemberNumber sql.NullString
ActivatedAt sql.NullTime
CreatedAt time.Time
UpdatedAt time.Time
}
Go has default values for every primitive data types.
int -> 0, string -> "", bool -> false likewise. So if you need to add null value, or load null value to a variable, it should be a pointer. Otherwise it is defaulted.
Default value of a pointer is nil in Go.
And complex data types such as slices, maps keep references. So their default value is nil. So, Images []string here images can be nil.
Below code with pointer types User1 and without pointer types User2 show the difference in default values.
package main
import (
"fmt"
"time"
)
type User1 struct {
Email *string
Images []string
Birthday *time.Time
}
type User2 struct {
Email string
Images []string
Birthday time.Time
}
func main() {
user1 := User1{}
user2 := User2{}
fmt.Printf("user1 := %+v \n", user1)
//Output : user1 := {Email:<nil> Images:[] Birthday:<nil>}
fmt.Printf("user2 := %+v \n", user2)
//Output : user2 := {Email: Images:[] Birthday:0001-01-01 00:00:00 +0000 UTC}
}
The main difference is that if you use pointers, you can put a null value into the DB, else you must put a string.
Esentially if the database field is nullable, you should use pointers.

GET blob from sql database Golang

import (
"database/sql"
"encoding/json"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
type User struct {
Name string `json:name`
Picture []uint8 `json:picture`
}
func main(){
//straight to the query
rows, err := 'SELECT name, picture FROM ms_users' // picture is longblob type in database
checkErr(err)
var usr User
for rows.Next(){
err = rows.Scan(&usr.Name, &usr.Picture)
checkErr(err)
}
jsn, err := json.Marshal(usr)
fmt.Printf("%v, "string(jsn))
}
With above code, I only get name value but the picture is empty.
How do I store blob value from databse to struct ?
Any answer will be appreciated! thank you!
I'm relatively new to GO I encountered this question while searching a solution for a similar problem I was able to find a solution.
When you get BLOB data from the database you get it as type []byte your struct can look like this below
type User struct {
Name string `json:name`
Picture []byte`json:picture`
}
I guess you can process the byte array according to you need later. In my case I needed a JSON object so I unmarshalled it to a type interface{} variable.

Could't convert <nil> into type ...?

I tried to using database/sql for query database row into a Go type, my codes snippet following:
type User struct {
user_id int64
user_name string
user_mobile string
password string
email interface{}
nickname string
level byte
locked bool
create_time string
comment string // convert <nil> to *string error
}
func TestQueryUser(t *testing.T) {
db := QueryUser(driverName, dataSourceName)
stmtResults, err := db.Prepare(queryAll)
defer stmtResults.Close()
var r *User = new(User)
arr := []interface{}{
&r.user_id, &r.user_name, &r.user_mobile, &r.password, &r.email,
&r.nickname, &r.level, &r.locked, &r.create_time, &r.comment,
}
err = stmtResults.QueryRow(username).Scan(arr...)
if err != nil {
t.Error(err.Error())
}
fmt.Println(r.email)
}
MySQL:
As your see, some fields that has NULL value, so I have to set interface{} type into User struct of Go, which convert NULL to nil.
--- FAIL: TestQueryUser (0.00s)
user_test.go:48: sql: Scan error on column index 9: unsupported Scan, storing driver.Value type <nil> into type *string
Somebody has a better way? or I must change the MySQL field and set its DEFAULT ' '
First the short answer : There is some types in sql package for example sql.NullString (for nullable string in your table and guess Nullint64 and NullBool and ... usage :) ) and you should use them in your struct.
The long one : There is two interface for this available in go , first is Scanner and the other is Valuer for any special type in database, (for example,I use this mostly with JSONB in postgres) you need to create a type, and implement this two(or one of them) interface on that type.
the scanner is used when you call Scan function. the data from the database driver, normally in []byte is the input and you are responsible for handling it. the other one, is used when the value is used as input in query. the result "normally" is a slice of byte (and an error) if you need to only read data, Scanner is enough, and vice-versa, if you need to write parameter in query the Valuer is enough
for an example of implementation, I recommend to see the types in sql package.
Also there is an example of a type to use with JSONB/JSON type in postgresql
// GenericJSONField is used to handle generic json data in postgres
type GenericJSONField map[string]interface{}
// Scan convert the json field into our type
func (v *GenericJSONField) Scan(src interface{}) error {
var b []byte
switch src.(type) {
case []byte:
b = src.([]byte)
case string:
b = []byte(src.(string))
case nil:
b = make([]byte, 0)
default:
return errors.New("unsupported type")
}
return json.Unmarshal(b, v)
}
// Value try to get the string slice representation in database
func (v GenericJSONField) Value() (driver.Value, error) {
return json.Marshal(v)
}
driver value is often []byte but string and nil is acceptable. so this could handle null-able fields too.

struct Time property doesn't load from Go sqlx library

I have a struct with a property time:
type Basket struct {
...
Created_at time.Time `db:"created_at"`
}
with the time saved as:
basket.Created_at = time.Now().UTC()
If I save it using Insert sql statement, it saves the time nicely in the SQLite3 but when I select the desired record using:
ret_basket := Basket{}
err := database.DB.Get(&ret_basket, "SELECT id, ..., created_at FROM baskets WHERE user_id = ?", some_user_id)
It returns the record with other properties loaded properly except the time property which is ret_basket.Created_at as 0001-01-01 00:00:00 +0000 UTC
Any suggestions?
There is no official package for Sqlite so I assume you are using https://github.com/mattn/go-sqlite3 Probably your issue is result of wrong created_at field declaration in database which should be DATETIME because the next code works perfectly on my machine (I've removed all error checks):
package main
import (
"github.com/jmoiron/sqlx"
_ "github.com/mattn/go-sqlite3"
"log"
"time"
)
type Post struct {
Id int64 `db:"post_id"`
Created time.Time `db:"created"`
}
func main() {
db, _ := sqlx.Connect("sqlite3", "post_db.db")
db.MustExec("DROP TABLE IF EXISTS posts; CREATE TABLE posts (post_id INT, created DATETIME);")
p1 := Post{Id: 1, Created: time.Now().UTC()}
p2 := Post{}
tx := db.MustBegin()
tx.NamedExec("INSERT INTO posts (post_id, created) VALUES (:post_id, :created)", &p1)
tx.Commit()
db.Get(&p2, "SELECT post_id, created FROM posts WHERE post_id = $1", p1.Id)
log.Println(p2.Created.Format("2006-01-02"))
}

Resources