I use go-pg library and specify row in table "unit"
type UnitModel struct {
Id int
Name string
TableName struct{} `sql:"unit"`
}
but table unit contains more then 2 fields and when i call
var unit UnitModel
err := db.Model(&unit).Where("id = ?", id).Select()
get error "pg: can't find column alter_name in model".
How specify ignore other fields in table "unit"?
Read go-pg manual. There's an example, for your case is:
err := db.Model(&unit).Column("id", "name").Where("id = ?", id).Select()
In version 4.8.10 go-pg this bug fixed. If db table contain fields which not exist in structure error will not occur.
Related
I need to store very large and high precision numbers with gORM, and using a pgtype.Numeric seems like the best bet. However, I cannot because I get an error: sql: Scan error on column index 4, name "foo": cannot scan int64
My model looks something like this:
type Model struct {
gorm.Model
Foo *pgtype.Numeric `gorm:"not null"`
}
Not sure if using pgtype.Numeric is the best (that's what i've seen everyone else use), or I'm doing something wrong. Thanks!
The code that caused the error:
package main
import (
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"math/big"
"github.com/jackc/pgtype"
)
type Model struct {
gorm.Model
Foo *pgtype.Numeric `gorm:"not null"`
}
func main() {
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
db.AutoMigrate(&Model{})
// Create
db.Create(&Model{Foo: &pgtype.Numeric{Int: big.NewInt(10000000), Status: pgtype.Present}})
var m Model
db.First(&m) // this line causes the error
}
Sqlite3 does not support big integer so there is no way you can accomplish that directly. I run the code and foo column is create as:
`foo` numeric NOT NULL
Which in sqlite https://www.sqlite.org/datatype3.html means
A column with NUMERIC affinity may contain values using all five storage classes... If the TEXT value is a well-formed integer literal that is too large to fit in a 64-bit signed integer, it is converted to REAL.
So your big int will turn into float64. Good thing it paniced instead of losing accuracy silently.
What you can do is convert the big int to string or bytes first and store that.
When debugging the sql.Scanner interface for database deserialization, it is noticeable that the value from the database arrives either as int64 or float64. This then leads to the corresponding error message.
A possible solution is to use a text data type in the database, by adding the type text to the field tag:
`gorm: "type:text;"`
Using the github.com/shopspring/decimal package, you can conveniently create a decimal number using the NewString function.
The adapted code to insert the data:
num, err := decimal.NewFromString("123456789012345.12345678901")
if err != nil {
panic(err)
}
db.Create(&Model{Foo: &num})
The model structure might then look something like this:
type Model struct {
gorm.Model
Foo *decimal.Decimal `gorm: "not null;type:text;"`
}
This would result in the following schema:
Test
If one inserts a breakpoint in decimal.Scan, one can see that the value comes from the database as expected as a string, resulting in the creation of a decimal with NewFromString (see Decimal's scan method).
If you add this line of code to the end of the main function
fmt.Println(m.Foo)
it would result in the following output in the debug console:
123456789012345.12345678901
Complete Program
Your complete program, slightly adapted to the above points, would then look something like this:
package main
import (
"fmt"
"github.com/shopspring/decimal"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
type Model struct {
gorm.Model
Foo *decimal.Decimal `gorm:"not null;type:text;"`
}
func main() {
db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
if err != nil {
panic("failed to connect database")
}
// Migrate the schema
db.AutoMigrate(&Model{})
// Create
num, err := decimal.NewFromString("123456789012345.12345678901")
if err != nil {
panic(err)
}
db.Create(&Model{Foo: &num})
var m Model
db.First(&m)
fmt.Println(m.Foo)
}
pgtype.Numeric and SQLite
If a PostgreSQL database is used, gorm can be used together with pgtype.Numeric to handle decimal numbers like 123456789012345.12345678901. You just need to use the numeric data type on the Postgres side with the appropriate desired precision (e.g. numeric(50,15)).
After all, this is exactly what pgtype is for, see the pgtype readme where it says:
pgtype is the type system underlying the https://github.com/jackc/pgx PostgreSQL driver.
However, if you use a text data type in SQLite for the reasons mentioned above, pgtype.Numeric will not work with SQLite. An attempt with the above number writes 12345678901234512345678901e-11 to the DB and when reading it out the following error occurs:
sql: Scan error on column index 4, name "foo": 12345678901234512345678901e-11 is not a number
I am developing an API using Go which connects to MySQL database for some query execution. Am using GORM for database operations. But am stuck at printing the SELECT query output for the tables which I don't have the column names.
My use case is that, I need to run the query on multiple tables where I don't have an idea about what their column names and types are. And so I cannot pre-define a struct for all the current and future tables which might get added.
Is there a way to print/save the SELECT query output without a pre-defined struct ?
I tried do some using empty struct but it didn't help me.
P.S: Am a beginner in Go
type Testing struct{}
var test Testing
dsn := fmt.Sprintf("%v:%v#tcp(%v:%v)/%v", myds.DBuser, myds.DBpassword, myds.DBhost, myds.DBport, myds.DBname)
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println(err)
}
tx := db.Raw(query).Scan(&test)
if tx.Error != nil {
fmt.Println(tx.Error)
}
fmt.Println(test)
You can use an anonymous struct
Let's say you have a struct:
type User struct{
FirstName string
LastName string
}
Query:
SELECT CONCAT(first_name,last_name) AS full_name from users;
Notice the new column full_name
you can simply do
var fullName = struct{FullName string}{}
Notice how I use pascal case & FullName has to be the field name
A capital letter in between will represent a _
Field is public so it can be accessed outside.
full_name(query) = FullName(field)
pass this fullName object as a bucket to your Scan and it should work.
db.Raw(query).Scan(&fullName)
EDIT:
Your query will have some result right?
Let me assume that you have
column_one,column_two... column_n
Now, to get the data from all the columns or selected ones if you want, you simply have to define fields (in anonymous struct) with specific names. In our case:
struct{ColumnOne,ColumnTwo,..ColumnN interface{}}{}
P.S. I have used interface{}, you can use types depending on the data your column returns.
It worked for me by using a map type with interface. This helped me to save the SELECT query results without pre-defined struct or the column names.
dsn := fmt.Sprintf("%v:%v#tcp(%v:%v)/%v", myds.DBuser, myds.DBpassword, myds.DBhost, myds.DBport, myds.DBname)
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println(err)
}
var result []map[string]interface{}
tx := db.Raw(query).Scan(&result)
if tx.Error != nil {
fmt.Println(tx.Error)
return
}
bytes, _ := json.Marshal(result)
fmt.Println(string(bytes))
I'm working on a golang application in which I need to fetch last record from the table so I'm not able to do it. I have models mention below:-
type SQLTransaction struct {
Id int `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Version uint64 `protobuf:"varint,1,opt,name=version,proto3" json:"version,omitempty" gorm:"primaryKey"`
Hash string `protobuf:"bytes,3,opt,name=hash,proto3" json:"hash,omitempty"`
VmStatusId int `protobuf:"bytes,6,opt,name=vm_status_id,proto3" json:"vm_status_id,omitempty"`
}
Here is my gorm function in which I want to fetch the last record
func (mgr *manager) GetLastTxn() (int, error) {
var versionId int
resp := mgr.connection.Table("transactions").Last("", "version")
return versionId, resp.Error
}
Error:-
model value required
[0.053ms] [rows:0] SELECT * FROM `transactions` WHERE version ORDER BY `transactions`. DESC LIMIT 1
0 model value required
How can I achieve it please any help. Thanks in advance.
There are a couple of issues in your example that should be handled.
First, the error model value required suggests that the Last method needs a model to store the result. If you want the entire record with all the data you can do it like this:
var lastVersion SQLTransaction
resp := mgr.connection.Table("transactions").Last(&lastVersion)
If you only want the ID of the record, you do something like this:
var lastVersion struct {
ID int
}
resp := mgr.connection.Table("transactions").Last(&lastVersion)
Next, you don't need Last("", "version") because it doesn't do what you think it does. The first parameter is a pointer to your resulting object (which for all go-gorm methods always should be an object or a slice). The second one is a condition for the WHERE clause. So the correct syntax, if you don't need an additional WHERE clause, is:
resp := mgr.connection.Table("transactions").Last(&lastVersion)
If you look at the code of the Last method, it considers the primary key when it executes the method, so you don't even need any additional parameters or conditions.
My question in a nutshell: can I use sqlx's StructScan to populate two embedded structs with values sourced from the same SQL table joined twice?
The help files to the useful sqlx package state this:
A StructScan will set an id column result in Person.AutoIncr.ID, also accessible as Person.ID. To avoid confusion, it's suggested that you use AS to create column aliases in your SQL instead.
Supposed I have this SQL query (parent-child, people to phones):
func getSQL() string {
return `SELECT *
FROM person
LEFT JOIN phones AS Phone1 ON Phone1.phone_id = person_phoneID1
LEFT JOIN phones AS Phone2 ON Phone2.phone_id = person_phoneID2
WHERE people_id = 1;`
}
Using sqlx and StructScan, I'd like to populate a struct full of embedded structs, something like this:
//Struct with embedded structs
type personHelper struct{
Person
Phone1 //Should I use the same name as SQL table alias?
Phone2
}
type Phone1 struct {
Phone //Underlying struct
}
type Phone2 struct{
Phone
}
//Base structs, with tags to match up fields
type Person struct{
ID `db:"person_id"`
Name `db:"person_name"`
Phone1 `db:"person_phoneID1"`
Phone2 `db:"person_phoneID2"`
}
type Phone struct{
ID int64 `db:"phone_id"`
Number string `db:"phone_no"`
//etc.
}
I might have a function something like this:
func getPeople(){
parseRows := func(rows *sqlx.Rows) {
for rows.Next() {
var ph personHelper
err := rows.StructScan(&ph)
if err != nil{
//etc.
}
}
}
sql := getSQL()
sqlutils.GetRows(parseRows, sql)//GetRows executes the SQL query and returns rows for processing
}
I can populate one phone number, but not both. I'm not sure whether I'm understanding the aliasing instructions correctly.
I'd appreciate any insights.
Thanks.
I am wondering how the correct approach is to load related fields in beego.
The doc explains it like this:
type User struct {
Id int
Name string
Posts []*Post `orm:"reverse(many)"`
}
user := User{Id: 1}
err := dORM.Read(&user)
num, err := dORM.LoadRelated(&user, "Posts")
This makes sense as long as I only query one record. What is the correct way to fetch related fields when I query all users?
A possible solution would be like this:
var users []*User
o.QueryTable(new(User)).All(&users)
for _, user := range users {
o.LoadRelated(controlCategory, "Posts")
}
However, this means I have to loop everytime over the complete list and make for every record a DB query to load all records.
Any suggestions? Thanks!