I'm trying to get the users from a DB as follow,
var users []User
_, err := dbMap.Select(&users, "select id,username,acctstarttime,acctlastupdatedtime,acctstoptime from accounting order by id")
Here I'm using gorp. When there are null values present, this throws exception
Select failed sql: Scan error on column index 3: unsupported driver -> Scan pair: <nil> -> *string
How can I solve this issue?. Here I used gorp because of the ease of mapping the output to a struct array.
Make whatever acctstarttime maps to a pointer to the type instead of a value of the type.
if the col is null, the pointer will be nil.
that or you can use the sql.NullXXX types, but I usually don't like those since they make everything else weird.
Related
I have the following model. Each instance is part of a group which is only defined as the string GroupName here because the actual group is defined in a different service using a different database.
type Instance struct {
gorm.Model
UserID uint
Name string `gorm:"index:idx_name_and_group,unique"`
GroupName string `gorm:"index:idx_name_and_group,unique"`
StackName string
DeployLog string `gorm:"type:text"`
Preset bool
PresetID uint
}
I'd like to scan, the above model, into the following struct. Thus grouping instances why their group name.
type GroupWithInstances struct {
Name string
Instances []*model.Instance
}
I'm been trying my luck with the following gorm code
var result []GroupWithInstances
err := r.db.
Model(&model.Instance{}).
Where("group_name IN ?", names).
Where("preset = ?", presets).
Group("group_name").
Scan(&result).Error
indent, _ := json.MarshalIndent(result, "", " ")
log.Println(string(indent))
But I'm getting the following error
ERROR: column "instances.id" must appear in the GROUP BY clause or be used in an aggregate function (SQLSTATE 42803)
I'm not sure how to deal with that since I don't want to group by instances but rather their groups.
The error indicates that your RDBMS is in Full Group By Mode - cannot select field that isn't in group by clause or used in an aggregate function (SUM, AVG,...). There are 2 solutions:
Disable Full Group By mode. Example in MySQL
Modify the query
Even when we go with Solution 1, gorm will throw another error about relationship between GroupWithInstances and Instance.
So I think we should review the feature and go with Solution 1 - only select what is needed.
We are updating our project from v1 to v2.
When we try to update a row by providing only changed fields as a struct, it tries to set created_at column and returns an error. This was working back in v1. According to documentation, during update operations, fields with default values are ignored.
err := r.writeDB.Save(&Record{
Model: Model{
ID: 1,
},
Name: "AB",
}).Error
if err != nil {
return err
}
Generates the following SQL statement
[3.171ms] [rows:0] UPDATE `records` SET `created_at`='0000-00-00 00:00:00',`updated_at`='2020-11-12 15:38:36.285',`name`='AB' WHERE `id` = 1
Returns following error
Error 1292: Incorrect datetime value: '0000-00-00' for column
'created_at' at row 1
With these entities
type Model struct {
ID int `gorm:"primary_key,type:INT;not null;AUTO_INCREMENT"`
CreatedAt time.Time `gorm:"type:TIMESTAMP(6)"`
UpdatedAt time.Time `gorm:"type:TIMESTAMP(6)"`
}
type Record struct {
Model
Name string
Details string
}
There is DB.Omit which allows ignoring a column when executing an update query. But this requires a lot of refactoring in the codebase. Does the behavior changed or is there something missing?
This might help you. Change the structure field (or add to replace default gorm.Model field) like this:
CreatedAt time.Time `gorm:"<-:create"` // allow read and create, but don't update
This tag helps to save created data from update.
In Gorm v2 Save(..) writes all fields to the database. As the CreatedAt field is the zero value, this is written to the database.
For you code to work, you can use a map:
err := r.writeDB.Model(&Record{Model:Model{ID:1}}).Update("name","AB").Error
Another option is to not fill in the Record struct, but just point to the table and use a Where clause:
err := r.writeDB.Model(&Record{}).Where("id = ?", 1).Update("name","AB").Error
If you have multiple updates you can use Updates(...) with a map, see here: https://gorm.io/docs/update.html
I was able to work around this problem using Omit() before save. Like this:
result := r.db.Omit("created_at").Save(item)
It omits the CreatedAt from the resulting update query, and updates everything else.
I am just getting my feet wet with go-pg, and having some issues utilizing the JSONB capabilities of PostgreSQL DB.
I have defined my model structs as:
type Chemical struct {
Id int
ChemicalName string
ListingDetails []ListingDetail `sql:",array"`
ParentChemical *Chemical `pg:"fk_parent_chemical_id"`
ParentChemicalId int
}
type ListingDetail struct {
TypeOfToxicity string
ListingMechanism string
CasNo string
ListedDate time.Time
SafeHarborInfo string
}
In the Chemical struct - by using the tag sql:",array" on the ListingDetails field - my understanding is this should get mapped to a jsonb column in the destination table. Using go-pg's CreateTable - a chemicals table gets created with a listing_details jsonb column - so all is good in that regards.
However, I cant seem to update this field (in the db) successfully. When I initially create a chemical record in teh database - there are no ListingInfos - I go back up update them at a later time. Below is a snippet showing how I do that.
detail := models.ListingDetail{}
detail.TypeOfToxicity = strings.TrimSpace(row[1])
detail.ListingMechanism = strings.TrimSpace(row[2])
detail.CasNo = strings.TrimSpace(row[3])
detail.SafeHarborInfo = strings.TrimSpace(row[5])
chemical.ListingDetails = append(chemical.ListingDetails, detail)
err = configuration.Database.Update(&chemical)
if err != nil {
panic(err)
}
But I always get error message shown below. What am I doing wrong??
panic: ERROR #22P02 malformed array literal: "{{"TypeOfToxicity":"cancer","ListingMechanism":"AB","CasNo":"26148-68-5","ListedDate":"1990-01-01T00:00:00Z","SafeHarborInfo":"2"}}"
I have the following 2 structs with a many-2-many relationship.
type Message struct {
gorm.Model
Body string `tag:"body" schema:"body"`
Locations []Location `tag:"locations" gorm:"many2many:message_locations;"`
TimeSent time.Time `tag:"timesent"`
TimeReceived time.Time `tag:"timereceived"`
User User
}
type Location struct {
gorm.Model
PlaceID string `tag:"loc_id" gorm:"unique"`
Lat float64 `tag:"loc_lat"`
Lng float64 `tag:"loc_lng"`
}
If I create a Message with DB.Create(my_message), everything works fine : the Message is created in DB, along with a Location and the join message_locations table is filled with the respective IDs of the Message and the Location.
What I was expecting though was that if the Location already exists in DB (based on the place_id field, which is passed on), gorm would create the Message, retrieve the Location ID and populate message_locations.That's not what is happening.
Since the PlaceID must be unique, gorm finds that a duplicate key value violates unique constraint "locations_place_id_key" and aborts the transaction.
If on the other hand I make the PlaceID not unique, gorm creates the message alright, with the association, but then that creates another, duplicate entry for the Location.
I can test if the location already exists before trying to save the message:
existsLoc := Location{}
DB.Where("place_id = ?", mssg.Locations[0].PlaceID).First(&existsLoc)
then if true switch the association off:
DB.Set("gorm:save_associations", false).Create(mssg)
DB.Create(mssg)
The message is saved without gorm complaining, but then message_locations is not filled.
I could fill it "manually" since I've retrieved the Location ID when testing for its existence, but it seems to me it kind of defeats the purpose of using gorm in the first place.
I'm not sure what the right way to proceed might be. I might be missing something obvious, I suspect maybe something's wrong with the way I declared my structs? Hints welcome.
UPDATE 2016/03/25
I ended up doing the following, which I'm pretty sure is not optimal. If you have a better idea, please chime in.
After testing if the location already exists and it does:
// in a transaction
tx := DB.Begin()
// create the message with transaction disabled
if errMssgCreate := tx.Set("gorm:save_associations", false).Create(mssg).Error; errMssgCreate != nil {
tx.Rollback()
log.Println(errMssgCreate)
}
// then create the association with existing location
if errMssgLocCreate := tx.Model(&mssg).Association("Locations").Replace(&existLoc).Error; errMssgLocCreate != nil {
tx.Rollback()
log.Println(errMssgLocCreate)
}
tx.Commit()
In my situation I was using a UUID for the ID. I coded a BeforeCreate hook to generate the uuid. When saving a new association, there was no need for the beforeCreate hook to create a new ID, but it did so anyway (that feels a bit like it could be a bug?).
Note that it did this even when using "association" mode to append a new relationship. The behaviour was not limited to when calling Create with a nested association.
It took me several hours to debug this because when I inspected the contents of the associated records they matched exactly the instances that I had just created.
In other words:
I made a bunch of Foo
I made some Bar, and tried to attach just certain Foo to each
The Foo in the Bar relationship had the same reference as the objects I had just created.
Removing the beforeCreate hook makes the code behave like I'd expect. And happily I was already in the habit of manually making a uuid whenever needed instead of relying on it so it didn't hurt me to remove it.
I've pasted a minimally reproducible example at https://pastebin.com/wV4h38Qz
package models
import (
"github.com/google/uuid"
"gorm.io/gorm"
"time"
)
type Model struct {
ID uuid.UUID `gorm:"type:char(36);primary_key"`
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt gorm.DeletedAt
}
// BeforeCreate will set a UUID rather than numeric ID.
func (m *Model) BeforeCreate(tx *gorm.DB) (err error) {
m.ID = uuid.New()
return
}
gorm:"many2many:message_locations;save_association:false"
Is getting closer to what you would like to have. You need to place it in your struct definition for Message. The field is then assumed to exist in the db and only the associations table will be populated with data.
Using gorp how can one insert multiple records efficiently? i.e instead of inserting one at a time, is there a batch insert?
var User struct {
Name string
Email string
Phone string
}
var users []Users
users = buildUsers()
dbMap.Insert(users...) //this fails compilation
//I am forced to loop over users and insert one user at a time. Error Handling omitted for brevity
Is there a better mechanism with gorp? Driver is MySQL.
As I found out on some other resource the reason this doesn't work is that interface{} and User{} do not have the same layout in memory, therefore their slices aren't of compatible types. Suggested solution was to convert []User{} into []interface{} in for loop, like shown here: https://golang.org/doc/faq#convert_slice_of_interface
There is still on caveat: you need to use pointers for DbMap.Insert()function.
Here's how I solved it:
s := make([]interface{}, len(users))
for i, v := range users {
s[i] = &v
}
err := dbMap.Insert(s...)
Note that &v is important, otherwise Insert will complain about non-pointers.
It doesn't look like gorp has anything that gives a wrapper for either raw SQL or multi value inserts (which is always SQL dialect dependent).
Are you worried about speed or transactions? If not, I would just do the inserts in a for loop.