I am trying to build a generic CrudRepository struct using Gorm for my api.
I know generics are coming to GoLang in version 2 but I try to build this lib using reflection or any other lib.
In my CrudRepository:
func (repository *BaseRepository) find(result interface{}, pageSize int, page int) error {
if page < 1 {
return errors.ExceedsMinimumInt("page", "", 0, true, nil)
}
offset := (page - 1) * pageSize
ent := reflect.Zero(reflect.TypeOf(result))
repository.db = repository.db.Limit(pageSize).Offset(offset)
err := repository.db.Find(&ent).Error
result = ent
if err != nil {
return err
}
return nil
}
And calling this method sth like:
func List(){
var entityList []MyEntity
find(entityList, 1, 10)
}
I think, I cannot pass any interface reference into Gorm.db.Find() method
Is there any other way to succeed?
Use a pointer of a slice as input argument of custom find method.
func (repository *BaseRepository) find(result interface{}, pageSize int, page int) error {
if page < 1 {
return errors.ExceedsMinimumInt("page", "", 0, true, nil)
}
if reflect.TypeOf(result).Kind() != reflect.Slice { 👈 check ❗️
return errors.New("`result` is not a slice")
}
offset := (page - 1) * pageSize
db = db.Limit(pageSize).Offset(offset)
if err := db.Find(result).Error; err != nil {
return err
}
return nil
}
usage 👇🏻
var entityList []MyEntity
err := find(&entityList, 10, 1)
Also you have to check input argument (result), because db.Find isn't fit to find single strut 👇🏻 (Retrieving a single object)
If you want to avoid the ErrRecordNotFound error, you could use Find
like db.Limit(1).Find(&user), the Find method accepts both struct and
slice data
For example (Book table is empty):
b := Book{}
rowsAffectedQuantity := db.Find(&b).RowsAffected // 👈 0
err = db.Find(&b).Error // 👈 nil
Related
I have the concept of Context which is a map that can hold any structure. Basically, I want to create a generic getter that adddressably 'populates' the destination interface (similarly to how json decoding works).
Here's an example of how I want this to work:
type Context map[string]interface{}
// Random struct that will be saved in the context
type Step struct {
Name string
}
func main() {
stepA := &Step{Name: "Cool Name"}
c := Context{}
c["stepA"] = stepA
var stepB *Step
err := c.Get("stepA", stepB)
if err != nil {
panic(err)
}
fmt.Println(stepB.Name) // Cool Name
stepB.Name = "CoolName2"
fmt.Println(stepA.Name) // I want to say: CoolName2
}
func (c Context) Get(stepId string, dest interface{}) error {
context, ok := c[stepId]
if !ok {
return nil
}
destinationValue := reflect.ValueOf(dest)
contextValue := reflect.ValueOf(context)
destinationValue.Set(contextValue) // Errors here
return nil
}
I leaned towards using reflect, but maybe I don't need it? - so opened to other suggestions (except for generics as that complicates other matters) I'm getting the following error with the above:
panic: reflect: reflect.Value.Set using unaddressable value
You can test it here.
The argument passed to Get must be a pointer type whose element type is identical to the type in the context map. So if the value in the context map is of type *Step, then the argument's type must be **Step. Also the passed in argument cannot be nil, it can be a pointer to nil, but it itself cannot be nil.
So in your case you should do:
var stepB *Step
err := c.Get("stepA", &stepB) // pass pointer-to-pointer
if err != nil {
panic(err)
}
And the Get method, fixed up a bit:
func (c Context) Get(stepId string, dest interface{}) error {
context, ok := c[stepId]
if !ok {
return nil
}
dv := reflect.ValueOf(dest)
if dv.Kind() != reflect.Ptr || dv.IsNil() {
return errors.New("dest must be non-nil pointer")
}
dv = dv.Elem()
cv := reflect.ValueOf(context)
if dv.Type() != cv.Type() {
return errors.New("dest type does not match context value type")
}
dv.Set(cv)
return nil
}
https://go.dev/play/p/OECttqp1aVg
I'm trying to seed my Postgres database as functionally. In my case, SeedSchema() function can take any type struct. So I define a interface and create functions to my structs which will seed. I tried with generics and without.
When I unmarshall any json array from file as byte array, json.Unmarshall method change my tempMember member of struct. Exp, models.Term to map[string]interface{}. I've used unmarshall before this function and I've not seen like this situation.
Here is my SeedSchema() function:
func (db *Database) SeedSchema(models ...globals.Seeder[any]) error {
var (
subjects []globals.Seeder[any]
fileByte []byte
err error
// tempMember any
)
if len(models) == 0 {
subjects = seederModelList
} else {
subjects = models
}
for _, model := range subjects {
fileName, tempMember := model.Seed()
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //1
if fileByte, err = os.ReadFile("db/seeds/" + fileName); err != nil {
fmt.Println(err)
return err
}
if err = json.Unmarshal(fileByte, &tempMember); err != nil {
fmt.Println(err)
return err
}
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //2
}
return nil
}
First print returns []models.AirportCodes and the second []interface {}.
Here is my interface and model:
func (AirportCodes) Seed() (string, any) {
return "airport_codes.json", []AirportCodes{}
}
type Seeder[T any] interface {
Seed() (string, T)
// Seed(*gorm.DB) error
TableName() string
}
seederModelList = []globals.Seeder[any]{
m.AirportCodes{},
m.Term{},
}
After a few weeks, I have looking for solve this problem and look unmarshaler interfaces and examples. Then Like what icza said, I started to look over the my code that convention between types and I solved like this. If you guys have better answer than mine, please add answer.
Data:
[
{
"id":1,
"name":"Term 1",
"content": [
"a1",
"a2",
"a3"
]
}
]
Result:
[{ID:1 Name:Term 1 Content:[a1 a2 a3]}]
UnmarshalJSON Function:
func (term *Term) UnmarshalJSON(data []byte) error {
tempMap := map[string]interface{}{}
if err := json.Unmarshal(data, &tempMap); err != nil {
return err
}
*term = Term{
Name: tempMap["name"].(string),
}
if tempMap["content"] != nil {
for _, v := range tempMap["content"].([]interface{}) {
(*term).Content = append((term).Content, v.(string))
}
}
return nil
}
Thank you for comments.
I'm stuck with an obvious operation: retrieving multiple rows using gorm.Find() method.
(resolver.go)
package resolver
type Root struct {
DB *gorm.DB
}
func (r *Root) Users(ctx context.Context) (*[]*UserResolver, error) {
var userRxs []*UserResolver
var users []model.User
// debug-start
// This is to prove r.DB is allocated and working
// It will print {2 alice#mail.com} in the console
var user model.User
r.DB.Find(&user)
log.Println(user)
// debug-end
if err := r.DB.Find(&users); err != nil { // <-- not working
log.Fatal(err)
}
for _, user := range users {
userRxs = append(userRxs, &UserResolver{user})
log.Println(user)
}
return &userRxs, nil
}
(model.go)
package model
type User struct {
ID graphql.ID `gorm:"primary_key"`
Email string `gorm:"unique;not null"`
}
The mysql table is filled with 2 values. Here is the content in json style:
{
{ Email: bob#mail.com },
{ Email: alice#mail.com },
}
This is the result when I run the program:
2020/05/13 12:23:17 Listening for requests on :8000
2020/05/13 12:23:22 {2 alice#mail.com}
2020/05/13 12:23:22 &{{{0 0} 0 0 0 0} 0xc0004cee40 <nil> 2 0xc00031e3c0 false 0 {0xc00035bea0} 0xc0004b3080 {{0 0} {<nil>} map[] 0} 0xc000408340 <nil> 0xc0004cee60 false <nil>}
What is wrong with my code? It seems from all the tuto/so/etc.. sources that I'm correctly defining a slice var and passing it to the Find() function..
if err := r.DB.Find(&users); err != nil { // <-- not working
log.Fatal(err)
}
Probably you forgot to mention Error property and returned object in this case is not nil for sure (please mention that Find returns not error interface in this case)
Please try something like that
if err := r.DB.Find(&users).Error; err != nil {
log.Fatal(err)
}
Hope it helps
You need to use a slice of pointers:
users := make([]*model.User, 0, 2)
if err := r.DB.Find(&users).Error; err != nil {
log.Fatal(err)
}
I am trying to code some generic methods (CRUD approach) to share it between my services. The following example is a GetAll() method that returns all the documents present in my collection:
func GetAll(out interface{}) error {
// mongodb operations
// iterate through all documents
for cursor.Next(ctx) {
var item interface{}
// decode the document
if err := cursor.Decode(&item); err != nil {
return err
}
(*out) = append((*out), item)
// arrays.AppendToArray(out, item) // Read below :)
}
return nil // if no error
}
I also tried with some reflection, but then:
package arrays
import "reflect"
func AppendToArray(slicePtrInterface interface{}, item interface{}) {
// enter `reflect`-land
slicePtrValue := reflect.ValueOf(slicePtrInterface)
// get the type
slicePtrType := slicePtrValue.Type()
// navigate from `*[]T` to `T`
_ = slicePtrType.Elem().Elem() // crashes if input type not `*[]T`
// we'll need this to Append() to
sliceValue := reflect.Indirect(slicePtrValue)
// append requested number of zeroes
sliceValue.Set(reflect.Append(sliceValue, reflect.ValueOf(item)))
}
panic: reflect.Set: value of type primitive.D is not assignable to type *mongodb.Test [recovered]
panic: reflect.Set: value of type primitive.D is not assignable to type *mongodb.Test
What I would like is to get the same approach as cursor.Decode(&item) (you can see above)
Here's how to do it:
// GetAll decodes the cursor c to slicep where slicep is a
// pointer to a slice of pointers to values.
func GetAll(ctx context.Context, c *Cursor, slicep interface{}) error {
// Get the slice. Call Elem() because arg is pointer to the slice.
slicev := reflect.ValueOf(slicep).Elem()
// Get value type. First call to Elem() gets slice
// element type. Second call to Elem() dereferences
// the pointer type.
valuet := slicev.Type().Elem().Elem()
// Iterate through the cursor...
for c.Next(ctx) {
// Create new value.
valuep := reflect.New(valuet)
// Decode to that value.
if err := c.Decode(valuep.Interface()); err != nil {
return err
}
// Append value pointer to slice.
slicev.Set(reflect.Append(slicev, valuep))
}
return c.Err()
}
Call it like this:
var data []*T
err := GetAll(ctx, c, &data)
if err != nil {
// handle error
}
Run it on the Go Playground.
Here's a generalization of the code to work with non-pointer slice elements:
func GetAll(ctx context.Context, c *Cursor, slicep interface{}) error {
slicev := reflect.ValueOf(slicep).Elem()
valuet := slicev.Type().Elem()
isPtr := valuet.Kind() == reflect.Ptr
if isPtr {
valuet = valuet.Elem()
}
for c.Next(ctx) {
valuep := reflect.New(valuet)
if err := c.Decode(valuep.Interface()); err != nil {
return err
}
if !isPtr {
valuep = valuep.Elem()
}
slicev.Set(reflect.Append(slicev, valuep))
}
return c.Err()
}
I have created an object mapping in Go that is not relational, it is very simple.
I have several structs that looks like this:
type Message struct {
Id int64
Message string
ReplyTo sql.NullInt64 `db:"reply_to"`
FromId int64 `db:"from_id"`
ToId int64 `db:"to_id"`
IsActive bool `db:"is_active"`
SentTime int64 `db:"sent_time"`
IsViewed bool `db:"is_viewed"`
Method string `db:"-"`
AppendTo int64 `db:"-"`
}
To create a new message I just run this function:
func New() *Message {
return &Message{
IsActive: true,
SentTime: time.Now().Unix(),
Method: "new",
}
}
And then I have a message_crud.go file for this struct that looks like this:
To find a message by a unique column (for example by id) I run this function:
func ByUnique(column string, value interface{}) (*Message, error) {
query := fmt.Sprintf(`
SELECT *
FROM message
WHERE %s = ?
LIMIT 1;
`, column)
message := &Message{}
err := sql.DB.QueryRowx(query, value).StructScan(message)
if err != nil {
return nil, err
}
return message, nil
}
And to save a message (insert or update in the database) I run this method:
func (this *Message) save() error {
s := ""
if this.Id == 0 {
s = "INSERT INTO message SET %s;"
} else {
s = "UPDATE message SET %s WHERE id=:id;"
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(this))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(*this)
if err != nil {
return err
}
if this.Id == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
this.Id = lastId
}
return nil
}
The sql.PlaceholderPairs() function looks like this:
func PlaceholderPairs(i interface{}) string {
s := ""
val := reflect.ValueOf(i).Elem()
count := val.NumField()
for i := 0; i < count; i++ {
typeField := val.Type().Field(i)
tag := typeField.Tag
fname := strings.ToLower(typeField.Name)
if fname == "id" {
continue
}
if t := tag.Get("db"); t == "-" {
continue
} else if t != "" {
s += t + "=:" + t
} else {
s += fname + "=:" + fname
}
s += ", "
}
s = s[:len(s)-2]
return s
}
But every time I create a new struct, for example a User struct I have to copy paste the "crud section" above and create a user_crud.go file and replace the words "Message" with "User", and the words "message" with "user". I repeat alot of code and it is not very dry. Is there something I could do to not repeat this code for things I would reuse? I always have a save() method, and always have a function ByUnique() where I can return a struct and search by a unique column.
In PHP this was easy because PHP is not statically typed.
Is this possible to do in Go?
Your ByUnique is almost generic already. Just pull out the piece that varies (the table and destination):
func ByUnique(table string, column string, value interface{}, dest interface{}) error {
query := fmt.Sprintf(`
SELECT *
FROM %s
WHERE %s = ?
LIMIT 1;
`, table, column)
return sql.DB.QueryRowx(query, value).StructScan(dest)
}
func ByUniqueMessage(column string, value interface{}) (*Message, error) {
message := &Message{}
if err := ByUnique("message", column, value, &message); err != nil {
return nil, err
}
return message, error
}
Your save is very similar. You just need to make a generic save function along the lines of:
func Save(table string, identifier int64, source interface{}) { ... }
Then inside of (*Message)save, you'd just call the general Save() function. Looks pretty straightforward.
Side notes: do not use this as the name of the object inside a method. See the link from #OneOfOne for more on that. And do not get obsessed about DRY. It is not a goal in itself. Go focuses on code being simple, clear, and reliable. Do not create something complicated and fragile just to avoid typing a simple line of error handling. This doesn't mean that you shouldn't extract duplicated code. It just means that in Go it is usually better to repeat simple code a little bit rather than create complicated code to avoid it.
EDIT: If you want to implement Save using an interface, that's no problem. Just create an Identifier interface.
type Ider interface {
Id() int64
SetId(newId int64)
}
func (msg *Message) Id() int64 {
return msg.Id
}
func (msg *Message) SetId(newId int64) {
msg.Id = newId
}
func Save(table string, source Ider) error {
s := ""
if source.Id() == 0 {
s = fmt.Sprintf("INSERT INTO %s SET %%s;", table)
} else {
s = fmt.Sprintf("UPDATE %s SET %%s WHERE id=:id;", table)
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(source))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(source)
if err != nil {
return err
}
if source.Id() == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
source.SetId(lastId)
}
return nil
}
func (msg *Message) save() error {
return Save("message", msg)
}
The one piece that might blow up with this is the call to Exec. I don't know what package you're using, and it's possible that Exec won't work correctly if you pass it an interface rather than the actual struct, but it probably will work. That said, I'd probably just pass the identifier rather than adding this overhead.
You probably want to use an ORM.
They eliminate a lot of the boilerplate code you're describing.
See this question for "What is an ORM?"
Here is a list of ORMs for go: https://github.com/avelino/awesome-go#orm
I have never used one myself, so I can't recommend any. The main reason is that an ORM takes a lot of control from the developer and introduces a non-negligible performance overhead. You need to see for yourself if they fit your use-case and/or if you are comfortable with the "magic" that's going on in those libraries.
I don't recommend doing this, i personally would prefer being explicit about scanning into structs and creating queries.
But if you really want to stick to reflection you could do:
func ByUnique(obj interface{}, column string, value interface{}) error {
// ...
return sql.DB.QueryRowx(query, value).StructScan(obj)
}
// Call with
message := &Message{}
ByUnique(message, ...)
And for your save:
type Identifiable interface {
Id() int64
}
// Implement Identifiable for message, etc.
func Save(obj Identifiable) error {
// ...
}
// Call with
Save(message)
The approach i use and would recommend to you:
type Redirect struct {
ID string
URL string
CreatedAt time.Time
}
func FindByID(db *sql.DB, id string) (*Redirect, error) {
var redirect Redirect
err := db.QueryRow(
`SELECT "id", "url", "created_at" FROM "redirect" WHERE "id" = $1`, id).
Scan(&redirect.ID, &redirect.URL, &redirect.CreatedAt)
switch {
case err == sql.ErrNoRows:
return nil, nil
case err != nil:
return nil, err
}
return &redirect, nil
}
func Save(db *sql.DB, redirect *Redirect) error {
redirect.CreatedAt = time.Now()
_, err := db.Exec(
`INSERT INTO "redirect" ("id", "url", "created_at") VALUES ($1, $2, $3)`,
redirect.ID, redirect.URL, redirect.CreatedAt)
return err
}
This has the advantage of using the type system and mapping only things it should actually map.