I have a REST API application written in Go, which uses GORM as ORM. During refactoring of some parts, I wanted to move some common operations to an external function ApplyToDBQuery(query *gorm.DB), which takes a pointer to the DB query and modifies it, so that the modified query can be used later.
Example usage:
query = shared.DB.Debug()
req.ApplytoDBQuery(query)
if query.find(&data).Error != nil {...}
func (this *MyCustomRequest) ApplyToDBQuery(query *gorm.DB) {
query.Limit(...)
query.Offset(...)
query.Where(...)
}
I thought, that since I'm passing a pointer to the function, the original query should've been modified, but nothing really happened to the original query.
I've also tried passing a pointer to pointer ApplyToDBQuery(query **gorm.DB), returning the modified pointer ApplyToDBQuery(query *gorm.DB) *gorm.DB and out of lack of ideas, even a combination of these two - ApplyToDBQuery(query **gorm.DB) *gorm.DB
Gorm object clone itself for every operation, therefore the original pointed value is never changed.
You should return the latest version of gorm.DB:
return query.Limit(...).Offset(...).Where(...)
Change the method receiver to
func (this *MyCustomRequest) ApplyToDBQuery(query *gorm.DB) *gorm.DB {
return query.Limit(...).
Offset(...).
Where(...)
}
then use it as:
query = req.ApplytoDBQuery(query)
if query.find(&data).Error != nil {...}
The reason is already pointed by #R3v4n
You should go with what the previous two answers recommend, but, if for some reason, you have to apply changes to the passed in pointer, you can still do it "manually".
query = shared.DB.Debug()
req.ApplytoDBQuery(query)
if query.find(&data).Error != nil {...}
func (r *MyCustomRequest) ApplyToDBQuery(query *gorm.DB) {
q := query.Limit(...).Offset(...).Where(...)
*query = *q
}
As a side note, it is generally discouraged to use receiver names like this and self, instead the preferred way is to use a short, let's say 1 to 3 letters, abbreviation of the type's name.
For example:
func (r *Request) AddCookie(c *Cookie)
func (c *Client) Get(url string) (resp *Response, err error)
func (srv *Server) ListenAndServe() error
Complementing #R3v4n answer:
Use chaining
Use db.Scopes for code reusage instead of usual func calls.
It can be like this:
query = shared.DB.Debug()
if query.Scopes(req.ApplyToDBQuery).find(&data).Error != nil {
// handle error
}
func (this *MyCustomRequest) ApplyToDBQuery(query *gorm.DB) *gorm.DB {
return query.Where(...).Limit(...).Offset(...)
}
Related
I'm trying to understand various coding architectures by looking into various public codes. One of which is the mime/multipart implementation by the Go team.
The below snippet is what I've seen. https://cs.opensource.google/go/go/+/refs/tags/go1.19.3:src/mime/multipart/formdata.go;l=156
func (r *Reader) ReadForm(maxMemory int64) (*Form, error) {
return r.readForm(maxMemory)
}
func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
form := &Form{make(map[string][]string), make(map[string][]*FileHeader)}
defer func() {
if err != nil {
form.RemoveAll()
}
}()
// Reserve an additional 10 MB for non-file parts.
maxValueBy
...more code here
I've read through some stuff about SOLID, DRY, public/ private relationships so I can't say I know a lot of best practices/ common strategies.
Looking at the above, it looks to me like its a function that makes a private function public.
The only thing that comes to my mind is that its purely for documentation sake? But nothing concrete in my mind.
So what I'm struggling to understand here is what's the benefit of doing so?
Thank you all for taking the time to read this.
Any comments/ reading suggestions is very much appreciated.
It is for the sake of the documentation. The PR comment explains:
Named returned values should only be used on public funcs and methods when it contributes to the documentation.
Named return values should not be used if they're only saving the programmer a few lines of code inside the body of the function, especially if that means there's stutter in the documentation or it was only there so the programmer could use a naked return statement. (Naked returns should not be used except in very small functions)
This change is a manual audit & cleanup of public func signatures.
To hide the return value names, the original function
func (r *Reader) ReadForm(maxMemory int64) (f *Form, err error) {
⋮
}
was changed to
func (r *Reader) ReadForm(maxMemory int64) (*Form, error) {
return r.readForm(maxMemory)
}
func (r *Reader) readForm(maxMemory int64) (_ *Form, err error) {
⋮
}
The error return value name can not be eliminated because a
deferred function accesses the error return value.
Is it possible to convert go-pg query
err = db.Model(story).
Relation("Author").
Where("story.id = ?", story1.Id).
Select()
into plain SQL?
It would be helpful for debugging. So I could copy this plain SQL query and run in psql client as a string.
Probably there is some kind of package for this?
This is listed in the project's wiki:
How to view queries this library generates?
How to view queries this library generates?
You can setup query logger like this:
type dbLogger struct { }
func (d dbLogger) BeforeQuery(c context.Context, q *pg.QueryEvent) (context.Context, error) {
return c, nil
}
func (d dbLogger) AfterQuery(c context.Context, q *pg.QueryEvent) (context.Context, error) {
fmt.Println(q.FormattedQuery())
return c, nil
}
db := pg.Connect(&pg.Options{...})
db.AddQueryHook(dbLogger{})
Problem: There is no default parsing to RAW sql from ORM since v10.
Well, that's too late i guess. Maybe someone (like me) will face this problem in 2021. There is some steps how you could solve this:
[x] Read DOCs.
[x] Check all structs.
[x] Implement all methods.
Solving the problem
This solve is "forked" from this issue but i'll explain it step by step.
First of all we need to read some source code of go-pg hook.
As I said before: we need to check all structs from this doc. But we're lucky. There is only 1 struct!
// QueryEvent ...
type QueryEvent struct {
StartTime time.Time
DB orm.DB
Model interface{}
Query interface{}
Params []interface{}
fmtedQuery []byte
Result Result
Err error
Stash map[interface{}]interface{}
}
We don't really need to implement this struct completely.
But when you use db.AddQueryHook() (where db is ref on our DB connection and AddQueryHook() is method) AddQueryHook() wait's from you this interface:
type QueryHook interface {
BeforeQuery(context.Context, *QueryEvent) (context.Context, error)
AfterQuery(context.Context, *QueryEvent) error
}
So, we already read DOCs, checked structs. And what's next? Answer is pretty easy:
Implement all methods.
TBH, I thought that this is harder than it is.
To implement it you just need to create 2 methods of current (new empty) structure that implements functionality of methods above, like this:
Creating empty structure
type dbLogger struct{}
Adding methods from doc:
func (d dbLogger) BeforeQuery(c context.Context, q *pg.QueryEvent) (context.Context, error) {
return c, nil
}
func (d dbLogger) AfterQuery(c context.Context, q *pg.QueryEvent) error {
fq, _ := q.FormattedQuery()
fmt.Println(string(fq))
return nil
}
I hope this helps everyone who ever encounters this problem.
I've just been upgrading from go-pg v7 to v10 & had a problem where Query.AppendFormat() which is what I was using to get the RAW SQL had been removed.
After using the comments in this post for inspiration I managed extract it, using the code below
import (
"github.com/go-pg/pg/v10/orm"
)
func QueryToString(q *orm.Query) string {
value, _ := q.AppendQuery(orm.NewFormatter(), nil)
return string(value)
}
Hope this helps future viewers
//db your *pg.DB
// q your *orm.Query = db.Model(&yourModel).
qq := pg.QueryEvent{
DB: db,
Model: q.TableModel(),
Query: q,
}
fmt.Println(qq.FormattedQuery())
So in your case
q:= db.Model(story).
Relation("Author").
Where("story.id = ?", story1.Id)
fmt.Println("running SQL:")
qq := pg.QueryEvent{
DB: db,
Model: q.TableModel(),
Query: q,
}
fmt.Println(qq.FormattedQuery())
q.Select()
I'm coming at Golang from a Python background and I am trying to wrap my head around various new concepts.
One thing I have come across is this function in net.go:
func (c *conn) ok() bool { return c != nil && c.fd != nil }
This function is called by multiple net.go methods, e.g. conn.Read:
// Read implements the Conn Read method.
func (c *conn) Read(b []byte) (int, error) {
if !c.ok() {
return 0, syscall.EINVAL
}
I am trying to understand how the ok() method can be called on conn, despite the fact that ok() does not appear to be an interface of conn.
Certainly I do not seem to be able to call ok() from my client code:
func main() {
conn, err := net.Dial("tcp", "www.reddit.com:80")
if err != nil {
os.Exit(-1)
}
fmt.Println(&conn.ok())
}
Output:
./server.go:14:22: conn.ok undefined (type net.Conn has no field or method ok)
Any pointers appreciated...
From Go document :
An identifier may be exported to permit access to it from another
package. An identifier is exported if the first character of the
identifier's name is a Unicode upper case letter
So , ok function is not exported and you can't access it outside of net package.
Go does not use public/private keywords for visibility of an identifier. If the initial character is an upper case letter, the identifier is exported(public); otherwise it is not:
upper case initial letter: Name is visible to clients of package
otherwise: name (or _Name) is not visible to clients of package
There is no field or method like ok in net.Conn that what the error says and that is correct.
when you try to read and write into the conn , you would get err and number of bytes read or write it into the connection.
I have this query in postgres which queries 1 or n users based on the parameters passed:
select name, phone from clients where id in ('id1','id2')
Now when I try to use this at golang I'm having problems approaching how to pass this type of variable arguments to the statement.Query() function:
ids := []string{"0aa6c0c5-e44e-4187-b128-6ae4b2258df0", "606b0182-269f-469a-bb29-26da4fa0302b"}
rows, err := stmt.Query(ids...)
This throws error: Cannot use 'ids' (type []string) as type []interface{}
When I check in source code query it can receive many variables of type interface:
func (s *Stmt) Query(args ...interface{}) (*Rows, error) {
return s.QueryContext(context.Background(), args...)
}
If I do this manually it works:
rows, err := stmt.Query("0aa6c0c5-e44e-4187-b128-6ae4b2258df0", "606b0182-269f-469a-bb29-26da4fa0302b")
But of course I need the args to be 1 or many more, and dynamically generated.
I'm using Sqlx lib.
As we can see on the Query() method scheme and also from the error message, the method requires an argument in []interface{} type.
func (s *Stmt) Query(args ...interface{}) (*Rows, error) {
return s.QueryContext(context.Background(), args...)
}
In your code, the ids variable hold []string data. Change it to []interface{} so it'll meet Query() requirements, then it'll work.
ids := []interface{}{
"0aa6c0c5-e44e-4187-b128-6ae4b2258df0",
"606b0182-269f-469a-bb29-26da4fa0302b",
}
rows, err := stmt.Query(ids...)
I have the following structure
type Groups struct {
sync.Mutex
Names []string
}
and the following function
func NewGroups(names ...string) (Groups, error) {
// ...
return groups, nil
}
When I check for semantic errors with go vet, I am getting this warning:
NewGroups returns Lock by value: Groups
As go vet is shouting, it is not good. What problems can this code bring? How can I fix this?
You need to embed the sync.Mutex as a pointer:
type Groups struct {
*sync.Mutex
Names []strng
}
Addressing your comment on your question: In the article http://blog.golang.org/go-maps-in-action notice Gerrand is not returning the struct from a function but is using it right away, that is why he isn't using a pointer. In your case you are returning it, so you need a pointer so as not to make a copy of the Mutex.
Update: As #JimB points out, it may not be prudent to embed a pointer to sync.Mutex, it might be better to return a pointer to the outer struct and continue to embed the sync.Mutex as a value. Consider what you are trying to accomplish in your specific case.
Return a pointer *Groups instead.
Embedding the mutex pointer also works but has two disadvantages that require extra care from your side:
the zero value of the struct would have a nil mutex, so you must explicitly initialize it every time
func main() {
a, _ := NewGroups()
a.Lock() // panic: nil pointer dereference
}
func NewGroups(names ...string) (Groups, error) {
return Groups{/* whoops, mutex zero val is nil */ Names: names}, nil
}
assigning a struct value, or passing it as function arg, makes a copy so you also copy the mutex pointer, which then locks all copies. (This may be a legit use case in some particular circumstances, but most of the time it might not be what you want.)
func main() {
a, _ := NewGroups()
a.Lock()
lockShared(a)
fmt.Println("done")
}
func NewGroups(names ...string) (Groups, error) {
return Groups{Mutex: &sync.Mutex{}, Names: names}, nil
}
func lockShared(g Groups) {
g.Lock() // whoops, deadlock! the mutex pointer is the same
}
Keep your original struct and return pointers. You don't have to explicitly init the embedded mutex, and it's intuitive that the mutex is not shared with copies of your struct.
func NewGroups(names ...string) (*Groups, error) {
// ...
return &Groups{}, nil
}
Playground (with the failing examples): https://play.golang.org/p/CcdZYcrN4lm