I have been doing some golang programming and usually has been lot of fun. Now I have this code I need to port from C# to go and it is just not working.
The idea is to fill a tree of employees from database but inner slices are not being filled up on each call.
Better to write the code here
func (db *DalBase) TitleAllChildren(tx *gorp.Transaction) (items []Title, err error) {
var dbChildren []entities.Title
_, err = tx.Select(&dbChildren, "select * from title where idparent is null order by name")
if err != nil {
return
}
items = make([]Title, 0)
for i := range dbChildren {
currItem := &dbChildren[i]
item := &Title{Id: currItem.Id, Name: currItem.Name}
err = db.TitleChildrenRecursive(tx, item)
if err != nil {
return
}
items = append(items, *item)
}
return
}
func (db *DalBase) TitleChildrenRecursive(tx *gorp.Transaction, u *Title) (err error) {
var dbChildren []entities.Title
_, err = tx.Select(&dbChildren, "select * from title where idparent = $1 order by name", u.Id)
if err != nil {
return
}
if len(dbChildren) != 0 {
u.Items = make([]Title, 0)
for i := range dbChildren {
currItem := &dbChildren[i]
item := &Title{Id: currItem.Id, Name: currItem.Name}
err = db.TitleChildrenRecursive(tx, item)
if err != nil {
return
}
u.Items = append(item.Items, *item)
}
}
return
}
I have logged the values on each recursive call and they are being filled inside the function, but when it bubbles up to the parent, slices are empty.
I wouldn't like to use pointers to slices, is it possible to implement?
Edit: here is the struct I am trying to fill
type Title struct {
Id string `json:"id"`
Name string `json:"name"`
Items []Title `json:"items"`
}
You won't need a pointer to a slice, as long you are passing around a pointer to the struct which contains your slice.
Each time you call TitleChildrenRecursive, you're replacing your slice with a new one before anything is appended:
u.Items = make([]Title, 0)
There's no need to make a new slice, since append works correctly with a nil slice.
You should also change []Title to []*Title, so that if any append operations happen to children items after they are added to the slice, it's reflected throughout the tree.
Related
I'm working on a resolver function for a GraphQL query for a BE I'm writing in Go. In the resolver, I have user data that I want to update, using an input value containing several possible update properties.
In JavaScript, this can be done quickly through destructuring (pseudo):
const mergedObj = {...oldProps, ...newProps}
For now, my resolver function looks like this (using gqlgen for GraphQL Go resolvers):
func (r *mutationResolver) ModifyUser(ctx context.Context, input *model.ModifyUserInput) (*model.User, error) {
id := input.ID
us, ok := r.Resolver.UserStore[id]
if !ok {
return nil, fmt.Errorf("not found")
}
if input.FirstName != nil {
us.FirstName = *input.FirstName
}
if input.LastName != nil {
us.LastName = *input.LastName
}
if input.ProfileImage != nil {
us.ProfileImage = input.ProfileImage
}
if input.Password != nil {
us.Password = *input.Password
}
if input.Email != nil {
us.Email = *input.Email
}
if input.InTomorrow != nil {
us.InTomorrow = input.InTomorrow
}
if input.DefaultDaysIn != nil {
us.DefaultDaysIn = input.DefaultDaysIn
}
r.Resolver.UserStore[id] = us
return &us, nil
}
This feels quite boilerplatey. Would it make sense in this situation to iterate through struct keys? Or is there another pattern I'm missing?
Use a function to reduce the boilerplate:
func mergef[T any](a, b *T) {
if b != nil {
*a = *b
}
}
...
mergef(&us.FirstName, input.FirstName)
mergef(&us.LastName, input.LastName)
...
Use the reflect package to reduce more boilerplate:
// merge sets fields in struct pointed to by d to
// dereferenced fields in struct pointed to by s.
//
// Argument s must point to a struct with pointer type
// fields.
// Argument d must point to a struct with fields that
// correspond to the fields in s: there must be a field
// in d with the same name as a field in s; the type of
// the field in s must be a pointer to the type of the field
// in d.
func merge(d, s any) {
sv := reflect.ValueOf(s).Elem()
dv := reflect.ValueOf(d).Elem()
for i := 0; i < sv.NumField(); i++ {
sf := sv.Field(i)
if sf.IsNil() {
continue
}
df := dv.FieldByName(sv.Type().Field(i).Name)
df.Set(sf.Elem())
}
}
Employ the function like this:
merge(us, input)
I am having trouble with scanning from a pgx query in Golang. The id field is always that of the last record. If I un-comment the var person Person declaration at the top of the function, every id is 3. There are 3 records with id's from 1 to 3 in my db. When I comment that declaration and declare the variable in the rows.Next() loop I get the correct id's. I can't figure out why the personId isn't being correctly overwritten
output from marshalled JSON with the var declared at the top of the function.
[{"person_id":3,"first_name":"Mark","last_name":"Brown"},{"person_id":3,"first_name":"Sam","last_name":"Smith"},{"person_id":3,"first_name":"Bob","last_name":"Jones"}]
output after declaring person every iteration of the scan loop
[{"person_id":1,"first_name":"Mark","last_name":"Brown"},{"person_id":2,"first_name":"Sam","last_name":"Smith"},{"person_id":3,"first_name":"Bob","last_name":"Jones"}]
I have this struct
// Person model
type Person struct {
PersonId *int64 `json:"person_id"`
FirstName *string `json:"first_name"`
LastName *string `json:"last_name"`
}
Here is my query function
func getPersons(rs *appResource, companyId int64) ([]Person, error) {
// var person Person
var persons []Person
queryString := `SELECT
user_id,
first_name,
last_name,
FROM users
WHERE company_id = $1`
rows, err := rs.db.Query(context.Background(), queryString, companyId)
if err != nil {
return persons, err
}
for rows.Next() {
var person Person
err = rows.Scan(
&person.PersonId,
&person.FirstName,
&person.LastName)
if err != nil {
return persons, err
}
log.Println(*person.PersonId) // 1, 2, 3 for both var patterns
persons = append(persons, person)
}
if rows.Err() != nil {
return persons, rows.Err()
}
return persons, err
}
I believe that you have discovered a bug (or, at least, unexpected behaviour) in github.com/jackc/pgx/v4. When running Scan it appears that if the pointer (so person.PersonId) is not nil then whatever it is pointing to is being reused. To prove this I replicated the issue and confirmed that you can also fix it with:
persons = append(persons, person)
person.PersonId = nil
I can duplicate the issue with this simplified code:
conn, err := pgx.Connect(context.Background(), "postgresql://user:password#127.0.0.1:5432/schema?sslmode=disable")
if err != nil {
panic(err)
}
defer conn.Close(context.Background())
queryString := `SELECT num::int FROM generate_series(1, 3) num`
var scanDst *int64
var slc []*int64
rows, err := conn.Query(context.Background(), queryString)
if err != nil {
panic(err)
}
for rows.Next() {
err = rows.Scan(&scanDst)
if err != nil {
panic(err)
}
slc = append(slc, scanDst)
// scanDst = nil
}
if rows.Err() != nil {
panic(err)
}
for _, i := range slc {
fmt.Printf("%v %d\n", i, *i)
}
The output from this is:
0xc00009f168 3
0xc00009f168 3
0xc00009f168 3
You will note that the pointer is the same in each case. I have done some further testing:
Uncommenting scanDst = nil in the above fixes the issue.
When using database/sql (with the "github.com/jackc/pgx/stdlib" driver) the code works as expected.
If PersonId is *string (and query uses num::text) it works as expected.
The issue appears to boil down to the following in convert.go:
if v := reflect.ValueOf(dst); v.Kind() == reflect.Ptr {
el := v.Elem()
switch el.Kind() {
// if dst is a pointer to pointer, strip the pointer and try again
case reflect.Ptr:
if el.IsNil() {
// allocate destination
el.Set(reflect.New(el.Type().Elem()))
}
return int64AssignTo(srcVal, srcStatus, el.Interface())
So this handles the case where the destination is a pointer (for some datatypes). The code checks if it is nil and, if so, creates a new instance of the relevant type as a destination. If it's not nil it just reuses the pointer. Note: I've not used reflect for a while so there may be issues with my interpretation.
As the behaviour differs from database/sql and is likely to cause confusion I believe it's probably a bug (I guess it could be an attempt to reduce allocations). I have had a quick look at the issues and could not find anything reported (will have a more detailed look later).
Im working with for each loop and var of information and filtering it by A) regex.matchString B)Timecomparrison. The filtering works well and I have the data I need but currently I'm outputting it to screen via fmt.Println in part of the loop. My goal is to take that data and build another var with the now filtered list. I guess I need to make a new variable and add to it? But how do I return that and something I can use later?
Any assistance is appreciated.
for _, thing := range things {
if thing.element1 != nil {
matched, err := regexp.MatchString(z, element1)
if err != nil {
fmt.Println(err)
}
if matched {
if timecomparrison(element2, a) {
// this is a section that needs to be added new var and returned as a var
fmt.Println("****")
fmt.Println("element1:", element1)
fmt.Println("element2:", element2)
}
}
}
}
}
I think you need something like this.
type Thing struct {
element1 string
element2 string
}
func filter() []Thing {
things := []Thing{
{element1: "element1", element2: "element2"},
}
var result []Thing
regex := "/{}d/"
date := time.Now
for _, thing := range things {
if thing.element1 != nil {
matched, err := regexp.MatchString(regex, thing.element1)
if err != nil {
fmt.Println(err)
}
if matched {
if timeComparison(thing.element2, date) {
// this is a section that needs to be added new var and returned as a var
fmt.Println("****")
fmt.Println("element1:", thing.element1)
fmt.Println("element2:", thing.element2)
result = append(result, thing)
}
}
}
}
return result
}
I cleaned the code, added a type and some data, fixed some issues and renamed some things, but you should get the idea :)
I'm trying to parse an HTML page and print its links.
I'm going over the parsed Html tree recursively adding links to a slice of strings.
I'm missing something out as I get out of memory error
here is my code:
package parser
import (
"errors"
"io"
"golang.org/x/net/html"
)
//URLParser returns all the urls inside a html page
type URLParser struct {
}
//GetURLS returns all
func (URLParser) GetURLS(htmlInput io.Reader) (*[]string, error) {
result := []string{}
htmlRoot, err := html.Parse(htmlInput)
//result := make([]string, 1000)
if err != nil {
parserError := errors.New("html parser failed with error" + err.Error())
return nil, parserError
}
finalResult := traverseHTMLTree(htmlRoot, &result)
return finalResult, nil
}
func traverseHTMLTree(node *html.Node, result *[]string) *[]string {
if node == nil {
return nil
}
if isLinkElement(node) {
currlink, shouldUse := getURLAttrb(node.Attr)
if shouldUse {
*result = append(*result, currlink)
}
}
for currNode := node.FirstChild; currNode != nil; currNode = currNode.NextSibling {
currRest := traverseHTMLTree(currNode, result)
if currRest != nil {
*result = append(*currRest, *result...)
}
}
return result
}
func getURLAttrb(attr []html.Attribute) (string, bool) {
for i := 0; i < len(attr); i++ {
if attr[i].Key == "href" {
return attr[i].Val, true
}
}
return "", false
}
func isLinkElement(node *html.Node) bool {
if node.Type == html.ElementNode {
if node.Data == "a" {
return true
}
}
return false
}
When just trying to print the links to stdout it works perfectly, so it got to be something with the way I handle the slice..
For each node in the HTML tree, you're adding the contents of the results array to itself, recursively. That should grow pretty quickly.
Note that when you check if isLinkElement, you add the item to the result list.
Then for each element of the html tree, you append the contents of the results array to itself, doubling it.
You're passing a pointer to a slice. You're effectively using a single slice for the whole program, and keep adding to it. What you're returning from the traverseHTMLTree is the same slice, not a copy of it. So it keeps growing.
One way to fix it is: do not pass the pointer to the slice. Pass the current slice, update it, and return the new slice.
I have created an object mapping in Go that is not relational, it is very simple.
I have several structs that looks like this:
type Message struct {
Id int64
Message string
ReplyTo sql.NullInt64 `db:"reply_to"`
FromId int64 `db:"from_id"`
ToId int64 `db:"to_id"`
IsActive bool `db:"is_active"`
SentTime int64 `db:"sent_time"`
IsViewed bool `db:"is_viewed"`
Method string `db:"-"`
AppendTo int64 `db:"-"`
}
To create a new message I just run this function:
func New() *Message {
return &Message{
IsActive: true,
SentTime: time.Now().Unix(),
Method: "new",
}
}
And then I have a message_crud.go file for this struct that looks like this:
To find a message by a unique column (for example by id) I run this function:
func ByUnique(column string, value interface{}) (*Message, error) {
query := fmt.Sprintf(`
SELECT *
FROM message
WHERE %s = ?
LIMIT 1;
`, column)
message := &Message{}
err := sql.DB.QueryRowx(query, value).StructScan(message)
if err != nil {
return nil, err
}
return message, nil
}
And to save a message (insert or update in the database) I run this method:
func (this *Message) save() error {
s := ""
if this.Id == 0 {
s = "INSERT INTO message SET %s;"
} else {
s = "UPDATE message SET %s WHERE id=:id;"
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(this))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(*this)
if err != nil {
return err
}
if this.Id == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
this.Id = lastId
}
return nil
}
The sql.PlaceholderPairs() function looks like this:
func PlaceholderPairs(i interface{}) string {
s := ""
val := reflect.ValueOf(i).Elem()
count := val.NumField()
for i := 0; i < count; i++ {
typeField := val.Type().Field(i)
tag := typeField.Tag
fname := strings.ToLower(typeField.Name)
if fname == "id" {
continue
}
if t := tag.Get("db"); t == "-" {
continue
} else if t != "" {
s += t + "=:" + t
} else {
s += fname + "=:" + fname
}
s += ", "
}
s = s[:len(s)-2]
return s
}
But every time I create a new struct, for example a User struct I have to copy paste the "crud section" above and create a user_crud.go file and replace the words "Message" with "User", and the words "message" with "user". I repeat alot of code and it is not very dry. Is there something I could do to not repeat this code for things I would reuse? I always have a save() method, and always have a function ByUnique() where I can return a struct and search by a unique column.
In PHP this was easy because PHP is not statically typed.
Is this possible to do in Go?
Your ByUnique is almost generic already. Just pull out the piece that varies (the table and destination):
func ByUnique(table string, column string, value interface{}, dest interface{}) error {
query := fmt.Sprintf(`
SELECT *
FROM %s
WHERE %s = ?
LIMIT 1;
`, table, column)
return sql.DB.QueryRowx(query, value).StructScan(dest)
}
func ByUniqueMessage(column string, value interface{}) (*Message, error) {
message := &Message{}
if err := ByUnique("message", column, value, &message); err != nil {
return nil, err
}
return message, error
}
Your save is very similar. You just need to make a generic save function along the lines of:
func Save(table string, identifier int64, source interface{}) { ... }
Then inside of (*Message)save, you'd just call the general Save() function. Looks pretty straightforward.
Side notes: do not use this as the name of the object inside a method. See the link from #OneOfOne for more on that. And do not get obsessed about DRY. It is not a goal in itself. Go focuses on code being simple, clear, and reliable. Do not create something complicated and fragile just to avoid typing a simple line of error handling. This doesn't mean that you shouldn't extract duplicated code. It just means that in Go it is usually better to repeat simple code a little bit rather than create complicated code to avoid it.
EDIT: If you want to implement Save using an interface, that's no problem. Just create an Identifier interface.
type Ider interface {
Id() int64
SetId(newId int64)
}
func (msg *Message) Id() int64 {
return msg.Id
}
func (msg *Message) SetId(newId int64) {
msg.Id = newId
}
func Save(table string, source Ider) error {
s := ""
if source.Id() == 0 {
s = fmt.Sprintf("INSERT INTO %s SET %%s;", table)
} else {
s = fmt.Sprintf("UPDATE %s SET %%s WHERE id=:id;", table)
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(source))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(source)
if err != nil {
return err
}
if source.Id() == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
source.SetId(lastId)
}
return nil
}
func (msg *Message) save() error {
return Save("message", msg)
}
The one piece that might blow up with this is the call to Exec. I don't know what package you're using, and it's possible that Exec won't work correctly if you pass it an interface rather than the actual struct, but it probably will work. That said, I'd probably just pass the identifier rather than adding this overhead.
You probably want to use an ORM.
They eliminate a lot of the boilerplate code you're describing.
See this question for "What is an ORM?"
Here is a list of ORMs for go: https://github.com/avelino/awesome-go#orm
I have never used one myself, so I can't recommend any. The main reason is that an ORM takes a lot of control from the developer and introduces a non-negligible performance overhead. You need to see for yourself if they fit your use-case and/or if you are comfortable with the "magic" that's going on in those libraries.
I don't recommend doing this, i personally would prefer being explicit about scanning into structs and creating queries.
But if you really want to stick to reflection you could do:
func ByUnique(obj interface{}, column string, value interface{}) error {
// ...
return sql.DB.QueryRowx(query, value).StructScan(obj)
}
// Call with
message := &Message{}
ByUnique(message, ...)
And for your save:
type Identifiable interface {
Id() int64
}
// Implement Identifiable for message, etc.
func Save(obj Identifiable) error {
// ...
}
// Call with
Save(message)
The approach i use and would recommend to you:
type Redirect struct {
ID string
URL string
CreatedAt time.Time
}
func FindByID(db *sql.DB, id string) (*Redirect, error) {
var redirect Redirect
err := db.QueryRow(
`SELECT "id", "url", "created_at" FROM "redirect" WHERE "id" = $1`, id).
Scan(&redirect.ID, &redirect.URL, &redirect.CreatedAt)
switch {
case err == sql.ErrNoRows:
return nil, nil
case err != nil:
return nil, err
}
return &redirect, nil
}
func Save(db *sql.DB, redirect *Redirect) error {
redirect.CreatedAt = time.Now()
_, err := db.Exec(
`INSERT INTO "redirect" ("id", "url", "created_at") VALUES ($1, $2, $3)`,
redirect.ID, redirect.URL, redirect.CreatedAt)
return err
}
This has the advantage of using the type system and mapping only things it should actually map.