Is it safe to return and modify primitive values during func return - go

(question reworded based on discussion, original question below)
The Go Language Specification gives examples showing that order of evaluation for primitives is unspecified with respect to function calls during assignment to slices and maps. None of the examples mention multi-value return though, so I'm not sure if they apply in this case or not.
Examples from the spec
a := 1
f := func() int { a++; return a }
x := []int{a, f()} // x may be [1, 2] or [2, 2]: evaluation order between a and f() is not specified
m := map[int]int{a: 1, a: 2} // m may be {2: 1} or {2: 2}: evaluation order between the two map assignments is not specified
n := map[int]int{a: f()} // n may be {2: 3} or {3: 3}: evaluation order between the key and the value is not specified
Using the language of the spec, is this functions return also unspecified:
func run() (int, int) {
a := 1
f := func() int { a++; return a }
return a, f() // always return 1,2 OR always returns 2,2 OR can return either?
}
If the order of evaluation is not specified then a non-toy example like the following could break at some time in the future if the compiler is updated:
func CountRows(ctx context.Context, db *pgxpool.Pool) (int, error) {
row := db.QueryRow(ctx, "SELECT COUNT(*)")
var count int
return count, row.Scan(&count)
}
Original Question
I'm unclear whether the go language specification is clear on whether values returned from funcs are "returned" one at a time or once all expressions are evaluated.
Aka is this code guaranteed to always output 10 <nil> (as it does in the playground) or can it ever output 0 <nil>?
package main
import "fmt"
func main() {
fmt.Println(run())
// Output: 10 <nil>
}
func run() (int, error) {
var i int
return i, inc(&i)
}
func inc(i *int) error {
*i = *i + 10
return nil
}
Edit
This related question suggests that order of evaluation of the primitive return value is not specified by the specification

"Is this function return also unspecified?"
Yes. As you found, the language specifies the evaluation order for some things:
when evaluating the operands of an expression, assignment, or return statement, all function calls, method calls, and communication operations are evaluated in lexical left-to-right order.
Naturally, anything that is not a function call, method call or communication operation is left unspecified with respect to evaluation order in an expression, assignment or return statement.
"a non-toy example like the following could break at some time in the future if the compiler is updated"
Yes. Even if the compiler doesn't update. If you expect it to have any particular result that's reliant on evaluation order, then the code is already broken in the sense that the language does not say the code will do what you think it will do.
"are values returned from funcs "returned" one at a time or once all expressions are evaluated."
There is no such distinction to be made.
"is this code guaranteed to always output 10 <nil> (as it does in the playground) or can it ever output 0 <nil>?"
Yes, it can output 0, <nil>. This is essentially the same as the previous run example, where now the closure f has been refactored as a function called inc.

Yes! This is a really great question.
Another way to write this code is
package main
import "fmt"
func main() {
fmt.Println(run())
// Output: 10 <nil>
}
func run() (int, error) {
var i int
err := inc(&i)
return i, err
}
func inc(i *int) error {
*i = *i + 10
return nil
}
This is the same reason why you can do the following
func main() {
callIt()
}
func callIt() (int, error) {
return multiReturn()
}
func multiReturn() (int, error) {
return 0, nil
}
And additionally, you can do this as well, which can be useful if you wrap a db transaction (for example) in an error handler
func main() {
result, err := assignInReturn()
if err != nil {
panic(err)
}
// the result here, assuming no error occurred, will be 1...
fmt.Println(result)
}
func assignInReturn() (int, error) {
var i int
return i, wrapErrFunc(func() error {
// db call
if err := dbCall(); err != nil {
return err
}
i = 1 // ...because of this. This will set `i` to 1 before `assignInReturn` returns to its caller
return nil
})
}
func wrapErrFunc(fn func() error) error {
return fn()
}
func dbCall() error {
// db query
return nil
}
In these situations, you can be certain that the items in the top-level return will evaluate prior to being returned to its caller

Related

Assign empty slice without referring to its type?

My code calls a library function which looks roughly like this:
func Search() ([]myLibrary.SomeObject, error) {
var results []apiv17.SomeObject
// ...
if (resultsFound) {
results = append(results, someResult)
}
return results
}
...and my code calls it and then marshals it to JSON.
results, err := myLibrary.Search()
bytes, err := json.Marshal(results)
Now the problem is that because of the way the Search function is written (and let's assume we can't change it), it'll return an uninitialized nil slice if there are no results. And unfortunately, there is no way to configure encoding/json to encode nil slices as [] (see e.g. this proposal with ongoing discussion).
Explicitly checking for nil solves the problem:
results, err := myLibrary.Search()
if results == nil {
results = []apiv17.SomeObject{}
}
bytes, err := json.Marshal(results)
...but it also adds an explicit dependency on the return type, apiv17.SomeObject. That's inconvenient because that type frequently changes in the library. E.g. in the next library version it might be apiv18.SomeObject.
With the nil check above, I'll have to update my code every time that happens.
Is there any way to avoid this and assign an empty, non-nil slice to the variable without explicitly referring to its type? Something like this:
results = [](type of results){}
Go 1.18
You can use a generic function that captures the slice's base type and returns a slice of length zero:
func echo[T any](v []T) []T {
return make([]T, 0)
}
func main() {
n := foo.GetFooBar()
if n == nil {
n = echo(n) // no need to refer to apiv17 here
}
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints []
}
The purpose of requiring a regular argument v []T in echo is to allow type inference to unify the slice []apiv17.SomeObject with the argument []T and infer T as the base type apiv17.SomeObject, so that you can call it just as echo(n) and no explicit type parameter.
The package apiv17 is of course known at compile time because it's transitively imported via myPackage, so you can take advantage of this and type inference to avoid adding an explicit import statement for apiv17.
This is how it looks like on the multi-file playground: https://go.dev/play/p/4ycTkaGLFpo
The type is declared in bar package, but main only imports play.ground/foo and only uses foo.GetFooBar.
Go 1.17 and below
Reflection. Just change the echo function from above to taking an interface{} argument (there's no any in Go 1.17, remember?) and do the deed with reflect.MakeSlice:
func set(v interface{}) {
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
panic("not a ptr")
}
reflect.Indirect(rv).Set(reflect.MakeSlice(rv.Type().Elem(), 0, 0))
}
Then pass a pointer to the slice, so that you can set its value with reflection.
func main() {
n := foo.GetFooBar()
if n == nil {
set(&n)
}
fmt.Printf("type: %T, val: %v, is nil: %t\n", n, n, n == nil)
// type: []bar.FooBar, val: [], is nil: false
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints [] again
}
Go 1.17 playground: https://go.dev/play/p/4jMkr22LMF7?v=goprev
The other answer describes how to create an empty slice.
But you can solve your original issue much simpler: if results is nil, you don't need to create a empty slice, regardless of whatever element type it would have, the JSON marshaling would be [] anyway. So if results is nil, no need to call json.Marshal(), just "output" []:
results, err := myLibrary.Search()
var bytes []byte
if results == nil {
bytes = []byte{'[', ']' } // JSON marshaling result is "[]"
} else {
bytes, err = json.Marshal(results)
// Handle error
}

Check named return error using defer function

Hi I want to write a generic function to trace error message when a function returns error. So I wrote this:
func TraceError1(err *error) {
if err != nil && *err != nil {
pc := make([]uintptr, 15)
n := runtime.Callers(2, pc)
frames := runtime.CallersFrames(pc[:n])
frame, _ := frames.Next()
fmt.Printf("%s:%d %s\n", frame.File, frame.Line, frame.Function)
}
}
func TraceError2(err error) {
if err != nil {
pc := make([]uintptr, 15)
n := runtime.Callers(2, pc)
frames := runtime.CallersFrames(pc[:n])
frame, _ := frames.Next()
fmt.Printf("%s:%d %s\n", frame.File, frame.Line, frame.Function)
}
}
func foo() (err error) {
defer TraceError1(&err)
defer TraceError2(err)
fmt.Println("do something")
return fmt.Errorf("haha")
}
TraceError1 works but TraceError2 didn't. In my understanding, error is an interface so it is a pointer/address, why do I need to pass its address? Why TraceError2 cannot work? Thanks.
In case of TraceError1 you are passing a pointer to the named return value err. The pointer is non-nil, but the value it points at (err) is nil (at the time of defer). However, it is not yet evaluated (dereferenced) because TraceError1 has not yet been called. By the time the function does run (after foo returns) and the pointer gets dereferenced, the value of err has been updated (by the return statement inside foo).
However, in case of TraceError2, a nil interface value is passed, which will stay nil even when TraceError2 executes eventually.
Here is a simpler example:
package main
import "fmt"
func intByValue(i int) {
fmt.Printf("i = %d\n", i)
// ^--- `i` is an integer value
// --- whatever i was passed to the function, gets printed
}
func intByRef(i *int) {
var v int = *i // i is a pointer to an int, which gets dereferenced here
// the *address* where the actual value resides was passed
// while the address stays the same, its value can change before
// i is dereferenced, and its value stored in v.
fmt.Printf("i = %d\n", v)
}
func main() {
var i int
defer intByValue(i) // passed the *value* of i, which is 0 right now
defer intByRef(&i) // passed a *pointer* to i, which contains 0 right now
i = 100 // before intByRef could "dereference" its argument, the value that it
// contained has been updated
// intByRef gets called, dereferences the value, finds 100, prints it.
// intByValue gets called, finds 0, prints it
// result should be:
// i = 100
// i = 0
}
So unfortunately, if you want the ability to update the error (e.g. by returning a named return value) before it gets used by the deferred function, you are going to have to pass around pointers to the variable.
In other words, TraceError2 is simply not suited for your use case.
Edit: use correct terminology and (questionably) improve example code.
As go blog explained
The behavior of defer statements is straightforward and predictable.
There are three simple rules:
A deferred function's arguments are evaluated when the defer statement is evaluated.
Deferred function calls are executed in Last In First Out order after the surrounding function returns.
Deferred functions may read and assign to the returning function's named return values.
According to first point, when you call defer TraceError2(err) , that err = nil and that is the value pass to the TraceError2 function.
TraceError1(err *error) works because it is getting a pointer to err, and that pointer value is assigned before defer func TraceError1 is executed.
Simple example code to explain the behaviour.
package main
import (
"fmt"
"runtime"
)
func main() {
i := 0
defer func(i int) {
fmt.Printf("%d\n",i) //Output: 0
}(i)
defer func(i *int) {
defer fmt.Printf("%d\n",*i) //Output: 1
}(&i)
i++
}

Go type for function call

Keywords like go and defer expect a function call as parameters. Is there a type available that can be used the same way? (e.g. to write a function that expects a function call - opposed to a function - as argument).
No there is not. You can't do the same with your function.
go and defer are backed by the language spec and the rule is enforced by the compiler.
What you may do is use a variable / value of function type, which you may call later / at any time as if it would be a function.
For example:
func myFunc() {
fmt.Println("hi")
}
func main() {
var f func()
f = myFunc
f() // This calls the function value stored in f: myFunc in this example
}
Edit: To have the functionality you mentioned in the comment: just wrap the function call with its arguments in a func(), and use / pass that.
For example:
func launch(f func()) {
fmt.Println("Before launch")
go func() {
defer fmt.Println("After completion")
f()
}()
}
Using it:
func main() {
launch(func() {
fmt.Println("Hello, playground")
})
time.Sleep(time.Second)
}
Which outputs (try it on the Go Playground):
Before launch
Hello, playground
After completion
Yes, this is not an exact workaround. If the params may change, you have to make a copy of them before calling launch(), and use the copy in the function literal (closure), like in this example:
s := "Hello, playground"
s2 := s // make a copy
launch(func() {
fmt.Println(s2) // Use the copy
})
s = "changed"
Mimicing automatic parameter saving
For a concrete function type we may construct a helper function which provides us automatic parameter saving. This helper function must have identical signature, and return a function without parameters. The returned function is a closure which calls the original function with the parameters. The act of calling this helper function is the mechanism to save the parameters, so the usage is identical to that of defer.
For example the helper for fmt.Println(s) is:
func wrapPrintln(s string) func() {
return func() {
fmt.Println(s)
}
}
And using it:
launch(wrapPrintln(s))
Example for a function with 2 int parameters:
func Sum(a, b int) {
fmt.Println("Sum:", a+b)
}
func WrapSum(a, b int) func() {
return func() {
Sum(a, b)
}
}
launch(WrapSum(a, b))
The above WrapPrintln() and WrapSum() wrapped a concrete function, and it can't be used for other functions (the wrapped function is "wired in"). We can make the wrapped functions a parameter too:
func WrapFuncIntInt(f func(a, b int), a, b int) func() {
return func() {
f(a, b)
}
}
And we may use it like this:
launch(WrapFuncIntInt(Sum, a, b))
Try this one on the Go Playground.
Using reflection to avoid the manual copies
You may use reflection to avoid having to make manual copies, but in this solution we're not actually calling the function, just passing it. Also due to using reflection, it will be slower. Another advantage is that this "feels" generic (we may use functions with different signatures), but we lose compile-time safety.
func launch(f interface{}, params ...interface{}) {
fmt.Println("Before launch")
go func() {
defer fmt.Println("After completion")
pv := make([]reflect.Value, len(params))
for i, v := range params {
pv[i] = reflect.ValueOf(v)
}
reflect.ValueOf(f).Call(pv)
}()
}
Example calling it:
func main() {
i, s := 1, "Hello, playground"
launch(fmt.Printf, "%d %q\n", i, s)
i, s = 2, "changed"
time.Sleep(time.Second)
}
Which outputs (try it on the Go Playground):
Before launch
1 "Hello, playground"
After completion
Single exception where you can utilize automatic parameter saving
There is a single exception which we may use. This is the Method value. If x has static type T and T's method set contains the method M, we may use x.M (without calling it).
The expression x.M is a method value, and it saves a copy of x which will be used as the receiver when the expression's result (which is a function value) is called.
Example:
type myParams struct {
format string
i int
s string
}
func (mp myParams) Call() {
fmt.Printf(mp.format, mp.i, mp.s)
}
func main() {
p := myParams{format: "%d %q\n", i: 1, s: "Hello, playground"}
launch(p.Call) // p is saved here
p.i, p.s = 2, "changed"
time.Sleep(time.Second)
}
func launch(f func()) {
fmt.Println("Before launch")
go func() {
defer fmt.Println("After completion")
f()
}()
}
It outputs the same. Try it on the Go Playground.

Tour of Go exercise #22: Reader, what does the question mean?

Exercise: Readers
Implement a Reader type that emits an infinite stream of the ASCII character 'A'.
I don't understand the question, how to emit character 'A'? into which variable should I set that character?
Here's what I tried:
package main
import "golang.org/x/tour/reader"
type MyReader struct{}
// TODO: Add a Read([]byte) (int, error) method to MyReader.
func main() {
reader.Validate(MyReader{}) // what did this function expect?
}
func (m MyReader) Read(b []byte) (i int, e error) {
b = append(b, 'A') // this is wrong..
return 1, nil // this is also wrong..
}
Ah I understand XD
I think it would be better to say: "rewrite all values in []byte into 'A's"
package main
import "golang.org/x/tour/reader"
type MyReader struct{}
// TODO: Add a Read([]byte) (int, error) method to MyReader.
func (m MyReader) Read(b []byte) (i int, e error) {
for x := range b {
b[x] = 'A'
}
return len(b), nil
}
func main() {
reader.Validate(MyReader{})
}
An io.Reader.Read role is to write a given memory location with data read from its source.
To implement a stream of 'A', the function must write given memory location with 'A' values.
It is not required to fill in the entire slice provided in input, it can decide how many bytes of the input slice is written (Read reads up to len(p) bytes into p), it must return that number to indicate to the consumer the length of data to process.
By convention an io.Reader indicates its end by returning an io.EOF error. If the reader does not return an error, it behaves as an infinite source of data to its consumer which can never detect an exit condition.
Note that a call to Read that returns 0 bytes read can happen and does not indicate anything particular, Callers should treat a return of 0 and nil as indicating that nothing happened; Which makes this non-solution https://play.golang.org/p/aiUyc4UDYi2 fails with a timeout.
In regard to that, the solution provided here https://stackoverflow.com/a/68077578/4466350 return copy(b, "A"), nil is really just right. It writes the minimum required, with an elegant use of built-ins and syntax facilities, and it never returns an error.
The alleged answer is didn't work for me, even without the typos.
Try as I did, that string would not go into b.
func (r MyReader) Read(b []byte) (int, error) {
return copy(b, "A"), nil
}
My solution: just add one byte at a time, store the index i using closure.
package main
import (
"golang.org/x/tour/reader"
)
type MyReader struct{}
func (mr MyReader) Read(b []byte) (int, error) {
i := 0
p := func () int {
b[i] = 'A'
i += 1
return i
}
return p(), nil
}
func main() {
reader.Validate(MyReader{})
}
Simplest one:
func (s MyReader) Read(b []byte) (int, error) {
b[0] = byte('A')
return 1, nil
}
You can generalize the idea to create an eternal reader, alwaysReader, from which you always read the same byte value over and over (it never results in EOF):
package readers
type alwaysReader struct {
value byte
}
func (r alwaysReader) Read(p []byte) (n int, err error) {
for i := range p {
p[i] = r.value
}
return len(p), nil
}
func NewAlwaysReader(value byte) alwaysReader {
return alwaysReader { value }
}
NewAlwaysReader() is the constructor for alwaysReader (which isn't exported). The result of NewAlwaysReader('A') is a reader from whom you will always read 'A'.
A clarifying unit test for alwaysReader:
package readers_test
import (
"bytes"
"io"
"readers"
"testing"
)
func TestEmptyReader(t *testing.T) {
const numBytes = 128
const value = 'A'
buf := bytes.NewBuffer(make([]byte, 0, numBytes))
reader := io.LimitReader(readers.NewAlwaysReader(value), numBytes)
n, err := io.Copy(buf, reader)
if err != nil {
t.Fatal("copy failed: %w")
}
if n != numBytes {
t.Errorf("%d bytes read but %d expected", n, numBytes)
}
for i, elem := range buf.Bytes() {
if elem != value {
t.Errorf("byte at position %d has not the value %v but %v", i, value, elem)
}
}
}
Since we can read from the alwaysReader forever, we need to decorate it with a io.LimitReader so that we end up reading at most numBytes from it. Otherwise, the bytes.Buffer will eventually run out of memory for reallocating its internal buffer because of io.Copy().
Note that the following implementation of Read() for alwaysReader is also valid:
func (r alwaysReader) Read(p []byte) (n int, err error) {
if len(p) > 0 {
p[0] = r.value
return 1, nil
}
return 0, nil
}
The former Read() implementation fills the whole byte slice with the byte value, whereas the latter writes a single byte.

How to send a message to an object in Golang? (.send() equivalent in go)

I am a Go beginner, coming from Ruby land.
In Ruby, you could do something like this.
Time.send("now") is equivalent to Time.now, as you are sending the message now to the object Time
Is there something similar in golang?
There is no built in way of calling an arbitrary function from a string in Go.
You can create something similar by registering functions to a map[string].
A working example:
package main
import "fmt"
var m = map[string]func(){
"now": func() { fmt.Println("The time is now") },
"then": func() { fmt.Println("Once upon a time") },
}
func main() {
cmd := "then"
m[cmd]()
}
play.golang.org
There is also the possibility of using reflection in order to call a method by name. You can look at the reflect package for MethodByName and Call. You can also check this Stackoverflow question.
As other suggested, you can do it yourself by mapping strings to functions, but the strong-typing nature of Go makes it difficult to translate .send directly into Go.
You can still use reflection if you really need to access a field or method by name:
import "reflect"
import "fmt"
type A struct {
Number int
}
func (a *A) Method(i int) int {
return a.Number + i;
}
func main() {
a := &A{Number: 1}
// Direct access
fmt.Printf("Direct -> Nb: %d, Nb + 2: %d\n", a.Number, a.Method(2));
v := reflect.ValueOf(*a)
vp := reflect.ValueOf(a)
field := v.FieldByName("Number")
meth := vp.MethodByName("Method")
args := []reflect.Value{reflect.ValueOf(2)}
// Reflection access
fmt.Printf("Reflect -> Nb: %d, Nb + 2: %d\n",
field.Interface().(int),
meth.Call(args)[0].Interface().(int))
}
Outputs:
Direct -> Nb: 1, Nb + 2: 3
Reflect -> Nb: 1, Nb + 2: 3
play.golang.org
Note however:
How cumbersome that is. Usually, performing a map as suggested by #ANisus is a more idiomatic way of doing
You still have to perform your conversions in the end.
Using the reflect packages changes your typed variable into more flexible Value objects, but these are very cumbersome to use in practice. It is usually better if you can find a way to express your intent without relying on reflection.
Also note that here, we had to use two Values, one for a (a pointer to A) for the method, and one for *a (a A structure) for the field. Trying to get a method defined with a pointer receiver with a non-pointer Value (or conversely, trying to obtain a field via a pointer Value) will result in a panic. More generally, due to the dynamic nature of reflected Values and its difference with the usual typed Go, expect a lot of convenience features (such as automatic referencing/dereferencing) to be absent on Values.
Also, expect quite a bit of runtime panics while debugging, as it is the only way for dynamic Value calls to fail !
Reference: the reflect package
No. Work your way through http://tour.golang.org/ and http://golang.org/doc/effective_go.html and you will have a proper understanding of how method invocation works.
Here is a working example using reflect
package main
import (
"fmt"
"os"
"reflect"
)
// Send sends a message to(calls a method of) obj, with args.
// The return value of the method call is set to ret and any error to err.
func Send(obj interface{}, method string, args ...interface{}) (ret []reflect.Value, err error) {
defer func() {
if e := recover(); e != nil {
err = fmt.Errorf("%v", e)
}
}()
objValue := reflect.ValueOf(obj)
argsValue := make([]reflect.Value, 0, len(args))
for _, arg := range args {
argsValue = append(argsValue, reflect.ValueOf(arg))
}
mtd := objValue.MethodByName(method)
if !mtd.IsValid() {
return nil, fmt.Errorf("%v does not have a method %v", reflect.TypeOf(obj), method)
}
ret = mtd.Call(argsValue)
return
}
// Then do some tests.
type A struct {
value int
}
func (a A) Value() int {
return a.value
}
func (a *A) SetValue(v int) {
a.value = v
}
func main() {
var (
ret []reflect.Value
err error
)
// StdOut.WriteString("Hello, World!\n")
_, err = Send(os.Stdout, "WriteString", "Hello, World!\n")
handleError(err)
var a = &A{100}
// ret = a.Value()
ret, err = Send(a, "Value")
handleError(err)
fmt.Printf("Return value is: %v\n", ret[0].Int())
// a.SetValue(200)
_, err = Send(a, "SetValue", 200)
handleError(err)
// ret = a.Value()
ret, err = Send(a, "Value")
handleError(err)
fmt.Printf("Return value is: %v", ret[0].Int())
}
func handleError(err error) {
if err != nil {
panic(err)
}
}
I based my code on this description of send.
class Klass
def hello(*args)
"Hello " + args.join(' ')
end
end
k = Klass.new
k.send :hello, "gentle", "readers" #=> "Hello gentle readers"
http://play.golang.org/p/lXlzBf_fGZ
package main
import "strings"
type Klass struct{}
func (k Klass) Hello(args ...string) string {
return "Hello " + strings.Join(args, " ")
}
func (k Klass) Send(symbol func(Klass, ...string) string, args ...string) string {
return symbol(k, args...)
}
func main() {
k := new(Klass)
k.Send(Klass.Hello, "gentle", "readers") //=> "Hello gentle readers"
}
The big difference between the two is that Go's Send function is only implemented for Klass and only works on methods that take a variable number of strings as parameters and return a single string. This is because Go is a statically typed language where Ruby is dynamically typed. Go does support dynamic typing via the reflect library, but it is an unpleasant experience and not the way general Go code is meant to be written.

Resources