say for some very simple Golang code:
package main
import "fmt"
func plus( a int, b int) int {
return a+b
}
func plusPlus(a,b,c int) int {
return a +b + c
}
func main() {
ptr := plus
ptr2 := plusPlus
fmt.Println(ptr)
fmt.Println(ptr2)
}
This has the following output:
0x2000
0x2020
What is going on here? This doesn't look like a function pointer, or any kind of pointer for that matter, that one would find in the stack. I also understand that Go, while offering some nice low level functionality in the threading department, also requires an OS for it to function; C is functional across all computer platforms and operating systems can be written in it while Go needs an operating system to function and in fact only works on a few OS right now. Do the very regular function pointers mean that this works on a VM? Or is the compiler just linked to low level C functions?
Go does not run on a virtual machine.
From the view of the language specification, ptr and ptr2 are function values. They can be called as ptr(1, 2) and ptr2(1, 2, 3).
Diving down into the implementation, the variables ptr and ptr2 are pointers to func values. See the Function Call design document for information on func values. Note the distinction between the language's "function" value and the implementation's "func" value.
Because the reflection API used by the fmt package indirects through the func values to get the pointer to print, the call tofmt.Println(ptr) prints the actual address of the plus function.
Go doesn't run on a virtual machine. Those are the actual addresses of the functions.
On my machine (go 1.4.1, Linux amd64) the program prints
0x400c00
0x400c20
which are different from the values in your example, but still pretty low. Checking the compiled code:
$ nm test | grep 'T main.plus'
0000000000400c00 T main.plus
0000000000400c20 T main.plusPlus
these are the actual addresses of the functions. func plus compiles to a mere 19 bytes of code, so plusPlus appears only 32 (0x20) bytes later to satisfy optimal alignment requirements.
For the sake of curiosity, here's the disassembly of func plus from objdump -d, which should dispell any doubts that Go compiles to anything but native code:
0000000000400c00 <main.plus>:
400c00: 48 8b 5c 24 08 mov 0x8(%rsp),%rbx
400c05: 48 8b 6c 24 10 mov 0x10(%rsp),%rbp
400c0a: 48 01 eb add %rbp,%rbx
400c0d: 48 89 5c 24 18 mov %rbx,0x18(%rsp)
400c12: c3 retq
They are function values:
package main
import "fmt"
func plus(a int, b int) int {
return a + b
}
func plusPlus(a, b, c int) int {
return a + b + c
}
func main() {
funcp := plus
funcpp := plusPlus
fmt.Println(funcp)
fmt.Println(funcpp)
fmt.Println(funcp(1, 2))
fmt.Println(funcpp(1, 2, 3))
}
Output:
0x20000
0x20020
3
6
Related
I want to dynamically generate a slice of bytes and prefill them with a a value. In this case, if difficulty is 3, I want to generate []byte("000").
I have the working code below, is there anyway to optimize this into a one-liner? (i.e. initialize it and not have to run a for loop and set each element manually)?
var targetPrefix []byte = make([]byte, difficulty)
for i := 0; i < difficulty; i++ {
targetPrefix[i] = 48 // UTF8 encoding for "0"
}
I guess it depends what you mean by optimize. If you mean performance, then no not really. Ultimately when you "request" some memory, it would have to be iterated across to set the values. This was often more noticeable when writing C/C++ after using malloc. If you looked at the contents of the memory, it might be a bunch of 0s, but its likely to be a bunch of random values.
If however you mean to just write less code and utilize something that already exists you could take a look at bytes.Repeat:
targetPrefix := bytes.Repeat([]byte("0"), difficulty)
In Go, write your code as:
package main
import "fmt"
func main() {
difficulty := 7
targetPrefix := make([]byte, difficulty)
for i := range targetPrefix {
targetPrefix[i] = '0'
}
fmt.Println(targetPrefix)
}
Playground: https://play.golang.org/p/QrxEvsnRKMC
Output:
[48 48 48 48 48 48 48]
Or, also in Go, write:
package main
import (
"bytes"
"fmt"
)
func main() {
difficulty := 7
targetPrefix := bytes.Repeat([]byte{'0'}, difficulty)
fmt.Println(targetPrefix)
}
Playground: https://play.golang.org/p/Setx4kXTo1_H
Output:
[48 48 48 48 48 48 48]
In trying to learn how to debug stack traces from panics, I came across something confusing.
package main
func F(a int) {
panic(nil)
}
func main() {
F(1)
}
outputs the following when I run it on the attached play link:
panic: nil
goroutine 1 [running]:
main.F(0x1, 0x10436000)
/tmp/sandbox090887108/main.go:4 +0x20
main.main()
/tmp/sandbox090887108/main.go:8 +0x20
I can't decipher what the second number means (the 0x10436000 in main.F(0x1, 0x10436000)). It doesn't appear if there's a second int argument, or if anything else if passed in as the first argument (can be seen in the second play link).
One arg: https://play.golang.org/p/3iV48xlNFR
Two args: https://play.golang.org/p/4jA7ueI86K
The data printed in the stack trace is the arguments, but the values don't correspond directly to the arguments passed in, it's the raw data printed in pointer-sized values (though usually this is the same as the native word size). The playground is slightly unique, in that it's a 64bit word architecture with 32bit pointers(GOARCH=amd64p32).
In traceback.go you can see that the values are printed by stepping through the arguments based on pointer size;
for i := uintptr(0); i < frame.arglen/sys.PtrSize; i++ {
So because the word size is twice as big as the pointer size in the playground, you will always have an even number of values printed in the frame arguments.
Another example of how the data is presented can be seen in the playground by using smaller types in the function arguments: https://play.golang.org/p/vHZDEHQZLh
func F(a uint8) {
panic(nil)
}
// F(1)
// main.F(0x97301, 0x10436000)
Only the first 8 bits of the 64bit word are used, which is 0x97301 & 0x0f, or simply 1. The extra 0x97300 from the first value and the entire 0x10436000 are just the remainder of that first 64bit word which is unused by the function.
You can see the same behavior on amd64 systems by using more arguments. This signature for example;
func F(a, b, c uint32)
when called via F(1, 1, 1), it will print a stack trace like:
main.F(0x100000001, 0xc400000001)
because the 3 32bit values take 2 words
The final set of values to note are return values, which are also allocated on the stack. The following function signature:
func F(a int64) (int, int)
on amd64, would show the stack frame arguments like:
main.F(0xa, 0x1054d60, 0xc420078058)
With one word for a, and two more for the (int, int) return values. The return values are not initialized however, so there's not much to me gained here other than to understand why these values are there.
For instance, github.com/yhat/scrape suggests using a closure like this:
func someFunc() {
...
matcher := func(n *html.Node) bool {
return n.DataAtom == atom.Body
}
body, ok := scrape.Find(root, matcher)
...
}
Since matcher doesn’t actually capture any local variables, this could equivalently be written as:
func someFunc() {
...
body, ok := scrape.Find(root, matcher)
...
}
func matcher(n *html.Node) bool {
return n.DataAtom == atom.Body
}
The first form looks better, because the matcher function is quite specific to that place in the code. But does it perform worse at runtime (assuming someFunc may be called often)?
I guess there must be some overhead to creating a closure, but this kind of closure could be optimized into a regular function by the compiler?
(Obviously the language spec doesn’t require this; I’m interested in what gc actually does.)
It seems like there is no difference. We can check in the generated machine code.
Here is a toy program:
package main
import "fmt"
func topLevelFunction(x int) int {
return x + 4
}
func useFunction(fn func(int) int) {
fmt.Println(fn(10))
}
func invoke() {
innerFunction := func(x int) int {
return x + 8
}
useFunction(topLevelFunction)
useFunction(innerFunction)
}
func main() {
invoke()
}
And here is its disassembly:
$ go version
go version go1.8.5 linux/amd64
$ go tool objdump -s 'main\.(invoke|topLevel)' bin/toy
TEXT main.topLevelFunction(SB) /home/vasiliy/cur/work/learn-go/src/my/toy/toy.go
toy.go:6 0x47b7a0 488b442408 MOVQ 0x8(SP), AX
toy.go:6 0x47b7a5 4883c004 ADDQ $0x4, AX
toy.go:6 0x47b7a9 4889442410 MOVQ AX, 0x10(SP)
toy.go:6 0x47b7ae c3 RET
TEXT main.invoke(SB) /home/vasiliy/cur/work/learn-go/src/my/toy/toy.go
toy.go:13 0x47b870 64488b0c25f8ffffff FS MOVQ FS:0xfffffff8, CX
toy.go:13 0x47b879 483b6110 CMPQ 0x10(CX), SP
toy.go:13 0x47b87d 7638 JBE 0x47b8b7
toy.go:13 0x47b87f 4883ec10 SUBQ $0x10, SP
toy.go:13 0x47b883 48896c2408 MOVQ BP, 0x8(SP)
toy.go:13 0x47b888 488d6c2408 LEAQ 0x8(SP), BP
toy.go:17 0x47b88d 488d052cfb0200 LEAQ 0x2fb2c(IP), AX
toy.go:17 0x47b894 48890424 MOVQ AX, 0(SP)
toy.go:17 0x47b898 e813ffffff CALL main.useFunction(SB)
toy.go:14 0x47b89d 488d0514fb0200 LEAQ 0x2fb14(IP), AX
toy.go:18 0x47b8a4 48890424 MOVQ AX, 0(SP)
toy.go:18 0x47b8a8 e803ffffff CALL main.useFunction(SB)
toy.go:19 0x47b8ad 488b6c2408 MOVQ 0x8(SP), BP
toy.go:19 0x47b8b2 4883c410 ADDQ $0x10, SP
toy.go:19 0x47b8b6 c3 RET
toy.go:13 0x47b8b7 e874f7fcff CALL runtime.morestack_noctxt(SB)
toy.go:13 0x47b8bc ebb2 JMP main.invoke(SB)
TEXT main.invoke.func1(SB) /home/vasiliy/cur/work/learn-go/src/my/toy/toy.go
toy.go:15 0x47b8f0 488b442408 MOVQ 0x8(SP), AX
toy.go:15 0x47b8f5 4883c008 ADDQ $0x8, AX
toy.go:15 0x47b8f9 4889442410 MOVQ AX, 0x10(SP)
toy.go:15 0x47b8fe c3 RET
As we can see, at least in this simple case, there is no structural difference in how topLevelFunction and innerFunction (invoke.func1), and their passing to useFunction, are translated to machine code.
(It is instructive to compare this to the case where innerFunction does capture a local variable; and to the case where, moreover, innerFunction is passed via a global variable rather than a function argument — but these are left as an exercise to the reader.)
It generally should. And probably even more so with compiler optimization taken into account (as reasoning about a function is generally easier then about a closure, so I would expect a compiler to tend to optimize a function more often then an equivalent closure). But it is not exactly black and white as many factors may affect the final code produced, including your platform and version of the compiler itself. And more importantly, your other code will typically affect performance much more then speed of making a call (both algorithm wise and lines of code wise), which seems to be the point JimB made.
For example, I wrote following sample code and then benchmarked it.
var (
test int64
)
const (
testThreshold = int64(1000000000)
)
func someFunc() {
test += 1
}
func funcTest(threshold int64) int64 {
test = 0
for i := int64(0); i < threshold; i++ {
someFunc()
}
return test
}
func closureTest(threshold int64) int64 {
someClosure := func() {
test += 1
}
test = 0
for i := int64(0); i < threshold; i++ {
someClosure()
}
return test
}
func closureTestLocal(threshold int64) int64 {
var localTest int64
localClosure := func() {
localTest += 1
}
localTest = 0
for i := int64(0); i < threshold; i++ {
localClosure()
}
return localTest
}
On my laptop, funcTest takes 2.0 ns per iteration, closureTest takes 2.2 ns and
closureTestLocal takes 1.9ns. Here, closureTest vs funcTest appears confirming your (and mine) assumption that a closure call will be slower then a function call. But please note that those test functions were intentionally made simple and small to make call speed difference to stand out and it's still only 10% difference. In fact, checking compiler output shows that actually in funcTest case compiler did inline funcTest instead of calling it. So, I would expect the difference be even smaller if it didn't. But more importantly, I'd like to point out that closureTestLocal is 5% faster then the (inlined) function even though this one is actually a capturing closure. Please note that neither of the closures was inlined or optimized out - both closure tests faithfully make all the calls. The only difference I see in the compiled code for local closure case operates completely on the stack, while both other functions access a global variable (somewhere in memory) by it's address. But whilst I easily can reason about the difference by looking at the compiled code, my point is - it's not exactly black and white even in the simplest cases.
So, if speed is really that important in your case, I would suggest benchmarking it instead (and with actual code). You also could use go tool objdump to analyze actual code produced to get a clue where difference comes from. But as a rule of thumb, I would suggest to rather focus on writing better code (whatever that means for you) and ignore speed of actual calls (as in "avoid premature optimization").
I don't think scope of function declaration can harm performance. Also it's common to inline lambda in call. I'd write it
body, ok := scrape.Find(root, func (n *html.Node) bool {return n.DataAtom == atom.Body})
There is a very nice description about loading a shared library and calling a function with the syscall package on Windows (https://github.com/golang/go/wiki/WindowsDLLs). However, the functions LoadLibrary and GetProcAddress that are used in this description are not available in the syscall package on Linux. I could not find documentation about how to do this on Linux (or Mac OS).
Thanks for help
Linux syscalls are used directly without loading a library, exactly how depends on which system call you would like to perform.
I will use the Linux syscall getpid() as an example, which returns the process ID of the calling process (our process, in this case).
package main
import (
"fmt"
"syscall"
)
func main() {
pid, _, _ := syscall.Syscall(syscall.SYS_GETPID, 0, 0, 0)
fmt.Println("process id: ", pid)
}
I capture the result of the syscall in pid, this particular call returns no errors so I use blank identifiers for the rest of the returns. Syscall returns two uintptr and 1 error.
As you can see I can also just pass in 0 for the rest of the function arguments, as I don't need to pass arguments to this syscall.
The function signature is: func Syscall(trap uintptr, nargs uintptr, a1 uintptr, a2 uintptr, a3 uintptr) (r1 uintptr, r2 uintptr, err Errno).
For more information refer to https://golang.org/pkg/syscall/
I find the provision of named returned variables in Go to be a useful feature, because it can avoid the separate declaration of a variable or variables. However, in some instances I want to return a different variable to the variable declared in the function as the return-variable. That appears to work ok, however I do find it a little strange to declare a return variable and then return something else.
While writing a test program to help learn Go (not the one below), I found it a little annoying specifying the return variable in the return statement of functions returning multiple variables. Particularly so, because the variables had been named in the function declaration. I now find while posting this, that it appears that where there are named return variables, they don't need to be used in the return statement, just "return" will suffice and will implicitly use the named variables. That I find this a great feature.
So, although I have possibly partly answered my own question, could someone advise if my usage below is acceptable? I'm sure this is documented, but I haven't come across it, and it doesn't appear to be in the reference-book that I purchased which I think overlooks this feature.
Basically, the rule appears to be (as far as I can determine), that where named return variables are used, that the function statement declares the variables, and also the function can optionally implicitly uses them as the return values, however this can be overridden by using explicit return values.
Example Program :
package main
func main() {
var sVar1, sVar2 string
println("Test Function return-values")
sVar1, sVar2 = fGetVal(1)
println("This was returned for '1' : " + sVar1 + ", " + sVar2)
sVar1, sVar2 = fGetVal(2)
println("This was returned for '2' : " + sVar1 + ", " + sVar2)
}
func fGetVal(iSeln int) (sReturn1 string, sReturn2 string) {
sReturn1 = "This is 'sReturn1'"
sReturn2 = "This is 'sReturn2'"
switch iSeln {
case 1 : return
default : return "This is not 'sReturn1'", "This is not 'sReturn2'"
}
}
Your usage is absolutely fine and you'll find plenty of similar examples in the Go source code.
I'll attempt to explain how the return statement actually works in Go to give a deeper appreciation of why. It is useful to have a think about how Go implements parameter passing and return from functions. Once you understand that you'll understand why named return variables are so natural.
All arguments to functions and all return values from functions are passed on the stack in Go. This varies from C which usually passes some parameters in registers. When a function is called in Go the caller makes space on the stack for both the arguments and the return values then calls the function.
Specifically, when this function is called, which has 3 input parameters a, b, c and two return values
func f(a int, b int, c int) (int, int)
The stack will look like this (low memory address at the top)
* a
* b
* c
* space for return parameter 1
* space for return parameter 2
Now it is obvious that naming your return parameter just names those locations on the stack.
func f(a int, b int, c int) (x int, y int)
* a
* b
* c
* x
* y
It should now also be obvious what an empty return statement does - it just returns to the caller with whatever the values of x and y are.
Now for some disassembly! Compiling this with go build -gcflags -S test.go
package a
func f(a int, b int, c int) (int, int) {
return a, 0
}
func g(a int, b int, c int) (x int, y int) {
x = a
return
}
Gives
--- prog list "f" ---
0000 (test.go:3) TEXT f+0(SB),$0-40
0001 (test.go:3) LOCALS ,$0
0002 (test.go:3) TYPE a+0(FP){int},$8
0003 (test.go:3) TYPE b+8(FP){int},$8
0004 (test.go:3) TYPE c+16(FP){int},$8
0005 (test.go:3) TYPE ~anon3+24(FP){int},$8
0006 (test.go:3) TYPE ~anon4+32(FP){int},$8
0007 (test.go:4) MOVQ a+0(FP),BX
0008 (test.go:4) MOVQ BX,~anon3+24(FP)
0009 (test.go:4) MOVQ $0,~anon4+32(FP)
0010 (test.go:4) RET ,
--- prog list "g" ---
0011 (test.go:7) TEXT g+0(SB),$0-40
0012 (test.go:7) LOCALS ,$0
0013 (test.go:7) TYPE a+0(FP){int},$8
0014 (test.go:7) TYPE b+8(FP){int},$8
0015 (test.go:7) TYPE c+16(FP){int},$8
0016 (test.go:7) TYPE x+24(FP){int},$8
0017 (test.go:7) TYPE y+32(FP){int},$8
0018 (test.go:7) MOVQ $0,y+32(FP)
0019 (test.go:8) MOVQ a+0(FP),BX
0020 (test.go:8) MOVQ BX,x+24(FP)
0021 (test.go:9) RET ,
Both functions assemble to pretty much the same code. You can see quite clearly the declarations of a,b,c,x,y on the stack in g, though in f, the return values are anonymous anon3 and anon4.
Note: CL 20024 (March 2016, for Go 1.7) clarifies the usage of named return values and illustrates within the code base of go itself when its usage is appropriate:
all: remove public named return values when useless
Named returned values should only be used on public funcs and methods
when it contributes to the documentation.
Named return values should not be used if they're only saving the
programmer a few lines of code inside the body of the function,
especially if that means there's stutter in the documentation or it
was only there so the programmer could use a naked return
statement. (Naked returns should not be used except in very small
functions)
This change is a manual audit & cleanup of public func signatures.
Signatures were not changed if:
the func was private (wouldn't be in public godoc)
the documentation referenced it
For instance, archive/zip/reader.go#Open() used
func (f *File) Open() (rc io.ReadCloser, err error) {
It now uses:
func (f *File) Open() (io.ReadCloser, error) {
Its named return values didn't add anything to its documentation, which was:
// Open returns a `ReadCloser` that provides access to the File's contents.
// Multiple files may be read concurrently.
Yes, it's totally acceptable. I usually use named returned variables to assure a default return in a defer error treatment, to ensure a minimum viable return, like example below:
//execute an one to one reflection + cache operation
func (cacheSpot CacheSpot) callOneToOne(originalIns []reflect.Value) (returnValue []reflect.Value) {
defer func() { //assure for not panicking
if r := recover(); r != nil {
log.Error("Recovering! Error trying recover cached values!! y %v", r)
//calling a original function
returnValue = reflect.ValueOf(cacheSpot.OriginalFunc).Call(originalIns)
}
}()
//... doing a really nasty reflection operation, trying to cache results. Very error prone. Maybe panic
return arrValues //.. it's ok, arrValues achieved
}