Consider the following code
package main
import (
"fmt"
)
func main() {
x := []byte("a")
fmt.Println(x)
fmt.Println(cap(x) == cap([]byte("a"))) // prints false
y := []byte("a")
fmt.Println(cap(y) == cap([]byte("a"))) // prints true
}
https://play.golang.org/p/zv8KQekaxH8
Calling simple Println with a slice variable, changes its capacity. I suspect calling any function with variadic parameters of ...interface{} produces the same effect. Is there any sane explanation for such behavior?
The explanation is, like bradfitz point in github, if you don't use make to create a slice, the compiler will use the cap it believes convenient. Creating multiple slices in different versions, or even the same, can result on slices of different capacities.
In short, if you need a concrete capacity, use make([]byte, len, cap). Otherwise you can't trust on a fixed capacity.
Related
In plain C, if I want a shallow heap copy of a struct, I would malloc() and memcpy() it.
In Go, I guess I have to do something like this:
original := Data{...}
copy := &Data{} // malloc
*copy = original // memcpy
But it doesn't look nice to me, nor idiomatic. What's the "right" way to do it?
The idiomatic way is to do a simple assignment and let the compiler allocate copy on the heap after performing escape analysis:
original := Data{...}
copy := original
return © // Or call some function with © as a parameter
Upon noticing that copy is used by reference and outlives the stack, Go will automatically allocate it on the heap rather than on the stack (the copy is still done properly of course)
We effectively no longer care about the heap, letting the compiler allocate it there as needed based on escape analysis. Our only concern is the copy itself.
You can see an example in action on godbolt:
Given the following simple code:
func main() {
type Data struct{
foo string
}
original := Data{"hi"}
copy := original
copyPtr := ©
fmt.Println(copyPtr)
}
Go will automatically allocate copy on the heap:
call runtime.newobject(SB)
We can also see this in action by passing extra flags at compile time showing escape and inlining decisions:
$ go build -gcflags '-m' .
...
./main.go:11:2: moved to heap: copy
...
Note: copy is a builtin function. It might be a good idea to avoid reusing the name (it works just fine, but it's not great practice).
A struct variable in Golang can be copied to another simply by an assignment statement:
https://play.golang.org/p/4Zcbxhy5UoB
package main
import (
"fmt"
)
type User struct {
name string
}
func main() {
u1 := User{name: "foo"}
u2 := u1
u2.name = "bar"
fmt.Println("u1: ", u1)
fmt.Println("u2: ", u2)
}
output:
u1: {foo}
u2: {bar}
I'm learning Go and have a C/C++ background. In the following example, is it safe to append the address of a into slice? When I run this example, the correct value (2) is printed, but wanted to be sure. If this is wrong, how should I do it?
func add(mapping map[string]*[]*int) {
sliceptr := &[]*int{}
mapping["foo"] = sliceptr
ele := mapping["foo"]
a := 2
// won't address of `a` go out of scope?
ele2 := append(*ele, &a)
mapping["foo"] = &ele2
}
func main() {
mapping := map[string]*[]*int{}
add(mapping)
fmt.Println(*(*mapping["foo"])[0])
}
It's safe to reference a after the function declaring it ends, because go does escape analysis. If the compiler can prove it can be accessed safely, it puts it on the stack, if not, it allocates it on the heap.
Build flags can give some insight into the escape analysis:
go build -gcflags "-m" main.go
...
./main.go:10:2: moved to heap: a
...
This might be helpful: Allocation efficiency.
Also, it's less common to see pointers to slices, since a slice is small: a pointer, length and capacity. See slice internals.
A related questions is here https://stackoverflow.com/a/12965872/6421681.
In go, you can do:
func numsInFactorial(n int) (nums []int) {
// `nums := make([]int)` is not needed
for i := 1; i <= n; i++ {
nums = append(nums, i)
}
return
}
However,the following doesn't work:
func mapWithOneKeyAndValue(k int, v int) (m map[int]int) {
m[k] = v
return
}
An error is thrown:
panic: assignment to entry in nil map
Instead, you must:
func mapWithOneKeyAndValue(k int, v int) map[int]int {
m := make(map[int]int)
m[k] = v
return
}
I can't find the documentation for this behavior.
I have read through all of effective go, and there's no mention of it there either.
I know that named return values are defined (i.e. memory is allocated; close to what new does) but not initialized (so make behavior isn't replicated).
After some experimenting, I believe this behavior can be reduced into understanding the behavior of the following code:
func main() {
var s []int // len and cap are both 0
var m map[int]int
fmt.Println(s) // works... prints an empty slice
fmt.Println(m) // works... prints an empty map
s = append(s, 10) // returns a new slice, so underlying array gets allocated
fmt.Println(s) // works... prints [10]
m[10] = 10 // program crashes, with "assignment to entry in nil map"
fmt.Println(m)
}
The issue seems that append likely calls make and allocates a new slice detecting that the capacity of s is 0. However, map never gets an explicit initialization.
The reason for this SO question is two-pronged. First, I would like to document the behavior on SO. Second, why would the language allow non-initializing definitions of slice and map? With my experience with go so far, it seems to be a pragmatic language (i.e. unused variables lead to compilation failure, gofmt forces proper formatting), so it would make sense for it to prevent the code from compiling.
Try to assign in nil slice by index - you will get "panic: runtime error: index out of range" (example: https://play.golang.org/p/-XHh1jNyn5g)
The only reason why append function works with nil, is that append function can do reallocation for the given slice.
For example, if you trying to to append 6th element to slice of 5 elements with current capacity 5, it will create the new array with new capacity, copy all the info from old one, and swap the data array pointers in the given slice. In my understanding, it is just golang implementation of dynamic arrays.
So, the nil slice is just a special case of slice with not enough capacity, so it would be reallocated on any append operation.
More details on https://blog.golang.org/go-slices-usage-and-internals
From https://blog.golang.org/go-maps-in-action
A nil map behaves like an empty map when reading, but attempts to write to a nil map will cause a runtime panic; don't do that. To initialize a map, use the built in make function
It seems like a nil map is considered a valid empty map and that's the reason they don't allocate memory for it automatically.
So I'm super new to Golang and seeing as the big buzz around the language seems to be concurrency I figured a good way to get my toes wet would be to write a generalized map function. In psudo code:
map(F funtion,A array){
return([F(k) for k in A])
}
And obviously I want the calculations for each F(k) occur concurrently. For organization I have a main file with my implementation and a supporting file Mr with my required definitions.
.
├── main.go
└── Mr
└── Mr.go
Main is a simple test implementation which should convert an array of strings to an array of ints where each member of the result is the length of the corresponding string in the input array.
package main
import(
"fmt"
"./Mr"
)
func exfunc(i int, c chan int){
c<-i
}
func main(){
data := make(map[int]string)
data[1]="these"
data[2]="are"
data[3]="some"
data[4]="words"
data[5]="and a few more..."
f := func(w string)int{
return(len(w))
}
testMr := Mr.Map(f,data) // this is line 22 (matters later)
fmt.Println(testMr)
}
Which works great with my Mr.Map function given that I specify all the types explicitly.
package Mr
type result struct{
key,value int
}
func wrapper(f func(string) int,k int,v string, c chan result){
c <- result{k,f(v)}
}
func Map(f func(string) int,m map[int]string) map[int]int{
c := make(chan result)
ret := make(map[int]int)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
However I was hoping that I would be able to generalize this mapping with the empty interface.
package Mr
type T interface{}
type result struct{
key,value T
}
func wrapper(f func(T) T,k T,v T, c chan result){
c <- result{k,f(v)}
}
func Map(f func(T) T,m map[T]T) map[T]T{
c := make(chan result)
ret := make(map[T]T)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
Unfortunately when I run main with this generalize Mr.Map I get the following error.
# command-line-arguments
./main.go:22: cannot use f (type func(string) int) as type func(Mr.T) Mr.T in argument to Mr.Map
./main.go:22: cannot use data (type map[int]string) as type map[Mr.T]Mr.T in argument to Mr.Map
So yeah, obviously I understand what the errors are telling me but it seems wild that I would have to re-write my Map function for each possible combination of key and value types.
Is there a work around here, or is this just the nature of the beast?
No real workaround there, that's the nature of the beast.
The language was designed after struggling with C++ for some time, and the idea of the creators was simplifying all non-vital things but at the same time make key additions to make the language more expressive.
You can read a bit about their reasoning here, which I believe is quite interesting even if you don't agree with all the decisions they made:
https://commandcenter.blogspot.com.ar/2012/06/less-is-exponentially-more.html
In your example, if you wanted to, you could make your maps and functions use interface{} (which by the way is called the empty interface and not "nil" interface).
But of course you would lose compile-time type checking and would have to add casts all around.
You can also try to find an interface to express the commonalities of the types you want to use (which might not be so easy or even possible at all), and then build your mapping API around that interface.
The philosophy of Go is not compatible with generalized functions such as is the style with popular dynamic languages. To properly inform the compiler of what you are trying to do, you should express your needed map through an interface or simply by writing it for each type you are using it with.
Mapping requires allocating an array, iterating through a collection, and adding to the array some data element for each element in the collection. If you need a map for a slice of structs, as is common in the application layer, you can express this tersely in Go:
https://play.golang.org/p/pk3Tl_BdlD
Dynamic languages build a "type tree" of "generic" types that allow for terse programming, such as functions like map being called by one symbol over any possible type. This provides a ton of developer productivity because code can be written loosely to allow easy experimentation.
Go is designed for writing semi-permanent software. It performs well because it requires more information to be supplied to the compiler. Map is only about three lines of code, so the cost/benefit of developer productivity v. efficiency lands on the performance side for Go. Functions like map, reduce and filter should be written explicitly as needed.
To evaluate the language, I would encourage you to try to solve a problem with a Go program and see where that takes you.
In Go, is there any way to compare two non-nil function pointers to test for equality? My standard of equality is pointer equality. If not, is there any particular reason why pointer equality is not allowed?
As of now, if I attempt to do this in the straight-forward way:
package main
import "fmt"
func SomeFun() {
}
func main() {
fmt.Println(SomeFun == SomeFun)
}
I get
./func-pointers.go:12: invalid operation: SomeFun == SomeFun (func can only be compared to nil)
It is my understanding that this behavior was introduced recently.
I've found an answer using the reflect package; however Atom suggests below that this actually produces undefined behavior. See Atom's post for more info and a possible alternative solution.
package main
import "fmt"
import "reflect"
func SomeFun() { }
func AnotherFun() { }
func main() {
sf1 := reflect.ValueOf(SomeFun)
sf2 := reflect.ValueOf(SomeFun)
fmt.Println(sf1.Pointer() == sf2.Pointer())
af1 := reflect.ValueOf(AnotherFun)
fmt.Println(sf1.Pointer() == af1.Pointer())
}
Outputs:
true
false
Note that there is a difference between equality and identity. The operators == and != in Go1 are comparing the values for equivalence (except when comparing channels), not for identity. Because these operators are trying not to mix equality and identity, Go1 is more consistent than pre-Go1 in this respect.
Function equality is different from function identity.
One reason for not allowing == and != on function types is performance. For example, the following closure is not using any variables from its environment:
f := func(){fmt.Println("foo")}
Disallowing comparisons of functions enables the compiler to generate a single implementation for the closure, instead of requiring the run-time to create a new closure (at run-time). So, from performance viewpoint the decision to disallow function comparisons was a good decision.
In relation to using the reflect package to determine function identity, a code like
func SomeFun() {}
func AnotherFun() {}
func main() {
sf1 := reflect.ValueOf(SomeFun)
sf2 := reflect.ValueOf(SomeFun)
fmt.Println(sf1.Pointer() == sf2.Pointer()) // Prints true
af1 := reflect.ValueOf(AnotherFun)
fmt.Println(sf1.Pointer() == af1.Pointer()) // Prints false
}
relies on undefined behavior. There are no guarantees as to what the program will print. The compiler may decide that it will merge SomeFun and AnotherFun into a single implementation, in which case the 2nd print statement would print true. In fact, there is absolutely no guarantee that the 1st print statement will print true (it may, under some other Go1 compiler and run-time, print false).
A correct answer to your original question is:
package main
import "fmt"
func F1() {}
func F2() {}
var F1_ID = F1 // Create a *unique* variable for F1
var F2_ID = F2 // Create a *unique* variable for F2
func main() {
f1 := &F1_ID // Take the address of F1_ID
f2 := &F2_ID // Take the address of F2_ID
// Compare pointers
fmt.Println(f1 == f1) // Prints true
fmt.Println(f1 == f2) // Prints false
}
The workaround depends on the situtation. I had to change a couple of places where I was comparing functions. In once case I just did something different so I wouldn't need to compare them any more. In another case I used a struct to associate functions with comparable strings, something like,
type nameFunc struct {
name string
fval func()
}
I only had a couple of functions I needed to compare so it was simplest to keep a slice of these structs and scan the slice as needed, comparing the name field and dispatching fval. If you have very many you might use a map instead. If your functions have different signatures you could use interfaces, and so on.
weekly.2011-11-18
Map and function value comparisons are now disallowed (except for
comparison with nil) as per the Go 1 plan. Function equality was
problematic in some contexts and map equality compares pointers, not
the maps' content.
Equality
Function equality was problematic in the presence of closures (when
are two closures equal?)