func (c *conn) setState(nc net.Conn, state ConnState) {
...
c.curState.Store(connStateInterface[state])
...
}
// connStateInterface is an array of the interface{} versions of
// ConnState values, so we can use them in atomic.Values later without
// paying the cost of shoving their integers in an interface{}.
var connStateInterface = [...]interface{}{
StateNew: StateNew,
StateActive: StateActive,
StateIdle: StateIdle,
StateHijacked: StateHijacked,
StateClosed: StateClosed,
}
I can't figure out the trick with connStateInterface, how exactly does it work?
There's a few things going on here...
The [...] declaration creates an actual array instead of a slice, so that indirection is removed. What's being declared here is an array of interface{} types... so you might wonder why the weird map-looking notation?
The StateXXX variables are simply constants declared further above, so they are ints... so the declaration is actually of the form index: value.
Here's a less obfuscated example of that using an array of ints:
var i = [...]int{4: 2, 2: 7}
This will allocate an array containing:
[0, 0, 7, 0, 2]
... note that index 2 has 7, index 4 has 2. Not a common way of declaring an array, but it's valid Go.
So going back to the original declaration, just take the example I gave above, and instead of int, make the array of type interface{}:
var i = [...]interface{}{4: 2, 2: 7}
And you'll get a similar array, but with nil interface values in place of zeroes.
Getting even closer to the original code, the StateXXX constants are just ints, only not literals like in my example.
So, what's the point of all this? Why all the obfuscation?
It's a performance hack. The function c.curState.Store() takes an argument of type interface{}. If you were to pass it an int, the compiled code would have to fumble about with converting the type on each call. A more clear (though obviously impractical) illustration of this might be:
var val interface{}
for i := 0; i < 1000000; i++ {
// the types are different, compiler has to fumble int vs. interface{}
val = i
// do something with val
}
Every time you do val = i a conversion between int and interface{} needs to happen. The code you posted avoids this by creating a static lookup table where all the values are already of type interface.
Therefore, this:
c.curState.Store(connStateInterface[state])
is more efficient than this:
c.curState.Store(state)
Since state would, in this case, need to undergo the int -> interface{} conversion. In the optimized code, state is merely an index looking up a value into an array, the result of which gets you an interface{}... so the int -> interface{} type conversion is avoided.
I'm not familiar with that code, but I'd imagine it's in a critical path and the nanoseconds or whatever savings shaved off likely makes a difference.
Related
I have a slice of structs that looks like:
type KeyValue struct {
Key uint32
Value uint32
}
var mySlice []Keyvalue
This slice mySlice can have a variable length.
I would like to pass all elements of this slice into a function with this signature:
takeKeyValues(keyValue []uint32)
Now I could easily allocate a slice of []uint32, populate it with all the keys/values of the elements in mySlice and then pass that slice as an argument into takeKeyValues()...
But I'm trying to find a way of doing that without any heap allocations, by directly passing all the keys/values on the stack. Is there some syntax trick which allows me to do that?
There is no safe way to arbitrarily reinterpret the memory layout of the your data. It's up to you whether the performance gain is worth the use of an unsafe type conversion.
Since the fields of KeyValue are of equal size and the struct has no padding, you can convert the underlying array of KeyValue elements to an array of uint32.
takeKeyValues((*[1 << 30]uint32)(unsafe.Pointer(&s[0]))[:len(s)*2])
https://play.golang.org/p/Jjkv9pdFITu
As an alternative to JimB's answer, you can also use the reflect (reflect.SliceHeader) and unsafe (unsafe.Pointer) packages to achieve this.
https://play.golang.org/p/RLrMgoWgI7t
s := []KeyValue{{0, 100}, {1, 101}, {2, 102}, {3, 103}}
var data []uint32
sh := (*reflect.SliceHeader)(unsafe.Pointer(&data))
sh.Data = uintptr(unsafe.Pointer(&s[0]))
sh.Len = len(s) * 2
sh.Cap = sh.Len
fmt.Println(data)
I a bit puzzled by what you want. If the slice is dynamic, you have to grow it dynamically as you add elements to it (at the call site).
Go does not have alloca(3), but I may propose another approach:
Come up with an upper limit of the elements you'd need.
Declare variable of the array type of that size (not a slice).
"Patch" as many elements you want (starting at index 0) in that array.
"Slice" the array and pass the slice to the callee.
Slicing an array does not copy memory — it just takes the address of the array's first element.
Something like this:
func caller(actualSize int) {
const MaxLen = 42
if actualSize > MaxLen {
panic("actualSize > MaxLen")
}
var kv [MaxLen]KeyValue
for i := 0; i < actualSize; i++ {
kv[i].Key, kv[i].Value = 100, 200
}
callee(kv[:actualSize])
}
func callee(kv []KeyValue) {
// ...
}
But note that when the Go compiler sees var kv [MaxLen]KeyValue, it does not have to allocate it on the stack. Basically, I beleive the language spec does not tell anything about the stack vs heap.
Let's say I have the following function:
func foo(bar []int) {
bar[0] = 456
}
I want to create an []int with one element that references a, such that when I:
var a int = 10
// mySlice := ... crazy hackery?
foo(mySlice)
The value of a will be 456.
I know this is crazy and stupid and not at all the right thing to do. I'm not looking for the "correct" way. The function prototype for foo must remain the same.
I can guarantee that while the underlying int will change, the slice that points to it is immutable.
The reason for the strange requirement is because this is for the c2go project.
You can cast any types you want through an unsafe.Pointer.
First convert the *int to a pointer to an array, then slice it:
(*[1]int)(unsafe.Pointer(&a))[:]
https://play.golang.org/p/bmKcMIj3pb
What's the cleanest way to handle a case such as this:
func a() string {
/* doesn't matter */
}
b *string = &a()
This generates the error:
cannot take the address of a()
My understanding is that Go automatically promotes a local variable to the heap if its address is taken. Here it's clear that the address of the return value is to be taken. What's an idiomatic way to handle this?
The address operator returns a pointer to something having a "home", e.g. a variable. The value of the expression in your code is "homeless". if you really need a *string, you'll have to do it in 2 steps:
tmp := a(); b := &tmp
Note that while there are completely valid use cases for *string, many times it's a mistake to use them. In Go string is a value type, but a cheap one to pass around (a pointer and an int). String's value is immutable, changing a *string changes where the "home" points to, not the string value, so in most cases *string is not needed at all.
See the relevant section of the Go language spec. & can only be used on:
Something that is addressable: variable, pointer indirection, slice indexing operation, field selector of an addressable struct, array indexing operation of an addressable array; OR
A composite literal
What you have is neither of those, so it doesn't work.
I'm not even sure what it would mean even if you could do it. Taking the address of the result of a function call? Usually, you pass a pointer of something to someone because you want them to be able to assign to the thing pointed to, and see the changes in the original variable. But the result of a function call is temporary; nobody else "sees" it unless you assign it to something first.
If the purpose of creating the pointer is to create something with a dynamic lifetime, similar to new() or taking the address of a composite literal, then you can assign the result of the function call to a variable and take the address of that.
In the end you are proposing that Go should allow you to take the address of any expression, for example:
i,j := 1,2
var p *int = &(i+j)
println(*p)
The current Go compiler prints the error: cannot take the address of i + j
In my opinion, allowing the programmer to take the address of any expression:
Doesn't seem to be very useful (that is: it seems to have very small probability of occurrence in actual Go programs).
It would complicate the compiler and the language spec.
It seems counterproductive to complicate the compiler and the spec for little gain.
I recently was tied up in knots about something similar.
First talking about strings in your example is a distraction, use a struct instead, re-writing it to something like:
func a() MyStruct {
/* doesn't matter */
}
var b *MyStruct = &a()
This won't compile because you can't take the address of a(). So do this:
func a() MyStruct {
/* doesn't matter */
}
tmpA := a()
var b *MyStruct = &tmpA
This will compile, but you've returned a MyStruct on the stack, allocated sufficient space on the heap to store a MyStruct, then copied the contents from the stack to the heap. If you want to avoid this, then write it like this:
func a2() *MyStruct {
/* doesn't matter as long as MyStruct is created on the heap (e.g. use 'new') */
}
var a *MyStruct = a2()
Copying is normally inexpensive, but those structs might be big. Even worse when you want to modify the struct and have it 'stick' you can't be copying then modifying the copies.
Anyway, it gets all the more fun when you're using a return type of interface{}. The interface{} can be the struct or a pointer to a struct. The same copying issue comes up.
You can't get the reference of the result directly when assigning to a new variable, but you have idiomatic way to do this without the use of a temporary variable (it's useless) by simply pre-declaring your "b" pointer - this is the real step you missed:
func a() string {
return "doesn't matter"
}
b := new(string) // b is a pointer to a blank string (the "zeroed" value)
*b = a() // b is now a pointer to the result of `a()`
*b is used to dereference the pointer and directly access the memory area which hold your data (on the heap, of course).
Play with the code: https://play.golang.org/p/VDhycPwRjK9
Yeah, it can be annoying when APIs require the use of *string inputs even though you’ll often want to pass literal strings to them.
For this I make a very tiny function:
// Return pointer version of string
func p(s string) *string {
return &s
}
and then instead of trying to call foo("hi") and getting the dreaded cannot use "hi" (type string) as type *string in argument to foo, I just wrap the argument in a call to to p():
foo(p("hi"))
a() doesn't point to a variable as it is on the stack. You can't point to the stack (why would you ?).
You can do that if you want
va := a()
b := &va
But what your really want to achieve is somewhat unclear.
At the time of writing this, none of the answers really explain the rationale for why this is the case.
Consider the following:
func main() {
m := map[int]int{}
val := 1
m[0] = val
v := &m[0] // won't compile, but let's assume it does
delete(m, 0)
fmt.Println(v)
}
If this code snippet actually compiled, what would v point to!? It's a dangling pointer since the underlying object has been deleted.
Given this, it seems like a reasonable restriction to disallow addressing temporaries
guess you need help from More effective Cpp ;-)
Temp obj and rvalue
“True temporary objects in C++ are invisible - they don't appear in your source code. They arise whenever a non-heap object is created but not named. Such unnamed objects usually arise in one of two situations: when implicit type conversions are applied to make function calls succeed and when functions return objects.”
And from Primer Plus
lvalue is a data object that can be referenced by address through user (named object). Non-lvalues include literal constants (aside from the quoted strings, which are represented by their addresses), expressions with multiple terms, such as (a + b).
In Go lang, string literal will be converted into StrucType object, which will be a non-addressable temp struct object. In this case, string literal cannot be referenced by address in Go.
Well, the last but not the least, one exception in go, you can take the address of the composite literal. OMG, what a mess.
The slices are references to the underlying array. This makes sense and seems to work on builtin/primitive types but why is not working on structs? I assume that even if I update a struct field the reference/address is still the same.
package main
import "fmt"
type My struct {
Name string
}
func main() {
x := []int{1}
update2(x)
fmt.Println(x[0])
update(x)
fmt.Println(x[0])
my := My{Name: ""}
update3([]My{my})
// Why my[0].Name is not "many" ?
fmt.Println(my)
}
func update(x []int) {
x[0] = 999
return
}
func update2(x []int) {
x[0] = 1000
return
}
func update3(x []My) {
x[0].Name = "many"
return
}
To clarify: I'm aware that I could use pointers for both cases. I'm only intrigued why the struct is not updated (unlike the int).
What you do when calling update3 is you pass a new array, containing copies of the value, and you immediately discard the array. This is different from what you do with the primitive, as you keep the array.
There are two approaches here.
1) use an array of pointers instead of an array of values:
You could define update3 like this:
func update3(x []*My) {
x[0].Name = "many"
return
}
and call it using
update3([]*My{&my})
2) write in the array (in the same way you deal with the primitive)
arr := make([]My,1)
arr[0] = My{Name: ""}
update3(arr)
From the GO FAQ:
As in all languages in the C family, everything in Go is passed by
value. That is, a function always gets a copy of the thing being
passed, as if there were an assignment statement assigning the value
to the parameter. For instance, passing an int value to a function
makes a copy of the int, and passing a pointer value makes a copy of
the pointer, but not the data it points to. (See the next section for
a discussion of how this affects method receivers.)
Map and slice values behave like pointers: they are descriptors that
contain pointers to the underlying map or slice data. Copying a map or
slice value doesn't copy the data it points to.
Thus when you pass my you are passing a copy of your struct and the calling code won't see any changes made to that copy.
To have the function change the data in teh struct you have to pass a pointer to the struct.
Your third test is not the same as the first two. Look at this (Playground). In this case, you do not need to use pointers as you are not modifying the slice itself. You are modifying an element of the underlying array. If you wanted to modify the slice, by for instance, appending a new element, you would need to use a pointer to pass the slice by reference. Notice that I changed the prints to display the type as well as the value.
It is just mind boggling how many different ways Go has for variable initialization. May be either i don't understand this completely or Go is a big after thought. Lot of things don't feel natural and looks like they added features as they found them missing. Initialization is one of them.
Here is running app on Go playground showing different ways of initialization
Here is what i understand
There are values and pointers. Values are initiated using var = or :=.
:= only works inside the methods
To Create value and reference you use new or &. And they only work on composite types.
There are whole new ways of creating maps and slices
Create slice and maps using either make or var x []int. Noticing there is no = or :=
Are there any easy way to understand all this for newbies? Reading specs gives all this in bits and pieces everywhere.
First, to mention some incorrect statements:
To Create value and reference you use new or &. And they only work on composite types.
new() works on everything. & doesn't
Create slice and maps using either make or var x []int. Noticing there is no = or :=
You can actually use := with x := []int{} or even x := []int(nil). However, the first is only used when you want to make a slice of len 0 that is not nil and the second is never used because var x []int is nicer.
Lets start from the beginning. You initialize variables with var name T. Type inference was added because you can type less and avoid worrying about the name of a type a function returns. So you are able to do var name = f().
If you want to allocate a pointer, you would need to first create the variable being pointed to and then get its pointer:
var x int
var px = &x
Requiring two statements can be bothersome so they introduced a built-in function called new(). This allows you to do it in one statement: var px = new(int)
However, this method of making new pointers still has an issue. What if you are building a massive literal that has structs that expect pointers? This is a very common thing to do. So they added a way (only for composite literals) to handle this.
var x = []*T{
&T{x: 1},
&T{x: 2},
}
In this case, we want to assign different values to fields of the struct the pointers reference. Handling this with new() would be awful so it is allowed.
Whenever a variable is initialized, it is always initialized to its zero value. There are no constructors. This is a problem for some of the builtin types. Specifically slices, maps, and channels. These are complicated structures. They require parameters and initialization beyond memory being set to zero. Users of Go who need to do this simply write initialization functions. So, that is what they did, they wrote make(). neW([]x) returns a pointer to a slice of x while make([]x, 5) returns a slice of x backed by an array of length 5. New returns a pointer while make returns a value. This is an important distinction which is why it is separate.
What about :=? That is a big clusterfuck. They did that to save on typing, but it led to some odd behaviors and the tacked on different rules until it became somewhat usable.
At first, it was a short form of var x = y. However, it was used very very often with multiple returns. And most of those multiple returns were errors. So we very often had:
x, err := f()
But multiple errors show up in your standard function so people started naming things like:
x, err1 := f()
y, err2 := g()
This was ridiculous so they made the rule that := only re-declares if it is not already declared in that scope. But that defeats the point of := so they also tacked on the rule that at least one must be newly declared.
Interestingly enough, this solves 95% of its problems and made it usable. Although the biggest problem I run into with it is that all the variables on the lefthand side must be identifiers. In other words, the following would be invalid because x.y is not an identifier:
x.y, z := f()
This is a leftover of its relation to var which can't declare anything other than an identifier.
As for why := doesn't work outside a function's scope? That was done to make writing the compiler easier. Outside a function, every part starts with a keyword (var, func, package, import). := would mean that declaration would be the only one that doesn't start with a keyword.
So, that is my little rant on the subject. The bottom line is that different forms of declaration are useful in different areas.
Yeah, this was one of those things that I found confusing early as well. I've come up with my own rules of thumb which may or may not be best practices, but they've served me well. Tweaked after comment from Stephen about zero values
Use := as much as possible, let Go infer types
If you just need an empty slice or map (and don't need to set an initial capacity), use {} syntax
s := []string{}
m := map[string]string{}
The only reason to use var is to initialize something to a zero value (pointed out by Stephen in comments) (and yeah, outside of functions you'll need var or const as well):
var ptr *MyStruct // this initializes a nil pointer
var t time.Time // this initializes a zero-valued time.Time
I think your confusion is coming from mixing up the type system with declaration and initialization.
In Go, variables can be declared in two ways: using var, or using :=. var only declares the variable, while := also assigns an initial value to it. For example:
var i int
i = 1
is equivalent to
i := 1
The reason this is possible is that := simply assumes that the type of i is the same as the type of the expression it's being initialized to. So, since the expression 1 has type int, Go knows that i is being declared as an integer.
Note that you can also explicitly declare and initialize a variable with the var keyword as well:
var i int = 1
Those are the only two delcaration/initialization constructs in Go. You're right that using := in the global scope is not allowed. Other than that, the two are interchangable, as long as Go is able to guess what type you're using on the right-hand-side of the := (which is most of the time).
These constructs work with any types. The type system is completely independent from the declaration/initialization syntax. What I mean by that is that there are no special rules about which types you can use with the declaration/initialization syntax - if it's a type, you can use it.
With that in mind, let's walk through your example and explain everything.
Example 1
// Value Type assignments [string, bool, numbers]
// = & := Assignment
// Won't work on pointers?
var value = "Str1"
value2 := "Str2"
This will work with pointers. The key is that you have to be setting these equal to expressions whose types are pointers. Here are some expressions with pointer types:
Any call to new() (for example, new(int) has type *int)
Referencing an existing value (if i has type int, then &i has type *int)
So, to make your example work with pointers:
tmp := "Str1"
var value = &tmp // value has type *int
value2 := new(string)
*value2 = "Str2"
Example 2
// struct assignments
var ref1 = refType{"AGoodName"}
ref2 := refType{"AGoodName2"}
ref3 := &refType{"AGoodName2"}
ref4 := new(refType)
The syntax that you use here, refType{"AGoodName"}, is used to create and initialize a struct in a single expression. The type of this expression is refType.
Note that there is one funky thing here. Normally, you can't take the address of a literal value (for example, &3 is illegal in Go). However, you can take the address of a literal struct value, like you do above with ref3 := &refType{"AGoodName2"}. This seems pretty confusing (and certainly confused me at first). The reason it's allowed is that it's actually just a short-hand syntax for calling ref3 := new(refType) and then initializing it by doing *ref3 = refType{"AGoodName2"}.
Example 3
// arrays, totally new way of assignment now, = or := now won't work now
var array1 [5]int
This syntax is actually equivalent to var i int, except that instead of the type being int, the type is [5]int (an array of five ints). If we wanted to, we could now set each of the values in the array separately:
var array1 [5]int
array1[0] = 0
array1[1] = 1
array1[2] = 2
array1[3] = 3
array1[4] = 4
However, this is pretty tedious. Just like with integers and strings and so on, arrays can have literal values. For example, 1 is a literal value with the type int. The way you create a literal array value in Go is by naming the type explicitly, and then giving the values in brackets (similar to a struct literal) like this:
[5]int{0, 1, 2, 3, 4}
This is a literal value just like any other, so we can use it as an initializer just the same:
var array1 [5]int = [5]int{0, 1, 2, 3, 4}
var array2 = [5]int{0, 1, 2, 3, 4}
array3 := [5]int{0, 1, 2, 3, 4}
Example 4
// slices, some more ways
var slice1 []int
Slices are very similar to arrays in how they are initialized. The only difference is that they aren't fixed at a particular length, so you don't have to give a length parameter when you name the type. Thus, []int{1, 2, 3} is a slice of integers whose length is initially 3 (although could be changed later). So, just like above, we can do:
var slice1 []int = []int{1, 2, 3}
var slice2 = []int{1, 2, 3}
slice3 := []int{1, 2, 3}
Example 5
var slice2 = new([]int)
slice3 := new([]int)
Reasoning about types can get tricky when the types get complex. As I mentioned above, new(T) returns a pointer, which has type *T. This is pretty straightforward when the type is an int (ie, new(int) has type *int). However, it can get confusing when the type itself is also complex, like a slice type. In your example, slice2 and slice3 both have type *[]int. For example:
slice3 := new([]int)
*slice3 = []int{1, 2, 3}
fmt.Println((*slice3)[0]) // Prints 1
You may be confusing new with make. make is for types that need some sort of initialization. For slices, make creates a slice of the given size. For example, make([]int, 5) creates a slice of integers of length 5. make(T) has type T, while new(T) has type *T. This can certainly get confusing. Here are some examples to help sort it out:
a := make([]int, 5) // s is now a slice of 5 integers
b := new([]int) // b points to a slice, but it's not initialized yet
*b = make([]int, 3) // now b points to a slice of 5 integers
Example 6
// maps
var map1 map[int]string
var map2 = new(map[int]string)
Maps, just like slices, need some initialization to work properly. That's why neither map1 nor map2 in the above example are quite ready to be used. You need to use make first:
var map1 map[int]string // Not ready to be used
map1 = make(map[int]string) // Now it can be used
var map2 = new(map[int]string) // Has type *map[int]string; not ready to be used
*map2 = make(map[int]string) // Now *map2 can be used
Extra Notes
There wasn't a really good place to put this above, so I'll just stick it here.
One thing to note in Go is that if you declare a variable without initializing it, it isn't actually uninitialized. Instead, it has a "zero value." This is distinct from C, where uninitialized variables can contain junk data.
Each type has a zero value. The zero values of most types are pretty reasonable. For example, the basic types:
int has zero value 0
bool has zero value false
string has zero value ""
(other numeric values like int8, uint16, float32, etc, all have zero values of 0 or 0.0)
For composite types like structs and arrays, the zero values are recursive. That is, the zero value of an array is an array with all of its entries set to their respective zero values (ie, the zero value of [3]int is [3]int{0, 0, 0} since 0 is the zero value of int).
Another thing to watch out for is that when using the := syntax, certain expressions' types cannot be inferred. The main one to watch out for is nil. So, i := nil will produce a compiler error. The reason for this is that nil is used for all of the pointer types (and a few other types as well), so there's no way for the compiler to know if you mean a nil int pointer, or a nil bool pointer, etc.