example code (edited code snippet): http://play.golang.org/p/eZV4WL-4N_
Why is it that
for x, _ := range body.Personality {
body.Personality[x].Mutate()
}
successfully mutates the structs' contents, but
for _, pf := range body.Personality{
pf.Mutate()
}
does not?
Is it that range creates new instances of each item it iterates over? because the struct does in fact mutate but it doesn't persist.
The range keyword copies the results of the array, thus making it impossible to alter the
contents using the value of range. You have to use the index or a slice of pointers instead of values
if you want to alter the original array/slice.
This behaviour is covered by the spec as stated here. The crux is that an assignment line
x := a[i] copies the value a[i] to x as it is no pointer. Since range is defined to use
a[i], the values are copied.
Your second loop is roughly equivalent to:
for x := range body.Personality {
pf := body.Personality[x]
pf.Mutate()
}
Since body.Personality is an array of structs, the assignment to pf makes a copy of the struct, and that is what we call Mutate() on.
If you want to range over your array in the way you do in the example, one option would be to make it an array of pointers to structs (i.e. []*PFile). That way the assignment in the loop will just take a pointer to the struct allowing you to modify it.
Related
Let's say a Go 1.18 program has a quite heavy struct, for which copying is to be considered costly:
type MyStruct struct {
P string
// a lot of properties
}
Now let's define a function, taking a slice of such elements as input parameter, which goal is to update properties of each slice element:
func myFunc(sl []MyStruct) {
for i := range sl {
p := &sl[i] // <-- HERE
p.P = "bar"
// other properties mutations
}
}
At the <-- HERE mark, is the Golang compiler making a temporary copy of the slice element into the loop's scope, or is it taking the address of the slice element in-place?
The idea is to avoid copying the whole slice element.
A working example: https://go.dev/play/p/jHOC2DauyrQ?v=goprev
&sl[i] does not copy the slice element, it just evaluates to the address of the ith element.
Slice elements act as variables, and &x evaluates to the address of the x variable. Think about it: since &sl[i] is the address of the ith element, the address does not need nor use the struct value, why would it make a copy?
If your slices are so big that you're worried about the performance impact of (implicit) copies, you really should consider storing pointers in the slice in the first place, and that way you can make your loop and accessing elements much simpler without having to worry about copies:
func myFunc(sl []*MyStruct) {
for _, v := range sl {
v.P = "bar"
// other properties mutations
}
}
Also note that if your slice holds non-pointers, and you want to change a field of a slice element, indexing the slice and referring to the field also does not involve copying the struct element:
func myFunc(sl []MyStruct) {
for i := range sl {
sl[i].P = "bar"
// other properties mutations
}
}
Yes, this may be more verbose and may be less efficient if you have to modify multiple fields (but compilers may also recognize and optimize the evaluation of multiple sl[i] expressions).
Well we can use Golang built in append method to add more data into a defined struct. e.g.
type aclStruct struct { acl string}
a := []aclStruct{aclStruct{"A"}, aclStruct{"B"}}
a = append(a, aclStruct{"C"})
No Doubt it's working. But when I tried without append method it's producing an error. E.g.
What I am trying to do is :
a := append(aclStruct{"A"}, aclStruct{"B"}, aclStruct{"C"})
Even not sure sure it's a right way for struct because on array it's works fine. Also is there any way of doing this without using of append ?
Any Help??
Even not sure sure it's a right way for struct because on array it's works fine.
The append function appends elements to the end of a slice.
structs are declared statically. There is simply no way in Go to change their structure to add fields to them at runtime.
If you want a slice of structs, which you may use to track multiple representations of the struct, perhaps with different data points, you can build that structure in several ways:
Using append, ensuring the first argument passed is a (possibly empty) slice:
a := append([]aclStruct{}, aclStruct{"A"}, ...)
Declare the slice variable a and pass this to append:
var a []aclStruct{}
a = append(a, aclStruct{"A"}, ...)
Declaring and initializing the slice with values inline:
a := []aclStruct{{"A"}, {"B"}}
Note that you don't need to re-specify the concrete type for each slice element, as it can be inferred from the type of the slice)
If you want to declare slice of three elements use:
a := []aclStruct{aclStruct{"A"}, aclStruct{"B"}, aclStruct{"C"}}
append should be used to modify currently existing slice (it appends to a slice). It is throwing an error, because first argument should be a slice.
if you want to use it:
var a []aclStruct
append(a, aclStruct{"A"})
GoDoc answers the question well:
func append(slice []Type, elems ...Type) []Type
The append built-in function appends elements to the end of a slice. If it has sufficient capacity, the destination is resliced to accommodate the new elements. If it does not, a new underlying array will be allocated. Append returns the updated slice. It is therefore necessary to store the result of append, often in the variable holding the slice itself:
slice = append(slice, elem1, elem2)
slice = append(slice, anotherSlice...)
first argument of append must be a slice so correct way of doing it is:
a := append([]aclStruct{aclStruct{"A"}}, aclStruct{"B"}, aclStruct{"C"})
or
append([]aclStruct{{"A"}}, aclStruct{"B"}, aclStruct{"C"})
a way of doing it without append is:
a := []aclStruct{aclStruct{"A"}, aclStruct{"B"}}
appendedArray := []aclStruct{aclStruct{"C"}, a...}
I'm struggling to understand why I have a bug in my code in one state but not the other. It's been a while since I've covered pointers, so I'm probably rusty!
Basically I have a repository structure I'm using to store an object in memory, that has a Store function.
type chartsRepository struct {
mtx sync.RWMutex
charts map[ChartName]*Chart
}
func (r *chartsRepository) Store(c *Chart) error {
r.mtx.Lock()
defer r.mtx.Unlock()
r.charts[c.Name] = c
return nil
}
So all it does is put a RW mutex lock on and adds the pointer to a map, referenced by an identifier.
Then I've got a function that will basically loop through a slice of these objects, storing them all in the repository.
type service struct {
charts Repository
}
func (svc *service) StoreCharts(arr []Chart) error {
hasError := false
for _, chart := range arr {
err := svc.repo.Store(&chart)
// ... error handling
}
if hasError {
// ... Deals with the error object
return me
}
return nil
}
The above doesn't work, it looks like everything works fine at first, but on trying to access the data later, the entries in the map all point to the same Chart object, despite having different keys.
If I do the following and move the pointer reference to another function, everything works as expected:
func (svc *service) StoreCharts(arr []Chart) error {
// ...
for _, chart := range arr {
err := svc.storeChart(chart)
}
// ...
}
func (svc *service) storeChart(c Chart) error {
return svc.charts.Store(&c)
}
I'm assuming the issue is that because the loop overwrites the reference to the chart in the for loop, the pointer reference also changes. When the pointer is generated in an independent function, that reference is never overwritten. Is that right?
I feel like I'm being stupid, but shouldn't the pointer be generated by &chart and that's independent of the chart reference? I also tried creating a new variable for the pointer p := &chart in the for loop and that didn't work either.
Should I just avoid generating pointers in loops?
This is because there is only a single loop variable chart, and in each iteration just a new value is assigned to it. So if you attempt to take the address of the loop variable, it will be the same in each iteration, so you will store the same pointer, and the pointed object (the loop variable) is overwritten in each iteration (and after the loop it will hold the value assigned in the last iteration).
This is mentioned in Spec: For statements: For statements with range clause:
The iteration variables may be declared by the "range" clause using a form of short variable declaration (:=). In this case their types are set to the types of the respective iteration values and their scope is the block of the "for" statement; they are re-used in each iteration. If the iteration variables are declared outside the "for" statement, after execution their values will be those of the last iteration.
Your second version works, because you pass the loop variable to a function, so a copy will be made of it, and then you store the address of the copy (which is detached from the loop variable).
You can achieve the same effect without a function though: just create a local copy and use the address of that:
for _, chart := range arr {
chart2 := chart
err := svc.repo.Store(&chart2) // Address of the local var
// ... error handling
}
Also note that you may also store the address of the slice elements:
for i := range arr {
err := svc.repo.Store(&arr[i]) // Address of the slice element
// ... error handling
}
The disadvantage of this is that since you store pointers to the slice elements, the whole backing array of the slice would have to be kept in memory for as long as you keep any of the pointers (the array cannot be garbage collected). Moreover, the pointers you store would share the same Chart values as the slice, so if someone would modify a chart value of the passed slice, that would effect the charts whose pointers you stored.
See related questions:
Golang: Register multiple routes using range for loop slices/map
Why do these two for loop variations give me different behavior?
I faced a similar issue today and creating this simple example helped me understand the problem.
// Input array of string values
inputList := []string {"1", "2", "3"}
// instantiate empty list
outputList := make([]*string, 0)
for _, value := range inputList {
// print memory address on each iteration
fmt.Printf("address of %v: %v\n", value, &value)
outputList = append(outputList, &value)
}
// show memory address of all variables
fmt.Printf("%v", outputList)
This printed out:
address of 1: 0xc00008e1e0
address of 2: 0xc00008e1e0
address of 3: 0xc00008e1e0
[0xc00008e1e0 0xc00008e1e0 0xc00008e1e0]
As you can see, the address of value in each iteration was always the same even though the actual value was different ("1", "2", and "3"). This is because value was getting reassigned.
In the end, every value in the outputList was pointing to the same address which is now storing the value "3".
I have read on the golang blog: https://blog.golang.org/go-maps-in-action that:
var m map[string]int
Map types are reference types, like pointers or slices, and so the
value of m above is nil; it doesn't point to an initialized map. A nil
map behaves like an empty map when reading, but attempts to write to a
nil map will cause a runtime panic; don't do that. To initialize a
map, use the built in make function:
m = make(map[string]int)
The make function allocates and initializes a hash map data structure
and returns a map value that points to it.
I have a hard time understanding some parts of this:
What does var m map[string]int do?
Why do I need to write m = make(map[string]int) but not i = make(int)
What does var m map[string]int do?
It tells the compiler that m is a variable of type map[string]int, and assigns "The Zero Value" of the type map[string] int to m (that's why m is nil as nil is The Zero Value of any map).
Why do I need to write m = make(map[string]int) but not i = make(int)
You don't need to. You can create a initialized map also like this:
m = map[string]int{}
which does exactly the same.
The difference between maps and ints is: A nil map is perfectly fine. E.g. len() of a nil map works and is 0. The only thing you cannot do with a nil map is store key-value-pairs. If you want to do this you'll have to prepare/initialize the map. This preparation/initialization in Go is done through the builtin make (or by a literal map as shown above). This initialization process is not needed for ints. As there are no nil ints this initialization would be total noise.
Note that you do not initialize the variable m: The variable m is a map of strings to ints, initialized or not. Like i is a variable for ints. Now ints are directly usable while maps require one more step because the language works that way.
What does var m map[string]int do?
You can think about it like pointer with nil value, it does not point to anything yet but able to point to concrete value.
Why do I need to write m = make(map[string]int) but not i = make(int)
https://golang.org/doc/effective_go.html#allocation_make
Back to allocation. The built-in function make(T, args) serves a purpose different from new(T). It creates slices, maps, and channels only, and it returns an initialized (not zeroed) value of type T (not *T). The reason for the distinction is that these three types represent, under the covers, references to data structures that must be initialized before use. A slice, for example, is a three-item descriptor containing a pointer to the data (inside an array), the length, and the capacity, and until those items are initialized, the slice is nil. For slices, maps, and channels, make initializes the internal data structure and prepares the value for use. For instance,
make([]int, 10, 100)
allocates an array of 100 ints and then creates a slice structure with length 10 and a capacity of 100 pointing at the first 10 elements of the array. (When making a slice, the capacity can be omitted; see the section on slices for more information.) In contrast, new([]int) returns a pointer to a newly allocated, zeroed slice structure, that is, a pointer to a nil slice value.
These examples illustrate the difference between new and make.
var p *[]int = new([]int) // allocates slice structure; *p == nil; rarely useful
var v []int = make([]int, 100) // the slice v now refers to a new array of 100 ints
// Unnecessarily complex:
var p *[]int = new([]int)
*p = make([]int, 100, 100)
// Idiomatic:
v := make([]int, 100)
Remember that make applies only to maps, slices and channels and does not return a pointer. To obtain an explicit pointer allocate with new or take the address of a variable explicitly.
All words have the same length of 32 bits (4 bytes) or 64 bits (8 bytes),
depending on the processor and the operating system. They are identified by their memory address (represented as a hexadecimal number).
All variables of primitive types like int, float, bool, string ... are value types, they point directly to the values contained in the memory. Also composite types like arrays and structs are value types. When assigning with = the value of a value type to another variable: j = i, a copy of the original value i is made in memory.
More complex data which usually needs several words are treated as reference types. A reference type variable r1 contains the address (a number) of the memory location where the value of r1 is stored (or at least the 1st word of it):
For reference types when assigning r2 = r1, only the reference (the address) is copied and not the value!!. If the value of r1 is modified, all references of that value (like r1 and r2) will be reflected.
In Go pointers are reference types, as well as slices, maps and channels. The variables that are referenced are stored in the heap, which is garbage collected.
In the light of the above statements it's clear why the article states:
To initialize a map, use the built in make function.
The make function allocates and initializes a hash map data structure and returns a map value that points to it. This means you can write into it, compare to
var m map[string]int
which is readable, resulting a nil map, but an attempt to write to a nil map will cause a runtime panic. This is the reason why it's important to initialize the map with make.
m = make(map[string]int)
I have a struct array of type []Struct. When I range over it in the form:
for i, val := range mystructarray
I understand that val is a local variable which contains a copy of mystructarray[i]. Is there a better way of iterating through the addressess of mystructarray than this:
for i := range mystructarray{
valptr = &mystructarray[i]
}
?
There is no way to iterate while receiving a pointer to the contents of the slice (unless of course, it is a slice of pointers).
Your example is the best way:
for i := range mySlice {
x = &mySlice[i]
// do something with x
}
Remember however, if your structs aren't very large, and you don't need to operate on them via a pointer, it may be faster to copy the struct, and provide you with clearer code.