I don't understand the behavior of the following piece of code. In creating a list of matching structs as a slice of struct pointers, the code always prints the last element of original array (which actually wasn't a match)—it prints 12 and 12. However, if I change matches to be []Widget instead of []*Widget, then it will print 10 and 11.
Why is this?
package main
import (
"fmt"
)
func main() {
type Widget struct {
id int
attrs []string
}
widgets := []Widget{
Widget{
id: 10,
attrs: []string{"blah", "foo"},
},
Widget{
id: 11,
attrs: []string{"foo", "bar"},
},
Widget{
id: 12,
attrs: []string{"xyz"},
},
}
var matches []*Widget
for _, w := range widgets {
for _, a := range w.attrs {
if a == "foo" {
matches = append(matches, &w)
break
}
}
}
for _, m := range matches {
fmt.Println(m.id)
}
}
That's because when you use the pointers you are adding &w to the array.
Note that w is actually the local variable used in the loop, so that's not the address you want to add to the matches array.
(even though the value of the variable w changes through the loop, its address stays the same)
When the loop ends, w ends up with the last value so that's why it prints 12 two times.
You need to add the address of the element that matched instead.
If you do this:
matches = append(matches, &widgets[i])
Then it'd work fine with pointers as well.
Modified Go playground for you to test it:
https://play.golang.org/p/YE-cokyEHu
Related
I'm currently trying to modify an element of a slice by passing a pointer to a function. Outside of the function the element is not modified.
Is there a way to modify the element without passing the slice itself along with the index of the desired element to alter?
package main
import (
"fmt"
)
type Item struct {
Value int
}
func alter(t *Item) {
(*t).Value = 100
}
func main() {
items := []Item{Item{0}, Item{1}}
for _, item := range items {
alter(&item)
}
fmt.Println(items) // Output is still [{0} {1}]
}
Golang Playground
for i := range items {
alter(&items[i])
}
Or
items := []*Item{{0}, {1}}
for _, item := range items {
alter(item)
}
The reason your version doesn't work is because the iteration variable item holds a copy of the element inside the slice, which means that what you're modifying is the copy and not the original. You can see that they are separate objects in memory if you run this: https://play.golang.org/p/vr9CfX0WQcB
Refer: https://tour.golang.org/moretypes/16
The range form of the for loop iterates over a slice or map.
When ranging over a slice, two values are returned for each iteration. The first is the index, and the second is a copy of the element at that index.
So,
for i, x := range arr {
// x is copy for arr[i]
}
Hence, we will directly used arr[i] and pass the address of the same to the alter function so that it could be modified.
Sample code:
package main
import "fmt"
type Item struct {
Value int
}
func alter(t *Item) {
(*t).Value = 100
}
func main() {
items := []Item{{0}, {1}}
for i := range items {
alter(&items[i])
}
fmt.Println(items)
}
[Playground]
I am dealing with data that is of the type map[string]interface{}. it can have unlimited number of nested objects inside (map[string]interface{}) types.
EDIT: This data comes from mongodb. I can't really apply golang's struct here because the attributes vary from document to document. All I want to do is get the most deeply nested object, add a new attribute to it and make sure the entire data object is updated after.
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
},
},
}
This data is coming from a remote API and I have no idea of the properties inside. All I want to add is add new attribute inside the last object (in this case "scores"), and lets say with this new attribute ("physics") the data would look like this
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
"physics": 95,
},
},
}
I am not sure how I could get that attribute added to the very last object.
I did recursive type checking and was able to get each field and print its value. But because maps are not referential I cannot add a value to the original map when I reach the map with the values that are not complex types.
package main
import "fmt"
func main() {
data := make(map[string]interface{})
data["person"] = map[string]interface{}{
"peter": map[string]interface{}{
"scores": map[string]interface{}{
"calculus": 88,
"algebra": 99,
"golang": 89,
},
},
}
parseMap(data)
}
func parseMap(aMap map[string]interface{}) interface{} {
var retVal interface{}
for _, val := range aMap {
switch val.(type) {
case map[string]interface{}:
retVal = parseMap(val.(map[string]interface{}))
//case []interface{}:
// retVal = parseArray(val.([]interface{}))
default:
//here i would have done aMap["physics"] = 95 if I could access the original map by reference, but that is not possible
retVal = aMap
}
}
return retVal
}
According the comments on the question, the goal is to set a value in the most deeply nested map.
Use the following function to find a map at the greatest nesting level. If there is more than one map at the greatest nesting level, this function returns an arbitrary one of those maps.
func findDeepest(m map[string]interface{}) (int, map[string]interface{}) {
depth := 0
candidate := m
for _, v := range m {
if v, ok := v.(map[string]interface{}); ok {
d, c := findDeepest(v)
if d+1 > depth {
depth = d + 1
candidate = c
}
}
}
return depth, candidate
}
Use it like this to set a value in the deeply nested map:
_, m := findDeepest(data)
m["physics"] = 95
Run it on the playground.
Try to avoid working with the raw map[string]interface{} type as much as you can. The Go encoding/json file can deal with string-keyed maps just fine, and hopefully the remote API has some sort of specification for what you're dealing with. (You know that you're expecting a person top-level key and scores in a specific point in the hierarchy, for example.)
I'm assuming the remote API is JSON-over-HTTP. You might model its structure as
type Input struct {
Person map[string]Person `json:"person"`
}
type Person struct {
Scores map[string]int `json:"scores"`
}
Once you've json.Unmarshal()ed data into this structure, you can directly set
data.Person["peter"].Scores["physics"] = 95
and then json.Marshal() the result again. https://play.golang.org/p/qoAVFodSvK2 has a complete example.
If you really wanted to directly manipulate the map[string]interface{} structure, I'd suggest splitting each "level" into a separate function call
func ParseTopLevel(data map[string]interface{}) {
switch peter := data["peter"].(type) {
case map[string]interface{}:
ParsePeter(peter)
}
}
map types are passed by reference, so when you get to the bottom of the stack you can directly set scores["physics"] = 95. (In your original code, I'd be surprised if you can't directly set aMap["physics"] as you propose, though it's rather imprecise on what gets set; compare https://play.golang.org/p/VuTjcjezwwU.)
I just want a function that having a slice of a struct type "t", returns the returns the element I'm looking for and the remaining, I tried with the partial solution for my problem like pointed out here:
Delete element in a slice
But for a weird reason, it does not work as expected
https://play.golang.org/p/tvJwkF5c_tj
func main() {
var names = []string{"john", "julio", "pepito","carlos"}
fmt.Println(getMe("john", names))
}
func getMe(me string, names []string) (string, []string, bool) {
for i := range names {
if names[i] == me {
return names[i], append(names[:i], names[i+1:]...), true
}
}
return "", nil, false
}
but the result gives me:
julio [julio pepito carlos] true
UPDATE:
https://play.golang.org/p/1xbu01rOiMg
Taking the answer from #Ullaakut
If I do: append(names[:i], names[i+1:]...), it changes the original slice, so this does not work for me, I do not want my slice to change, because I will be using it later on
Simply use the range to get both the value and the index, instead of accessing the value by using the index.
package main
import (
"fmt"
)
func main() {
var names = []string{"john", "julio", "pepito", "carlos"}
name, newNames, _ := getMe("john", names)
fmt.Println("extracted name:\t\t\t\t", name)
fmt.Println("new slice without extracted name:\t", newNames)
fmt.Println("old slice still intact:\t\t\t", names)
}
func getMe(me string, names []string) (string, []string, bool) {
var newSlice []string
for i := 0; i < len(names); i++ {
if names[i] == me {
newSlice = append(newSlice, names[:i]...)
newSlice = append(newSlice, names[i+1:]...)
return names[i], newSlice, true
}
}
return "", nil, false
}
Outputs
extracted name: john
new slice without extracted name: [julio pepito carlos]
old slice still intact: [john julio pepito carlos]
See playground example
Edit after request for a faster version: Using the manual for instead of the range loop is much faster. Since you need to create a new slice without the element, it's necessary to build a new slice within the function, which is always going to take some processing power.
In Go, I am trying to make a scramble slice function for my traveling salesman problem. While doing this I noticed when I started editing the slice I gave the scramble function was different every time I passed it in.
After some debugging I found out it was due to me editing the slice inside the function. But since Go is supposed to be a "pass by value" language, how is this possible?
https://play.golang.org/p/mMivoH0TuV
I have provided a playground link to show what I mean.
By removing line 27 you get a different output than leaving it in, this should not make a difference since the function is supposed to make its own copy of the slice when passed in as an argument.
Can someone explain the phenomenon?
Everything in Go is passed by value, slices too. But a slice value is a header, describing a contiguous section of a backing array, and a slice value only contains a pointer to the array where the elements are actually stored. The slice value does not include its elements (unlike arrays).
So when you pass a slice to a function, a copy will be made from this header, including the pointer, which will point to the same backing array. Modifying the elements of the slice implies modifying the elements of the backing array, and so all slices which share the same backing array will "observe" the change.
To see what's in a slice header, check out the reflect.SliceHeader type:
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
See related / possible duplicate question:
Performance of function slice parameter vs global variable?
Read blog post: Go Slices: usage and internals
Please note that when you pass a slice to a function, if the function modifies the "existing" elements of the slice, the caller will see / observe the changes. If the function adds new elements to the slice, that requires changing the slice header (the length at a minimum, but may also involve allocating a new backing array), which the caller will not see (not without returning the new slice header).
Not with maps, because maps are pointers under the hood, and if you pass a map to a function and the function adds a new entry to the map, the map pointer will not change so the caller will see the changed map (the new entry) without returning the map after change.
Also regarding slices and maps, see Map initialization in Go and why slice values can sometimes go stale but never map values?
You can find an example below. Briefly slices is also passed by value but original slice and copied slice are linked to the same underlying array. If one of this slice changes, then underlying array changes, then other slice changes.
package main
import "fmt"
func main() {
x := []int{1, 10, 100, 1000}
double(x)
fmt.Println(x) // ----> 3 will print [2, 20, 200, 2000] (original slice changed)
}
func double(y []int) {
fmt.Println(y) // ----> 1 will print [1, 10, 100, 1000]
for i := 0; i < len(y); i++ {
y[i] *= 2
}
fmt.Println(y) // ----> 2 will print [2, 20, 200, 2000] (copy slice + under array changed)
}
Slices when its passed it’s passed with the pointer to underlying array, so a slice is a small structure that points to an underlying array. The small structure is copied, but it still points to the same underlying array. the memory block containing the slice elements is passed by "reference". The slice information triplet holding the capacity, the number of element and the pointer to the elements is passed by value.
The best way to handle slices passing to function (if the elements of the slice are manipulated into the function, and we do not want this to be reflected at the elements memory block is to copy them using copy(s, *c) as:
package main
import "fmt"
type Team []Person
type Person struct {
Name string
Age int
}
func main() {
team := Team{
Person{"Hasan", 34}, Person{"Karam", 32},
}
fmt.Printf("original before clonning: %v\n", team)
team_cloned := team.Clone()
fmt.Printf("original after clonning: %v\n", team)
fmt.Printf("clones slice: %v\n", team_cloned)
}
func (c *Team) Clone() Team {
var s = make(Team, len(*c))
copy(s, *c)
for index, _ := range s {
s[index].Name = "change name"
}
return s
}
But be careful, if this slice is containing a sub slice further copying is required, as we'll still have the sub slice elements sharing pointing to the same memory block elements, an example is:
type Inventories []Inventory
type Inventory struct { //instead of: map[string]map[string]Pairs
Warehouse string
Item string
Batches Lots
}
type Lots []Lot
type Lot struct {
Date time.Time
Key string
Value float64
}
func main() {
ins := Inventory{
Warehouse: "DMM",
Item: "Gloves",
Batches: Lots{
Lot{mustTime(time.Parse(custom, "1/7/2020")), "Jan", 50},
Lot{mustTime(time.Parse(custom, "2/1/2020")), "Feb", 70},
},
}
inv2 := CloneFrom(c Inventories)
}
func (i *Inventories) CloneFrom(c Inventories) {
inv := new(Inventories)
for _, v := range c {
batches := Lots{}
for _, b := range v.Batches {
batches = append(batches, Lot{
Date: b.Date,
Key: b.Key,
Value: b.Value,
})
}
*inv = append(*inv, Inventory{
Warehouse: v.Warehouse,
Item: v.Item,
Batches: batches,
})
}
(*i).ReplaceBy(inv)
}
func (i *Inventories) ReplaceBy(x *Inventories) {
*i = *x
}
Slice will work with pass by value to the function, But we should not use append to add values to slice in the function, instead we should use the assignment directly. Reason being that append will create new memory and copy values to that. Here is the example.
Go playground
// Go program to illustrate how to
// pass a slice to the function
package main
import "fmt"
// Function in which slice
// is passed by value
func myfun(element []string) {
// Here we only modify the slice
// Using append function
// Here, this function only modifies
// the copy of the slice present in
// the function not the original slice
element = append(element, "blackhole")
fmt.Println("Modified slice: ", element)
}
func main() {
// Creating a slice
slc := []string{"rocket", "galaxy", "stars", "milkyway"}
fmt.Println("Initial slice: ", slc)
//slice pass by value
myfun(slc)
fmt.Println("Final slice: ", slc)
}
Output-
Initial slice: [rocket galaxy stars milkyway]
Modified slice: [rocket galaxy stars milkyway blackhole]
Final slice: [rocket galaxy stars milkyway]
Go Playground
// Go program to illustrate how to
// pass a slice to the function
package main
import "fmt"
// Function in which slice
// is passed by value
func myfun(element []string) {
// Here we only modify the slice
// Using append function
// Here, this function only modifies
// the copy of the slice present in
// the function not the original slice
element[0] = "Spaceship"
element[4] = "blackhole"
element[5] = "cosmos"
fmt.Println("Modified slice: ", element)
}
func main() {
// Creating a slice
slc := []string{"rocket", "galaxy", "stars", "milkyway", "", ""}
fmt.Println("Initial slice: ", slc)
//slice pass by value
myfun(slc)
fmt.Println("Final slice: ", slc)
}
Output-
Initial slice: [rocket galaxy stars milkyway ]
Modified slice: [Spaceship galaxy stars milkyway blackhole cosmos]
Final slice: [Spaceship galaxy stars milkyway blackhole cosmos]
To complement this post, here is an example of passing by reference for the Golang PlayGround you shared:
type point struct {
x int
y int
}
func main() {
data := []point{{1, 2}, {3, 4}, {5, 6}, {7, 8}}
makeRandomDatas(&data)
}
func makeRandomDatas(dataPoints *[]point) {
for i := 0; i < 10; i++ {
if len(*dataPoints) > 0 {
fmt.Println(makeRandomData(dataPoints))
} else {
fmt.Println("no more elements")
}
}
}
func makeRandomData(cities *[]point) []point {
solution := []point{(*cities)[0]} //create a new slice with the first item from the old slice
*cities = append((*cities)[:0], (*cities)[1:]...) //remove the first item from the old slice
return solution
}
Here is example code:
package main
import (
"fmt"
)
type Product struct {
Id int64
Title string
AttrVals []string
}
type ProductAttrValView struct {
Product
Attr string
}
type ProductAttrVal struct {
Attr string
Product int64
Value string
}
func main() {
p := Product{Id: 1, Title: "test", AttrVals: []string{}}
var prod *Product
prodViews := []ProductAttrValView{
ProductAttrValView{ Product: p, Attr: "text1" },
ProductAttrValView{ Product: p, Attr: "text2" },
ProductAttrValView{ Product: p, Attr: "text3" },
ProductAttrValView{ Product: p, Attr: "text4" },
}
// collapse join View to Product with Attrs
for _, pview := range prodViews {
if prod == nil {
prod = &pview.Product
prod.AttrVals = make([]string, 0, len(prodViews))
}
if pview.Attr != "" {
fmt.Printf("appending '%s' to %p\n", pview.Attr, prod) // output for debug
prod.AttrVals = append(prod.AttrVals, pview.Attr)
}
}
fmt.Printf("%+v\n", prod) // output for debug
}
http://play.golang.org/p/949w5tYjcH
Here i have some returned data from DB in ProductAttrValView struct and want
put it into Product struct and also fill Product.AttrVals
It prints:
&{Id:1 Title:test AttrVals:[text4]}
While i expect this:
&{Id:1 Title:test AttrVals:[text1 text2 text3 text4]}
So, all text should be appended, but for some reason only the last element stays in Attrs slice.
You are re-using variables in your for-range loop, and each iteration modifies the value of that same variable. You can create a new value each iteration with the trick:
pview := pview
http://play.golang.org/p/qtJXxdtuq2
You also initialize the slice with a length of 4, but you append another value (ignoring the first 4). You likely meant to set the capacity of the slice as opposed to the length:
prod.AttrVals = make([]string, 0, len(prodViews))
Because the value of prod is changing each iteration, the code would be a lot less confusing if you specifically initialized the prod value, instead of assigning the address of &pview.Product
prod = &Product{AttrVals: make([]string, 0, len(prodViews))}
[time line]
You create a single product p, containing an initialized []string
That same p is assigned to all prodViews that we will iterate over.
On the first iteration through the loop, you assign *prod to that initial p value, then change AttrVals to a new []string of length 4. This doesn't alter the original p.
pview.Attr is appended to prod.AttrVals, making a it length 5, and creating a new underlying array. This isn't reflected in the values of p.
On subsequent iterations, pview is overwritten with the next value in prodViews. This overwrites the value of prod too, since it points to the pview.Product value. This means that prod.AttrVals is now the same one from the original p.
pview.Attr is appended to a slice of length 0, and the underlying array is replaced with more capacity, so pview.Attr still isn't contained in the original p.
pview is again overwritten with the next value, which contains original p values, setting your AttrVals length back to 0 with an empty array.
The cycle continues until the final single value is printed.