A related questions is here https://stackoverflow.com/a/12965872/6421681.
In go, you can do:
func numsInFactorial(n int) (nums []int) {
// `nums := make([]int)` is not needed
for i := 1; i <= n; i++ {
nums = append(nums, i)
}
return
}
However,the following doesn't work:
func mapWithOneKeyAndValue(k int, v int) (m map[int]int) {
m[k] = v
return
}
An error is thrown:
panic: assignment to entry in nil map
Instead, you must:
func mapWithOneKeyAndValue(k int, v int) map[int]int {
m := make(map[int]int)
m[k] = v
return
}
I can't find the documentation for this behavior.
I have read through all of effective go, and there's no mention of it there either.
I know that named return values are defined (i.e. memory is allocated; close to what new does) but not initialized (so make behavior isn't replicated).
After some experimenting, I believe this behavior can be reduced into understanding the behavior of the following code:
func main() {
var s []int // len and cap are both 0
var m map[int]int
fmt.Println(s) // works... prints an empty slice
fmt.Println(m) // works... prints an empty map
s = append(s, 10) // returns a new slice, so underlying array gets allocated
fmt.Println(s) // works... prints [10]
m[10] = 10 // program crashes, with "assignment to entry in nil map"
fmt.Println(m)
}
The issue seems that append likely calls make and allocates a new slice detecting that the capacity of s is 0. However, map never gets an explicit initialization.
The reason for this SO question is two-pronged. First, I would like to document the behavior on SO. Second, why would the language allow non-initializing definitions of slice and map? With my experience with go so far, it seems to be a pragmatic language (i.e. unused variables lead to compilation failure, gofmt forces proper formatting), so it would make sense for it to prevent the code from compiling.
Try to assign in nil slice by index - you will get "panic: runtime error: index out of range" (example: https://play.golang.org/p/-XHh1jNyn5g)
The only reason why append function works with nil, is that append function can do reallocation for the given slice.
For example, if you trying to to append 6th element to slice of 5 elements with current capacity 5, it will create the new array with new capacity, copy all the info from old one, and swap the data array pointers in the given slice. In my understanding, it is just golang implementation of dynamic arrays.
So, the nil slice is just a special case of slice with not enough capacity, so it would be reallocated on any append operation.
More details on https://blog.golang.org/go-slices-usage-and-internals
From https://blog.golang.org/go-maps-in-action
A nil map behaves like an empty map when reading, but attempts to write to a nil map will cause a runtime panic; don't do that. To initialize a map, use the built in make function
It seems like a nil map is considered a valid empty map and that's the reason they don't allocate memory for it automatically.
Related
I'm looking for something like the c++ function .clear() for the primitive type map.
Or should I just create a new map instead?
Update: Thank you for your answers. By looking at the answers I just realized that sometimes creating a new map may lead to some inconsistency that we don't want. Consider the following example:
var a map[string]string
var b map[string]string
func main() {
a = make(map[string]string)
b=a
a["hello"]="world"
a = nil
fmt.Println(b["hello"])
}
I mean, this is still different from the .clear() function in c++, which will clear the content in the object.
You should probably just create a new map. There's no real reason to bother trying to clear an existing one, unless the same map is being referred to by multiple pieces of code and one piece explicitly needs to clear out the values such that this change is visible to the other pieces of code.
So yeah, you should probably just say
mymap = make(map[keytype]valtype)
If you do really need to clear the existing map for whatever reason, this is simple enough:
for k := range m {
delete(m, k)
}
Unlike C++, Go is a garbage collected language. You need to think things a bit differently.
When you make a new map
a := map[string]string{"hello": "world"}
a = make(map[string]string)
the original map will be garbage-collected eventually; you don't need to clear it manually. But remember that maps (and slices) are reference types; you create them with make(). The underlying map will be garbage-collected only when there are no references to it.
Thus, when you do
a := map[string]string{"hello": "world"}
b := a
a = make(map[string]string)
the original array will not be garbage collected (until b is garbage-collected or b refers to something else).
// Method - I , say book is name of map
for k := range book {
delete(book, k)
}
// Method - II
book = make(map[string]int)
// Method - III
book = map[string]int{}
Go 1.18 and above
You can use maps.Clear. The function belongs to the package golang.org/x/exp/maps (experimental and not covered by the compatibility guarantee)
Clear removes all entries from m, leaving it empty.
Example usage:
func main() {
testMap := map[string]int{"gopher": 1, "badger": 2}
maps.Clear(testMap)
fmt.Println(testMap)
testMap["zebra"] = 2000
fmt.Println(testMap)
}
Playground: https://go.dev/play/p/qIdnGrd0CYs?v=gotip
If you don't want to depend on experimental packages, you can copy-paste the source, which is actually extremely simple:
func Clear[M ~map[K]V, K comparable, V any](m M) {
for k := range m {
delete(m, k)
}
}
IMPORTANT NOTE: just as with the builtin delete — which the implementation of maps.Clear uses —, this does not remove irreflexive keys from the map. The reason is that for irreflexive keys, by definition, x == x is false. Irreflexive keys are NaN floats and every other type that supports comparison operators but contains NaN floats somewhere.
See this code to understand what this entails:
func main() {
m := map[float64]string{}
m[1.0] = "foo"
k := math.NaN()
fmt.Println(k == k) // false
m[k] = "bar"
maps.Clear(m)
fmt.Printf("len: %d, content: %v\n", len(m), m)
// len: 1, content: map[NaN:bar]
a := map[[2]float64]string{}
a[[2]float64{1.0, 2.0}] = "foo"
h := [2]float64{1.0, math.NaN()}
fmt.Println(h == h) // false
a[h] = "bar"
maps.Clear(a)
fmt.Printf("len: %d, content: %v\n", len(a), a)
// len: 1, content: map[[1 NaN]:bar]
}
Playground: https://go.dev/play/p/LWfiD3iPA8Q
A clear builtin is being currently discussed (Autumn 2022) that, if added to next Go releases, will delete also irreflexive keys.
For the method of clearing a map in Go
for k := range m {
delete(m, k)
}
It only works if m contains no key values containing NaN.
delete(m, k) doesn't work for any irreflexive key (such as math.NaN()), but also structs or other comparable types with any NaN float in it. Given struct{ val float64 } with NaN val is also irreflexive (Quote by blackgreen comment)
To resolve this issue and support clearing a map in Go, one buildin clear(x) function could be available in the new release, for more details, please refer to add clear(x) builtin, to clear map, zero content of slice, ptr-to-array
If you are trying to do this in a loop, you can take advantage of the initialization to clear out the map for you. For example:
for i:=0; i<2; i++ {
animalNames := make(map[string]string)
switch i {
case 0:
animalNames["cat"] = "Patches"
case 1:
animalNames["dog"] = "Spot";
}
fmt.Println("For map instance", i)
for key, value := range animalNames {
fmt.Println(key, value)
}
fmt.Println("-----------\n")
}
When you execute this, it clears out the previous map and starts with an empty map. This is verified by the output:
$ go run maptests.go
For map instance 0
cat Patches
-----------
For map instance 1
dog Spot
-----------
I'm trying to learn more regarding memory usage.
Doing some tests with interface{} and struct{} slices, I noticed that a slice of struct{} doesn't allocate any memory whereas a slice of interface{} does. It doesn't make so much sense to me, I'm actually expecting the same behavior (ie. both allocate nothing). Anyway I couldn't find any explanation regarding this particular case.
Could someone explain me why this happens?
package main
import (
"runtime"
"fmt"
)
func main() {
// Below is an example of using our PrintMemUsage() function
// Print our starting memory usage (should be around 0mb)
fmt.Println("Start")
PrintMemUsage()
fmt.Println("")
structContainer := make([]struct{}, 1000000)
for i := 0; i<1000000; i++ {
structContainer[i] = struct{}{}
}
fmt.Println("With 1kk struct{}")
PrintMemUsage()
fmt.Println("")
nilContainer := make([]interface{}, 1000000)
for i := 0; i<1000000; i++ {
nilContainer[i] = nil
}
fmt.Println("With 1kk nil interface{}")
PrintMemUsage()
fmt.Println("")
}
// PrintMemUsage outputs the current, total and OS memory being used. As well as the number
// of garage collection cycles completed.
func PrintMemUsage() {
var m runtime.MemStats
runtime.ReadMemStats(&m)
// For info on each, see: https://golang.org/pkg/runtime/#MemStats
fmt.Printf("Alloc = %v KiB", bToMb(m.Alloc))
fmt.Printf("\tTotalAlloc = %v KiB", bToMb(m.TotalAlloc))
fmt.Printf("\tSys = %v KiB", bToMb(m.Sys))
fmt.Printf("\tNumGC = %v\n", m.NumGC)
}
func bToMb(b uint64) uint64 {
return b / 1024
}
Playground link.
A variable of type interface{} can hold any value. E.g. it can hold the integer 8, it can hold the string value "hi", it can hold the struct value image.Point{X: 1, Y: 2} and pretty much everything else.
If you allocate a slice having interface{} as its element type, memory have to be allocated so that you can store any values in its elements. When using make() to allocate it, all its elements will get the zero value of the element type (which is nil for the interface{}), but memory still has to be allocated else you couldn't set elements later on.
On the other hand, the empty struct struct{} has no fields, it cannot hold any values (other than struct{}). When you allocate a slice having struct{} as its element type, memory does not need to be allocated because you won't be able to store anything in it that would require memory. So it's a simple and clever optimization not to allocate memory for such a type.
This is because an empty struct contains no value.
This is not very useful for arrays or slices. But it is useful for maps. A map without value is like a set. You can insert keys and test if they are present. The absence of value save space as you discovered.
I know everything is passed by value in Go, meaning if I give a slice to a function and that function appends to the slice using the builtin append function, then the original slice will not have the values that were appended in the scope of the function.
For instance:
nums := []int{1, 2, 3}
func addToNumbs(nums []int) []int {
nums = append(nums, 4)
fmt.Println(nums) // []int{1, 2, 3, 4}
}
fmt.Println(nums) // []int{1, 2, 3}
This causes a problem for me, because I am trying to do recursion on an accumulated slice, basically a reduce type function except the reducer calls itself.
Here is an example:
func Validate(obj Validatable) ([]ValidationMessage, error) {
messages := make([]ValidationMessage, 0)
if err := validate(obj, messages); err != nil {
return messages, err
}
return messages, nil
}
func validate(obj Validatable, accumulator []ValidationMessage) error {
// If something is true, recurse
if something {
if err := validate(obj, accumulator); err != nil {
return err
}
}
// Append to the accumulator passed in
accumulator = append(accumulator, message)
return nil
}
The code above gives me the same error as the first example, in that the accumulator does not get all the appended values because they only exist within the scope of the function.
To solve this, I pass in a pointer struct into the function, and that struct contains the accumulator. That solution works nicely.
My question is, is there a better way to do this, and is my approach idiomatic to Go?
Updated solution (thanks to icza):
I just return the slice in the recursed function. Such a facepalm, should have thought of that.
func Validate(obj Validatable) ([]ValidationMessage, error) {
messages := make([]ValidationMessage, 0)
return validate(obj, messages)
}
func validate(obj Validatable, messages []ValidationMessage) ([]ValidationMessage, error) {
err := v.Struct(obj)
if _, ok := err.(*validator.InvalidValidationError); ok {
return []ValidationMessage{}, errors.New(err.Error())
}
if _, ok := err.(validator.ValidationErrors); ok {
messageMap := obj.Validate()
for _, err := range err.(validator.ValidationErrors) {
f := err.StructField()
t := err.Tag()
if v, ok := err.Value().(Validatable); ok {
return validate(v, messages)
} else if _, ok := messageMap[f]; ok {
if _, ok := messageMap[f][t]; ok {
messages = append(messages, ValidationMessage(messageMap[f][t]))
}
}
}
}
return messages, nil
}
If you want to pass a slice as a parameter to a function, and have that function modify the original slice, then you have to pass a pointer to the slice:
func myAppend(list *[]string, value string) {
*list = append(*list, value)
}
I have no idea if the Go compiler is naive or smart about this; performance is left as an exercise for the comment section.
For junior coders out there, please note that this code is provided without error checking. For example, this code will panic if list is nil.
Slice grows dynamically as required if the current size of the slice is not sufficient to append new value thereby changing the underlying array. If this new slice is not returned, your append change will not be visible.
Example:
package main
import (
"fmt"
)
func noReturn(a []int, data ...int) {
a = append(a, data...)
}
func returnS(a []int, data ...int) []int {
return append(a, data...)
}
func main() {
a := make([]int, 1)
noReturn(a, 1, 2, 3)
fmt.Println(a) // append changes will not visible since slice size grew on demand changing underlying array
a = returnS(a, 1, 2, 3)
fmt.Println(a) // append changes will be visible here since your are returning the new updated slice
}
Result:
[0]
[0 1 2 3]
Note:
You don't have to return the slice if you are updating items in the slice without adding new items to slice
Slice you passed is an reference to an array, which means the size is fixed. If you just modified the stored values, that's ok, the value will be updated outside the called function.
But if you added new element to the slice, it will reslice to accommodate new element, in other words, a new slice will be created and old slice will not be overwritten.
As a summary, if you need to extend or cut the slice, pass the pointer to the slice.Otherwise, use slice itself is good enough.
Update
I need to explain some important facts. For adding new elements to a slice which was passed as a value to a function, there are 2 cases:
A
the underlying array reached its capacity, a new slice created to replace the origin one, obviously the origin slice will not be modified.
B
the underlying array has not reached its capacity, and was modified. BUT the field len of the slice was not overwritten because the slice was passed by value. As a result, the origin slice will not aware its len was modified, which result in the slice not modified.
When appending data into slice, if the underlying array of the slice doesn't have enough space, a new array will be allocated. Then the elements in old array will be copied into this new memory, accompanied with adding new data behind
I am trying to create a generic function that can handle actions on slices in Go... for instance, append an item of any type to a slice of that same type. This is simply a generic purpose for a more complex solution, but overall the issue boils down to this example:
package main
type car struct {
make string
color string
}
type submarine struct {
name string
length int
}
func genericAppender(thingList interface{}, thing interface{}) []interface{} {
return append(thingList, thing)
}
func main() {
cars := make([]car, 0, 10)
cars[0] = car{make: "ford", color: "red"}
cars[1] = car{make: "chevy", color: "blue"}
subs := make([]submarine, 0, 10)
subs[0] = submarine{name: "sally", length: 100}
subs[1] = submarine{name: "matilda", length: 200}
newCar := car{make: "bmw", color: "white"}
genericAppender(&cars, newCar)
}
The code playground is at this location
The above errors as follows:
prog.go:14: first argument to append must be slice; have interface {}
After this change you're still getting a runtime error (index out of range) however the problem is that thingList is not of type []interface{} but rather interface{} so you can't append to it. Here's an updated version of your code on playground that does a type assertion to convert it to an []interface{} in line with the append. In reality you need to do that on a separate line and check for errors.
https://play.golang.org/p/YMed0VDZrv
So to put some code here;
func genericAppender(thingList interface{}, thing interface{}) []interface{} {
return append(thingList.([]interface{}), thing)
}
will solve the basic problem you're facing. As noted, you still get runtime errors when indexing into the slice. Also, you could change the argument to avoid this by making it;
func genericAppender(thingList []interface{}, thing interface{}) []interface{} {
return append(thingList, thing)
}
Here's a complete example of the second type; https://play.golang.org/p/dIuW_UG7XY
Note I also corrected the runtime error. When you use make with 3 args they are, in this order, type, length, capacity. This means the length of the array is 0 so when you try to assign to indexes 0 and 1 it was causing a panic for IndexOutoFRange. Instead I removed the middle argument so it's make([]interface{}, 10) meaning the length is initially set to 10 so you can assign to those indexes.
In the answer above if you do the following then it throws error. This is what the original question was about:
//genericAppender(subs, newCar). // Throws "cannot use subs (type []submarine) as type []interface {} in argument to genericAppender"
The trick is to convert your slice of specific type into a generic []interface{}.
func convertToGeneric(thingList interface{}) []interface{} {
input := reflect.ValueOf(thingList)
length := input.Len()
out := make([]interface{},length)
for i:=0 ;i < length; i++ {
out[i] = input.Index(i).Interface()
}
return out
}
This you can call the function like this:
genericAppender(convertToGeneric(subs), newCar)
You can check modified working code here: https://play.golang.org/p/0_Zmme3c8lT
With Go 1.19 (Q4 2022), no need for interface, or "convert your slice of specific type into a generic []interface{}"
CL 363434 comes with a new slices packages:
// Package slices defines various functions useful with slices of any type.
// Unless otherwise specified, these functions all apply to the elements
// of a slice at index 0 <= i < len(s).
package slices
import "constraints"
// Grow increases the slice's capacity, if necessary, to guarantee space for
// another n elements. After Grow(n), at least n elements can be appended
// to the slice without another allocation. If n is negative or too large to
// allocate the memory, Grow panics.
func Grow[S ~[]T, T any](s S, n int) S {
return append(s, make(S, n)...)[:len(s)]
}
// Equal reports whether two slices are equal: the same length and all
// elements equal. If the lengths are different, Equal returns false.
// Otherwise, the elements are compared in index order, and the
// comparison stops at the first unequal pair.
// Floating point NaNs are not considered equal.
func Equal[T comparable](s1, s2 []T) bool {
if len(s1) != len(s2) {
return false
}
for i, v1 := range s1 {
v2 := s2[i]
if v1 != v2 {
return false
}
}
return true
}
// ...
Ian Lance Taylor confirms in issue 45955:
This package is now available at golang.org/x/exp/slices.
Per this thread, it will not be put into standard library until the 1.19 release.
We may of course adjust it based on anything we learn about having it in x/exp.
I'm looking for something like the c++ function .clear() for the primitive type map.
Or should I just create a new map instead?
Update: Thank you for your answers. By looking at the answers I just realized that sometimes creating a new map may lead to some inconsistency that we don't want. Consider the following example:
var a map[string]string
var b map[string]string
func main() {
a = make(map[string]string)
b=a
a["hello"]="world"
a = nil
fmt.Println(b["hello"])
}
I mean, this is still different from the .clear() function in c++, which will clear the content in the object.
You should probably just create a new map. There's no real reason to bother trying to clear an existing one, unless the same map is being referred to by multiple pieces of code and one piece explicitly needs to clear out the values such that this change is visible to the other pieces of code.
So yeah, you should probably just say
mymap = make(map[keytype]valtype)
If you do really need to clear the existing map for whatever reason, this is simple enough:
for k := range m {
delete(m, k)
}
Unlike C++, Go is a garbage collected language. You need to think things a bit differently.
When you make a new map
a := map[string]string{"hello": "world"}
a = make(map[string]string)
the original map will be garbage-collected eventually; you don't need to clear it manually. But remember that maps (and slices) are reference types; you create them with make(). The underlying map will be garbage-collected only when there are no references to it.
Thus, when you do
a := map[string]string{"hello": "world"}
b := a
a = make(map[string]string)
the original array will not be garbage collected (until b is garbage-collected or b refers to something else).
// Method - I , say book is name of map
for k := range book {
delete(book, k)
}
// Method - II
book = make(map[string]int)
// Method - III
book = map[string]int{}
Go 1.18 and above
You can use maps.Clear. The function belongs to the package golang.org/x/exp/maps (experimental and not covered by the compatibility guarantee)
Clear removes all entries from m, leaving it empty.
Example usage:
func main() {
testMap := map[string]int{"gopher": 1, "badger": 2}
maps.Clear(testMap)
fmt.Println(testMap)
testMap["zebra"] = 2000
fmt.Println(testMap)
}
Playground: https://go.dev/play/p/qIdnGrd0CYs?v=gotip
If you don't want to depend on experimental packages, you can copy-paste the source, which is actually extremely simple:
func Clear[M ~map[K]V, K comparable, V any](m M) {
for k := range m {
delete(m, k)
}
}
IMPORTANT NOTE: just as with the builtin delete — which the implementation of maps.Clear uses —, this does not remove irreflexive keys from the map. The reason is that for irreflexive keys, by definition, x == x is false. Irreflexive keys are NaN floats and every other type that supports comparison operators but contains NaN floats somewhere.
See this code to understand what this entails:
func main() {
m := map[float64]string{}
m[1.0] = "foo"
k := math.NaN()
fmt.Println(k == k) // false
m[k] = "bar"
maps.Clear(m)
fmt.Printf("len: %d, content: %v\n", len(m), m)
// len: 1, content: map[NaN:bar]
a := map[[2]float64]string{}
a[[2]float64{1.0, 2.0}] = "foo"
h := [2]float64{1.0, math.NaN()}
fmt.Println(h == h) // false
a[h] = "bar"
maps.Clear(a)
fmt.Printf("len: %d, content: %v\n", len(a), a)
// len: 1, content: map[[1 NaN]:bar]
}
Playground: https://go.dev/play/p/LWfiD3iPA8Q
A clear builtin is being currently discussed (Autumn 2022) that, if added to next Go releases, will delete also irreflexive keys.
For the method of clearing a map in Go
for k := range m {
delete(m, k)
}
It only works if m contains no key values containing NaN.
delete(m, k) doesn't work for any irreflexive key (such as math.NaN()), but also structs or other comparable types with any NaN float in it. Given struct{ val float64 } with NaN val is also irreflexive (Quote by blackgreen comment)
To resolve this issue and support clearing a map in Go, one buildin clear(x) function could be available in the new release, for more details, please refer to add clear(x) builtin, to clear map, zero content of slice, ptr-to-array
If you are trying to do this in a loop, you can take advantage of the initialization to clear out the map for you. For example:
for i:=0; i<2; i++ {
animalNames := make(map[string]string)
switch i {
case 0:
animalNames["cat"] = "Patches"
case 1:
animalNames["dog"] = "Spot";
}
fmt.Println("For map instance", i)
for key, value := range animalNames {
fmt.Println(key, value)
}
fmt.Println("-----------\n")
}
When you execute this, it clears out the previous map and starts with an empty map. This is verified by the output:
$ go run maptests.go
For map instance 0
cat Patches
-----------
For map instance 1
dog Spot
-----------