I'm still new to programming. Forgive my lack of computer science knowledge. Not sure if this question is specific to Golang or computer science in general...
I always thought that functions do not alter variables/data held outside their own scope unless you use a return statement back into the other scope, or unless they are higher in the hierarchy of scopes. One may argue that functions f1 and f2 in this example are called from a lower scope. However, this still doesn't explain why I'm getting different results for variable num and nums.
package main
import "fmt"
func f1(a int) {
a = 50 // this will not work, as it shouldn't
}
func f2(a ...int) {
a[0] = 50 // this will work without return statement
a[1] = 50 // this will work without return statement
}
func main() {
num := 2
nums := []int{2, 2}
f1(num)
f2(nums...)
fmt.Printf("function f1 doesn't affect the variable num and stays: %v\n", num)
fmt.Printf("function f2 affects the variable nums and results in: %v", nums)
Questions:
Why doesn't f2 require a return statement to modify nums like num
would within f1?
Golang functions are said to pass values (rather than reference),
shouldn't that force the function to return copies?
Can this happen in other languages? (I think I may have
seen this in other languages).
This is the correct behaviour, since a ...int is equal to a slice e.g.: a []int
func f2(a []int) {
a[0] = 50
a[1] = 50
}
func main() {
b := []int{2, 2}
f2(b)
fmt.Println(b) // [50 50]
}
And a slice is a view to the original data, here 'b'.
"Why doesn't f2 require a return statement to modify nums like num would in f1?"
In f2 you are using the slice, which has a pointer to the original array, so f2 can change the outside array.
"Golang functions are said to pass values (not reference), shouldn't that force to return copies? (If the question is related...)"
In f2 the slice itself is passed by value, meaning pointer and length and capacity of the original array.
"Can this happen in other languages? (I think I may have seen this in other langues)"
Too broad to answer, there are many languages and in general if you have a pointer to the outside world array, yes.
Edit:
package main
import "fmt"
func sum(a ...int) int {
s := 0
for _, v := range a {
s += v
}
return s
}
func f2(a []int) {
c := make([]int, len(a))
copy(c, a)
c[0] = 50
fmt.Println(sum(c...)) // 52
}
func main() {
b := []int{2, 2}
fmt.Println(sum(1, 2, 3, 4)) // 10
fmt.Println(sum(b...)) // 4
f2(b)
fmt.Println(b) // [2 2]
}
Notes:
The sum() function above is a pure function, since it has no side effect.
The new f2 function above is a pure function, since it has no side effect: it makes a copy of a into c then calls the sum.
In go, function arguments are passed by value. That means, if you pass an int (like in f1), compiler will pass the value of f1, essentially copying it. If the function takes a *int and you pass &num, then the compiler passes the value of &num, which is a pointer to num. When the function changes *num, the value of the variable outside the function will change. If the function changes num, the pointer value of num will change, and it will point to a different variable.
As a contrast, Java passes all primitive values as value, and all objects by reference. That is, if you pass an int, there is no way for the function to modify the value of that int that is visible to the caller. If you want to pass an int the function can modify, you put that in a class and pass an instance of that class in Java.
A slice (as in f2) contains a pointer to the underlying array. When you call a function with a slice, the slice header (containing a pointer to the underlying array) is copied, so when the function changes the slice elements, the underlying array elements change.
The question of scope is somewhat different. Scope of a function is all the variables it can see. Those are the global variables (if from different packages, exported global variables), function arguments, and if the function is declared nested within another function, all the variables visible in that function at that point.
1 & 2) Both questions can be answered when looking at how slices work in Go. There's a blog article on it.
In general, all variables are passed by value in Go. You can use pointers (e.g. *int for f1) to pass by reference (or more correct, the address of the pointer).
However, slices are technically also passed by value.
When we look here, we can get an idea how they work:
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
Len and Cap are integers, but Data is a pointer.
When this struct is copied (when passing by value), a copy of Len, Cap and Data will be made. Since Data is a pointer, any modifications made to the value it's pointing to will be visible after your function returns.
You can also read this
Related
In go, I'm trying to iterate through an array that was created in C. I have the length of the array (int arrc) and its pointer (mytype *arrv).
I have found a way, but it involves transferring back and forth between go and c and is super super hacky.
// void *nextelement(void *p, int i, int size) {
// return (void*)((uint64_t)p+i*size);
// }
#import "C"
...
for i := 0; i < protoc; i++ {
adr := (*C.mytype)(C.nextelement(unsafe.Pointer(myarr), C.int(i), C.sizeof_mytype))
All that code just to get myarr[i]... it doesn't feel right.
Note: this is simply copied from the CGo documentation.
Turning C arrays into Go slices
C arrays are typically either null-terminated or have a length kept
elsewhere.
Go provides the following function to make a new Go byte slice from a
C array:
func C.GoBytes(cArray unsafe.Pointer, length C.int) []byte
To create a Go slice backed by a C array (without copying the original
data), one needs to acquire this length at runtime and use a type
conversion to a pointer to a very big array and then slice it to the
length that you want (also remember to set the cap if you're using Go
1.2 or later), for example (see http://play.golang.org/p/XuC0xqtAIC for a runnable example):
var theCArray *C.YourType = C.getTheArray()
length := C.getTheArrayLength()
slice := (*[1 << 28]C.YourType)(unsafe.Pointer(theCArray))[:length:length]
It is important to keep in mind that the Go garbage collector will not
interact with this data, and that if it is freed from the C side of
things, the behavior of any Go code using the slice is
nondeterministic.
The reason for the magic array size constant is described in What does (*[1 << 30]C.YourType) do exactly in CGo? The actual value of the constant, whether it's 1 << 30 or 1 << 28 or whatever, is not crucial, but it must be at least as large as the largest length value will be.
Is there a way to make this golang code shorter?
func MergeSlices(s1 []float32, s2 []int32) []int {
var slice []int
for i := range s1 {
slice = append(slice, int(s1[i]))
}
for i := range s2 {
slice = append(slice, int(s2[i]))
}
return slice
}
You can't eliminate the loops to convert each element to int individually, because you can't convert whole slices of different element types. For explanation, see this question: Type converting slices of interfaces in go
The most you can do is use named result type, and a for range with 2 iteration values, where you can omit the first (the index) by assigning it to the blank identifier, and the 2nd will be the value:
func MergeSlices(s1 []float32, s2 []int32) (s []int) {
for _, v := range s1 {
s = append(s, int(v))
}
for _, v := range s2 {
s = append(s, int(v))
}
return
}
But know that your code is fine as-is. My code is not something to always follow, it was to answer your question: how to make your code shorter. If you want to improve your code, you could start by looking at its performance, or even refactoring your code to not end up needing to merge slices of different types.
Your code should be correct, maintainable, readable, and reasonably efficient. Note that shortness of code is not one of the important goals. For good reason, Stack Exchange has another site for Code Golf questions: Programming Puzzles & Code Golf.
Your code could be improved; it's inefficient. For example, merging two len(256) slices,
BenchmarkMergeSlices 200000 8350 ns/op 8184 B/op 10 allocs/op
Here's a more efficient (and longer) version:
BenchmarkMergeSlices 300000 4420 ns/op 4096 B/op 1 allocs/op
.
func MergeSlices(s1 []float32, s2 []int32) []int {
slice := make([]int, 0, len(s1)+len(s2))
for i := range s1 {
slice = append(slice, int(s1[i]))
}
for i := range s2 {
slice = append(slice, int(s2[i]))
}
return slice
}
Use the Go Code Review Comments for Named Result Parameters. For example: "Don't name result parameters just to avoid declaring a var inside the function; that trades off a minor implementation brevity at the cost of unnecessary API verbosity. Clarity of docs is always more important than saving a line or two in your function."
var s1 []int
var s2 []int
newSlice = append(s1, s2...)
The code can't get any shorter, but that's a goal of dubious value to begin with; it's not overly verbose as-is. You can, however, likely improve performance by eliminating the intermediate allocations. Every time you call append, if the target slice doesn't have enough space, it expands it, guessing at the necessary size since you haven't told it how much space it will need.
The simplest would just be to presize your target slice (replace var slice []int with slice := make([]int, 0, len(s1) + len(s2)); that way the appends never have to expand it. Setting the second parameter to 0 is important, that sets the length to zero, and the capacity to the total size needed, so that your appends will work as expected.
Once you've presized it though, you can get rid of the appends entirely, and directly set each index:
func MergeSlices(s1 []float32, s2 []int32) []int {
slice := make([]int, len(s1) + len(s2))
for i,v := range s1 {
slice[i] = int(v)
}
for i,v := range s2 {
slice[i+len(s1)] = int(v)
}
return slice
}
Playground link
Here is the code I've been trying to understand:
package main
import (
"fmt"
)
func squares() func() int {
var x int
return func() int {
x = x + 2
return x * x
}
}
func main() {
f := squares()
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares()())
fmt.Println(squares()())
fmt.Println(squares()())
}
The result we got:
4
16
36
4
4
4
My question is: Why does the value of x in fmt.Println(squares()()) stay unchanged?
Short version
You are building a new closure each time you call squares.
This is exactly as if you built a new Counter object in an object-oriented language:
new Counter().increment(); // 4
new Counter().increment(); // 4
...as opposed to:
c = new Counter();
c.increment(); // 4
c.increment(); // 16
Longer version
In your function, var x int declares a local variable x:
func squares() func() int {
var x int
return func() int {
x = x + 2
return x * x
}
}
As for any function, local variables are visible only inside the function. If you call a function in different contexts, each call has a separate set of memory storage addressable by your local symbols (here x). What happens when you return a function is that any binding currently visible in the scope of your code is kept alongside your function, which is then called a closure.
Closures can hold a state, like objects do. Thus your closure can refer to the local variables that were visible when it was created, even when you escape the block where local variables were introduced (thankfully, the GC is here to keep track of the memory associated with those variables).
When you define f, you create a fresh closure. Each time you call it you modify the same place referenced by the internal x variable. But if you create fresh closures and call them once each, then you won't see the same side-effects, because each x names a different place in memory.
The reason you are getting the same result when you are calling the second variant with closure function is a consequence of scope rules. All function values created by the new self invoking, or anonymous function, capture and share the same variable - an addressable storage location, not it's value at that particular moment.
The anonymous function introduce a new lexical block which shares the same logical address to which the function values points to, so each time you invoke the function without to enclose the function to a new scope it will share the same logical address. This is the reason why you'll see in many places an anonymous function called in this way:
for i := range mymap {
func(n int) {
fmt. Println(n)
}(i) // note the scope
}
To address your issue one way would be to use pointer variables, this way you will be absolute sure that you will share the variable allocated to the same memory address. Here is the updated and working code:
package main
import (
"fmt"
)
func squares() func(x *int) int {
return func(x *int) int {
*x = *x + 2
return *x * *x
}
}
func main() {
f := squares()
x := 0
fmt.Println(f(&x))
fmt.Println(f(&x))
fmt.Println(f(&x))
fmt.Println(squares()(&x))
fmt.Println(squares()(&x))
fmt.Println(squares()(&x))
}
Go Playground
Another way is to expose the variable x as a global variable. This will guarantee that you won't create a new variable x each time you run the anonymous function.
Here's a different perspective to answer your question.
First break down the squares() function first to understand what is going on:
func squares() func() int {
The above defines a func named squares that returns another function of type func() int (and that returns an int, but that's not the focus here).
Drill that into your head first: it returns a function.
Now, let's see what happens when we call squares():
var x int
That's it. It defines a varibale x, which defaults to value 0 per Go specs.
OK, now we have a variable in scope called x and it has a value of 0. Now, we return a function:
return func() int {
x = x + 2
return x * x
}
If we had not defined x earlier, this would be a build error because x must be defined. But, it was defined in the previous scope.
Go's use of closures allows another scope to be defined, which uses variables in the previous scope. var x int in this case.
Also, you are able to modify closures (variables in previous scope). And in this case, we are modifying the previous var x int that was defined in the previous scope.
Hold onto those thoughts for a moment, let's run some code...
f := squares()
Here we run squares(), which defines var x int as zero and returns a func() named f() that can do more work on x.
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
Since we continue to reuse f(), this keeps the scope of var x int on a stack, and that stack remains around as long as you have this f() variable. Therefore, x continues to retain its modified value and be modified.
With all of this knowledge, the answer to your question is simple: if you don't retain the scope, the f() definition like above, then you define a new var x int = 0 at every invitation of squares():
fmt.Println(squares()())
This invokes a new squares() at every call. And therefore, a new var x int = 0.
So the output of the returning func of squares, which is invoked with squares()(), always operates on var x int = 0, at every invokation since a new squares is called each time.
Simply because (1) x is a captured closure for the anonymous function and (2) the default value for type int is 0. So every time you call it, you will see the same output.
Let's rewrite the function squares like this:
func squares(initialValue int) func() int {
var x int
x = initialValue
return func() int {
x = x + 2
return x * x
}
}
Now for this:
func main() {
f := squares(0)
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares(0)())
fmt.Println(squares(0)())
fmt.Println(squares(0)())
}
We will see the exact output! Because we are initializing x with 0. If we use 1 as the initial value of x:
func main() {
f := squares(1)
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares(1)())
fmt.Println(squares(1)())
fmt.Println(squares(1)())
}
We will see this result:
9
25
49
9
9
9
As you can see, it's just about the initial value of x, which when not initialized explicitly, will have it's default value, zero.
I wanted to implement time based slots for holding data using golang slices. I managed to come up with a go program like this and it also works. But I have few questions regarding garbage collection and the general performance of this program. Does this program guarantee garbage collection of items once slice is equated to nil? And while shuffling slices, I hope this program does not do any deep copying.
type DataSlots struct {
slotDuration int //in milliseconds
slots [][]interface{}
totalDuration int //in milliseconds
}
func New(slotDur int, totalDur int) *DataSlots {
dat := &DataSlots{slotDuration: slotDur,
totalDuration: totalDur}
n := totalDur / slotDur
dat.slots = make([][]interface{}, n)
for i := 0; i < n; i++ {
dat.slots[i] = make([]interface{}, 0)
}
go dat.manageSlots()
return dat
}
func (self *DataSlots) addData(data interface{}) {
self.slots[0] = append(self.slots[0], data)
}
// This should be a go routine
func (self *DataSlots) manageSlots() {
n := self.totalDuration / self.slotDuration
for {
time.Sleep(time.Duration(self.slotDuration) * time.Millisecond)
for i := n - 1; i > 0; i-- {
self.slots[i] = self.slots[i-1]
}
self.slots[0] = nil
}
}
I removed critical section handling in this snippet to make it concise.
Once your slice is set too nil, any values contained in the slice are available for garbage collection, provided that the underlying array isn't shared with another slice.
Since there are no slice operations in your program, you never have multiple references to the same array, nor are you leaving data in any inaccessible portions of the underlying array.
What you need to be careful of, is when you're using slice operations:
a := []int{1, 2, 3, 4}
b := a[1:3]
a = nil
// the values 1 and 4 can't be collected, because they are
// still contained in b's underlying array
c := []int{1, 2, 3, 4}
c = append(c[1:2], 5)
// c is now []int{2, 5}, but again the values 1 and 4 are
// still in the underlying array. The 4 may be overwritten
// by a later append, but the 1 is inaccessible and won't
// be collected until the underlying array is copied.
While append does copy values when the capacity of the slice in insufficient, only the values contained in the slice are copied. There is no deep copy of any of the values.
The code snippet below is the library implementation of the push methods for a priority queue. I am wondering why the line with the code a = a[0 : n+1] does not throw an out of bounds errors.
func (pq *PriorityQueue) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
// To simplify indexing expressions in these methods, we save a copy of the
// slice object. We could instead write (*pq)[i].
a := *pq
n := len(a)
a = a[0 : n+1]
item := x.(*Item)
item.index = n
a[n] = item
*pq = a
}
a slice is not an array; it is a view onto an existing array. The slice in question is backed by an array larger than itself. When you define a slice of an existing slice, you're actually slicing the underlying array, but the indexes referenced are relative to the source slice.
That's a mouthful. Let's prove this in the following way: we'll create a slice of zero length, but we'll force the underlying array to be larger. When creating a slice with make, the third parameter will set the size of the underlying array. The expression make([]int, 0, 2) will allocate an array of size 2, but it evaluates to a size-zero slice.
package main
import ("fmt")
func main() {
// create a zero-width slice over an initial array of size 2
a := make([]int, 0, 2)
fmt.Println(a)
// expand the slice. Since we're not beyond the size of the initial
// array, this isn't out of bounds.
a = a[0:len(a)+1]
a[0] = 1
fmt.Println(a)
fmt.Println(a[0:len(a)+1])
}
see here. You can use the cap keyword to reference the size of the array that backs a given slice.
The specific code that you asked about loops over cap(pq) in the calling context (container/heap/example_test.go line 90). If you modify the code at the call site and attempt to push another item into the queue, it will panic like you expect. I ... probably wouldn't suggest writing code like this. Although the code in the standard library executes, I would be very sour if I found that in my codebase. It's generally safer to use the append keyword.
Because it works in a specific example program. Here are the important parts from the original/full example source)
const nItem = 10
and
pq := make(PriorityQueue, 0, nItem)
and
for i := 0; i < cap(pq); i++ {
item := &Item{
value: values[i],
priority: priorities[i],
}
heap.Push(&pq, item)
}
Is it an example from container/heap? If yes, then it doesn't throws an exception because capacity is big enough (see how the Push method is used). If you change the example to Push more items then the capacity, then it'll throw.
It does in general; it doesn't in the container/heap example. Here's the general fix I already gave you some time ago.
func (pq *PriorityQueue) Push(x interface{}) {
a := *pq
n := len(a)
item := x.(*Item)
item.index = n
a = append(a, item)
*pq = a
}
Golang solution to Project Euler problem #81