I have a question about the reader interface, the definition looks like:
type Reader interface {
Read(p []byte) (n int, err error)
}
I have following code that use the reader interface:
package main
import (
"fmt"
"os"
)
// Reading files requires checking most calls for errors.
// This helper will streamline our error checks below.
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
// You'll often want more control over how and what
// parts of a file are read. For these tasks, start
// by `Open`ing a file to obtain an `os.File` value.
f, err := os.Open("configuration.ini")
check(err)
// Read some bytes from the beginning of the file.
// Allow up to 5 to be read but also note how many
// actually were read.
b1 := make([]byte, 10)
n1, err := f.Read(b1)
check(err)
fmt.Printf("%d bytes: %s\n", n1, string(b1))
f.Close()
}
As you can see the code above, b1 is defined as byte slice and it passed to the Read method as value argument. After the Read method, the b1 contains the first 10 letters from file.
What for me very confusing about the code above is, why does b1 contains suddenly values after the Read method.
In Golang, when I pass a value to the method, it will be passed as value and not as reference. To clarify, what I talking about, I made a sample application:
package main
import (
"fmt"
)
func passAsValue(p []byte) {
c := []byte("Foo")
p = c
}
func main() {
b := make([]byte, 10)
passAsValue(b)
fmt.Println(string(b))
}
After passAsValue function, b does not contain any values and that what I expected in golang, arguments will be pass as value to the function or method.
Why then, the first code snippet can change the content of the passed argument? If the Read method expects a pointer of []byte slice, then I would be agreed, but on this case not.
Everything is passed by value (by creating a copy of the value being passed).
But since slices in Go are just descriptors for a contiguous segment of an underlying array, the descriptor will be copied which will refer to the same underlying array, so if you modify the contents of the slice, the same underlying array is modified.
If you modify the slice value itself in the function, that is not reflected at the calling place, because the slice value is just a copy and the copy will be modified (not the original slice descriptor value).
If you pass a pointer, the value of the pointer is also passed by value (the pointer value will be copied), but in this case if you modify the pointed value, that will be the same as at the calling place (the copy of the pointer and the original pointer points to the same object/value).
Related blog articles:
Go Slices: usage and internals
Arrays, slices (and strings): The mechanics of 'append'
The slice header in Go contains in itself a pointer to the underlaying array.
You can read from the official blog post: https://blog.golang.org/slices
Even though the slice header is passed by value, the header includes a pointer to elements of an array, so both the original slice header and the copy of the header passed to the function describe the same array. Therefore, when the function returns, the modified elements can be seen through the original slice variable.
It is the exact same behaviour as passing a pointer in C :
#include <stdio.h>
#include <stdlib.h>
// p is passed by value ; however, this function does not modify p,
// it modifies the values pointed by p.
void read(int* p) {
int i;
for( i=0; i<10; i++) {
p[i] = i+1;
}
}
// p is passed by value, so changing p itself has no effect out
// of the function's scope
void passAsValue(int*p) {
int* c = (int*)malloc(3*sizeof(int));
c[0] = 15; // 'F' in hex is 15 ...
c[1] = 0;
c[2] = 0;
p = c;
}
int main() {
int* p = (int*)malloc(10*sizeof(int));
int i;
for( i=0; i<10; i++) {
p[i] = 0;
}
printf(" init : p[0] = %d\n", p[0]);
read(p);
printf(" after read : p[0] = %d\n", p[0]);
passAsValue(p);
printf("after passAsValue : p[0] = %d\n", p[0]);
return 0;
}
output :
// init : p[0] = 0
// after read : p[0] = 1
//after passAsValue : p[0] = 1 // <- not 15, the modification from
// // within passAsValue is not persistent
(for the record : this C program leaks the int* c array)
A Go slice contains more info than just the pointer : it is a small struct, which contains the pointer, the length, and the max capacity of the allocated array (see the link mentioned in other answers : https://blog.golang.org/slices ).
But from the code's perspective, it behaves exactly like the C pointer.
Related
I'm still new to programming. Forgive my lack of computer science knowledge. Not sure if this question is specific to Golang or computer science in general...
I always thought that functions do not alter variables/data held outside their own scope unless you use a return statement back into the other scope, or unless they are higher in the hierarchy of scopes. One may argue that functions f1 and f2 in this example are called from a lower scope. However, this still doesn't explain why I'm getting different results for variable num and nums.
package main
import "fmt"
func f1(a int) {
a = 50 // this will not work, as it shouldn't
}
func f2(a ...int) {
a[0] = 50 // this will work without return statement
a[1] = 50 // this will work without return statement
}
func main() {
num := 2
nums := []int{2, 2}
f1(num)
f2(nums...)
fmt.Printf("function f1 doesn't affect the variable num and stays: %v\n", num)
fmt.Printf("function f2 affects the variable nums and results in: %v", nums)
Questions:
Why doesn't f2 require a return statement to modify nums like num
would within f1?
Golang functions are said to pass values (rather than reference),
shouldn't that force the function to return copies?
Can this happen in other languages? (I think I may have
seen this in other languages).
This is the correct behaviour, since a ...int is equal to a slice e.g.: a []int
func f2(a []int) {
a[0] = 50
a[1] = 50
}
func main() {
b := []int{2, 2}
f2(b)
fmt.Println(b) // [50 50]
}
And a slice is a view to the original data, here 'b'.
"Why doesn't f2 require a return statement to modify nums like num would in f1?"
In f2 you are using the slice, which has a pointer to the original array, so f2 can change the outside array.
"Golang functions are said to pass values (not reference), shouldn't that force to return copies? (If the question is related...)"
In f2 the slice itself is passed by value, meaning pointer and length and capacity of the original array.
"Can this happen in other languages? (I think I may have seen this in other langues)"
Too broad to answer, there are many languages and in general if you have a pointer to the outside world array, yes.
Edit:
package main
import "fmt"
func sum(a ...int) int {
s := 0
for _, v := range a {
s += v
}
return s
}
func f2(a []int) {
c := make([]int, len(a))
copy(c, a)
c[0] = 50
fmt.Println(sum(c...)) // 52
}
func main() {
b := []int{2, 2}
fmt.Println(sum(1, 2, 3, 4)) // 10
fmt.Println(sum(b...)) // 4
f2(b)
fmt.Println(b) // [2 2]
}
Notes:
The sum() function above is a pure function, since it has no side effect.
The new f2 function above is a pure function, since it has no side effect: it makes a copy of a into c then calls the sum.
In go, function arguments are passed by value. That means, if you pass an int (like in f1), compiler will pass the value of f1, essentially copying it. If the function takes a *int and you pass &num, then the compiler passes the value of &num, which is a pointer to num. When the function changes *num, the value of the variable outside the function will change. If the function changes num, the pointer value of num will change, and it will point to a different variable.
As a contrast, Java passes all primitive values as value, and all objects by reference. That is, if you pass an int, there is no way for the function to modify the value of that int that is visible to the caller. If you want to pass an int the function can modify, you put that in a class and pass an instance of that class in Java.
A slice (as in f2) contains a pointer to the underlying array. When you call a function with a slice, the slice header (containing a pointer to the underlying array) is copied, so when the function changes the slice elements, the underlying array elements change.
The question of scope is somewhat different. Scope of a function is all the variables it can see. Those are the global variables (if from different packages, exported global variables), function arguments, and if the function is declared nested within another function, all the variables visible in that function at that point.
1 & 2) Both questions can be answered when looking at how slices work in Go. There's a blog article on it.
In general, all variables are passed by value in Go. You can use pointers (e.g. *int for f1) to pass by reference (or more correct, the address of the pointer).
However, slices are technically also passed by value.
When we look here, we can get an idea how they work:
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
Len and Cap are integers, but Data is a pointer.
When this struct is copied (when passing by value), a copy of Len, Cap and Data will be made. Since Data is a pointer, any modifications made to the value it's pointing to will be visible after your function returns.
You can also read this
(I'm new to Go.)
I am working on this leetcode problem: https://leetcode.com/problems/pascals-triangle/
package main
import "fmt"
func main() {
arrRes := [][]int{}
gen(5, arrRes)
fmt.Println(arrRes)
}
func gen(numRows int, arrRes [][]int) {
build(numRows, 0, arrRes)
}
func build(n int, level int, arrRes [][]int) {
if(n == level) {
return
}
arr := []int{}
if level == 0 {
arr = append(arr, 1)
} else if level == 1 {
arr = append(arr, 1, 1)
} else {
// get it out
tmp := arrRes[level-1]
arr = comb(tmp)
}
arrRes = append(arrRes, arr)
build(n, level+1, arrRes)
}
func comb(arr []int) []int{
// arr type init
tmpArr := []int{1}
for i:=1; i<len(arr); i++ {
sum := arr[i-1] + arr[i]
tmpArr = append(tmpArr, sum)
}
// go use val, not ref
tmpArr = append(tmpArr, 1)
return tmpArr;
}
I want to define an accumulated variable arrRes := [][]int{} and keep passing into the recursive function. I think Go is pass-by-value instead of pass-by-reference. Is there a way to keep this pattern?
I've got two alternative methods:
passing a global var.
pass a 2D array into the func then return the new 2D array.
https://github.com/kenpeter/go_tri/blob/master/tri_global.go
https://github.com/kenpeter/go_tri/blob/master/tri.go
A slice is (basically) three things: a length, a capacity, and a pointer to an underlying array. Everything in Go is pass-by-value, so when you pass a slice to a function you are passing its current length, current capacity, and the memory address of the pointer. Changes made to length and capacity inside the function are made to a copy, and will not affect the length and capacity of the slice that was passed as an argument in the function call.
Printing a slice doesn't print its underlying array, it prints the part of the underlying array that is visible in the slice (which could be none of it if len = 0), based on (1) the pointer to the first element in the underlying array that's supposed to be visible to the slice; and (2) the length in the slice variable.
If you are modifying the length or capacity of a slice inside a function and you want those changes to be visible outside the function, you can either return the slice to update the context outside the function, like append does:
numbers := append(numbers, 27)
Or you can pass in a pointer to a slice:
func ChangeNumbersLenOrCap(numbers *[]int) {
// make your changes, no return value required
}
For your program, it looks like you could get away with a pointer to a slice of int slices:
var arrRes *[][]int
...because you're not modifying the int slice across another function boundary. Some programs would need a pointer to a slice of pointers to int slices:
var arrRes *[]*[]int
Here are some simple edits to get you started:
arrRes := [][]int{}
gen(5, &arrRes)
fmt.Println(arrRes)
}
func gen(numRows int, arrRes *[][]int) {
// ...
func build(n int, level int, arrRes *[][]int) {
// ...
tmp := *arrRes[level-1]
// ...
*arrRes = append(*arrRes, arr)
build(n, level+1, arrRes)
consider this piece of code:
package main
import (
"fmt"
)
func main() {
fmt.Println(Part(11))
}
func Part(n int) string {
enumResult := [][]int{}
enum(n, n, []int{}, &enumResult)
fmt.Println(enumResult)
fmt.Println(40, enumResult[40])
return ""
}
var abc int = 0
func enum(n int, top int, pre []int, result *[][]int) {
var i int
if n > top {
i = top
} else {
i = n
}
for ; i > 0; i-- {
tempResult := append(pre, i)
if n-i == 0 {
/* if tempResult[0] == 3 && tempResult[1] == 3 && tempResult[2] == 3 && tempResult[3] == 2 {
tempResult = append(tempResult, 12345)
}*/
fmt.Println(abc, tempResult)
abc++
*result = append(*result, tempResult)
} else {
enum(n-i, i, tempResult, result)
}
}
}
When I run this code
I append value '[3,3,3,2]' to 'enumResult'
but If I check the value of 'enumResult' then '[3,3,3,1]' is appear
it`s index is 40 =>enumResult[40]
(other value is correct)
I don`t know why this is happening
Can you explain to me why?
The problem is indeed due to append.
There are two thing about append. First is, that append doe not necessarily copy memory. As the spec specifies:
If the capacity of s is not large enough to fit the additional values,
append allocates a new, sufficiently large underlying array that fits
both the existing slice elements and the additional values. Otherwise,
append re-uses the underlying array.
This may cause unexpected behavior if you are not clear. A playground example: https://play.golang.org/p/7A3JR-5IX8o
The second part is, that when append does copy memory, it grows the capacity of the slice. However, it does not grow it just by 1. A playground example: https://play.golang.org/p/STr9jMqORUz
How much append grows a slice is undocumented and considered an implentation details. But till Go 1.10, it follows this rule:
Go slices grow by doubling until size 1024, after which they grow by
25% each time.
Note that when enabling race-detector, this may change. The code for growing slice is located in $GOROOT/src/runtime/slice.go in growslice function.
Now back to the question. It should be clear now that your code did append from a same slice with sufficient capacity due to growth of the slice from append before. To solve it, make a new slice and copy the memory.
tempResult := make([]int,len(pre)+1)
copy(tempResult,pre)
tempResult[len(pre)] = i
I wanted to implement time based slots for holding data using golang slices. I managed to come up with a go program like this and it also works. But I have few questions regarding garbage collection and the general performance of this program. Does this program guarantee garbage collection of items once slice is equated to nil? And while shuffling slices, I hope this program does not do any deep copying.
type DataSlots struct {
slotDuration int //in milliseconds
slots [][]interface{}
totalDuration int //in milliseconds
}
func New(slotDur int, totalDur int) *DataSlots {
dat := &DataSlots{slotDuration: slotDur,
totalDuration: totalDur}
n := totalDur / slotDur
dat.slots = make([][]interface{}, n)
for i := 0; i < n; i++ {
dat.slots[i] = make([]interface{}, 0)
}
go dat.manageSlots()
return dat
}
func (self *DataSlots) addData(data interface{}) {
self.slots[0] = append(self.slots[0], data)
}
// This should be a go routine
func (self *DataSlots) manageSlots() {
n := self.totalDuration / self.slotDuration
for {
time.Sleep(time.Duration(self.slotDuration) * time.Millisecond)
for i := n - 1; i > 0; i-- {
self.slots[i] = self.slots[i-1]
}
self.slots[0] = nil
}
}
I removed critical section handling in this snippet to make it concise.
Once your slice is set too nil, any values contained in the slice are available for garbage collection, provided that the underlying array isn't shared with another slice.
Since there are no slice operations in your program, you never have multiple references to the same array, nor are you leaving data in any inaccessible portions of the underlying array.
What you need to be careful of, is when you're using slice operations:
a := []int{1, 2, 3, 4}
b := a[1:3]
a = nil
// the values 1 and 4 can't be collected, because they are
// still contained in b's underlying array
c := []int{1, 2, 3, 4}
c = append(c[1:2], 5)
// c is now []int{2, 5}, but again the values 1 and 4 are
// still in the underlying array. The 4 may be overwritten
// by a later append, but the 1 is inaccessible and won't
// be collected until the underlying array is copied.
While append does copy values when the capacity of the slice in insufficient, only the values contained in the slice are copied. There is no deep copy of any of the values.
The code snippet below is the library implementation of the push methods for a priority queue. I am wondering why the line with the code a = a[0 : n+1] does not throw an out of bounds errors.
func (pq *PriorityQueue) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
// To simplify indexing expressions in these methods, we save a copy of the
// slice object. We could instead write (*pq)[i].
a := *pq
n := len(a)
a = a[0 : n+1]
item := x.(*Item)
item.index = n
a[n] = item
*pq = a
}
a slice is not an array; it is a view onto an existing array. The slice in question is backed by an array larger than itself. When you define a slice of an existing slice, you're actually slicing the underlying array, but the indexes referenced are relative to the source slice.
That's a mouthful. Let's prove this in the following way: we'll create a slice of zero length, but we'll force the underlying array to be larger. When creating a slice with make, the third parameter will set the size of the underlying array. The expression make([]int, 0, 2) will allocate an array of size 2, but it evaluates to a size-zero slice.
package main
import ("fmt")
func main() {
// create a zero-width slice over an initial array of size 2
a := make([]int, 0, 2)
fmt.Println(a)
// expand the slice. Since we're not beyond the size of the initial
// array, this isn't out of bounds.
a = a[0:len(a)+1]
a[0] = 1
fmt.Println(a)
fmt.Println(a[0:len(a)+1])
}
see here. You can use the cap keyword to reference the size of the array that backs a given slice.
The specific code that you asked about loops over cap(pq) in the calling context (container/heap/example_test.go line 90). If you modify the code at the call site and attempt to push another item into the queue, it will panic like you expect. I ... probably wouldn't suggest writing code like this. Although the code in the standard library executes, I would be very sour if I found that in my codebase. It's generally safer to use the append keyword.
Because it works in a specific example program. Here are the important parts from the original/full example source)
const nItem = 10
and
pq := make(PriorityQueue, 0, nItem)
and
for i := 0; i < cap(pq); i++ {
item := &Item{
value: values[i],
priority: priorities[i],
}
heap.Push(&pq, item)
}
Is it an example from container/heap? If yes, then it doesn't throws an exception because capacity is big enough (see how the Push method is used). If you change the example to Push more items then the capacity, then it'll throw.
It does in general; it doesn't in the container/heap example. Here's the general fix I already gave you some time ago.
func (pq *PriorityQueue) Push(x interface{}) {
a := *pq
n := len(a)
item := x.(*Item)
item.index = n
a = append(a, item)
*pq = a
}
Golang solution to Project Euler problem #81