The following code:
package main
import "fmt"
// fibonacci is a function that returns
// a function that returns an int.
func fibonacci() func() int {
first, second := 0, 1
return func() int {
// return next fibonacci number here.
first, second = second, first+second
return first
}
}
func main() {
f := fibonacci()
for i := 0; i < 10; i++ {
fmt.Println(f())
}
}
returns 10 numbers of the fibonacci sequence. What's confusing to me is why is works. It seems like the values first and second are somehow saved in memory, since each time the code is executed, a new fibonacci number in sequence with the previous one is returned. I thought that functions lost their remembered variables when they were done executing. What is going on here?
first, and second are variables in the fibonacci() func, that were 'closed over' by the returned func() int that was returned from fibonacci().
So they are in the closure associated with f, so f has access to those variables as long as it exists.
See this Go Tour slide (and the ones around it) for some explanation of Go closures.
Related
package main
import (
"fmt"
)
func main(){
f,val,val1:=fibonacci()
fmt.Println(val,val1)
for i:=0;i<=10;i++ {
fmt.Println(f(i),val,val1)
}
_,val,val1=fibonacci()
fmt.Println(val,val1)
}
func fibonacci()(func(n int)int,int,int){
var val int
var val1 int
f:=func(n int)int{
if n==0||n==1{
val,val1=1,1
}else{
val,val1=val+val1,val
}
return val
}
fmt.Println("fibonacci val =",val,"val1 =",val1)
return f,val,val1
}
Here is my code on sloving fibonacci without using recursion when I
read about lambda function/closure. And the Go Documentary says a
closure will capture some external state. My understanding is the
closure will keep a copy of state of the function which it is
declared. These states are just copies whatever I do on them won't
modify the original, is that so?
from your test code here: https://play.golang.org/p/pajT2bAIe2
your fibonacci function is working out the nth numbers in the sequence provided it's called in an incremental fashion as you are doing. but the values you return from the initial call to fibonacci are not pointers (or references) to those integer values they are just the values of those integers at that time, think of them as being copied out of the function, try using integer pointers instead like this: https://play.golang.org/p/-vLja7Fpsq
package main
import (
"fmt"
)
func main() {
f, val, val1 := fibonacci()
fmt.Println(val, val1)
for i := 0; i <= 10; i++ {
fmt.Println(f(i), *val, *val1) //dereference the pointer to get its value at the current time
}
_, val, val1 = fibonacci()
fmt.Println(*val, *val1)
}
func fibonacci() (func(n int) int, *int, *int) {
var val int
var val1 int
f := func(n int) int {
if n == 0 || n == 1 {
val, val1 = 1, 1
} else {
val, val1 = val+val1, val
}
return val
}
fmt.Println("fibonacci val =", val, "val1 =", val1)
return f, &val, &val1 // return pointers to the closured values instead of just the values
}
Although you have accepted the above answer i'm giving another explanation. The reason why you receive the last value of the loop operation has to do with the Go's lexical scoping. The for loop introduces a new lexical block in which the value is referenced by it's memory address, so by pointer and not by it's value. In order to get the value, you have to de-reference.
Each time the for loop makes an iteration the value processed is pointing to the same memory address. All the function values created by this loop "capture" and share the same variable - and addressable storage location, not it's value at that particular moment. Thus when the last iteration is finished, the variable holds the value from the final step.
A much better approach for these kind of operations would be to use goroutines, because in these cases you are not communicating through sharing the same memory address, but you are sharing the memory through communication.
Here is a more elegant solution using goroutine:
package main
import (
"fmt"
)
func fibonacci(ch chan interface{}, quit chan struct{}) {
x, y := 1, 1
for {
select {
case ch <- x:
x, y = y, x+y
fmt.Println(x , y)
case <-quit:
fmt.Println("Quiting...")
return
}
}
}
func main() {
ch := make(chan interface{})
quit := make(chan struct{})
go func() {
for i := 0; i < 10; i++ {
<-ch
}
quit <- struct{}{}
}()
fibonacci(ch, quit)
}
https://play.golang.org/p/oPQgXWyV9u
Here is the code I've been trying to understand:
package main
import (
"fmt"
)
func squares() func() int {
var x int
return func() int {
x = x + 2
return x * x
}
}
func main() {
f := squares()
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares()())
fmt.Println(squares()())
fmt.Println(squares()())
}
The result we got:
4
16
36
4
4
4
My question is: Why does the value of x in fmt.Println(squares()()) stay unchanged?
Short version
You are building a new closure each time you call squares.
This is exactly as if you built a new Counter object in an object-oriented language:
new Counter().increment(); // 4
new Counter().increment(); // 4
...as opposed to:
c = new Counter();
c.increment(); // 4
c.increment(); // 16
Longer version
In your function, var x int declares a local variable x:
func squares() func() int {
var x int
return func() int {
x = x + 2
return x * x
}
}
As for any function, local variables are visible only inside the function. If you call a function in different contexts, each call has a separate set of memory storage addressable by your local symbols (here x). What happens when you return a function is that any binding currently visible in the scope of your code is kept alongside your function, which is then called a closure.
Closures can hold a state, like objects do. Thus your closure can refer to the local variables that were visible when it was created, even when you escape the block where local variables were introduced (thankfully, the GC is here to keep track of the memory associated with those variables).
When you define f, you create a fresh closure. Each time you call it you modify the same place referenced by the internal x variable. But if you create fresh closures and call them once each, then you won't see the same side-effects, because each x names a different place in memory.
The reason you are getting the same result when you are calling the second variant with closure function is a consequence of scope rules. All function values created by the new self invoking, or anonymous function, capture and share the same variable - an addressable storage location, not it's value at that particular moment.
The anonymous function introduce a new lexical block which shares the same logical address to which the function values points to, so each time you invoke the function without to enclose the function to a new scope it will share the same logical address. This is the reason why you'll see in many places an anonymous function called in this way:
for i := range mymap {
func(n int) {
fmt. Println(n)
}(i) // note the scope
}
To address your issue one way would be to use pointer variables, this way you will be absolute sure that you will share the variable allocated to the same memory address. Here is the updated and working code:
package main
import (
"fmt"
)
func squares() func(x *int) int {
return func(x *int) int {
*x = *x + 2
return *x * *x
}
}
func main() {
f := squares()
x := 0
fmt.Println(f(&x))
fmt.Println(f(&x))
fmt.Println(f(&x))
fmt.Println(squares()(&x))
fmt.Println(squares()(&x))
fmt.Println(squares()(&x))
}
Go Playground
Another way is to expose the variable x as a global variable. This will guarantee that you won't create a new variable x each time you run the anonymous function.
Here's a different perspective to answer your question.
First break down the squares() function first to understand what is going on:
func squares() func() int {
The above defines a func named squares that returns another function of type func() int (and that returns an int, but that's not the focus here).
Drill that into your head first: it returns a function.
Now, let's see what happens when we call squares():
var x int
That's it. It defines a varibale x, which defaults to value 0 per Go specs.
OK, now we have a variable in scope called x and it has a value of 0. Now, we return a function:
return func() int {
x = x + 2
return x * x
}
If we had not defined x earlier, this would be a build error because x must be defined. But, it was defined in the previous scope.
Go's use of closures allows another scope to be defined, which uses variables in the previous scope. var x int in this case.
Also, you are able to modify closures (variables in previous scope). And in this case, we are modifying the previous var x int that was defined in the previous scope.
Hold onto those thoughts for a moment, let's run some code...
f := squares()
Here we run squares(), which defines var x int as zero and returns a func() named f() that can do more work on x.
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
Since we continue to reuse f(), this keeps the scope of var x int on a stack, and that stack remains around as long as you have this f() variable. Therefore, x continues to retain its modified value and be modified.
With all of this knowledge, the answer to your question is simple: if you don't retain the scope, the f() definition like above, then you define a new var x int = 0 at every invitation of squares():
fmt.Println(squares()())
This invokes a new squares() at every call. And therefore, a new var x int = 0.
So the output of the returning func of squares, which is invoked with squares()(), always operates on var x int = 0, at every invokation since a new squares is called each time.
Simply because (1) x is a captured closure for the anonymous function and (2) the default value for type int is 0. So every time you call it, you will see the same output.
Let's rewrite the function squares like this:
func squares(initialValue int) func() int {
var x int
x = initialValue
return func() int {
x = x + 2
return x * x
}
}
Now for this:
func main() {
f := squares(0)
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares(0)())
fmt.Println(squares(0)())
fmt.Println(squares(0)())
}
We will see the exact output! Because we are initializing x with 0. If we use 1 as the initial value of x:
func main() {
f := squares(1)
fmt.Println(f())
fmt.Println(f())
fmt.Println(f())
fmt.Println(squares(1)())
fmt.Println(squares(1)())
fmt.Println(squares(1)())
}
We will see this result:
9
25
49
9
9
9
As you can see, it's just about the initial value of x, which when not initialized explicitly, will have it's default value, zero.
I am new to Go language and got confused with the following code
package main
import "fmt"
// fibonacci is a function that returns
// a function that returns an int.
func fibonacci() func() int {
previous := 0
current := 1
return func () int{
current = current+previous
previous = current-previous
return current
}
}
func main() {
f := fibonacci
for i := 0; i < 10; i++ {
fmt.Println(f()())
}
}
This code is supposed to print out the Fibonacci Sequence (first 10), but only print out 10 times 1.
but if I change the code to:
func main() {
f := fibonacci()
for i := 0; i < 10; i++ {
fmt.Println(f())
}
}
Then it is working fine. The output is the Fibonacci sequence.
Could any one help me explain this?
Thanks
fibonacci() creates a new fibonacci generator function. fibonacci()() does the same, and then calls it once, returns the result and discards the generator, never to be used again. If you call that in a loop, it'll just keep creating new generators and only using their first value.
If you want more than just the first value, you need to do exactly what you did in your second example. Store the generator itself in a variable and then call the same generator multiple times.
This has to do with how variables are encapsulated in closures after returning the closure.
Consider the following example (live code on play):
func newClosure() func() {
i := 0
fmt.Println("newClosure with &i=", &i)
return func() {
fmt.Println(i, &i)
i++
}
}
func main() {
a := newClosure()
a()
a()
a()
b := newClosure()
b()
a()
}
Running this code will yield something like the following output. I annotated
which line comes from which statement:
newClosure with &i= 0xc010000000 // a := newClosure()
0 0xc010000000 // a()
1 0xc010000000 // a()
2 0xc010000000 // a()
newClosure with &i= 0xc010000008 // b := newClosure()
0 0xc010000008 // b()
3 0xc010000000 // a()
In the example, the closure returned by newClosure encapsulates the local variable i.
This corresponds to current and the like in your code. You can see that a and b
have different instances of i, or else the call b() would've printed 3 instead.
You can also see that the i variables have different addresses. (The variable is already
on the heap as go does not have a separate stack memory, so using it in the closure is
no problem at all.)
So, by generating a new closure you're automatically creating a new context for the
closure and the local variables are not shared between the closures. This is the reason
why creating a new closure in the loop does not get you further.
The equivalent in for your code in terms of this example would be:
for i:=0; i < 10; i++ {
newClosure()()
}
And you've already seen by the output that this will not work.
func fibonacci() func() int return a function literal (closure) that returns an int representing the last generated number in the list.
The first main() method
f := fibonacci
for i := 0; i < 10; i++ {
fmt.Println(f()())
}
f is the generator function, every iteration in the loop invoke f()() that generates a new closure with a new envirement ( previous := 0, current := 1), so , the second invocation returns current which is always equal to 1.
The second main() method
func main() {
f := fibonacci()
for i := 0; i < 10; i++ {
fmt.Println(f())
}
}
f is the closure (not the generator) with an initial envirement ( previous := 0, current := 1), every iteration in the loop invoke f() witch returns current and modify the envirement, so whith the next call, previous will be 1 and current 2.
This is homework
I'm working on a project, and a very small (very small, once I get this working...it's basically the pre-req for the rest of the project) part of it is to generate some combinations using a Go routine.
The code I have is thusly:
package bruteforce
// GenerateCombinations is an iterator function. Given an alphabet and a
// length, it will generate every possible combination of the letters in
// alphabet of the specified length.
//
// It is meant to be consumed by using the range clause:
//
// for combination := range GenerateCombinations(alphabet, length) {
// process(combination)
// }
//
func GenerateCombinations(alphabet string, length int) <-chan string {
GenerateCombinations(alphabet, length):
if length == 0:
yield ""
else:
for i := range alphabet{
for j := range GenerateCombinations(alphabet, length-1){
i + j
}
}
return nil
}
I seriously do not get this at all. As you can see, there is some instructor-provided pseudo code there, but the implementation of it is frying my brain.
Example I/O would be something like this:
If the alphabet is {0, 1} and the password was length 2, then it would need to generate {0, 1, 00, 01, 10, 11}.
I appreciate all suggestions, but please recognize that the term "beginner" doesn't begin to describe my competency with go. Saying things like "use channels" doesn't help me at all. Explanations are my friend... something that I haven't had a lot of luck getting out of my professor aside from "use channels."
Your teacher has already hinted that you should use a channel instead of returning a big array. By solving it like that, you will not have to store this big chunk of data containing all combination, but rather feed your function with different iterations and process them one at a time.
We can see your teachers hint in that the GenerateCombinations returns a chan string and not a []string:
func GenerateCombinations(alphabet string, length int) <-chan string
This also means that the function must 1) create a channel to return, and 2) start a goroutine that feeds iterations to the channel. This function would look something like this:
func GenerateCombinations(alphabet string, length int) <-chan string {
c := make(chan string)
// Starting a separate goroutine that will create all the combinations,
// feeding them to the channel c
go func(c chan string) {
defer close(c) // Once the iteration function is finished, we close the channel
// This is where the iteration will take place
// Your teacher's pseudo code uses recursion
// which mean you might want to create a separate function
// that can call itself.
}(c)
return c // Return the channel to the calling function
}
While I will leave the actual iteration to you, every loop should result in you putting a combination string into the channel. Because it is not a buffered channel, the iterating function will wait til the main function has read the value before continue to process the next iteration.
A playground version including the main function: http://play.golang.org/p/CBkSjpmQ0t
A full working solution using recursion: http://play.golang.org/p/0bWDCibSUJ
This is a tricky problem if you are entirely unfamiliar with Go and/or how to generate permutations. Below is a full implementation of the solution. I suggest you only look at this if you really get stuck, because doing so will obviously remove the learning experience.
You can run it on the go playground to see it work.
This approach is not recursive as your instructor's example suggests, but it gets the job done quite nicely.
package main
import "fmt"
import "sync"
func main() {
// Make sure the alphabet is sorted.
const alphabet = "abcde"
for str := range generate(alphabet) {
fmt.Println(str)
}
}
func generate(alphabet string) <-chan string {
c := make(chan string, len(alphabet))
go func() {
defer close(c)
if len(alphabet) == 0 {
return
}
// Use a sync.WaitGroup to spawn permutation
// goroutines and allow us to wait for them all
// to finish.
var wg sync.WaitGroup
wg.Add(len(alphabet))
for i := 1; i <= len(alphabet); i++ {
go func(i int) {
// Perform permutation on a subset of
// the alphabet.
Word(alphabet[:i]).Permute(c)
// Signal Waitgroup that we are done.
wg.Done()
}(i)
}
// Wait for all routines to finish.
wg.Wait()
}()
return c
}
type Word []rune
// Permute generates all possible combinations of
// the current word. This assumes the runes are sorted.
func (w Word) Permute(out chan<- string) {
if len(w) <= 1 {
out <- string(w)
return
}
// Write first result manually.
out <- string(w)
// Find and print all remaining permutations.
for w.next() {
out <- string(w)
}
}
// next performs a single permutation by shuffling characters around.
// Returns false if there are no more new permutations.
func (w Word) next() bool {
var left, right int
left = len(w) - 2
for w[left] >= w[left+1] && left >= 1 {
left--
}
if left == 0 && w[left] >= w[left+1] {
return false
}
right = len(w) - 1
for w[left] >= w[right] {
right--
}
w[left], w[right] = w[right], w[left]
left++
right = len(w) - 1
for left < right {
w[left], w[right] = w[right], w[left]
left++
right--
}
return true
}
I'm currently working through the Tour of Go, and I thought that goroutines have been used similarly to Python generators, particularly with Question 66. I thought 66 looked complex, so I rewrote it to this:
package main
import "fmt"
func fibonacci(c chan int) {
x, y := 1, 1
for {
c <- x
x, y = y, x + y
}
}
func main() {
c := make(chan int)
go fibonacci(c)
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
}
This seems to work. A couple of questions:
If I turn up the buffer size on the channel to say, 10, fibonacci would fill up 10 further spots, as quickly as possible, and main would eat up the spots as quickly as it could go. Is this right? This would be more performant than a buffer size of 1 at the expense of memory, correct?
As the channel doesn't get closed by the fibonacci sender, what happens memory-wise when we go out of scope here? My expectation is that once c and go fibonacci is out of scope, the channel and everything on it gets garbage-collected. My gut tells me this is probably not what happens.
Yes, increasing the buffer size might drastically increase the execution speed of your program, because it will reduce the number of context switches. Goroutines aren't garbage-collected, but channels are. In your example, the fibonacci goroutine will run forever (waiting for another goroutine to read from the channel c), and the channel c will never be destroyed, because the fib-goroutine is still using it.
Here is another, sightly different program, which does not leak memory and is imho more similar to Python's generators:
package main
import "fmt"
func fib(n int) chan int {
c := make(chan int)
go func() {
x, y := 0, 1
for i := 0; i <= n; i++ {
c <- x
x, y = y, x+y
}
close(c)
}()
return c
}
func main() {
for i := range fib(10) {
fmt.Println(i)
}
}
Alternatively, if you do not know how many Fibonacci numbers you want to generate, you have to use another quit channel so that you can send the generator goroutine a signal when it should stop. This is whats explained in golang's tutorial https://tour.golang.org/concurrency/4.
I like #tux21b's answer; having the channel created in the fib() function makes the calling code nice and clean. To elaborate a bit, you only need a separate 'quit' channel if there's no way to tell the function when to stop when you call it. If you only ever care about "numbers up to X", you can do this:
package main
import "fmt"
func fib(n int) chan int {
c := make(chan int)
go func() {
x, y := 0, 1
for x < n {
c <- x
x, y = y, x+y
}
close(c)
}()
return c
}
func main() {
// Print the Fibonacci numbers less than 500
for i := range fib(500) {
fmt.Println(i)
}
}
If you want the ability to do either, this is a little sloppy, but I personally like it better than testing the condition in the caller and then signalling a quit through a separate channel:
func fib(wanted func (int, int) bool) chan int {
c := make(chan int)
go func() {
x, y := 0, 1
for i := 0; wanted(i, x); i++{
c <- x
x, y = y, x+y
}
close(c)
}()
return c
}
func main() {
// Print the first 10 Fibonacci numbers
for n := range fib(func(i, x int) bool { return i < 10 }) {
fmt.Println(n)
}
// Print the Fibonacci numbers less than 500
for n := range fib(func(i, x int) bool { return x < 500 }) {
fmt.Println(n)
}
}
I think it just depends on the particulars of a given situation whether you:
Tell the generator when to stop when you create it by
Passing an explicit number of values to generate
Passing a goal value
Passing a function that determines whether to keep going
Give the generator a 'quit' channel, test the values yourself, and tell it to quit when appropriate.
To wrap up and actually answer your questions:
Increasing the channel size would help performance due to fewer context switches. In this trivial example, neither performance nor memory consumption are going to be an issue, but in other situations, buffering the channel is often a very good idea. The memory used by make (chan int, 100) hardly seems significant in most cases, but it could easily make a big performance difference.
You have an infinite loop in your fibonacci function, so the goroutine running it will run (block on c <- x, in this case) forever. The fact that (once c goes out of scope in the caller) you won't ever again read from the channel you share with it doesn't change that. And as #tux21b pointed out, the channel will never be garbage collected since it's still in use. This has nothing to do with closing the channel (the purpose of which is to let the receiving end of the channel know that no more values will be coming) and everything to do with not returning from your function.
You could use closures to simulate a generator. Here is the example from golang.org.
package main
import "fmt"
// fib returns a function that returns
// successive Fibonacci numbers.
func fib() func() int {
a, b := 0, 1
return func() int {
a, b = b, a+b
return a
}
}
func main() {
f := fib()
// Function calls are evaluated left-to-right.
fmt.Println(f(), f(), f(), f(), f())
}
Using channels to emulate Python generators kind of works, but they introduce concurrency where none is needed, and it adds more complication than's probably needed. Here, just keeping the state explicitly is easier to understand, shorter, and almost certainly more efficient. It makes all your questions about buffer sizes and garbage collection moot.
type fibState struct {
x, y int
}
func (f *fibState) Pop() int {
result := f.x
f.x, f.y = f.y, f.x + f.y
return result
}
func main() {
fs := &fibState{1, 1}
for i := 0; i < 10; i++ {
fmt.Println(fs.Pop())
}
}