Go goroutines leaking - go

Following this post about go-routines leaking, https://www.ardanlabs.com/blog/2018/11/goroutine-leaks-the-forgotten-sender.html, i tried to solve my leaking code. But adding a buffer to the channel it did not do the trick.
My code
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
fmt.Println(runtime.NumGoroutine())
leaking()
time.Sleep(5)
fmt.Println(runtime.NumGoroutine())
}
func leaking() {
errChang := make(chan int, 1)
go func() {
xx := return666()
errChang <- xx
}()
fmt.Println("hola")
return
fmt.Println(<-errChang)
}
func return666() int {
time.Sleep(time.Second * 1)
return 6
}
My initial code did not use a buffer, leading to the go-routine in the leaking function, .. to leak. Following the post i expected that by adding a buffer to the channel, it would have avoided leaking.

Here, in the Go Playground, is your original code with a few slight modifications:
the delays are reduced, except for time.Sleep(5) which becomes time.Sleep(time.Second);
a return is removed because it becomes unnecessary;
a fmt.Println is commented out, because with both the return and the uncommented fmt.Println, go vet complains about the unreachable fmt.Println;
the channel stored in errChang is changed to unbuffered.
When run, its output is:
1
hola
2
(with a small delay before the 2), showing that indeed, the anonymous goroutine you started in function leaking is still running.
If we uncomment the commented-out fmt.Println, the output is:
1
hola
6
1
(with the same slight delay before the final 1) because we now wait for (and then print) the value computed in return666 and sent over channel errChang.
If we keep the commented-out fmt.Println commented out and make the channel buffered, the output becomes:
1
hola
1
as the anonymous goroutine is now able to push its value (6) into the channel.
The channel itself would be garbage collected, along with the single value stored inside it, as there are no remaining references to the channel at this point. Note, however, that simply making the channel buffered is not always sufficient. If we send two values down the channel, the program returns to printing:
1
hola
2
as the anonymous goroutine succeeds in putting 6 into the channel but then blocks trying to put 42 in as well.

Related

Doesn't go routine and the channels work in order of call?

Doesn't go routine and the channels worked in the order they were called.
and go routine share values between the region variables?
main.go
var dataSendChannel = make(chan int)
func main() {
a(dataSendChannel)
time.Sleep(time.Second * 10)
}
func a(c chan<- int) {
for i := 0; i < 1000; i++ {
go b(dataSendChannel)
c <- i
}
}
func b(c <-chan int) {
val := <-c
fmt.Println(val)
}
output
> go run main.go
0
1
54
3
61
5
6
7
8
9
Channels are ordered. Goroutines are not. Goroutines may run, or stall, more or less at random, whenever they are logically allowed to run. They must stop and wait whenever you force them to do so, e.g., by attempting to write on a full channel, or using a mutex.Lock() call on an already-locked mutex, or any of those sorts of things.
Your dataSendChannel is unbuffered, so an attempt to write to it will pause until some goroutine is actively attempting to read from it. Function a spins off one goroutine that will attempt one read (go b(...)), then writes and therefore waits for at least one reader to be reading. Function b immediately begins reading, waiting for data. This unblocks function a, which can now write some integer value. Function a can now spin off another instance of b, which begins reading; this may happen before, during, or after the b that got a value begins calling fmt.Println. This second instance of b must now wait for someone—which in this case is always function a, running the loop—to send another value, but a does that as quickly as it can. The second instance of b can now begin calling fmt.Println, but it might, mostly-randomly, not get a chance to do that yet. The first instance of b might already be in fmt.Println, or maybe it isn't yet, and the second one might run first—or maybe both wait around for a while and a third instance of b spins up, reads some value from the channel, and so on.
There's no guarantee which instance of b actually gets into fmt.Println when, so the values you see printed will come out in some semi-random order. If you want the various b instances to sequence themselves, they will need to do that somehow.

Why doesn't this buffered channel block in my code?

I'm learning the Go language. Can someone please explain the output here?
package main
import "fmt"
var c = make(chan int, 1)
func f() {
c <- 1
fmt.Println("In f()")
}
func main() {
go f()
c <- 2
fmt.Println(<-c)
fmt.Println(<-c)
}
Output:
In f()
2
1
Process finished with exit code 0
Why did "In f()" occur before "2"? If "In f()" is printed before "2", the buffered channel should block. But this didn't happen, why?
The other outputs are reasonable.
Image of my confusing result
The order of events that cause this to run through are these:
Trigger goroutine.
Write 2 to the channel. Capacity of the channel is now exhausted.
Read from the channel and output result.
Write 1 to the channel.
Read from the channel and output result.
The order of events that cause this to deadlock are these:
Trigger goroutine.
Write 1 to the channel. Capacity of the channel is now exhausted.
Write 2 to the channel. Since the channel's buffer is full, this blocks.
The output you provide seems to indicate that the goroutine finishes first and the program doesn't deadlock, which contradicts above two scenarios as explanation. Here's what happens though:
Trigger goroutine.
Write 2 to the channel.
Read 2 from the channel.
Write 1 to the channel.
Output In f().
Output the 2 received from the channel.
Read 1 from the channel.
Output the 1 received from the channel.
Keep in mind that you don't have any guarantees concerning the scheduling of goroutines unless you programmatically enforce them. When you start a goroutine, it is undefined when the first code of that goroutine are actually executed and how much the starting code progresses until then. Note that since your code relies on a certain order of events, it is broken by that definition, just to say that explicitly.
Also, at any point in your program, the scheduler can decide to switch between different goroutines. Even the single line fmt.Printtln(<-c) consists of multiple steps and in between each step the switch can occur.
To reproduce block you have to run that code many times the easiest way to do it with test. You don't have block cause of luck. But it is not guaranteed:
var c = make(chan int, 1)
func f() {
c <- 1
fmt.Println("In f()")
}
func TestF(t *testing.T) {
go f()
c <- 2
fmt.Println(<-c)
fmt.Println(<-c)
}
and run with command:
go test -race -count=1000 -run=TestF -timeout=4s
where count number of tests. It reproduces blocking to me.
Provide timeout to not wait default 10 minutes

Confused about channel argument for Goroutines

I am learning about channels and concurrency in Go and I am confused on how the code below works.
package main
import (
"fmt"
"time"
"sync/atomic"
)
var workerID int64
var publisherID int64
func main() {
input := make(chan string)
go workerProcess(input)
go workerProcess(input)
go workerProcess(input)
go publisher(input)
go publisher(input)
go publisher(input)
go publisher(input)
time.Sleep(1 * time.Millisecond)
}
// publisher pushes data into a channel
func publisher(out chan string) {
atomic.AddInt64(&publisherID, 1)
thisID := atomic.LoadInt64(&publisherID)
dataID := 0
for {
dataID++
fmt.Printf("publisher %d is pushing data\n", thisID)
data := fmt.Sprintf("Data from publisher %d. Data %d", thisID, dataID)
out <- data
}
}
func workerProcess(in <-chan string) {
atomic.AddInt64(&workerID, 1)
thisID := atomic.LoadInt64(&workerID)
for {
fmt.Printf("%d: waiting for input...\n", thisID)
input := <-in
fmt.Printf("%d: input is: %s\n", thisID, input)
}
}
This is what I understand please correct me if I'm wrong:
workerProcess Goroutine:
Input channel taken as the argument which is assigned as the in channel.
In the for loop, the first printf is executed.
The value off the in channel is assigned to the variable input.
The last printf is executed to show the value of thisID and input.(This doesn't really execute till after other Goroutines run).
publisher Goroutine:
Input channel taken as the argument which is assigned as the out channel.
In the for loop, dataID is incremented then first printf is executed.
The string is assigned to data.
The value of data is passed to out.
Questions:
Where is the value from out getting passed to in the publisher Goroutine? It seems to be only in the scope of the for loop, wouldn't this cause deadlock. Since this is an unbuffered channel.
I don't get how the workerProcess gets the data from publisher if
all the Goroutines in main have the arguments as the channel input.
I'm used to having code written like this if the output of one function
is used in another:
foo = fcn1(input)
fcn2(foo)
I suspect it has something with how Goroutines run in the background but I'm not to sure I would appreciate an explanation.
Why isn't the last printf statement in workerProcess not executed? My guess is the channel is empty, so it's waiting for a value.
Part of the output:
1: waiting for input...
publisher 1 is pushing data
publisher 1 is pushing data
1: input is: Data from publisher 1. Data 1
1: waiting for input...
1: input is: Data from publisher 1. Data 2
1: waiting for input...
publisher 1 is pushing data
publisher 1 is pushing data
publisher 2 is pushing data
2: waiting for input..
You have one channel, it is made with make in main.
This one single channel is named in in workerProcess and out in publisher. out is a function argument to publisher which answers question 1.
Question 2: That's the whole purpose of a channel. What you stuff into a channel on its input side comes out at its output side. If a function has reference to (one end of) such a channel it may communicate with someone else having reference to the same channel (its other end). What producer send to the channel is received by workerProcess. This send and receive is done through the special operator <- on Go. A fact you got slightly wrong in your explanation.
out <- data takes data and sends it through the channel named out until <- in reads it from the channel (remember in and out name the same channel from main). That's how workerProcess and publisher communicate.
Question 3 is a duplicate. Your whole program terminates once main is done (in your case after 1 millisecond. Nothing happens after termination of the program. Give the program more time to execute. (The non-printing is unrelated to the channel).

Golang concurrency and blocking with a single channel, explanation needed

I'm playing around with the code presented on https://tour.golang.org/concurrency/5. My idea was that I could simplify the code by getting rid of the quit channel and still keep the correct program behavior - just for learning purposes.
Here is the code I've got (simplified it even more for better readability):
package main
import "fmt"
import "time"
func sendNumbers(c chan int) {
for i := 0; i < 2; i++ {
c <- i
}
}
func main() {
c := make(chan int)
go func() {
for i := 0; i < 2; i++ {
fmt.Println(<-c)
}
}()
time.Sleep(0 * time.Second)
sendNumbers(c)
}
In this code, the go routine I spawn should be able to receive 2 numbers before it returns. The sendNumbers() function I call next sends exactly 2 numbers to the channel c.
So, the expected output of the program is 2 lines: 0 and 1. However, what I'm getting when I run the code on the page, is just a single line containing 0.
Note the
time.Sleep(0 * time.Second)
though that I deliberately added before calling sendNumbers(c). If I change it to
time.Sleep(1 * time.Second)
The output becomes as expected:
0
1
So, I'm confused with what happens. Can someone explain what is going on? Shouldn't both sends and receives block regardless of how much time is passed before I call sendNumbers()?
In Go, the program exits as soon as the main function returns regardless of whether other goroutines are still running or not. If you want to make sure the main function does not exit prematurely, you need some synchronization mechanism. https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/
You don’t absolutely have to use synchronization primitives, you could also do it with channels only, arguably a more idiomatic way to do it.

Why is this program producing output

I am new to GO from google. In buffered channels here is a program:
package main
import "fmt"
import "time"
func main() {
c := make(chan int, 2)
c <- 1
fmt.Println(<-c)
time.Sleep(1000 * time.Millisecond)
c <- 2
fmt.Println(<-c)
}
It produces output. But according to http://tour.golang.org/#64 it says:
Sends to a buffered channel block only when the buffer is full.
Receives block when the buffer is empty.
As it says it send only when FULL why does the program produce an output instead of waiting infinity for c to full up at the first statement. ?
Your channel has a buffer size of two. You're putting one int in then pulling one int out. Then you sleep and repeat the process. The channel will not block until you try to insert a third int without pulling any ints out. The first two ints will be buffered.
I guess you didn't understood the slide properly.
It says "block only" you understood "work only".
What the slide said is:
If the buffer is not full, then your send will work properly and it won't block
If the buffer is full, then your send will block until the buffer is not full.
So your example is working as specified.

Resources