I'm trying to understand go channel and go routine. To do so, I'm doing online exercises. I found one here: http://whipperstacker.com/2015/10/05/3-trivial-concurrency-exercises-for-the-confused-newbie-gopher/
I resolved the 3rd one (named "Internet cafe").
But there's something I resolved by "luck", and it's bother me because I don't understand my issue and why my "hack" fixed it.
In my code below, I replace "enterChan <- next" by "go func() { enterChan <- next }()", and it solved my deadlock.
Can someone explain to me why it deadlock before, and why it works with this hack ? Is it a proper solution, or an ugly one ?
Don't hesitate to criticize my code, I'm searching to improve :)
Many thanks!
This is my code:
package main
import (
"fmt"
"math/rand"
"strconv"
"time"
)
const (
maxNumberOfUser = 8
)
func useComputer(tourist string, leaverChan chan string) {
seed := rand.NewSource(time.Now().UnixNano())
random := rand.New(seed)
fmt.Println(tourist, "is online")
d := random.Intn(120-15) + 15
time.Sleep(time.Duration(d) * time.Millisecond * 10)
fmt.Println(tourist, "is done, having spent", d, "minutes online.")
leaverChan <- tourist
}
func manageUsers(enterChan, leaverChan chan string, stopChan chan struct{}) {
nbUsed := 0
queue := make([]string, 0)
for {
select {
case tourist := <-enterChan:
if nbUsed < maxNumberOfUser {
nbUsed++
go useComputer(tourist, leaverChan)
} else {
fmt.Println(tourist, "waiting for turn.")
queue = append(queue, tourist)
}
case tourist := <-leaverChan:
nbUsed--
fmt.Println(tourist, "is leaving, number of free place is now:", maxNumberOfUser-nbUsed)
if len(queue) > 0 {
next := queue[0]
queue = queue[1:]
go func() {
enterChan <- next
}()
} else if nbUsed == 0 {
close(stopChan)
return
}
}
}
}
func main() {
enterChan := make(chan string)
leaverChan := make(chan string)
stopChan := make(chan struct{})
go manageUsers(enterChan, leaverChan, stopChan)
for i := 1; i <= 25; i++ {
enterChan <- "Tourist " + strconv.Itoa(i)
}
<-stopChan
fmt.Println("The place is empty, let's close up and go to the beach!")
}
As explained by #dev.bmax, enterChan is not a buffered channel and you are trying to send data to a channel that is not being read. For example, below code results in a deadlock error:
package main
import (
"fmt"
)
func main() {
stream := make(chan int)
<-stream
stream<-1
fmt.Println("Yayyy!! no deadlock!")
}
you can run the above code here and check that it has deadlock. But if I change it to this:
package main
import (
"fmt"
)
func main() {
stream := make(chan int)
go func(){<-stream}()
stream<-1
fmt.Println("Yayyy!! no deadlock!")
}
we have no deadlock. Because we started another GoRoutine which is reading from the channel while the main GoRoutine is pushing data to channel. Hope this helps!
Related
I am getting fatal error: all goroutines are asleep - deadlock!
on the line wg.Wait()
It happens for about ~30% of the runs, the rest are finished with no error. I guess I am using WaitGroup the wrong way, but not sure what am I doing wrong.
Maybe someone can help me identify my bug? Thanks!
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
const (
numOfPhilosophers = 5
numOfMeals = 3
maxEaters = 2
)
var doOnce sync.Once
func main() {
chopsticks := make([]sync.Mutex, 5)
permissionChannel := make(chan bool)
finishEating := make(chan bool)
go permissionFromHost(permissionChannel,finishEating)
var wg sync.WaitGroup
wg.Add(numOfPhilosophers)
for i:=1 ; i<=numOfPhilosophers ; i++ {
go eat(i, chopsticks[i-1], chopsticks[i%numOfPhilosophers], &wg, permissionChannel, finishEating)
}
wg.Wait()
}
func eat(philosopherId int, left sync.Mutex, right sync.Mutex, wg *sync.WaitGroup, permissionChannel <-chan bool, finishEatingChannel chan<- bool) {
defer wg.Done()
for i:=1 ; i<=numOfMeals ; i++ {
//lock chopsticks in random order
if RandBool() {
left.Lock()
right.Lock()
} else {
right.Lock()
left.Lock()
}
fmt.Printf("waiting for permission from host %d\n",philosopherId)
<-permissionChannel
fmt.Printf("starting to eat %d (time %d)\n", philosopherId, i)
fmt.Printf("finish to eat %d (time %d)\n", philosopherId, i)
//release chopsticks
left.Unlock()
right.Unlock()
//let host know I am done eating
finishEatingChannel<-true
}
}
func permissionFromHost(permissionChannel chan<-bool, finishEating <-chan bool) {
ctr := 0
for {
select {
case <-finishEating:
ctr--
default:
if ctr<maxEaters {
ctr++
permissionChannel<-true
}
}
}
}
func RandBool() bool {
rand.Seed(time.Now().UnixNano())
return rand.Intn(2) == 1
}
Edit 1: I fixed the mutex to be passed by reference. It didn't solve the problem.
Edit 2: I tried to use buffered channel permissionChannel:=make(chan bool, numOfPhilosophers) which makes it work
Edit 3: also #Jaroslaw example makes it work
The last goroutine will not exit, it will get blocked in its last iteration when it is writing to the finishEatingChannel channel as there are no consumers for it.
The reason there are no consumers for the finishEatingChannel is that the select case in the function permissionFromHost is writing to permissionChannel<-true but there are no consumers for permissionChannel as it is waiting for it to be read so we have a deadlock.
You can make the permissionFromHost channel buffered, it will resolve the issue.
There is also a bug in your code, you are passing mutex by value which is not allowed
The go vet command says
./main.go:26:13: call of eat copies lock value: sync.Mutex
./main.go:26:30: call of eat copies lock value: sync.Mutex
./main.go:31:34: eat passes lock by value: sync.Mutex
./main.go:31:52: eat passes lock by value: sync.Mutex
Another problem is that there are times when goroutines (philosophers) get blocked when trying to send an acknowledgement on finishEatingChannel, because the goroutine (host) responsible for reading data from this unbuffered channel is busy trying to send a permission. Here is the exact part of code:
if ctr<maxEaters {
ctr++
// This goroutine stucks since the last philosopher is not reading from permissionChannel.
// Philosopher is not reading from this channel at is busy trying to write finishEating channel which is not read by this goroutine.
// Thus the deadlock happens.
permissionChannel<-true
}
Deadlock is 100% reproducible when there is only one philosopher left who needs to eat twice.
Fixed version of code:
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
const (
numOfPhilosophers = 5
numOfMeals = 3
maxEaters = 2
)
func main() {
chopsticks := make([]sync.Mutex, 5)
permissionChannel := make(chan bool)
finishEating := make(chan bool)
go permissionFromHost(permissionChannel, finishEating)
var wg sync.WaitGroup
wg.Add(numOfPhilosophers)
for i := 1; i <= numOfPhilosophers; i++ {
go eat(i, &chopsticks[i-1], &chopsticks[i%numOfPhilosophers], &wg, permissionChannel, finishEating)
}
wg.Wait()
}
func eat(philosopherId int, left *sync.Mutex, right *sync.Mutex, wg *sync.WaitGroup, permissionChannel <-chan bool, finishEatingChannel chan<- bool) {
defer wg.Done()
for i := 1; i <= numOfMeals; i++ {
//lock chopsticks in random order
if RandBool() {
left.Lock()
right.Lock()
} else {
right.Lock()
left.Lock()
}
fmt.Printf("waiting for permission from host %d\n", philosopherId)
<-permissionChannel
fmt.Printf("starting to eat %d (time %d)\n", philosopherId, i)
fmt.Printf("finish to eat %d (time %d)\n", philosopherId, i)
//release chopsticks
left.Unlock()
right.Unlock()
//let host know I am done eating
finishEatingChannel <- true
}
}
func permissionFromHost(permissionChannel chan<- bool, finishEating <-chan bool) {
ctr := 0
for {
if ctr < maxEaters {
select {
case <-finishEating:
ctr--
case permissionChannel <- true:
ctr++
}
} else {
<-finishEating
ctr--
}
}
}
func RandBool() bool {
rand.Seed(time.Now().UnixNano())
return rand.Intn(2) == 1
}
I would like to understand why this case deadlock and why it's not in the other case.
If I close the channel inside the goroutine, it works fine, but if I close it after the WaitGroup.Wait() it cause a deadlock.
package main
import (
"fmt"
"io/ioutil"
"os"
"sync"
)
var (
wg = sync.WaitGroup{}
links = make(chan string)
)
func rec_readdir(depth int, path string) {
files, _ := ioutil.ReadDir(path)
for _, f := range files {
if symlink, err := os.Readlink(path + "/" + f.Name()); err == nil {
links <- path + "/" + symlink
}
rec_readdir(depth+1, path+"/"+f.Name())
}
if depth == 0 {
wg.Done()
// close(links) // if close here ok
}
}
func main() {
wg.Add(1)
go rec_readdir(0, ".")
for slink := range links {
fmt.Println(slink)
}
wg.Wait()
close(links) // if close here deadlock
}
https://play.golang.org/p/Ntl_zsV5nwO
for slink := range links will continue looping until the channel is closed. So you obviously can't close after that loop. When you do, you get a deadlock, as you have observed.
I recently started learning go and I am really impressed with all the features. I been playing with go routines and term-ui and facing some trouble. I am trying to exit this code from console after I run it but it just doesn't respond. If I run it without go-routine it does respond to my q key press event.
Any help is appreciated.
My code
package main
import (
"fmt"
"github.com/gizak/termui"
"time"
"strconv"
)
func getData(ch chan string) {
i := 0
for {
ch <- strconv.Itoa(i)
i++
time.Sleep(time.Second)
if i == 20 {
break
}
}
}
func Display(ch chan string) {
err := termui.Init()
if err != nil {
panic(err)
}
defer termui.Close()
termui.Handle("/sys/kbd/q", func(termui.Event) {
fmt.Println("q captured")
termui.Close()
termui.StopLoop()
})
for elem := range ch {
par := termui.NewPar(elem)
par.Height = 5
par.Width = 37
par.Y = 4
par.BorderLabel = "term ui example with chan"
par.BorderFg = termui.ColorYellow
termui.Render(par)
}
}
func main() {
ch := make(chan string)
go getData(ch)
Display(ch)
}
This is possibly the answer you are looking for. First off, you aren't using termui correctly. You need to call it's Loop function to start the Event loop so that it can actually start listening for the q key. Loop is called last because it essentially takes control of the main goroutine from then on until StopLoop is called and it quits.
In order to stop the goroutines, it is common to have a "stop" channel. Usually it is a chan struct{} to save memory because you don't ever have to put anything in it. Wherever you want the goroutine to possibly stop and shutoff (or do something else perhaps), you use a select statement with the channels you are using. This select is ordered, so it will take from them in order unless they block, in which case it tries the next one, so the stop channel usually goes first. The stop channel normally blocks, but to get it to take this path, simply close()ing it will cause this path to be chosen in the select. So we close() it in the q keyboard handler.
package main
import (
"fmt"
"github.com/gizak/termui"
"strconv"
"time"
)
func getData(ch chan string, stop chan struct{}) {
i := 0
for {
select {
case <-stop:
break
case ch <- strconv.Itoa(i):
}
i++
time.Sleep(time.Second)
if i == 20 {
break
}
}
}
func Display(ch chan string, stop chan struct{}) {
for {
var elem string
select {
case <-stop:
break
case elem = <-ch:
}
par := termui.NewPar(elem)
par.Height = 5
par.Width = 37
par.Y = 4
par.BorderLabel = "term ui example with chan"
par.BorderFg = termui.ColorYellow
termui.Render(par)
}
}
func main() {
ch := make(chan string)
stop := make(chan struct{})
err := termui.Init()
if err != nil {
panic(err)
}
defer termui.Close()
termui.Handle("/sys/kbd/q", func(termui.Event) {
fmt.Println("q captured")
close(stop)
termui.StopLoop()
})
go getData(ch, stop)
go Display(ch, stop)
termui.Loop()
}
This question already has answers here:
What is channel buffer size?
(3 answers)
Closed 7 years ago.
I went through a series of definitions to figure out how the buffer works but I just don't get it. Here is an example below, I changed the value of the buffer but I have no clue about what it does. Can some one explain it to me based on this example and provide some test cases of how/why it's working? Thanks.
package main
import (
"fmt"
"time"
)
func send(out, finish chan bool) {
for i := 0; i < 5; i++ {
out <- true
time.Sleep(1 * time.Second)
fmt.Println("Fin d'une écriture")
}
finish <- true
close(out)
}
func recv(in, finish chan bool) {
for _ = range in {
fmt.Println("Fin d'une lecture")
time.Sleep(10 * time.Second)
}
finish <- true
}
func main() {
chanFoo := make(chan bool, 3)
chanfinish := make(chan bool)
go send(chanFoo, chanfinish)
go recv(chanFoo, chanfinish)
<-chanfinish
<-chanfinish
}
If a channel does not have a buffer then only a single item can be sent on it at a time. This means code that sends on it will block until some receiver reads the item out of the channel. Here's a contrived example; https://play.golang.org/p/HM8jdIFqsN
package main
import (
"fmt"
)
func main() {
blocker := make(chan bool)
nonBlocker := make(chan bool, 5)
for i := 0; i < 5; i++ {
nonBlocker <- true
fmt.Println("We keep going")
}
go func () {
for i := 0; i < 5; i++ {
blocker <- true
fmt.Println("We block cause that channel is full")
} }()
}
There's plenty of other things I could do to demonstrate the same but the basic idea is, if you pass a channel into some goroutine and the channel is not buffered, the goroutine which sends on the channel will block until the item it sent is received. With a buffered channel you can send as long as the buffer isn't at capacity. Basically, if you spin up goroutines which are doing work and returning the results and they're moving faster than the code that spawned them, you may want to use a buffered channel to open up that bottle neck.
EDIT: If it's still not obvious what's happening look at this; https://play.golang.org/p/9SXc4M1to4
package main
import (
"fmt"
)
func main() {
blocker := make(chan bool)
nonBlocker := make(chan bool, 5)
for i := 0; i < 5; i++ {
nonBlocker <- true
fmt.Println("We keep going")
}
go func () {
for i := 0; i < 5; i++ {
blocker <- true
fmt.Println("Now we see this cause the reciever keeps opening the channel up again!")
} }()
for i := 0; i < 5; i++ {
<-blocker
}
}
I am trying to understand how goroutines and channels work. I have a loop sending values to a channel and I'd like to iterate over all the values the channel sends until it is closed.
I have written a simple example here:
package main
import (
"fmt"
)
func pinger(c chan string) {
for i := 0; i < 3; i++ {
c <- "ping"
}
close(c)
}
func main() {
var c chan string = make(chan string)
go pinger(c)
opened := true
var msg string
for opened {
msg, opened = <-c
fmt.Println(msg)
}
}
This gives the expected result but I'd like to know if there is a shorter way of doing this.
Many thanks for your help
You can use the range over the channel. The loop will continue until the channel is closed as you want:
package main
import (
"fmt"
)
func pinger(c chan string) {
for i := 0; i < 3; i++ {
c <- "ping"
}
close(c)
}
func main() {
var c chan string = make(chan string)
go pinger(c)
for msg := range c {
fmt.Println(msg)
}
}