Deadlock in channels communication between two packages - Golang.
I have two packages that are communicated by two channels. One is main and the other has a function. When I run it, I obtain a deadlock.
package main
import (
functionspackage "GoEjemplos/subFolder"
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
ChannelSendData := make(chan functionspackage.FunctionStruct, 1)
defer close(ChannelSendData)
data := functionspackage.FunctionStruct{
FieldOne: 3.56,
FieldTwo: 23,
}
ChannelSendData <- data
wg.Add(1)
go functionspackage.FunctionExt(ChannelSendData, &wg)
recibe := <-functionspackage.ChannelOutFunct
fmt.Println("channelOut: ", recibe)
close(functionspackage.ChannelOutFunct)
wg.Wait()
}
The other package is
package functionspackage
import "sync"
type FunctionStruct struct {
FieldOne float64
FieldTwo int
}
var ChannelOutFunct chan float64
func FunctionExt(RecibeChan chan FunctionStruct, wg *sync.WaitGroup) (ChannelOutFunct chan float64) {
reciveData := <-RecibeChan
result := reciveData.FieldOne * float64(reciveData.FieldTwo)
ChannelOutFunct <- result
wg.Done()
return ChannelOutFunct
}
This is the deadlock.
PS C:\Go-Project\src\GoEjemplos> go run main.go
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive (nil chan)]:
main.main()
C:/Go-Project/src/GoEjemplos/main.go:32 +0x13d
goroutine 19 [chan send (nil chan)]:
GoEjemplos/subFolder.FunctionExt(0xc0000d4000, 0xc0000a2070, 0xc0000c9f18)
C:/Go-Project/src/GoEjemplos/subFolder/functionsPackage.go:19 +0x85
created by main.main
C:/Go-Project/src/GoEjemplos/main.go:30 +0x11a
exit status 2
PS C:\Go-Project\src\GoEjemplos>
Could you explain it to me where is the problem?
Thanks!
The channel functionspackage.ChannelOutFunct is not initialized, thus it is a nil channel. Writing to a nil-channel or reading from a nil-channel will always block.
https://dave.cheney.net/2014/03/19/channel-axioms
I got it to work. This is the code
package main
import (
packagefunctions "GoEjemplos/subFolder"
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
var ChannelSendData = make(chan packagefunctions.FunctionStruct, 0)
defer close(ChannelSendData)
var ChanReturn = make(chan float64)
defer close(ChanReturn)
data := packagefunctions.FunctionStruct{
FieldOne: 3.56,
FieldTwo: 2,
}
wg.Add(1)
go func() { ChannelSendData <- data }()
wg.Add(1)
go func() {
ChanReturn = packagefunctions.FunctionExt(ChannelSendData, &wg)
recibeChanReturn := <-ChanReturn
fmt.Println("channelOut: ", recibeChanReturn)
wg.Done()
}()
wg.Wait()
}
the other package is
package packagefunctions
import (
"fmt"
"sync"
)
type FunctionStruct struct {
FieldOne float64
FieldTwo int
}
func FunctionExt(ChanIn chan FunctionStruct, wg *sync.WaitGroup) chan float64 {
reciveData, ok := <-ChanIn
if ok == false {
fmt.Println("channel closed")
}
var result float64
result = reciveData.FieldOne * float64(reciveData.FieldTwo)
var ChannelReturn = make(chan float64, 1)
defer close(ChannelReturn)
ChannelReturn <- result
wg.Done()
return ChannelReturn
}
Related
My task is to sync 2 goroutines so the output should look like that:
foobarfoobarfoobarfoobar
.The issue is that when I call them they come out completely randomized. This is my code:
package main
import (
"fmt"
"sync"
"time"
)
type ConcurrentPrinter struct {
sync.WaitGroup
sync.Mutex
}
func (cp *ConcurrentPrinter) printFoo(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("foo")
cp.Unlock()
}()
}
func (cp *ConcurrentPrinter) printBar(times int) {
cp.WaitGroup.Add(times)
go func() {
cp.Lock()
fmt.Print("bar")
cp.Unlock()
}()
}
func main() {
times := 10
cp := &ConcurrentPrinter{}
for i := 0; i <= times; i++ {
cp.printFoo(i)
cp.printBar(i)
}
time.Sleep(10 * time.Millisecond)
}
As outlined in the comments, using goroutines may not be the best use case for what you are trying to achieve - and thus this may be an XY problem.
Having said that, if you want to ensure two independent goroutines interleave their work in an alternating sequence, you can implement a set of "ping-pong" mutexs:
var ping, pong sync.Mutex
pong.Lock() // ensure the 2nd goroutine waits & the 1st goes first
go func() {
for {
ping.Lock()
foo()
pong.Unlock()
}
}()
go func() {
for {
pong.Lock()
bar()
ping.Unlock()
}
}()
https://go.dev/play/p/VO2LoMJ8fek
Using channel:
func printFoo(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("foo")
ch <- true
}()
}
func printBar(i int, ch chan<- bool, wg *sync.WaitGroup) {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Print("bar")
ch <- true
}()
}
func main() {
times := 4
firstchan := make(chan bool)
secondchan := make(chan bool)
var wg sync.WaitGroup
for i := 0; i <= times; i++ {
printFoo(i, firstchan, &wg)
<-firstchan
printBar(i, secondchan, &wg)
<-secondchan
}
wg.Wait()
}
https://go.dev/play/p/MlZ9dHkUXGb
I'm trying to figure out why I have a dead lock with waitgroup.Wait()
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func foo(c chan int, i int) {
defer wg.Done()
c <- i
}
func main() {
ch := make(chan int)
for i := 0; i < 10; i++ {
wg.Add(1)
go foo(ch, i)
}
wg.Wait()
close(ch)
for item := range ch {
fmt.Println(item)
}
}
When I run it like this, it prints fatal error: all goroutines are asleep - deadlock!
I tried to change ch to a buffered channel and that solved the problem. But I really want to know why is there a dead lock.
I've commented out the parts where your program's logic is not correct:
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func foo(c chan int, i int) {
defer wg.Done()
c <- i
}
func main() {
ch := make(chan int) // unbuffered channel
for i := 0; i < 10; i++ {
wg.Add(1)
go foo(ch, i)
}
// wg.Wait is waiting for all goroutines to finish but that's
// only possible if the send to channel succeeds. In this case,
// it is not possible as your receiver "for item := range ch" is below
// this. Hence, a deadlock.
wg.Wait()
// Ideally, it should be the sender's duty to close the channel.
// And closing a channel before the receiver where the channel
// is unbuffered is not correct.
close(ch)
for item := range ch {
fmt.Println(item)
}
}
Corrected program:
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func foo(c chan int, i int) {
defer wg.Done()
c <- i
}
func main() {
ch := make(chan int)
go func() {
for item := range ch {
fmt.Println(item)
}
}()
for i := 0; i < 10; i++ {
wg.Add(1)
go foo(ch, i)
}
wg.Wait()
close(ch)
}
Say I have the following code (playground):
package main
import (
"fmt"
"sync"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
go func() {
wg.Add(1)
defer wg.Done()
s = createStr()
}()
wg.Wait()
fmt.Printf("s after: %v", s)
}
I would have expected s to not equal nil.
However, if I add a small wait, I get s != nil (playground):
package main
import (
"fmt"
"sync"
"time"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
go func() {
wg.Add(1)
defer wg.Done()
s = createStr()
}()
wg.Wait()
time.Sleep(time.Second)
fmt.Printf("s after: %v", s)
}
This just caused a bug in a program I wrote. What is happening with sync.WaitGroup that's not causing my program to wait for s to be assigned a string pointer in my go func?
Your placement of Add method for sync.WaitGroup is wrong. Do not use Add inside a goroutine (anonymous goroutine here) but use it in the goroutine (main goroutine here) that's going to wait for it.
A probable situation that was happening in your code was that wg.Wait() didn't wait as the goroutine's wg.Add(1) wasn't called yet and hence s == nil. The following code fixes the issue:
Go Playground
package main
import (
"fmt"
"sync"
)
func createStr() *string {
tmp := "foo"
return &tmp
}
func main() {
var (
s *string
wg sync.WaitGroup
)
// Use wg.Add() here
wg.Add(1)
go func() {
defer wg.Done()
s = createStr()
}()
wg.Wait()
fmt.Printf("s after: %v", *s)
}
There are several channels to monitor, their type are different and irrelevant(since we only care about len and cap), but golang compiler does not accept following code, whatever T is:
func monitorChan(ch chan T) {
for {
if len(ch) == cap(ch) {
log.Warn("log")
}
time.Sleep(chanMonitorInterval)
}
}
it shows error:
cannot use ch (type chan []byte) as type chan interface {} in argument
to monitorChan.
How can this function be modified to write once monitor every channel?
Here is my code:
package main
import (
"fmt"
"time"
)
func monitorChan(ch chan interface{}) {
for {
if len(ch) == cap(ch) {
fmt.Println("log")
}
time.Sleep(1 * time.Second)
}
}
func main() {
ch := make(chan []byte, 100)
go monitorChan(ch)
// actual things below ...
}
Playground: https://play.golang.org/p/t7T28IpLNAs
Use reflection. For example,
package main
import (
"log"
"reflect"
"time"
)
func monitorChan(ch interface{}, intvl time.Duration) {
v := reflect.ValueOf(ch)
if v.Kind() != reflect.Chan {
return
}
c := v.Cap()
if c == 0 {
return
}
for {
if l := v.Len(); l == c {
log.Printf("log: len(%d) cap(%d)", l, c)
}
time.Sleep(intvl)
}
}
func main() {
log.Print("main")
c := make(chan []byte, 10)
var chanMonitorInterval = 1 * time.Second
go monitorChan(c, chanMonitorInterval)
log.Print("monitor")
time.Sleep(5 * chanMonitorInterval)
for len(c) != cap(c) {
c <- []byte{}
}
log.Print("len(c) == cap(c)")
time.Sleep(3 * chanMonitorInterval)
<-c
log.Print("len(c) < cap(c)")
time.Sleep(5 * chanMonitorInterval)
log.Print("main")
}
Playground: https://play.golang.org/p/c5VhIIO0pik
Output:
2009/11/10 23:00:00 main
2009/11/10 23:00:00 monitor
2009/11/10 23:00:05 len(c) == cap(c)
2009/11/10 23:00:06 log: len(10) cap(10)
2009/11/10 23:00:07 log: len(10) cap(10)
2009/11/10 23:00:08 log: len(10) cap(10)
2009/11/10 23:00:08 len(c) < cap(c)
2009/11/10 23:00:13 main
References:
Package reflect
The Go Blog: The Laws of Reflection
Go 1.18
This is now trivial to accomplish, using type parameters:
func monitorChan[T any](ch chan T) {
for {
if len(ch) == cap(ch) {
log.Warn("log")
}
time.Sleep(chanMonitorInterval)
}
}
Type inference also allows to infer the type parameter T from the concrete type of the function argument, so the code in main doesn't even need to be rewritten:
func main() {
ch := make(chan []byte, 100)
go monitorChan(ch) // T instantiated as []byte
// actual things below ...
}
Create an interface{} type channel and pass any type wrapping around interface{}, then fetch the use type assert on receiving end.
package main
import (
"fmt"
"sync"
)
var wg sync.WaitGroup
func monitorChan(ch chan interface{}) {
val := <-ch
fmt.Println(string(val.(interface{}).([]uint8)))
wg.Done()
}
func main() {
ch := make(chan interface{}, 100)
wg.Add(1)
ch <- []byte("hello")
go monitorChan(ch)
wg.Wait()
// actual things below ...
}
Working code on Go Playground
Edited :- you can also go for reflect package to get the values of channels after wrapping the channels inside interface{}
package main
import (
"fmt"
"sync"
"reflect"
)
var wg sync.WaitGroup
func monitorChan(i interface{}) {
defer wg.Done()
v := reflect.ValueOf(i)
fmt.Printf("%s size: %d/%d\n", v.Kind(), v.Len(), v.Cap())
}
func main() {
ch := make(chan []byte, 100)
wg.Add(1)
go monitorChan(ch)
wg.Wait()
// actual things below ...
}
Playground example
I spend my evening looking at how to fix this error but I haven't succeeded. When I run the program I have the following error : "all goroutines are asleep - deadlock!". I understand this is because the main program is exited before the routine has the possibility to do its tasks and I thought using sync.WaitGroup would help but not really :/
I want to set a number of routine and use channels to send urls in order to check http status code. I want to limit the number of concurrency call to a website. I've followed examples doing the same thing with string instead of struct and it worked.
Any help would be well appreciated :)
package main
import (
"fmt"
"sync"
"time"
)
const (
numPollers = 2 // number of Poller goroutines to launch
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://blog.golang.org/",
"http://golangtutorials.blogspot.fr",
"https://gobyexample.com/",
}
// Resource represents an HTTP URL to be polled by this program.
type Resource struct {
url string
}
func Poller(in <-chan *Resource, wg *sync.WaitGroup) {
//defer wg.Done()
for r := range in {
fmt.Printf("Finished: %v - %v\n", r.url, time.Now())
}
wg.Done()
}
func main() {
var wg sync.WaitGroup
pending := make(chan *Resource)
wg.Add(len(urls))
go Poller(pending, &wg)
go func() {
for _, url := range urls {
wg.Add(1)
fmt.Println("SENT > Pending url " + url)
pending <- &Resource{url: url}
}
}()
wg.Wait()
fmt.Printf("Finished all goroutines: %v\n", time.Now())
}
https://play.golang.org/p/B-HSiDo2Qg
First, you have too many calls to wg.Add(). You call that once for each goroutine you're running. See http://golang.org/pkg/sync/#WaitGroup. Second, you didn't close the channel after you were done writing to it. Here's a modified version of your code:
package main
import (
"fmt"
"sync"
"time"
)
const (
numPollers = 2 // number of Poller goroutines to launch
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://blog.golang.org/",
"http://golangtutorials.blogspot.fr",
"https://gobyexample.com/",
}
// Resource represents an HTTP URL to be polled by this program.
type Resource struct {
url string
}
func Poller(in <-chan *Resource, wg *sync.WaitGroup) {
defer wg.Done()
for r := range in {
fmt.Printf("Finished: %v - %v\n", r.url, time.Now())
}
}
func main() {
var wg sync.WaitGroup
pending := make(chan *Resource)
wg.Add(2)
go Poller(pending, &wg)
go func() {
defer close(pending)
defer wg.Done()
for _, url := range urls {
fmt.Println("SENT > Pending url " + url)
pending <- &Resource{url: url}
}
}()
wg.Wait()
fmt.Printf("Finished all goroutines: %v\n", time.Now())
}
and https://play.golang.org/p/ucUlZEZMZM
You forget to close channel, and your wait group is too long. This works for me: https://play.golang.org/p/yasIzaCbmQ