I try to send an error in the channel on recovery
Why this error is not sent to the channel?
package main
import (
"fmt"
"sync"
"errors"
)
func main() {
var wg sync.WaitGroup
wg.Add(1)
batchErrChan := make(chan error)
go func(errchan chan error) {
defer func() {
if r := recover(); r != nil {
errchan <- errors.New("recover err")
}
close(batchErrChan)
wg.Done()
}()
panic("ddd")
}(batchErrChan)
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
}()
wg.Wait()
}
https://play.golang.org/p/0ytunuYDWZU
I expect "err in range" to be printed, but it is not. Why?
Your program ends before the goroutine gets a chance to print the message. Try waiting to it:
...
done:=make(chan struct{})
go func() {
for _ = range batchErrChan {
fmt.Println("err in range")
}
close(done)
}()
wg.Wait()
<-done
}
Related
This question already has answers here:
fatal error: all goroutines are asleep - deadlock! when channel is not buffered
(1 answer)
Why is the order of channels receiving causing/resolving a deadlock in Golang?
(3 answers)
Closed 4 months ago.
I wrote the following code trying to implement future for a group of async jobs. and it works. but I don't understand why I have to wrap wg.Wait() in a goroutine w/o it main will block.
package main
import (
"fmt"
"net/http"
"sync"
)
var urls = []string{
"https://www.easyjet.com/",
"https://www.skyscanner.de/",
"https://www.ryanair.com",
"https://wizzair.com/",
"https://www.swiss.com/",
}
func checker(urls []string) func() chan urlStatus {
wg := &sync.WaitGroup{}
c := make(chan urlStatus)
for _, url := range urls {
wg.Add(1)
go checkUrl(url, c, wg)
}
return func() chan urlStatus {
go func() { // <--- why this goroutine is necessary. change it to sync will block
wg.Wait()
close(c)
}()
return c
}
}
func main() {
done := make(chan bool)
future := checker(urls)
go consumer(future(), done)
<-done
}
func consumer(c <-chan urlStatus, done chan bool) {
for r := range c {
if r.status {
fmt.Println(r.url, "is up.")
} else {
fmt.Println(r.url, "is down!!")
}
}
fmt.Println("end of consumer")
done <- true
}
//checks and prints a message if a website is up or down
func checkUrl(url string, c chan<- urlStatus, wg *sync.WaitGroup) {
defer wg.Done()
_, err := http.Get(url)
if err != nil {
// The website is down
c <- urlStatus{url, false}
} else {
// The website is up
c <- urlStatus{url, true}
}
}
type urlStatus struct {
url string
status bool
}
I think it is because of this line
go consumer(future(), done)
When future() is called, the main go routine is blocked by wg.Wait()
I'm looking for the simplest code for looping over a dataset in parallel. The requirements are that the number of goroutines is fixed and that they can return an error code. The following is a quick attempt which doesn't work, since the loops will deadlock as both goroutines are blocking on the error channel
package main
import (
"fmt"
"sync"
)
func worker(wg *sync.WaitGroup, intChan chan int, errChan chan error) {
defer wg.Done()
for i := range intChan {
fmt.Printf("Got %d\n", i)
errChan <- nil
}
}
func main() {
ints := []int{1,2,3,4,5,6,7,8,9,10}
intChan := make(chan int)
errChan := make(chan error)
wg := new(sync.WaitGroup)
for i := 0; i < 2; i++ {
wg.Add(1)
go worker(wg, intChan, errChan)
}
for i := range ints {
intChan <- i
}
for range ints {
err := <- errChan
fmt.Printf("Error: %v\n", err)
}
close(intChan)
wg.Wait()
}
What is the simplest pattern for doing this?
Listen for errors in a goroutine:
go func() {
for err:=range errChan {
// Deal with err
}
}()
for i := 0; i < 2; i++ {
wg.Add(1)
go worker(wg, intChan, errChan)
}
for i := range ints {
intChan <- i
}
close(errChan) // Error listener goroutine terminates after this
This code is for a fairly simple demo for my programming language class. I'm trying to display a few different techniques that Go allows, like interfaces and concurrency, but I can't seem to get the WaitGroups to work right, so it keeps deadlocking on me at the end. My biggest question is: how do I get the WaitGroups to synchronize correctly and not deadlock the system when the goroutines stop? I'm most likely missing something obvious.
package main
import (
"bufio"
"fmt"
"os"
"sync"
)
func Reader(wg *sync.WaitGroup, message chan string, done chan bool){
defer wg.Done()
reader := bufio.NewReader(os.Stdin)
for {
msg, _ := reader.ReadString('\n')
if msg == "exit\n" {
<-done
return
} else {
message <- msg
}
}
}
func main() {
message := make(chan string)
done := make(chan bool)
wg := &sync.WaitGroup{}
wg.Add(1)
go Reader(wg, message, done)
wg.Add(1)
go func(){
defer wg.Done()
for {
select {
case <-done:
return
case msg := <-message:
fmt.Println("main: "+msg)
}
}
}()
wg.Wait()
close(message)
close(done)
}
Your break statement in main breaks the select, not the for loop. Use return or a label instead:
go func() {
defer wg.Done()
for {
select {
case <-done:
return // don't break here without label
case msg := <-message:
fmt.Println("main: " + msg)
}
}
}()
In addition, both functions try to receive from done. Reader should close the channel instead to signal completion:
func Reader(wg *sync.WaitGroup, message chan string, done chan bool) {
defer wg.Done()
defer close(done) // close channel to signal completion
reader := bufio.NewReader(os.Stdin)
for {
msg, _ := reader.ReadString('\n')
if msg == "exit\n" {
return
} else {
message <- msg
}
}
}
Don't close the channel in main. Channels should always be closed on the sender's side.
Once you have done all that you should recognize that message and done are redunant. The whole program can be simplified to:
package main
import (
"bufio"
"fmt"
"os"
)
func Reader(message chan string) {
defer close(message)
reader := bufio.NewReader(os.Stdin)
for {
msg, _ := reader.ReadString('\n')
if msg == "exit\n" {
return
} else {
message <- msg
}
}
}
func main() {
message := make(chan string)
go Reader(message)
for msg := range message {
fmt.Println("main: " + msg)
}
}
I use sync.WaitGroup, defer wg.Close() and wg.Wait() to wait for my goroutines to complete.
The program do wait, but it never exits.
This is my program (runnable):
package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"sync"
)
var symbols = []string{
"ASSA-B.ST",
"ELUX-B.ST",
"HM-B.ST",
}
func main() {
fmt.Println("fetching quotes...")
fetchedSymbols := make(chan string)
var wg sync.WaitGroup
wg.Add(len(symbols))
for _, symbol := range symbols {
go fetchSymbol(symbol, &wg, fetchedSymbols)
}
for response := range fetchedSymbols {
fmt.Println("fetched " + response)
}
wg.Wait()
fmt.Println("done")
}
func fetchSymbol(symbol string, wg *sync.WaitGroup, c chan<- string) {
defer wg.Done()
resp, err := http.Get("http://ichart.yahoo.com/table.csv?s=" + symbol + "&a=0&b=1&c=2000")
defer resp.Body.Close()
if err != nil {
log.Fatal(err)
}
out, err := os.Create("./stock-quotes/" + symbol + ".csv")
defer out.Close()
if err != nil {
log.Fatal(err)
}
io.Copy(out, resp.Body)
c <- symbol
}
Shouldn't this program exit when all the quotes have been downloaded? (FYI: I just started learning GO)
You're never closing the fetchedSymbols channel, so that range loop will never exit.
One way to handle this is to use use the WaitGroup you already have to signal when to close the channel. Ranging over fetchedSymbols is enough to block the progress in main, and you don't need another channel or WaitGroup.
...
go func() {
wg.Wait()
close(fetchedSymbols)
}()
for response := range fetchedSymbols {
fmt.Println("fetched " + response)
}
...
I have the following golang program;
package main
import (
"fmt"
"net/http"
"time"
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://yahoo.com/",
}
type HttpResponse struct {
url string
response *http.Response
err error
status string
}
func asyncHttpGets(url string, ch chan *HttpResponse) {
client := http.Client{}
if url == "http://www.google.com/" {
time.Sleep(500 * time.Millisecond) //google is down
}
fmt.Printf("Fetching %s \n", url)
resp, err := client.Get(url)
u := &HttpResponse{url, resp, err, "fetched"}
ch <- u
fmt.Println("sent to chan")
}
func main() {
fmt.Println("start")
ch := make(chan *HttpResponse, len(urls))
for _, url := range urls {
go asyncHttpGets(url, ch)
}
for i := range ch {
fmt.Println(i)
}
fmt.Println("Im done")
}
Run it on Playground
However when I run it; it hangs (ie the last part that ought to print Im done doesnt run.)
Here's the terminal output;;
$ go run get.go
start
Fetching http://yahoo.com/
Fetching http://golang.org/
Fetching http://www.google.com/
sent to chan
&{http://www.google.com/ 0xc820144120 fetched}
sent to chan
&{http://golang.org/ 0xc82008b710 fetched}
sent to chan
&{http://yahoo.com/ 0xc82008b7a0 fetched}
The problem is that ranging over a channel in a for loop will continue forever unless the channel is closed. If you want to read precisely len(urls) values from the channel, you should loop that many times:
for i := 0; i < len(urls); i++ {
fmt.Println(<-ch)
}
Another good dirty devious trick would be to use sync.WaitGroup and increment it per goroutine and then monitor it with a Wait and after its done it will close your channel allowing the next blocks of code to run, the reason I am offering you this approach is because it gets away from using a static number in a loop like len(urls) so that you can have a dynamic slice that might change and what not.
The reason Wait and close are in their own goroutine is so that your code can reach the for loop to range over your channel
package main
import (
"fmt"
"net/http"
"time"
"sync"
)
var urls = []string{
"http://www.google.com/",
"http://golang.org/",
"http://yahoo.com/",
}
type HttpResponse struct {
url string
response *http.Response
err error
status string
}
func asyncHttpGets(url string, ch chan *HttpResponse, wg *sync.WaitGroup) {
client := http.Client{}
if url == "http://www.google.com/" {
time.Sleep(500 * time.Millisecond) //google is down
}
fmt.Printf("Fetching %s \n", url)
resp, err := client.Get(url)
u := &HttpResponse{url, resp, err, "fetched"}
ch <- u
fmt.Println("sent to chan")
wg.Done()
}
func main() {
fmt.Println("start")
ch := make(chan *HttpResponse, len(urls))
var wg sync.WaitGroup
for _, url := range urls {
wg.Add(1)
go asyncHttpGets(url, ch, &wg)
}
go func() {
wg.Wait()
close(ch)
}()
for i := range ch {
fmt.Println(i)
}
fmt.Println("Im done")
}