Closing a channel after a timeout - go

Question

I have a small Go program that makes a number of requests every tick (1 second). I'm attempting to make these requests concurrently. I want to count and log the number of successful requests made in one tick, and then move on. If requests don't complete in time, I don't want to block the main ticker.
The code below achieves this, but I don't believe I'm closing the channel in concurrentReqs correctly. As any requests that miss the deadline still log with the previous tick. I also believe the ticker in the main function will block waiting for the concurrentReqs to finish. I tried moving the close(ch) inside of the timeout case in my select, but this results in a 'send on closed channel' error.
My understanding is that using contexts with a deadline (probably set in my main ticker) might be a solution for this, but I'm struggling to wrap my head around them, and I wonder if there's something else I can try.
Note: the timeout in concurrentReqs is deliberately low, since I'm testing locally.
package main
import (
"fmt"
"time"
"net/http"
)
type response struct {
num int
statusCode int
requestDuration time.Duration
}
func singleRequest(url string, i int, tick int) response {
start := time.Now()
client := http.Client{ Timeout: 100 * time.Millisecond }
resp, _ := client.Get(url)
fmt.Printf("%d: %d\n", tick, i)
defer resp.Body.Close()
return response{statusCode: int(resp.StatusCode), requestDuration: time.Since(start)}
}
func concurrentReqs(url string, reqsPerTick int, tick int) (results []response){
ch := make(chan response, reqsPerTick)
timeout := time.After(20 * time.Millisecond) // deliberately low
results = make([]response, 0)
for i := 0; i < reqsPerTick; i++ {
go func(i int, t int) {
ch <- singleRequest(url, i, tick)
}(i, tick)
}
for i := 0; i < reqsPerTick; i++ {
select {
case response := <- ch:
results = append(results, response)
case <- timeout:
return
}
}
close(ch)
return
}
func main() {
var url string = "http://end-point.svc/req"
c := time.Tick(1 * time.Second)
for next := range c {
things := concurrentReqs(url, 100, next.Second())
fmt.Printf("%s: Successful Reqs - %d\n", int(next.Second()), len(things))
}
}

Answer 1

I suggest to use context with timeout for cancellation and timing out. Also I think using wait group and mutex protected result writing helps simplicity here by eliminating second loop.
package main
import (
"context"
"fmt"
"log"
"net/http"
"sync"
"time"
)
type response struct {
num int
statusCode int
requestDuration time.Duration
}
func singleRequest(ctx context.Context, url string, i int, tick int) (response, error) {
start := time.Now()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return response{requestDuration: time.Since(start)}, err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return response{requestDuration: time.Since(start)}, err
}
fmt.Printf("%d: %d\n", tick, i)
defer resp.Body.Close()
return response{statusCode: int(resp.StatusCode), requestDuration: time.Since(start)}, nil
}
func concurrentReqs(url string, reqsPerTick int, tick int) (results []response) {
mu := sync.Mutex{}
results = make([]response, 0)
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond)
defer cancel()
wg := sync.WaitGroup{}
for i := 0; i < reqsPerTick; i++ {
wg.Add(1)
go func(i int, t int) {
defer wg.Done()
response, err := singleRequest(ctx, url, i, tick)
if err != nil {
log.Print(err)
return
}
mu.Lock()
results = append(results, response)
mu.Unlock()
}(i, tick)
}
wg.Wait()
return results
}
func main() {
var url string = "http://end-point.svc/req"
c := time.Tick(1 * time.Second)
for next := range c {
// You may want to wrap this in a goroutine to make sure tick is not skipped.
// Otherwise if concurrentReqs takes more than a tick time for whatever reason, a tick will be skipped.
things := concurrentReqs(url, 100, next.Second())
fmt.Printf("%s: Successful Reqs - %d\n", int(next.Second()), len(things))
}
}