Is there a way to get transfer speed from io.Copy? - go

Question

I am copying a network stream to a file using io.Copy. I would like to extract the current speed, preferably in bytes per second, that the transfer is operating at.
res, err := http.Get(url)
if err != nil {
panic(err)
}
// Open output file
out, err := os.OpenFile("output", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
panic(err)
}
// Close output file as well as body
defer out.Close()
defer func(Body io.ReadCloser) {
err := Body.Close()
if err != nil {
panic(err)
}
}(res.Body)
_, err := io.Copy(out, res.Body)

Answer 1

As noted in the comments - the entire transfer rate is easily computed after the fact - especially when using io.Copy. If you want to track "live" transfer rates - and poll the results over a long file transfer - then a little more work is involved.
Below I've outlined a simple io.Reader wrapper to track the overall transfer rate. For brevity, it is not goroutine safe, but would be trivial do make it so. And then one could poll from another goroutine the progress, while the main goroutine does the reading.
You can create a io.Reader wrapper - and use that to track the moment of first read - and then track future read byte counts. The final result may look like this:
r := NewRater(resp.Body) // io.Reader wrapper
n, err := io.Copy(out, r)
log.Print(r) // stringer method shows human readable "b/s" output
To implement this, one approach:
type rate struct {
r io.Reader
count int64 // may have large (2GB+) files - so don't use int
start, end time.Time
}
func NewRater(r io.Reader) *rate { return &rate{r: r} }
then we need the wrapper Read to track the underlying io.Readers progress:
func (r *rate) Read(b []byte) (n int, err error) {
if r.start.IsZero() {
r.start = time.Now()
}
n, err = r.r.Read(b) // underlying io.Reader read
r.count += int64(n)
if err == io.EOF {
r.end = time.Now()
}
return
}
the rate at any time can be polled like so - even before EOF:
func (r *rate) Rate() (n int64, d time.Duration) {
end := r.rend
if end.IsZero() {
end = time.Now()
}
return r.count, end.Sub(r.start)
}
and a simple Stringer method to show b/s:
func (r *rate) String() string {
n, d := r.Rate()
return fmt.Sprintf("%.0f b/s", float64(n)/(d.Seconds()))
}
Note: the above io.Reader wrapper has no locking in place, so operations must be from the same goroutine. Since the question relates to io.Copy - then this is a safe assumption to make.