http: read on closed response body - httptest.NewServer - go

Question

I am trying to get to grips with testing using the httptest.NewServer and I am hitting a roadblock.
In my code I am making a GET request to an external API and I want to write a test for this using httptest.NewServer.
Here is my code making the request (main.go):
package main
import (
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
)
type HTTPClient interface {
Do(req *http.Request) (*http.Response, error)
}
type NewRequest interface {
NewRequest(method string, url string, body io.Reader) (*http.Request, error)
}
var (
Client HTTPClient
)
func init() {
Client = &http.Client{}
}
func main() {
url := "https://httpbin.org/get"
GetData(url)
}
func GetData(url string) (*http.Response, error) {
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
log.Fatalln(err)
return nil, err
}
resp, err := Client.Do(req)
if err != nil {
log.Fatalln(err)
return nil, err
}
defer resp.Body.Close()
responseBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
return nil, err
}
fmt.Println(resp.Status)
fmt.Println(string(responseBody))
return resp, nil
}
When I run this it works fine.
Here is my test file:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"testing"
)
func TestYourHTTPGet(t *testing.T){
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, `response from the mock server goes here`)
}))
defer ts.Close()
mockServerURL := ts.URL
resp, err := GetData(mockServerURL)
if err != nil {
fmt.Println("Error 1: ", err)
}
defer resp.Body.Close()
responseBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal("Error 2: ", err)
}
fmt.Println(resp.Status)
fmt.Println(string(responseBody))
}
When I run go test I receive the error: http: read on closed response body. If I remove defer resp.Body.Close() from main.go the test passes correctly.
I am not sure why this is happening and was hoping that someone could explain what is going on here?

Answer 1

As #Cerise Limón says you call resp.Body.Close() twice and then try to read closed body. To fix yor code you can remove body processing from GetData function and do it outside GetData or return the body and do not read it in test.
main.go:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
)
var Client = &http.Client{}
func main() {
url := "https://httpbin.org/get"
status, data, err := GetData(url)
if err != nil {
log.Fatalln(err)
}
fmt.Println(status)
fmt.Println(string(data))
}
func GetData(url string) (status string, body []byte, err error) {
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return
}
resp, err := Client.Do(req)
if err != nil {
return
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return resp.Status, body, nil
}
main_test.go:
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
)
func TestYourHTTPGet(t *testing.T){
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, `response from the mock server goes here`)
}))
defer ts.Close()
mockServerURL := ts.URL
status, data, err := GetData(mockServerURL)
if err != nil {
fmt.Println("Error 1: ", err)
}
fmt.Println(status)
fmt.Println(string(data))
}

Answer 2

Your GetData()'s return is a pointer. You run GetData() in main.go, when retun, it will close the resp.body. And if you read it again, it cause http: read on closed response body
So if you want read the body again, you should not return *http.Response, you should clone the resp.body to return