Golang Type system inconsistency (http package) - go

I am trying to wrap my head around the GoLang type system, and there area a few things the confuse me.
So I have been working on the http library to try to understand this and I have come across the following that makes no sense.
package main
import (
"net/http"
"fmt"
"io/ioutil"
"io"
)
func convert(closer io.Closer) ([]byte) {
body, _ := ioutil.ReadAll(closer);
return body
}
func main() {
client := &http.Client{}
req, _ := http.NewRequest("GET", "https://www.google.com", nil)
response, _ := client.Do(req);
body, _ := ioutil.ReadAll(response.Body)
fmt.Println(body);
fmt.Println(convert(response.Body))
}
The Go Playground
this is not about the fact that the convert function is not needed it is the fact that the response body is of type io.closer and the ioutil.Readall takes a io.reader, yet I can pass it in, in one instance but not if in another. Is there something that I am missing that is magically happening.
I know that the closer technically passes the reader interface as it implements the the Read method but that should be true in both the function and the main body.
Any insight would be great.
Thanks

it is the fact that the response body is of type io.closer
No, it is not. Declaration of Request.Body is at http.Request:
Body io.ReadCloser
The Request.Body field is of type io.ReadCloser, it is both an io.Reader and an io.Closer.
Since it is an io.Reader (dynamic value of Request.Body implements io.Reader), you may use / pass it where an io.Reader is required, e.g. to ioutil.ReadAll().
Since it also implements io.Closer, you can also pass it where io.Closer is required, like your convert() function.
But inside convert the closer param has static type io.Closer, you can't use closer where an in.Reader is required. It might be (and in your case it is) that the dynamic type stored in closer also implements io.Reader, but there is no guarantee for this. Like in this example:
type mycloser int
func (mycloser) Close() error { return nil }
func main() {
var m io.Closer = mycloser(0)
convert(m)
}
In the above example closer inside convert() will hold a value of type mycloser, which truly does not implement io.Reader.
If your convert() function intends to treat its parameter also as an io.Reader, the parameter type should be io.ReadCloser:
func convert(rc io.ReadCloser) ([]byte, error) {
body, err := ioutil.ReadAll(rc)
if err != nil {
return body, err
}
err = rc.Close()
return body, err
}

Related

Golang: Intercepting and Mocking an HTTP Response with httptest

I've looked into various different tools that can be used for mock testing in golang, but I'm trying to accomplish this task using httptest. In particular, I have a function as such:
type contact struct {
username string
number int
}
func getResponse(c contact) string {
url := fmt.Sprintf("https://mywebsite/%s", c.username)
req, err := http.NewRequest(http.MethodGet, url, nil)
// error checking
resp, err := http.DefaultClient.Do(req)
// error checking
return response
}
A lot of the documentation I've read seems to require creating a client interface or a custom transport. Is there no way to mock a response in a test file without changing this main code at all? I want to keep my client, response, and all the related details within the getResponse function. I could have the wrong idea, but I'm trying to find a way to intercept the http.DefaultClient.Do(req) call and return a custom response, is that possible?
https://pkg.go.dev/net/http/httptest#example-Server is a good example for your use case with a small refactoring of your code.
You just have to change the getResponse() by getResponse(url string) to be able to give the server mock url.
I've read seems to require creating a client interface
without changing this main code at all
Keeping your code clean is a good practice and you'll finally get used to it, a testable code is cleaner and a cleaner code is more testable, so don't worry to change your code (using interfaces) so it can accept mock objects.
Your code in its simplest form can be like this:
package main
import (
"fmt"
"net/http"
)
type contact struct {
username string
number int
}
type Client interface {
Do(req *http.Request) (*http.Response, error)
}
func main() {
getResponse(http.DefaultClient, contact{})
}
func getResponse(client Client, c contact) string {
url := fmt.Sprintf("https://mywebsite/%s", c.username)
req, _ := http.NewRequest(http.MethodGet, url, nil)
// error checking
resp, _ := http.DefaultClient.Do(req)
// error checking and response processing
return response
}
And your test can be like this:
package main
import (
"net/http"
"testing"
)
type mockClient struct {
}
// Do function will cause mockClient to implement the Client interface
func (tc mockClient) Do(req *http.Request) (*http.Response, error) {
return &http.Response{}, nil
}
func TestGetResponse(t *testing.T) {
client := new(mockClient)
getResponse(client, contact{})
}
But if you prefer to use httptest:
package main
import (
"fmt"
"io"
"net/http"
"net/http/httptest"
)
type contact struct {
username string
number int
}
func main() {
fmt.Println(getResponse(contact{}))
}
func getResponse(c contact) string {
// Make a test server
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "your response")
}))
defer ts.Close()
// You should still set your base url
base_url := ts.URL
url := fmt.Sprintf("%s/%s", base_url, c.username)
req, _ := http.NewRequest(http.MethodGet, url, nil)
// Use ts.Client() instead of http.DefaultClient in your tests.
resp, _ := ts.Client().Do(req)
// Processing the response
response, _ := io.ReadAll(resp.Body)
resp.Body.Close()
return string(response)
}

Is this example of method overriding in Go?

package main
import (
"encoding/json"
"fmt"
"log"
"strings"
)
type Animal int
const (
Unknown Animal = iota
Gopher
Zebra
)
func (a *Animal) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
switch strings.ToLower(s) {
default:
*a = Unknown
case "gopher":
*a = Gopher
case "zebra":
*a = Zebra
}
return nil
}
func (a Animal) MarshalJSON() ([]byte, error) {
var s string
switch a {
default:
s = "unknown"
case Gopher:
s = "gopher"
case Zebra:
s = "zebra"
}
return json.Marshal(s)
}
func main() {
blob := `["gopher","armadillo","zebra","unknown","gopher","bee","gopher","zebra"]`
var zoo []Animal
if err := json.Unmarshal([]byte(blob), &zoo); err != nil {
log.Fatal(err)
}
census := make(map[Animal]int)
for _, animal := range zoo {
census[animal] += 1
}
fmt.Printf("Zoo Census:\n* Gophers: %d\n* Zebras: %d\n* Unknown: %d\n",
census[Gopher], census[Zebra], census[Unknown])
}
This is the code snippet of Json custom marshal example in go doc. My question is where is the call to MarshalJSON and UnmarshalJSON method in this code. Are these method somehow overriding Json package's UnmarshalJSON and MarshalJSON method. I thought go does not support method overriding this way. Pls help, i am not able to understand what is happening in this code!!
The documentation says:
To unmarshal JSON into a value implementing the Unmarshaler interface, Unmarshal calls that value's UnmarshalJSON method, including when the input is a JSON null.
Somewhere in the json.Unmarshal implementation, there's code similar this:
u, ok := v.(Unmarshaler)
if ok {
err := u.Unmarshal(data)
if err != nil { /* handle error */}
} else {
// handle other kinds of values
}
The code uses a type assertion to determine if the value satisfies the json.Unmarshaler interface. If the value does satisfy the method, the value's UnmarshalJSON function is called.
The (*Animal).UnmarshalJSON function is called because *Animal satisfies the json.Unmarshaler interface.
This is an example of implementing an interface from a different package.
There's no need to explicitly declare that you're implementing an interface in Go, like there is in Java or C++ for example. You just have to implement all the functions it declares. In this case, you're implementing the Unmarshaler interface declared in the json package which is used by the Unmarshal function.

How to get http.ResponseWriter and http.Request in Revel controller

I am trying to implement an oauth server and the package I am using needs the complete http.ResponseWriter and http.Request types.
c.Response does not contain all the methods that http.ResponseWriter does and c.Request gives error incompatible type.
How do I get http.ResponseWriter and http.Request in a Revel controller?
type client struct {
ClientId string
ClientSecret string
}
type App struct {
*revel.Controller
}
func (c App) TokenRequest() {
r := c.Request
w := c.Response
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
log.Println(string(body))
var cli client
err = json.Unmarshal(body, &cli)
if err != nil {
panic(err)
}
log.Println(cli.ClientId)
err = OauthSrv.HandleTokenRequest(w, r)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
Warning
I am generally not fond of frameworks like Revel in Go, for reasons that I hope demonstrate themselves on this page. My first recommendation would be that you examine closely what you are actually getting out of Revel that merits the use of such a heavy abstraction layer, and if it's really that valuable, you may want to ask questions going in the other direction, such as how one might make OauthSrv work within Revel's customized ecosystem.
Using a Revel controller as a ResponseWriter
For something to be an http.ResponseWriter, it just needs to have these methods.
Header
You need a method named Header() that returns an http.Header, which you can build out of any map[string][]string. Revel provides similar functionality, but through several layers of abstraction. You will need to unravel them:
c.Response is a *Response, so it has a field named Out containing an OutResponse.
An OutResponse has a Header() method—but it doesn't return an http.Header. Instead, it returns a *RevelHeader.
A *RevelHeader has a GetAll(key string) []string method—which is very similar to the API already provided by the built-in map type, but isn't exactly the same. So, you will need to copy the returned values into a new map every time Header() is called, in order to fully satisfy the function signature requirements.
Also, GetAll() requires you to know the key name you are interested in, and *RevelHeader on its own does not provide a way to look up which keys are available. For now we can rely on the fact that the current implementation only has one field, a ServerHeader that does provide a GetKeys() []string method.
Putting all this together, we can build our Header method:
func (rrw RevelResponseWrapper) Header() http.Header {
revelHeader := rrw.Response.Out.Header()
keys := revelHeader.Server.GetKeys()
headerMap := make(map[string][]string)
for _, key := range keys {
headerMap[key] = revelHeader.GetAll(key)
}
return http.Header(headerMap)
}
Write and WriteHeader
You would use similar anti-patterns to expose rrw.Write([]byte) (int, error) so that it calls through to c.Response.Out.Write(data []byte) (int, error), and rrw.WriteHeader(int) error so that it calls c.Response.WriteHeader(int, string). Depending on what is considered appropriate for the framework, either panic on errors or fail silently, since their API doesn't expect WriteHeader errors to be handle-able.
Getting an http.Request from Revel
Unfortunately, the http.Request type is a struct, so you can't just simulate it. You basically have two options: reconstruct it using the net/http package from all the properties you are able to access, or hope that the *revel.Request you have is secretly an http.Request under the hood. In the latter case, you can use a type assertion:
revelReq, ok := c.Request.In.(*revel.GoRequest)
if !ok {
// handle this somehow
}
r := revelReq.Original

gin/golang - Empty Req Body

I'm new to Go and Gin, and am having trouble printing out the full request body.
I want to be able to read the request body from third party POST, but I'm getting empty request body
curl -u dumbuser:dumbuserpassword -H "Content-Type: application/json" -X POST --data '{"events": "3"}' http://localhost:8080/events
My entire code is as below. Any pointer is appreciated!
package main
import (
"net/http"
"fmt"
"github.com/gin-gonic/gin"
)
func main() {
router := gin.Default()
authorized := router.Group("/", gin.BasicAuth(gin.Accounts{
"dumbuser": "dumbuserpassword",
}))
authorized.POST("/events", events)
router.Run(":8080")
}
func events(c *gin.Context) {
fmt.Printf("%s", c.Request.Body)
c.JSON(http.StatusOK, c)
}
The problem here is that you're printing out the string value of c.Request.Body, which is of interface type ReadCloser.
What you can do to satisfy yourself that it does in fact contain the body you want is to read the value out of c.Request.Body to a string, and then print that out. This is for your learning process only!
Learning code:
func events(c *gin.Context) {
x, _ := ioutil.ReadAll(c.Request.Body)
fmt.Printf("%s", string(x))
c.JSON(http.StatusOK, c)
}
However, this is not the way you should access the body of the request. Let gin do the parsing of the body for you, by using a binding.
More correct code:
type E struct {
Events string
}
func events(c *gin.Context) {
data := &E{}
c.Bind(data)
fmt.Println(data)
c.JSON(http.StatusOK, c)
}
This is a more correct way to access the data in the body, since it will be already parsed out for you. Note that if you read the body first, as we did above in the learning step, the c.Request.Body will have been emptied, and so there will be nothing left in the body for Gin to read.
Broken code:
func events(c *gin.Context) {
x, _ := ioutil.ReadAll(c.Request.Body)
fmt.Printf("%s", string(x))
data := &E{}
c.Bind(data) // data is left unchanged because c.Request.Body has been used up.
fmt.Println(data)
c.JSON(http.StatusOK, c)
}
You're probably also curious why the JSON returned from this endpoint shows and empty Request.Body. This is for the same reason. The JSON Marshalling method cannot serialize a ReadCloser, and so it shows up as empty.

How to read the Go docs?

I've created a simple go program (basically just example code):
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func getPage(url string) (body []byte, err error) {
resp, err := http.Get(url)
body = nil
if (err != nil) {
return
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
return
}
func main() {
startUrl := "http://slashdot.org/"
body, err := getPage(startUrl)
if (err != nil) {
fmt.Println("Error: " , err)
}
fmt.Println(body)
}
I'm trying to go through the docs to understand how it all fits together.
First issue: http.Get(). It's not in the docs (at http://golang.org/pkg/net/http/). Except it is, but under Response. However there are 2 other Get() functions. How do I know that net/http.Get is actually the Get() on the Response type?
Anyway, so http.Get() returns a Response with a Body of io.ReadCloser. ioutil.ReadAll() takes an io.Reader - but how can I find other functions that accept this as a parameter? It kind of seems like the docs are 'backwards' - if I know which function I want I can find docs, but if I have a type, how can I find what functions will work with it?
The functions are defined like this :
func (c *Client) Get(url string) (resp *Response, err error)
func (h Header) Get(key string) string
func Get(url string) (resp *Response, err error)
See how functions are declared.
Between func and the name of the function (Get), you have the receiver type and name (between parenthesis). Only one of those functions has no receiver and must be called directly prefixed by the package name (http). That's the one you need.
Concerning your io.Reader and io.ReadCloser question:
Those are interfaces, if you are not familiar with them read up on them here
Interfaces are basically sets of methods, these two are defined as follows:
type Reader interface{
Read([]byte)(int, error)
}
type ReadCloser interface{
Read([]byte)(int, error)
Close()
}
This means any concrete datatype that has a Read Method with the above signature can be passed on as a io.Reader. A datatype that satisfies io.ReadCloser definitely does this, since it has to provide the Read method and an additional close Method.
So you can simply pass your ReadCloser on as Reader.
The way interfaces work in go is a little hard to grasp at the beginning since they are so implicit,
however they are very powerful and give you lot's of possibilities. Definitely read the text I linked above. I read the whole thing down, when I was starting with go and it made things a lot easier.

Resources