I've created a simple go program (basically just example code):
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func getPage(url string) (body []byte, err error) {
resp, err := http.Get(url)
body = nil
if (err != nil) {
return
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
return
}
func main() {
startUrl := "http://slashdot.org/"
body, err := getPage(startUrl)
if (err != nil) {
fmt.Println("Error: " , err)
}
fmt.Println(body)
}
I'm trying to go through the docs to understand how it all fits together.
First issue: http.Get(). It's not in the docs (at http://golang.org/pkg/net/http/). Except it is, but under Response. However there are 2 other Get() functions. How do I know that net/http.Get is actually the Get() on the Response type?
Anyway, so http.Get() returns a Response with a Body of io.ReadCloser. ioutil.ReadAll() takes an io.Reader - but how can I find other functions that accept this as a parameter? It kind of seems like the docs are 'backwards' - if I know which function I want I can find docs, but if I have a type, how can I find what functions will work with it?
The functions are defined like this :
func (c *Client) Get(url string) (resp *Response, err error)
func (h Header) Get(key string) string
func Get(url string) (resp *Response, err error)
See how functions are declared.
Between func and the name of the function (Get), you have the receiver type and name (between parenthesis). Only one of those functions has no receiver and must be called directly prefixed by the package name (http). That's the one you need.
Concerning your io.Reader and io.ReadCloser question:
Those are interfaces, if you are not familiar with them read up on them here
Interfaces are basically sets of methods, these two are defined as follows:
type Reader interface{
Read([]byte)(int, error)
}
type ReadCloser interface{
Read([]byte)(int, error)
Close()
}
This means any concrete datatype that has a Read Method with the above signature can be passed on as a io.Reader. A datatype that satisfies io.ReadCloser definitely does this, since it has to provide the Read method and an additional close Method.
So you can simply pass your ReadCloser on as Reader.
The way interfaces work in go is a little hard to grasp at the beginning since they are so implicit,
however they are very powerful and give you lot's of possibilities. Definitely read the text I linked above. I read the whole thing down, when I was starting with go and it made things a lot easier.
Related
I am trying to implement an oauth server and the package I am using needs the complete http.ResponseWriter and http.Request types.
c.Response does not contain all the methods that http.ResponseWriter does and c.Request gives error incompatible type.
How do I get http.ResponseWriter and http.Request in a Revel controller?
type client struct {
ClientId string
ClientSecret string
}
type App struct {
*revel.Controller
}
func (c App) TokenRequest() {
r := c.Request
w := c.Response
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
log.Println(string(body))
var cli client
err = json.Unmarshal(body, &cli)
if err != nil {
panic(err)
}
log.Println(cli.ClientId)
err = OauthSrv.HandleTokenRequest(w, r)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
Warning
I am generally not fond of frameworks like Revel in Go, for reasons that I hope demonstrate themselves on this page. My first recommendation would be that you examine closely what you are actually getting out of Revel that merits the use of such a heavy abstraction layer, and if it's really that valuable, you may want to ask questions going in the other direction, such as how one might make OauthSrv work within Revel's customized ecosystem.
Using a Revel controller as a ResponseWriter
For something to be an http.ResponseWriter, it just needs to have these methods.
Header
You need a method named Header() that returns an http.Header, which you can build out of any map[string][]string. Revel provides similar functionality, but through several layers of abstraction. You will need to unravel them:
c.Response is a *Response, so it has a field named Out containing an OutResponse.
An OutResponse has a Header() method—but it doesn't return an http.Header. Instead, it returns a *RevelHeader.
A *RevelHeader has a GetAll(key string) []string method—which is very similar to the API already provided by the built-in map type, but isn't exactly the same. So, you will need to copy the returned values into a new map every time Header() is called, in order to fully satisfy the function signature requirements.
Also, GetAll() requires you to know the key name you are interested in, and *RevelHeader on its own does not provide a way to look up which keys are available. For now we can rely on the fact that the current implementation only has one field, a ServerHeader that does provide a GetKeys() []string method.
Putting all this together, we can build our Header method:
func (rrw RevelResponseWrapper) Header() http.Header {
revelHeader := rrw.Response.Out.Header()
keys := revelHeader.Server.GetKeys()
headerMap := make(map[string][]string)
for _, key := range keys {
headerMap[key] = revelHeader.GetAll(key)
}
return http.Header(headerMap)
}
Write and WriteHeader
You would use similar anti-patterns to expose rrw.Write([]byte) (int, error) so that it calls through to c.Response.Out.Write(data []byte) (int, error), and rrw.WriteHeader(int) error so that it calls c.Response.WriteHeader(int, string). Depending on what is considered appropriate for the framework, either panic on errors or fail silently, since their API doesn't expect WriteHeader errors to be handle-able.
Getting an http.Request from Revel
Unfortunately, the http.Request type is a struct, so you can't just simulate it. You basically have two options: reconstruct it using the net/http package from all the properties you are able to access, or hope that the *revel.Request you have is secretly an http.Request under the hood. In the latter case, you can use a type assertion:
revelReq, ok := c.Request.In.(*revel.GoRequest)
if !ok {
// handle this somehow
}
r := revelReq.Original
Is it possible to have my function definition below accept any type of struct?
I've tried to refactor like so:
// This method should accept any type of struct
// Once I receive my response from the database,
// I scan the rows to create a slice of type struct.
func generateResponse(rows *sqlx.Rows, structSlice []struct{}, structBody struct{}) ([]struct{}, error) {
for rows.Next() {
err := rows.StructScan(&structBody)
if err != nil {
return nil, err
}
structSlice = append(structSlice, structBody)
}
err := rows.Err()
if err != nil {
return nil, err
}
return structSlice, nil
}
Assume my struct is of type OrderRevenue.
When I call the function above:
structSlice, err := generateResponse(rows, []OrderRevenue{}, OrderRevenue{})
The error I get is:
cannot use []OrderRevenue literal as type []struct{} in argument...
Am I going about this the wrong way?
This is considered the cornerstone (or more of a limitation) of Go's type system. struct{} is an unnamed type that is different from struct{ field1 int } and of course is not the same as OrderRevenue{}.
Go emphasizes abstraction through interfaces, and perhaps you should try that. Here is the first take:
type OrderRevenue interface {
MarshalMyself() ([]byte, error)
}
type Anonymous struct {}
func (a Anonymous) MarshalMyself() ([]byte, error) {
// implementation's up to you
return []byte{}, nil
}
// the function signature
generateResponse(rows *sqlx.Rows, structSlice []OrderRevenue, structBody Body) ([]Body, error) {
// ...
}
In this case you can also use empty interface interface{}, which all types implement, but you'll have to recursively go through the structure to do manual type assertion. The best approach in Go is to know the shape of your data in advance, at least partially.
I am trying to wrap my head around the GoLang type system, and there area a few things the confuse me.
So I have been working on the http library to try to understand this and I have come across the following that makes no sense.
package main
import (
"net/http"
"fmt"
"io/ioutil"
"io"
)
func convert(closer io.Closer) ([]byte) {
body, _ := ioutil.ReadAll(closer);
return body
}
func main() {
client := &http.Client{}
req, _ := http.NewRequest("GET", "https://www.google.com", nil)
response, _ := client.Do(req);
body, _ := ioutil.ReadAll(response.Body)
fmt.Println(body);
fmt.Println(convert(response.Body))
}
The Go Playground
this is not about the fact that the convert function is not needed it is the fact that the response body is of type io.closer and the ioutil.Readall takes a io.reader, yet I can pass it in, in one instance but not if in another. Is there something that I am missing that is magically happening.
I know that the closer technically passes the reader interface as it implements the the Read method but that should be true in both the function and the main body.
Any insight would be great.
Thanks
it is the fact that the response body is of type io.closer
No, it is not. Declaration of Request.Body is at http.Request:
Body io.ReadCloser
The Request.Body field is of type io.ReadCloser, it is both an io.Reader and an io.Closer.
Since it is an io.Reader (dynamic value of Request.Body implements io.Reader), you may use / pass it where an io.Reader is required, e.g. to ioutil.ReadAll().
Since it also implements io.Closer, you can also pass it where io.Closer is required, like your convert() function.
But inside convert the closer param has static type io.Closer, you can't use closer where an in.Reader is required. It might be (and in your case it is) that the dynamic type stored in closer also implements io.Reader, but there is no guarantee for this. Like in this example:
type mycloser int
func (mycloser) Close() error { return nil }
func main() {
var m io.Closer = mycloser(0)
convert(m)
}
In the above example closer inside convert() will hold a value of type mycloser, which truly does not implement io.Reader.
If your convert() function intends to treat its parameter also as an io.Reader, the parameter type should be io.ReadCloser:
func convert(rc io.ReadCloser) ([]byte, error) {
body, err := ioutil.ReadAll(rc)
if err != nil {
return body, err
}
err = rc.Close()
return body, err
}
I'm still quite new to Go and I was surprised to not be able to use the subtype of an embedded interface.
Here is a small example to explain what I mean:
func test(sl bufio.ReadWriter){
// cannot use sl(type bufio.ReadWriter) as type bufio.Reader in function argument
readStuff(sl)
[...]
writeStuff(sl) // same kind of error
}
func readStuff(sl bufio.Reader){
[...]
}
As every interface have the same memory layout and ReadWriter is a Reader and a Writer, I was expecting this code to work.
I did try to convert the interface type with:
readStuff(sl.(buffio.Reader))
But it doesn't work either. So I've got two questions:
Why doesn't it work?
What's the go philosophy about that problem?
They're different types. However, a bufio.ReadWriter contains a pointer to both a bufio.Reader type and a bufio.Writer type as elements of its struct. So passing the correct one should be easy enough. Try this:
func test(sl bufio.ReadWriter){
readStuff(sl.Reader)
[...]
writeStuff(sl.Writer)
}
// Changed this bufio.Reader to a pointer receiver
func readStuff(sl *bufio.Reader) {
[...]
}
bufio.ReadWriter is a concrete type, not an interface. However, it does satisfy an interface (io.ReadWriter) so it can be assigned to a variable/function argument of an appropriate interface type. Then it works the way you may have anticipated (your code actually doesn't use any interfaces):
package main
import (
"bufio"
"bytes"
"fmt"
"io"
"log"
)
func readStuff(r io.Reader) {
b := make([]byte, 10)
n, err := r.Read(b)
if err != nil && err != io.EOF {
log.Fatal(err)
}
fmt.Printf("readStuff: %q\n", b[:n])
}
func writeStuff(w io.Writer) {
b := []byte("written")
n, err := w.Write(b)
if n != len(b) {
log.Fatal("Short write")
}
if err != nil {
log.Fatal(err)
}
}
func test(rw io.ReadWriter) {
readStuff(rw)
writeStuff(rw)
}
func main() {
r := io.Reader(bytes.NewBufferString("source"))
var uw bytes.Buffer
w := io.Writer(&uw)
rw := bufio.NewReadWriter(bufio.NewReader(r), bufio.NewWriter(w))
test(rw)
rw.Flush()
fmt.Printf("The underlying bytes.Buffer writer contains %q\n", uw.Bytes())
}
(Also here)
Output:
readStuff: "source"
The underlying bytes.Buffer writer contains "written"
This way test can consume any io.ReadWriter, not only a specific one. Which is a hint towards your question about go "philosophy".
In order to determine whether a given type implements an interface using the reflect package, you need to pass a reflect.Type to reflect.Type.Implements(). How do you get one of those types?
As an example, trying to get the type of an uninitialized error (interface) type does not work (it panics when you to call Kind() on it)
var err error
fmt.Printf("%#v\n", reflect.TypeOf(err).Kind())
Do it like this:
var err error
t := reflect.TypeOf(&err).Elem()
Or in one line:
t := reflect.TypeOf((*error)(nil)).Elem()
Even Shaws response is correct, but brief. Some more details from the reflect.TypeOf method documentation:
// As interface types are only used for static typing, a common idiom to find
// the reflection Type for an interface type Foo is to use a *Foo value.
writerType := reflect.TypeOf((*io.Writer)(nil)).Elem()
fileType := reflect.TypeOf((*os.File)(nil)).Elem()
fmt.Println(fileType.Implements(writerType))
For googlers out there I just ran into the dreaded scannable dest type interface {} with >1 columns (XX) in result error.
Evan Shaw's answer did not work for me. Here is how I solved it. I am also using the lann/squirrel library, but you could easily take that out.
The solution really isn't that complicated, just knowing the magic combination of reflect calls to make.
The me.GetSqlx() function just returns an instance to *sqlx.DB
func (me *CommonRepo) Get(query sq.SelectBuilder, dest interface{}) error {
sqlst, args, err := query.ToSql()
if err != nil {
return err
}
// Do some reflection magic so that Sqlx doesn't hork on interface{}
v := reflect.ValueOf(dest)
return me.GetSqlx().Get(v.Interface(), sqlst, args...)
}
func (me *CommonRepo) Select(query sq.SelectBuilder, dest interface{}) error {
sqlst, args, err := query.ToSql()
if err != nil {
return err
}
// Do some reflection magic so that Sqlx doesn't hork on interface{}
v := reflect.ValueOf(dest)
return me.GetSqlx().Select(v.Interface(), sqlst, args...)
}
Then to invoke it you can do:
func (me *myCustomerRepo) Get(query sq.SelectBuilder) (rec Customer, err error) {
err = me.CommonRepo.Get(query, &rec)
return
}
func (me *myCustomerRepo) Select(query sq.SelectBuilder) (recs []Customer, err error) {
err = me.CommonRepo.Select(query, &recs)
return
}
This allows you to have strong types all over but have all the common logic in one place (CommonRepo in this example).