Attempting to map a slice of []byte to a slice of []*byte.Reader for use in mapping to Decode images results in cannot use jpeg.Decode (value of type func(r io.Reader) (image.Image, error)) as func(item *bytes.Reader) (image.Image, error) with go1.18 using generics
package main
import (
"bytes"
"fmt"
"image"
"image/jpeg"
"log"
)
func main() {
imagesBytes := [][]byte{
[]byte("test1"),
[]byte("test2"),
}
jpeg.Decode(bytes.NewReader(imagesBytes[0])) // works
readers := Map(imagesBytes, bytes.NewReader)
images, err := MapWithErr(readers, jpeg.Decode) // cannot use jpeg.Decode (value of type func(r io.Reader) (image.Image, error)) as func(item *bytes.Reader) (image.Image, error) value in argument to MapWithErr[*bytes.Reader, image.Image]compilerIncompatibleAssign
if err != nil {
log.Fatal(err)
}
fmt.Println(images)
}
func MapWithErr[T1 any, T2 any](items []T1, predicate func(item T1) (T2, error)) ([]T2, error) {
output := make([]T2, len(items))
for idx, item := range items {
o, err := predicate(item)
if err != nil {
output = nil
return output, err
}
output[idx] = o
}
return output, nil
}
func Map[T1 any, T2 any](items []T1, predicate func(item T1) T2) []T2 {
output := make([]T2, len(items))
for idx, item := range items {
output[idx] = predicate(item)
}
return output
}
Why is it that I can directly pass a *bytes.Reader into jpeg.Decode() but when mapping and using generics, it doesn't work?
Because the type of acceptable parameter (io.Reader) in the jpeg.Decode() function is interface type
And if you output bytes.NewReader function
and you pass a type of *byte.Reader to the jpeg.Decode () function, there will be no problem
But the second parameter in the MapWithErr function is of the function type, not the interface, and now the input of this parameter must be fully compatible with the received function and its parameters.
In other words:
*Bytes.Reader and io.Reader types is not equal
only *Bytes.Reader can be converted to io.Reader.
Related
Preference: I'm new to Golang and keen to improve.
So I'm trying to learn from Dave Cheney's talk here: https://youtu.be/NwEuRO_w8HE?t=812 where we pass interfaces to methods to make code clearer and more generic.
I've implemented an example below, where I have a struct that can be outputted either to the std.out or to a file. However, I feel like it's a bit redundant just making empty structs (called "print" and "save" in ,my example), just so I can attach methods. How can I improve this?
package main
import (
"fmt"
"io"
"io/ioutil"
"log"
)
type file struct {
data string
}
func (f *file) output(w io.Writer) error {
len, err := w.Write([]byte(f.data))
if err != nil {
return err
} else {
log.Println("Bytes Out: ", len)
return nil
}
}
type print struct{}
func (print *print) Write(p []byte) (n int, err error) {
return fmt.Printf("%s\n", p)
}
type save struct{}
func (s *save) Write(p []byte) (n int, err error) {
err = ioutil.WriteFile("./dat1", []byte(p), 0644)
if err != nil {
return -1, err
}
return len(p), nil
}
func main() {
f := file{"test"}
s := save{}
p := print{}
f.output(&s)
f.output(&p)
}
Your intentions aren't clear, but to answer your original question: how to pass functions as interface values?
You could create a single adapter-like type that implements io.Writer, and when its Write() method is called, it forwards to a function of your choice. A typical example of this is the http.HandlerFunc: it's a function type that implements http.Handler, so a function can be passed when an http.Handler implementation is required.
For an io.Writer adapter, it could look like this:
type WriteFunc func(p []byte) (n int, err error)
func (w WriteFunc) Write(p []byte) (n int, err error) {
return w(p)
}
And if you have functions like these:
func PrintWrite(p []byte) (n int, err error) {
return fmt.Printf("%s\n", p)
}
func SaveWrite(p []byte) (n int, err error) {
err = ioutil.WriteFile("./dat1", []byte(p), 0644)
if err != nil {
return -1, err
}
return len(p), nil
}
You can use them as io.Writer like this:
f.output(WriteFunc(PrintWrite))
f.output(WriteFunc(SaveWrite))
Try it on the Go Playground.
You don't need print or save types here. You have a method, output, that takes an io.Writer; the whole point of that is that you can pass it any writer. Both stdout and files are writers, so everything else here is unnecessary. You could just:
func main() {
f := file{"test"}
fp,err := os.Create("./dat1")
if err != nil {
panic(err)
}
f.output(fp) // This writes to a file
f.output(os.Stdout) // This writes to stdout
}
I have 2 functions: GetRequest that has 3 arguments and returns *http.Request and RunRequest that has *http.Request as an argument and resp, content, err as results and I'm trying to connect them together like:
resp, content, err := myStruct.RunRequest(args for GetRequest)
type myStruct struct {}
func GetRequest(method string, path string, token string, body interface{}) (req *http.Request, err error) {...}
func RunRequest(req *http.Request) (resp *http.Response, content []byte, err error) {...}
resp, content, err := myStruct.RunRequest("GET", "/users", "", nil)
In Go this is called "defining a method on a type". It works like this:
package main
import (
"fmt"
"math"
)
type Vertex struct {
X, Y float64
}
func (v Vertex) Abs() float64 {
return math.Sqrt(v.X*v.X + v.Y*v.Y)
}
func main() {
v := Vertex{3, 4}
fmt.Println(v.Abs())
}
This example is taken straight from the Tour of Go which I highly recommend any newcomer should take.
I am attempting to unmarshal a custom/alias Hash of type [32]byte but the results are returning the zero value. The following code is just a snippet of what I am trying to do. In this snippet I am producing the json just to give an example but in my actual codebase it would be produced and read from another source. Here is the snippet:
package main
import (
"os"
"fmt"
"crypto/sha256"
"encoding/hex"
"encoding/json"
)
type Block struct {
Key Hash `json:"hash"`
}
type Hash [32]byte
func (h Hash) MarshalText() ([]byte, error) {
return []byte(hex.EncodeToString(h[:])), nil
}
func (h Hash) UnmarshalText(data []byte) error {
_, err := hex.Decode(h[:], data)
return err
}
func main() {
b := Block{sha256.Sum256([]byte("afasdfasfasfjiuoiuioupio"))}
j, _ := json.Marshal(b)
var unmarshalled Block
err := json.Unmarshal(j, &unmarshalled)
if err != nil {
os.Exit(1)
}
fmt.Printf("%x\n", unmarshalled.Key)
// prints out 0000000000000000000000000000000000000000000000000000000000000000
}
The UnmarshalText method modifies the value h, not the value in the caller. Use a pointer receiver to modify the caller's value.
func (h *Hash) UnmarshalText(data []byte) error {
_, err := hex.Decode(h[:], data)
return err
}
I'm trying to select a struct by string input and then depending on the return JSON Object or Array, unmarshall the JSON. Is it correct to think of a way to reflect the struct to slice struct? if so how to do that with reflection?
Regards,
Peter
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
type NameStruct struct {
Name string
}
func main() {
jsonData := []byte(`[{"name":"james"},{"name":"steven"}]`)
returnModel := InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
jsonData = []byte(`{"name":"james"}`)
returnModel = InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
}
func getModelByName(modelType string) interface{} {
modelMap := make(map[string]interface{})
modelMap["NameStruct"] = new(NameStruct)
//don't want to do this
modelMap["arrNameStruct"] = new([]NameStruct)
return modelMap[modelType]
}
func InitializeModel(modelName string, jsonData []byte) interface{} {
switch IsArray(jsonData) {
case true:
// some conversion here, how?
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
case false:
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
}
return nil
}
func IsArray(jsonData []byte) bool {
return (bytes.HasPrefix(jsonData, []byte("["))) && (bytes.HasSuffix(jsonData, []byte("]")))
}
Expanding on my comment, you can create a Factory where pre-defined types are registered:
type Factory struct {
m map[string]reflect.Type
}
func (f *Factory) Register(v interface{}) {
vt := reflect.TypeOf(v)
n := vt.Name()
f.m[n] = vt
f.m["[]"+n] = reflect.SliceOf(vt) // implicitly register a slice of type too
}
these types can be looked up by name at runtime and initialized with JSON data:
func (f *Factory) Make(k string, bs []byte) (interface{}, error) {
vt, ok := f.m[k]
if !ok {
return nil, fmt.Errorf("type %q not registered", k)
}
pv := reflect.New(vt).Interface()
err := json.Unmarshal(bs, pv)
if err != nil {
return nil, err
}
return pv, nil
}
To use:
type Place struct {
City string `json:"city"`
}
factory.Register(Place{})
p, err := factory.Make("Place", []byte(`{"city":"NYC"}`))
fmt.Printf("%#v\n", p) // &main.Place{City:"NYC"}
Slices also work:
ps, err := factory.Make("[]Place", []byte(`[{"city":"NYC"},{"city":"Dublin"}]`))
fmt.Printf("%#v\n", p, p) // &[]main.Place{main.Place{City:"NYC"}, main.Place{City:"Dublin"}}
Playground: https://play.golang.org/p/qWEdwk-YUug
In my current Golang project, I use buffering of logs before sending them to Elasticsearch within my log library. I want to introduce something like C atexit() function to flush all pending logs in case of an unexpected exit.
I have found atexit library but it was insufficient in my case because it does not allow passing of arguments to handler functions. I decided to write my version and ended up with roughly similar structure. I have module atexit:
package atexit
import (
"fmt"
"os"
"reflect"
)
var handlers []interface{}
var params []interface{}
func runHandler(handler interface{}, p interface{}) {
defer func() {
if err := recover(); err != nil {
fmt.Fprintln(os.Stderr, "error: atexit handler error:", err)
}
}()
f := reflect.ValueOf(handler)
s := reflectSlice(p)
fmt.Printf("%#v", s)
f.Call(s)
}
func reflectSlice(slice interface{}) []reflect.Value {
s := reflect.ValueOf(slice)
if s.Kind() != reflect.Slice {
panic("InterfaceSlice() given a non-slice type")
}
ret := make([]reflect.Value, s.Len())
for i:=0; i<s.Len(); i++ {
ret[i] = reflect.ValueOf(s.Index(i))
}
return ret
}
func runHandlers() {
for i, handler := range handlers {
runHandler(handler, params[i])
}
}
func Exit(code int) {
runHandlers()
os.Exit(code)
}
func Register(handler interface{}, p interface{}) {
f := reflect.TypeOf(handler)
if f.Kind() != reflect.Func {
panic("Register() given a non-function type")
}
handlers = append(handlers, handler)
params = append(params, p)
}
and I am calling it from the main program:
package main
import (
"fmt"
"./atexit"
"encoding/json"
"reflect"
)
type Batch struct {
Index string `json:"index"`
Type string `json:"_Type"`
Content interface{} `json:"Content"`
}
func flush(b ...interface{}) {
for _, entry := range(b) {
fmt.Printf("%#v\n", entry)
fmt.Println(reflect.TypeOf(reflect.ValueOf(entry)))
fmt.Println(reflect.TypeOf(entry))
a, err := json.Marshal(entry)
if err != nil {
fmt.Println("error!")
}
fmt.Println(string(a))
}
}
func handler(v ...interface{}) {
fmt.Println("Exiting")
for _, batch := range(v) {
fmt.Printf("%#v\n", batch)
}
}
func main() {
type ColorGroup struct {
ID int
Name string
Colors []string
}
group := ColorGroup{
ID: 1,
Name: "Reds",
Colors: []string{"Crimson", "Red", "Ruby", "Maroon"},
}
fmt.Printf("%#v\n", group)
r, err := json.Marshal(group)
if err != nil {
fmt.Println("error:", err)
}
fmt.Println(string(r))
b := []Batch{Batch{"index", "type", "content1"},Batch{"index", "type", "content2"}}
atexit.Register(handler, b)
atexit.Register(flush, []ColorGroup{group})
atexit.Exit(0)
}
As you can see, by calling reflect.ValueOf() I get structure reflect.Value which is then passed to callback function. The problem seems to be that this structure does not contain metadata about json export or is not handled correctly with json.Marshal(), which then outputs empty json. Is there any way I can pass correct []interface{} structure to the callback function or some similar mechanism which would do roughly what I'm trying to accomplish? Please note that I want a general callback mechanism which should be independent of the type passed to it. So far it seems to be impossible or at least very limited by f.Call(s), which is called in runHandler() and takes reflect.Value as its parameter.
You should not be using reflection for this. You should make your handlers be of type func(), then just pass in an anonymous function (possibly a closure) that handles any necessary arguments directly. Example:
atexit.Register(func() {
handler(b)
})