I have a couple of example nested structs and need to serialize them. I am using the encoding/gob library, which should convert the struct data to bytes and the encoding/base64 library to convert the bytes to a readable base64 format. However, when I run my example code I get a serialization error error. I don't understand why this happens and how to fix the problem.
I followed this example: Golang serialize and deserialize back
Here is the code:
package main
import (
"bytes"
"encoding/base64"
"encoding/gob"
"errors"
"fmt"
)
type Hello struct {
greeting string
}
type Bye struct {
helloSaid Hello
byesaid Hello
}
func (b1 Bye) Serialize() (string, error) {
b := bytes.Buffer{}
e := gob.NewEncoder(&b)
err := e.Encode(b1)
if err != nil {
return string(b.Bytes()[:]), errors.New("serialization failed")
}
return base64.StdEncoding.EncodeToString(b.Bytes()), nil
}
func DeserializeBye(str string) (Bye, error) {
m := Bye{}
by, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return m, errors.New("deserialization failed")
}
b := bytes.Buffer{}
b.Write(by)
d := gob.NewDecoder(&b)
err = d.Decode(&m)
if err != nil {
return m, errors.New("deserialization failed")
}
return m, nil
}
func main() {
h := Hello{greeting: "hello"}
b := Bye{helloSaid: h, byesaid: h}
serialized, err := b.Serialize()
if err != nil {
fmt.Println(err)
}
fmt.Println(serialized)
}
Please, make the fields of the Hello and Bye structures public. Please see the documentation for the gob package:
Structs encode and decode only exported fields.
Related
I need to get the filename of certain file(s) that receives backend from the frontend. Backend (implemented in Go) will receive the file as io.ReadCloser. Is there way I could extract it from the io.ReadCloser?
Backend (implemented in Go) will receive the file as io.ReadCloser. Is there way I could extract it from the io.ReadCloser?
No.
Take a look at which methods an io.ReadCloser provides by running go doc io.ReadCloser and note that there isn't a method which will provide a name. So unless you know nothing more that that it is an io.ReadCloser you simply cannot do it.
package main
import (
"errors"
"fmt"
"io"
"os"
)
func fatalln(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
// hasName interface is an interface that expects types
// that implements it to have "Name() string" method.
type hasName interface {
Name() string
}
func open(name string) (io.ReadCloser, error) {
f, err := os.Open(name)
if err != nil {
return nil, err
}
// f implements io.ReadCloser interface as *os.File
// has Read and Close methods.
return f, nil
}
func main() {
// rc is of the type io.ReadCloser
rc, err := open("example.txt")
if err != nil {
fatalln(err)
}
defer rc.Close()
// Type assetion to check rc's underlying type has
// a method "Name() string".
f, ok := rc.(hasName)
if !ok {
fatalln(errors.New("type assertion failed"))
}
// Yay, type assertion succeeded. Print the name!
fmt.Println("Name:", f.Name())
}
The io.ReadCloser here is a reader for runtime reader which reads file from network as the frontend sends it to backend. You'll have to work on request itself to get that file name.
This is an assumption but in most such cases for file upload, the request is a multipart request. If you have the same situation, you can read the headers, typically Content-Disposition to identify the file type. Go native http.Request has ability to parse the details. You can try this :
formFile, handler, err := r.FormFile("file") // read file from network with key "file"
defer formFile.Close()
fileName := handler.Filename // Get file name
By defining an interface which embeds io.Reader you can require a Name() method up front:
package main
import (
"fmt"
"io"
"log"
"os"
)
type NamedReadCloser interface {
io.ReadCloser
Name() string
}
func doThings(f NamedReadCloser) error {
defer f.Close()
b, err := io.ReadAll(f)
if err != nil {
return err
}
fmt.Printf("Name: %s, Content: %s\n", f.Name(), b)
return nil
}
func main() {
f, err := os.Open("/etc/hosts")
if err != nil {
log.Fatal("Cannot open file: ", err)
}
err = doThings(f)
if err != nil {
log.Fatal("Error doing things: ", err)
}
}
This will only work if what is passed in has a name method, like an *os.File. If it does not, then what you are trying to do is not possible.
You'll have to cast it to a type with a Name method:
package main
import (
"io"
"os"
)
func open(name string) (io.ReadCloser, error) {
return os.Open(name)
}
func main() {
c, e := open("file.txt")
if e != nil {
panic(e)
}
defer c.Close()
f := c.(*os.File)
println(f.Name())
}
In go I have a function:
func UrlGET(url string, headers string) string { // inputs are url and headers for a http request
...
req, err := http.NewRequest("GET", url, nil)
...
resp, err := client.Do(req)
defer resp.Body.Close()
if rc := resp.Cookies(); len(rc) > 0 {
return string(rc)
}
return ""
}
However, you cannot convert type Cookie ([]*http.Cookie to type string (cannot convert rc (type []*http.Cookie) to type string). What would be an alternative or another way to convert to type string, ideally I would still return type string. I'm relatively new to go so at a bit of a wall as to what else to try.
Ideally, it would return like cookie=some_cookie_value as a string.
If you just want one big string, you can do:
package main
import "net/http"
func main() {
r, e := http.Get("https://stackoverflow.com")
if e != nil {
panic(e)
}
defer r.Body.Close()
s := r.Header.Get("Set-Cookie")
println(s)
}
Or you could build a map:
package main
import (
"fmt"
"net/http"
)
func main() {
r, e := http.Get("https://stackoverflow.com")
if e != nil {
panic(e)
}
defer r.Body.Close()
m := make(map[string]string)
for _, c := range r.Cookies() {
m[c.Name] = c.Value
}
fmt.Println(m)
}
https://golang.org/pkg/net/http#Response.Cookies
https://golang.org/pkg/net/http#Response.Header
I have gobs of unknown type. Is there way to print it to view inside?
There might be gob.Debug but it is not available for me
https://golang.org/src/encoding/gob/debug.go
Googling advices to use DecodeValue but it requires initialised reflect.Value
If I get unknown gob blob then I can't pass initialized value on unknown type
https://play.golang.org/p/OWxX1kPJ6Qa
package main
import (
"bytes"
"encoding/gob"
"fmt"
"reflect"
)
func encode1() []byte {
x := "123"
buf := &bytes.Buffer{}
enc := gob.NewEncoder(buf)
err := enc.Encode(x)
if err != nil {
panic(err)
}
return buf.Bytes()
}
func decode1(b1 []byte) {
var x string
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.Decode(&x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
}
func decode2(b1 []byte) {
// var x reflect.Value
x := reflect.New(reflect.TypeOf(""))
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.DecodeValue(x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
fmt.Printf("%#v\n", reflect.Indirect(x))
}
func main() {
b1 := encode1()
decode1(b1)
decode2(b1)
}
you need to Register your type before encode decoding
Register records a type, identified by a value for that type, under
its internal type name. That name will identify the concrete type of a
value sent or received as an interface variable. Only types that will
be transferred as implementations of interface values need to be
registered. Expecting to be used only during initialization, it panics
if the mapping between types and names is not a bijection.
I have a struct that I want to marshal to JSON. It has a defined field called Foo (exported as foo) and a data interface field to which I want to pass a dynamic struct with additional JSON fields.
However when the data field is an interface instead of the specific struct it never gets exported as JSON. How can I make this work?
package main
import (
"encoding/json"
"fmt"
)
type data interface{}
type foo struct {
Foo string `json:"foo,omitempty"`
data
}
type bar struct {
Bar string `json:"bar,omitempty"`
}
func main() {
b := bar{"bar"}
f := foo{"foo", b}
byt, err := json.Marshal(f)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(byt))
}
I need to output to look like this (it needs to be flat, not nested):
{"foo": "foo", "bar": "bar"}
You could do this with a custom json.Marshaler implementation and a little bit of byte slicing.
func (f foo) MarshalJSON() ([]byte, error) {
type goo foo
g := goo(f)
b1, err := json.Marshal(g)
if err != nil {
return nil, err
}
b2, err := json.Marshal(g.data)
if err != nil {
return nil, err
}
s1 := string(b1[:len(b1)-1])
s2 := string(b2[1:])
return []byte(s1 + ", " + s2), nil
}
https://play.golang.org/p/NYTNWIL-xu
Please note that this is not checking whether the bytes can actually be sliced and it does also not consider the possible case of the data field being a slice or an array, which i'm unsure how you would want that flattened anyway.
I would write a custom marshaller, like so:
func (f foo) MarshalJSON() ([]byte, error) {
type tmp foo
g := tmp(f)
first, err := json.Marshal(g)
if err != nil {
return nil, err
}
second, err := json.Marshal(f.data)
if err != nil {
return nil, err
}
data := make(map[string]interface{})
json.Unmarshal(first, &data)
json.Unmarshal(second, &data)
return json.Marshal(data)
//{"bar":"bar","foo":"foo"}
}
https://play.golang.org/p/TENiCe9nR0
2 options:
Set to it type json.RawMessage so it won’t be decoded automatically and left as an interface. https://golang.org/pkg/encoding/json/#RawMessage
Write custom unmarshaler on the structure.
I have read the documentation of ( gob) and I have some problems :
Now I know how to encode structure and decode like that:
func main() {
s1 := &S{
Field1: "Hello Gob",
Field2: 999,
}
log.Println("Original value:", s1)
buf := new(bytes.Buffer)
err := gob.NewEncoder(buf).Encode(s1)
if err != nil {
log.Println("Encode:", err)
return
}
s2 := &S{}
err = gob.NewDecoder(buf).Decode(s2)
if err != nil {
log.Println("Decode:", err)
return
}
log.Println("Decoded value:", s2)
}
But I don't know the purpose of this method gob.Register() can someone explain to me when to use it and why?
If you're dealing with concrete types (structs) only, you don't really need it. Once you're dealing with interfaces you must register your concrete type first.
For example, let's assume we have these struct and interface (the struct implements the interface):
type Getter interface {
Get() string
}
type Foo struct {
Bar string
}
func (f Foo)Get() string {
return f.Bar
}
To send a Foo over gob as a Getter and decode it back, we must first call
gob.Register(Foo{})
So the flow would be:
// init and register
buf := bytes.NewBuffer(nil)
gob.Register(Foo{})
// create a getter of Foo
g := Getter(Foo{"wazzup"})
// encode
enc := gob.NewEncoder(buf)
enc.Encode(&g)
// decode
dec := gob.NewDecoder(buf)
var gg Getter
if err := dec.Decode(&gg); err != nil {
panic(err)
}
Now try removing the Register and this won't work because gob wouldn't know how to map things back to their appropriate type.
As http://golang.org/pkg/encoding/gob/#Register said:
Only types that will be transferred as implementations of interface
values need to be registered.
So it doesn't needed by your demo.
If you want to encode / decode a map[string]interface{}, since the field of the map is enclosed as interface type, then we need to register the specific type before.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type SomeStruct struct {
Text string
}
func main() {
var bytes bytes.Buffer
// Remove one of these, then the decoding will produce error
gob.Register(SomeStruct{})
gob.Register([]interface{}{})
gob.Register([]SomeStruct{})
gob.Register(map[string]SomeStruct{})
writer := gob.NewEncoder(&bytes)
err := writer.Encode(map[string]interface{}{
"SomeStruct": SomeStruct{"Halo"},
"SomeSlice": []interface{}{},
"SomeSliceStruct": []SomeStruct{
{
Text: "SomeText",
},
},
"SomeMapStruct": map[string]SomeStruct{
"S": {"Test"},
},
})
if err != nil {
log.Fatalf("Error on encode process: %v\n", err)
return
}
reader := gob.NewDecoder(&bytes)
var aMap map[string]interface{}
err = reader.Decode(&aMap)
if err != nil {
log.Fatalf("Error on decode process: %v\n", err)
return
}
fmt.Printf("Decode is successful: %+v\n", aMap)
}