I am attempting to implement an interface based message queue where jobs are pushed as bytes to a redis queue. But I keep receiving an EOF error when attempting to decode the byte stream.
https://play.golang.org/p/l9TBvcn9qg
Could someone point me in the right direction?
Thank you!
In your Go Playground example, you're trying to encode an interface and interfaces don't have a concrete implementation. If you remove the interface from your A struct, that should work. Like the following:
package main
import "fmt"
import "encoding/gob"
import "bytes"
type testInterface interface{}
type A struct {
Name string
Interface *B // note this change here
}
type B struct {
Value string
}
func main() {
var err error
test := &A {
Name: "wut",
Interface: &B{Value: "BVALUE"},
}
buf := bytes.NewBuffer([]byte{})
enc := gob.NewEncoder(buf)
dec := gob.NewDecoder(buf)
// added error checking as per Mark's comment
err = enc.Encode(test)
if err != nil {
panic(err.Error())
}
result := &A{}
err := dec.Decode(result)
fmt.Printf("%+v\n", result)
fmt.Println("Error is:", err)
fmt.Println("Hello, playground")
}
Also, just as a side note you will see some sort of output like the following: &{Name:wut Interface:0x1040a5a0} because A is referencing a reference to a B struct. To clean that up further:
type A struct{
Name string
Interface B // no longer a pointer
}
func main() {
// ...
test := &A{Name: "wut", Interface: B{Value: "BVALUE"}}
// ...
}
Found the answer to the problem from Mark above. I have forgotten to do a gob.Register(B{})
https://play.golang.org/p/7rQDHvMhD7
Related
package main
import (
"encoding/json"
"fmt"
"log"
"strings"
)
type Animal int
const (
Unknown Animal = iota
Gopher
Zebra
)
func (a *Animal) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
switch strings.ToLower(s) {
default:
*a = Unknown
case "gopher":
*a = Gopher
case "zebra":
*a = Zebra
}
return nil
}
func (a Animal) MarshalJSON() ([]byte, error) {
var s string
switch a {
default:
s = "unknown"
case Gopher:
s = "gopher"
case Zebra:
s = "zebra"
}
return json.Marshal(s)
}
func main() {
blob := `["gopher","armadillo","zebra","unknown","gopher","bee","gopher","zebra"]`
var zoo []Animal
if err := json.Unmarshal([]byte(blob), &zoo); err != nil {
log.Fatal(err)
}
census := make(map[Animal]int)
for _, animal := range zoo {
census[animal] += 1
}
fmt.Printf("Zoo Census:\n* Gophers: %d\n* Zebras: %d\n* Unknown: %d\n",
census[Gopher], census[Zebra], census[Unknown])
}
This is the code snippet of Json custom marshal example in go doc. My question is where is the call to MarshalJSON and UnmarshalJSON method in this code. Are these method somehow overriding Json package's UnmarshalJSON and MarshalJSON method. I thought go does not support method overriding this way. Pls help, i am not able to understand what is happening in this code!!
The documentation says:
To unmarshal JSON into a value implementing the Unmarshaler interface, Unmarshal calls that value's UnmarshalJSON method, including when the input is a JSON null.
Somewhere in the json.Unmarshal implementation, there's code similar this:
u, ok := v.(Unmarshaler)
if ok {
err := u.Unmarshal(data)
if err != nil { /* handle error */}
} else {
// handle other kinds of values
}
The code uses a type assertion to determine if the value satisfies the json.Unmarshaler interface. If the value does satisfy the method, the value's UnmarshalJSON function is called.
The (*Animal).UnmarshalJSON function is called because *Animal satisfies the json.Unmarshaler interface.
This is an example of implementing an interface from a different package.
There's no need to explicitly declare that you're implementing an interface in Go, like there is in Java or C++ for example. You just have to implement all the functions it declares. In this case, you're implementing the Unmarshaler interface declared in the json package which is used by the Unmarshal function.
I am trying to parse out a specific field from a json string, and currently I have the following code snippet.
package main
import (
"encoding/json"
"fmt"
)
type PlatformID string
type Map map[string]interface{}
func Str2Map(str string) (Map, error) {
var dictionary Map
bytes := []byte(str)
err := json.Unmarshal(bytes, &dictionary)
if err != nil {
fmt.Println(err)
}
return dictionary, err
}
func parsePlatformID(str string) (PlatformID, error) {
fmt.Println(str)
dict, err := Str2Map(str)
fmt.Println(dict)
return dict["platform-id"].(PlatformID), err
}
func main() {
PlatformDictStr := "{\"platform-id\":\"platform_BnjliXLEUV26\",\"platform-labels\":\"test\",\"OptimizeScheme\":\"None\"}"
fmt.Println(PlatformDictStr)
ID, _ := parsePlatformID(PlatformDictStr)
fmt.Println(ID)
}
When I try to run it, it gives me the following error
{"platform-id":"platform_BnjliXLEUV26","platform-labels":"test","OptimizeScheme":"None"}
{"platform-id":"platform_BnjliXLEUV26","platform-labels":"test","OptimizeScheme":"None"}
map[platform-id:platform_BnjliXLEUV26 platform-labels:test OptimizeScheme:None]
panic: interface conversion: interface is string, not main.PlatformID
goroutine 1 [running]:
panic(0x126aa0, 0x10532300)
/usr/local/go/src/runtime/panic.go:500 +0x720
main.parsePlatformID(0x13e6fe, 0x58, 0x0, 0x0, 0x0, 0x0)
/tmp/sandbox256874711/main.go:26 +0x220
main.main()
/tmp/sandbox256874711/main.go:34 +0x100
This question sort of answers why I got panic: interface conversion: interface is string
If I try to change the type assertion to string, the underlying type of PlatformID, it won't even compile tmp/sandbox325023244/main.go:26: cannot use dict["platform-id"].(string) (type string) as type PlatformID in return argument
So how should I modify the return line so that I can retrieve PlatformID?
After looking at PlatformDictStr, I think var dictionary map[string]string should do the job.
PlatformID(dict["platform-id"].(string)) can be avoided in that case. Working example here
https://play.golang.org/p/A5kiVm_XbP
After playing around with the syntax a bit more, I think I need to do both type conversion and type assertion.
So the following line solves the problem
return PlatformID(dict["platform-id"].(string)), err
In retrospect, I need to first assert the interface type to a base type string, and from there I can just do a type conversion to PlatformID
PS 1: The use case is that I got the raw string in the request body, REST layer will parse out certain fields in a dictionary, then forward the rest of unparsed string to API layer for further processing. The keys in dictionary vary depends on workload, so I can't really Unmarshal it to a well defined struct.
Is there any specific reason for which you have created types for string and map? If not, I think you are over-engineering a simple use-case. The following works fine:
package main
import (
"encoding/json"
"fmt"
)
func Str2Map(str string) (map[string]interface{}, error) {
var dictionary map[string]interface{}
bytes := []byte(str)
err := json.Unmarshal(bytes, &dictionary)
if err != nil {
fmt.Println(err)
}
return dictionary, err
}
func parsePlatformID(str string) (string, error) {
fmt.Println(str)
dict, err := Str2Map(str)
fmt.Println(dict)
return dict["platform-id"].(string), err
}
func main() {
PlatformDictStr := "{\"platform-id\":\"platform_BnjliXLEUV26\",\"platform-labels\":\"test\",\"OptimizeScheme\":\"None\"}"
fmt.Println(PlatformDictStr)
ID, _ := parsePlatformID(PlatformDictStr)
fmt.Println(ID)
}
Working Playground example: https://play.golang.org/p/mpPpDmiz7x
I am currently unable to unmarshall the data correctly from a map into a structure. Following is a code snippet (Brief Code at playground):
Request you to kindly provide the reason of getting default values on unmarshlling back the data.
package main
import (
"fmt"
"encoding/json"
"os"
)
func main() {
fmt.Println("Hello, playground")
type PDPOffer struct {
cart_value int `json:"cart_value"`
discount_amount_default int `json:"discount_amount_default"`
max_discount string `json:"max_discount"`
}
a:= map[string]interface{} {
"cart_value" : 1,
"max_discount" : 2,
}
var pdf PDPOffer
b, err := json.Marshal(a)
if err != nil {
fmt.Println("error:", err)
}
os.Stdout.Write(b)//working
err1 := json.Unmarshal(b, &pdf)
if err1 != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", pdf)//displaying just the defualt values????????
}
json.Marshal and json.Unmarshal can only work on exported struct fields. Your fields aren't exported and aren't visible to the json code.
The reason for your unmarshal not working is that you need to expose fields of the struct and for doing that you need to start field name with Capital letters. Something has below:
type PDPOffer struct {
Cart_value int `json:"cart_value"`
Discount_amount_default int `json:"discount_amount_default"`
Max_discount string `json:"max_discount"`
}
Also, you were trying to marshal an int value into a string for "max_discount", you need to store it as a string in your map that you're marshaling:
a := map[string]interface{}{
"cart_value": 1,
"max_discount": "2",
}
The error handling had a bug checking for err1 != nil then printing err which was hiding the message error: json: cannot unmarshal number into Go value of type string
Working example with all the fixes: http://play.golang.org/p/L8VC-531nS
I'm writing a websocket client in Go. I'm receiving the following JSON from the server:
{"args":[{"time":"2013-05-21 16:57:17"}],"name":"send:time"}
I'm trying to access the time parameter, but just can't grasp how to reach deep into an interface type:
package main;
import "encoding/json"
import "log"
func main() {
msg := `{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`
u := map[string]interface{}{}
err := json.Unmarshal([]byte(msg), &u)
if err != nil {
panic(err)
}
args := u["args"]
log.Println( args[0]["time"] ) // invalid notation...
}
Which obviously errors, since the notation is not right:
invalid operation: args[0] (index of type interface {})
I just can't find a way to dig into the map to grab deeply nested keys and values.
Once I can get over grabbing dynamic values, I'd like to declare these messages. How would I write a type struct to represent such complex data structs?
You may like to consider the package github.com/bitly/go-simplejson
See the doc： http://godoc.org/github.com/bitly/go-simplejson
Example:
time, err := json.Get("args").GetIndex(0).String("time")
if err != nil {
panic(err)
}
log.Println(time)
The interface{} part of the map[string]interface{} you decode into will match the type of that field. So in this case:
args.([]interface{})[0].(map[string]interface{})["time"].(string)
should return "2013-05-21 16:56:16"
However, if you know the structure of the JSON, you should try defining a struct that matches that structure and unmarshal into that. Ex:
type Time struct {
Time time.Time `json:"time"`
Timezone []TZStruct `json:"tzs"` // obv. you need to define TZStruct as well
Name string `json:"name"`
}
type TimeResponse struct {
Args []Time `json:"args"`
}
var t TimeResponse
json.Unmarshal(msg, &t)
That may not be perfect, but should give you the idea
I'm extremely new to Golang coming from Python, and have always struggled with encode/decoding json. I found gjson at https://github.com/tidwall/gjson, and it helped me immensely:
package main
import "github.com/tidwall/gjson"
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
value := gjson.Get(msg, "args.#.time")
println(value.String())
}
-----------------------
["2013-05-21 16:56:16"]
Additionally, I noticed the comment of how to convert into Struct
package main
import (
"encoding/json"
"fmt"
)
type msgFormat struct {
Time string `json:"time"`
Tzs msgFormatTzs `json:"tzs"`
Name string `json:"name"`
}
type msgFormatTzs struct {
TzsName string `json:"name"`
}
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
r, err := json.Marshal(msgFormatTzs{msg})
if err != nil {
panic(err)
}
fmt.Printf("%v", r)
}
Try on Go playground
I'm still quite new to Go and I was surprised to not be able to use the subtype of an embedded interface.
Here is a small example to explain what I mean:
func test(sl bufio.ReadWriter){
// cannot use sl(type bufio.ReadWriter) as type bufio.Reader in function argument
readStuff(sl)
[...]
writeStuff(sl) // same kind of error
}
func readStuff(sl bufio.Reader){
[...]
}
As every interface have the same memory layout and ReadWriter is a Reader and a Writer, I was expecting this code to work.
I did try to convert the interface type with:
readStuff(sl.(buffio.Reader))
But it doesn't work either. So I've got two questions:
Why doesn't it work?
What's the go philosophy about that problem?
They're different types. However, a bufio.ReadWriter contains a pointer to both a bufio.Reader type and a bufio.Writer type as elements of its struct. So passing the correct one should be easy enough. Try this:
func test(sl bufio.ReadWriter){
readStuff(sl.Reader)
[...]
writeStuff(sl.Writer)
}
// Changed this bufio.Reader to a pointer receiver
func readStuff(sl *bufio.Reader) {
[...]
}
bufio.ReadWriter is a concrete type, not an interface. However, it does satisfy an interface (io.ReadWriter) so it can be assigned to a variable/function argument of an appropriate interface type. Then it works the way you may have anticipated (your code actually doesn't use any interfaces):
package main
import (
"bufio"
"bytes"
"fmt"
"io"
"log"
)
func readStuff(r io.Reader) {
b := make([]byte, 10)
n, err := r.Read(b)
if err != nil && err != io.EOF {
log.Fatal(err)
}
fmt.Printf("readStuff: %q\n", b[:n])
}
func writeStuff(w io.Writer) {
b := []byte("written")
n, err := w.Write(b)
if n != len(b) {
log.Fatal("Short write")
}
if err != nil {
log.Fatal(err)
}
}
func test(rw io.ReadWriter) {
readStuff(rw)
writeStuff(rw)
}
func main() {
r := io.Reader(bytes.NewBufferString("source"))
var uw bytes.Buffer
w := io.Writer(&uw)
rw := bufio.NewReadWriter(bufio.NewReader(r), bufio.NewWriter(w))
test(rw)
rw.Flush()
fmt.Printf("The underlying bytes.Buffer writer contains %q\n", uw.Bytes())
}
(Also here)
Output:
readStuff: "source"
The underlying bytes.Buffer writer contains "written"
This way test can consume any io.ReadWriter, not only a specific one. Which is a hint towards your question about go "philosophy".