I am trying to write a simple custom marshaler and failing. Notice I have an interface that has three functions. Both Happy and Sad structs implement this interface by embedding the emotion struct which implements all the three required functions.
The problem is UnmarshalJSON does not get invoked when I call json.Unmarshal() on the pointer to either Happy or Sad and I can't understand why. You can reproduce the exact codebase in Go Playground or just look below. You will notice that while MarshalJSON is correctly called, UnmarshalJSON isn't.
type Emotion interface {
String() string
MarshalJSON() ([]byte, error)
UnmarshalJSON(data []byte) error
}
type emotion struct {
status string
}
func (s emotion) String() string {
return s.status
}
func (s emotion) MarshalJSON() ([]byte, error) {
fmt.Println("MarshalJSON is overriden: I am called fine")
x := struct {
Status string
}{
Status: s.String(),
}
return json.Marshal(x)
}
func (s *emotion) UnmarshalJSON(data []byte) error {
fmt.Println("MarshalJSON is overriden: I am never called")
y := struct {
Status string
}{
Status: "",
}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
s.status = y.Status
return nil
}
type Happy struct {
*emotion
}
// Job is not in any detention
type Sad struct {
*emotion
}
func main() {
x := Happy{&emotion{status: "happy"}}
jsonX, _ := json.Marshal(x)
var y Emotion
err := json.Unmarshal(jsonX, &y)
fmt.Printf("%v", err)
}
You cannot unmarshal into an abstract interface type.
An interface value is just a pointer to a type (associating that types methods) - it has no storage behind it - because an abstract type cannot know the exact size of any concrete value it may have in the future.
Using a concrete value type (that also implements that interface) will work:
y2 := emotion{}
err = json.Unmarshal(jsonX, &y2)
Playground: https://play.golang.org/p/8aCEjLgfKVQ
MarshalJSON is overriden: I am called fine
EXPECTED ERROR, Can't unmarshal into a non-concrete value: json: cannot unmarshal object into Go value of type main.Emotion
MarshalJSON is overriden: I am (fixed) and now called
SHOULD NOT ERROR: <nil>
VALUE: happy
Related
package main
import (
"encoding/json"
"fmt"
"log"
"strings"
)
type Animal int
const (
Unknown Animal = iota
Gopher
Zebra
)
func (a *Animal) UnmarshalJSON(b []byte) error {
var s string
if err := json.Unmarshal(b, &s); err != nil {
return err
}
switch strings.ToLower(s) {
default:
*a = Unknown
case "gopher":
*a = Gopher
case "zebra":
*a = Zebra
}
return nil
}
func (a Animal) MarshalJSON() ([]byte, error) {
var s string
switch a {
default:
s = "unknown"
case Gopher:
s = "gopher"
case Zebra:
s = "zebra"
}
return json.Marshal(s)
}
func main() {
blob := `["gopher","armadillo","zebra","unknown","gopher","bee","gopher","zebra"]`
var zoo []Animal
if err := json.Unmarshal([]byte(blob), &zoo); err != nil {
log.Fatal(err)
}
census := make(map[Animal]int)
for _, animal := range zoo {
census[animal] += 1
}
fmt.Printf("Zoo Census:\n* Gophers: %d\n* Zebras: %d\n* Unknown: %d\n",
census[Gopher], census[Zebra], census[Unknown])
}
This is the code snippet of Json custom marshal example in go doc. My question is where is the call to MarshalJSON and UnmarshalJSON method in this code. Are these method somehow overriding Json package's UnmarshalJSON and MarshalJSON method. I thought go does not support method overriding this way. Pls help, i am not able to understand what is happening in this code!!
The documentation says:
To unmarshal JSON into a value implementing the Unmarshaler interface, Unmarshal calls that value's UnmarshalJSON method, including when the input is a JSON null.
Somewhere in the json.Unmarshal implementation, there's code similar this:
u, ok := v.(Unmarshaler)
if ok {
err := u.Unmarshal(data)
if err != nil { /* handle error */}
} else {
// handle other kinds of values
}
The code uses a type assertion to determine if the value satisfies the json.Unmarshaler interface. If the value does satisfy the method, the value's UnmarshalJSON function is called.
The (*Animal).UnmarshalJSON function is called because *Animal satisfies the json.Unmarshaler interface.
This is an example of implementing an interface from a different package.
There's no need to explicitly declare that you're implementing an interface in Go, like there is in Java or C++ for example. You just have to implement all the functions it declares. In this case, you're implementing the Unmarshaler interface declared in the json package which is used by the Unmarshal function.
I'm developing a tool that can be implemented to simplify the process of creating simple CRUD operations/endpoints. Since my endpoints don't know what kind of struct they'll be receiving, I've created an interface that users can implement, and return an empty object to be filled.
type ItemFactory interface {
GenerateEmptyItem() interface{}
}
And the users would implement something like:
type Test struct {
TestString string `json:"testString"`
TestInt int `json:"testInt"`
TestBool bool `json:"testBool"`
}
func (t Test) GenerateEmptyItem() interface{} {
return Test{}
}
When the Test object gets created, its type is "Test", even though the func returned an interface{}. However, as soon as I try to unmarshal some json of the same format into it, it strips it of its type, and becomes of type "map[string]interface {}".
item := h.ItemFactory.GenerateEmptyItem()
//Prints "Test"
fmt.Printf("%T\n", item)
fmt.Println(reflect.TypeOf(item))
err := ConvertRequestBodyIntoObject(r, &item)
if err != nil {...}
//Prints "map[string]interface {}"
fmt.Printf("%T\n", item)
Func that unmarshalls item:
func ConvertRequestBodyIntoObject(request *http.Request, object interface{}) error {
body, err := ioutil.ReadAll(request.Body)
if err != nil {
return err
}
// Unmarshal body into request object
err = json.Unmarshal(body, object)
if err != nil {
return err
}
return nil
}
Any suggestions as to why this happens, or how I can work around it?
Thanks
Your question lacks an example showing this behavior so I'm just guessing this is what is happening.
func Generate() interface{} {
return Test{}
}
func GeneratePointer() interface{} {
return &Test{}
}
func main() {
vi := Generate()
json.Unmarshal([]byte(`{}`), &vi)
fmt.Printf("Generate Type: %T\n", vi)
vp := GeneratePointer()
json.Unmarshal([]byte(`{}`), vp)
fmt.Printf("GenerateP Type: %T\n", vp)
}
Which outputs:
Generate Type: map[string]interface {}
GenerateP Type: *main.Test
I suggest you return a pointer in GenerateEmptyItem() instead of the actual struct value which is demonstrated in the GenerateP() example.
Playground Example
Is it possible to have my function definition below accept any type of struct?
I've tried to refactor like so:
// This method should accept any type of struct
// Once I receive my response from the database,
// I scan the rows to create a slice of type struct.
func generateResponse(rows *sqlx.Rows, structSlice []struct{}, structBody struct{}) ([]struct{}, error) {
for rows.Next() {
err := rows.StructScan(&structBody)
if err != nil {
return nil, err
}
structSlice = append(structSlice, structBody)
}
err := rows.Err()
if err != nil {
return nil, err
}
return structSlice, nil
}
Assume my struct is of type OrderRevenue.
When I call the function above:
structSlice, err := generateResponse(rows, []OrderRevenue{}, OrderRevenue{})
The error I get is:
cannot use []OrderRevenue literal as type []struct{} in argument...
Am I going about this the wrong way?
This is considered the cornerstone (or more of a limitation) of Go's type system. struct{} is an unnamed type that is different from struct{ field1 int } and of course is not the same as OrderRevenue{}.
Go emphasizes abstraction through interfaces, and perhaps you should try that. Here is the first take:
type OrderRevenue interface {
MarshalMyself() ([]byte, error)
}
type Anonymous struct {}
func (a Anonymous) MarshalMyself() ([]byte, error) {
// implementation's up to you
return []byte{}, nil
}
// the function signature
generateResponse(rows *sqlx.Rows, structSlice []OrderRevenue, structBody Body) ([]Body, error) {
// ...
}
In this case you can also use empty interface interface{}, which all types implement, but you'll have to recursively go through the structure to do manual type assertion. The best approach in Go is to know the shape of your data in advance, at least partially.
I have created a new declared type and added a method to marshal the value into JSON
type TextOutput string
func (t *TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}
When I try to marshal an instance of the type I get the raw value returned. What am I missing?
var t TextOutput
t = `Test test`
output, err := json.Marshal(t)
if err != nil {
fmt.Println(err)
} else {
fmt.Println(string(output))
}
// prints Test Test. Expected {"data": "Test test"}
You have to define the MarshalJSON interface as a non-pointer.
func (t TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}
Play Link: https://play.golang.org/p/lLK6zsAkOi
Output:
{"data":"Test test"}
The root of the problem stems from how interfaces in Go are implicitly satisfied.
In this particular case, the json.Marshal method uses type assertion at runtime to see if the given value implements json.Marshaler. Effective Go mentions this very case.
You could have satisfied the json.Marshaler for the *TextOutput type using a pointer-receiver like so:
func (t *TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data":"%s"}`, *t)), nil
}
And for this to work properly, pass the reference to the json.Marshal function:
var t TextOutput
t = `Test test`
output, err := json.Marshal(&t)
However, implementing it using a value-receiver ensures that both TextOutput and *TextOutput types implement json.Marshaler
func (t TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}
lets say i have the following json
{
name: "John",
birth_date: "1996-10-07"
}
and i want to decode it into the following structure
type Person struct {
Name string `json:"name"`
BirthDate time.Time `json:"birth_date"`
}
like this
person := Person{}
decoder := json.NewDecoder(req.Body);
if err := decoder.Decode(&person); err != nil {
log.Println(err)
}
which gives me the error parsing time ""1996-10-07"" as ""2006-01-02T15:04:05Z07:00"": cannot parse """ as "T"
if i were to parse it manually i would do it like this
t, err := time.Parse("2006-01-02", "1996-10-07")
but when the time value is from a json string how do i get the decoder to parse it in the above format?
That's a case when you need to implement custom marshal and unmarshal functions.
UnmarshalJSON(b []byte) error { ... }
MarshalJSON() ([]byte, error) { ... }
By following the example in the Golang documentation of json package you get something like:
// First create a type alias
type JsonBirthDate time.Time
// Add that to your struct
type Person struct {
Name string `json:"name"`
BirthDate JsonBirthDate `json:"birth_date"`
}
// Implement Marshaler and Unmarshaler interface
func (j *JsonBirthDate) UnmarshalJSON(b []byte) error {
s := strings.Trim(string(b), "\"")
t, err := time.Parse("2006-01-02", s)
if err != nil {
return err
}
*j = JsonBirthDate(t)
return nil
}
func (j JsonBirthDate) MarshalJSON() ([]byte, error) {
return json.Marshal(time.Time(j))
}
// Maybe a Format function for printing your date
func (j JsonBirthDate) Format(s string) string {
t := time.Time(j)
return t.Format(s)
}
If there are lots of struct and you just implement custom marshal und unmarshal functions, that's a lot of work to do so. You can use another lib instead,such as a json-iterator extension jsontime:
import "github.com/liamylian/jsontime"
var json = jsontime.ConfigWithCustomTimeFormat
type Book struct {
Id int `json:"id"`
UpdatedAt *time.Time `json:"updated_at" time_format:"sql_date" time_utc:"true"`
CreatedAt time.Time `json:"created_at" time_format:"sql_datetime" time_location:"UTC"`
}
I wrote a package for handling yyyy-MM-dd and yyyy-MM-ddThh:mm:ss dates at https://github.com/a-h/date
It uses the type alias approach in the answer above, then implements the MarshalJSON and UnmarshalJSON functions with a few alterations.
// MarshalJSON outputs JSON.
func (d YYYYMMDD) MarshalJSON() ([]byte, error) {
return []byte("\"" + time.Time(d).Format(formatStringYYYYMMDD) + "\""), nil
}
// UnmarshalJSON handles incoming JSON.
func (d *YYYYMMDD) UnmarshalJSON(b []byte) (err error) {
if err = checkJSONYYYYMMDD(string(b)); err != nil {
return
}
t, err := time.ParseInLocation(parseJSONYYYYMMDD, string(b), time.UTC)
if err != nil {
return
}
*d = YYYYMMDD(t)
return
}
It's important to parse in the correct timezone. My code assumes UTC, but you may wish to use the computer's timezone for some reason.
I also found that solutions which involved using the time.Parse function leaked Go's internal mechanisms as an error message which clients didn't find helpful, for example: cannot parse "sdfdf-01-01" as "2006". That's only useful if you know that the server is written in Go, and that 2006 is the example date format, so I put in more readable error messages.
I also implemented the Stringer interface so that it gets pretty printed in log or debug messages.