I'm trying to write non-string value into io.Writer like integer, float, slices, or even map[string]interface{}. I'm expecting the result written returned as expected type written. If I wrote int into the Write, then I will get integer type value after decoding the []byte returned. How to do it in Go?
What you're probably looking for is encoding/gob since that encoding retains the Go type information. Out of the box it supports some of the builtin Go types and some of the basic gob types. If you want to encode/decode types not supported out of the box by gob you can use the gob.Register function to register those types.
To encode:
var v interface{} = uint8(123)
if err := gob.NewEncoder(w).Encode(&v); err != nil {
panic(err)
}
Note that the above passes a value of type *interface{} to Encode, this is necessary if, at the other end, the decoder doesn't know the type beforehand and has to also use type *interface{} as the argument to Decode. If you have a scenario where the decoder knows the concrete type the of the incoming data then you can also pass a value of that concrete type to Encode.
To decode:
var v interface{}
if err := gob.NewDecoder(r).Decode(&v); err != nil {
panic(err)
}
fmt.Println(v) // output: 123
fmt.Printf("%T", v) // output: uint8
https://play.golang.org/p/cCtQse8BoqZ
This seems to do it:
package main
import "encoding/json"
func main() {
a := []interface{}{
31, 3.1, []int{12,31}, map[string]interface{}{"month": 12, "day": 31},
}
b, err := json.Marshal(a)
if err != nil {
panic(err)
}
println(string(b)) // [31,3.1,[12,31],{"day":31,"month":12}]
}
https://pkg.go.dev/encoding/json#Marshal
Related
My code calls a library function which looks roughly like this:
func Search() ([]myLibrary.SomeObject, error) {
var results []apiv17.SomeObject
// ...
if (resultsFound) {
results = append(results, someResult)
}
return results
}
...and my code calls it and then marshals it to JSON.
results, err := myLibrary.Search()
bytes, err := json.Marshal(results)
Now the problem is that because of the way the Search function is written (and let's assume we can't change it), it'll return an uninitialized nil slice if there are no results. And unfortunately, there is no way to configure encoding/json to encode nil slices as [] (see e.g. this proposal with ongoing discussion).
Explicitly checking for nil solves the problem:
results, err := myLibrary.Search()
if results == nil {
results = []apiv17.SomeObject{}
}
bytes, err := json.Marshal(results)
...but it also adds an explicit dependency on the return type, apiv17.SomeObject. That's inconvenient because that type frequently changes in the library. E.g. in the next library version it might be apiv18.SomeObject.
With the nil check above, I'll have to update my code every time that happens.
Is there any way to avoid this and assign an empty, non-nil slice to the variable without explicitly referring to its type? Something like this:
results = [](type of results){}
Go 1.18
You can use a generic function that captures the slice's base type and returns a slice of length zero:
func echo[T any](v []T) []T {
return make([]T, 0)
}
func main() {
n := foo.GetFooBar()
if n == nil {
n = echo(n) // no need to refer to apiv17 here
}
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints []
}
The purpose of requiring a regular argument v []T in echo is to allow type inference to unify the slice []apiv17.SomeObject with the argument []T and infer T as the base type apiv17.SomeObject, so that you can call it just as echo(n) and no explicit type parameter.
The package apiv17 is of course known at compile time because it's transitively imported via myPackage, so you can take advantage of this and type inference to avoid adding an explicit import statement for apiv17.
This is how it looks like on the multi-file playground: https://go.dev/play/p/4ycTkaGLFpo
The type is declared in bar package, but main only imports play.ground/foo and only uses foo.GetFooBar.
Go 1.17 and below
Reflection. Just change the echo function from above to taking an interface{} argument (there's no any in Go 1.17, remember?) and do the deed with reflect.MakeSlice:
func set(v interface{}) {
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
panic("not a ptr")
}
reflect.Indirect(rv).Set(reflect.MakeSlice(rv.Type().Elem(), 0, 0))
}
Then pass a pointer to the slice, so that you can set its value with reflection.
func main() {
n := foo.GetFooBar()
if n == nil {
set(&n)
}
fmt.Printf("type: %T, val: %v, is nil: %t\n", n, n, n == nil)
// type: []bar.FooBar, val: [], is nil: false
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints [] again
}
Go 1.17 playground: https://go.dev/play/p/4jMkr22LMF7?v=goprev
The other answer describes how to create an empty slice.
But you can solve your original issue much simpler: if results is nil, you don't need to create a empty slice, regardless of whatever element type it would have, the JSON marshaling would be [] anyway. So if results is nil, no need to call json.Marshal(), just "output" []:
results, err := myLibrary.Search()
var bytes []byte
if results == nil {
bytes = []byte{'[', ']' } // JSON marshaling result is "[]"
} else {
bytes, err = json.Marshal(results)
// Handle error
}
Starting to play around with golang and was looking at a custom json.Unmarshal. In a blog post the had the following:
type FlexInt int
func (fi *FlexInt) UnmarshalJSON(data []byte) error {
if data[0] != '"' {
return json.Unmarshal(data, (*int)(fi))
}
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
i, err := strconv.Atoi(s)
if err != nil {
return err
}
*fi = FlexInt(i)
return nil
}
And I understand what it is doing - but I dont understand (*int)(fi) part. Looking at the value of the fi pointer it is the same as (*int)(fi) - (*int)(fi) == fi. Yet when I change that line to simply fi it then does an infinite loop
The expression converts fi to an *int. The result contains the same address, but has a different type.
If a *FlexInt is passed to json.Unmarshal, then json.Unmarshal will call the *FlexInt.UnmarshalJSON method which calls json.Unmarshal and so on.
It's the same pointer, but with a different type. When json.Unmarshal is called with an interface{} that contains a FlexInt*, it calls FlexInt's UnmarshalJSON method. When it's called with an interface{} that contains an int*, it uses the builtin behavior. Since FlexInt and int have the same underlying type, it's acceptable to convert a pointer to one into a pointer to the other, but it's the "actual" type that the interface will be marked with.
I have a struct that embeds an embedded pointer to another struct. When I use the default json.Unmarshal behavior, it works perfectly. But when I implement UnmarshalJSON for the embedded struct's type but not the outer struct, then go panics with null pointer dereference.
If I implement UnmarshalJSON for the outer struct type as well, then it works. However, the outer struct has many fields that I would rather not have to manually unmarshal.
Why does implementing UnmarshalJSON on one and not the other cause a panic?
Is there a way to get it work without implemented UnmarshalJSON for the outer type?
If not, is there a simpler/less manual way to implement UnmarshalJSON for the outer type?
Note: There is a question with a similar title, "json.Unmarshal fails when embedded type has UnmarshalJSON
", but the issue there is different from mine.
tl;dr: The rest of this question is just a lengthy example of the above.
Base Example
(play.golang.org version of example)
The two structs, one with an embedded field pointer to the other:
(Simplified for example -- this doesn't really need its own UnmarshalJSON but it demonstrates the problem.)
type Obj struct {
X int `json:"x"`
}
type Container struct {
*Obj
Y int `json:"y"`
}
Invoking unmarshal:
func main() {
b := []byte(`{"x": 5, "y": 3}`)
c := &Container{}
err := json.Unmarshal(b, c)
if err != nil {
fmt.Printf("error ummarshalling json: %+v\n", err)
return
}
fmt.Printf("unmarshalled: %+v --> %+v\n", c, c.Obj)
}
Without implementing any UnmarshalJSON funcs, this works fine:
unmarshalled: &{Obj:0x416080 Y:3} --> &{X:5}
Panic
But, if I add UnmarshalJSON to the embedded Obj type only, then the program panics, as the json.Unmarshal call passes a nil pointer when it tries to unmarshal *Obj.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Output:
panic: runtime error: invalid memory address or nil pointer dereference
[...]
main.(*Obj).UnmarshalJSON(0x0, 0x416030, 0x10, 0x10, 0x0, 0x0)
/tmp/sandbox185809294/main.go:18 +0x130
[...]
Question: Why does it panic here but not with the default unmarshal behavior? I'd think that if a nil *Obj is being passed here, then the default behavior also passes around a nil pointer...
Fixing the panic
It no longer panics when I implement UnmarshalJSON for the outer Container type:
func (c *Container) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return err
}
c.Obj = &Obj{X: m["x"]}
c.Y = m["y"]
return nil
}
But unmarshalling Container manually this way gets tedious if both the real Container and real Obj have more fields than this, each with different types.
Question: Is there a simpler way to prevent this panic?
Because the default behavior checks for nil and your custom unmarshaller does not. You need some logic in your UnmarshalJSON to check if o is nil and behave appropriately, instead of assuming o is not nil (by trying to access one of its fields), thereby triggering a panic.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
if o == nil {
return nil // maybe? What do you want to happen in this case?
}
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Also just for future reference, your *Obj field is not an "anonymous field", it is an embedded field: https://golang.org/ref/spec#Struct_types
I am json.Decode()'ing (forgive the shorthand) a json response from an api into a large struct. Within that struct are a few types that are of type []interface{}. I can't figure out how to extract any data from those special nested structs. I have tried using a case switch type checking solution but came up empty handed still. Can someone share their experience with a similar case or point me in the right direction?
m := new(largestruct)
if err := json.NewDecoder(resp.Body).Decode(&m); err != nil{
return err
}
The struct field for interface is:
Strings []interface{} `json:"strings"`
Using the switch case you can fetch the value underlying the interface. The function will run recursively until it gets the original type of the parsed json.
func fetchValue(value interface{}) { // pass the interface value from the struct in this function as it will run recursively to get the value.
switch value.(type) {
case string:
fmt.Printf("%v is an string \n ", value.(string))
case bool:
fmt.Printf("%v is bool \n ", value.(bool))
case float64:
fmt.Printf("%v is float64 \n ", value.(float64))
case []interface{}:
fmt.Printf("%v is a slice of interface \n ", value)
for _, v := range value.([]interface{}) {
fetchValue(v)
}
case map[string]interface{}:
fmt.Printf("%v is a map \n ", value)
for _, v := range value.(map[string]interface{}) {
fetchValue(v)
}
default:
fmt.Printf("%v is unknown \n ", value)
}
}
The reason behind the types in switch to be limited is defined in the golang spec for unmarshal where it is clearly described what values the json will parse into when unmarshaling using interface{}:
To unmarshal JSON into an interface value, Unmarshal stores one of
these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
I am calling a function to do a http request, two pass by reference parameter is used for the function. I pass the []byte to v interface. I want the function to update the v interface reference value. The response body is a string, I want to pass the string value to v interface. However, tried many ways but not success.
Here is the code, you can see I declare byts as v.(*[]byte) in order to make v updated with the string value of response body. But it does not work. The v is always nil. Please suggest any way to make v can be updated with the string value.
func (s *BackendConfiguration) Do(req *http.Request, v interface{}) error {
res, err := s.HTTPClient.Do(req)
defer res.Body.Close()
resBody, err := ioutil.ReadAll(res.Body)
if v != nil {
byts, ok := v.(*[]byte)
if len(resBody) > 0 {
byts = append(byts, resBody...)
return nil
}
}
}
return nil
}
Well, the main reason this does not work is because you think of "call by reference", a concept completely unknown to Go. Absolutely everything is called by value in Go and once you spell out what is a byte slice, a pointer to a byte slice, a pointer to byte slice wrapped inside an interface, a copy of the pointer to a byte slice extracted from the interface, and so on you'll see how to update the value the pointer to byte slice points to:
package main
import "fmt"
func f(v interface{}) {
pbs := v.(*[]byte)
*pbs = append(*pbs, []byte{9,8,7}...)
}
func main() {
bs := []byte{1,2,3}
pbs := &bs
var v interface{} = pbs
f(v)
fmt.Printf("%v\n", *pbs)
}