I am json.Decode()'ing (forgive the shorthand) a json response from an api into a large struct. Within that struct are a few types that are of type []interface{}. I can't figure out how to extract any data from those special nested structs. I have tried using a case switch type checking solution but came up empty handed still. Can someone share their experience with a similar case or point me in the right direction?
m := new(largestruct)
if err := json.NewDecoder(resp.Body).Decode(&m); err != nil{
return err
}
The struct field for interface is:
Strings []interface{} `json:"strings"`
Using the switch case you can fetch the value underlying the interface. The function will run recursively until it gets the original type of the parsed json.
func fetchValue(value interface{}) { // pass the interface value from the struct in this function as it will run recursively to get the value.
switch value.(type) {
case string:
fmt.Printf("%v is an string \n ", value.(string))
case bool:
fmt.Printf("%v is bool \n ", value.(bool))
case float64:
fmt.Printf("%v is float64 \n ", value.(float64))
case []interface{}:
fmt.Printf("%v is a slice of interface \n ", value)
for _, v := range value.([]interface{}) {
fetchValue(v)
}
case map[string]interface{}:
fmt.Printf("%v is a map \n ", value)
for _, v := range value.(map[string]interface{}) {
fetchValue(v)
}
default:
fmt.Printf("%v is unknown \n ", value)
}
}
The reason behind the types in switch to be limited is defined in the golang spec for unmarshal where it is clearly described what values the json will parse into when unmarshaling using interface{}:
To unmarshal JSON into an interface value, Unmarshal stores one of
these in the interface value:
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
Related
My code calls a library function which looks roughly like this:
func Search() ([]myLibrary.SomeObject, error) {
var results []apiv17.SomeObject
// ...
if (resultsFound) {
results = append(results, someResult)
}
return results
}
...and my code calls it and then marshals it to JSON.
results, err := myLibrary.Search()
bytes, err := json.Marshal(results)
Now the problem is that because of the way the Search function is written (and let's assume we can't change it), it'll return an uninitialized nil slice if there are no results. And unfortunately, there is no way to configure encoding/json to encode nil slices as [] (see e.g. this proposal with ongoing discussion).
Explicitly checking for nil solves the problem:
results, err := myLibrary.Search()
if results == nil {
results = []apiv17.SomeObject{}
}
bytes, err := json.Marshal(results)
...but it also adds an explicit dependency on the return type, apiv17.SomeObject. That's inconvenient because that type frequently changes in the library. E.g. in the next library version it might be apiv18.SomeObject.
With the nil check above, I'll have to update my code every time that happens.
Is there any way to avoid this and assign an empty, non-nil slice to the variable without explicitly referring to its type? Something like this:
results = [](type of results){}
Go 1.18
You can use a generic function that captures the slice's base type and returns a slice of length zero:
func echo[T any](v []T) []T {
return make([]T, 0)
}
func main() {
n := foo.GetFooBar()
if n == nil {
n = echo(n) // no need to refer to apiv17 here
}
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints []
}
The purpose of requiring a regular argument v []T in echo is to allow type inference to unify the slice []apiv17.SomeObject with the argument []T and infer T as the base type apiv17.SomeObject, so that you can call it just as echo(n) and no explicit type parameter.
The package apiv17 is of course known at compile time because it's transitively imported via myPackage, so you can take advantage of this and type inference to avoid adding an explicit import statement for apiv17.
This is how it looks like on the multi-file playground: https://go.dev/play/p/4ycTkaGLFpo
The type is declared in bar package, but main only imports play.ground/foo and only uses foo.GetFooBar.
Go 1.17 and below
Reflection. Just change the echo function from above to taking an interface{} argument (there's no any in Go 1.17, remember?) and do the deed with reflect.MakeSlice:
func set(v interface{}) {
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
panic("not a ptr")
}
reflect.Indirect(rv).Set(reflect.MakeSlice(rv.Type().Elem(), 0, 0))
}
Then pass a pointer to the slice, so that you can set its value with reflection.
func main() {
n := foo.GetFooBar()
if n == nil {
set(&n)
}
fmt.Printf("type: %T, val: %v, is nil: %t\n", n, n, n == nil)
// type: []bar.FooBar, val: [], is nil: false
bytes, _ := json.Marshal(n)
fmt.Println(string(bytes)) // prints [] again
}
Go 1.17 playground: https://go.dev/play/p/4jMkr22LMF7?v=goprev
The other answer describes how to create an empty slice.
But you can solve your original issue much simpler: if results is nil, you don't need to create a empty slice, regardless of whatever element type it would have, the JSON marshaling would be [] anyway. So if results is nil, no need to call json.Marshal(), just "output" []:
results, err := myLibrary.Search()
var bytes []byte
if results == nil {
bytes = []byte{'[', ']' } // JSON marshaling result is "[]"
} else {
bytes, err = json.Marshal(results)
// Handle error
}
I'm trying to write non-string value into io.Writer like integer, float, slices, or even map[string]interface{}. I'm expecting the result written returned as expected type written. If I wrote int into the Write, then I will get integer type value after decoding the []byte returned. How to do it in Go?
What you're probably looking for is encoding/gob since that encoding retains the Go type information. Out of the box it supports some of the builtin Go types and some of the basic gob types. If you want to encode/decode types not supported out of the box by gob you can use the gob.Register function to register those types.
To encode:
var v interface{} = uint8(123)
if err := gob.NewEncoder(w).Encode(&v); err != nil {
panic(err)
}
Note that the above passes a value of type *interface{} to Encode, this is necessary if, at the other end, the decoder doesn't know the type beforehand and has to also use type *interface{} as the argument to Decode. If you have a scenario where the decoder knows the concrete type the of the incoming data then you can also pass a value of that concrete type to Encode.
To decode:
var v interface{}
if err := gob.NewDecoder(r).Decode(&v); err != nil {
panic(err)
}
fmt.Println(v) // output: 123
fmt.Printf("%T", v) // output: uint8
https://play.golang.org/p/cCtQse8BoqZ
This seems to do it:
package main
import "encoding/json"
func main() {
a := []interface{}{
31, 3.1, []int{12,31}, map[string]interface{}{"month": 12, "day": 31},
}
b, err := json.Marshal(a)
if err != nil {
panic(err)
}
println(string(b)) // [31,3.1,[12,31],{"day":31,"month":12}]
}
https://pkg.go.dev/encoding/json#Marshal
I am looking for a way to cast an interface into the concrete type to save a lot of source code.
The initial situation are two functions of a webserver handler. They differ only in that one function decodes an array of structs and the other decodes a single struct and stores it in the database. The functions which have to be called for saving are identical depending on the type.
To decide whether an array or a struct is passed, it is tried to cast the interface into the type and then to pass this appropriately as a parameter of the function. Similar as it is described in the documentation and in the stackoverflow post.
However, I do not get the concrete type as expected and the program always runs into the default section. What am I doing wrong or have I not taken into account?
These are the outputs of the default section:
# interface is a struct
... or a single repository struct: map[string]interface{}
# interface is an array of structs
... or a single repository struct: []interface{}
Below is the source code with the functions
func (rh *RouteHandler) AddOrUpdateRepository(rw http.ResponseWriter, req *http.Request) {
repository := new(types.Repository)
rh.addOrUpdateRepositories(rw, req, repository)
}
func (rh *RouteHandler) AddOrUpdateRepositories(rw http.ResponseWriter, req *http.Request) {
repositories := make([]*types.Repository, 0)
rh.addOrUpdateRepositories(rw, req, repositories)
}
func (rh *RouteHandler) addOrUpdateRepositories(rw http.ResponseWriter, req *http.Request, v interface{}) {
defer req.Body.Close()
switch req.Header.Get("Content-Type") {
case "application/xml":
xmlDecoder := xml.NewDecoder(req.Body)
err := xmlDecoder.Decode(&v)
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to decode repositories or repository")
rh.ulogger.Error("Failed to decode repositories or repository: %v", err)
return
}
case "application/json":
fallthrough
default:
jsonDecoder := json.NewDecoder(req.Body)
err := jsonDecoder.Decode(&v)
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to decode repositories or repository")
rh.ulogger.Error("Failed to decode repositories or repository: %v", err)
return
}
}
var err error
switch x := v.(type) {
case map[string]*types.Repository:
for _, repository := range x {
err = rh.manager.AddOrUpdateRepository(context.Background(), repository)
}
case *types.Repository:
err = rh.manager.AddOrUpdateRepository(context.Background(), x)
case map[string][]*types.Repository:
for i := range x {
for j := range x[i] {
err = rh.manager.AddOrUpdateRepository(context.Background(), x[i][j])
}
}
case []*types.Repository:
err = rh.manager.AddOrUpdateRepository(context.Background(), x...)
case nil:
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to cast interface")
rh.ulogger.Error("Failed to cast interface. Interface is a type of nil")
return
default:
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to cast interface")
rh.ulogger.Error("Failed to cast interface. Interface does not match onto an array of repositories or a single repository struct: %T", x)
return
}
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to add repositories or repository")
rh.ulogger.Error("Failed to add repositories or repository: %v", err)
return
}
rw.WriteHeader(http.StatusCreated)
}
(Simplified a bit.)
You have a function with the following signature:
func addOrUpdateRepositories(v interface{})
and then you call it like this:
repository := new(types.Repository)
addOrUpdateRepositories(repository)
and like this:
repositories := make([]*types.Repository, 0)
addOrUpdateRepositories(repositories)
In the first call, the concrete type of the value stored in v will be *types.Repository (as new returns a pointer to the allocated value) and in the second call the concrete type of the value stored in v will be []*types.Repository—because that's what make was told to create.
Now you do a type-switch on v, which reads:
switch x := v.(type) {
case map[string]*types.Repository:
case map[string][]*types.Repository:
case nil:
default:
}
Leaving aside the case that if you do not call addOrUpdateRepositories passing it a nil v which cannot happen in the snippet from your question, the switch will always pick the default branch because the type of the concrete value stored in v is never map[string]*types.Repository or map[string][]*types.Repository.
I am not sure why you fail to see this, so probably you should refine your question or may be try to clear up your confusion in a comment to my answer?
Another shot in the dark: type conversions (note that Go does not have type casts, as #Flimzy pointed out) and type switches in Go do not actually change the underlying representation of the value they operate on—except for the limited set of ("everyone expects this") cases such as type-converting a float64 to int64, which are precisely documented.
So you cannot take a []*types.Repository (a slice of pointers to values of type types.Repository) and somehow force it "to become" map[string][]*types.Repository: that'd be a nonsensical thing to do for a number of reasons, with the most glaring being this: if you were writing the Go compiler, how would you carry out such a "type cast"? Suppose you were to really allocate a map, but then which key in that map should have been assigned the original (source) slice? What about type-converting a []*types.Repository to struct {foo []*types.Repository; bar []*types.Repository}?
Starting to play around with golang and was looking at a custom json.Unmarshal. In a blog post the had the following:
type FlexInt int
func (fi *FlexInt) UnmarshalJSON(data []byte) error {
if data[0] != '"' {
return json.Unmarshal(data, (*int)(fi))
}
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
i, err := strconv.Atoi(s)
if err != nil {
return err
}
*fi = FlexInt(i)
return nil
}
And I understand what it is doing - but I dont understand (*int)(fi) part. Looking at the value of the fi pointer it is the same as (*int)(fi) - (*int)(fi) == fi. Yet when I change that line to simply fi it then does an infinite loop
The expression converts fi to an *int. The result contains the same address, but has a different type.
If a *FlexInt is passed to json.Unmarshal, then json.Unmarshal will call the *FlexInt.UnmarshalJSON method which calls json.Unmarshal and so on.
It's the same pointer, but with a different type. When json.Unmarshal is called with an interface{} that contains a FlexInt*, it calls FlexInt's UnmarshalJSON method. When it's called with an interface{} that contains an int*, it uses the builtin behavior. Since FlexInt and int have the same underlying type, it's acceptable to convert a pointer to one into a pointer to the other, but it's the "actual" type that the interface will be marked with.
How can I get the underlying value of a field having the fields name as a string?
I understand I need to use reflection but if I do will I have to continue using it throughout my code? Is there any way to assert?
I would just like to get the value of the field, the underlying struct, in this case a []Dice.
http://play.golang.org/p/KYOH8C7TAl
type Dice struct {
In int
}
type SliceNDice struct {
Unknown []Dice
}
func main() {
structure := SliceNDice{make([]Dice, 10)}
refValue := reflect.ValueOf(&structure).Elem().FieldByName(string("Unknown"))
slice := refValue.Slice(0, refValue.Len())
// cannot range over slice (type reflect.Value)
//for i,v := range slice {
// fmt.Printf("%v %v\n", i, v.In)
//}
for i := 0; i < slice.Len(); i++ {
v := slice.Index(i)
// v.In undefined (type reflect.Value has no field or method In)
fmt.Printf("%v %v\n", i, v.In)
}
}
If you know that the "Unknown" field is of type []Dice, you can use Value.Interface to get the underlying value and convert it with a type assertion:
slice := refValue.Interface().([]Dice)
for i,v := range slice {
fmt.Printf("%v %v\n", i, v.In)
}
http://play.golang.org/p/2lV106b6dH