How can I convert a JSON string to a byte array? - go

I need some help with unmarshaling. I have this example code:
package main
import (
"encoding/json"
"fmt"
)
type Obj struct {
Id string `json:"id"`
Data []byte `json:"data"`
}
func main() {
byt := []byte(`{"id":"someID","data":["str1","str2"]}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Println(obj)
}
What I try to do here - convert bytes to the struct, where type of one field is []byte. The error I get:
panic: json: cannot unmarshal string into Go struct field Obj.data of
type uint8
That's probably because parser already sees that "data" field is already a slice and tries to represent "str1" as some char bytecode (type uint8?).
How do I store the whole data value as one bytes array? Because I want to unmarshal the value to the slice of strings later. I don't include a slice of strings into struct because this type can change (array of strings, int, string, etc), I wish this to be universal.

My first recommendation would be for you to just use []string instead of []byte if you know the input type is going to be an array of strings.
If data is going to be a JSON array with various types, then your best option is to use []interface{} instead - Go will happily unmarshal the JSON for you and you can perform checks at runtime to cast those into more specific typed variables on an as-needed basis.

If []byte really is what you want, use json.RawMessage, which is of type []byte, but also implements the methods for JSON parsing. I believe this may be what you want, as it will accept whatever ends up in data. Of course, you then have to manually parse Data to figure out just what actually IS in there.
One possible bonus is that this skips any heavy parsing because it just copies the bytes over. When you want to use this data for something, you use a []interface{}, then use a type switch to use individual values.
https://play.golang.org/p/og88qb_qtpSGJ
package main
import (
"encoding/json"
"fmt"
)
type Obj struct {
Id string `json:"id"`
Data json.RawMessage `json:"data"`
}
func main() {
byt := []byte(`{"id":"someID","data":["str1","str2", 1337, {"my": "obj", "id": 42}]}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Printf("%+v\n", obj)
fmt.Printf("Data: %s\n", obj.Data)
// use it
var d []interface{}
if err := json.Unmarshal(obj.Data, &d); err != nil {
panic(err)
}
fmt.Printf("%+v\n", d)
for _, v := range d {
// you need a type switch to deterine the type and be able to use most of these
switch real := v.(type) {
case string:
fmt.Println("I'm a string!", real)
case float64:
fmt.Println("I'm a number!", real)
default:
fmt.Printf("Unaccounted for: %+v\n", v)
}
}
}

Your question is:
convert bytes array to struct with a field of type []byte
But you do not have a bytearray but a string array. Your question is not the same as your example. So let answer your question, there are more solutions possible depending in how far you want to diverge from your original requirements.
One string can be converted to one byte-slice, two strings need first to be transformed to one string. So that is problem one. The second problem are the square-brackets in your json-string
This works fine, it implicitly converts the string in the json-string to a byte-slice:
byt := []byte(`{"id":"someID","data":"str1str2"}`)
var obj Obj
if err := json.Unmarshal(byt, &obj); err != nil {
panic(err)
}
fmt.Println(obj)

Related

go-redis Eval func return value type, when Lua script returns an array

When a Lua script returns a table array during an Eval call, how can it be converted to a []string in go?
The redis cli returns bulk replies in the following format.
1) val1
2) val2
Will go-redis eval function return the bulk entries as
["val1", "val2"]
Redis returns Lua table arrays as RESP2 arrays. The Go client then will map that response to Go native types. The relevant documentation for go-redis is found here: Lua and Go types.
The tl;dr is that Lua tables are indeed mapped to a bulk reply, and the Go client maps that to a slice of interface: []interface{}.
Both go-redis script Run and Eval return a *Cmd. You can use methods on this type to retrieve the output as Go types. Result gives (interface{}, error), which you can type-assert to whatever you want, otherwise, StringSlice is a convenience getter to retrieve []string right away.
So it looks like:
script := redis.NewScript(`
local foo = {"val1", "val2"}
return foo
`)
cmd := script.Run(/* parameters */)
i, err := cmd.Result() // (interface, error)
// or
ss, err := cmd.StringSlice() // ([]string, error)
If the values aren't actually all strings, use Slice to get the []interface{} slice, and then inspect the elements individually.
You can use the encoding/json package to convert a JSON string to a slice of strings.
package main
import (
"encoding/json"
"fmt"
)
// jsonString is the JSON string that you want to convert to a slice of strings.
const jsonString = `["value1", "value2"]`
func main() {
var stringSlice []string
// Unmarshal the JSON string into the stringSlice variable.
err := json.Unmarshal([]byte(jsonString), &stringSlice)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(stringSlice) // ["value1", "value2"]
}

Extracting Generic Struct Values by Reflection

I'm trying to extract all of the values for a struct into a string slice.
func structValues(item Item) []string {
values := []string{}
e := reflect.ValueOf(&item).Elem()
for i := 0; i < e.NumField(); i++ {
fieldValue := e.Field(i).Interface()
values = append(values, fmt.Sprintf("%#v", fieldValue))
}
return values
}
I'd like to use this function with any struct, so I thought I could just change the type signature to func structValues(item interface{}) but then I got a panic:
panic: reflect: call of reflect.Value.NumField on interface Value
Working example: https://repl.it/#fny/stackoverflow61719532
I'd like to use this function with any struct ...
You can do this, but note that it gives up type-safety. Moreover, the only way to do this is to allow a call with any type, not just any type that is some structure type, so you have to check that what you got was in fact some struct type:
func structValues(item interface{}) {
if reflect.ValueOf(item).Kind() != reflect.Struct {
... do something here ...
}
Having made that check—or deferring it slightly, or omitting it to allow reflect to panic instead—you then need to replace reflect.ValueOf(&item).Elem() with the simpler reflect.ValueOf(item).
If you wish to allow pointers to structures as well as actual structures, you can make that happen pretty simply by using reflect.Indirect first. The result is:
func structValues(item interface{}) []string {
e := reflect.Indirect(reflect.ValueOf(item))
if e.Kind() != reflect.Struct {
panic("not a struct")
}
values := []string{}
for i := 0; i < e.NumField(); i++ {
fieldValue := e.Field(i).Interface()
values = append(values, fmt.Sprintf("%#v", fieldValue))
}
return values
}
Leave out the reflect.Indirect if you want to make sure that callers do their own indirection when they have a pointer.
(Note that the panic here is not very friendly. If you want proper debugging, consider either just printing the struct directly with %v or %#v, or for something much more thorough, the spew package.)
Complete example here on the Go Playground uses your type Item struct from your own link.

How do you pass a slice of *interface{} as arguments?

I want to use Scan() in package sql, but the number of columns, and hence the number of arguments, will change at runtime. This is the signature of Scan():
func (rs *Rows) Scan(dest ...interface{}) error
According to the documentation, *interface{} is one of the types accepted by Scan(). So I want to create a slice of []*interface{} and that expand as arguments.
This is what I thought would work:
func query(database *sql.DB) {
rows, _ := database.Query("select * from testTable")
for rows.Next() {
data := make([]*interface{}, 2)
err := rows.Scan(data...) // Compilation error
fmt.Printf("%v%v\n", *data[0], *data[1])
if err != nil {
fmt.Println(err.Error())
}
}
}
Compilation fails with cannot use data (type []*interface {}) as type []interface {} in argument to rows.Scan. I thought that data... would expand to &data[0], &data[1], but apparently not. I don't understand the error message. *interface{} is compatible with interface{}, so why can't I expand the slice of pointers to interface types?
This works:
func query(database *sql.DB) {
rows, _ := database.Query("select * from testTable")
for rows.Next() {
data := make([]*interface{}, 2)
err := rows.Scan(&data[0], &data[1]) // Only changed this line
fmt.Printf("%v%v\n", *data[0], *data[1]) // Outputs "[48][116 101 120 116]"
if err != nil {
fmt.Println(err.Error())
}
}
}
I can't use this however, because the number of columns is unknown at compile time. How can I write this code so that I can pass a variable number of *interface{} to rows.Scan()?
First, you must not use []*interface{} slice of pointers to interface rather than []interface{} where the interfaces are pointers. []*interface{} is different from []interface{}. Just create a slice of interfaces where each element is a pointer to a concrete type.
Here is a snippet how you would do this.
var x int
var s string
data := []interface{}{&x, &s}
rows.Scan(data...)
Note on the use of the ... spread operator.
Here are some related questions that will explain a bit more:
golang: slice of struct != slice of interface it implements?
Cannot convert []string to []interface {}
If you really want to pass a []*interface{} (perhaps you don't know the concrete types of the output) you must first wrap each *interface{} in a interface{}:
values := make([]interface{}, columnsCount)
for i := range values {
values[i] = new(interface{})
}
Individual values passed into a ...interface{} parameter are automatically wrapped in a interface{}, but just like []int... won't satisfy ...interface{}, neither will []*interface{}....

how to insert an array in sqlite?

I have struct like:
type Foo struct {
bars []string
}
Since sqlite3 doesn't have array data type supported, can we store []string as string and while retrieving return as slice of string? Was trying to implement like below, but getting error because of type mismatch. What need to be done here?
Edit: I have changed the code and look like working
type strArray []string
func (strarr StrArray) Value() (driver.Value, error) {
if strarr != nil {
resarr := strings.Join(strarr, "")
return resarr, nil
}
return nil, nil
}
Complementary to database/sql/driver.Valuer you need also to implement database/sql.Scanner for reading your type from the database.
When you think of how to implement it, it's obvious that in Valuer you should Join your slice with some delimiter character/string (not occurring in the data of course) to be able to Split it back when retrieving.
Assuming that such delimiter would be ; (my wild guess), the code for reading would look like:
func (a *strArray) Scan(value interface{}) error {
if value == nil {
return nil // case when value from the db was NULL
}
s, ok := value.(string)
if !ok {
return fmt.Errorf("failed to cast value to string: %v", value)
}
*a = strings.Split(s, ";")
return nil
}
For writing, you'd need to use strings.Join(strarr, ";") in Valuer implementation.
Other less-trivial implementation would require marshaling your slice and encoding the resulting bytes as string somehow (base32/64? json?). In any case you need to not loose the information what are distinct slice elements when saving them to the database.

how to access nested Json key values in Golang

Team,
new to Programming.
I have data available after unmarshaling the Json as shown below, which has nested Key values. flat key values I am able to access, how do I access nested key values.
Here is the byte slice data shown below after unmarshaling —>
tables:[map[name:basic__snatpool_members] map[name:net__snatpool_members] map[name:optimizations__hosts] map[columnNames:[name] name:pool__hosts rows:[map[row:[ry.hj.com]]]] traffic_group:/Common/traffic-group-1
Flat key values I am able to access by using the following code
p.TrafficGroup = m[“traffic_group”].(string)
here is the complete function
func dataToIapp(name string, d *schema.ResourceData) bigip.Iapp {
var p bigip.Iapp
var obj interface{}
jsonblob := []byte(d.Get("jsonfile").(string))
err := json.Unmarshal(jsonblob, &obj)
if err != nil {
fmt.Println("error", err)
}
m := obj.(map[string]interface{}) // Important: to access property
p.Name = m[“name”].(string)
p.Partition = m[“partition”].(string)
p.InheritedDevicegroup = m[“inherited_devicegroup”].(string)
}
Note: This may not work with your JSON structure. I inferred what it would be based on your question but without the actual structure, I cannot guarantee this to work without modification.
If you want to access them in a map, you need to assert that the interface pulled from the first map is actually a map. So you would need to do this:
tmp := m["tables"]
tables, ok := tmp.(map[string]string)
if !ok {
//error handling here
}
r.Name = tables["name"].(string)
But instead of accessing the unmarshaled JSON as a map[string]interface{}, why don't you create structs that match your JSON output?
type JSONRoot struct {
Name string `json:"name"`
Partition string `json:"partition"`
InheritedDevicegroup string `json:"inherited_devicegroup"`
Tables map[string]string `json:"tables"` //Ideally, this would be a map of structs
}
Then in your code:
func dataToIapp(name string, d *schema.ResourceData) bigip.Iapp {
var p bigip.Iapp
var obj &JSONRoot{}
jsonblob := []byte(d.Get("jsonfile").(string))
err := json.Unmarshal(jsonblob, &obj)
if err != nil {
fmt.Println("error", err)
}
p.Name = obj.Name
p.Partition = obj.Partition
p.InheritedDevicegroup = obj.InheritedDevicegroup
p.Name = obj.Tables["name"]
}
JSON objects are unmarshaled into map[string]interface{}, JSON arrays into []interface{}, same applies for nested objects/arrays.
So for example if a key/index maps to a nested object you need to type assert the value to map[string]interface{} and if the key/index maps to an array of objects you first need to assert the value to []interface{} and then each element to map[string]interface{}.
e.g. (for brevity this code is not guarding against panic)
tables := obj.(map[string]interface{})["tables"]
table1 := tables.([]interface{})[0]
name := table1.(map[string]interface{})["name"]
namestr := name.(string)
However, if it's the case that the json you are parsing is not dynamic but instead has a specific structure you should define a struct type that mirrors that structure and unmarshal the json into that.
All you have to do is repeatedly accessing the map via type-switching or assertion:
for _, table := range m["tables"] {
switch val := table {
case string:
fmt.Println("table is string")
case int:
fmt.Println("table is integer")
// This is your case, since JSON is unmarshaled to type []interface{} and map[string]interface{}
case []interface{}:
fmt.Println("table is a slice of interface{}")
for _, tb := range value {
if m, ok := tb.(map[string]interface{}); ok {
// Now it's accessible
fmt.Println(m["name"])
}
}
default:
fmt.Println("unknown type")
}
}
You might want to handle errors better than this.
To read more, check out my writing from a while ago https://medium.com/code-zen/dynamically-creating-instances-from-key-value-pair-map-and-json-in-go-feef83ab9db2.

Resources