How to iterate over an []interface{} in Go - go

I'm struggling to get the keys and values of the following interface, which is the result of JSON marshaling the result returned by Execute as demonstrated in this example:
[
[
{
"id": 36,
"label": "TestThing",
"properties": {
"schema__testBoolean": [
{
"id": 40,
"value": true
}
],
"schema__testInt": [
{
"id": 39,
"value": 1
}
],
"schema__testNumber": [
{
"id": 38,
"value": 1.0879834
}
],
"schema__testString": [
{
"id": 37,
"value": "foobar"
}
],
"uuid": [
{
"id": 41,
"value": "7f14bf92-341f-408b-be00-5a0a430852ee"
}
]
},
"type": "vertex"
}
]
]
A reflect.TypeOf(result) results in: []interface{}.
I've used this to loop over the array:
s := reflect.ValueOf(result)
for i := 0; i < s.Len(); i++ {
singleVertex := s.Index(i).Elem() // What to do here?
}
But I'm getting stuck with errors like:
reflect.Value.Interface: cannot return value obtained from unexported
field or method

If you know that's your data structure, there's no reason to use reflection at all. Just use a type assertion:
for key, value := range result.([]interface{})[0].([]interface{})[0].(map[string]interface{}) {
// key == id, label, properties, etc
}

For getting the underlying value of an interface use type assertion. Read more about Type assertion and how it works.
package main
import (
"fmt"
)
func main() {
res, err := g.Execute( // Sends a query to Gremlin Server with bindings
"g.V(x)",
map[string]string{"x": "1234"},
map[string]string{},
)
if err != nil {
fmt.Println(err)
return
}
fetchValue(res)
}
func fetchValue(value interface{}) {
switch value.(type) {
case string:
fmt.Printf("%v is an interface \n ", value)
case bool:
fmt.Printf("%v is bool \n ", value)
case float64:
fmt.Printf("%v is float64 \n ", value)
case []interface{}:
fmt.Printf("%v is a slice of interface \n ", value)
for _, v := range value.([]interface{}) { // use type assertion to loop over []interface{}
fetchValue(v)
}
case map[string]interface{}:
fmt.Printf("%v is a map \n ", value)
for _, v := range value.(map[string]interface{}) { // use type assertion to loop over map[string]interface{}
fetchValue(v)
}
default:
fmt.Printf("%v is unknown \n ", value)
}
}

Related

Convert JSON key value pair into single field value in Go

I have a json like following, where value can be int or string
{
"data": [
{
"Name": "a_name",
"value": 1
},
{
"Name": "b_name",
"value": "val"
},
{
"Name": "c_name",
"value": 2
}
]
}
Now I want to convert that json into following struct, like only extract a_name and b_name value.
type Data struct {
AName int `json: "a_name"`
BName string `json: "b_name"`
}
I can do it by following way
import (
"encoding/json"
"fmt"
)
type TmpData struct {
Data []struct {
Name string `json:"Name"`
Value interface{} `json:"value"`
} `json:"data"`
}
type ExpectedData struct {
AName int `json: "a_name"`
BName string `json: "b_name"`
}
func main() {
data := `{
"data": [
{
"Name": "a_name",
"value": 1
},
{
"Name": "b_name",
"value": "val"
},
{
"Name": "c_name",
"value": 2
}
]
}`
tmpData := &TmpData{}
json.Unmarshal([]byte(data), tmpData)
ans := &ExpectedData{}
for _, d := range tmpData.Data {
if d.Name == "a_name" {
ans.AName = int(d.Value.(float64))
} else if d.Name == "b_name" {
ans.BName = d.Value.(string)
}
}
fmt.Println(ans)
}
Is there any better solution for this?
Not possible with the standard JSON un-marshalling unless you write a custom un-marshaller for your Data type.
The key here is to define the type for value to be an interface{}, so that multiple types could be stored in your b_name record.
func (d *Data) UnmarshalJSON(data []byte) error {
var result Details
if err := json.Unmarshal(data, &result); err != nil {
return err
}
for _, value := range result.Data {
switch value.Name {
// The json package will assume float64 when Unmarshalling with an interface{}
case "a_name":
v, ok := (value.Value).(float64)
if !ok {
return fmt.Errorf("a_name got data of type %T but wanted float64", value.Value)
}
d.AName = int(v)
case "b_name":
v, ok := (value.Value).(string)
if !ok {
return fmt.Errorf("b_name got data of type %T but wanted string", value.Value)
}
d.BName = v
}
}
return nil
}
Playground - https://go.dev/play/p/GrXKAE87d1F

Extract all JSON keys and values as map

i want to get all keys and values of json fields as type map[string][]string. The objects with same keys, should be in same map field.
I can convert JSON to map[string]interface{} now with this function, but it doesn't meet my needs.
func jsonToMap(data []byte) map[string]interface{} {
x := map[string]interface{}
json.Unmarshal(data, &x)
return x
}
Sample json:
{
"data": {
"name": "John",
"age": 30
},
"items": [
1,
2,
3
],
"data2": {
"name": "Johns",
"age": {
"test": [
1,
2,
3,
{
"test2": "123"
}
]
}
}
}
Expected result:
map[string][]string
map["data"][]string{""}
map["name"][]string{"John", "Johns"}
map["age"][]string{"30",""}
map["items"][]string{"1","2","3"}
map["test"][]string{"1","2","3"}
map["test2"][]string{"123"}
The solution is using recursive function.
https://en.wikipedia.org/wiki/Recursion_(computer_science)
Here is a simple example:
https://gobyexample.com/recursion
The following code can do what you want:
package main
import (
"encoding/json"
"fmt"
)
func main() {
JSONStr := `{
"data": {"name": "John", "age": 30},
"items": [1, 2, 3],
"data2": {
"name": "Johns",
"age": {"test": [1, 2, 3, {"test2": "123"}]}
}
}`
var JSON map[string]interface{}
json.Unmarshal([]byte(JSONStr), &JSON)
neededOutput := jsonToMap(JSON)
fmt.Println(neededOutput)
}
func jsonToMap(data map[string]interface{}) map[string][]string {
// final output
out := make(map[string][]string)
// check all keys in data
for key, value := range data {
// check if key not exist in out variable, add it
if _, ok := out[key]; !ok {
out[key] = []string{}
}
if valueA, ok := value.(map[string]interface{}); ok { // if value is map
out[key] = append(out[key], "")
for keyB, valueB := range jsonToMap(valueA) {
if _, ok := out[keyB]; !ok {
out[keyB] = []string{}
}
out[keyB] = append(out[keyB], valueB...)
}
} else if valueA, ok := value.([]interface{}); ok { // if value is array
for _, valueB := range valueA {
if valueC, ok := valueB.(map[string]interface{}); ok {
for keyD, valueD := range jsonToMap(valueC) {
if _, ok := out[keyD]; !ok {
out[keyD] = []string{}
}
out[keyD] = append(out[keyD], valueD...)
}
} else {
out[key] = append(out[key], fmt.Sprintf("%v", valueB))
}
}
} else { // if string and numbers and other ...
out[key] = append(out[key], fmt.Sprintf("%v", value))
}
}
return out
}
Output:
map[age:[30 ] data:[] data2:[] items:[1 2 3] name:[John Johns] test:[1 2 3] test2:[123]]

golang unmarshal map[string]interface{} to a struct containing an array with meta

I have the following json data coming through an API. I want to unmarshal this data into a different way of structure as it is defined below. How can I do it in an elegant way?
{
"_meta": {
"count": 2,
"total": 2
},
"0": {
"key": "key0",
"name": "name0"
},
"1": {
"key": "key1",
"name": "name1"
},
"2": {
"key": "key2",
"name": "name2"
}
// It goes on..
}
type Data struct {
Meta Meta `json:"_meta,omitempty"`
Elements []Element
}
type Element struct {
Key string
Name string
}
type Meta struct{
Count int
Total int
}
This can be quite tricky because you have a json object that holds everything. So i went with the approach of unmarshalling to map of string to *json.RawMessage and then fixing the struct from there.
To do that you will be using a custom Unmarshaler and the benefit of it is that you delay the actual parsing of the inner messages until you need them.
So for example if your meta field was wrong or the numbers it said didn't match the length of the map-1 you could exit prematurely.
package main
import (
"encoding/json"
"fmt"
)
type jdata map[string]*json.RawMessage
type data struct {
Meta Meta
Elements []Element
}
//Element is a key val assoc
type Element struct {
Key string
Name string
}
//Meta holds counts and total of elems
type Meta struct {
Count int
Total int
}
var datain = []byte(`
{
"_meta": {
"count": 2,
"total": 2
},
"0": {
"key": "key0",
"name": "name0"
},
"1": {
"key": "key1",
"name": "name1"
},
"2": {
"key": "key2",
"name": "name2"
}
}`)
func (d *data) UnmarshalJSON(buf []byte) (err error) {
var (
meta *json.RawMessage
ok bool
)
jdata := new(jdata)
if err = json.Unmarshal(buf, jdata); err != nil {
return
}
if meta, ok = (*jdata)["_meta"]; !ok {
return fmt.Errorf("_meta field not found in JSON")
}
if err = json.Unmarshal(*meta, &d.Meta); err != nil {
return
}
for k, v := range *jdata {
if k == "_meta" {
continue
}
elem := &Element{}
if err = json.Unmarshal(*v, elem); err != nil {
return err
}
d.Elements = append(d.Elements, *elem)
}
return nil
}
func main() {
data := &data{}
if err := data.UnmarshalJSON(datain); err != nil {
panic(err)
}
fmt.Printf("decoded:%v\n", data)
}

How to return custom error if validation failed with Gin [duplicate]

This question already has an answer here:
Return custom error message from struct tag validation
(1 answer)
Closed 1 year ago.
For example I've got the following struct
type Address struct {
City string `json:"city" binding:"required"`
AddressLine string `json:"address_line" binding:"required"`
}
and I've got the following function to handle request from users
func AddressCreate(c *gin.Context) {
var address Address
if err := c.BindJSON(&address); err == nil {
// if everything is good save to database
// and return success message
db.Create(&address)
c.JSON(http.StatusOK, gin.H {"status":"success"})
} else {
c.JSON(http.StatusBadRequest, err)
}
}
Expected behavior is to return JSON, formatted this way
[
{
"city":"required"
}
{
"address_line":"required"
}
]
But I'm getting an error formatted like this
"Address.City": {
"FieldNamespace": "Address.City",
"NameNamespace": "City",
"Field": "City",
"Name": "City",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
},
"Address.AddressLine": {
"FieldNamespace": "AddressLine",
"NameNamespace": "AddressLine",
"Field": "AddressLine",
"Name": "AddressLine",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
}
What I tried:
I created function which casts error to ValidationErrors and iterates through all FieldError's in it
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
errors[e.Name] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
The output look's much better
[
{
"City":"required"
},
{
"AddressLine":"required"
}
]
But I cannot solve the problem with the name of the fields. I cannot swap Name into name which I noted in structs tag json:"city". So my question is did I choose correct way to solve the problem if the answer is yes how to get structs JSON tag for field?
If you want it to be same as defined in your json tag, then you should use reflection to pull that tag from your data type.
I don't have your libraries, so can't compile and check it. But I believe what you are after should go along those lines:
func ListOfErrors(address *Address, e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
field, _ := reflect.TypeOf(address).Elem().FieldByName(e.Name)
jsonTag := string(field.Tag.Get("json"))
errors[jsonTag] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
Note that it is a bit contrived as type of address parameter is essentially known. So, not strictly required as a function parameter. But you can change address *Address to address interface{} and use it for other types too.
Disclaimer: I skipped error checking for brevity, but you certainly should check for errors in your code (e.g. no such field error or no json tag on that field).
You can use ToSnake to snake case the names:
import (
"unicode"
)
// ToSnake convert the given string to snake case following the Golang format:
// acronyms are converted to lower-case and preceded by an underscore.
func ToSnake(in string) string {
runes := []rune(in)
length := len(runes)
var out []rune
for i := 0; i < length; i++ {
if i > 0 && unicode.IsUpper(runes[i]) && ((i+1 < length && unicode.IsLower(runes[i+1])) || unicode.IsLower(runes[i-1])) {
out = append(out, '_')
}
out = append(out, unicode.ToLower(runes[i]))
}
return string(out)
}
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
invalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
errors[ToSnake(e.Name)] = e.Tag
invalidFields = append(InvalidFields, errors)
}
return invalidFields
}

Parsing nested JSON in go language

How could I parse and extract certain values from below JSON
Here is the sample JSON response
{
"success":true,
"endpoint":"https://api.abcxyz.com",
"info":{
"Guestconnected":134,
"Guestratio":100000.06963,
"symbol1":{
"code":"NY",
"symbol":"*",
"name":"newyear",
"codev":391.78161,
"symbolAppearsAfter":false,
"local":true
},
"symbol2":{
"code":"HNY",
"symbol":"#",
"name":"HappyNewYear",
"codev":1000000.0960,
"symbolAppearsAfter":true,
"local":false
},
"latest":{
"value":1597509,
"autovalue":"00099cf8da58a36c08f2ef98650ff6043ddfb",
"height":474696,
"time":1499527696
}
},
"Allguest":{
"all":4,
"filtered":4,
"total_invitations":15430,
"sent_invitations":15430,
"final_invitations":0
},
"Guestlist":[
{
"GuestCode":"369AR",
"all":2,
"total_invitations":5430,
"sent_invitations":5430,
"final_invitations":0,
"change":0,
"accounts":0
},
{
"GuestCode":"6POIA96TY",
"all":2,
"total_invitations":10000,
"sent_invitations":10000,
"final_invitations":0,
"change":0,
"accounts":0
}
]
}
My Code is :
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
type object struct {
Success bool `json:"success"`
Endpoint string `json:"endpoint"`
Allguest struct {
All int `json:"all"`
Filtered int `json:"filtered"`
TotalInvitations int `json:"total_invitations"`
SentInvitations int `json:"sent_invitations"`
FinalInvitations int `json:"final_invitations"`
} `json:"Allguest"`
Guestlist []struct {
GuestCode string `json:"GuestCode"`
All int `json:"all"`
TotalInvitations int `json:"total_invitations"`
SentInvitations int `json:"sent_invitations"`
FinalInvitations int `json:"final_invitations"`
Change int `json:"change"`
Accounts int `json:"accounts"`
} `json:"Guestlist"`
}
func main() {
uri := "https://siteurl.com/api?lists=1"
res, err := http.Get(uri)
fmt.Println(uri)
if err != nil {
fmt.Println("Error:", err)
log.Fatal(err)
}
defer res.Body.Close()
var s object
err := json.NewDecoder(res.Body).Decode(&s)
if err != nil {
log.Fatal(err)
fmt.Println("Error:", err)
}
fmt.Println(s.Success)
fmt.Println(s.Allguest.TotalInvitations)
for i := 0; i < 6; i++ {
fmt.Println(s.Guestlist[i].TotalInvitations)
)
}
The problem is :
If the response is null it is giving index out of range error how can we avoid this ?
Applying condition if TotalInvitations value is greater than 100 then save in 100.csv else save in others.csv
If you need only certain entries from JSON you can define a structure with only those fields you are interested in. And if value can be null it is better to declare a pointer in the structure. Please take a look at an example at go playground: https://play.golang.org/p/mEwSXvPg3D
use gjson pkg to get value from json document an simple way
example:
package main
import (
"fmt"
"github.com/tidwall/gjson"
)
func main() {
data := []byte(`{
"success": true,
"endpoint": "https://api.abcxyz.com",
"info": {
"Guestconnected": 134,
"Guestratio": 100000.06963,
"symbol1": {
"code": "NY",
"symbol": "*",
"name": "newyear",
"codev": 391.78161,
"symbolAppearsAfter": false,
"local": true
},
"symbol2": {
"code": "HNY",
"symbol": "#",
"name": "HappyNewYear",
"codev": 1000000.096,
"symbolAppearsAfter": true,
"local": false
},
"latest": {
"value": 1597509,
"autovalue": "00099cf8da58a36c08f2ef98650ff6043ddfb",
"height": 474696,
"time": 1499527696
}
},
"Allguest": {
"all": 4,
"filtered": 4,
"total_invitations": 15430,
"sent_invitations": 15430,
"final_invitations": 0
},
"Guestlist": [
{
"GuestCode": "369AR",
"all": 2,
"total_invitations": 5430,
"sent_invitations": 5430,
"final_invitations": 0,
"change": 0,
"accounts": 0
},
{
"GuestCode": "6POIA96TY",
"all": 2,
"total_invitations": 10000,
"sent_invitations": 10000,
"final_invitations": 0,
"change": 0,
"accounts": 0
}
]
}
`)
r := gjson.GetBytes(data, "Allguest.total_invitations")
fmt.Println(r.Value()) // output 15430
r = gjson.GetBytes(data, "Guestlist.#.total_invitations")
fmt.Println(r.Value()) // output [5430 10000]
}
If the response is null it is giving index out of range error how can
we avoid this?
Unless I'm misunderstanding you, the response being null shouldn't matter. If you receive 2 Guestlist items, you can only loop over those 2, meaning for i := 0; i < 6; i++ is wrong. Change it to for i := 0; i < len(s.Guestlist); i++ or for i := range s.Guestlist.
Applying condition if TotalInvitations value is greater than 100 then
save in 100.csv else save in others.csv
You will probably want the os package's Create function to create a new file for writing or the OpenFile function, the O_CREATE, O_WRONLY, and O_APPEND file-open flags, and you can pass 0755 as the permission mode (or another set of octal permissions if you want to change access of a file).
You can then use csvWriter := csv.NewWriter(csvFile) to wrap the resulting io.Writer (assuming there is no error opening the file). From there, you can just write whatever information you require to the file as CSV records. Don't forget to call csvWriter.Flush to flush the output buffer and csvWriter.Error to check whether there was an error flushing the output buffer. Also don't forget to close the file you opened.

Resources