Unmarshal unknown JSON fields without structs - go

I am trying to unmarshal a JSON object which has an optional array, I am doing this without an array and this is what I got so far:
import (
"encoding/json"
"fmt"
)
func main() {
jo := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
var raw map[string]interface{}
err := json.Unmarshal([]byte(jo), &raw)
if err != nil {
panic(err)
}
fmt.Println(raw["emails"][0])
}
The emails field might or might not come sometime. I know I can use struct and unmarshal it twice for with and without array. When I try to get the index 0 of raw["emails"][0] I get the following error
invalid operation: raw["emails"][0] (type interface {} does not support indexing)
Is there a way to get the index of the emails field?
Update 1
I can do something like this fmt.Println(raw["emails"].([]interface{})[0]) and it works. Is this the only way?

The easiest way is with a struct. There's no need to unmarshal twice.
type MyStruct struct {
// ... other fields
Emails []string `json:"emails"`
}
This will work, regardless of whether the JSON input contains the emails field. When it is missing, your resulting struct will just have an uninitialized Emails field.

You can use type assertions. The Go tutorial on type assertions is here.
A Go playground link applying type assertions to your problem is here. For ease of reading, that code is replicated below:
package main
import (
"encoding/json"
"fmt"
)
func main() {
jo := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
var raw map[string]interface{}
err := json.Unmarshal([]byte(jo), &raw)
if err != nil {
panic(err)
}
emails, ok := raw["emails"]
if !ok {
panic("do this when no 'emails' key")
}
emailsSlice, ok := emails.([]interface{})
if !ok {
panic("do this when 'emails' value is not a slice")
}
if len(emailsSlice) == 0 {
panic("do this when 'emails' slice is empty")
}
email, ok := (emailsSlice[0]).(string)
if !ok {
panic("do this when 'emails' slice contains non-string")
}
fmt.Println(email)
}

As always you can use additional libraries for work with your json data. For example with gojsonq package it will like so:
package main
import (
"fmt"
"github.com/thedevsaddam/gojsonq"
)
func main() {
json := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
first := gojsonq.New().JSONString(json).Find("emails.[0]")
if first != nil {
fmt.Println(first.(string))
} else {
fmt.Println("There isn't emails")
}
}

Related

Golang REST api request with token auth to json array reponse

EDIT
This is the working code incase someone finds it useful. The title to this question was originally
"How to parse a list fo dicts in golang".
This is title is incorrect because I was referencing terms I'm familiar with in python.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
//Regional Strut
type Region []struct {
Region string `json:"region"`
Description string `json:"Description"`
ID int `json:"Id"`
Name string `json:"Name"`
Status int `json:"Status"`
Nodes []struct {
NodeID int `json:"NodeId"`
Code string `json:"Code"`
Continent string `json:"Continent"`
City string `json:"City"`
} `json:"Nodes"`
}
//working request and response
func main() {
url := "https://api.geo.com"
// Create a Bearer string by appending string access token
var bearer = "TOK:" + "TOKEN"
// Create a new request using http
req, err := http.NewRequest("GET", url, nil)
// add authorization header to the req
req.Header.Add("Authorization", bearer)
//This is what the response from the API looks like
//regionJson := `[{"region":"GEO:ABC","Description":"ABCLand","Id":1,"Name":"ABCLand [GEO-ABC]","Status":1,"Nodes":[{"NodeId":17,"Code":"LAX","Continent":"North America","City":"Los Angeles"},{"NodeId":18,"Code":"LBC","Continent":"North America","City":"Long Beach"}]},{"region":"GEO:DEF","Description":"DEFLand","Id":2,"Name":"DEFLand","Status":1,"Nodes":[{"NodeId":15,"Code":"NRT","Continent":"Asia","City":"Narita"},{"NodeId":31,"Code":"TYO","Continent":"Asia","City":"Tokyo"}]}]`
//Send req using http Client
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Println("Error on response.\n[ERROR] -", err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Println("Error while reading the response bytes:", err)
}
var regions []Region
json.Unmarshal([]byte(body), &regions)
fmt.Printf("Regions: %+v", regions)
}
Have a look at this playground example for some pointers.
Here's the code:
package main
import (
"encoding/json"
"log"
)
func main() {
b := []byte(`
[
{"key": "value", "key2": "value2"},
{"key": "value", "key2": "value2"}
]`)
var mm []map[string]string
if err := json.Unmarshal(b, &mm); err != nil {
log.Fatal(err)
}
for _, m := range mm {
for k, v := range m {
log.Printf("%s [%s]", k, v)
}
}
}
I reformatted the API response you included because it is not valid JSON.
In Go it's necessary to define types to match the JSON schema.
I don't know why the API appends % to the end of the result so I've ignored that. If it is included, you will need to trim the results from the file before unmarshaling.
What you get from the unmarshaling is a slice of maps. Then, you can iterate over the slice to get each map and then iterate over each map to extract the keys and values.
Update
In your updated question, you include a different JSON schema and this change must be reflect in the Go code by update the types. There are some other errors in your code. Per my comment, I encourage you to spend some time learning the language.
package main
import (
"bytes"
"encoding/json"
"io/ioutil"
"log"
)
// Response is a type that represents the API response
type Response []Record
// Record is a type that represents the individual records
// The name Record is arbitrary as it is unnamed in the response
// Golang supports struct tags to map the JSON properties
// e.g. JSON "region" maps to a Golang field "Region"
type Record struct {
Region string `json:"region"`
Description string `json:"description"`
ID int `json:"id"`
Nodes []Node
}
type Node struct {
NodeID int `json:"NodeId`
Code string `json:"Code"`
}
func main() {
// A slice of byte representing your example response
b := []byte(`[{
"region": "GEO:ABC",
"Description": "ABCLand",
"Id": 1,
"Name": "ABCLand [GEO-ABC]",
"Status": 1,
"Nodes": [{
"NodeId": 17,
"Code": "LAX",
"Continent": "North America",
"City": "Los Angeles"
}, {
"NodeId": 18,
"Code": "LBC",
"Continent": "North America",
"City": "Long Beach"
}]
}, {
"region": "GEO:DEF",
"Description": "DEFLand",
"Id": 2,
"Name": "DEFLand",
"Status": 1,
"Nodes": [{
"NodeId": 15,
"Code": "NRT",
"Continent": "Asia",
"City": "Narita"
}, {
"NodeId": 31,
"Code": "TYO",
"Continent": "Asia",
"City": "Tokyo"
}]
}]`)
// To more closely match your code, create a Reader
rdr := bytes.NewReader(b)
// This matches your code, read from the Reader
body, err := ioutil.ReadAll(rdr)
if err != nil {
// Use Printf to format strings
log.Printf("Error while reading the response bytes\n%s", err)
}
// Initialize a variable of type Response
resp := &Response{}
// Try unmarshaling the body into it
if err := json.Unmarshal(body, resp); err != nil {
log.Fatal(err)
}
// Print the result
log.Printf("%+v", resp)
}

Consume a nested array inside JSON request in Go

How do I go about consuming the following json post in Go?
{
"notificationType": "email",
"client": "The CLient",
"content": "Hellow World",
"recipients": [
{
"address": "email1#example.com"
},
{
"address": "email2#example.com"
}
]
}
I've managed to get the string types but I really don't know enough about Go to deal with the recipients array.
My code looks like this:
package handlers
import (
"net/http"
"encoding/json"
"notificationservice/src/validation"
"io/ioutil"
"fmt"
)
func EmailHandler(w http.ResponseWriter, r *http.Request) {
b, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
var postData validation.EmailValidator
err = json.Unmarshal(b, &postData)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
fmt.Println(postData.Client)
fmt.Println(postData.Content)
}
My struct:
package validation
import "fmt"
type EmailValidator struct {
Client string `json:"client"`
Content string `json:"content"`
//Recipients []string `json:"recipients"`
}
func (validator * EmailValidator) ValidateEmail() (bool) {
var required = []string {validator.Client, validator.Content, validator.Stuff}
for _, param := range required {
fmt.Println(param)
}
return true;
}
I've tried setting Recipients to []string and [][]string but I really don't know what I'm doing.
In PHP I would use the var_dump command to print out the entire object and debug step by step from there, but Go doesn't appear to have that functionality.
You can try something like this:
package main
import (
"encoding/json"
"fmt"
)
type Email struct {
Adress string `json:"address"`
}
type EmailValidator struct {
Client string `json:"client"`
Content string `json:"content"`
Recipients []Email `json:"recipients"`
}
func main() {
j := `{
"notificationType": "email",
"client": "The CLient",
"content": "Hellow World",
"recipients": [
{
"address": "email1#example.com"
},
{
"address": "email2#example.com"
}
]
}`
result := EmailValidator{}
json.Unmarshal([]byte(j), &result)
fmt.Printf("%+v", result) // something like var_dump in PHP
}
You need an array of "things with an address":
type EmailValidator struct {
Client string `json:"client"`
Content string `json:"content"`
Recipients []Recipient `json:"recipients"`
}
type Recipient struct {
Address string `json:"address"`
}
You can nest objects, and Unmarshal will handle the entire tree for you.
type Recipient struct {
Address string `json:"address"`
}
type EmailValidator struct {
Client string `json:"client"`
Content string `json:"content"`
Recipients []Recipient `json:"recipients"`
}
The rest of your code looks good.

Recieving a EOF Panic error

I'm trying to decode a json I get. Here's an example json I get:
{"response":"1","number":"1234","id":nil}
Here's my struct:
type AutoGenerated struct {
Response string `json:"response"`
Number string `json:"number"`
ID interface{} `json:"id"`
}
I use the decode function in encode/json. What Am I getting wrong? ID has the chance to be both a string or a nil value.
Here's me exact error incase it helps.
panic: EOF
Without you showing how you're doing it, I think the best answer is to show you how to do it.
package main
import (
"fmt"
"log"
"encoding/json"
)
func main() {
j := []byte(`{"response":"1","number":"1234","id":null}`)
data := AutoGenerated{}
err := json.Unmarshal(j, &data)
if err != nil {
log.Println(err.Error())
return
}
fmt.Println(data)
}
type AutoGenerated struct {
Response string `json:"response"`
Number string `json:"number"`
ID interface{} `json:"id"`
}
The JSON string you put here is invalid. You can find this code sample for reference.
If you're going to set the id field to nil, just don't put it in the JSON string.
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strings"
)
type AutoGenerated struct {
Response string `json:"response"`
Number string `json:"number"`
ID interface{} `json:"id"`
}
func main() {
jsonStream := `
{ "response": "1", "number": "1234" }
{ "response": "1", "number": "1234", "id": "nil" }
`
decoder := json.NewDecoder(strings.NewReader(jsonStream))
for {
var m AutoGenerated
if err := decoder.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Println(m)
}
}
The output of the program is:
{1 1234 <nil>}
{1 1234 nil}

Golang Json Not returning expected values

I have a code # http://play.golang.org/p/HDlJJ54YqW
I wanted to print the Phone and email of a person.
It can be of multiple entries.
But getting the error undefined.
Can anyone help out.
Thanks.
Small details: you are referencing twice: you give the address of the address of the object to json.Unmarshal. Just give the address.
` allows for multiline, no need to split your json input.
I don't know what you where trying to achieve with u.Details[Phone:"1111"].Email, but this is no Go syntax. your Details member is a slice off Detail. A slice is similar to an array and can be accessed by index.
Also, your json does not match your object structure. If you want to have multiple details in one content, then it needs to be embed in an array ([ ])
You could do something like this: (http://play.golang.org/p/OP1zbPW_wk)
package main
import (
"encoding/json"
"fmt"
)
type Content struct {
Owner string
Details []*Detail
}
type Detail struct {
Phone string
Email string
}
func (c *Content) SearchPhone(phone string) *Detail {
for _, elem := range c.Details {
if elem.Phone == phone {
return elem
}
}
return nil
}
func (c *Content) SearchEmail(email string) *Detail {
for _, elem := range c.Details {
if elem.Email == email {
return elem
}
}
return nil
}
func main() {
encoded := `{
"Owner": "abc",
"Details": [
{
"Phone": "1111",
"Email": "#gmail"
},
{
"Phone": "2222",
"Email": "#yahoo"
}
]
}`
// Decode the json object
u := &Content{}
if err := json.Unmarshal([]byte(encoded), u); err != nil {
panic(err)
}
// Print out Email and Phone
fmt.Printf("Email: %s\n", u.SearchPhone("1111").Email)
fmt.Printf("Phone: %s\n", u.SearchEmail("#yahoo").Phone)
}

Traversing a Golang Map

I initialized a variable named data like this:
var data interface{}
Then I unmarshalled raw json into.
err = json.Unmarshal(raw, &data)
I've run these two functions on it:
fmt.Println(reflect.TypeOf(data))
fmt.Println(data)
and those return this:
map[string]interface {}
map[tasks:[map[payload:map[key:36A6D454-FEEE-46EB-9D64-A85ABEABBCB7] code_name:image_resize]]]
and I need to access the "key". I have tried these approaches and a few more:
data["tasks"][0]["payload"]["key"]
data[0][0][0][0]
Those have all given me an error similar to this one:
./resize.go:44: invalid operation: data["tasks"] (index of type interface {})
Any advice on how to grab the "key" value out of this interface? Thanks in advance.
Since you already know your schema, the best way to do this is to unmarshal directly into structs you can use.
http://play.golang.org/p/aInZp8IZQA
package main
import (
"encoding/json"
"fmt"
)
type msg struct {
Tasks []task `json:"tasks"`
}
type task struct {
CodeName string `json:"code_name"`
Payload payload `json:"payload"`
}
type payload struct {
Key string `json:"key"`
}
func main() {
var data msg
raw := `{ "tasks": [ { "code_name": "image_resize", "payload": { "key": "36A6D454-FEEE-46EB-9D64-A85ABEABBCB7" } } ] }`
err := json.Unmarshal([]byte(raw), &data)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(data.Tasks[0].Payload.Key)
}
If you insist on doing things the hard way using your original code, you need to do type assertions. I highly recommend avoiding this route when possible. It is not fun. Every step needs to be checked to ensure it matches the data structure you expect.
http://play.golang.org/p/fI5sqKV19J
if m, ok := data.(map[string]interface{}); ok {
if a, ok := m["tasks"].([]interface{}); ok && len(a) > 0 {
if e, ok := a[0].(map[string]interface{}); ok {
if p, ok := e["payload"].(map[string]interface{}); ok {
if k, ok := p["key"].(string); ok {
fmt.Println("The key is:", k)
}
}
}
}
}
In response to Goodwine's question: You can read further about how to marshal and unmarshal by reading the encoding/json godoc. I suggest starting here:
http://golang.org/pkg/encoding/json/#Marshal
http://golang.org/pkg/encoding/json/#Unmarshal
Another solution is to use 3rd-party package like https://github.com/Jeffail/gabs
With gabs, The example above by #stephen-weinberg can be written as:
raw := `{ "tasks": [ { "code_name": "image_resize", "payload": { "key": "36A6D454-FEEE-46EB-9D64-A85ABEABBCB7" } } ] }`
j, _ := gabs.ParseJSON([]byte(raw))
fmt.Println("The key is", j.S("tasks").Index(0).S("payload", "key").Data().(string))
other popular json handling packages I've stumbled upon are: https://github.com/bitly/go-simplejson and https://github.com/antonholmquist/jason.

Resources