Unmarshalling a complicated JSON ad bin it with a struct - go

I have the following JSOn response from a webhook call
{
"responseId": "d5c70d8b-e8ad-41df-bb3b-26b0e51d60ca-a14fa99c",
"queryResult": {
"queryText": "1111111111",
"parameters": {
"phone-number": "1111111111"
},
"allRequiredParamsPresent": true,
"fulfillmentText": "Thats great! You payment link has been sent to Gaf ( Mobile number 1111111111 )",
"fulfillmentMessages": [{
"text": {
"text": ["Thats great! You payment link has been sent to Far ( Mobile number 1111111111 )"]
}
}],
"outputContexts": [{
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name",
"lifespanCount": 2,
"parameters": {
"name": ["Gar"],
"name.original": ["Gar"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_number",
"lifespanCount": 4,
"parameters": {
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name_confirm",
"lifespanCount": 3,
"parameters": {
"name": ["Gaf"],
"name.original": ["Far"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/__system_counters__",
"parameters": {
"no-input": 0.0,
"no-match": 0.0,
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}],
"intent": {
"name": "projects/open-prod-bot-pfgibi/agent/intents/d21f7be5-0f77-4cb8-9857-26ba04964317",
"displayName": "GetMobileNumber"
},
"intentDetectionConfidence": 1.0,
"languageCode": "en"
},
"originalDetectIntentRequest": {
"payload": {
}
},
"session": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829"
}
I wanted to extract out
"outputContexts": [{
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name",
"lifespanCount": 2,
"parameters": {
"name": ["Gar"],
"name.original": ["Gar"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}
out of this and bind this with a struct. But I couldn't do it. I am trying to loop it through the map as below
var f interface{}
json.Unmarshal(b, &f)
for k, v := range f.(map[string]interface{}) {
if k == "queryResultmap" {
fmt.Println(v)
}
}
but not working. I am new to Go. Tried few examples in google but since this one is a complicated JSON I am unable to do it. Please help

I would suggest you declare a struct with the fields you care about and unmarshal into that, but if you want to stick to interface{} try this:
m := f.(map[string]interface{})
r := m["queryResult"].(map[string]interface{})
fmt.Println(r["outputContext"])
https://play.golang.com/p/gA5wWcPyd6E
Using struct:
type OutputContext struct {
Name string `json:"name"`
LifespanCount int `json:"lifespanCount"`
Parameters struct {
Name []string `json:"name"`
NameOriginal []string `json:"name.original"`
PhoneNumber string `json:"phone-number"`
PhoneNumberOriginal string `json:"phone-number.original"`
NoInput float64 `json:"no-input"`
NoMatch float64 `json:"no-match"`
} `json:"parameters"`
}
type QueryResult struct {
OutputContexts []OutputContext `json:"outputContexts"`
}
// ...
var dest struct {
QueryResult QueryResult `json:"queryResult"`
}
if err := json.Unmarshal(data, &dest); err != nil {
panic(err)
}
for _, v := range dest.QueryResult.OutputContexts {
fmt.Printf("%+v\n", v)
}
https://play.golang.com/p/EfIugDQJ651

Related

When the first stage is array, How to handle with go-simplejson

JSON struct like below:
[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}
]
I don't know how to get the location 0's data.
content, _ := simplejson.NewJson(body)
arr, _ := content.Array() // Here can get the all data, It's []interface{} type.
I cannot get the next data with arr[0]["sha"]. How to handle it?
It is not clear to the compiler that arr is an array of map[string]interface{} at compile time, as arr[0] is of type interface{}. This basically means that the compiler knows nothing about this type, which is why you can't do a map lookup operation here.
You can add a type assertion to make sure you can use it as a map like this:
asMap := arr[0].(map[string]interface{})
fmt.Println(asMap["sha"])
To get the SHA as string, you can again add a type assertion behind it as well:
asString := asMap["sha"].(string)
This is also shown in this working example. The downside of this is that your program will panic in case the given data is not of the specified type. You could instead use a type assertion with a check if it worked (asString, ok := ...), but it gets cumbersome with more complex data.
This does work, but isn't really nice. I would recommend using a tool like this to generate Go structs and then use them in a type-safe way. First define a struct with all the info you need:
type ArrayElement struct {
Sha string `json:"sha"`
// Add more fields if you need them
}
Then you can just use the standard-library json package to unmarshal your data:
// This should have the same structure as the data you want to parse
var result []ArrayElement
err := json.Unmarshal([]byte(str), &result)
if err != nil {
panic(err)
}
fmt.Println(result[0].Sha)
Here is an example for that -- this is a more Go-like approach for converting JSON data.
Your json data is wrong formatted. First of all, remove , after "id": "xxxxx", line:
...
"id": "xxxxx"
...
You should check errors after NewJson to prevent find out if there is a problem:
content, err := simplejson.NewJson(body)
if err != nil {
// log err
}
For getting sha from first index, you simply can use simplejson built-in methods:
shaVal := content.GetIndex(0).Get("sha").String()
Here is how you can get the desired value.
This worked for me.
package main
import (
"encoding/json"
"fmt"
)
type MyData []struct {
Sha string `json:"sha"`
NodeID string `json:"node_id"`
Commit struct {
Author struct {
Name string `json:"name"`
} `json:"author"`
Committer struct {
Name string `json:"name"`
} `json:"committer"`
Message string `json:"message"`
Tree struct {
Sha string `json:"sha"`
URL string `json:"url"`
} `json:"tree"`
URL string `json:"url"`
CommentCount int `json:"comment_count"`
Verification struct {
Verified bool `json:"verified"`
Reason string `json:"reason"`
Signature interface{} `json:"signature"`
Payload interface{} `json:"payload"`
} `json:"verification"`
} `json:"commit"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
CommentsURL string `json:"comments_url"`
Author struct {
Login string `json:"login"`
ID string `json:"id"`
NodeID string `json:"node_id"`
AvatarURL string `json:"avatar_url"`
GravatarID string `json:"gravatar_id"`
Type string `json:"type"`
SiteAdmin bool `json:"site_admin"`
} `json:"author"`
Committer struct {
Login string `json:"login"`
ID string `json:"id"`
} `json:"committer"`
Parents []struct {
Sha string `json:"sha"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
} `json:"parents"`
}
func main() {
my_json_data := `[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}]`
var data MyData
err := json.Unmarshal([]byte(my_json_data), &data)
if err != nil {
panic(err)
}
fmt.Println("data --> sha: ", data[0].Sha)
}

GORM .Save don't save "has one" relation to the database

I have struct:
type Book struct {
gorm.Model
Title string `json:"title"`
Author string `json:"author"`
Description string `json:"description"`
Category string `json:"Category"`
Publisher string `json:"publisher"`
AuthorsCard AuthorsCard `gorm:"foreignKey:BookID" json:"authorscard"`
}
type AuthorsCard struct {
//gorm.Model // I purposely Don't want to use gorm.model
ID uint `gorm:"primarykey"`
BookID uint
Name string `json:"name"`
Age int `json:"age"`
YearOfBirth int `json:"year"`
Biography string `json:"biography"`
}
And I'm trying to make update function:
// controller.go
func UpdateBook(ctx *gin.Context) {
enableCors(&ctx.Writer)
id := ctx.Param("ID")
var updateBook = &models.Book{}
if err := ctx.BindJSON(updateBook); err != nil {
ctx.AbortWithStatus(http.StatusBadRequest)
log.Fatal(err)
} else {
repository.UpdateBook(updateBook, id)
ctx.JSON(http.StatusOK, updateBook)
log.Println(updateBook)
}
}
//repository.go
func UpdateBook(updateBook *models.Book, ID string) {
book, db := GetBookById(ID)
if updateBook.Title != "" {
book.Title = updateBook.Title
}
if updateBook.Author != "" {
book.Author = updateBook.Author
}
if updateBook.Publisher != "" {
book.Publisher = updateBook.Publisher
}
if updateBook.Description != "" {
book.Description = updateBook.Description
}
if updateBook.Category != "" {
book.Category = updateBook.Category
}
if updateBook.AuthorsCard.Name != "" {
book.AuthorsCard.Name = updateBook.AuthorsCard.Name
}
if updateBook.AuthorsCard.Age != 0 {
book.AuthorsCard.Age = updateBook.AuthorsCard.Age
}
if updateBook.AuthorsCard.YearOfBirth != 0 {
book.AuthorsCard.YearOfBirth = updateBook.AuthorsCard.YearOfBirth
}
if updateBook.AuthorsCard.Biography != "" {
book.AuthorsCard.Biography = updateBook.AuthorsCard.Biography
}
db.Save(&book)
// same with db.Preload("AuthorsCard").Save(&book)
}
The issue is: When I make an PUT request, I receive Fully updated data. And when I'm trying to make GET request all my fields, except related AuthorsCard, are been updated.
PUT response:
200 Code
{
"ID": 0,
"CreatedAt": "0001-01-01T00:00:00Z",
"UpdatedAt": "0001-01-01T00:00:00Z",
"DeletedAt": null,
"title": "Test",
"author": "author",
"description": "something",
"Category": "Category",
"publisher": "PB",
"authorscard": {
"ID": 0,
"BookID": 0,
"name": "Updated",
"age": 22,
"year": 1999,
"biography": "biography Updated"
}
}
Get response after that:
[
{
"ID": 1,
"CreatedAt": "2022-06-29T14:57:37.489639+03:00",
"UpdatedAt": "2022-06-29T15:50:11.578724+03:00",
"DeletedAt": null,
"title": "Test",
"author": "author",
"description": "something",
"Category": "Category",
"publisher": "PB",
"authorscard": {
"ID": 1,
"BookID": 1,
"name": "test",
"age": 23,
"year": 1999,
"biography": "23fdgsdddTEST"
}
}
]
As you can see "authorscard" hasn't changed. Please, can someone tell me what I'm doing wrong?
You need to explicitly tell Gorm to store/update associations.
db.Session(&gorm.Session{FullSaveAssociations: true}).Updates(&book)

How to replace json values from key-value map in GOlang

I have below json & I need to replace employee_names to emp_id from a map.
Tried this GolangPlay , but not sure how to replace values from the given map and error handling if the value is not present in given map.
Json data:
[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]
Map data:
[{emp1 14325} {emp3 49184} {emp2 21518}]
Expected output:
[
{
"dept": "IT",
"condition": {
"employee": [
14325
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
21518,
49184
]
}
}
]
code :
Started with below code , but not sure how to use the given map to replace with error handling.
package main
import (
"encoding/json"
"fmt"
//"strconv"
//"log"
)
func main() {
jsonStr := `[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]`
empMap := `[{emp1 14325} {emp3 49184} {emp2 21518}]`
type GetEmployee []struct {
Dept string `json:"dept"`
Condition struct {
Employee []string `json:"employee"`
} `json:"condition"`
}
var empResponse GetEmployee
unmarshallingError := json.Unmarshal([]byte(string(jsonStr)), &empResponse)
if unmarshallingError != nil {
fmt.Println(unmarshallingError.Error())
}
fmt.Println(empResponse)
fmt.Println(empMap)
for i := range empResponse {
fmt.Println(i)
}
}
Instead of storing the ids in an array of {ids: value}, it will be better for them to be in a map instead.
The above will range over the Employees' name and change it to the id. A check is made to see if there is a certain id key in the map.
for i, e := range empResponse {
fmt.Println(e)
for j,val := range empResponse[i].Condition.Employee {
if _, ok := ids[val]; ok {
empResponse[i].Condition.Employee[j] = ids[val]
}
}
}
Full code
package main
import (
"encoding/json"
"fmt"
//"strconv"
//"log"
)
func main() {
jsonStr := `[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]`
empMap := `{"emp1": "14325", "emp3": "49184", "emp2": "21518"}`
type GetEmployee []struct {
Dept string `json:"dept"`
Condition struct {
Employee []string `json:"employee"`
} `json:"condition"`
}
var empResponse GetEmployee
var ids map[string]string
unmarshallingError := json.Unmarshal([]byte(string(jsonStr)), &empResponse)
if unmarshallingError != nil {
fmt.Println(unmarshallingError.Error())
}
json.Unmarshal([]byte(empMap), &ids)
fmt.Println(empResponse)
fmt.Println(ids)
for i, e := range empResponse {
fmt.Println(e)
for j,val := range empResponse[i].Condition.Employee {
if _, ok := ids[val]; ok {
empResponse[i].Condition.Employee[j] = ids[val]
}
}
}
fmt.Println(empResponse)
}
playground
In the above, the id are strings since the name to be replaced are strings. Actually Employee are of type []string. If a string is to be replaced by an int, then Employee type needs to be changed to []interface{}.
playground

Unmarshalling nested json object from http request returns nil

I've been going through other similar questions here but I don't know what I'm doing wrong.
I am calling this API:
https://coronavirus-tracker-api.herokuapp.com/v2/locations
Which returns a JSON object like this one:
{
"latest": {
"confirmed": 272166,
"deaths": 11299,
"recovered": 87256
},
"locations": [
{
"id": 0,
"country": "Thailand",
"country_code": "TH",
"province": "",
"last_updated": "2020-03-21T06:59:11.315422Z",
"coordinates": {
"latitude": "15",
"longitude": "101"
},
"latest": {
"confirmed": 177,
"deaths": 1,
"recovered": 41
}
},
{
"id": 39,
"country": "Norway",
"country_code": "NO",
"province": "",
"last_updated": "2020-03-21T06:59:11.315422Z",
"coordinates": {
"latitude": "60.472",
"longitude": "8.4689"
},
"latest": {
"confirmed": 1463,
"deaths": 3,
"recovered": 1
}
}
]
}
So I have written a small program to parse it but I can only parse the outer object ("latest") while the inner array ("locations") always returns nil.
Code is here (even if TCP calls don't work on the playground):
https://play.golang.org/p/ma225d07iRA
and here:
package main
import (
"encoding/json"
"fmt"
"net/http"
"time"
)
type AutoGenerated struct {
Latest Latest `json:"latest"`
Locations []Locations `json:"locations"`
}
type Latest struct {
Confirmed int `json:"confirmed"`
Deaths int `json:"deaths"`
Recovered int `json:"recovered"`
}
type Coordinates struct {
Latitude string `json:"latitude"`
Longitude string `json:"longitude"`
}
type Locations struct {
ID int `json:"id"`
Country string `json:"country"`
CountryCode string `json:"country_code"`
Province string `json:"province"`
LastUpdated time.Time `json:"last_updated"`
Coordinates Coordinates `json:"coordinates"`
Latest Latest `json:"latest"`
}
var latestUrl = "https://coronavirus-tracker-api.herokuapp.com/v2/latest"
func getJson(url string, target interface{}) {
req, err := http.NewRequest("GET", url, nil)
if err != nil {
fmt.Println(err)
}
req.Header.Add("content-type", "application/json")
res, err := http.DefaultClient.Do(req)
if err != nil {
fmt.Println(err)
}
decoder := json.NewDecoder(res.Body)
var data AutoGenerated
err = decoder.Decode(&data)
if err != nil {
fmt.Println(err)
}
for i, loc := range data.Locations {
fmt.Printf("%d: %s", i, loc.Country)
}
defer res.Body.Close()
}
func main() {
var a AutoGenerated
getJson(latestUrl, &a)
}
The problem is that the endpoint https://coronavirus-tracker-api.herokuapp.com/v2/latest does not return locations. This is the response I get by calling it:
{
"latest": {
"confirmed": 304524,
"deaths": 12973,
"recovered": 91499
}
}
However if you call the correct endpoint https://coronavirus-tracker-api.herokuapp.com/v2/locations, it might work.

Protocol buffer serialization Golang

I am using DialogFlow V2 official GoLang SDK. In my webhook, I am returning a payload, which I'm obtaining using the function GetWebhookPayload().
This returns *google_protobuf4.Struct. I would like to turn this struct into a map[string]interface{}. How is this possible?
This is what the struct looks like when serialized:
"payload": {
"fields": {
"messages": {
"Kind": {
"ListValue": {
"values": [
{
"Kind": {
"StructValue": {
"fields": {
"title": {
"Kind": {
"StringValue": "Hi! How can I help?"
}
},
"type": {
"Kind": {
"StringValue": "message"
}
}
}
}
}
}
]
}
}
}
}
What I essentially need is for it to be serialized as such:
"payload": {
"messages": [
{
"title": "Hi! How can I help?",
"type": "message"
}
]
}
This can be solved using jsonpb.
package main
import (
"bytes"
"encoding/json"
"github.com/golang/protobuf/jsonpb"
)
func main() {
...
payload := qr.GetWebhookPayload()
b, marshaler := bytes.Buffer{}, jsonpb.Marshaler{}
if err := marshaler.Marshal(&b, payload.GetFields()["messages"]); err != nil {
// handle err
}
msgs := []interface{}{}
if err := json.Unmarshal(b.Bytes(), &msgs); err != nil {
// handle err
}
// msgs now populated
}

Resources