How to populate an arbitrary struct in Go? - go

I want to create a function (in the example code called gremlinResultToStruct()) that takes an arbitrary struct and populates the values from a database query. Please note that the "Properties" key in the GremlinResultStruct are arbitrary too (that is why it is a Properties map[string][]struct)
GOAL:
I want gremlinResultToStruct() to populate the given struct based on its keys if they are available in the database results.
WHY:
I have hundreds of structs like Foo and Bar in my codebase and I want to have a single function that populates the struct needed in the code instead of creating semi-duplicate code.
CODE:
My code looks like this, it also contains the question:
// Gremlin Query result struct
type GremlinResult struct {
AtType string `json:"#type"`
AtValue []struct {
AtType string `json:"#type"`
AtValue struct {
ID struct {
AtType string `json:"#type"`
AtValue int `json:"#value"`
} `json:"id"`
Label string `json:"label"`
Properties map[string][]struct { // <== IMPORTANT, CONTAINS A VALUE THAT IS THE EQUIVALENT OF THE KEY IN THE STRUCT (like Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse)
AtType string `json:"#type"`
AtValue struct {
ID struct {
AtType string `json:"#type"`
AtValue int `json:"#value"`
} `json:"id"`
Label string `json:"label"`
Value string `json:"value"`
} `json:"#value"`
} `json:"#properties"`
} `json:"#value"`
} `json:"#value"`
}
type Foo struct {
Something string
SomethingElse bool
}
type Bar struct {
EvenMoreSomethingElse string
AndMoreSomethingElse int
}
func gremlinResultToStruct(result *GremlinResult, ???){ // What to do at ???
// How to itterate over the ??? key values (like: Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse)
// How to populate:
// VARIABLE := Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse
// result.AtValue[0].AtValue.Properties[ VARIABLE ][0].AtValue.Value
}
func main {
// Do stuff to get JSON results. HOW is not relevant for question
dbResults := doDbStuffToPopulateResuts()
// Create the results variable as gremlinResults struct
results := GremlinResult{}
// Unmarshall to this struct
err = json.Unmarshal(dbResults, &results)
// 1. Expected result should be "result-a" (see below)
// Create the Foo struct
foo := Foo{}
// HERE IS MY QUESTION ABOUT (see function above)
gremlinResultToStruct(&results, &foo)
// NOW foo SHOULD HAVE Something and SomethingElse set
fmt.Println(foo.Something, foo.SomethingElse)
// 2. Expected result should be "result-b" (see below)
// Create the Bar struct
bar := Bar{}
// HERE IS MY QUESTION ABOUT (see function above)
gremlinResultToStruct(&results, &bar)
// NOW foo SHOULD HAVE Something and SomethingElse set
fmt.Println(bar.EvenMoreSomethingElse, bar.AndMoreSomethingElse)
}
Example JSON results.
result-A:
{
"#type": "g:List",
"#value": [{
"#type": "g:Vertex",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 200
},
"label": "key",
"properties": {
"SomeThing": [{ // <== IMPORTANT, same as struct field in Foo{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 201
},
"value": true,
"label": "Write"
}
}],
"SomeThingElse": [{ // <== IMPORTANT, same as struct field in Foo{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 202
},
"value": true,
"label": "Read"
}
}]
}
}
}]
}
result-B:
{
"#type": "g:List",
"#value": [{
"#type": "g:Vertex",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 200
},
"label": "key",
"properties": {
"EvenMoreSomethingElse": [{ // <== IMPORTANT, same as struct field in Bar{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 201
},
"value": true,
"label": "Write"
}
}],
"AndEvenMoreSomethingElse": [{ // <== IMPORTANT, same as struct field in Bar{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 202
},
"value": true,
"label": "Read"
}
}]
}
}
}]
}

Related

When the first stage is array, How to handle with go-simplejson

JSON struct like below:
[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}
]
I don't know how to get the location 0's data.
content, _ := simplejson.NewJson(body)
arr, _ := content.Array() // Here can get the all data, It's []interface{} type.
I cannot get the next data with arr[0]["sha"]. How to handle it?
It is not clear to the compiler that arr is an array of map[string]interface{} at compile time, as arr[0] is of type interface{}. This basically means that the compiler knows nothing about this type, which is why you can't do a map lookup operation here.
You can add a type assertion to make sure you can use it as a map like this:
asMap := arr[0].(map[string]interface{})
fmt.Println(asMap["sha"])
To get the SHA as string, you can again add a type assertion behind it as well:
asString := asMap["sha"].(string)
This is also shown in this working example. The downside of this is that your program will panic in case the given data is not of the specified type. You could instead use a type assertion with a check if it worked (asString, ok := ...), but it gets cumbersome with more complex data.
This does work, but isn't really nice. I would recommend using a tool like this to generate Go structs and then use them in a type-safe way. First define a struct with all the info you need:
type ArrayElement struct {
Sha string `json:"sha"`
// Add more fields if you need them
}
Then you can just use the standard-library json package to unmarshal your data:
// This should have the same structure as the data you want to parse
var result []ArrayElement
err := json.Unmarshal([]byte(str), &result)
if err != nil {
panic(err)
}
fmt.Println(result[0].Sha)
Here is an example for that -- this is a more Go-like approach for converting JSON data.
Your json data is wrong formatted. First of all, remove , after "id": "xxxxx", line:
...
"id": "xxxxx"
...
You should check errors after NewJson to prevent find out if there is a problem:
content, err := simplejson.NewJson(body)
if err != nil {
// log err
}
For getting sha from first index, you simply can use simplejson built-in methods:
shaVal := content.GetIndex(0).Get("sha").String()
Here is how you can get the desired value.
This worked for me.
package main
import (
"encoding/json"
"fmt"
)
type MyData []struct {
Sha string `json:"sha"`
NodeID string `json:"node_id"`
Commit struct {
Author struct {
Name string `json:"name"`
} `json:"author"`
Committer struct {
Name string `json:"name"`
} `json:"committer"`
Message string `json:"message"`
Tree struct {
Sha string `json:"sha"`
URL string `json:"url"`
} `json:"tree"`
URL string `json:"url"`
CommentCount int `json:"comment_count"`
Verification struct {
Verified bool `json:"verified"`
Reason string `json:"reason"`
Signature interface{} `json:"signature"`
Payload interface{} `json:"payload"`
} `json:"verification"`
} `json:"commit"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
CommentsURL string `json:"comments_url"`
Author struct {
Login string `json:"login"`
ID string `json:"id"`
NodeID string `json:"node_id"`
AvatarURL string `json:"avatar_url"`
GravatarID string `json:"gravatar_id"`
Type string `json:"type"`
SiteAdmin bool `json:"site_admin"`
} `json:"author"`
Committer struct {
Login string `json:"login"`
ID string `json:"id"`
} `json:"committer"`
Parents []struct {
Sha string `json:"sha"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
} `json:"parents"`
}
func main() {
my_json_data := `[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}]`
var data MyData
err := json.Unmarshal([]byte(my_json_data), &data)
if err != nil {
panic(err)
}
fmt.Println("data --> sha: ", data[0].Sha)
}

Golang struct to Json schema

Hi I need to infer a json schema (github.com/go-openapi/spec.Schema) from a struct :
type Testcase struct {
Id string `json:"id,omitempty"` // id of this test case
Name string `json:"name,omitempty"` // name of this test case
CreationDate time.Time `json:"creation_date,omitempty"` // timestamp when the scenario was first created
UpdateDate time.Time `json:"update_date,omitempty"` // last update timestamp
Steps []Step `json:"steps,omitempty"` // list of steps type:"[]StepCcs"
}
I can't find a simple way to do so.
I guess that this is a prerequisite for many REST frameworks out there that generate open api spec from the code.
Can someone point me out to a package that contains such helper : ie
func toSchema(obj interface{}) (spec.Schema, error)
https://github.com/invopop/jsonschema
is what you are looking for.
type TestUser struct {
ID int `json:"id"`
Name string `json:"name" jsonschema:"title=the name,description=The name of a friend,example=joe,example=lucy,default=alex"`
Friends []int `json:"friends,omitempty" jsonschema_description:"The list of IDs, omitted when empty"`
Tags map[string]interface{} `json:"tags,omitempty" jsonschema_extras:"a=b,foo=bar,foo=bar1"`
BirthDate time.Time `json:"birth_date,omitempty" jsonschema:"oneof_required=date"`
YearOfBirth string `json:"year_of_birth,omitempty" jsonschema:"oneof_required=year"`
Metadata interface{} `json:"metadata,omitempty" jsonschema:"oneof_type=string;array"`
FavColor string `json:"fav_color,omitempty" jsonschema:"enum=red,enum=green,enum=blue"`
}
Results in following JSON Schema:
jsonschema.Reflect(&TestUser{})
Huma's schema package can do this. Be sure to check out all the validation tags that are available. Example:
import "github.com/danielgtaylor/huma/schema"
// First define some structs:
type Child struct {
Name string `json:"name" doc:"Child name"`
}
type Hello struct {
Greeting string `json:"greeting" maxLength:"80" doc:"A greeting message"`
Optional uint `json:"optional,omitempty" doc:"Optional integer example"`
Child *Child `json:"child,omitempty" doc:"Child struct example"`
Labels []string `json:"labels" maxItems:"10" doc:"Array example"`
Created time.Time `json:"created" doc:"Created time for this greeting"`
}
// Then later you can do:
s, err := schema.Generate(reflect.TypeOf(Hello{}))
https://go.dev/play/p/4F8NpcgZ4Yh
This will output:
{
"type": "object",
"properties": {
"child": {
"type": "object",
"description": "Child struct example",
"properties": {
"name": {
"type": "string",
"description": "Child name"
}
},
"additionalProperties": false,
"required": [
"name"
]
},
"created": {
"type": "string",
"description": "Created time for this greeting",
"format": "date-time"
},
"greeting": {
"type": "string",
"description": "A greeting message",
"maxLength": 80
},
"labels": {
"type": "array",
"description": "Array example",
"items": {
"type": "string"
},
"maxItems": 10
},
"optional": {
"type": "integer",
"description": "Optional integer example",
"format": "int32",
"minimum": 0
}
},
"additionalProperties": false,
"required": [
"greeting",
"labels",
"created"
]
}

array of struct object not getting return in response [duplicate]

This question already has answers here:
json.Marshal(struct) returns "{}"
(3 answers)
Closed 1 year ago.
My model having following data:
package main
type Subject struct {
name string `json:name`
section int `json:section`
}
var subjects = map[string][]Subject{
"1001": []Subject{
{
name: "Phy",
section: 1,
},
{
name: "Phy",
section: 2,
},
},
"1002": []Subject{
{
name: "Chem",
section: 1,
},
{
name: "Chem",
section: 2,
},
},
"1003": []Subject{
{
name: "Math",
section: 1,
},
{
name: "Math",
section: 2,
},
},
"1004": []Subject{
{
name: "Bio",
section: 1,
},
{
name: "Bio",
section: 2,
},
},
}
I am creating route as follows:
route.GET("/subjects/:id", func(c *gin.Context) {
id := c.Param("id")
subjects := subjects[id]
c.JSON(http.StatusOK, gin.H{
"StudentID": id,
"Subject": subjects,
})
})
It tried to call it using postman as : localhost:8080/subjects/1001
but it just shows {} {} instead of array of subject struct's objects.
Output:
{
"StudentID": "1001",
"Subject": [
{},
{}
]
}
This is because your Subject uses lowercase fields name and section and thus will not be serialized.
Changing it to:
type Subject struct {
Name string `json:"name"`
Section int `json:"section"`
}
Will show the fields:
{
"StudentID": "1001",
"Subject": [
{"name":"Phy","section":1},
{"name":"Phy","section":2}
]
}

How to replace json values from key-value map in GOlang

I have below json & I need to replace employee_names to emp_id from a map.
Tried this GolangPlay , but not sure how to replace values from the given map and error handling if the value is not present in given map.
Json data:
[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]
Map data:
[{emp1 14325} {emp3 49184} {emp2 21518}]
Expected output:
[
{
"dept": "IT",
"condition": {
"employee": [
14325
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
21518,
49184
]
}
}
]
code :
Started with below code , but not sure how to use the given map to replace with error handling.
package main
import (
"encoding/json"
"fmt"
//"strconv"
//"log"
)
func main() {
jsonStr := `[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]`
empMap := `[{emp1 14325} {emp3 49184} {emp2 21518}]`
type GetEmployee []struct {
Dept string `json:"dept"`
Condition struct {
Employee []string `json:"employee"`
} `json:"condition"`
}
var empResponse GetEmployee
unmarshallingError := json.Unmarshal([]byte(string(jsonStr)), &empResponse)
if unmarshallingError != nil {
fmt.Println(unmarshallingError.Error())
}
fmt.Println(empResponse)
fmt.Println(empMap)
for i := range empResponse {
fmt.Println(i)
}
}
Instead of storing the ids in an array of {ids: value}, it will be better for them to be in a map instead.
The above will range over the Employees' name and change it to the id. A check is made to see if there is a certain id key in the map.
for i, e := range empResponse {
fmt.Println(e)
for j,val := range empResponse[i].Condition.Employee {
if _, ok := ids[val]; ok {
empResponse[i].Condition.Employee[j] = ids[val]
}
}
}
Full code
package main
import (
"encoding/json"
"fmt"
//"strconv"
//"log"
)
func main() {
jsonStr := `[
{
"dept": "IT",
"condition": {
"employee": [
"emp1"
]
}
},
{
"dept": "HR",
"condition": {
"employee": [
"emp2",
"emp3"
]
}
}
]`
empMap := `{"emp1": "14325", "emp3": "49184", "emp2": "21518"}`
type GetEmployee []struct {
Dept string `json:"dept"`
Condition struct {
Employee []string `json:"employee"`
} `json:"condition"`
}
var empResponse GetEmployee
var ids map[string]string
unmarshallingError := json.Unmarshal([]byte(string(jsonStr)), &empResponse)
if unmarshallingError != nil {
fmt.Println(unmarshallingError.Error())
}
json.Unmarshal([]byte(empMap), &ids)
fmt.Println(empResponse)
fmt.Println(ids)
for i, e := range empResponse {
fmt.Println(e)
for j,val := range empResponse[i].Condition.Employee {
if _, ok := ids[val]; ok {
empResponse[i].Condition.Employee[j] = ids[val]
}
}
}
fmt.Println(empResponse)
}
playground
In the above, the id are strings since the name to be replaced are strings. Actually Employee are of type []string. If a string is to be replaced by an int, then Employee type needs to be changed to []interface{}.
playground

Unmarshalling a complicated JSON ad bin it with a struct

I have the following JSOn response from a webhook call
{
"responseId": "d5c70d8b-e8ad-41df-bb3b-26b0e51d60ca-a14fa99c",
"queryResult": {
"queryText": "1111111111",
"parameters": {
"phone-number": "1111111111"
},
"allRequiredParamsPresent": true,
"fulfillmentText": "Thats great! You payment link has been sent to Gaf ( Mobile number 1111111111 )",
"fulfillmentMessages": [{
"text": {
"text": ["Thats great! You payment link has been sent to Far ( Mobile number 1111111111 )"]
}
}],
"outputContexts": [{
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name",
"lifespanCount": 2,
"parameters": {
"name": ["Gar"],
"name.original": ["Gar"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_number",
"lifespanCount": 4,
"parameters": {
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name_confirm",
"lifespanCount": 3,
"parameters": {
"name": ["Gaf"],
"name.original": ["Far"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}, {
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/__system_counters__",
"parameters": {
"no-input": 0.0,
"no-match": 0.0,
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}],
"intent": {
"name": "projects/open-prod-bot-pfgibi/agent/intents/d21f7be5-0f77-4cb8-9857-26ba04964317",
"displayName": "GetMobileNumber"
},
"intentDetectionConfidence": 1.0,
"languageCode": "en"
},
"originalDetectIntentRequest": {
"payload": {
}
},
"session": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829"
}
I wanted to extract out
"outputContexts": [{
"name": "projects/open-prod-bot-pfgibi/agent/sessions/80f2fb70-01d0-fc1d-200a-ccbae5572829/contexts/awaiting_name",
"lifespanCount": 2,
"parameters": {
"name": ["Gar"],
"name.original": ["Gar"],
"phone-number": "1111111111",
"phone-number.original": "1111111111"
}
}
out of this and bind this with a struct. But I couldn't do it. I am trying to loop it through the map as below
var f interface{}
json.Unmarshal(b, &f)
for k, v := range f.(map[string]interface{}) {
if k == "queryResultmap" {
fmt.Println(v)
}
}
but not working. I am new to Go. Tried few examples in google but since this one is a complicated JSON I am unable to do it. Please help
I would suggest you declare a struct with the fields you care about and unmarshal into that, but if you want to stick to interface{} try this:
m := f.(map[string]interface{})
r := m["queryResult"].(map[string]interface{})
fmt.Println(r["outputContext"])
https://play.golang.com/p/gA5wWcPyd6E
Using struct:
type OutputContext struct {
Name string `json:"name"`
LifespanCount int `json:"lifespanCount"`
Parameters struct {
Name []string `json:"name"`
NameOriginal []string `json:"name.original"`
PhoneNumber string `json:"phone-number"`
PhoneNumberOriginal string `json:"phone-number.original"`
NoInput float64 `json:"no-input"`
NoMatch float64 `json:"no-match"`
} `json:"parameters"`
}
type QueryResult struct {
OutputContexts []OutputContext `json:"outputContexts"`
}
// ...
var dest struct {
QueryResult QueryResult `json:"queryResult"`
}
if err := json.Unmarshal(data, &dest); err != nil {
panic(err)
}
for _, v := range dest.QueryResult.OutputContexts {
fmt.Printf("%+v\n", v)
}
https://play.golang.com/p/EfIugDQJ651

Resources