Golang struct to Json schema - go

Hi I need to infer a json schema (github.com/go-openapi/spec.Schema) from a struct :
type Testcase struct {
Id string `json:"id,omitempty"` // id of this test case
Name string `json:"name,omitempty"` // name of this test case
CreationDate time.Time `json:"creation_date,omitempty"` // timestamp when the scenario was first created
UpdateDate time.Time `json:"update_date,omitempty"` // last update timestamp
Steps []Step `json:"steps,omitempty"` // list of steps type:"[]StepCcs"
}
I can't find a simple way to do so.
I guess that this is a prerequisite for many REST frameworks out there that generate open api spec from the code.
Can someone point me out to a package that contains such helper : ie
func toSchema(obj interface{}) (spec.Schema, error)

https://github.com/invopop/jsonschema
is what you are looking for.
type TestUser struct {
ID int `json:"id"`
Name string `json:"name" jsonschema:"title=the name,description=The name of a friend,example=joe,example=lucy,default=alex"`
Friends []int `json:"friends,omitempty" jsonschema_description:"The list of IDs, omitted when empty"`
Tags map[string]interface{} `json:"tags,omitempty" jsonschema_extras:"a=b,foo=bar,foo=bar1"`
BirthDate time.Time `json:"birth_date,omitempty" jsonschema:"oneof_required=date"`
YearOfBirth string `json:"year_of_birth,omitempty" jsonschema:"oneof_required=year"`
Metadata interface{} `json:"metadata,omitempty" jsonschema:"oneof_type=string;array"`
FavColor string `json:"fav_color,omitempty" jsonschema:"enum=red,enum=green,enum=blue"`
}
Results in following JSON Schema:
jsonschema.Reflect(&TestUser{})

Huma's schema package can do this. Be sure to check out all the validation tags that are available. Example:
import "github.com/danielgtaylor/huma/schema"
// First define some structs:
type Child struct {
Name string `json:"name" doc:"Child name"`
}
type Hello struct {
Greeting string `json:"greeting" maxLength:"80" doc:"A greeting message"`
Optional uint `json:"optional,omitempty" doc:"Optional integer example"`
Child *Child `json:"child,omitempty" doc:"Child struct example"`
Labels []string `json:"labels" maxItems:"10" doc:"Array example"`
Created time.Time `json:"created" doc:"Created time for this greeting"`
}
// Then later you can do:
s, err := schema.Generate(reflect.TypeOf(Hello{}))
https://go.dev/play/p/4F8NpcgZ4Yh
This will output:
{
"type": "object",
"properties": {
"child": {
"type": "object",
"description": "Child struct example",
"properties": {
"name": {
"type": "string",
"description": "Child name"
}
},
"additionalProperties": false,
"required": [
"name"
]
},
"created": {
"type": "string",
"description": "Created time for this greeting",
"format": "date-time"
},
"greeting": {
"type": "string",
"description": "A greeting message",
"maxLength": 80
},
"labels": {
"type": "array",
"description": "Array example",
"items": {
"type": "string"
},
"maxItems": 10
},
"optional": {
"type": "integer",
"description": "Optional integer example",
"format": "int32",
"minimum": 0
}
},
"additionalProperties": false,
"required": [
"greeting",
"labels",
"created"
]
}

Related

When the first stage is array, How to handle with go-simplejson

JSON struct like below:
[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}
]
I don't know how to get the location 0's data.
content, _ := simplejson.NewJson(body)
arr, _ := content.Array() // Here can get the all data, It's []interface{} type.
I cannot get the next data with arr[0]["sha"]. How to handle it?
It is not clear to the compiler that arr is an array of map[string]interface{} at compile time, as arr[0] is of type interface{}. This basically means that the compiler knows nothing about this type, which is why you can't do a map lookup operation here.
You can add a type assertion to make sure you can use it as a map like this:
asMap := arr[0].(map[string]interface{})
fmt.Println(asMap["sha"])
To get the SHA as string, you can again add a type assertion behind it as well:
asString := asMap["sha"].(string)
This is also shown in this working example. The downside of this is that your program will panic in case the given data is not of the specified type. You could instead use a type assertion with a check if it worked (asString, ok := ...), but it gets cumbersome with more complex data.
This does work, but isn't really nice. I would recommend using a tool like this to generate Go structs and then use them in a type-safe way. First define a struct with all the info you need:
type ArrayElement struct {
Sha string `json:"sha"`
// Add more fields if you need them
}
Then you can just use the standard-library json package to unmarshal your data:
// This should have the same structure as the data you want to parse
var result []ArrayElement
err := json.Unmarshal([]byte(str), &result)
if err != nil {
panic(err)
}
fmt.Println(result[0].Sha)
Here is an example for that -- this is a more Go-like approach for converting JSON data.
Your json data is wrong formatted. First of all, remove , after "id": "xxxxx", line:
...
"id": "xxxxx"
...
You should check errors after NewJson to prevent find out if there is a problem:
content, err := simplejson.NewJson(body)
if err != nil {
// log err
}
For getting sha from first index, you simply can use simplejson built-in methods:
shaVal := content.GetIndex(0).Get("sha").String()
Here is how you can get the desired value.
This worked for me.
package main
import (
"encoding/json"
"fmt"
)
type MyData []struct {
Sha string `json:"sha"`
NodeID string `json:"node_id"`
Commit struct {
Author struct {
Name string `json:"name"`
} `json:"author"`
Committer struct {
Name string `json:"name"`
} `json:"committer"`
Message string `json:"message"`
Tree struct {
Sha string `json:"sha"`
URL string `json:"url"`
} `json:"tree"`
URL string `json:"url"`
CommentCount int `json:"comment_count"`
Verification struct {
Verified bool `json:"verified"`
Reason string `json:"reason"`
Signature interface{} `json:"signature"`
Payload interface{} `json:"payload"`
} `json:"verification"`
} `json:"commit"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
CommentsURL string `json:"comments_url"`
Author struct {
Login string `json:"login"`
ID string `json:"id"`
NodeID string `json:"node_id"`
AvatarURL string `json:"avatar_url"`
GravatarID string `json:"gravatar_id"`
Type string `json:"type"`
SiteAdmin bool `json:"site_admin"`
} `json:"author"`
Committer struct {
Login string `json:"login"`
ID string `json:"id"`
} `json:"committer"`
Parents []struct {
Sha string `json:"sha"`
URL string `json:"url"`
HTMLURL string `json:"html_url"`
} `json:"parents"`
}
func main() {
my_json_data := `[
{
"sha": "eb08dc1940e073a5c40d8b53a5fd58760fde8f27",
"node_id": "C_kwDOHb9FrtoAKGViMDhkYzE5NDBlMDczYTVjNDBkOGI1M2E1ZmQ1ODc2MGZkZThmMjc",
"commit": {
"author": {
"name": "xxxx"
},
"committer": {
"name": "xxxxx"
},
"message": "update DownLoad_Stitch_ACM.py",
"tree": {
"sha": "a30aab98319846f0e86da4a39ec05786e04c0a4f",
"url": "xxxxx"
},
"url": "xxxxx",
"comment_count": 0,
"verification": {
"verified": false,
"reason": "unsigned",
"signature": null,
"payload": null
}
},
"url": "xxxxx",
"html_url": "xxxxx",
"comments_url": "xxxxx",
"author": {
"login": "xxxxx",
"id": "xxxxx",
"node_id": "U_kgDOBkuicQ",
"avatar_url": "https://avatars.githubusercontent.com/u/105620081?v=4",
"gravatar_id": "",
"type": "User",
"site_admin": false
},
"committer": {
"login": "xxxxx",
"id": "xxxxx"
},
"parents": [
{
"sha": "cf867ec9dc4b904c466d9ad4b9338616d1213a06",
"url": "xxxxx",
"html_url": "xxxxx"
}
]
}]`
var data MyData
err := json.Unmarshal([]byte(my_json_data), &data)
if err != nil {
panic(err)
}
fmt.Println("data --> sha: ", data[0].Sha)
}

How to Marshal/Unmarshal a common JSON & BSON key/field that can have two different formats in Go?

I currently have mongo data stored in two forms (specifically for content key) in a collection. Partial sample data shown below:
Format 1.
{
"type": "text",
"content": "foobar",
"extraTextData": "hello text"
}
Format 2
{
"type": "group",
"content": [
{
"type": "text",
"content": "grouped-foobar"
},
{
"type": "image",
"url": "https://abc.jpg"
},
],
"extraGroupData": "hello group"
}
My attempt to structure this in golang is below.
type C struct {
Type string `json:"type" bson:"type"`
Content ???
*TextC
*GroupC
}
type TextC struct {
ExtraTextData `json:"extraTextData" bson:"extraTextData"`
}
type GroupC struct {
ExtraGroupData `json:"extraGroupData" bson:"extraGroupData"`
}
I am having issues on how to setup the structure for "content" field that works for both the formats, TextC and GroupC.
Content for GroupC can be array of C like - Content []C
Content for TextC can also be string type.
Can someone please help & give an example on how to tackle this situation?
Format2 json is Invalid. You can check it here : https://jsonlint.com/
I have created a sample scenario for your case.
You can try it here: https://go.dev/play/p/jaUE3rjI-Ik
Use interface{} like this:
type AutoGenerated struct {
Type string `json:"type"`
Content interface{} `json:"content"`
ExtraTextData string `json:"extraTextData,omitempty"`
ExtraGroupData string `json:"extraGroupData,omitempty"`
}
And you should also remove comma from Format2:
{
"type": "group",
"content": [
{
"type": "text",
"content": "grouped-foobar"
},
{
"type": "image",
"url": "https://abc.jpg"
}
],
"extraGroupData": "hello group"
}
If you don't remove comma then it will through an error like this:
invalid character ']' looking for beginning of value

How to make query result structure match to what i've been declared on GORM Select

I want to make the structure of the query results match what I have stated in GORM Select because right now it only matches the Struct structure. How do i make it work? Thank you in advance
i've tried to make new Struct and it works, but i dont know is it a best practice or not
type User struct {
User_Id uint `json:"user_id" gorm:"column:user_id; PRIMARY_KEY"`
Email string `json:"email"`
Password string `json:"password"`
Token string `json:"token" gorm:"-"`
}
func GetUsers() map[string]interface{} {
users := []User{}
GetDB().Table("app_user").Select("user_id, email").Find(&users)
resp := u.Message(true, "All users")
resp["users"] = users
return resp
}
//actual result
{
"message": "All users",
"status": true,
"users": [
{
"user_id": 1732,
"email": "aaaaaaa#gmail.com",
"password": "",
"token": ""
},
{
"user_id": 1733,
"email": "bbbbbbb#gmail.com",
"password": "",
"token": ""
},
]
}
//Expected result
{
"message": "All users",
"status": true,
"users": [
{
"user_id": 1732,
"email": "aaaaaaa#gmail.com"
},
{
"user_id": 1733,
"email": "bbbbbbb#gmail.com"
}
]
}
It looks like all you need to do is to omit empty fields. You can do that by adding omitempty to json tags:
Password string `json:"password,omitempty"`
Token string `json:"token,omitempty" gorm:"-"`

Unable to do a mutation with a property of type "array of objects" in apollo

I'm new to all graphql world, so this might be a very easy question, sorry
I'm using graphql-compose-mongoose to generate my graphql schema, here's my mongoose schema:
const ComplainSchema = new Schema({
entityId: {type: String, required: true},
user: {type: UserInfoSchema, required: true},
title: String, // standard types
desc: String,
state: {required: true, type: String, enum: ["DRAFT", "MODERATION", "PUBLIC", "SOLVED"]},
attachments: [{
url: {type: String, required: true},
name: String,
mimeType: String,
attachmentId: Schema.Types.ObjectId
}],
createdAt: {type: Date, index: true},
updatedAt: {type: Date, index: true},
}, {timestamps: {}})
export default mongoose.model('Complaint', ComplainSchema)
If I attempt the following mutation in graphiql it works fine
mutation {
complaintUpdateById(record:{_id:"5bdd9350fe144227042e6a20", title:"ok", desc:"updated", attachments:[{name:"zied", url:"http://zied.com"}]}){
recordId,
record{
_id,
entityId,
user {
userId,
userName,
roleInShop
},
title,
desc,
createdAt,
updatedAt,
attachments{
name,
url
}
}
}
}
and returns this (in case there could be helpful to see the response)
{
"data": {
"complaintUpdateById": {
"recordId": "5bdd9350fe144227042e6a20",
"record": {
"_id": "5bdd9350fe144227042e6a20",
"entityId": "5bd9b1858788f51f44ab678a",
"user": {
"userId": "5bd9ac078788f51f44ab6785",
"userName": "Zied Hamdi",
"roleInShop": "ASA"
},
"title": "ok",
"desc": "updated",
"createdAt": "2018-11-03T12:23:44.565Z",
"updatedAt": "2018-11-05T09:02:51.494Z",
"attachments": [
{
"name": "zied",
"url": "http://zied.com"
}
]
}
}
}
}
Now if I try to pass the attachments to apollo, I don't know how to do that, I don't know which type to provide (Attachment is not the right type obvisouly):
const UPDATE_COMPLAINT = gql `mutation complaintUpdateById($_id:MongoID!, $title: String!, $desc: String!, $attachments: [Attachment]
)
{
complaintUpdateById(record:{_id:$_id, title:$title, desc:$desc, attachments:$attachments}){
recordId,
record{
_id,
entityId,
user {
userId,
userName,
roleInShop
},
title,
desc,
createdAt,
updatedAt
}
}
}`
So searching for the right type, I did a introspection of my object, the issue is that I get the type of attachment as null for this query:
{
__type(name: "Complaint") {
kind
name
fields {
name
description
type {
name
}
}
}
}
this is the response:
{
"data": {
"__type": {
"kind": "OBJECT",
"name": "Complaint",
"fields": [
{
"name": "entityId",
"description": null,
"type": {
"name": "String"
}
},
{
"name": "user",
"description": null,
"type": {
"name": "ComplaintUser"
}
},
{
"name": "title",
"description": null,
"type": {
"name": "String"
}
},
{
"name": "desc",
"description": null,
"type": {
"name": "String"
}
},
{
"name": "state",
"description": null,
"type": {
"name": "EnumComplaintState"
}
},
{
"name": "attachments",
"description": null,
"type": {
"name": null
}
},
{
"name": "createdAt",
"description": null,
"type": {
"name": "Date"
}
},
{
"name": "updatedAt",
"description": null,
"type": {
"name": "Date"
}
},
{
"name": "_id",
"description": null,
"type": {
"name": null
}
}
]
}
}
}
googling didn't help since I don't know how is this operation called, I don't think it's a nested mutation from what I found...
Ok fixed,
I did these steps:
I first introspected the type of attachment in a regular query using the __typename keyword: as follows
mutation {
complaintUpdateById(record:{_id:"5bdd9350fe144227042e6a20", title:"ok", desc:"updated", attachments:[{name:"zied", url:"http://zied.com"}]}){
recordId,
record{
_id,
entityId,
user {
userId,
userName,
roleInShop
},
title,
desc,
createdAt,
updatedAt,
attachments{
__typename,
name,
url
}
}
}
}
it showed up a type named ComplaintAttachments
when replacing the Attachment type with this new value, ComplaintAttachments, an error occured and that error message helped me out:
Variable "$attachments" of type "[ComplaintAttachments]" used in
position expecting type "[ComplaintComplaintAttachmentsInput]"
so the array is of type ComplaintComplaintAttachmentsInput, I still don't know how to introspect it directly, but I'm already happy with the result :)

How to populate an arbitrary struct in Go?

I want to create a function (in the example code called gremlinResultToStruct()) that takes an arbitrary struct and populates the values from a database query. Please note that the "Properties" key in the GremlinResultStruct are arbitrary too (that is why it is a Properties map[string][]struct)
GOAL:
I want gremlinResultToStruct() to populate the given struct based on its keys if they are available in the database results.
WHY:
I have hundreds of structs like Foo and Bar in my codebase and I want to have a single function that populates the struct needed in the code instead of creating semi-duplicate code.
CODE:
My code looks like this, it also contains the question:
// Gremlin Query result struct
type GremlinResult struct {
AtType string `json:"#type"`
AtValue []struct {
AtType string `json:"#type"`
AtValue struct {
ID struct {
AtType string `json:"#type"`
AtValue int `json:"#value"`
} `json:"id"`
Label string `json:"label"`
Properties map[string][]struct { // <== IMPORTANT, CONTAINS A VALUE THAT IS THE EQUIVALENT OF THE KEY IN THE STRUCT (like Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse)
AtType string `json:"#type"`
AtValue struct {
ID struct {
AtType string `json:"#type"`
AtValue int `json:"#value"`
} `json:"id"`
Label string `json:"label"`
Value string `json:"value"`
} `json:"#value"`
} `json:"#properties"`
} `json:"#value"`
} `json:"#value"`
}
type Foo struct {
Something string
SomethingElse bool
}
type Bar struct {
EvenMoreSomethingElse string
AndMoreSomethingElse int
}
func gremlinResultToStruct(result *GremlinResult, ???){ // What to do at ???
// How to itterate over the ??? key values (like: Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse)
// How to populate:
// VARIABLE := Something, SomethingElse, EvenMoreSomethingElse or AndMoreSomethingElse
// result.AtValue[0].AtValue.Properties[ VARIABLE ][0].AtValue.Value
}
func main {
// Do stuff to get JSON results. HOW is not relevant for question
dbResults := doDbStuffToPopulateResuts()
// Create the results variable as gremlinResults struct
results := GremlinResult{}
// Unmarshall to this struct
err = json.Unmarshal(dbResults, &results)
// 1. Expected result should be "result-a" (see below)
// Create the Foo struct
foo := Foo{}
// HERE IS MY QUESTION ABOUT (see function above)
gremlinResultToStruct(&results, &foo)
// NOW foo SHOULD HAVE Something and SomethingElse set
fmt.Println(foo.Something, foo.SomethingElse)
// 2. Expected result should be "result-b" (see below)
// Create the Bar struct
bar := Bar{}
// HERE IS MY QUESTION ABOUT (see function above)
gremlinResultToStruct(&results, &bar)
// NOW foo SHOULD HAVE Something and SomethingElse set
fmt.Println(bar.EvenMoreSomethingElse, bar.AndMoreSomethingElse)
}
Example JSON results.
result-A:
{
"#type": "g:List",
"#value": [{
"#type": "g:Vertex",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 200
},
"label": "key",
"properties": {
"SomeThing": [{ // <== IMPORTANT, same as struct field in Foo{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 201
},
"value": true,
"label": "Write"
}
}],
"SomeThingElse": [{ // <== IMPORTANT, same as struct field in Foo{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 202
},
"value": true,
"label": "Read"
}
}]
}
}
}]
}
result-B:
{
"#type": "g:List",
"#value": [{
"#type": "g:Vertex",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 200
},
"label": "key",
"properties": {
"EvenMoreSomethingElse": [{ // <== IMPORTANT, same as struct field in Bar{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 201
},
"value": true,
"label": "Write"
}
}],
"AndEvenMoreSomethingElse": [{ // <== IMPORTANT, same as struct field in Bar{}
"#type": "g:VertexProperty",
"#value": {
"id": {
"#type": "g:Int64",
"#value": 202
},
"value": true,
"label": "Read"
}
}]
}
}
}]
}

Resources