I want to append a value to the returned response of my code, this is what I already have:
publisherShare := 25 // I also have this variable that want to append to the returned response.
c.JSON(http.StatusOK, nf) // nf is a row found and returned from database.
This returns such json:
{
"id": 105324,
"title": "test title",
"last_update": "2021-03-10T12:50:37+03:30",
"created_at": "2021-03-10T12:50:36+03:30",
"updated_at": "2021-05-05T05:46:39.352859604Z"
}
I need to have a json result like this:
{
"id": 105324,
"title": "test title",
"last_update": "2021-03-10T12:50:37+03:30",
"created_at": "2021-03-10T12:50:36+03:30",
"updated_at": "2021-05-05T05:46:39.352859604Z",
"publisher_share": 25 // I want this to be added.
}
This is what I have tried so far, but it changes the schema and will not be backward compatible anymore:
c.Json(http.StatusOK, gin.H{"book": nf, "publisher_share": publisherShare})
but this is not the json result I want. I want publisher_share alongside other fields just like the json result I mentioned above.
You can add publisher share to the same type as
type nfType struct {
// All of the db row fields
PublisherShare int `json:"publisher_share,omitempty" db:"-"`
}
Simply create a struct that matches the structure of your json responce. Key is to add aditional fild to it like
type Responce struct {
// other fields. They have to be written in same way to keep conventions
// (Go-ish name and needed name as tag)
PublisherShare int `json:"publisher_share"`
}
Then when you handle responce you can just unmarshal and marshal json.
responce := Responce{PublisherShare: 25}
err := json.Unmarshal(bytes, &responce)
if err != nil {
//handle error
}
newResponce, err := jsom.Marshal(responce)
if err != nil {
//handle error
}
// newResponce is now in form you want
Define a new type. It embeds the type of the db row, and it has a field for the extra value. E.G: if the type of the db row is DatabaseRow:
type CompleteReturn struct {
DatabaseRow
PublisherShare int `json:"publisher_share"`
}
Later use it as:
complete := CompleteReturn{
DatabaseRow: nf, // Serialize in json as before
PublisherShare: 25, // Serialize in json as publisher_share
}
c.Json(http.StatusOk, complete)
Related
func (m *TodoServer) GetTodos(ctx context.Context, empty *emptypb.Empty) (*desc.GetTodosResponse, error) {
todos, err := m.todoService.GetTodos()
if err != nil {
return nil, err
}
todosResp := make([]*desc.GetTodosResponse_Todo, 0, len(todos))
for _, todo := range todos {
todosResp = append(todosResp, &desc.GetTodosResponse_Todo{
Id: todo.ID,
Title: todo.Title,
IsCompleted: todo.IsCompleted,
})
}
return &desc.GetTodosResponse{Todos: todosResp}, nil
}
service TodoService {
rpc GetTodos(google.protobuf.Empty) returns (GetTodosResponse) {}
}
message GetTodosResponse {
repeated Todo todos = 1;
message Todo {
int64 id = 1;
string title = 2;
bool is_completed = 3;
}
}
I have one record in the db
| id | title | is_completed |
|-|-|-|
| 1 | aaa | false |
the function above returns {"todos": [{"id": "1", "title": "aaa"}]} but once I change the is_completed to true the result is correct {"todos": [{"id": "1", "title": "aaa", "isCompleted": true}]}
This is by design and for efficiency.
The "zero" value for a bool is false - so when initializing your protobuf struct with a false value, the field does not need to be explicitly stated when using the standard libary's encoding/json unmarshaler. On the encoding end, if the field's JSON tag includes an omitempty qualifier, the standard libary's encoding/json marshaler will remove any zero values - which is what you are seeing.
You will see the same behavior with the Title string field if it was "" (i.e. the zero value of a string).
Looking at your generated code (*.pb.go) the struct's bool field definition will look something like this:
type Todo struct {
// ...
IsCompleted bool `protobuf:"varint,5,opt,name=is_complete,proto3" json:"is_complete,omitempty"`
}
So the json:"...,omitempty" instructs the encoding/json marshaler to omit any zero values during marshalling with tags like theses.
If you want to override this behavior:
one could remove the omitempty directive from your generated code (not recommended - as edits would need to be managed over the lifecycle of development). But if you must, refer to this answer;
if using grpc-gateway, override this at runtime e.g.
gwmux := runtime.NewServeMux(runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONPb{OrigName: true, EmitDefaults: true}))
or if exporting the JSON yourself, instead of using the standard library (encoding/json) use the JSON marshaler from this package "google.golang.org/protobuf/encoding/protojson":
protojson.Marshaler{EmitDefaults: true}.Marshal(w, resp)
as outlined in this answer.
I'm trying to figure out how to create a slice I can more easily manipulate and use JUST the values from to later iterate over to make a number of API requests. The slice of integers are API IDs. I am successfully making a struct with custom types after making a GET to retrieve the JSON Array of IDs, but I now need to pull only the values from that JSON array and dump them into a slice without the key "id" (which will likely need to change over time in size).
This is my JSON:
{
"data": [
{
"id": 38926
},
{
"id": 38927
}
],
"meta": {
"pagination": {
"total": 163795,
"current_page": 3,
"total_pages": 81898
}
}
}
And I would like this from it:
{38926, 38927}
If you want custom Unmarshaling behavior, you need a custom type with its own json.Unmarshaler e.g.
type ID int
func (i *ID) UnmarshalJSON(data []byte) error {
id := struct {
ID int `json:"id"`
}{}
err := json.Unmarshal(data, &id)
if err != nil {
return err
}
*i = ID(id.ID)
return nil
}
To use this, reference this type in your struct e.g.
type data struct {
IDs []ID `json:"data"`
}
var d data
working example: https://go.dev/play/p/i3MAy85nr4X
I'm trying to get values/index/key of a JSON but no success. I found some answers and tutorials but I couldn't get them working for me.
I have the following JSON:
{
"id": "3479",
"product": "Camera",
"price": "",
"creation": 04032020,
"products": [
{
"camera": "Nikon",
"available": true,
"freeshipping": false,
"price": "1,813",
"color": "black"
},
{
"camera": "Sony",
"available": true,
"freeshipping": true,
"price": "931",
"color": "black"
}
],
"category": "eletronics",
"type": "camera"
}
I have tried several examples but none worked for this type of Json.
The error I'm getting:
panic: interface conversion: interface {} is nil, not
map[string]interface {}
I believe it's because of "products[]" I tried map[string]interface{} and []interface{} it compiles but gives me that error afterwards.
Could you give me an example on how I can extract these values?
The code I'm using:
//gets the json
product,_:=conn.GetProductData(shop.Info.Id)
// assing a variable
productInformation:=<-product
//prints the json
fmt.Printf(productInformation)
//try to get json values
type Product struct {
Id string
Product string
}
var product Product
json.Unmarshal([]byte(productInformation), &product)
fmt.Printf("Id: %s, Product: %s", product.Id, product.Product)
This code does not panic but it doesn't print all results either so I tried
this one below (which was supposed to give me all results) but it panics
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].(map[string]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value.(string))
}
You need to add json tag to specify the field name in json as it is in lowercase
type Product struct {
Id string `json:"id"`
Product string `json:"product"`
}
And in the second case according to json it is a slice not a map so you need to cast it to []interface{}
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].([]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value)
}
I am trying to build aggregation services, to all third party APIs that's I used,
this aggregation services taking json values coming from my main system and it will put this value to key equivalent to third party api key then, aggregation services it will send request to third party api with new json format.
example-1:
package main
import (
"encoding/json"
"fmt"
"log"
"github.com/tidwall/gjson"
)
func main() {
// mapping JSON
mapB := []byte(`
{
"date": "createdAt",
"clientName": "data.user.name"
}
`)
// from my main system
dataB := []byte(`
{
"createdAt": "2017-05-17T08:52:36.024Z",
"data": {
"user": {
"name": "xxx"
}
}
}
`)
mapJSON := make(map[string]interface{})
dataJSON := make(map[string]interface{})
newJSON := make(map[string]interface{})
err := json.Unmarshal(mapB, &mapJSON)
if err != nil {
log.Panic(err)
}
err = json.Unmarshal(dataB, &dataJSON)
if err != nil {
log.Panic(err)
}
for i := range mapJSON {
r := gjson.GetBytes(dataB, mapJSON[i].(string))
newJSON[i] = r.Value()
}
newB, err := json.MarshalIndent(newJSON, "", " ")
if err != nil {
log.Println(err)
}
fmt.Println(string(newB))
}
output:
{
"clientName": "xxx",
"date": "2017-05-17T08:52:36.024Z"
}
I use gjson package to get values form my main system request in simple way from a json document.
example-2:
import (
"encoding/json"
"fmt"
"log"
"github.com/tidwall/gjson"
)
func main() {
// mapping JSON
mapB := []byte(`
{
"date": "createdAt",
"clientName": "data.user.name",
"server":{
"google":{
"date" :"createdAt"
}
}
}
`)
// from my main system
dataB := []byte(`
{
"createdAt": "2017-05-17T08:52:36.024Z",
"data": {
"user": {
"name": "xxx"
}
}
}
`)
mapJSON := make(map[string]interface{})
dataJSON := make(map[string]interface{})
newJSON := make(map[string]interface{})
err := json.Unmarshal(mapB, &mapJSON)
if err != nil {
log.Panic(err)
}
err = json.Unmarshal(dataB, &dataJSON)
if err != nil {
log.Panic(err)
}
for i := range mapJSON {
r := gjson.GetBytes(dataB, mapJSON[i].(string))
newJSON[i] = r.Value()
}
newB, err := json.MarshalIndent(newJSON, "", " ")
if err != nil {
log.Println(err)
}
fmt.Println(string(newB))
}
output:
panic: interface conversion: interface {} is map[string]interface {}, not string
I can handle this error by using https://golang.org/ref/spec#Type_assertions, but what if this json object have array and inside this array have json object ....
my problem is I have different apis, every api have own json schema, and my way for mapping json only work if
third party api have json key value only, without nested json or array inside this array json object.
is there a way to mapping complex json schema, or golang package to help me to do that?
EDIT:
After comment interaction and with updated question. Before we move forward, I would like to mention.
I just looked at your example-2 Remember one thing. Mapping is from one form to another form. Basically one known format to targeted format. Each data type have to handled. You cannot do generic to generic mapping logically (technically feasible though, would take more time & efforts, you can play around on this).
I have created sample working program of one approach; it does a mapping of source to targeted format. Refer this program as a start point and use your creativity to implement yours.
Playground link: https://play.golang.org/p/MEk_nGcPjZ
Explanation: Sample program achieves two different source format to one target format. The program consist of -
Targeted Mapping definition of Provider 1
Targeted Mapping definition of Provider 2
Provider 1 JSON
Provider 2 JSON
Mapping function
Targeted JSON marshal
Key elements from program: refer play link for complete program.
type MappingInfo struct {
TargetKey string
SourceKeyPath string
DataType string
}
Map function:
func mapIt(mapping []*MappingInfo, parsedResult gjson.Result) map[string]interface{} {
mappedData := make(map[string]interface{})
for _, m := range mapping {
switch m.DataType {
case "time":
mappedData[m.TargetKey] = parsedResult.Get(m.SourceKeyPath).Time()
case "string":
mappedData[m.TargetKey] = parsedResult.Get(m.SourceKeyPath).String()
}
}
return mappedData
}
Output:
Provider 1 Result: map[date:2017-05-17 08:52:36.024 +0000 UTC clientName:provider1 username]
Provider 1 JSON: {
"clientName": "provider1 username",
"date": "2017-05-17T08:52:36.024Z"
}
Provider 2 Result: map[date:2017-05-12 06:32:46.014 +0000 UTC clientName:provider2 username]
Provider 2 JSON: {
"clientName": "provider2 username",
"date": "2017-05-12T06:32:46.014Z"
}
Good luck, happy coding!
Typically Converting/Transforming one structure to another structure, you will have to handle this with application logic.
As you mentioned in the question:
my problem is I have different apis, every api have own json schema
This is true for every aggregation system.
One approach to handle this requirement effectively; is to keep mapping of keys for each provider JSON structure and targeted JSON structure.
For example: This is an approach, please go with your design as you see fit.
JSON structures from various provider:
// Provider 1 : JSON structrure
{
"createdAt": "2017-05-17T08:52:36.024Z",
"data": {
"user": {
"name": "xxx"
}
}
}
// Provider 2 : JSON structrure
{
"username": "yyy"
"since": "2017-05-17T08:52:36.024Z",
}
Mapping for target JSON structure:
jsonMappingByProvider := make(map[string]string)
// Targeted Mapping for Provider 1
jsonMappingByProvider["provider1"] = `
{
"date": "createdAt",
"clientName": "data.user.name"
}
`
// Targeted Mapping for Provider 2
jsonMappingByProvider["provider2"] = `
{
"date": "since",
"clientName": "username"
}
`
Now, based the on the provider you're handling, get the mapping and map the response JSON into targeted structure.
// get the mapping info by provider
mapping := jsonMappingByProvider["provider1"]
// Parse the response JSON
// Do the mapping
This way you can control each provider and it's mapping effectively.
I have a code # http://play.golang.org/p/HDlJJ54YqW
I wanted to print the Phone and email of a person.
It can be of multiple entries.
But getting the error undefined.
Can anyone help out.
Thanks.
Small details: you are referencing twice: you give the address of the address of the object to json.Unmarshal. Just give the address.
` allows for multiline, no need to split your json input.
I don't know what you where trying to achieve with u.Details[Phone:"1111"].Email, but this is no Go syntax. your Details member is a slice off Detail. A slice is similar to an array and can be accessed by index.
Also, your json does not match your object structure. If you want to have multiple details in one content, then it needs to be embed in an array ([ ])
You could do something like this: (http://play.golang.org/p/OP1zbPW_wk)
package main
import (
"encoding/json"
"fmt"
)
type Content struct {
Owner string
Details []*Detail
}
type Detail struct {
Phone string
Email string
}
func (c *Content) SearchPhone(phone string) *Detail {
for _, elem := range c.Details {
if elem.Phone == phone {
return elem
}
}
return nil
}
func (c *Content) SearchEmail(email string) *Detail {
for _, elem := range c.Details {
if elem.Email == email {
return elem
}
}
return nil
}
func main() {
encoded := `{
"Owner": "abc",
"Details": [
{
"Phone": "1111",
"Email": "#gmail"
},
{
"Phone": "2222",
"Email": "#yahoo"
}
]
}`
// Decode the json object
u := &Content{}
if err := json.Unmarshal([]byte(encoded), u); err != nil {
panic(err)
}
// Print out Email and Phone
fmt.Printf("Email: %s\n", u.SearchPhone("1111").Email)
fmt.Printf("Phone: %s\n", u.SearchEmail("#yahoo").Phone)
}