I'm currently using gRPC for my API communication. Now I need the value of my request can accept any data type, be it a struct object or just a primitive data type like single integer, string, or boolean. I tried using google.protobuf.Any as below but it doesn't work when I want to marshal and unmarshal it.
message MyRequest {
google.protobuf.Any item = 1; // could be 9, "a string", true, false, {"key": value}, etc.
}
proto-compiled result:
type MyRequest struct {
Item *anypb.Any `protobuf:"bytes,1,opt,name=item,proto3" json:"item,omitempty"`
}
I will marshal the incoming request through gRPC and save it as a JSON string in the database:
bytes, err = json.Marshal(m.request.Item)
if err != nil {
return err
}
itemStr := string(bytes)
But when I want to unmarshal the string back into *anypb.Any by:
func getItem(itemStr string) (structpb.Value, error) {
var item structpb.Value
err := json.Unmarshal([]byte(itemStr), &item)
if err != nil {
return item, err
}
// also I've no idea on how to convert `structpb.Value` to `anypb.Any`
return item, nil
}
It returns an error which seems to be because the struct *anypb.Any doesn't contain any of the fields in my encoded item string. Is there a correct way to solve this problem?
Consider using anypb
In the documentation it shows an example of how to unmarshal it
m := new(foopb.MyMessage)
if err := any.UnmarshalTo(m); err != nil {
... // handle error
}
... // make use of m
Related
I'm trying to add string "Employee" to my existing JSON response. Also, we need to be able to generate this version of json based on an user condition. Only if the user condition is met, I need to generate second version of json with string "Employee" added. If not the first version without string "Employee" should be generated. How can I achieve it with out updating the existing struct and how can I check this with if clause to check for the condition and then generate json based on it?
Below is my existing json response in go
[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
How can I get my json response like below with out changing existing struct based on input parameter?
{
"Employee":[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
}
Below is my code:
Step 1: model folder
type EmployeeWithRoot struct {
Employee []Employee
}
type Employee struct {
EmpNbr sql.NullString `json:"EmpNbr"`
DateofJoin sql.NullString `json:"DateofJoin"`
DeptId sql.NullString `json:"DeptId"`
DeptName sql.NullString `json:"DeptName"`
}
Step 2: code folder
func GetEmpDetails(logRequestId string, logNestedLevel int, EmpNbr string, DateofJoin string) ([]model.EmployeeWithRoot, error) {
logFunctionFunctionName := "code.GetEmpDetails"
logStartTime := time.Now()
logNestedLevel++
defer configurations.TimeTrack(logFunctionFunctionName, logRequestId, logStartTime, logNestedLevel)
rows, err := db.Query(utils.SELECT_OF_EMP_AGGR, EmpNbr, DateofJoin, DeptId, DeptName)
if err != nil {
return nil, err
}
defer rows.Close()
var e []model.EmployeeWithRoot
for rows.Next() {
var x model.EmployeeWithRoot
err := rows.Scan(&x.Employee.EmpNbr, &x.Employee.DateofJoin, &x.Employee.DeptId,&x.Employee.DeptName)
if err != nil {
return nil, err
}
e = append(e, x)
}
err = rows.Err()
if err != nil {
return nil, err
}
return e, nil
}
STEP 3: API folder
Employee, err := code.GetEmpDetails(logRequestId, logNestedLevel, EmpNbr, DateofJoin)
if err != nil {
log.Panic(err)
}
marshalDataForRequestContentType(logRequestId, logNestedLevel, w, r, Employee)
I'm getting the below error.
x.Employee.EmpNbr undefined (type []model.Employee has no field or method EmpNbr)
x.Employee.DateofJoin undefined (type []model.Employee has no field or method DateofJoin)enter code here
x.Employee.DeptId undefined (type []model.Employee has no field or method DeptId)
x.Employee.DeptName undefined (type []model.Employee has no field or method DeptName)
Considering you're just wrapping it it an outer object, I don't see any reason you'd need to change the existing struct, just wrap it in a new one. I'll have to make some guesses/assumptions here since you've only shown the JSON and not the Go code that produces it, but assuming your existing JSON is produced by marshaling something like var response []Employee, the desired JSON could be produced in your condition by marshaling instead:
json.Marshal(struct{Employee []Employee}{response})
Working example: https://go.dev/play/p/vwDvxnQ96G_2
Use string concatenation:
func addRoot(json string) string {
return `{ "Employee":` + json + `}`
}
Run an example on the GoLang playground.
Here's the code if you are working with []byte instead of string:
func addRoot(json []byte) []byte {
const prefix = `{ "Employee":`
const suffix = `}`
result := make([]byte, 0, len(prefix)+len(json)+len(suffix))
return append(append(append(result, prefix...), json...), suffix...)
}
Run this example on the GoLang playground.
If you have some JSON in a byte slice ([]byte) then you can just add the outer element directly - e.g. (playground):
existingJSON := []byte(`[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]`)
b := bytes.NewBufferString(`{"Employee":`)
b.Write(existingJSON)
b.WriteString(`}`)
fmt.Println(string(b.Bytes()))
If this is not what you are looking for please add further details to your question (ideally your attempt as a minimal, reproducible, example)
I'm using range to loop through an array of structs to extract data which will be used as a URL parameter for my API calls. Within this loop, I'm trying to push response data from one struct to another.
I'm able to get everything working, except for moving data from one struct to another, but not entirely sure how to solve for the errors I keep getting. I've tried multiple methods and seem to be stuck in the mud here for something I don't consider to be too hard, until now... In my code I'm using the append method but I'm not so sure that might be the correct way to proceed.
Presenting my code:
models.go
//Here is my existing struct, with populated data that I get from a CSV
type TravelItenaries struct {
Origin string
Destination string
Flight_num string
Origin_latitude string
Origin_longitude string
Destination_latitude string
Destination_longitude string
Origin_weather string
Destination_weather string
Coordinates_ori string
Coordinates_dest string
Temp_c_ori string
Temp_f_ori string
Temp_c_dest string
Temp_f_dest string
}
//Here is the response data that I'm expected to get from my API calls.
//I'm trying to "push" Temp_c_dest and Temp_f_dest data into TravelItenaries.Temp_f_dest and TravelItenaries.Temp_c_dest
//While also changing the data types to fit above.
type Response struct {
Current struct {
LastUpdatedEpoch int `json:"last_updated_epoch"`
LastUpdated string `json:"last_updated"`
Temp_c_dest float64 `json:"temp_c"`
Temp_c_dest float64 `json:"temp_f"`
IsDay int `json:"is_day"`
} `json:"current"
}
weather.go
func (s *Server) getWeather(w http.ResponseWriter, r *http.Request) {
// open file
f, err := os.Open("challenge_dataset.csv")
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// remember to close the file at the end of the program
defer f.Close()
// read csv values using csv.Reader
csvReader := csv.NewReader(f)
data, err := csvReader.ReadAll()
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// convert records to array of structs
travelItenaries := createTravelItenaries(data)
// remove duplicate flight records
cleanTravelItenaries:= remDupKeys(travelItenaries)
// set up params for API get request
params := url.Values{
"key": []string{"xxx"},
"q": []string{""},
}
// Construct URL for API request
u := &url.URL{
Scheme: "https",
Host: "api.weatherapi.com",
Path: "/v1/current.json",
RawQuery: params.Encode(),
}
client := &http.Client{}
// Will need this to populate the params using a range over a struct
values := u.Query()
// loop through cleaned data set
for _, service := range cleanTravelItenaries {
// dynamically acquire data from struct to pass as parameter
values.Set("q", service.Coordinates_dest)
u.RawQuery = values.Encode()
req, err := http.NewRequest("GET", u.String(), nil)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
resp, err := client.Do(req)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
body, err := ioutil.ReadAll(resp.Body)
// create empty struct to parse response data with using Inmarshal
var responseData models.Response
json.Unmarshal(body, &responseData)
// Here is the issue, I don't think append might be the correct procedure here?
// I simply just need to pass this response data to my already existing struct
service.Temp_c_dest = append(responseData.Current.Temp_c_dest , cleanTravelItenaries )
service.Temp_f_dest = append(responseData.Current.Temp_f_dest , cleanTravelItenaries )
}
}
The errors I get are related to both append statements at the end of the range function.
first argument to append must be slice; have float64
first argument to append must be slice; have float64
for both append methods.
Also, note how type TravelItenaries struct uses string type for:
Temp_c_dest string
Temp_f_dest string
Hence why I also need to do some field type conversion from Float64 to string.
How can I extract the fields Temp_c_dest and Temp_f_dest from API response struct to TravelItenaries struct fields while changing datatypes?
EDIT:
I've managed to get this somewhat working, but only inside the for loop. The data is not being saved outside the function.
service.Temp_f_dest = strconv.FormatFloat(responseData.Current.Temp_f_dest, 'g', -1, 64)
service.Temp_c_dest = strconv.FormatFloat(responseData.Current.Temp_c_dest, 'g', -1, 64)
I am a little bit confused here and although I have searched a lot on this, something is clearly missing from my knowledge and I am asking your help.
I have created a Hyperledger Fabric Network and installed a chaincode in it. And I want to make a function that retrieves all the World State inputs about the Keys. I have done it already with the bytes.Buffer and it worked. But what I want to do is to do it with a struct.
So, I created the following struct that has only the key:
type WSKeys struct {
Key string `json: "key"`
Namespace string `json: "Namespace"`
}
And this is my code function:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArrayStr []WSKeys
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
var qry_key_json WSKeys
json.Unmarshal([]byte(queryResponse), &qry_key_json)
keyArray = append(keyArray, qry_key_json)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
When executing the above I get the following error:
cannot convert queryResponse (type *queryresult.KV) to type []byte
I can get the results correctly if I, for example do this:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArray []string
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
keyArray = append(keyArray, queryResponse.Key)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
But, why I get the above error when trying to add the queryResponse into a custom struct?
Do I need to add it to a struct that is only its type?
Please someone can explain what I am missing here?
The error statement is verbose enough to indicate, that your []byte conversion failed for the type queryResponse which, with a bit of lookup seems to be a struct type. In Go you cannot natively convert a struct instance to its constituent bytes without encoding using gob or other means.
Perhaps your intention was to use the Key record in the struct for un-marshalling
json.Unmarshal([]byte(queryResponse.Key), &qry_key_json)
I have a use case where I have the code as below. I have a request coming in to hit the backend where I need to append data to a map. My question is how do I convert the below type to a []byte to unmarshal?
Any ideas would be appreciated.
type Example struct {
Category string `json:"category"`
Name string `json:"name"`
}
Incoming Postman request json looks like this:
[{"Category":"TestCategory", "Name":"Sample1"}]
but after doing
jsonString Type: []Example
if err := gc.ShouldBindJSON(&jsonString) it looks like [{TestCategory Sample1}] ; how do I convert this to a []byte?
for _, req := range blob{
var jsonString Example
if err := json.Unmarshal([]byte(jsonString), &blob); err != nil { //this does not work
logger.Fatal(err)
}
//I am checking if a key-value is present and appending it to the map
dict := make(map[string][]Example)
dict[req.Category] = append(dict[req.Category], req)
fmt.Println(dict)
if value, ok := dict["TestCategory"]; ok {
fmt.Printf("Found %d\n", value)
} else {
fmt.Println("not found")
}
}
//I was able to test the above logic by declaring the jsonString as a const and it works
There are two directions in which you can move the data:
from JSON to a Go data structure
// This is your payload coming from the request.
jsonStr := `[{"Category":"TestCategory", "Name":"Sample1"}]`
// This is the Go struct that will hold the unmarshalled data.
var examples []Example
err := json.Unmarshal([]byte(jsonStr), &examples)
if err != nil {
log.Fatal(err)
}
fmt.Println("Examples:", examples) // prints "Examples: [{TestCategory Sample1}]"
from a Go data structure to JSON (either string or []byte)
exampleBytes, err := json.Marshal(examples)
if err != nil {
log.Fatal(err)
}
fmt.Println("Example bytes:", string(exampleBytes)) // prints "Example bytes: [{"category":"TestCategory","name":"Sample1"}]"
You should check out "Go by Example" if you haven't already: https://gobyexample.com/json
Looking at your code:
You are looping on blob but instead of using the req you are trying to unmarshal onto the entire blob each time. I'm not sure what you are trying to achieve there but nothing good can come out of changing a struct you're looping over from within the loop.
The request JSON you are listing is an array of JSON objects. You are trying to unmarshal that into a single Example struct. That won't work, you need an array of those.
I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.