Error unmarshalling message body from amqp - go

I'm trying to send a JSON inside the body of the message using AMQP and receive it. The message sends without any error, however I can't unmarshall it on the other side. I get a unexpected end of JSON input error.
JSON:
var jsonStr = []byte(`
{
"name": "Test",
"phone_number": "12345"
}`)
err = files.SendMessageToCore(jsonStr, "event.saveorder")
if err != nil {
logr.Error("Error sending a message to rabbitmq:", fmt.Sprintf("%v", err))
}
SendMessageToCore function:
func SendMessageToCore(body []byte, routingKey string) error {
pb, err := GetRabbitPublisher()
if err != nil {
return err
}
var message amqp.Message
message.Exchange = "example"
message.RoutingKey = routingKey
message.Body = body
messageByte, err := json.Marshal(message)
if err != nil {
return err
}
err = (*pb).Publish(amqp.Message{Exchange: message.Exchange, RoutingKey: message.RoutingKey, Body: messageByte})
if err != nil {
return err
}
fmt.Println("Message was successfully sent")
return nil
}
Unmarshalling part:
func SaveOrderEventHandler(mes amqp.Message) *amqp.Message {
var incomingMessage amqp.Message
if err := json.Unmarshal(mes.Body, &incomingMessage); err != nil {
fmt.Println(err)
return nil
}
var user foodDomain.User
if err := json.Unmarshal(incomingMessage.Body, &user); err != nil {
fmt.Println("Error unmarshalling incomingMessage body: ", err) //Error appears here
return nil
}
fmt.Println(user.PhoneNumber)
return nil
}
User struct:
type User struct {
Name string `json:"name"`
PhoneNumber string `json:"phone_number"`
}

In your SendMessageToCore you already initialized amqb.Message. No need to re-initialize in the Publish func, just pass your initialized message. Also, your json string body is already in jsonByte, so no need to marshal it once more.
For instance -
func SendMessageToCore(body []byte, routingKey string) error {
pb, err := GetRabbitPublisher()
if err != nil {
return err
}
var message amqp.Message
message.Exchange = "example"
message.RoutingKey = routingKey
message.Body = body
err = (*pb).Publish(message)
if err != nil {
return err
}
fmt.Println("Message was successfully sent")
return nil
}
And your mes already in amqp.Message type, so no need to unmarshal mes.Body into amqp.Message type variable.
Just directly unmarshal your mes.Body into user.
func SaveOrderEventHandler(mes amqp.Message) *amqp.Message {
var user foodDomain.User
if err := json.Unmarshal(mes.Body, &user); err != nil {
fmt.Println(err)
return nil
}
fmt.Println(user.PhoneNumber)
return nil
}

Related

Using or in bind and validation?

I'm using go gin with validation package called ozzo-validation and when I bind json data and validation error like this:
err := c.ShouldBind(&user)
valErrors := user.LoginValidate()
if err != nil || valErrors != nil {
util.ErrorJSON(c, http.StatusBadRequest, err.Error())
return
}
I got errors in console:
/usr/local/go/src/net/http/server.go:2878 +0x43b
net/http.(*conn).serve(0xc0002cebe0, {0x17430c0, 0xc0004a6f30})
/usr/local/go/src/net/http/server.go:1929 +0xb08
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:3033 +0x4e8
when I convert the above code to the following it works!
var user models.User
// Decode request data
if err := c.ShouldBind(&user); err != nil {
util.ErrorJSON(c, http.StatusBadRequest, err.Error())
return
}
// Validate Error
if err := user.LoginValidate(); err != nil {
util.ErrorJSON(c, http.StatusBadRequest, err.Error())
return
}
what is variable user? is it a model? why don't you declare validation on your model with validatin-ozzo like :
func (f *user) Validate() error {
if err := validation.Validate(f.name, validation.Required); err != nil {
return &response.ErrorResponse{
ErrorID: 422,
Msg: map[string]string{
"en": "name cannot be empty",
},
}
}
return nil
}
type ErrorResponse struct {
ErrorID int `json:"error_id"`
Msg map[string]string `json:"message"`
}
func (c *ErrorResponse) Error() string {
b, _ := json.Marshal(c)
return string(b)
}
// and validate it on your code like :
if err := user.Validate(); err != nil {
return nil, err
}

Reuse a method on a struct

I have a method that takes a URL to a yaml file and unmarshals it into a struct. There's nothing unique about it so I'd like to use it on another struct.
type SWConfig struct {
Name string `yaml:"name"`
Version string `yaml:"version"`
BuildType string `yaml:"buildType"`
}
func (c *SWConfig) getConfiguration(url string) {
resp, err := http.Get(url)
if err != nil {
log.Fatalf("ERROR: GET request failed: %s\n", err.Error())
}
if resp.StatusCode != http.StatusOK {
log.Fatalf("ERROR: %v: %s\n", resp.Status, url)
}
source, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("ERROR: could not read response: %s\n", err.Error())
}
err = yaml.Unmarshal(source, c)
if err != nil {
log.Fatalf("ERROR: could not read YAML: %s\n", err.Error())
}
}
var swConfig SWConfig
swConfig.getConfiguration(swURL)
fmt.Println(swConfig.Name)
The other struct would have different fields, but again that shouldn't matter for this method. Is it possible to reuse this method or do I need to convert it to a function?
Use this utility function. It works with a pointer to any type.
// fetchYAML unmarshals the YAML document at url to the value
// pointed to by v.
func fetchYAML(url string, v interface{}) error {
resp, err := http.Get(url)
if err != nil {
return err
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("bad status: %v: %s\n", resp.Status, url)
}
source, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
err = yaml.Unmarshal(source, v)
return err
}
Call it like this:
func (c *SWConfig) getConfiguration(url string) {
if err := fetchYAML(url, c); err != nil {
log.Fatalf("ERROR: get value failed: %s\n", err.Error())
}
}

gob deserialization can not work well on the nested struct

I am writing an rpc call flow. When I use tcp read and write, I found another nested struct problem. I created a request and response object that contains a traceId and struct request object or return object struct, but when I write back the client's response object on the server, I can't get the correct reply on the client gob deserialization The object entity can only get traceId. I want to ask what is the reason for this. The code is as follows
enter image description here
// Server
for {
// Read data from connection
data, err := readData(conn)
if err != nil {
if err.Error() == "EOF" {
return
} else {
log.Printf("Server readData error: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
} else if len(data) == 0 {
continue
}
// Decode the data from client request
var req Request
edcode, err := GetEdcode()
if err != nil {
log.Printf("Server encode error: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
err = edcode.Decode(data, &req)
if err != nil {
log.Printf("Server decode error: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
// Get the service from the registerd service map
methodStr := strings.Split(req.MethodName, ".")
if len(methodStr) != 2 {
log.Printf("Server methodStr length invalid: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
service := server.ServiceMap[methodStr[0]][methodStr[1]]
// Construct the request args
argv, err := req.MakeArgs(edcode, *service)
// Construct reply data
reply := reflect.New(service.ReplyType.Elem())
// Call the method related
function := service.Method.Func
out := function.Call([]reflect.Value{reflect.New(server.ServerType.Elem()), argv, reply})
if out[0].Interface() != nil {
log.Printf("Server out result length invalid: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
// Construct response data
response := NewResponse(req.TraceId, reply.Elem().Interface())
// We should register the matching type related to the GOB encoding to prevent the encoding error
codeErr := response.RegisterGobRespType()
if codeErr != nil {
log.Printf("Server response.RegisterGobRespType eror : %v, remoteAddr:%v", codeErr.Error(), conn.RemoteAddr().String())
return
}
// Encode the reply data to send to the client
//replyData, err := edcode.Encode(reply.Elem().Interface())
replyData, err := edcode.Encode(response)
if err != nil {
log.Printf("Server replayData encode error: %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return
}
// Write data to connection
_, err = writeData(conn, replyData)
if err != nil {
return
}
}
Client
// Receive and read the encoding response data from client
replyData, err := readData(conn)
if err != nil {
log.Printf("Client writeData eror : %v, remoteAddr:%v", err.Error(), conn.RemoteAddr().String())
return err
}
var response etnet.Response;
// Decode data and assign it to reply variable
//var response etnet.Responseļ¼Œ response reply is always nil and I check the write value in server it is correct
edcode.Decode(replyData, &response)
type Request struct {
TraceId string
MethodName string
Args interface{}
}
type Response struct {
TraceId string
Reply interface{}
}
func NewRequest(traceId string, methodName string, args interface{}) *Request {
return &Request{
TraceId: traceId,
MethodName: methodName,
Args: args,
}
}
func NewResponse(traceId string, reply interface{}) *Response {
return &Response{
TraceId: traceId,
Reply: reply,
}
}
// If encode in GOB we should register the type of Args to prevent the GOB encode error
func (request *Request) RegisterGobArgsType() error {
edcodeStr := new(Config).GetEdcodeConf()
switch edcodeStr {
case "gob":
args := reflect.New(reflect.TypeOf(request.Args))
if args.Kind() == reflect.Ptr {
args = args.Elem()
}
gob.Register(args.Interface())
return nil
case "json":
return nil
default:
return errors.New("Unknown edcode protocol: " + edcodeStr)
}
}
// If encode in GOB we should register the type of Args to prevent the GOB encode error
func (response *Response) RegisterGobRespType() error {
edcodeStr := new(Config).GetEdcodeConf()
switch edcodeStr {
case "gob":
reply := reflect.New(reflect.TypeOf(response.Reply))
if reply.Kind() == reflect.Ptr {
reply = reply.Elem()
}
gob.Register(reply.Interface())
return nil
case "json":
return nil
default:
return errors.New("Unknown edcode protocol: " + edcodeStr)
}
}
// Return the reflect.Value type of Args
func (request *Request) MakeArgs(edcode Edcode, service Service) (reflect.Value, error) {
switch edcode.(type) {
case GobEncode:
return reflect.ValueOf(request.Args), nil
case JsonEncode:
reqArgs := request.Args.(map[string]interface{})
argv := reflect.New(service.ArgType)
err := MakeArgType(reqArgs, argv)
if err != nil {
log.Println(err.Error())
return reflect.New(nil), err
}
if argv.Kind() == reflect.Ptr {
argv = argv.Elem()
}
return argv, nil
default:
return reflect.ValueOf(request.Args), errors.New("Unknown edcode")
}
}

Dynamic JSON Decoding

I have this function to parse HTTP results:
func (a *Admin) ResponseDecode(structName string, body io.Reader) (interface{}, error) {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
switch structName {
case "[]Cat":
var data []Cat
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
case "[]Dog":
var data []Dog
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
default:
log.Fatal("Can't decode " + structName)
}
return nil, nil
}
I do a type assertion after this method :
parsed, err := a.ResponseDecode("[]Cat", resp.Body)
if err != nil {
return nil, err
}
return parsed.([]Cat), nil
but how can I do to avoid the repetition of the code :
var data []Stuff
err = json.Unmarshal(content, &data)
if err != nil {
return nil, err
}
return data, err
Every time I add an object? Usually I would use generics, but this is Go. What's the good way to do that ?
You are passing in the name of the struct, and then expecting data of that type. Instead, you can simply pass the struct:
var parsed []Cat
err := a.ResponseDecode(&parsed, resp.Body)
where:
func (a *Admin) ResponseDecode(out interface{}, body io.Reader) error {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
return json.Unmarshal(content,out)
}
In fact, you can get rid of ResponseDecode function:
var parsed []Cat
err:=json.NewDecoder(body).Decode(&parsed)
Found what I was looking for here.
This function does the job :
func (a *Admin) ResponseDecode(body io.Reader, value interface{}) (interface{}, error) {
content, err := ioutil.ReadAll(body)
if err != nil {
return nil, err
}
err = json.Unmarshal(content, &value)
if err != nil {
return nil, err
}
return value, nil}

How to refactor semantic duplication

I have defined two funcs that do slightly different things but are syntactically the same.
Functions in question send POST requests to an api.
The duplication occurs in constructing the request, adding headers, etc.
How can I refactor the code to remove said duplication.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httputil"
)
type token struct {
Token string
}
type config struct {
Foo string
}
func main() {
token, err := getAuthToken()
if err != nil {
log.Fatal(err)
}
config, err := getConfig("foo", token)
if err != nil {
log.Fatal(err)
}
_ = config
}
func getAuthToken() (string, error) {
endpoint := "foo"
body := struct {
UserName string `json:"username"`
Password string `json:"password"`
}{
UserName: "foo",
Password: "bar",
}
jsnBytes, err := json.Marshal(body)
if err != nil {
return "", err
}
req, err := http.NewRequest("POST", endpoint, bytes.NewReader(jsnBytes))
if err != nil {
return "", fmt.Errorf("Unable to create request. %v", err)
}
req.Header.Add("Content-Type", "application/json")
dump, err := httputil.DumpRequest(req, true)
if err != nil {
return "", fmt.Errorf("Could not dump request. ", err)
}
log.Println("Request: ", string(dump))
client := http.Client{}
log.Println("Initiating http request")
resp, err := client.Do(req)
if err != nil {
return "", fmt.Errorf("HTTP Error: %v", err)
}
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("Error reading response body: %v", err)
}
var token token
err = json.Unmarshal(bytes, &token)
if err != nil {
return "", fmt.Errorf("Could not unamrshal json. ", err)
}
return token.Token, nil
}
func getConfig(id string, token string) (*config, error) {
endpoint := "foo"
body := struct {
ID string `json:"id"`
}{
ID: id,
}
jsnBytes, err := json.Marshal(body)
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", endpoint, bytes.NewReader(jsnBytes))
if err != nil {
return nil, fmt.Errorf("Unable to create request. %v", err)
}
req.Header.Add("Authorization", "Bearer "+token)
req.Header.Add("Content-Type", "application/json")
dump, err := httputil.DumpRequest(req, true)
if err != nil {
return nil, fmt.Errorf("Could not dump request. ", err)
}
log.Println("Request: ", string(dump))
client := http.Client{}
log.Println("Initiating http request")
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("HTTP Error: %v", err)
}
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("Error reading response body: %v", err)
}
var config config
err = json.Unmarshal(bytes, &config)
if err != nil {
return nil, fmt.Errorf("Could not unamrshal json. ", err)
}
return &config, nil
}
I would say the essence of sending the request is that you are sending a body to an endpoint and parsing a result. The headers are then optional options that you can add to the request along the way. With this in mind I would make a single common function for sending the request with this signature:
type option func(*http.Request)
func sendRequest(endpoint string, body interface{}, result interface{}, options ...option) error {
Note this is using functional options which Dave Cheney did an excellent description of here:
https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
The complete code then becomes:
https://play.golang.org/p/GV6FeipIybA
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/http/httputil"
)
type token struct {
Token string
}
type config struct {
Foo string
}
func main() {
token, err := getAuthToken()
if err != nil {
log.Fatal(err)
}
config, err := getConfig("foo", token)
if err != nil {
log.Fatal(err)
}
_ = config
}
func getAuthToken() (string, error) {
endpoint := "foo"
body := struct {
UserName string `json:"username"`
Password string `json:"password"`
}{
UserName: "foo",
Password: "bar",
}
var token token
err := sendRequest(endpoint, body, &token)
if err != nil {
return "", err
}
return token.Token, nil
}
func getConfig(id string, token string) (*config, error) {
endpoint := "foo"
body := struct {
ID string `json:"id"`
}{
ID: id,
}
var config config
err := sendRequest(endpoint, body, &config, header("Content-Type", "application/json"))
if err != nil {
return nil, err
}
return &config, nil
}
type option func(*http.Request)
func header(key, value string) func(*http.Request) {
return func(req *http.Request) {
req.Header.Add(key, value)
}
}
func sendRequest(endpoint string, body interface{}, result interface{}, options ...option) error {
jsnBytes, err := json.Marshal(body)
if err != nil {
return err
}
req, err := http.NewRequest("POST", endpoint, bytes.NewReader(jsnBytes))
if err != nil {
return fmt.Errorf("Unable to create request. %v", err)
}
req.Header.Add("Content-Type", "application/json")
for _, option := range options {
option(req)
}
dump, err := httputil.DumpRequest(req, true)
if err != nil {
return fmt.Errorf("Could not dump request. ", err)
}
log.Println("Request: ", string(dump))
client := http.Client{}
log.Println("Initiating http request")
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("HTTP Error: %v", err)
}
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("Error reading response body: %v", err)
}
err = json.Unmarshal(bytes, result)
if err != nil {
return fmt.Errorf("Could not unamrshal json. ", err)
}
return nil
}
The way I would do this is to extract the two parts that are common to both request executions: 1) create a request and 2) execute the request.
Gist with new code using HTTP Bin as an example
Creating the request includes setting up the endpoint, headers and marshaling the request body to JSON. In your case, you're also dumping the request to the log, that can also go in there. This is how it would look like:
func buildRequest(endpoint string, body interface{}, extraHeaders map[string]string) (*http.Request, error) {
jsnBytes, err := json.Marshal(body)
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", endpoint, bytes.NewReader(jsnBytes))
if err != nil {
return nil, err
}
req.Header.Add("Content-Type", "application/json")
for name, value := range extraHeaders {
req.Header.Add(name, value)
}
dump, err := httputil.DumpRequest(req, true)
if err != nil {
return nil, err
}
log.Println("Request: ", string(dump))
return req, nil
}
If you have no extra headers, you can pass nil as the third argument here.
The second part to extract is actually executing the request and unmarshalling the data. This is how the executeRequest would look like:
func executeRequest(req *http.Request, responseBody interface{}) error {
client := http.Client{}
log.Println("Initiating http request")
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
log.Printf("Response is: %s\n", string(bytes))
err = json.Unmarshal(bytes, &responseBody)
return err
}

Resources