Complete tracing using zipkin-go - go

I have some services. I want to trace those services using zipkin-go. In every service, I am calling some of my other internal services or db calls. I want to trace every activity like how much time it has taken to call internal services or db. I have implemented using available tutorials on internet. Below is my code:
package main
import (
"fmt"
"os"
"net/http"
"log"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
"Upload-Image-API/services/controllers/updatemodrefid"
"Upload-Image-API/services/controllers/uploadimage"
"Upload-Image-API/services/utils"
"github.com/openzipkin/zipkin-go"
"github.com/openzipkin/zipkin-go/model"
zipkinhttp "github.com/openzipkin/zipkin-go/middleware/http"
reporterhttp "github.com/openzipkin/zipkin-go/reporter/http"
)
const endpointURL = "http://localhost:9411/api/v2/spans"
func newTracer() (*zipkin.Tracer, error) {
// The reporter sends traces to zipkin server
reporter := reporterhttp.NewReporter(endpointURL)
port := utils.GetString("serviceListeningPort")
if port == "" {
fmt.Println("Port is not defined in configuration ...!!!")
os.Exit(1)
}
// Local endpoint represent the local service information
localEndpoint := &model.Endpoint{ServiceName: "Upload-Image-API", Port: 7795}
// Sampler tells you which traces are going to be sampled or not. In this case we will record 100% (1.00) of traces.
sampler, err := zipkin.NewCountingSampler(1)
if err != nil {
return nil, err
}
t, err := zipkin.NewTracer(
reporter,
zipkin.WithSampler(sampler),
zipkin.WithLocalEndpoint(localEndpoint),
)
if err != nil {
return nil, err
}
return t, err
}
func main() {
var err error
var environment = os.Args[1]
err = utils.LoadDefaultConfig(environment)
tracer, err := newTracer()
if err != nil {
log.Fatal(err)
}
// We add the instrumented transport to the defaultClient
// that comes with the zipkin-go library
http.DefaultClient.Transport, err = zipkinhttp.NewTransport(
tracer,
zipkinhttp.TransportTrace(true),
)
if err != nil {
log.Fatal(err)
}
if err != nil {
fmt.Println("Could not load the configuration because of following err " + err.Error() + " ...!!!")
os.Exit(1)
} else {
fmt.Println("Configurations sucessfully loaded for environment " + environment + " ...!!!")
}
allowedHeaders := handlers.AllowedHeaders([]string{"X-Requested-With"})
allowedOrigins := handlers.AllowedOrigins([]string{"*"})
allowedMethods := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "DELETE", "OPTIONS"})
port := utils.GetString("serviceListeningPort")
if port == "" {
fmt.Println("Port is not defined in configuration ...!!!")
os.Exit(1)
}
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/uploadimage", uploadimage.UploadImages).Methods("POST", "GET")
router.HandleFunc("/updatemodrefid", updatemodrefid.UpdateModRefId).Methods("POST", "GET")
router.Use(zipkinhttp.NewServerMiddleware(
tracer,
zipkinhttp.SpanName("request")), // name for request span
)
err = http.ListenAndServe(port, handlers.CORS(allowedHeaders, allowedOrigins, allowedMethods)(router))
if err != nil {
fmt.Println("Count not start the server because of following err " + err.Error())
return
}
}
I am getting my request traced but I am not able to trace what is happening inside the uploadimage controller. Below is the screenshot of my zipkin UI:
I want to trace all the activities happening inside uploadimage controller. What should I need to pass so that I can trace all.

I am new to zipkin and golang, If you want to trace internal process, then you can create span from context
example: say you have api called Login, inside login you might perform database operation or any other operations
func Login(res http.ResponseWriter, req *http.Request) {
span, _ := tracer.StartSpanFromContext(req.Context(), "database")
databaseOperation()
span.Finish()
}

Related

Google Cloud Vertex AI with Golang: rpc error: code = Unimplemented desc = unexpected HTTP status code received from server: 404 (Not Found)

I have a Vertex AI model deployed on an endpoint and want to do some prediction from my app in Golang.
To do this I create code inspired by this example : https://cloud.google.com/go/docs/reference/cloud.google.com/go/aiplatform/latest/apiv1?hl=en
const file = "MY_BASE64_IMAGE"
func main() {
ctx := context.Background()
c, err := aiplatform.NewPredictionClient(cox)
if err != nil {
log.Printf("QueryVertex NewPredictionClient - Err:%s", err)
}
defer c.Close()
parameters, err := structpb.NewValue(map[string]interface{}{
"confidenceThreshold": 0.2,
"maxPredictions": 5,
})
if err != nil {
log.Printf("QueryVertex structpb.NewValue parameters - Err:%s", err)
}
instance, err := structpb.NewValue(map[string]interface{}{
"content": file,
})
if err != nil {
log.Printf("QueryVertex structpb.NewValue instance - Err:%s", err)
}
reqP := &aiplatformpb.PredictRequest{
Endpoint: "projects/PROJECT_ID/locations/LOCATION_ID/endpoints/ENDPOINT_ID",
Instances: []*structpb.Value{instance},
Parameters: parameters,
}
resp, err := c.Predict(cox, reqP)
if err != nil {
log.Printf("QueryVertex Predict - Err:%s", err)
}
log.Printf("QueryVertex Res:%+v", resp)
}
I put the path to my service account JSON file on GOOGLE_APPLICATION_CREDENTIALS environment variable.
But when I run my test app I obtain this error message:
QueryVertex Predict - Err:rpc error: code = Unimplemented desc = unexpected HTTP status code received from server: 404 (Not Found); transport: received unexpected content-type "text/html; charset=UTF-8"
QueryVertex Res:<nil>
As #DazWilkin suggested, configure the client option to specify the specific regional endpoint with a port 443:
option.WithEndpoint("<region>-aiplatform.googleapis.com:443")
Try like below:
func main() {
ctx := context.Background()
c, err := aiplatform.NewPredictionClient(
ctx,
option.WithEndpoint("<region>-aiplatform.googleapis.com:443"),
)
if err != nil {
log.Printf("QueryVertex NewPredictionClient - Err:%s", err)
}
defer c.Close()
.
.
I'm unfamiliar with Google's (Vertex?) AI Platform and unable to test this hypothesis but it appears that the API uses location-specific endpoints.
Can you try configuring the client's ClientOption to specify the specific regional endpoint, i.e.:
url := fmt.Sprintf("https://%s-aiplatform.googleapis.com", location)
opts := []option.ClientOption{
option.WithEndpoint(url),
}
And:
package main
import (
"context"
"fmt"
"log"
"os"
aiplatform "cloud.google.com/go/aiplatform/apiv1"
"google.golang.org/api/option"
aiplatformpb "google.golang.org/genproto/googleapis/cloud/aiplatform/v1"
"google.golang.org/protobuf/types/known/structpb"
)
const file = "MY_BASE64_IMAGE"
func main() {
// Values from the environment
project := os.Getenv("PROJECT")
location := os.Getenv("LOCATION")
endpoint := os.Getenv("ENDPOINT")
ctx := context.Background()
// Configure the client with a region-specific endpoint
url := fmt.Sprintf("https://%s-aiplatform.googleapis.com", location)
opts := []option.ClientOption{
option.WithEndpoint(url),
}
c, err := aiplatform.NewPredictionClient(ctx, opts...)
if err != nil {
log.Fatal(err)
}
defer c.Close()
parameters, err := structpb.NewValue(map[string]interface{}{
"confidenceThreshold": 0.2,
"maxPredictions": 5,
})
if err != nil {
log.Fatal(err)
}
instance, err := structpb.NewValue(map[string]interface{}{
"content": file,
})
if err != nil {
log.Printf("QueryVertex structpb.NewValue instance - Err:%s", err)
}
rqst := &aiplatformpb.PredictRequest{
Endpoint: fmt.Sprintf("projects/%s/locations/%s/endpoints/%s",
project,
location,
endpoint,
),
Instances: []*structpb.Value{
instance,
},
Parameters: parameters,
}
resp, err := c.Predict(ctx, rqst)
if err != nil {
log.Fatal(err)
}
log.Printf("QueryVertex Res:%+v", resp)
}
Try to do something like this
[...]
url := fmt.Sprintf("%s-aiplatform.googleapis.com:443", location)
[..]

Using Colly framework I can't login to the Evernote account

I am using colly framework for scraping the website. Am trying to login the Evernote account for scraping some things. But I can't go through it. I used "username" and "password" titles for giving the credentials. Is this the right way ?.
Thank you in advance.
package main
import (
"log"
"github.com/gocolly/colly"
)
func main() {
// create a new collector
c := colly.NewCollector()
// authenticate
err := c.Post("https://www.evernote.com/Login.action",
map[string]string{"username":
"XXXXXX#XXX.com", "password": "*********"})
if err != nil {
log.Fatal("Error : ",err)
}
// attach callbacks after login
c.OnResponse(func(r *colly.Response) {
log.Println("response received", r.StatusCode)
})
// start scraping
c.Visit("https://www.evernote.com/")
}
You should try to mimic the browser behavior, take a look at this implementation, I've added comments on each step:
package evernote
import (
"bytes"
"errors"
"fmt"
"io/ioutil"
"net/http"
"net/http/cookiejar"
"net/url"
"regexp"
"strings"
)
const (
evernoteLoginURL = "https://www.evernote.com/Login.action"
)
var (
evernoteJSParamsExpr = regexp.MustCompile(`document.getElementById\("(.*)"\).value = "(.*)"`)
evernoteRedirectExpr = regexp.MustCompile(`Redirecting to <a href="(.*)">`)
errNoMatches = errors.New("No matches")
errRedirectURL = errors.New("Redirect URL not found")
)
// EvernoteClient wraps all methods required to interact with the website.
type EvernoteClient struct {
Username string
Password string
httpClient *http.Client
// These parameters persist during the login process:
hpts string
hptsh string
}
// NewEvernoteClient initializes a new Evernote client.
func NewEvernoteClient(username, password string) *EvernoteClient {
// Allocate a new cookie jar to mimic the browser behavior:
cookieJar, _ := cookiejar.New(nil)
// Fill up basic data:
c := &EvernoteClient{
Username: username,
Password: password,
}
// When initializing the http.Client, copy default values from http.DefaultClient
// Pass a pointer to the cookie jar that was created earlier:
c.httpClient = &http.Client{
Transport: http.DefaultTransport,
CheckRedirect: http.DefaultClient.CheckRedirect,
Jar: cookieJar,
Timeout: http.DefaultClient.Timeout,
}
return c
}
func (e *EvernoteClient) extractJSParams(body []byte) (err error) {
matches := evernoteJSParamsExpr.FindAllSubmatch(body, -1)
if len(matches) == 0 {
return errNoMatches
}
for _, submatches := range matches {
if len(submatches) < 3 {
err = errNoMatches
break
}
key := submatches[1]
val := submatches[2]
if bytes.Compare(key, hptsKey) == 0 {
e.hpts = string(val)
}
if bytes.Compare(key, hptshKey) == 0 {
e.hptsh = string(val)
}
}
return nil
}
// Login handles the login action.
func (e *EvernoteClient) Login() error {
// First step: fetch the login page as a browser visitor would do:
res, err := e.httpClient.Get(evernoteLoginURL)
if err != nil {
return err
}
if res.Body == nil {
return errors.New("No response body")
}
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
err = e.extractJSParams(body)
if err != nil {
return err
}
// Second step: we have extracted the "hpts" and "hptsh" parameters
// We send a request using only the username and setting "evaluateUsername":
values := &url.Values{}
values.Set("username", e.Username)
values.Set("evaluateUsername", "")
values.Set("analyticsLoginOrigin", "login_action")
values.Set("clipperFlow", "false")
values.Set("showSwitchService", "true")
values.Set("hpts", e.hpts)
values.Set("hptsh", e.hptsh)
rawValues := values.Encode()
req, err := http.NewRequest(http.MethodPost, evernoteLoginURL, bytes.NewBufferString(rawValues))
if err != nil {
return err
}
req.Header.Set("Accept", "application/json")
req.Header.Set("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8")
req.Header.Set("x-requested-with", "XMLHttpRequest")
req.Header.Set("referer", evernoteLoginURL)
res, err = e.httpClient.Do(req)
if err != nil {
return err
}
body, err = ioutil.ReadAll(res.Body)
if err != nil {
return err
}
bodyStr := string(body)
if !strings.Contains(bodyStr, `"usePasswordAuth":true`) {
return errors.New("Password auth not enabled")
}
// Third step: do the final request, append password to form data:
values.Del("evaluateUsername")
values.Set("password", e.Password)
values.Set("login", "Sign in")
rawValues = values.Encode()
req, err = http.NewRequest(http.MethodPost, evernoteLoginURL, bytes.NewBufferString(rawValues))
if err != nil {
return err
}
req.Header.Set("Accept", "text/html")
req.Header.Set("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8")
req.Header.Set("x-requested-with", "XMLHttpRequest")
req.Header.Set("referer", evernoteLoginURL)
res, err = e.httpClient.Do(req)
if err != nil {
return err
}
// Check the body in order to find the redirect URL:
body, err = ioutil.ReadAll(res.Body)
if err != nil {
return err
}
bodyStr = string(body)
matches := evernoteRedirectExpr.FindAllStringSubmatch(bodyStr, -1)
if len(matches) == 0 {
return errRedirectURL
}
m := matches[0]
if len(m) < 2 {
return errRedirectURL
}
redirectURL := m[1]
fmt.Println("Login is ok, redirect URL:", redirectURL)
return nil
}
After you successfully get the redirect URL, you should be able to send authenticated requests as long as you keep using the HTTP client that was used for the login process, the cookie jar plays a very important role here.
To call this code use:
func main() {
evernoteClient := NewEvernoteClient("user#company", "password")
err := evernoteClient.Login()
if err != nil {
panic(err)
}
}

gRPC - GoLang - Stackdriver tracer

I am trying to get the stackdriver tracer to work with gRPC and I need some help. I have been looking at these two links for reference and I still can't get it to work:
https://medium.com/#harlow/tracing-grpc-calls-in-golang-with-google-stackdriver-b22495763a06#.81oa9q21v
https://rakyll.org/grpc-trace/
For simplicity, I am just working with the hello world gRPC example. Here's my client:
func main() {
// Set up a connection to the server.
conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure(), grpc.WithUnaryInterceptor(grpc.UnaryClientInterceptor(clientInterceptor)))
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewGreeterClient(conn)
ctx := context.Background()
tc, err := trace.NewClient(ctx, "{PROJECT-ID}")
if err != nil {
log.Fatal(err)
}
span := tc.NewSpan("/greeter/SayHello")
defer span.Finish()
ctx = trace.NewContext(ctx, span)
r, err := c.SayHello(ctx, &pb.HelloRequest{Name: "world"})
if err != nil {
log.Fatalf("could not greet: %v", err)
}
println("Response:", r.Message)
}
func clientInterceptor(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
// trace current request w/ child span
span := trace.FromContext(ctx).NewChild(method)
defer span.Finish()
// new metadata, or copy of existing
md, ok := metadata.FromContext(ctx)
if !ok {
md = metadata.New(nil)
} else {
md = md.Copy()
}
// append trace header to context metadata
// header specification: https://cloud.google.com/trace/docs/faq
md["X-Cloud-Trace-Context"] = append(
md["X-Cloud-Trace-Context"], fmt.Sprintf("%s/%d;o=1", span.TraceID(), 0),
)
ctx = metadata.NewContext(ctx, md)
return invoker(ctx, method, req, reply, cc, opts...)
}
.. and my gRPC server:
// server is used to implement helloworld.GreeterServer.
type server struct{}
// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
println("HERE")
return &pb.HelloReply{Message: "Hello " + in.Name}, nil
}
func main() {
lis, err := net.Listen("tcp", ":50051")
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
ctx := context.Background()
tc, err := trace.NewClient(ctx, "{PROJECT-ID}")
if err != nil {
log.Fatal(err)
}
s := grpc.NewServer(EnableGRPCTracingServerOption(tc))
pb.RegisterGreeterServer(s, &server{})
println("listening on :50051")
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
// EnableGRPCTracingServerOption enables parsing google trace header from metadata
// and adds a new child span to the incoming request context.
func EnableGRPCTracingServerOption(traceClient *trace.Client) grpc.ServerOption {
return grpc.UnaryInterceptor(serverInterceptor(traceClient))
}
func serverInterceptor(traceClient *trace.Client) grpc.UnaryServerInterceptor {
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (resp interface{}, err error) {
// fetch metadata from request context
md, ok := metadata.FromContext(ctx)
if !ok {
md = metadata.New(nil)
}
header := strings.Join(md["X-Cloud-Trace-Context"], "")
// create new child span from google trace header, add to
// current request context
span := traceClient.SpanFromHeader(info.FullMethod, header)
defer span.Finish()
ctx = trace.NewContext(ctx, span)
return handler(ctx, req)
}
}
I when I run the client to initiate the trace, I get the error:
rpc error: code = 13 desc = stream terminated by RST_STREAM with error code: 1
I'm confused because I don't see anything else about authentication; only providing the project ID which can't be enough to initiate tracing for a specific project. What am I missing?
The issue was with:
defer span.Finish()
That call does not block so because I was just doing preliminary testing with one call my program was exiting before the traces could be uploaded. I contacted the author of https://rakyll.org/grpc-trace/ and she actually updated her post with the option of using:
defer span.FinishWait()
which blocks and that fixed it by allowing the traces to be successfully uploaded before the program exited.
Also, with a long running webserver this wouldn't have been an issue because the process wouldn't have been terminated.
I followed those same tutorials and ran into similar problems.
Header keys are converted to lowercase. If you retrieve it on the server side with header := strings.Join(md["x-cloud-trace-context"], "") you should be good.
You can also define your metadata headers with:
span := trace.FromContext(ctx).NewChild(method)
defer span.Finish()
md := metadata.Pairs(
"x-cloud-trace-context", fmt.Sprintf("%s/%d;o=1", span.TraceID(), 0),
)
ctx = metadata.NewContext(ctx, md)

rpc.ServerCodec Still Serving?

I was performing some RPC tests, and stumbled across a problem I can't seem to solve. In my testing I create three separate RPC servers, all of which I try to close and shutdown. However upon performing my last test (TestRpcCodecServerClientComm), it seems my client connection is connecting to the first RPC server I started (I know this because I at some point attached IDs to the RPCHandlers), even though I attempted everything I could to make sure it was shutdown. Though the code is not there I have attempted to inspect every single error I could, but that did not bring about anything.
rpc.go
package rbot
import (
"io"
"net"
"net/rpc"
"net/rpc/jsonrpc"
)
func RpcCodecClientWithPort(port string) (rpc.ClientCodec, error) {
conn, err := net.Dial("tcp", "localhost:"+port)
if err != nil {
return nil, err
}
return jsonrpc.NewClientCodec(conn), nil
}
func RpcCodecServer(conn io.ReadWriteCloser) rpc.ServerCodec {
return jsonrpc.NewServerCodec(conn)
}
rpc_test.go
package rbot
import (
"errors"
"fmt"
"net"
"net/rpc"
"testing"
)
type RPCHandler struct {
RPCServer net.Listener
conn rpc.ServerCodec
done chan bool
TestPort string
stop bool
GotRPC bool
}
func (r *RPCHandler) SetupTest() {
r.stop = false
r.GotRPC = false
r.done = make(chan bool)
r.TestPort = "5556"
}
// TODO: Create separate function to handle erroring
func (r *RPCHandler) CreateRPCServer() error {
rpc.RegisterName("TestMaster", TestAPI{r})
var err error
r.RPCServer, err = net.Listen("tcp", ":"+r.TestPort)
if err != nil {
return err
}
go func() {
for {
conn, err := r.RPCServer.Accept()
if err != nil || r.stop {
r.done <- true
return
}
r.conn = RpcCodecServer(conn)
rpc.ServeCodec(r.conn)
}
}()
return nil
}
func (r *RPCHandler) CloseRPCServer() error {
r.stop = true
if r.conn != nil {
err := r.conn.Close()
if err != nil {
fmt.Println(err)
}
}
err := r.RPCServer.Close()
<-r.done
return err
}
type TestAPI struct {
t *RPCHandler
}
func (tapi TestAPI) Send(msg string, result *string) error {
if msg == "Got RPC?" {
tapi.t.GotRPC = true
return nil
}
return errors.New("Didn't receive right message")
}
// Check if we can create and close an RPC server successfully using the RPC server codec.
func TestRpcCodecServer(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
err := r.CreateRPCServer()
if err != nil {
t.Fatalf("Could not create rpc server! %s:", err.Error())
}
err = r.CloseRPCServer()
if err != nil {
t.Fatalf("Could not close RPC server! %s:", err.Error())
}
}
// Check if we can create a client without erroring.
func TestRpcCodecClientWithPortt(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
r.CreateRPCServer()
defer r.CloseRPCServer()
RPCClient, err := RpcCodecClientWithPort(r.TestPort)
defer RPCClient.Close()
if err != nil {
t.Fatalf("Could not create an RPC client! %s:", err.Error())
}
}
// Let's double check and make sure our server and client can speak to each other
func TestRpcCodecServerClientComm(t *testing.T) {
r := RPCHandler{}
r.SetupTest()
r.CreateRPCServer()
defer r.CloseRPCServer()
RPCCodec, _ := RpcCodecClientWithPort(r.TestPort)
RPCClient := rpc.NewClientWithCodec(RPCCodec)
defer RPCClient.Close()
var result string
err := RPCClient.Call("TestMaster.Send", "Got RPC?", &result)
if err != nil {
t.Fatalf("Error while trying to send RPC message: %s", err.Error())
}
if !r.GotRPC {
t.Fatalf("Could not send correct message over RPC")
}
}
Not sure if I'm just mishandling the connection or something of the like, any help would be much appreciated.
For the Record The RPC api does receive the correct string message
While not the source of your problems, your test configuration has a few race conditions which you should take care of before they cause problems. Always check for issues with the -race option. You should also let the OS allocate the port so you don't run into conflicts. See for example how httptest.Server works.
Your failure here is that you're not creating a new rpc.Server for each test, you're reusing the rpc.DefaultServer. The first call to CreateRPCServer registers a TestAPI under the name TestMaster. Each subsequent call uses the already registered instance.
If you create a new rpc.Server each time you setup the test and register a new TestAPI, the final test will pass.
srv := rpc.NewServer()
srv.RegisterName("TestMaster", testAPI)
...
// and then use srv to handle the new connection
srv.ServeCodec(RpcCodecServer(conn))

Go Error handling on a REST API

I am very new to go and have deployed a small service with an API endpoint.
I have heard/read that go doesn't use try/catch so I am trying to figure out how I can "catch" any problems happening from my service call from my API and make sure that the resource server doesn't go down.
My code for my API looks like the following..
I have a routes.go file with the following
package main
import (
"net/http"
"github.com/gorilla/mux"
)
type Route struct {
Name string
Method string
Pattern string
HandlerFunc http.HandlerFunc
}
type Routes []Route
func NewRouter() *mux.Router {
router := mux.NewRouter().StrictSlash(true)
for _, route := range routes {
router.
Methods(route.Method).
Path(route.Pattern).
Name(route.Name).
Handler(route.HandlerFunc)
}
return router
}
var routes = Routes{
Route{
"CustomerLocationCreate",
"POST",
"/tracking/customer",
CustomerLocationCreate,
},
}
I have a handlers.go
package main
import (
"encoding/json"
"net/http"
"io"
"io/ioutil"
)
//curl -H "Content-Type: application/json" -d '{"userId":"1234"}' http://localhost:8181/tracking/customer
func CustomerLocationCreate(w http.ResponseWriter, r *http.Request) {
var location CustomerLocation
body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))
if err != nil {
panic(err)
}
if err := r.Body.Close(); err != nil {
panic(err)
}
if err := json.Unmarshal(body, &location); err != nil {
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.WriteHeader(422) // unprocessable entity
if err := json.NewEncoder(w).Encode(err); err != nil {
panic(err)
}
}
c := RepoCreateCustomerLocation(location)
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.WriteHeader(http.StatusCreated)
if err := json.NewEncoder(w).Encode(c); err != nil {
panic(err)
}
HandleCustomerLocationChange(c);
}
and I have a bus.go which has the HandleCustomerLocationChange(...) function.
func HandleCustomerLocationChange(custLoc CustomerLocation) {
endpoint := og.Getenv("RABBIT_ENDPOINT")
conn, err := amqp.Dial("amqp://guest:guest#" + endpoint)
failOnError(err, "Failed to connect to RabbitMQ")
defer conn.Close()
ch, err := conn.Channel()
failOnError(err, "Failed to open a channel")
defer ch.Close()
topic := "locationChange"
err = ch.ExchangeDeclare(
topic, // name
"topic", // type
true, // durable
false, // auto-deleted
false, // internal
false, // no-wait
nil, // arguments
)
failOnError(err, "Failed to declare an exchange")
// Create JSON from the instance data.
body, _ := json.Marshal(custLoc)
// Convert bytes to string.
err = ch.Publish(
topic, // exchange
"", // routing key
false, // mandatory
false, // immediate
amqp.Publishing{
ContentType: "text/plain",
Body: body,
})
failOnError(err, "Failed to publish a message")
log.Printf(" [x] Sent %s", body)
}
My question is how should I modify both the HandleCustomerLocationChange(...) function and if necessaryCustomerLocationChange(..)` handler to handle errors properly so that if an error occurs, my entire API doesn't go down?
Go suggests a different approach, that errors are not exceptional, they're normal events, just less common.
Taking an example from the code above:
body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))
if err != nil {
panic(err)
}
Here, a panic (without recovery) terminates the process, shutting down the web server. Seems an overly severe response to not fully reading a request.
What do you want to do? It may be appropriate to tell the client who made the request:
body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
You might want to return a json encoded response, or give a generic message to the client avoid exposing too much, and log the specific error details.
For general functions it's idiomatic to return the error as the last return parameter. In the specific example you mentioned:
func HandleCustomerLocationChange(custLoc CustomerLocation)
...
conn, err := amqp.Dial(...)
failOnError(err, "Failed to connect to RabbitMQ")
Instead, check if the connection failed, and return the error to the caller. Handle it in the calling function, or add information and propagate it up the call stack.
func HandleCustomerLocationChange(custLoc CustomerLocation) error
...
conn, err := amqp.Dial(...)
if err != nil {
return fmt.Errorf("failed to connect to RabbitMQ: %s", err)
}
Propagating the error in this way gives a concise explanation of the root cause, like the 5 whys technique, eg:
"did not update client location: did not connect to rabbitmq: network address 1.2.3 unreachable"
Another convention is to deal with errors first and return early. This helps to reduce nesting.
See also the many error handling resources, like error handling in a web application, Go by Example, Error Handling and Go, errors are values and Defer, Panic & Recover. The source code of the error package is interesting, as is Russ Cox's comment on error handling, and Nathan Youngman's To Err is Human.
Also interesting is Upspin's concept of an operational trace, rather than a stack trace.

Resources