HttpMock is not intercepting Resty call - go

I have a function that calls an external api that I want to mock out in the test.
func ApiWrapper(...) (...) {
client := resty.New()
var r apiResponse
apiPath := "..." // In the test will be http://localhost:{PORT}/path/to/endpoint
_, e := client.R().SetResult(&r).Get(apiPath)
...
return ...
}
The test looks like this:
func TestApiWrapper(t *testing.T) {
client := resty.New()
httpmock.ActivateNonDefault(client.GetClient())
defer httpmock.DeactivateAndReset()
mock_resp = `...`
responder := httpmock.NewStringResponder(200, mock_resp)
api_url := "same string used in the function"
httpmock.RegisterResponder("GET", api_url, responder)
res, e := ApiWrapper(...)
...
}
The issue I'm having is that the mock is not being used also the external api will not be available in our CI.
In the test the client has:
httpClient: *net/http.Client {
Transport: net/http.RoundTripper(*github.com/jarcoal/httpmock.MockTransport)
In the function the client has:
httpClient: *net/http.Client {
Transport: net/http.RoundTripper(*net/http.Transport)

I was able to get around the problem by using a function to inject the resty Client. Don't really like this approach as it leaves a couple of lines of code that are not executed during my test.
func ApiWrapper(...) {
client := resty.New()
resp, err := ApiWrapperWrapper(client)
return resp, err
}
func ApiWrapperWrapper(client *resty.Client) (...) {
copy all the code into here
}
Then in my test I just call ApiWrapperWrapper and pass in the mocked client.
func TestApiWrapper(...) {
...
// change last line in example to this
res, e := ApiWrapperWrapper(client)
}

Related

Manually extracting OpenTelemetry context from golang into a string?

I'm building a simple client server app which I want to trace across the client execution to a server microservice that calls a second server microservice.
Simply speaking, it's not more complicated than CLI -> ServiceA -> ServiceB.
The challenge I'm having is how to serialize the context - most of the docs I've looked at appear to do some form of automated HTTP header injection (e.g. https://opentelemetry.lightstep.com/core-concepts/context-propagation/) , but I do not have access to that. I need to serialize (I think) the context of the trace/span in the client and push it to the server, where I'll rehydrate it. (Mind you, I'd love this to be simpler, but I cannot figure it out).
So the object looks like this (called "job"):
args := &types.SubmitArgs{
SerializedOtelContext: serializedOtelContext,
}
job := &types.Job{}
tracer := otel.GetTracerProvider().Tracer("myservice.org")
_, span := tracer.Start(ctx, "Submitting Job to RPC")
err := system.JsonRpcMethod(rpcHost, rpcPort, "Submit", args, job)
The function to submit to JsonRpcMethod is here:
func JsonRpcMethod(
host string,
port int,
method string,
req, res interface{},
) error {
client, err := rpc.DialHTTP("tcp", fmt.Sprintf("%s:%d", host, port))
if err != nil {
return fmt.Errorf("Error in dialing. %s", err)
}
return client.Call(fmt.Sprintf("JobServer.%s", method), req, res)
}
And the function that receives it is here:
func (server *JobServer) Submit(args *types.SubmitArgs, reply *types.Job) error {
//nolint
job, err := server.RequesterNode.Scheduler.SubmitJob(args.Spec, args.Deal)
if err != nil {
return err
}
*reply = *job
return nil
}
My question is how do I, in the receiving function ("Submit" above) extract the trace/span from the sender?
Here is a small program to illustrate the usage. Hope this makes it clear.
package main
import (
"context"
"fmt"
"go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
"go.opentelemetry.io/otel/propagation"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
)
func main() {
// common init
// You may also want to set them as globals
exp, _ := stdouttrace.New(stdouttrace.WithPrettyPrint())
bsp := sdktrace.NewSimpleSpanProcessor(exp) // You should use batch span processor in prod
tp := sdktrace.NewTracerProvider(
sdktrace.WithSampler(sdktrace.AlwaysSample()),
sdktrace.WithSpanProcessor(bsp),
)
propgator := propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{})
ctx, span := tp.Tracer("foo").Start(context.Background(), "parent-span-name")
defer span.End()
// Serialize the context into carrier
carrier := propagation.MapCarrier{}
propgator.Inject(ctx, carrier)
// This carrier is sent accros the process
fmt.Println(carrier)
// Extract the context and start new span as child
// In your receiving function
parentCtx := propgator.Extract(context.Background(), carrier)
_, childSpan := tp.Tracer("foo").Start(parentCtx, "child-span-name")
childSpan.AddEvent("some-dummy-event")
childSpan.End()
}

How do I distribute tracing between 2 Go microservices using Opentelemetry & Otelgin?

I am trying to distribute tracing between 2 Go microservices using Opentelemetry and Gin-Gonic.
Please help, I have come across Otelhttp examples, but I couldn't find examples with Otelgin.
Using "go.opentelemetry.io/otel/sdk/trace" as tracesdk
Microservice 1
func TracerProvider() (*tracesdk.TracerProvider, error) {
otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}))
exporter, _:= stdouttrace.New(stdouttrace.WithPrettyPrint())
tp := tracesdk.NewTracerProvider(
tracesdk.WithSampler(tracesdk.AlwaysSample()),
tracesdk.WithBatcher(exporter),
tracesdk.WithResource(resource.NewWithAttributes(
semconv.ServiceNameKey.String("microservice-1"),
attribute.String("environment", "test"),
)),
)
otel.SetTracerProvider(tp)
return tp, nil
}
func main() {
TracerProvider()
resty := resty.New()
router := gin.Default()
router.Use(otelgin.Middleware("microservice-1"))
{
router.GET("/ping", func(c *gin.Context) {
result := Result{}
req := resty.R().SetHeader("Content-Type", "application/json")
ctx := req.Context()
span := trace.SpanFromContext(ctx)
defer span.End()
otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(req.Header))
resp, _ := req.Get("http://localhost:8088/pong")
json.Unmarshal([]byte(resp.String()), &result)
c.IndentedJSON(200, gin.H{
"message": result.Message,
})
})
}
router.Run(":8085")
}
// Microservice 2
//TracerProvider func is the same as Microservice 1
//main
TracerProvider()
router := gin.Default()
router.Use(otelgin.Middleware("microservice-2"))
{
router.GET("/pong", func(c *gin.Context) {
ctx := c.Request.Context()
span := trace.SpanFromContext(otel.GetTextMapPropagator().Extract(ctx, propagation.HeaderCarrier(c.Request.Header)))
defer span.End()
c.IndentedJSON(200, gin.H{
"message": "pong",
})
})
}
router.Run(":8088")
There is an example of go-gin instrumentation in the official repository of OpenTelemetry Go Contrib:
https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/instrumentation/github.com/gin-gonic/gin/otelgin/example
Basically, you need use the package:
"go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
And you can also add the otelgin.HTML to render your response.
HTML will trace the rendering of the template as a child of the span in the given context.
This is a replacement for gin.Context.HTML function - it invokes the original function after setting up the span.
As detailed in its implementation: https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/instrumentation/github.com/gin-gonic/gin/otelgin/gintrace.go
It might be a bit late but it will still be useful.
In the 1st microservice, you need to put:
req := resty.R().SetHeader("Content-Type", "application/json")
req.SetContext(c.Request.Context())
That will fix the problem.
On this screenshot you are able to see the 2 spans on the 1st microservice

How to mock memcache in go lang for unit testing?

I want to mock memcache cache data in go lang to avoid authhorization
i tried with gomock but couldn't work out as i dont have any interface for it.
func getAccessTokenFromCache(accessToken string)
func TestSendData(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
mockObj := mock_utils.NewMockCacheInterface(mockCtrl)
mockObj.EXPECT().GetAccessToken("abcd")
var jsonStr = []byte(`{
"devices": [
{"id": "avccc",
"data":"abcd/"
}
]
}`)
req, err := http.NewRequest("POST", "/send/v1/data",
bytes.NewBuffer(jsonStr))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "d958372f5039e28")
rr := httptest.NewRecorder()
handler := http.HandlerFunc(SendData)
handler.ServeHTTP(rr, req)
if status := rr.Code; status != 200 {
t.Errorf("handler returned wrong status code: got %v want %v",
status, http.StatusOK)
}
expected := `{"error":"Invalid access token"}`
body, _ := ioutil.ReadAll(rr.Body)
if string(body) != expected {
t.Errorf("handler returned unexpected body: got %v want %v",
string(body), expected)
}
func SendData(w http.ResponseWriter, r *http.Request) {
accessToken := r.Header.Get(constants.AUTHORIZATION_HEADER_KEY)
t := utils.CacheType{At1: accessToken}
a := utils.CacheInterface(t)
isAccessTokenValid := utils.CacheInterface.GetAccessToken(a, accessToken)
if !isAccessTokenValid {
RespondError(w, http.StatusUnauthorized, "Invalid access token")
return
}
response := make(map[string]string, 1)
response["message"] = "success"
RespondJSON(w, http.StatusOK, response)
}
tried to mock using gomock
package mock_utils
gen mock for utils for get access controler
(1) Define an interface that you wish to mock.
(2) Use mockgen to generate a mock from the interface.
(3) Use the mock in a test:
You need to architect your code such that every such access to a service happens via an interface implementation. In your case, you should ideally create an interface like
type CacheInterface interface {
Set(key string, val interface{}) error
Get(key string) (interface{},error)
}
Your MemcacheStruct should implement this interface and all your memcache related calls should happen from there. Like in your case GetAccessToken should call cacheInterface.get(key) wherein your cacheInterface should refer to memcache implementation of this interface. This is a way better way to design your go programs and this would not only help you in writing tests but would also help in case let's say you want to use a different in memory database to help with caching. Like for ex., let's say in future if you want to use redis as your cache storage, then all you need to change is create a new implementation of this interface.

Gikngo Tests hang during goapp tests

I'm trying to use Gikngo to write some tests for appengine.
My setup for the tests is as follows:
suite_test.go:
BeforeSuite() {
inst, err = aetest.NewInstance(options)
if err != nil {
Fail(fmt.Sprintf("%s", err))
}
}
var(
req *http.Request
ctx context.Context
)
BeforeEach() {
req = inst.NewRequest()
ctx = appengine.NewContext(req)
// Clean up local datastore using the context.
}
validation_test.go
Describe("Some Test", func() {
It("ValidateFoo", func() {
// Access ctx here
})
...
It("ValidateBar", func() {
// Access ctx here.
})
})
I see our tests consistently hanging with the error of the type:
Expected success, but got an error:
<*url.Error | 0xc8210570b0>: {
Op: "Post",
URL: "http://localhost:59072",
Err: {s: "EOF"},
}
Post http://localhost:59072: EOF
This seems to indicate that the API server has become inaccessible. However, the test output does not seem to indicate this.
What are the ways in which we can debug a goapp test?
It turns out that Ginkgo or Golang had nothing to do with this. There seems to be some limitation on the number of fields one can read per second from dev_appserver.py. (I suspect that it might be related to SQLite which is the DB that dev_appserver uses internally).
The following code points out the problem:
package preorder
import (
"fmt"
"testing"
"google.golang.org/appengine"
"google.golang.org/appengine/aetest"
"google.golang.org/appengine/datastore"
)
func TestLoad(t *testing.T) {
opts := aetest.Options{StronglyConsistentDatastore: true}
inst, _ := aetest.NewInstance(&opts)
defer inst.Close()
for i := 0; i < 10000; i++ {
req, _ := inst.NewRequest("GET", "/", nil)
ctx := appengine.NewContext(req)
k := datastore.NewKey(ctx, ENTITY_NAME, "", 12345, nil)
var entity Entity
datastore.Get(ctx, k, &entity)
fmt.Println("Iteration Count: ", i)
ctx.Done()
}
}
Any help on figuring out how to work around the limit of 240 operations would be appreciated. One technique I can think of is to artificially inject delays.

Go lang Redis PubSub in different go routes for publish and subscribe

Am new to go programming language, and I have a requirement to create Redis PubSub with websocket.
My reference code is here
https://github.com/vortec/orchestrate
Am using following libraries
"golang.org/x/net/websocket"
"github.com/garyburd/redigo/redis"
Everything is working for me like this way, but I don't understand what is "websocket.Handler(handleWSConnection)" here.
I need 2 different go routes for /ws-subscribe and /ws-publish. I don't know anything wrong in this concept?
Doubts
Can I do this way, http.HandleFunc("/ws", handleWSConnection) // Tried this way but am getting "not enough arguments in call to handleWSConnection"
Is there any way to call "handleWSConnection()" as a normal function.
Any suggestions how to write /ws-publish as a different go route
Following is my code
main function
func (wsc *WSConnection) ReadWebSocket() {
for {
var json_data []byte
var message WSMessage
// Receive data from WebSocket
err := websocket.Message.Receive(wsc.socket, &json_data)
if err != nil {
return
}
// Parse JSON data
err = json.Unmarshal(json_data, &message)
if err != nil {
return
}
switch message.Action {
case "SUBSCRIBE":
wsc.subscribe.Subscribe(message.Channel)
case "UNSUBSCRIBE":
wsc.subscribe.Unsubscribe(message.Channel)
case "PUBLISH":
wsc.publish.Conn.Do("PUBLISH", message.Channel, message.Data)
}
}
}
func handleWSConnection(socket *websocket.Conn) {
wsc := &WSConnection {socket: socket}
defer wsc.Uninitialize()
wsc.Initialize()
go wsc.ProxyRedisSubscribe()
wsc.ReadWebSocket()
}
func serveWeb() {
http.Handle("/ws", websocket.Handler(handleWSConnection)) // I need to call this route as funciton
if err := http.ListenAndServe(":9000", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
Done following way, I dont know is it the proper way to do this
http.HandleFunc("/publish", publishHandler)
func publishHandler(conn http.ResponseWriter, req *http.Request) {
log.Println("PUBLISH HANDLER")
wsHandler := websocket.Handler(func(ws *websocket.Conn) {
handleWSConnection(ws)
})
wsHandler.ServeHTTP(conn, req)
}

Resources