Testing Nats subscription in Go - go

I have a function designed to listen to a Nats subject and route the messages as it receives them:
func (conn *JetStreamConnection) SubscribeMultiple(ctx context.Context, subject string,
subscribers ...*SubscriptionCallback) error {
callbacks := make(map[string]func(*pnats.NatsMessage) (func(context.Context), error))
for _, subscriber := range subscribers {
callbacks[subscriber.Category] = subscriber.Callback
}
fullSubject := fmt.Sprintf("%s.*", subject)
sub, err := conn.context.SubscribeSync(fullSubject, nats.Context(ctx))
if err != nil {
return err
}
loop:
for {
select {
case <-ctx.Done():
break loop
default:
}
msg, err := sub.NextMsgWithContext(ctx)
if err != nil {
return err
}
msg.InProgress()
var message pnats.NatsMessage
if err := conn.unmarshaller(msg.Data, &message); err != nil {
msg.Term()
return err
}
actualSubject := fmt.Sprintf("%s.%s", subject, message.Context.Category)
subscriber, ok := callbacks[message.Context.Category]
if !ok {
msg.Nak()
continue
}
callback, err := subscriber(&message)
if err == nil {
msg.Ack()
} else {
msg.Nak()
return err
}
callback(ctx)
}
if err := sub.Unsubscribe(); err != nil {
return err
}
return nil
}
My problem is that, since the SubscribeSync function produces a *nats.Subscription object, I have no way to mock out the test. How can I test around this object?

You can put your loop in a separate function. This func can accept an interface that describes nats Subscription instead of *nats.Subscription. This way you will be able to create Subscription mocks with gomock or other tools. After that you can test the inside func separately
Something like this:
func (conn *JetStreamConnection) SubscribeMultiple(ctx context.Context, subject string,
subscribers ...*SubscriptionCallback) error {
callbacks := make(map[string]func(*pnats.NatsMessage) (func(context.Context), error))
for _, subscriber := range subscribers {
callbacks[subscriber.Category] = subscriber.Callback
}
fullSubject := fmt.Sprintf("%s.*", subject)
sub, err := conn.context.SubscribeSync(fullSubject, nats.Context(ctx))
if err != nil {
return err
}
return run(ctx, sub)
}
//go:generate mockgen -source conn.go -destination ../mocks/conn.go -package mocks
type ISubscription interface{
NextMsgWithContext(ctx context.Context) (*nats.Msg, error)
Unsubscribe() error
}
func (conn *JetStreamConnection) run(ctx context.Context, sub ISubscription) error {
loop:
for {
select {
case <-ctx.Done():
break loop
default:
}
msg, err := sub.NextMsgWithContext(ctx)
if err != nil {
return err
}
msg.InProgress()
var message pnats.NatsMessage
if err := conn.unmarshaller(msg.Data, &message); err != nil {
msg.Term()
return err
}
actualSubject := fmt.Sprintf("%s.%s", subject, message.Context.Category)
subscriber, ok := callbacks[message.Context.Category]
if !ok {
msg.Nak()
continue
}
callback, err := subscriber(&message)
if err == nil {
msg.Ack()
} else {
msg.Nak()
return err
}
callback(ctx)
}
if err := sub.Unsubscribe(); err != nil {
return err
}
}
upd: if you still want to test SubscribeMultiple, you can create a Runner that will have only one func Run and take it as dependency for JetStreamConnection. Again, you can create a mock for Runner and test with it

Related

Passing an CSVExport function to a handler Gin

I have created a CSV export function for my to-do list application. The function is working, the handler is returning a written file but I get a strange panic in the console from the Gin framework:
http: wrote more than the declared Content-Length
Is that something crucial and how can I fix this panic.
This is my function:
func (r *Repository) CSVExport() (*os.File, error) {
tasks, err := r.getAllTasks()
if err != nil {
return nil, err
}
file, err := os.Create("tasks.csv")
if err != nil {
return nil, err
}
defer file.Close()
writer := csv.NewWriter(file)
defer writer.Flush()
var taskNameList []string
for _, task := range tasks {
taskNameList = append(taskNameList, task.Text)
}
err = writer.Write(taskNameList)
if err != nil {
return nil, err
}
return file, nil
}
And this is the handler:
func CSVExport(data model.ListOperations) gin.HandlerFunc {
return func(c *gin.Context) {
tasks, err := data.CSVExport()
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid Task"})
}
c.FileAttachment("./tasks.csv", "tasks.csv")
c.Writer.Header().Set("attachment", "filename=tasks.csv")
c.JSON(200, tasks)
}
}
Your code has some error:
You need to return on error
You can't return JSON after returning your file with fileAttachment (it already does this stuff)
func CSVExport(data model.ListOperations) gin.HandlerFunc {
return func(c *gin.Context) {
tasks, err := data.CSVExport()
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid Task"})
return //stop it on error
}
c.FileAttachment("./tasks.csv", "tasks.csv")
c.Writer.Header().Set("attachment", "filename=tasks.csv")
//c.JSON(200, tasks) not need the fileAttachement func do it
}
}

Where is the message handler in go-ethereum?

I'm currently trying to figure out the Ethereum code and have learned how to send transactions to the blockchain using the client module.Here is an example of a contract call function:
func (ec *Client) CallContract(ctx context.Context, msg ethereum.CallMsg, blockNumber *big.Int) ([]byte, error) {
var hex hexutil.Bytes
err := ec.c.CallContext(ctx, &hex, "eth_call", toCallArg(msg), toBlockNumArg(blockNumber))
if err != nil {
return nil, err
}
return hex, nil
}
, where CallContext defined as:`
func (c *Client) CallContext(ctx context.Context, result interface{}, method string, args ...interface{}) error {
if result != nil && reflect.TypeOf(result).Kind() != reflect.Ptr {
return fmt.Errorf("call result parameter must be pointer or nil interface: %v", result)
}
msg, err := c.newMessage(method, args...)
if err != nil {
return err
}
op := &requestOp{ids: []json.RawMessage{msg.ID}, resp: make(chan *jsonrpcMessage, 1)}
if c.isHTTP {
err = c.sendHTTP(ctx, op, msg)
} else {
err = c.send(ctx, op, msg)
}
if err != nil {
return err
}
// dispatch has accepted the request and will close the channel when it quits.
switch resp, err := op.wait(ctx, c); {
case err != nil:
return err
case resp.Error != nil:
return resp.Error
case len(resp.Result) == 0:
return ErrNoResult
default:
return json.Unmarshal(resp.Result, &result)
}`
And my question is: Where is the handler for these messages implemented in go - ethereum?
For example:
switch msg.name:
case "eth_call": ...
case "eth_sendTx": ...
...

I cannot escape a for loop with a ticker

I have this API that scans for drivers' locations and send them via web-socket every 1 second. The issue is that the loop cannot be escaped when client disconnects. It seems it is alive for ever. I am using Gin with nhooyr websocket library.
var GetDriverLocations = func(c *gin.Context) {
wsoptions := websocket.AcceptOptions{InsecureSkipVerify: true}
wsconn, err := websocket.Accept(c.Writer, c.Request, &wsoptions)
if err != nil {
return
}
defer wsconn.Close(websocket.StatusInternalError, "the sky is falling")
driverLocation := &models.DriverLocation{}
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
case <-c.Request.Context().Done():
fmt.Println("done") //this never gets printed
return
}
coords, err := driverLocation.GetDrivers()
if err != nil {
break
}
err = wsjson.Write(c.Request.Context(), wsconn, &coords)
if websocket.CloseStatus(err) == websocket.StatusNormalClosure {
break
}
if err != nil {
break
}
}
fmt.Println("conn ended") //this never gets printed
}
I also tried this loop but also has the same issue:
for range ticker.C{
coords, err := driverLocation.GetDrivers()
if err != nil {
break
}
err = wsjson.Write(c.Request.Context(), wsconn, &coords)
if websocket.CloseStatus(err) == websocket.StatusNormalClosure {
break
}
if err != nil {
break
}
}
Because the network connection is hijacked from the net/http server by the nhooyr websocket library, the context c.Request.Context() is not canceled until handler returns.
Call CloseRead to get a context that's canceled when the connection is closed. Use that context in the loop.
var GetDriverLocations = func(c *gin.Context) {
wsoptions := websocket.AcceptOptions{InsecureSkipVerify: true}
wsconn, err := websocket.Accept(c.Writer, c.Request, &wsoptions)
if err != nil {
return
}
defer wsconn.Close(websocket.StatusInternalError, "")
ctx := wsconn.CloseRead(c.Request.Context())
driverLocation := &models.DriverLocation{}
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
case <-ctx.Done():
return
}
coords, err := driverLocation.GetDrivers()
if err != nil {
break
}
err = wsjson.Write(c.Request.Context(), wsconn, &coords)
if err != nil {
break
}
}
}

Create function that receives any function with specific amount of parameters

Say I have several different gRPC servers, for example x.Server, y.Server and z.Server, and in order to spin them up, I have a lot of repeated code inside their main function, e.g.:
func main() {
if err := config.EnsureArgLength(1); err != nil {
log.Fatalln(err.Error())
}
srv := &x.Server{}
if err := srv.ReadServerConfig(os.Args[1]); err != nil {
log.Fatalln(err.Error())
}
if err := srv.RegisterListener(); err != nil {
log.Fatalln(err.Error())
}
if err := srv.RegisterClients(); err != nil {
log.Fatalln(err.Error())
}
s := grpc.NewServer()
proto.RegisterXServer(s, srv)
if err := srv.Serve(s); err != nil {
log.Fatalf("failed to serve: %s", err.Error())
}
}
I would love to refactor this main function to make it one or two lines long, something like the following:
func main() {
srv := x.Server{}
if err := srv.RegisterAndServe(); err != nil {
log.Fatal("failed to serve: %s", err.Error())
}
}
But each server will have an auto-generated function proto.RegisterXServer which is not part of x.Server struct, and I'm also not able to modify the file which contains it, since it is auto generated. How should I proceed?
in regards to op changes, which was radical,
I can suggest using a reducer pattern like this.
package main
import (
"fmt"
)
func main() {
fail(reduce(sayHello(), sayGoodbye))
}
func sayHello() func() error {
return func() error { fmt.Println("Hello, playground"); return nil }
}
func sayGoodbye() error {
fmt.Println("Goodbye from the playground")
return nil
}
func reduce(h ...func() error) error {
for _, hh := range h {
if err := hh(); err != nil {
return err
}
}
return nil
}
func fail(err error) {
if err != nil {
panic(err)
}
}

How to gently defer execution of a function that might return an error?

Most cleanup functions, especially those related to the IO operations, return an error, and normally we'd prefer to defer their execution in case if we'd not forget to call them when we're done with acquired resources. For example, at some point in the code we might write something like this:
var r *SomeResource
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer r.Close() // This might return an error
It seems that if Close function returns an error, it'll be ignored. How can we gently process the returned error from such a function?
Using defer with a func() {}() like so.
var r *SomeResource
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if err = r.Close(); err != nil {
fmt.Printf("ERROR: %v", err)
}
}()
Fail gracefully with an error. Report the first error. Don't overwrite earlier errors. For example,
package main
import (
"fmt"
"os"
)
func demo() (name string, err error) {
filename := `test.file`
f, err := os.Open(filename)
if err != nil {
return "", err
}
defer func() {
e := f.Close()
if e != nil {
if err == nil {
err = e
}
}
}()
// do someting with the file
name = f.Name()
fi, err := f.Stat()
if err != nil {
return name, err
}
if fi.Size() == 0 {
err = fmt.Errorf("%s: empty file", filename)
return name, err
}
return name, err
}
func main() {
name, err := demo()
fmt.Println(name, err)
}
We can handle this in ways like:
way-1:
func myFn() error {
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if cErr = r.Close(); cErr != nil {
err = cErr
}
}()
return err
}
way-2:
func myFn() error {
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if cErr = r.Close(); cErr != nil {
// we can log the error
// or
// whatever we want to do
}
}()
return err
}
I have also find a nice blog on this topic, i mean handling error when defer func returns an error. Check here https://blog.learngoprogramming.com/5-gotchas-of-defer-in-go-golang-part-iii-36a1ab3d6ef1.

Resources