How to reduce repetitive http handler code in golang? - go

I'm designing a API server in Go. I have many database tables, each with a matching struct. Each has a route and handler:
type Thing1 struct {
ID int64
Name string
...
}
func main() {
...
router := mux.NewRouter()
apiRouter := router.PathPrefix("/v1").Subrouter()
apiRouter.HandleFunc("/thing1/{id}", Thing1ShowHandler).Methods("GET")
}
func Thing1ShowHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id, err := strconv.ParseInt(vars["id"], 10, 64)
if err != nil {
errorHandler(w, err)
return
}
thing1 := Thing1{ID: id}
err = db.First(&thing1, id).Error
if thing1.ID > 0 {
jsonHeaders(w, http.StatusOK)
if err := json.NewEncoder(w).Encode(thing1); err != nil {
errorHandler(w, err)
}
return
}
notFoundHandler(w, r)
}
The code for Thing2 is pretty much identical, as it is for Thing3 and so on. I will end up with hundreds of things, and therefore lots of duplicated code. It feels like I'm doing something horribly wrong. What's the best way to make this more DRY?

Why not create a factory function for the http.Handler used with each Thing? This allows you to write the showHandler logic once and parameterize the instantiation of individual things.
// A ThingFactory returns a Thing struct configured with the given ID.
type ThingFactory func(id int64) interface{}
// The createShowHandler function is a factory function for creating a handler
// which uses the getThing factory function to obtain an instance of a
// thing to use when generating a view.
func createShowHandler(getThing ThingFactory) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id, err := strconv.ParseInt(vars["id"], 10, 64)
if err != nil {
errorHandler(w, err)
return
}
thing := getThing(id)
err = db.First(&thing, id).Error
if err != nil {
errorHandler(w, err)
}
if thing1.ID > 0 {
jsonHeaders(w, http.StatusOK)
if err := json.NewEncoder(w).Encode(thing1); err != nil {
errorHandler(w, err)
}
return
}
notFoundHandler(w, r)
}
}
This function can be used to systematically create routes for a given router. For instance, I can create an explicit registry which keeps track of the path for each thing as well as a ThingFactory instance which is used when calling the createShowHandler factory function.
router := mux.NewRouter()
apiRouter := router.PathPrefix("/v1").Subrouter()
registry := []struct {
path string
handler ThingFactory
}{
{"/thing1/{id}", func(id int64) interface{} { return Thing1{ID: id} }},
{"/thing2/{id}", func(id int64) interface{} { return Thing2{ID: id} }},
{"/thing3/{id}", func(id int64) interface{} { return Thing3{ID: id} }},
}
for _, registrant := range registry {
apiRouter.HandleFunc(registrant.path, createShowHandler(registrant.handler)).Methods("GET")
}
Naturally, you would want to define interfaces for the various interaction points in a program like this to gain more type safety when dealing with a large number of instances. A more robust registry could be implemented that provided an interface for Things to register themselves with.

Related

Maximize the number of CustomResources that a CustomResourceDefinition can have | kubebuilder & operator-sdk

I'm developing a kubernetes operator that represents a very simple api and a controller.
I would like to maximize the number of the CustomResources those could belonging to the specific CustomResourceDefinition that the operator defines. (As specially I would like to allow just one CR, if it is already defined, the operator should throw an error message and skip reconciling it.)
If I generate the api, there is a KindList struct default generated, and if I understand correctly, it should keep track of the CRs already defined for my CRD. It is also added to the scheme by default. See the example from kubebuilder documentation:
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// CronJob is the Schema for the cronjobs API
type CronJob struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec CronJobSpec `json:"spec,omitempty"`
Status CronJobStatus `json:"status,omitempty"`
}
//+kubebuilder:object:root=true
// CronJobList contains a list of CronJob
type CronJobList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []CronJob `json:"items"`
}
func init() {
SchemeBuilder.Register(&CronJob{}, &CronJobList{})
}
Unfortunately, I can not find out how to access this List from the controller. I have tried like this, but r.Get can not accept cacheList:
cronjob/cronjob_controller.go
package controllers
import (
"context"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
cronjobv1alpha1 "github.com/example/cronjob-operator/api/v1alpha1"
)
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
// TODO(user): your logic here
cronjob := cronjobv1alpha1.Memcached{}
if err := r.Get(ctx, req.NamespacedName, &cronjob); err != nil {
return ctrl.Result{}, err
}
cronjobList := cachev1alpha1.MemcachedList{}
if err := r.Get(ctx, req.NamespacedName, &cronjobList); err != nil {
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
If I get the list, I could validate the length of it, and do or skip the reconcile.
Is it even a correct approach? Is there a better way to achieve my goal? Should I create a webhook instead?
Assuming you are using the default sigs.k8s.io/controller-runtime/pkg/client's client.Client, you get access to the List() function.
In your case r.List(...).
Usage:
case 1: list by label
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.MatchingLabels{"foo": "bar"})
if err != nil {
return ctrl.Result{}, err
}
}
case 2: list all in namespace
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.InNamespace("default"))
if err != nil {
return ctrl.Result{}, err
}
}
case 3: list by field i.e. metadata.name
// in your Reconciler Setup function create an index
func SetupWithManager(mgr ctrl.Manager) error {
r := &CronJobReconciler{
Client: mgr.GetClient(),
}
mgr.GetFieldIndexer().IndexField(context.TODO(), &cronjobv1alpha1.CronJob{}, "metadata.name", NameIndexer)
return ctrl.NewControllerManagedBy(mgr).
For(&cronjobv1alpha1.CronJob{}).
Complete(r)
}
func NameIndexer(o client.Object) []string {
m := o.(*cronjobv1alpha1.CronJob)
return []string{m.ObjectMeta.Name}
}
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.MatchingFields{"metadata.name": "test"})
if err != nil {
return ctrl.Result{}, err
}
}

Go create a mock for gcp compute sdk

I use the following function, and I need to raise the coverage of it (if possible to 100%), the problem is that typically I use interface to handle such cases in Go and for this specific case not sure how to do it, as this is a bit more tricky, any idea?
The package https://pkg.go.dev/google.golang.org/genproto/googleapis/cloud/compute/v1
Which I use doesn't have interface so not sure how can I mock it?
import (
"context"
"errors"
"fmt"
"os"
compute "cloud.google.com/go/compute/apiv1"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
computev1 "google.golang.org/genproto/googleapis/cloud/compute/v1"
)
func Res(ctx context.Context, project string, region string,vpc string,secret string) error {
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
return err
}
defer c.Close()
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := c.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Whenever I find myself in this scenario, I found that the easiest solution is to create missing interfaces myself. I limit these interfaces to the types and functions that I am using, instead of writing interfaces for the entire library. Then, in my code, instead of accepting third-party concrete types, I accept my interfaces for those types. Then I use gomock to generate mocks for these interfaces as usual.
The following is a descriptive example inspired by your code.
type RestClient interface {
List(context.Context, *computev1.ListAddressesRequest) (ListResult, error) // assuming List returns ListResult type.
Close() error
}
func newRestClient(ctx context.Context, secret string) (RestClient, error) {
return compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
}
func Res(ctx context.Context, project string, region string, vpc string, secret string) error {
c, err := newRestClient(ctx, secret)
if err != nil {
return err
}
defer c.Close()
return res(ctx, project, region, vpc, c)
}
func res(ctx context.Context, project string, region string, vpc string, c RestClient) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it, err := c.List(ctx, addrReq)
if err != nil {
return err
}
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Now you can test the important bits of the Res function by injecting a mock RestClient to the internal res function.
One obstacle to testability here is that you instantiate a client inside your Res function rather than injecting it. Because
the secret doesn't change during the lifetime of the programme,
the methods of *compute.AddressesClient (other than Close) are concurrency-safe,
you could create one client and reuse it for each invocation or Res. To inject it into Res, you can declare some Compute type and turn Res into a method on that type:
type Compute struct {
Lister Lister // some appropriate interface type
}
func (cp *Compute) Res(ctx context.Context, project, region, vpc string) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := cp.Lister.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
One question remains: how should you declare Lister? One possibility is
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) *compute.AddressIterator
}
However, because compute.AddressIterator is a struct type with some unexported fields and for which package compute provides no factory function, you can't easily control how the iterator returned from List behaves in your tests. One way out is to declare an additional interface,
type Iterator interface {
Next() (*computev1.Address, error)
}
and change the result type of List from *compute.AddressIterator to Iterator:
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
}
Then you can declare another struct type for the real Lister and use that on the production side:
type RealLister struct {
Client *compute.AddressesClient
}
func (rl *RealLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return rl.Client.List(ctx, req, opts...)
}
func main() {
secret := "don't hardcode me"
ctx, cancel := context.WithCancel(context.Background()) // for instance
defer cancel()
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
log.Fatal(err) // or deal with the error in some way
}
defer c.Close()
cp := Compute{Lister: &RealLister{Client: c}}
if err := cp.Res(ctx, "my-project", "us-east-1", "my-vpc"); err != nil {
log.Fatal(err) // or deal with the error in some way
}
}
For your tests, you can declare another struct type that will act as a configurable test double:
type FakeLister func(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
func (fl FakeLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return fl(ctx, req, opts...)
}
To control the behaviour of the Iterator in your test, you can declare another configurable concrete type:
type FakeIterator struct{
Err error
Status string
}
func (fi *FakeIterator) Next() (*computev1.Address, error) {
addr := computev1.Address{Status: &fi.Status}
return &addr, fi.Err
}
A test function may look like this:
func TestResStatusInUse(t *testing.T) {
// Arrange
l := func(_ context.Context, _ *computev1.ListAddressesRequest, _ ...gax.CallOption) Iterator {
return &FakeIterator{
Status: "IN_USE",
Err: nil,
}
}
cp := Compute{Lister: FakeLister(l)}
dummyCtx := context.Background()
// Act
err := cp.Res(dummyCtx, "my-project", "us-east-1", "my-vpc")
// Assert
if err != nil {
// ...
}
}

Nesting method calls

How can I nest methods in Go?
Let's say, I have 2 files, each in a different package.
First file: handlers/user.go
type Resolver struct {
client *elastic.Client
index string
}
func (r *Resolver) CreateUser(ctx context.Context, name string) (*model.User, error) {
u, err := services.CreateUserService(ctx, r.client, r.index, name)
if err != nil {
return nil, err
}
return u, nil
}
and another file: services/user.go
func CreateUserService(ctx context.Context, client *elastic.Client, index string, name string) (*model.User, error) {
u := &model.User{
Name: name
}
//doing some other function call
err := dao.CreateUserDAO(ctx, client, index, s)
if err != nil {
return nil, err
}
return u, nil
}
This works fine. But I am still having to pass 2 parameters as r.client and r.index.
What I want to do, is make a call like
services.r.CreateUserService(ctx, name).
This makes my code more readable and less cluttered.
But I am not sure, how to change the services package to accommodate this change.
How can I do that?
If I understand correctly, try to create such a method.
func (r *Resolver) CreateUser(ctx context.Context, name string) (*model.User, error) {
u, err := r.CreateUserService(ctx, name)
if err != nil {
return nil, err
}
return u, nil
}
func (r *Resolver) CreateUserService(ctx context.Context, name string) (*model.User, error) {
return services.CreateUserService(ctx, r.client, r.index, name)
}
But I have no idea how to call like services.r.CreateUserService(ctx, name) unless services is a class which includes a Resolver, that's so weird.

VS Code error when running go application

I am new to go and I am following a tutorial online. I get this error from VS Code
"cannot use c.ReadConfig (type func(http.ResponseWriter, *http.Request)) as type http.Handler in argument to router.Get:
func(http.ResponseWriter, *http.Request) does not implement http.Handler (missing ServeHTTP method)".
I checked the Get and Redconfig functions and they look alright. The teacher on his end does not get the error and he is able to run the Go code fine. this is the snippet in the main
This the main function
func main() {
config := domain.Config{}
configService := service.ConfigService{
Config: &config,
Location: "config.yaml",
}
go configService.Watch(time.Second * 30)
c := controller.Controller{
Config: &config,
}
router := muxinator.NewRouter()
router.Get("/read/{serviceName}", c.ReadConfig)
log.Fatal(router.ListenAndServe(":8080"))
}
This is the Get function
// Get returns the config for a particular service
func (c *Config) Get(serviceName string) (map[string]interface{}, error) {
c.lock.RLock()
defer c.lock.RUnlock()
a, ok := c.config["base"].(map[string]interface{})
if !ok {
return nil, fmt.Errorf("base config is not a map")
}
// If no config is defined for the service
if _, ok = c.config[serviceName]; !ok {
// Return the base config
return a, nil
}
b, ok := c.config[serviceName].(map[string]interface{})
if !ok {
return nil, fmt.Errorf("service %q config is not a map", serviceName)
}
// Merge the maps with the service config taking precedence
config := make(map[string]interface{})
for k, v := range a {
config[k] = v
}
for k, v := range b {
config[k] = v
}
return config, nil
}
This is ReadConfig
// ReadConfig writes the config for the given service to the ResponseWriter
func (c *Controller) ReadConfig(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
vars := mux.Vars(r)
serviceName, ok := vars["serviceName"]
if !ok {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(w, "error")
}
config, err := c.Config.Get(serviceName)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "error")
}
rsp, err := json.Marshal(&config)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "error")
}
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, string(rsp))
}
What should happen is that I should be able to run and I can go to http://localhost:8080/read/base
Use http.HandlerFunc:
router := muxinator.NewRouter()
router.Get("/read/{serviceName}", http.HandlerFunc(c.ReadConfig))
It's expecting a ServeHTTP method, but you gave it a direct function. http.HandlerFunc acts as a wrapper so you can use a plain function as your handler.

Test wrap function with dependency injection

I have this function which I need to mock in test,
I was able to mock it as expected with http mock package , but now I’ve function that are calling
To the HttpReq method and here I cannot use http mock package
I read about dependency injection and tried something but I wasn’t able to fully do it,
This is the function
type params struct {
cs string
ci string
method string
url string
}
// I added this struct but not sure if it's needed ... probably for test purpose but not sure how to use it.
type Impl struct {
client *http.Client
}
func (i *Impl) HttpReq(p *params) ([]byte, error) {
httpClient := i.client
req, err := http.NewRequest(p.method, p.url, nil)
if err != nil {
fmt.Sprintf(err)
}
req.SetBasicAuth(p.cs, p.ci)
res, err := httpClient.Do(req)
if err != nil {
fmt.Sprintf(err)
}
t, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Sprintf(err)
}
defer res.Body.Close()
return t, nil
}
This is what I tried
I’ve created interface
type Req interface {
HttpReq(params) ([]byte, error)
}
Now I’ve created a struct which contain the interface
type Service struct {
req Req
}
This is the function which I need to tests
func (c *Service) execute(cli Connection , args []string) (error, []byte) {
sk, err := c.doSomthing(cli, args)
sc, err := c.doSometing2(serviceK, []string{"url", "cl", "ct"})
cc := strings.Fields(serviceCredentials)
// ----------Here is what I need to mock ----------
t, err := c.req.HttpReq(params{cs: cc[1],
ci: cc[2],
method: http.MethodPost,
url: cc[0],})
return err, t
}
Any idea how I can run test for this function ??? Im struggling with it a lot.
Independent of the original question, you should not create new HTTP clients for each request. Client's maintain a connection pool and should be reused as much as possible.
You can fix that, and continue using your existing mock server by injecting the HTTP client.
Note also that the interface definition in the question doesn't match the implementation. These two method signatures are not the same:
HttpReq(params) ([]byte, error) // Req
HttpReq(*params) ([]byte, error) // Impl
Pick one. I would probably go with the non-pointer type here. And upper case initials are idiomatic in Go (HTTPReq, not HttpReq).
Add the client to the Impl type and use it in HTTPReq:
type Impl struct {
client *http.Client
}
func (i *Impl) HTTPReq(p params) ([]byte, error) {
req, err := http.NewRequest(p.method, p.url, nil)
if err != nil {
return nil, err
}
req.SetBasicAuth(p.cs, p.ci)
res, err := i.client.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
return ioutil.ReadAll(res.Body)
}
The Service type doesn't have to change.
In the tests, simply inject a test client into the Impl value:
import (
"context"
"net"
"net/http"
"net/http/httptest"
"testing"
)
func TestService_execute(t *testing.T) {
var testHandler http.Handler // TODO
srv := httptest.NewServer(testHandler)
defer srv.Close()
client := srv.Client()
tp := client.Transport.(*http.Transport)
// Direct all requests to the test server, no matter what the request URL is.
tp.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
// Note that we ignore the network and addr parameters, since these are
// derived from the request URL and are unrelated to the test server.
srvAddr := srv.Listener.Addr()
return (&net.Dialer{}).DialContext(ctx, srvAddr.Network(), srvAddr.String())
}
svc := &Service{
req: &Impl{
client: client,
},
}
svc.execute(/* ... */)
// assert request, response, etc.
}
Since Service struct already has an req interface, during tests initialise service object with mock that satisfies req interface.
Something like this
https://stackoverflow.com/a/53805535/3968921

Resources