Is it possible to reload html templates after app is started? - go

Right now I currently parse all the templates into a variable like so:
var templates = template.New("Template")
func LoadTemplates() {
filepath.Walk("view/", func(path string, info os.FileInfo, err error) error {
if strings.HasSuffix(path, ".html") {
templates.ParseFiles(path)
}
return nil
})
}
func Render(w http.ResponseWriter, tmplName string, data interface{}) {
if err := templates.ExecuteTemplate(w, tmplName, data); err != nil {
log.Println(err)
}
}
So if I make any changes, I need to restart the entire app. Is there any way I can make it so that the changes are reflected when they're made

It is completly OK to reload your templates on every request when developing / in debug mode.
You can define an interface and two "template executors" like so
type TemplateExecutor interface{
ExecuteTemplate(wr io.Writer, name string, data interface{}) error
}
type DebugTemplateExecutor struct {
Glob string
}
func (e DebugTemplateExecutor) ExecuteTemplate(wr io.Writer, name string, data interface{}) error {
t := template.Must(template.ParseGlob(e.Glob))
return t.ExecuteTemplate(wr, name, data)
}
type ReleaseTemplateExecutor struct {
Template *template.Template
}
func (e ReleaseTemplateExecutor) ExecuteTemplate(wr io.Writer, name string, data interface{}) error {
return e.Template.ExecuteTemplate(wr, name, data)
}
that wrap template.Execute(). Pick which one you want at runtime e.g.
const templateGlob = "templates/*.html"
const debug = true
var executor TemplateExecutor
func main() {
if debug {
executor = DebugTemplateExecutor{templateGlob}
} else {
executor = ReleaseTemplateExecutor{
template.Must(template.ParseGlob(templateGlob)),
}
}
http.HandleFunc("/test", func(w http.ResponseWriter, r *http.Request) {
executor.ExecuteTemplate(w, "test.html", nil)
})
http.ListenAndServe(":3000", nil)
}
If you need hot reload in production, i'd suggest watching the directory for changes instead of compiling it on every request.

Yes, it is possible to reload (html or text) templates at runtime.
As long as the folder that you are reading the templates from is accessible by the application you can simply repeat the template parsing.
You could do something like this:
var templates = template.New("Template")
func folderChanged(folder string) bool {
// todo: implement filesystem watcher here
return true
}
func ReloadTemplates(templateFolder string) {
newTemplates, err := LoadTemplates(templateFolder)
if err != nil {
log.Println("Unable to load templates: %s", err)
return
}
// override existing templates variable
templates = newTemplates
}
func LoadTemplates(folder string) (*template.Template, error) {
template := template.New("Template")
walkError := filepath.Walk(folder, func(path string, info os.FileInfo, err error) error {
if strings.HasSuffix(path, ".html") {
_, parseError := template.ParseFiles(path)
if parseError != nil {
return parseError
}
}
return nil
})
return template, walkError
}
func Render(w http.ResponseWriter, tmplName string, data interface{}) {
templateFolder := "view"
if folderChanged(templateFolder) {
ReloadTemplates(templateFolder)
}
if err := templates.ExecuteTemplate(w, tmplName, data); err != nil {
log.Println(err)
}
}
If you are using a relative path like the ./view-folder from your example it will only work if the directory from which you are executing the application has such a sub-folder. If that is not the case I would suggest using a path relative to the users' home directory or something like that.

Related

Maximize the number of CustomResources that a CustomResourceDefinition can have | kubebuilder & operator-sdk

I'm developing a kubernetes operator that represents a very simple api and a controller.
I would like to maximize the number of the CustomResources those could belonging to the specific CustomResourceDefinition that the operator defines. (As specially I would like to allow just one CR, if it is already defined, the operator should throw an error message and skip reconciling it.)
If I generate the api, there is a KindList struct default generated, and if I understand correctly, it should keep track of the CRs already defined for my CRD. It is also added to the scheme by default. See the example from kubebuilder documentation:
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// CronJob is the Schema for the cronjobs API
type CronJob struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec CronJobSpec `json:"spec,omitempty"`
Status CronJobStatus `json:"status,omitempty"`
}
//+kubebuilder:object:root=true
// CronJobList contains a list of CronJob
type CronJobList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []CronJob `json:"items"`
}
func init() {
SchemeBuilder.Register(&CronJob{}, &CronJobList{})
}
Unfortunately, I can not find out how to access this List from the controller. I have tried like this, but r.Get can not accept cacheList:
cronjob/cronjob_controller.go
package controllers
import (
"context"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
cronjobv1alpha1 "github.com/example/cronjob-operator/api/v1alpha1"
)
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
// TODO(user): your logic here
cronjob := cronjobv1alpha1.Memcached{}
if err := r.Get(ctx, req.NamespacedName, &cronjob); err != nil {
return ctrl.Result{}, err
}
cronjobList := cachev1alpha1.MemcachedList{}
if err := r.Get(ctx, req.NamespacedName, &cronjobList); err != nil {
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
If I get the list, I could validate the length of it, and do or skip the reconcile.
Is it even a correct approach? Is there a better way to achieve my goal? Should I create a webhook instead?
Assuming you are using the default sigs.k8s.io/controller-runtime/pkg/client's client.Client, you get access to the List() function.
In your case r.List(...).
Usage:
case 1: list by label
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.MatchingLabels{"foo": "bar"})
if err != nil {
return ctrl.Result{}, err
}
}
case 2: list all in namespace
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.InNamespace("default"))
if err != nil {
return ctrl.Result{}, err
}
}
case 3: list by field i.e. metadata.name
// in your Reconciler Setup function create an index
func SetupWithManager(mgr ctrl.Manager) error {
r := &CronJobReconciler{
Client: mgr.GetClient(),
}
mgr.GetFieldIndexer().IndexField(context.TODO(), &cronjobv1alpha1.CronJob{}, "metadata.name", NameIndexer)
return ctrl.NewControllerManagedBy(mgr).
For(&cronjobv1alpha1.CronJob{}).
Complete(r)
}
func NameIndexer(o client.Object) []string {
m := o.(*cronjobv1alpha1.CronJob)
return []string{m.ObjectMeta.Name}
}
func (r *CronJobReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
cronjobList := cronjobv1alpha1.CronJobList{}
err = r.List(ctx, &cronjobList, client.MatchingFields{"metadata.name": "test"})
if err != nil {
return ctrl.Result{}, err
}
}

Go create a mock for gcp compute sdk

I use the following function, and I need to raise the coverage of it (if possible to 100%), the problem is that typically I use interface to handle such cases in Go and for this specific case not sure how to do it, as this is a bit more tricky, any idea?
The package https://pkg.go.dev/google.golang.org/genproto/googleapis/cloud/compute/v1
Which I use doesn't have interface so not sure how can I mock it?
import (
"context"
"errors"
"fmt"
"os"
compute "cloud.google.com/go/compute/apiv1"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
computev1 "google.golang.org/genproto/googleapis/cloud/compute/v1"
)
func Res(ctx context.Context, project string, region string,vpc string,secret string) error {
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
return err
}
defer c.Close()
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := c.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Whenever I find myself in this scenario, I found that the easiest solution is to create missing interfaces myself. I limit these interfaces to the types and functions that I am using, instead of writing interfaces for the entire library. Then, in my code, instead of accepting third-party concrete types, I accept my interfaces for those types. Then I use gomock to generate mocks for these interfaces as usual.
The following is a descriptive example inspired by your code.
type RestClient interface {
List(context.Context, *computev1.ListAddressesRequest) (ListResult, error) // assuming List returns ListResult type.
Close() error
}
func newRestClient(ctx context.Context, secret string) (RestClient, error) {
return compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
}
func Res(ctx context.Context, project string, region string, vpc string, secret string) error {
c, err := newRestClient(ctx, secret)
if err != nil {
return err
}
defer c.Close()
return res(ctx, project, region, vpc, c)
}
func res(ctx context.Context, project string, region string, vpc string, c RestClient) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it, err := c.List(ctx, addrReq)
if err != nil {
return err
}
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Now you can test the important bits of the Res function by injecting a mock RestClient to the internal res function.
One obstacle to testability here is that you instantiate a client inside your Res function rather than injecting it. Because
the secret doesn't change during the lifetime of the programme,
the methods of *compute.AddressesClient (other than Close) are concurrency-safe,
you could create one client and reuse it for each invocation or Res. To inject it into Res, you can declare some Compute type and turn Res into a method on that type:
type Compute struct {
Lister Lister // some appropriate interface type
}
func (cp *Compute) Res(ctx context.Context, project, region, vpc string) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := cp.Lister.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
One question remains: how should you declare Lister? One possibility is
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) *compute.AddressIterator
}
However, because compute.AddressIterator is a struct type with some unexported fields and for which package compute provides no factory function, you can't easily control how the iterator returned from List behaves in your tests. One way out is to declare an additional interface,
type Iterator interface {
Next() (*computev1.Address, error)
}
and change the result type of List from *compute.AddressIterator to Iterator:
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
}
Then you can declare another struct type for the real Lister and use that on the production side:
type RealLister struct {
Client *compute.AddressesClient
}
func (rl *RealLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return rl.Client.List(ctx, req, opts...)
}
func main() {
secret := "don't hardcode me"
ctx, cancel := context.WithCancel(context.Background()) // for instance
defer cancel()
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
log.Fatal(err) // or deal with the error in some way
}
defer c.Close()
cp := Compute{Lister: &RealLister{Client: c}}
if err := cp.Res(ctx, "my-project", "us-east-1", "my-vpc"); err != nil {
log.Fatal(err) // or deal with the error in some way
}
}
For your tests, you can declare another struct type that will act as a configurable test double:
type FakeLister func(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
func (fl FakeLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return fl(ctx, req, opts...)
}
To control the behaviour of the Iterator in your test, you can declare another configurable concrete type:
type FakeIterator struct{
Err error
Status string
}
func (fi *FakeIterator) Next() (*computev1.Address, error) {
addr := computev1.Address{Status: &fi.Status}
return &addr, fi.Err
}
A test function may look like this:
func TestResStatusInUse(t *testing.T) {
// Arrange
l := func(_ context.Context, _ *computev1.ListAddressesRequest, _ ...gax.CallOption) Iterator {
return &FakeIterator{
Status: "IN_USE",
Err: nil,
}
}
cp := Compute{Lister: FakeLister(l)}
dummyCtx := context.Background()
// Act
err := cp.Res(dummyCtx, "my-project", "us-east-1", "my-vpc")
// Assert
if err != nil {
// ...
}
}

Nesting method calls

How can I nest methods in Go?
Let's say, I have 2 files, each in a different package.
First file: handlers/user.go
type Resolver struct {
client *elastic.Client
index string
}
func (r *Resolver) CreateUser(ctx context.Context, name string) (*model.User, error) {
u, err := services.CreateUserService(ctx, r.client, r.index, name)
if err != nil {
return nil, err
}
return u, nil
}
and another file: services/user.go
func CreateUserService(ctx context.Context, client *elastic.Client, index string, name string) (*model.User, error) {
u := &model.User{
Name: name
}
//doing some other function call
err := dao.CreateUserDAO(ctx, client, index, s)
if err != nil {
return nil, err
}
return u, nil
}
This works fine. But I am still having to pass 2 parameters as r.client and r.index.
What I want to do, is make a call like
services.r.CreateUserService(ctx, name).
This makes my code more readable and less cluttered.
But I am not sure, how to change the services package to accommodate this change.
How can I do that?
If I understand correctly, try to create such a method.
func (r *Resolver) CreateUser(ctx context.Context, name string) (*model.User, error) {
u, err := r.CreateUserService(ctx, name)
if err != nil {
return nil, err
}
return u, nil
}
func (r *Resolver) CreateUserService(ctx context.Context, name string) (*model.User, error) {
return services.CreateUserService(ctx, r.client, r.index, name)
}
But I have no idea how to call like services.r.CreateUserService(ctx, name) unless services is a class which includes a Resolver, that's so weird.

Dynamically parsing files

For parsing files i have setup a variable for template.ParseFiles and i currently have to manually set each file.
Two things:
How would i be able to walk through a main folder and a multitude of subfolders and automatically add them to ParseFiles so i dont have to manually add each file individually?
How would i be able to call a file with the same name in a subfolder because currently I get an error at runtime if i add same name file in ParseFiles.
var templates = template.Must(template.ParseFiles(
"index.html", // main file
"subfolder/index.html" // subfolder with same filename errors on runtime
"includes/header.html", "includes/footer.html",
))
func main() {
// Walk and ParseFiles
filepath.Walk("files", func(path string, info os.FileInfo, err error) {
if !info.IsDir() {
// Add path to ParseFiles
}
return
})
http.HandleFunc("/", home)
http.ListenAndServe(":8080", nil)
}
func home(w http.ResponseWriter, r *http.Request) {
render(w, "index.html")
}
func render(w http.ResponseWriter, tmpl string) {
err := templates.ExecuteTemplate(w, tmpl, nil)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
To walk a directory looking for files see: http://golang.org/pkg/path/filepath/#Walk or http://golang.org/pkg/html/template/#New and http://golang.org/pkg/html/template/#Template.Parse
As for your other question ParseFiles uses the base name of the file as the template name which results in a collision in your template. You have two choices
Rename the file.
use t := template.New("name of your choice") to create an initial template
Use the walk function you already have started and call t.Parse("text from file") for each template file. You'll have to open and read the contents of the template files yourself to pass in here.
Edit: Code example.
func main() {
// Walk and ParseFiles
t = template.New("my template name")
filepath.Walk("files", func(path string, info os.FileInfo, err error) {
if !info.IsDir() {
// Add path to ParseFiles
content := ""
// read the file into content here
t.Parse(content)
}
return
})
http.HandleFunc("/", home)
http.ListenAndServe(":8080", nil)
}
So basically i set New("path name i want").Parse("String from read file") while walking through the folders.
var templates = template.New("temp")
func main() {
// Walk and ParseFiles
parseFiles()
http.HandleFunc("/", home)
http.ListenAndServe(":8080", nil)
}
//////////////////////
// Handle Functions //
//////////////////////
func home(w http.ResponseWriter, r *http.Request) {
render(w, "index.html")
render(w, "subfolder/index.html")
}
////////////////////////
// Reusable functions //
////////////////////////
func parseFiles() {
filepath.Walk("files", func(path string, info os.FileInfo, err error) error {
if !info.IsDir() {
filetext, err := ioutil.ReadFile(path)
if err != nil {
return err
}
text := string(filetext)
templates.New(path).Parse(text)
}
return nil
})
}
func render(w http.ResponseWriter, tmpl string) {
err := templates.ExecuteTemplate(w, tmpl, nil)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}

Specify names for parsed templates

I am trying to dynamically parse files using walk in a folder and I want to be able to set the path of the file "path/file.html". But my issue is if I have a file in a folder "path/folder/files.html" I can't do it because when I ExecuteTemplate the file name will be the same "files.html". Is it possible to name each template as I ParseFiles?
Im ok with doing a file one at a time if trying to do them all at once wont work.
// Parse file and send to responsewriter
func View(w http.ResponseWriter, path string) {
temp, err := template.ParseFiles("application/views/"+path+".html")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
} else {
temp.ExecuteTemplate(w, path, nil)
}
}
Walk the filesystem using filepath.Walk and a consumer method that will create templates with the full file paths as names:
package main
import (
"fmt"
"html/template"
"os"
"path/filepath"
)
func consumer(p string, i os.FileInfo, e error) error {
t := template.New(p)
fmt.Println(t.Name())
return nil
}
func main() {
filepath.Walk("/path/to/template/root", filepath.WalkFunc(consumer))
}
You can try template.Lookup, the whole process looks like:
var (
templates *template.Template
)
func loadTemplate() {
funcMap := template.FuncMap{
"safe":func(s string) template.HTML {
return template.HTML(s)
},
}
var err error
templates, err = utils.BuildTemplate("/theme/path/", funcMap)
if err != nil {
log.Printf("Can't read template file %v,", err)
}
}
func homeHandler(w http.ResponseWriter, r *http.Request) {
//lookup the theme your want to use
templ = templates.Lookup("theme.html")
err := templ.Execute(w, data)
if err != nil {
log.Println(err)
}
}
func main() {
loadTemplate()
}
BuildTemplate looks like:
func BuildTemplate(dir string, funcMap template.FuncMap) (*template.Template, error) {
fs, err := ioutil.ReadDir(dir)
if err != nil {
fmt.Printf("Can't read template folder: %s\n", dir)
return nil, err
}
files := make([]string, len(fs))
for i, f := range (fs) {
files[i] = path.Join(dir, f.Name())
}
return template.Must(template.New("Template").Funcs(funcMap).ParseFiles(files...)), nil
}

Resources