Usage of global variables with Http handlers in Golang - go

I know there are some questions and posts/articles regarding this question, but from my newbie view, not exactly.
The thing is, I've got a main program listening to a port and redirecting the calls to a specific handler. The typical structure:
func main() {
http.HandleFunc("/something", specificHandler)
http.ListenAndServe(":8080", nil)
}
With the handler being something like:
func specificHandler(w http.ResponseWriter, r *http.Request) {
somepackage.foo()
}
Then somepackage, which contains the function foo, has some global variables, basically because they're needed for functions to share (e.g., when using a priority queue implemented with a container/heap, which will get the priorities in the Swap function from a global matrix of distances which is of course changable). And many other examples. In summary, global variables...
The problem is, as you might see, that those variables are shared among all the calls to the handler. And that's bad.
How can I actually solve this? There must be an easy to way to do it that I haven't got to yet, because it looks like something so usual...
Thanks in advance.
EDIT
To make it clearer. For example, in my A* package, I've got the following global variables:
var openVerticesAS PriorityQueueAStar
// which vertices are closed
var closedVertices map[int]bool
// which vertices are currently open
var openVertices map[int]bool
// cost from start to node
var gScore map[int]float64
// cost from start to end, passing by node i (g+h)
var fScore map[int]float64
Then, PriorityQueueAStar is implemented as follows:
type PriorityQueueAStar []int // rel id
func (pq PriorityQueueAStar) Len() int { return len(pq) }
func (pq PriorityQueueAStar) Empty() bool { return len(pq) == 0 }
func (pq PriorityQueueAStar) Less(i, j int) bool {
return fScore[pq[i]] < fScore[pq[j]]
}
func (pq PriorityQueueAStar) Swap(i, j int) {
pq[i], pq[j] = pq[j], pq[i]
}
func (pq *PriorityQueueAStar) Push(x interface{}) {
*pq = append(*pq, x.(int))
}
func (pq *PriorityQueueAStar) Pop() interface{} {
old := *pq
n := len(old)
rel := old[n-1]
*pq = old[0 : n-1]
return rel
}
func (pq PriorityQueueAStar) Top() interface{} {
return pq[0]
}
The question then, is, how do I keep doing this without having all those maps as global variables? If they are part of the struct, how do I access the struct from the priority queue functions?

When your handler needs a variable, usually that means you should implement the Handler interface instead of providing a HandlerFunc function.
Here is a BAD example (using global variables):
var globalThing string
func specificHandler(w http.ResponseWriter, r *http.Request) {
w.Write(globalConfigThing)
}
func main() {
globalThing = "Hello world!"
http.HandleFunc("/something", specificHandler)
http.ListenAndServe(":8080", nil)
}
Here is a BETTER example (not using global variables):
type specificHandler struct {
Thing string
}
func (h *specificHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
w.Write(h.Thing)
}
func main() {
http.Handle("/something", &specificHandler{Thing: "Hello world!"})
http.ListenAndServe(":8080", nil)
}
As you can see, a Handler can encapsulate variables.
For completeness, the other approach is to use a function closure. This works well for once-off handlers but is not re-usable and is harder to write unit tests for.
func main() {
scopedThing := "Hello world!"
http.HandleFunc("/something", func (w http.ResponseWriter, r *http.Request) {
w.Write(scopedThing)
})
http.ListenAndServe(":8080", nil)
}
Done correctly, you can now avoid global variables in your package somepackage by passing them as arguments, etc.
EDIT: For example, you can define your handler struct with several PriorityQueueAStar fields from the somepackage package:
type specificHandler struct {
QueueA somepackage.PriorityQueueAStar
QueueB somepackage.PriorityQueueAStar
QueueC somepackage.PriorityQueueAStar
}
func (h *specificHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
h.QueueA.Push(h.QueueB.Pop)
h.QueueB.Push(h.QueueC.Pop)
w.Write([]byte("Queues pushed and popped"))
}

The closure method was mentioned by chowey, but had a caveat that it isn't testable or reusable. It is actually testable and reusable if you have a function to return the closure. For some types of data this may make for a cleaner abstraction and implementation:
func handleThing(thing string) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
w.Write(thing)
}
}
func main() {
http.HandleFunc("/something", handleThing("Hello world!"))
http.ListenAndServe(":8080", nil)
}

Related

Dependency injection in Go

I'm looking for an appropriate way to inject dependencies.
Say I have this code where the FancyWrite and FancyRead functions have a dependency on the WriteToFile and ReadFromFile functions. Since these have side effects I'd like to be able to inject them so I can replace them in tests.
package main
func main() {
FancyWrite()
FancyRead()
}
////////////////
func FancyWrite() {
WriteToFile([]byte("content..."))
}
func FancyRead() {
ReadFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
One thing I tried is just put them as parameters into the functions:
package main
func main() {
FancyWrite(WriteToFile)
FancyRead(ReadFromFile)
}
////////////////
func FancyWrite(writeToFile func(content []byte) (bool, error)) {
writeToFile([]byte("content..."))
}
func FancyRead(readFromFile func(file string) ([]byte, error)) {
readFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
So, this actually works great, but I could see this becoming harder to maintain for more dependencies. I also tried a factory pattern like the following so that the main function doesn't have to concern itself with building the FancyWrite function. But, the syntax is getting a little hard to read and with even more functions would be hard to maintain.
func FancyWriteFactory(writeToFile func(content []byte) (bool, error)) func() {
return func() {
FancyWrite(writeToFile)
}
}
So next I tried housing the functions as methods in a struct:
package main
func main() {
dfu := DefaultFileUtil{}
ffm := FancyFileModule{
FileUtil: &dfu,
}
ffm.FancyWrite()
ffm.FancyRead()
}
////////////////
type FileUtil interface {
WriteToFile(content []byte) (bool, error)
ReadFromFile(file string) ([]byte, error)
}
type FancyFileModule struct {
FileUtil
}
func (fm *FancyFileModule) FancyWrite() {
fm.FileUtil.WriteToFile([]byte("content..."))
}
func (fm *FancyFileModule) FancyRead() {
fm.FileUtil.ReadFromFile("/path/to/file")
}
////////////////
type DefaultFileUtil struct{}
func (fu *DefaultFileUtil) WriteToFile(content []byte) (bool, error) {
return true, nil
}
func (fu *DefaultFileUtil) ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
Now, this actually works well and is cleaner. However, I'm worried I am just shoehorning my functions as methods now and something just felt odd about that. I guess I can reason about it because structs are good when you have some state, and I guess I can count the dependencies as state?
Those are the things I tried. So my question is, what is the proper way to do dependency injection in this case when the only reason to put functions as methods is to make them be a collection of dependencies elsewhere?
Thanks!
The simple answer is that you cannot cleanly use dependency injection with functions, only with methods. Technically, you could make the functions global vars instead (ex. var WriteToFile = func(content []byte) (bool, error) { [...] }), but this is rather brittle code.
The more proper solution, from an idiomatic perspective, is to make any behavior you want to replace, inject, or wrap into a method that is then wrapped in an interface.
For example:
type (
FancyReadWriter interface {
FancyWrite()
FancyRead()
}
fancyReadWriter struct {
w Writer
r Reader
}
Writer interface {
Write([]byte) (bool, error)
}
Reader interface {
Read() ([]byte, error)
}
fileWriter struct {
path string
// or f *os.File
}
fileReader struct {
path string
// or f *os.File
}
)
func (w fileWriter) Write([]byte) (bool, error) {
// Write to the file
return true, nil
}
func (r fileReader) Read() ([]byte, error) {
// Read from the file
return nil, nil
}
func (f fancyReadWriter) FancyWrite() {
// I like to be explicit when I'm ignoring return values,
// hence the underscores.
_, _ = f.w.Write([]byte("some content..."))
}
func (f fancyReadWriter) FancyRead() {
_, _ = f.r.Read()
}
func NewFancyReadWriter(w Writer, r Reader) FancyReadWriter {
// NOTE: Returning a pointer to the struct type, but it is actually
// returned as an interface instead, abstracting the underlying
// implementation.
return &fancyReadWriter{
w: w,
r: r,
}
}
func NewFileReader(path string) Reader {
// Same here, returning a pointer to the struct as the interface
return &fileReader {
path: path
}
}
func NewFileWriter(path string) Writer {
// Same here, returning a pointer to the struct as the interface
return &fileWriter {
path: path
}
}
func Main() {
w := NewFileWriter("/var/some/path")
r := NewFileReader("/var/some/other/path")
f := NewFancyReadWriter(w, r)
f.FancyWrite()
f.FancyRead()
}
And then in the test file (or wherever you want to do the dependency injection):
type MockReader struct {}
func (m MockReader) Read() ([]byte, error) {
return nil, fmt.Errorf("test error 1")
}
type MockWriter struct {}
func (m MockWriter) Write([]byte) (bool, error) {
return false, fmt.Errorf("test error 2")
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
f := NewFancyReadWriter(w, r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
}
You could then go a step further and make the mock or injection framework functional and thus flexible. This is my preferred style for mocks for tests, actually, as it lets me define the behavior of the mocked dependency within the test using that behavior. Example:
type MockReader struct {
Readfunc func() ([]byte, error)
ReadCalled int
}
func (m *MockReader) Read() (ret1 []byte, ret2 error) {
m.ReadCalled++
if m.Readfunc != nil {
// Be *very* careful that you don't just call m.Read() here.
// That would result in an infinite recursion.
ret1, ret2 = m.Readfunc()
}
// if Readfunc == nil, this just returns the zero values
return
}
type MockWriter struct {
Writefunc func([]byte) (bool, error)
WriteCalled int
}
func (m MockWriter) Write(arg1 []byte) (ret1 bool, ret2 error) {
m.WriteCalled++
if m.Writefunc != nil {
ret1, ret2 = m.Writefunc(arg1)
}
// Same here, zero values if the func is nil
return
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
// Note that these definitions are optional. If you don't provide a
// definition, the mock will just return the zero values for the
// return types, so you only need to define these functions if you want
// custom behavior, like different returns or test assertions.
w.Writefunc = func(d []byte) (bool, error) {
// Whatever tests you want, like assertions on the input or w/e
// Then whatever returns you want to test how the caller handles it.
return false, nil
}
r.Readfunc = func() ([]byte, error) {
return nil, nil
}
// Since the mocks now define the methods as *pointer* receiver methods,
// so the mock can keep track of the number of calls, we have to pass in
// the address of the mocks rather than the mocks as struct values.
f := NewFancyReadWriter(&w, &r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
// Now you have a simple way to assert that the calls happened:
if w.WriteCalled < 1 {
t.Fail("Missing expected call to Writer.Write().")
}
if r.ReadCalled < 1 {
t.Fail("Missing expected call to Reader.Read().")
}
}
Since all of the types involved here (the Reader, Writer, and the FancyReadWriter) are all handed around as interfaces rather than concrete types, it also becomes trivial to wrap them with middleware or similar (ex. logging, metrics/tracing, timeout aborts, etc).
This is hands down the most power strength of Go's interface system. Start thinking of types as bags of behavior, attach your behavior to types that can hold them, and pass all behavior types around as interfaces rather than concrete structs (data structs that are just used to organize specific bits of data are perfectly fine without interfaces, else you have to define Getters and Setters for everything and it's a real chore without much benefit). This lets you isolate, wrap, or entirely replace any particular bit of behavior you want at any time.

Add a cache to a go function as if it were a static member

Say I have an expensive function
func veryExpensiveFunction(int) int
and this function gets called a lot for the same number.
Is there a good way to allow this function to store previous results to use if the function gets called again that is perhaps even reusable for veryExpensiveFunction2?
Obviously, it would be possible to add an argument
func veryExpensiveFunctionCached(p int, cache map[int]int) int {
if val, ok := cache[p]; ok {
return val
}
result := veryExpensiveFunction(p)
cache[p] = result
return result
}
But now I have to create the cache somewhere, where I don't care about it. I would rather have it as a "static function member" if this were possible.
What is a good way to simulate a static member cache in go?
You can use closures; and let the closure manage the cache.
func InitExpensiveFuncWithCache() func(p int) int {
var cache = make(map[int]int)
return func(p int) int {
if ret, ok := cache[p]; ok {
fmt.Println("from cache")
return ret
}
// expensive computation
time.Sleep(1 * time.Second)
r := p * 2
cache[p] = r
return r
}
}
func main() {
ExpensiveFuncWithCache := InitExpensiveFuncWithCache()
fmt.Println(ExpensiveFuncWithCache(2))
fmt.Println(ExpensiveFuncWithCache(2))
}
output:
4
from cache
4
veryExpensiveFunctionCached := InitExpensiveFuncWithCache()
and use the wrapped function with your code.
You can try it here.
If you want it to be reusable, change the signature to InitExpensiveFuncWithCache(func(int) int) so it accept a function as a parameter. Wrap it in the closure, replacing the expensive computation part with it.
You need to be careful about synchronization if this cache will be used in http handlers. In Go standard lib, each http request is processed in a dedicated goroutine and at this moment we are at the domain of concurrency and race conditions. I would suggest a RWMutex to ensure data consistency.
As for the cache injection, you may inject it at a function where you create the http handler.
Here it is a prototype
type Cache struct {
store map[int]int
mux sync.RWMutex
}
func NewCache() *Cache {
return &Cache{make(map[int]int), sync.RWMutex{}}
}
func (c *Cache) Set(id, value int) {
c.mux.Lock()
c.store[id] = id
c.mux.Unlock()
}
func (c *Cache) Get(id int) (int, error) {
c.mux.RLock()
v, ok := c.store[id]
c.mux.RUnlock()
if !ok {
return -1, errors.New("a value with given key not found")
}
return v, nil
}
func handleComplexOperation(c *Cache) http.HandlerFunc {
return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request){
})
}
The Go standard library uses the following style for providing "static" functions (e.g. flag.CommandLine) but which leverage underlying state:
// "static" function is just a wrapper
func Lookup(p int) int { return expCache.Lookup(p) }
var expCache = NewCache()
func newCache() *CacheExpensive { return &CacheExpensive{cache: make(map[int]int)} }
type CacheExpensive struct {
l sync.RWMutex // lock for concurrent access
cache map[int]int
}
func (c *CacheExpensive) Lookup(p int) int { /*...*/ }
this design pattern not only allows for simple one-time use, but also allows for segregated usage:
var (
userX = NewCache()
userY = NewCache()
)
userX.Lookup(12)
userY.Lookup(42)

Find structure to which *ast.FuncDecl attached

Here is simple realisation of Handler interface:
type someApi struct {
mu *sync.RWMutex
}
func (api *someApi) ServeHTTP(w http.ResponseWriter, r *http.Request) {}
func NewSomeApi(mu *sync.RWMutex) *someApi {
return &someApi{
mu: mu,
}
}
func (srv *someApi) Create() {
// some realisation
}
func Create() {
// some realisation
}
I want parse file with go/ast and create decorator for someApi.Create function.
It simple to get func name with *ast.FuncDecl.Name, but how can I find which of Create funcs attached to someApi?
Sloved this by iterating through *ast.FuncDecl.Recv.List:
for _, l := range fn.Recv.List { // fn is *ast.FuncDecl
star, ok := l.Type.(*ast.StarExpr)
if !ok {
continue
}
fmt.Println(star.X) // someApi
}

Passing in a function that calls another func

Hello I have 2 funcs that look similar and I would like to create one generic func. My problem is: I am unsure how to pass in another func:
func (b *Business) StreamHandler1(sm streams.Stream, p []*types.People) {
guard := make(chan struct{}, b.maxManifestGoRoutines)
for _, person := range p {
guard <- struct{}{} // would block if guard channel is already filled
go func(n *types.People) {
b.PeopleHandler(sm, n)
<-guard
}(person)
}
}
func (b *Business) StreamHandler2(sm streams.Stream, pi []*types.PeopleInfo) {
guard := make(chan struct{}, b.maxManifestGoRoutines)
for _, personInfo := range pi {
guard <- struct{}{} // would block if guard channel is already filled
go func(n *types.PeopleInfo) {
b.PeopleInfoHandler(sm, n)
<-guard
}(personInfo)
}
}
You can see they both look very, very similar so I would like to make one generic func that I can pass in PeopleInfoHandler and PeopleHandler . Any idea how I can do this correctly? It looks like the syntax from Go I should be able to do something like this:
func (b *Business) StreamHandler1(f func(streams.Stream, interface{}), sm streams.Stream, p []*interface{}) {
But that doesn't seem to be working. Any ideas on how I can make this generic?
You can create abstractions for the type you passed with specific interface types defined.
I use the Peopler interface to either get the People or the PeopleInfo, based on the handler that I defined and that I pass to the new StreamHandler. You can pass the *Business as well in the handler if you need any of its field/method.
But as would Sergio say, if the method is only 5 lines long, even if it is mostly the same, it might not be worth it.
For your pattern with the guard struct, you could use a sync.WaitGroup that would fit better.
package main
import (
"fmt"
"time"
)
func (b *Business) StreamHandler(sm streamsStream, p []Peopler, handler func(streamsStream, Peopler)) {
guard := make(chan struct{}, b.maxManifestGoRoutines)
for _, person := range p {
guard <- struct{}{} // would block if guard channel is already filled
go func(p Peopler) {
handler(sm, p)
<-guard
}(person)
}
}
func peopleInfoHandler(s streamsStream, p Peopler) {
fmt.Println("info:", p.PeopleInfo())
}
func peopleHandler(s streamsStream, p Peopler) {
fmt.Println("people:", p.People())
}
func main() {
b := &Business{maxManifestGoRoutines: 2}
s := streamsStream{}
p := []Peopler{
&People{
Info: PeopleInfo{Name: "you"},
},
}
b.StreamHandler(s, p, peopleInfoHandler)
b.StreamHandler(s, p, peopleHandler)
time.Sleep(time.Second)
}
type streamsStream struct {
}
type People struct {
Info PeopleInfo
}
func (tp *People) People() People {
return *tp
}
type PeopleInfo struct {
Name string
}
func (tp *People) PeopleInfo() PeopleInfo {
return tp.Info
}
type Peopler interface {
People() People
PeopleInfo() PeopleInfo
}
type Business struct {
maxManifestGoRoutines int
}
play link
If possible, you can use dependency inversion to utilize interfaces instead. An association between a type and a function is pretty much the definition of a method. So use an interface to specify the method definition, and pass Business into that method.
The interface will be necessary to prevent an import cycle along with making the code more permissive.
Normally with dependency inversion, the handlers would be explicitly implementing an interface in the same package as Business, but that is all implicit in Go.
For example:
type Handler interface {
Handle(*Business, streams.Stream)
}
func (b *Business) StreamHandler1(sm streams.Stream, hs []Handler) {
guard := make(chan struct{}, b.maxManifestGoRoutines)
for _, h := range hs {
guard <- struct{}{} // would block if guard channel is already filled
go func(n Handler) {
n.Handle(b, sm)
<-guard
}(h)
}
}
Alternatively, if you need a specific functionality from the type, you can have the Business method accept the interface for that behavior. This would be a bit more elegant, but it takes more planning ahead of time across multiple types or a large refactor. For example:
type TypeDoThinger interface {
Type() Schema
DoThing() ImportantValue
}
func (b *Business) HandleTypeDoThinger(sm streams.Stream, t TypeDoThinger) {
sch := t.Type()
// use schema for something
v := t.DoThing()
// save the important data
}
func (b *Business) StreamHandler(sm streams.Stream, ts []TypeDoThinger) {
guard := make(chan struct{}, b.maxManifestGoRoutines)
for _, t := range ts {
guard <- struct{}{} // would block if guard channel is already filled
go func(t TypeDoThinger) {
b.HandleTypeDoThinger(sm, t)
<-guard
}(t)
}
}

http HandleFunc argument in golang

I want to use a rate limiting or throttler library to limit the number of client requests. I use a vendor library in my code base. I want to pass in a ResponseWriter, Request and a third variable retrieved from the URL. When I use the library for throttling, it gives me back a handler that only handles two arguments. How can I pass my third argument into the handler?
Here is my current code:
package main
import (
"fmt"
"github.com/didip/tollbooth"
"net/http"
)
func viewHandler(
w http.ResponseWriter,
r *http.Request,
uniqueId string,
) {
//data := getData(uniqueId)
fmt.Println("Id:", uniqueId)
p := &objects.ModelApp{LoggedUser: "Ryan Hardy", ViewData: "data"}
renderTemplate(w, "view", p)
}
//URL validation for basic web services
var validPath = regexp.MustCompile("^/$|/(home|about|view)/(|[a-zA-Z0-9]+)$")
func makeHandler(
fn func(
http.ResponseWriter,
*http.Request, string,
)) http.HandlerFunc {
return func(
w http.ResponseWriter,
r *http.Request,
) {
m := validPath.FindStringSubmatch(r.URL.Path)
if m == nil {
http.NotFound(w, r)
return
}
fn(w, r, m[2])
}
}
func main() {
http.Handle("/view/", makeHandler(tollbooth.LimitFuncHandler(tollbooth.NewLimiter(1, time.Second), viewHandler)))
http.ListenAndServe(":8080", nil)
}
Could anyone help me with this?
I'm on my phone so this may be difficult to type but you could use the http.Handle function which takes an interface of Handler something like
type makeHandler struct {
YourVariable string
}
func (m *makeHandler) ServeHTTP (w http.ResponseWriter, r *http.Request) {
yourVariableYouNeed := m.YourVariable
// do whatever
w.Write()
}
// do whatever you need to get your variable
blah := &makeHandler{ yourThing }
http.Handle("/views", blah)
On my phone so can't test but it should work, let me know if it doesn't.

Resources