I have an interface called localcache:
package localcache
type Cache interface {
Set(k string, v interface{}) error
Get(k string) (interface{}, error)
}
and another file containing its implementation
type cache struct {
pool map[string]value
}
type value struct {
data interface{}
expiredAt time.Time
}
func New() *cache {
if cacheInstance == nil {
once.Do(func() {
cacheInstance = &cache{
pool: make(map[string]value),
}
go cacheInstance.spawnCacheChcker()
})
return cacheInstance
}
return cacheInstance
}
func (c *cache) Set(k string, v interface{}) (e error) {
expiredAt := time.Now().Add(expiredIn)
c.pool[k] = value{data: v, expiredAt: expiredAt}
e = nil
return
}
func (c *cache) Get(k string) (v interface{}, e error) {
v = c.pool[k].data
e = nil
return
}
func (c *cache) spawnCacheChcker() {
for {
for k, v := range c.pool {
if !(v.expiredAt.Before(time.Now())) {
continue
}
c.evictCache(k)
}
time.Sleep(checkInBetween)
}
}
a cache will be expired 30 seconds after it's been set, how can I test this functionality?
I'm using testify rn, my brute force solution was to time.Sleep in the test function, but I feel like this will prolong the entire test process, which is not the best practice.
Is there any ways to mock the expiredAt inside the Set function? Or is there any workaround that tests this better?
You can move time.Now() to be a field in cache struct like
type cache struct {
nowFunc func() time.Time
}
then assign time.Now to it in the constructor. Within the test, just modify it to your mock function to return any time you want.
Related
There is an Interface declare and many structs that implement it
type DataInterface interface {
Get(string) string
}
type DataA struct {
d map[string]string
}
func (d *DataA) Get(key string) string {
return d.d[key]
}
func (d *DataA) GetId() string {
return d.Get("id")
}
type DataB struct {
d map[string]string
}
func (d *DataB) Get(key string) string {
return d.d[key]
}
func (d *DataB) GetFile() string {
return d.Get("file")
}
type DataC...
Also includes DataC,D,E...
and I will store these DataX structs instance into a type DataSlice []DataInterface
Now, If I want to get DataX , I can do this:
type DataSlice []DataInterface
func (d DataSlice) GetA() []*DataA {
var ret []*DataA
for _, di := range d {
if v, ok := di.(*DataA); ok {
ret = append(ret, v)
}
}
return ret
}
func (d DataSlice) GetB() []*DataB {
var ret []*DataB
for _, di := range d {
if v, ok := di.(*DataB); ok {
ret = append(ret, v)
}
}
return ret
}
func (d DataSlice) GetC() .....
Obviously there's a lot of repetitive code here:
var ret []*DataX
for _, di := range d {
if v, ok := di.(*DataX); ok {
ret = append(ret, v)
}
}
So I think about that I can use generic to slove this, then I define this function:
func GetDataX[T any] (d DataInterface) *T {
return d.(*T)
}
but got error: Impossible type assertion: '*T' does not implement 'DataInterface
So, I want to know is this way really impossible? Or it could be completed by the other way?
You should be able to handle your needs with the following code:
package main
import "fmt"
// interface
type DataInterface interface {
Get(string) string
}
// struct implementing the interface
type DataA struct {
d map[string]string
}
func (d DataA) Get(key string) string {
return d.d[key]
}
type DataB struct {
d map[string]string
}
func (d DataB) Get(key string) string {
return d.d[key]
}
type DataSlice []DataInterface
func GetDataX[T any](d DataInterface) T {
return d.(T)
}
func main() {
a := DataA{map[string]string{"a": "1"}}
b := DataB{map[string]string{"b": "2"}}
ds := DataSlice{a, b}
for _, v := range ds {
if value, ok := v.(DataA); ok {
fmt.Printf("A\t%q\n", GetDataX[DataA](value))
continue
}
if value, ok := v.(DataB); ok {
fmt.Printf("B\t%q\n", GetDataX[DataB](value))
continue
}
// add unknown type handling logic here
}
}
First, I simplified the code to take into consideration only the DataA and DataB structs. Then, I changed the pointer receivers to value receivers as you're not going to change the state of the actual instance passed to the methods. Thanks to this change the GetDataX works successfully and you're able to get the info for all of your similar structs.
Let me know if this solves your issues or if you need something else, thanks!
I am using Echo framework and want to pass the Go's built-in context.Context underlying echo.Context after setting some custom values.
To achieve it, I think I could first apply Set(key string, val interface{}) method of echo.Context and then extract the underlying context.Context.
Question is is it possible to do it this way? In other words, does echo.Context.Set(...) sets the value directly on the context.Context just like WithValue does? Or should I take extra steps to copy my custom entries down.
P.S. I do not want to pass echo.Context to deeper layers of my app, that's why I do not want to directly use it but get the referring context.Context
Method 1: Reimplement the echo.Context.Get and echo.Context.Set methods to manipulate the ctx.Request().Context() object.
Disadvantages: http.Request.WithContext will be called once for each Set method, and *http.Request will be copied once. See the implementation of WithContext method for details.
Method 2: Reimplement the echo.Context.Get and echo.Context.Set methods to manipulate the contextValueData2 object, and set http.Request.WithContext to a custom context.Context contextValueData2.
Disadvantages: Before go1.13, context.Context requires Type assertions. Don't implement the context.Context method. Compared with method 1, the implementation only requires WithContext once.
It is recommended to use method 1, which is clear and simple, and method 2 is complicated and not fully tested.
The example import package uses gopath, and the implementation of this feature also reflects the advantage of echo.Context as an interface.
package main
import (
"context"
"fmt"
"github.com/labstack/echo"
"github.com/labstack/echo/middleware"
"net/http"
)
func main() {
// Echo instance
e := echo.New()
// Middleware
e.Use(NewMiddlewareContextValue)
e.Use(middleware.Logger())
e.Use(middleware.Recover())
// Routes
e.GET("/", hello)
e.GET("/val", getval)
// Start server
e.Logger.Fatal(e.Start(":1323"))
}
// Handler
func hello(c echo.Context) error {
return c.String(http.StatusOK, "Hello, World!")
}
func getval(c echo.Context) error {
c.Set("111", "aa")
c.Set("222", "bb")
return c.String(http.StatusOK, fmt.Sprint(c.Request().Context()))
}
// ---------- method1 ----------
func NewMiddlewareContextValue(fn echo.HandlerFunc) echo.HandlerFunc {
return func(ctx echo.Context) error {
return fn(contextValue{ctx})
}
}
type contextValue struct {
echo.Context
}
// Get retrieves data from the context.
func (ctx contextValue) Get(key string) interface{} {
// get old context value
val := ctx.Context.Get(key)
if val != nil {
return val
}
return ctx.Request().Context().Value(key)
}
// Set saves data in the context.
func (ctx contextValue) Set(key string, val interface{}) {
ctx.SetRequest(ctx.Request().WithContext(context.WithValue(ctx.Request().Context(), key, val)))
}
// ---------- method2 ----------
func NewMiddlewareContextValue2(fn echo.HandlerFunc) echo.HandlerFunc {
return func(ctx echo.Context) error {
ctxdata := contextValueData2{
Context: ctx.Request().Context(),
}
ctx.SetRequest(ctx.Request().WithContext(ctxdata))
return fn(&contextValue2{Context: ctx, contextValueData2: ctxdata})
}
}
type contextValue2 struct {
echo.Context
contextValueData2
}
type contextValueData2 struct {
context.Context
Data map[string]interface{}
}
// Get retrieves data from the context.
func (ctx *contextValue2) Get(key string) interface{} {
// get old context value
val := ctx.Context.Get(key)
if val != nil {
return val
}
// get my data value
val, ok := ctx.contextValueData2.Data[key]
if ok {
return val
}
return ctx.contextValueData2.Context.Value(key)
}
// Set saves data in the context.
func (ctx *contextValue2) Set(key string, val interface{}) {
if ctx.Data == nil {
ctx.contextValueData2.Data = make(map[string]interface{})
}
ctx.contextValueData2.Data[key] = val
}
func (ctx contextValueData2) Value(key interface{}) interface{} {
str, ok := key.(string)
if ok {
val, ok := ctx.Data[str]
if ok {
return val
}
}
return ctx.Context.Value(key)
}
I'm looking for an appropriate way to inject dependencies.
Say I have this code where the FancyWrite and FancyRead functions have a dependency on the WriteToFile and ReadFromFile functions. Since these have side effects I'd like to be able to inject them so I can replace them in tests.
package main
func main() {
FancyWrite()
FancyRead()
}
////////////////
func FancyWrite() {
WriteToFile([]byte("content..."))
}
func FancyRead() {
ReadFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
One thing I tried is just put them as parameters into the functions:
package main
func main() {
FancyWrite(WriteToFile)
FancyRead(ReadFromFile)
}
////////////////
func FancyWrite(writeToFile func(content []byte) (bool, error)) {
writeToFile([]byte("content..."))
}
func FancyRead(readFromFile func(file string) ([]byte, error)) {
readFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
So, this actually works great, but I could see this becoming harder to maintain for more dependencies. I also tried a factory pattern like the following so that the main function doesn't have to concern itself with building the FancyWrite function. But, the syntax is getting a little hard to read and with even more functions would be hard to maintain.
func FancyWriteFactory(writeToFile func(content []byte) (bool, error)) func() {
return func() {
FancyWrite(writeToFile)
}
}
So next I tried housing the functions as methods in a struct:
package main
func main() {
dfu := DefaultFileUtil{}
ffm := FancyFileModule{
FileUtil: &dfu,
}
ffm.FancyWrite()
ffm.FancyRead()
}
////////////////
type FileUtil interface {
WriteToFile(content []byte) (bool, error)
ReadFromFile(file string) ([]byte, error)
}
type FancyFileModule struct {
FileUtil
}
func (fm *FancyFileModule) FancyWrite() {
fm.FileUtil.WriteToFile([]byte("content..."))
}
func (fm *FancyFileModule) FancyRead() {
fm.FileUtil.ReadFromFile("/path/to/file")
}
////////////////
type DefaultFileUtil struct{}
func (fu *DefaultFileUtil) WriteToFile(content []byte) (bool, error) {
return true, nil
}
func (fu *DefaultFileUtil) ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
Now, this actually works well and is cleaner. However, I'm worried I am just shoehorning my functions as methods now and something just felt odd about that. I guess I can reason about it because structs are good when you have some state, and I guess I can count the dependencies as state?
Those are the things I tried. So my question is, what is the proper way to do dependency injection in this case when the only reason to put functions as methods is to make them be a collection of dependencies elsewhere?
Thanks!
The simple answer is that you cannot cleanly use dependency injection with functions, only with methods. Technically, you could make the functions global vars instead (ex. var WriteToFile = func(content []byte) (bool, error) { [...] }), but this is rather brittle code.
The more proper solution, from an idiomatic perspective, is to make any behavior you want to replace, inject, or wrap into a method that is then wrapped in an interface.
For example:
type (
FancyReadWriter interface {
FancyWrite()
FancyRead()
}
fancyReadWriter struct {
w Writer
r Reader
}
Writer interface {
Write([]byte) (bool, error)
}
Reader interface {
Read() ([]byte, error)
}
fileWriter struct {
path string
// or f *os.File
}
fileReader struct {
path string
// or f *os.File
}
)
func (w fileWriter) Write([]byte) (bool, error) {
// Write to the file
return true, nil
}
func (r fileReader) Read() ([]byte, error) {
// Read from the file
return nil, nil
}
func (f fancyReadWriter) FancyWrite() {
// I like to be explicit when I'm ignoring return values,
// hence the underscores.
_, _ = f.w.Write([]byte("some content..."))
}
func (f fancyReadWriter) FancyRead() {
_, _ = f.r.Read()
}
func NewFancyReadWriter(w Writer, r Reader) FancyReadWriter {
// NOTE: Returning a pointer to the struct type, but it is actually
// returned as an interface instead, abstracting the underlying
// implementation.
return &fancyReadWriter{
w: w,
r: r,
}
}
func NewFileReader(path string) Reader {
// Same here, returning a pointer to the struct as the interface
return &fileReader {
path: path
}
}
func NewFileWriter(path string) Writer {
// Same here, returning a pointer to the struct as the interface
return &fileWriter {
path: path
}
}
func Main() {
w := NewFileWriter("/var/some/path")
r := NewFileReader("/var/some/other/path")
f := NewFancyReadWriter(w, r)
f.FancyWrite()
f.FancyRead()
}
And then in the test file (or wherever you want to do the dependency injection):
type MockReader struct {}
func (m MockReader) Read() ([]byte, error) {
return nil, fmt.Errorf("test error 1")
}
type MockWriter struct {}
func (m MockWriter) Write([]byte) (bool, error) {
return false, fmt.Errorf("test error 2")
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
f := NewFancyReadWriter(w, r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
}
You could then go a step further and make the mock or injection framework functional and thus flexible. This is my preferred style for mocks for tests, actually, as it lets me define the behavior of the mocked dependency within the test using that behavior. Example:
type MockReader struct {
Readfunc func() ([]byte, error)
ReadCalled int
}
func (m *MockReader) Read() (ret1 []byte, ret2 error) {
m.ReadCalled++
if m.Readfunc != nil {
// Be *very* careful that you don't just call m.Read() here.
// That would result in an infinite recursion.
ret1, ret2 = m.Readfunc()
}
// if Readfunc == nil, this just returns the zero values
return
}
type MockWriter struct {
Writefunc func([]byte) (bool, error)
WriteCalled int
}
func (m MockWriter) Write(arg1 []byte) (ret1 bool, ret2 error) {
m.WriteCalled++
if m.Writefunc != nil {
ret1, ret2 = m.Writefunc(arg1)
}
// Same here, zero values if the func is nil
return
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
// Note that these definitions are optional. If you don't provide a
// definition, the mock will just return the zero values for the
// return types, so you only need to define these functions if you want
// custom behavior, like different returns or test assertions.
w.Writefunc = func(d []byte) (bool, error) {
// Whatever tests you want, like assertions on the input or w/e
// Then whatever returns you want to test how the caller handles it.
return false, nil
}
r.Readfunc = func() ([]byte, error) {
return nil, nil
}
// Since the mocks now define the methods as *pointer* receiver methods,
// so the mock can keep track of the number of calls, we have to pass in
// the address of the mocks rather than the mocks as struct values.
f := NewFancyReadWriter(&w, &r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
// Now you have a simple way to assert that the calls happened:
if w.WriteCalled < 1 {
t.Fail("Missing expected call to Writer.Write().")
}
if r.ReadCalled < 1 {
t.Fail("Missing expected call to Reader.Read().")
}
}
Since all of the types involved here (the Reader, Writer, and the FancyReadWriter) are all handed around as interfaces rather than concrete types, it also becomes trivial to wrap them with middleware or similar (ex. logging, metrics/tracing, timeout aborts, etc).
This is hands down the most power strength of Go's interface system. Start thinking of types as bags of behavior, attach your behavior to types that can hold them, and pass all behavior types around as interfaces rather than concrete structs (data structs that are just used to organize specific bits of data are perfectly fine without interfaces, else you have to define Getters and Setters for everything and it's a real chore without much benefit). This lets you isolate, wrap, or entirely replace any particular bit of behavior you want at any time.
Say I have an expensive function
func veryExpensiveFunction(int) int
and this function gets called a lot for the same number.
Is there a good way to allow this function to store previous results to use if the function gets called again that is perhaps even reusable for veryExpensiveFunction2?
Obviously, it would be possible to add an argument
func veryExpensiveFunctionCached(p int, cache map[int]int) int {
if val, ok := cache[p]; ok {
return val
}
result := veryExpensiveFunction(p)
cache[p] = result
return result
}
But now I have to create the cache somewhere, where I don't care about it. I would rather have it as a "static function member" if this were possible.
What is a good way to simulate a static member cache in go?
You can use closures; and let the closure manage the cache.
func InitExpensiveFuncWithCache() func(p int) int {
var cache = make(map[int]int)
return func(p int) int {
if ret, ok := cache[p]; ok {
fmt.Println("from cache")
return ret
}
// expensive computation
time.Sleep(1 * time.Second)
r := p * 2
cache[p] = r
return r
}
}
func main() {
ExpensiveFuncWithCache := InitExpensiveFuncWithCache()
fmt.Println(ExpensiveFuncWithCache(2))
fmt.Println(ExpensiveFuncWithCache(2))
}
output:
4
from cache
4
veryExpensiveFunctionCached := InitExpensiveFuncWithCache()
and use the wrapped function with your code.
You can try it here.
If you want it to be reusable, change the signature to InitExpensiveFuncWithCache(func(int) int) so it accept a function as a parameter. Wrap it in the closure, replacing the expensive computation part with it.
You need to be careful about synchronization if this cache will be used in http handlers. In Go standard lib, each http request is processed in a dedicated goroutine and at this moment we are at the domain of concurrency and race conditions. I would suggest a RWMutex to ensure data consistency.
As for the cache injection, you may inject it at a function where you create the http handler.
Here it is a prototype
type Cache struct {
store map[int]int
mux sync.RWMutex
}
func NewCache() *Cache {
return &Cache{make(map[int]int), sync.RWMutex{}}
}
func (c *Cache) Set(id, value int) {
c.mux.Lock()
c.store[id] = id
c.mux.Unlock()
}
func (c *Cache) Get(id int) (int, error) {
c.mux.RLock()
v, ok := c.store[id]
c.mux.RUnlock()
if !ok {
return -1, errors.New("a value with given key not found")
}
return v, nil
}
func handleComplexOperation(c *Cache) http.HandlerFunc {
return http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request){
})
}
The Go standard library uses the following style for providing "static" functions (e.g. flag.CommandLine) but which leverage underlying state:
// "static" function is just a wrapper
func Lookup(p int) int { return expCache.Lookup(p) }
var expCache = NewCache()
func newCache() *CacheExpensive { return &CacheExpensive{cache: make(map[int]int)} }
type CacheExpensive struct {
l sync.RWMutex // lock for concurrent access
cache map[int]int
}
func (c *CacheExpensive) Lookup(p int) int { /*...*/ }
this design pattern not only allows for simple one-time use, but also allows for segregated usage:
var (
userX = NewCache()
userY = NewCache()
)
userX.Lookup(12)
userY.Lookup(42)
Suppose I have the following struct:
package manager
type Manager struct {
strings []string
}
func (m *Manager) AddString(s string) {
m.strings = append(m.strings, s)
}
func (m *Manager) RemoveString(s string) {
for i, str := range m.strings {
if str == s {
m.strings = append(m.strings[:i], m.strings[i+1:]...)
}
}
}
This pattern is not thread safe, so the following test fails due to some race condition (array index out of bounds):
func TestManagerConcurrently(t *testing.T) {
m := &manager.Manager{}
wg := sync.WaitGroup{}
for i:=0; i<100; i++ {
wg.Add(1)
go func () {
m.AddString("a")
m.AddString("b")
m.AddString("c")
m.RemoveString("b")
wg.Done()
} ()
}
wg.Wait()
fmt.Println(m)
}
I'm new to Go, and from googling around I suppose I should use channels (?). So one way to make this concurrent would be like this:
type ManagerA struct {
Manager
addStringChan chan string
removeStringChan chan string
}
func NewManagerA() *ManagerA {
ma := &ManagerA{
addStringChan: make(chan string),
removeStringChan: make(chan string),
}
go func () {
for {
select {
case msg := <-ma.addStringChan:
ma.AddString(msg)
case msg := <-ma.removeStringChan:
ma.RemoveString(msg)
}
}
}()
return ma
}
func (m* ManagerA) AddStringA(s string) {
m.addStringChan <- s
}
func (m* ManagerA) RemoveStringA(s string) {
m.removeStringChan <- s
}
I would like to expose an API similar to the non-concurrent example, hence AddStringA, RemoveStringA.
This seems to work as expected concurrently (although I guess the inner goroutine should also exit at some point). My problem with this is that there is a lot of extra boilerplate:
need to define & initialize channels
define inner goroutine loop with select
map functions to channel calls
It seems a bit much to me. Is there a way to simplify this (refactor / syntax / library)?
I think the best way to implement this would be to use a Mutex instead? But is it still possible to simplify this sort of boilerplate?
Using a mutex would be perfectly idiomatic like this:
type Manager struct {
mu sync.Mutex
strings []string
}
func (m *Manager) AddString(s string) {
m.mu.Lock()
m.strings = append(m.strings, s)
m.mu.Unlock()
}
func (m *Manager) RemoveString(s string) {
m.mu.Lock()
for i, str := range m.strings {
if str == s {
m.strings = append(m.strings[:i], m.strings[i+1:]...)
}
}
m.mu.Unlock()
}
You could do this with channels, but as you noted it is a lot of extra work for not much gain. Just use a mutex is my advice!
If you simply need to make the access to the struct thread-safe, use mutex:
type Manager struct {
sync.Mutex
data []string
}
func (m *Manager) AddString(s string) {
m.Lock()
m.strings = append(m.strings, s)
m.Unlock()
}