Lets take the following Golang expressions :
type MyStruct struct {
foo int,
bar map[string]MyInterface
}
type MyInterface interface { /* ... */ }
func firstFunc() { /* ...*/ }
func secondFunc() { /* ...*/ }
with the following builder function :
func NewMyStruct() *MyStruct {
// Building a map made of only static content, known at compile time
// Assigning it to a local variable
m := map[string]MyInterface {
"first": firstFunc,
"second": secondFunc,
}
/* ... */
return &MyStruct{
bar: m, // Copying the map into the struct field
/* ... */
}
}
I came across this code and decided to try optimizing it in terms of memory management and/or execution time.
Coming from the C++ world, I am used to the "static vs dynamic allocation" dilema and the use of constexpr. I am trying to achieve the same goal here in Golang: avoid constructing/allocating a data structure too often in the limits of what the language permits.
Question 1: In NewMyStruct(), is the temporary map effectively assigned to m constructed/allocated at each call?
Question 2: Can the compiler detect that the temporary map is made of static content, and construct it once for all?
My other solution is to go for a globally defined map and refer to it from the MyStruct instances using a pointer.
As Volker suggests, this is almost certainly premature optimization. However, if you've already found that this has some significant time cost in your program and are just looking for options, this Playground link shows a way to construct a map at program startup time and share it. The essence is just:
return &MyStruct{bar: sharedMap, /*...*/}
The shared map needs to be created by this point. If a simple static init is not possible, use an init function, or add a sync.Once to construct the map only once on the first call to your New function.
Related
I'm surprised to see the lack of (in any) modules for a configuration module in Go which is thread safe for concurrent reads/writes.
I'm surprised there is no easy method to basically have something like https://github.com/spf13/viper, but thread safe.. where the Get/Set holds a Lock.
what's the right Go way to handle this without bloating code?
I normally use: https://github.com/spf13/viper however if the program for example as a GUI configuration is changeable during runtime, this package doesn't work.
I started doing the following
var config struct {
lock sync.RWMutex
myString string
myInt int
}
func main() {
config.lock.RLock()
_ = config.myString // any read
config.lock.RUnlock()
}
however this becomes very very tedious when accessing members to each time Lock/Unlock every single access for a read or a write and code becomes bloated and repetitive.
I typically use a configuration struct that is retrieved and updated atomically. This allows multiple fields to be updated in a single "transaction" and is easy to implement/use.
With Go1.18 and earlier this can be implemented with atomic.Value and wrap with some simple helper functions to convert the type from interface{} to your *config.
It is even simpler with atomic.Pointer in Go 1.19:
package main
import (
"sync/atomic"
)
type config struct {
str string
num int
}
var cfg atomic.Pointer[config]
func main() {
// Store initial configuration.
cfg.Store(&config{str: "foo", num: 42})
// Clone and modify multiple fields.
newCfg := *cfg.Load()
newCfg.str = "bar"
newCfg.num++
cfg.Store(&newCfg)
// Retrieve a value.
_ = cfg.Load().str
}
With Go1.18 and earlier you can use atomic.Value and wrap them with some simple helper functions to convert the type from interface{} to your *config.
Here's an implementation of a type which stores a syslog Facility value:
package main
import (
"fmt"
)
// Facility - as defined in RFC 5424
type Facility uint8
var facilityName []string = []string{
"kernel",
"user",
"...",
"local7",
}
func (s Facility) String() string {
if int(s) < len(facilityName) {
return facilityName[s]
} else {
return fmt.Sprintf("facility(%d)!", int(s))
}
}
func UnknownFacility() Facility {
return Facility(1) // "user"
}
The problem with this is that it puts it puts facilityName in the package namespace. The alternative is to create this array or slice inside the String() function, but this creates a new slice/array on every call to String(). The generated code isn't efficient.
Is there a way of doing this efficiently and not polluting the namespace?
UnknownFacility() also feels like Hungarian Notation...
The short answer seems to be No.
The Go compiler (1.18.2) doesn't optimize creating a "constant" array / slice in a function, and there's no equivalent in Go of a C/C++ static variable. Such arrays therefore need to be stored in a package variable to avoid the overhead of recreating them on every function call. (See https://godbolt.org/z/TGPE75Kdf for example compiler output).
There's no mechanism to limit the visibility of a variable to a limited number of functions inside a package. (Go lacks an equivalent to a C++ anonymous namespace)
I'm working around this with a naming convention of prefixing variable names which only need file scope with _filename so they can't collide and are easy to distinguish in IDE auto-completion.
Whether adding variable names which only need file or function scope to the internal package namespace is a significant concern is a different question.
I would like, at any stage in a program, to perform an operation on all instances of a struct defined in a given package. In order to do that, I have contemplated developping a function that gather those instances. But I haven't found anything yet.
To clarify my goal (following '#Muffin Top' and '#Volker' answer), I currently do this registration manualy into a pointer store present in each package. There are two problems with this approach, the registration is a mundane task and it is error prone.
To make my case clear, consider a package foo with one struct Foo. Foo has one string field Name. The package also defines two instances of Foo declared as Foo1 & Foo2
package foo
type Foo struct {
Name string
}
var Foo1, Foo2 = Foo{Name: "Foo1"}, Foo{Name: "Foo2"}
The GetInstancesOfPkg I am trying to implement would be called like in the below example. The operation to perform is just to print the Name field.
This is a working example but with a mockup version of GetInstancesOfPkg.
package main
import (
"log"
"reflect"
"github.com/thomaspeugeot/test-heapdump/foo"
)
func main() {
s := GetInstancesOfPkg("github.com/thomaspeugeot/test-heapdump/foo")
for _, instance := range s {
// cast instance on foo.Foo
if f, ok := instance.(*foo.Foo); !ok {
log.Panic("Unknown Struct " + reflect.TypeOf(instance).Name())
} else {
log.Printf("Instance of Foo has name %s ", f.Name)
}
}
}
// GetInstancesOfPkg return a slice of instances of Struct defined in the package pkgPath
// --- this is the mockup verion ---
func GetInstancesOfPkg(pkgPath string) (interfaceSlice []interface{}) {
var listOfFooInstances = []*foo.Foo{&foo.Foo1, &foo.Foo2}
interfaceSlice = make([]interface{}, len(listOfFooInstances))
for i, d := range listOfFooInstances {
interfaceSlice[i] = d
}
return
}
For the solution, I know there is no such function in the reflect standard library or any other library I know of. However, I was contemplating :
an algorithm that parses the entire program memory, identify each variable, get its type and gather those that match the struct type of the package. Since delve and heapdump do parse the entire memory and that delve can guess the variable type, I have a hunch that my desired function could be implemented. I have been trying to see how delve is doing or how heapcoredump is developped but this is a too big piece of knowledge to swallow for the desired goal.
an alternative would be to add a pre-compilation step that modifies the abstract syntax tree (AST) by adding the registration each time there is a a variable initialisation.
I am not even sure if this level of reflexion is possible with go (or if it is portable ?)
Performance is not an issue, the program could be compiled with wathever flag necessary or the entire memory could be parsed.
Is there a way in go to dynamicaly gather instances of a given struct type?
No.
You must redesign, e.g. by allocating the instances through a function which memorizes them.
I'm making an app that'll need sets of rules to run a job. The app offers the possibility to express the rules in one of several different languages. I therefore have defined an interface to a live rules engine, that offers the methods that the app will need to query the current set of rules. Behind this interface, there will be one different type of engine, according to the source language.
Now I'd like to instantiate a rules engine according to the rule file's extension. But I get some errors which I have a hard time to overcome.
Let me first offer this simplified skeleton :
package main
//
//
// The interface
type RulesEngine interface {
SomeRuleEvaluator(string) bool
}
//
//
// An implementation, with its constructor
type ASimpleRulesEngine struct {
// I've also tried with :
// RulesEngine
}
func NewASimpleRulesEngine(context string) *ASimpleRulesEngine {
re := ASimpleRulesEngine{}
return &re
}
func (re *ASimpleRulesEngine) SomeRuleEvaluator(dummy string) bool {
return true
}
//
//
// A client, that'll want to be able to choose a constructor
var rulesEngineConstructorsPerExtension map[string](func(string) RulesEngine)
func init() {
rulesEngineConstructorsPerExtension = make(map[string](func(string)RulesEngine))
rulesEngineConstructorsPerExtension[".ini"] = NewASimpleRulesEngine
}
func main() {
}
When I try to build this, I get 35: cannot use NewASimpleRulesEngine (type func(string) *ASimpleRulesEngine) as type func(string) RulesEngine in assignment
I've also tried :
assigning without a pointer, although I felt stupid while trying it
having an intermediate step in the initfunction, where I'd create a new(func (string) RulesEngine) and then assign to it, with and without pointer.
storing function pointers like in C, but the compiler said it could not take the adress of my function.
I'm not that familiar with Go and this felt a bit surprising. What would be the proper type signature to use ? Is this possible at all ? If it's unavoidable, I'll obviously have a simple array of extensions on one side (to check if a file is potentially a rules file), and a big switch on the other side to provide the adequate constructor, but as much possible I'd like to avoid such duplication.
Thank you for any insight !
(edited : I've accepted my own answer for lack of any other, but its most relevant part is #seh 's comment below)
Following #JorgeMarey 's comment, and not wanting to sacrifice the constructor's type signature, I came up with this. But it does feel very tacky to me. I'll be glad to hear about a cleaner way.
func init() {
rulesEngineConstructorsPerExtension = make(map[string](func(string)RulesEngine))
cast_NewASimpleRulesEngine := func(content string) RulesEngine {
return NewASimpleRulesEngine(content)
}
rulesEngineConstructorsPerExtension[".ini"] = cast_NewASimpleRulesEngine
}
(an attempt to explicitly cast with (func(string)RulesEngine)( NewASimpleRulesEngine) was deemed unfit by the compiler too)
I am trying to interface with some C code from Go. Using cgo, this has been relatively straight-forward until I hit this (fairly common) case: needing to pass a pointer to a structure that itself contains a pointer to some data. I cannot seem to figure out how to do this from Go without resorting to putting the creation of the structure into the C code itself, which I'd prefer not to do. Here is a snippet that illustrates the problem:
package main
// typedef struct {
// int size;
// void *data;
// } info;
//
// void test(info *infoPtr) {
// // Do something here...
// }
import "C"
import "unsafe"
func main() {
var data uint8 = 5
info := &C.info{size: C.int(unsafe.Sizeof(data)), data: unsafe.Pointer(&data)}
C.test(info)
}
While this compiles fine, trying to run it results in:
panic: runtime error: cgo argument has Go pointer to Go pointer
In my case, the data being passed to the C call doesn't persist past the call (i.e. the C code in question digs into the structure, copies what it needs, then returns).
See "Passing pointers" section in cgo docs:
Go code may pass a Go pointer to C provided the Go memory to which it points does not contain any Go pointers.
And also:
These rules are checked dynamically at runtime. The checking is controlled by the cgocheck setting of the GODEBUG environment variable. The default setting is GODEBUG=cgocheck=1, which implements reasonably cheap dynamic checks. These checks may be disabled entirely using GODEBUG=cgocheck=0. Complete checking of pointer handling, at some cost in run time, is available via GODEBUG=cgocheck=2.
If you run the snippet you've provided with:
GODEBUG=cgocheck=0 go run snippet.go
Then there is no panic. However, the correct way to go is to use C.malloc (or obtain a "C pointer" from somewhere else):
package main
// #include <stdlib.h>
// typedef struct {
// int size;
// void *data;
// } info;
//
// void test(info *infoPtr) {
// // Do something here...
// }
import "C"
import "unsafe"
func main() {
var data uint8 = 5
cdata := C.malloc(C.size_t(unsafe.Sizeof(data)))
*(*C.char)(cdata) = C.char(data)
defer C.free(cdata)
info := &C.info{size: C.int(unsafe.Sizeof(data)), data: cdata}
C.test(info)
}
It works because while regular Go pointers are not allowed, C.malloc returns a "C pointer":
Go pointer means a pointer to memory allocated by Go (such as by using the & operator or calling the predefined new function) and the term C pointer means a pointer to memory allocated by C (such as by a call to C.malloc). Whether a pointer is a Go pointer or a C pointer is a dynamic property determined by how the memory was allocated.
Note that you need to include stdlib.h to use C.free.