I'm interested in using a Go package that uses a global variable which, for my application, is a key "variable". Each time I call this package, I want to set that variable (it's exported). And I'm calling it from various Go routines. Is there a recommended strategy for sequencing/syncing my calls to that package so that my code isn't colliding on that global variable?
Here's an example:
package main
import (
"fmt"
"sync"
"time"
"github.com/jameshaskell/_sketches/globalvars/testlib"
)
var wg sync.WaitGroup
func gr(i int) {
defer wg.Done()
testlib.GlobalVar = i
duration := time.Duration(i) * time.Millisecond
time.Sleep(duration)
fmt.Printf(" pause %d DONE\n", i)
fmt.Printf(" global var: %d should be: %d TestFunc: %s\n", testlib.GlobalVar, i, testlib.TestFunc())
}
func main() {
for i := 0; i <= 10; i += 1 {
wg.Add(1)
go gr(i)
}
wg.Wait()
}
The package I'm trying to access would be like:
package testlib
import "fmt"
var GlobalVar int
func TestFunc() string {
return fmt.Sprintf("GlobalVar: %d\n", GlobalVar)
}
NOTE: the package I'm hoping to use (having the global var) isn't mine...
Is this variable intended to be used this way? May be there’re some specific setters/getters for it? Does documentation have some info about?
I think code becomes very fragile because you do not control all accesses to the variable. You may serialize access from your code with a mutex. But another lib’s code doesn’t know about your it. Moreover it can change suddenly in new versions or bug fixes.
So I prefer:
To read about it in official docs
If documentation allows, and I can’t avoid to use it, protect access to the variable with sync/mutex/atomic/channel depending on a task.
Prepare tests to cover different cases with the variable
Run race condition detector as part of my CI tests.
You could use the atomic package, but beware the warning there:
These functions require great care to be used correctly. Except for
special, low-level applications, synchronization is better done with
channels or the facilities of the sync package. Share memory by
communicating; don't communicate by sharing memory.
Or you could use sync.Mutex to protect the variable but this would require wrapping it in a struct or an access function to properly control access.
There is a broader question though which you should attempt to answer - if you set this variable every time you access the package, why is it a package variable at all? Why not instead just pass it in to the function which needs it? Since it changes every time anyway, there's little point in a package variable quite apart from the synchronisation problems it poses.
Use the atomic package with the Value type
Related
I'm setting up some integration testing, which I'm doing in a separate test package to my src code. This is done to prevent circular dependencies. Unit tests are not stored here, they are stored alongside the files they are testing.
My golang project hierarchy looks like:
cmd
public
...
testing/
main_test.go
database_test.go
in main_test.go, I plan to initialise the connections to external dependencies, such as my test database.
package tests
type Database struct {
...
}
var DB Database
func TestMain(m *testing.M){
SetUpDatabase()
exitCode := m.Run()
os.Exit(exitCode)
}
database_integration_test.go
func Test(t *testing.T) {
tests := []struct {
title string
run func(t *testing.T)
}{
{"should make query", testQuery},
}
for _, test := range tests {
t.Run(test.title, func(t *testing.T) {
test.run(t)
})
}
}
func testQuery(t *testing.T) {
var r result.Result
err := DB.database.DoQuery("").Decode(&r)
if err != nil {
t.Errorf(err.Error())
t.Fatal()
}
}
This setup works when I run it, however, I would like to add build tags to these files, or the type: // +build integration
However, as soon as I use the build tag, the database_integration_test.go file cannot see the initalised Database type. How can I stop this? Also as a side note, should I change the name of main_test.go. I only called it that due to main being the standrd entry point.
Firstly, Regarding this:
Also as a side note, should I change the name of main_test.go. I only
called it that due to main being the standard entry point.
I think it is confusing to name it as main_test.go as it might indicate that you are testing the main function in this file (according to golang convention)
Secondly, Regarding this:
However, as soon as I use the build tag, the
database_integration_test.go file cannot see the initialised Database
type. How can I stop this?
A Build Constraint or also known as Build Tag is used to include or exclude files in a package during a build process. With this, we can build different types of builds from the same source code.
So if you are not seeing the Database Type initialized then most probably the definition of the Database Type and the integration test are tagged with different build tags. Make sure they are present in the same build tags. Also, I think you can use more than one build tag to label a file. So you can try that as well.
For more details on the Build Tags check out the following article by Dave Cheney here
You could simply add a flag to your TestMain:
var isIntegration bool
func init() {
flag.StringVar(&isIntegration, "mytest.integration", "Set flag to set up DB for integration tests")
}
func TestMain(t *testing.M) {
SetUpDatabase(isIntegration)
//etc...
}
Then just simply have the SetUpDatabase call different, unexported, functions based on whether or not the argument is true or false. That'd be a quick way to get the behaviour, without having to much about with custom build constraints. Especially considering you're running tests, not building the application as-such.
As far as renaming main_test.go is concerned: I don't see why you'd need to change it. It does what it says on the tin. When someone else wants to see how the tests are structured/run, or what possible flags have been added, it's a lot easier to just check the directory and look for a main_test.go file (along with init.go, that'd be first file I'd look for). Any other name like setup_integration_test.go, integration_setup_test.go, integration_start_test.go, ... is just going to muddy the waters.
I have this:
var wd, _ = os.Getwd()
var advanceAgreementTemplateBytes, _ = ioutil.ReadFile(path.Join(wd, "/docs/templates/advance-agreement.template"))
var advanceAgreementTemplate = string(advanceAgreementTemplateBytes)
var t, _ = template.New("advance-agreement").Parse(string(advanceAgreementTemplate))
func sendAdvanceEmail(user *common.User, amount *apd.Decimal, fee *apd.Decimal, created time.Time) {
// use variables `t`, `advanceAgreementTemplate` etc here
}
I want to be able to handle the errors in the cached variables instead of using _ instead of err. Is the only way to do this is to use func init()?
If you want to error-check these, use func init or just initialize them in main.
Not all calls have Must variants, and such variants don't make sense for all calls. Don't forget that the Go way is to do proper error checking. Having Must wrappers for all possible failing calls would encourage a bad programming style, where instead of handling and propagating errors, every error would crash by default.
It's very easy to write your own Must wrappers if all you need is to crash the program on error.
The benefit of writing your own Must wrapper is that you could provide a much better error message - another thing Go encourages. ioutil.ReadFile will just say "open /docs/templates/advance-agreement.template: No such file or directory".
I am doing some web develoment using the go programming language using the package html/template. At some point of the code, I need to call the function template.ParseFiles(...) so I can create a template from those files ad then execute it using temp.Execute(w,data). I would like to know if it is better to create the template on each request or to do it once in the main and declare a global variable.
Right now I do it on each request on my handle functions, like most tutorials do. However, I don't know If I'm wasting resources by doing it on each request instead of having them as global variables.
This is how it looks on each request
func ViewStats(w http.ResponseWriter, r *http.Request) {
//Get stuff from db and put them in data
//...
//return data to user
tmp, err := template.ParseFiles("views/guest-layout.html",
"views/stats.html")
if err != nil {
fmt.Println(err)
} else {
tmp.Execute(w,data)
}
}
I would like to know if this is better:
var temp1 template.Template
func main() {
temp1, err = template.ParseFiles("file1","file2")
//...
}
As usual: It depends.
But first some nuance:
You should never do template parsing (or anything else interesting) in your main() function. Instead, your main() function should call methods (or a single method) that kicks off the chain of interesting things in your program.
Go doesn't have globals, so it's not actually an option to store your parsed templates in a global variable in the first place. The closest Go has to global variables is package variables. You could store your parsed templates in a package variable in the main package, but this is also bad form, as your main package (except for tiny, trivial programs), should just be an entry point, and otherwise nearly empty.
But now, on to the core of your question:
Should you parse templates per request, or per run?
And here it depends.
If you have templates that change frequently (i.e. during development, when you're constantly editing your HTML files), once per request can be best.
But this is far less efficient than just parsing once, so in production, you may wish to parse the templates once on startup only. Then you can store the templates in a package variable, or better, in a struct that is initialized at runtime. But I leave that to you.
But what may be best is actually a bit of a compromise between the two approaches. It may be best to load your templates at start-up, and re-load them occasionally, either automatically (say, every 5 minutes), or watch your filesystem, and reload them whenever the on-disk representation of the templates changes.
How to do this is left as an exercise for the reader.
I'm using gin gonic as an HTTP framework and I need to group some paths with shared variable by like this:
ur := r.Group("/")
ur.Use(package.Prepare)
{
ur.GET("/", package.Home)
}
Inside the Prepare handler, I declare package variable like
package.tplPath because I want all sub routes can access to this variable instead of rewrite the code in each http handler.
var tplPath = ""
func Prepare(c *gin.Context) {
_, filename, _, _ := runtime.Caller(0)
s := helper.RelativeFilePath(filename)
tplPath = s[1:len(s)] + "template/"
}
I don't know how Go works with each process, and variables for each http request. If a variable was declared in package level, will it be set after each http request?
Is this considered good practice? If not, why not?
This is not threadsafe, package variables are shared across all goroutines, and modification in one routine will change the value in all the others, causing a data race.
In general; try and avoid using package level variables where possible.
Edit:
In go, packages are a kind of module. There exists only one instance of a package for any given import path (basically package name). This means that there is only 1 instance of any variable at package level.
Package variables are shared, global state. All accessors of that variable will be accessing the exact same memory.
It does not matter what type the package variable is, struct / string / int etc. If it is defined at package level, all accessors of that variable will share the same instance of it.
If (as with http servers) you have concurrent code, there will be multiple accessors of that variable at the same time. In some situations this is fine, like when that variable is only ever read, but it appears that in your case it will be modified. Making this code racy!
The tplPath is a global variable, all routine will access the same memory address, and set after each http request.
If you want set just once, and the tplPath not depend on the http request. You can set it in the init function.
func init(){
_, filename, _, _ := runtime.Caller(0)
s := helper.RelativeFilePath(filename)
tplPath = s[1:len(s)] + "template/"
}
The init function will run before main, and just do once.
In Go one can create one-way channels. It's a very convenient feature in case of one want to restrict a set of operations available on the given channel. However, as far as I can see, this feature is useful only for function's arguments and variable's type specification, while creating one-way channels via make looks strange for me. I've read this question, but it's not about creating read (or write)-only channels in Go, it's about usage in general. So, my question is about use cases of the next code:
writeOnly := make(chan<- string)
readOnly := make(<-chan string)
Theoretically you can use write only channels for unit testing to ensure for example that your code is not writing more than specific number of times to a channel.
Something like this: http://play.golang.org/p/_TPtvBa1OQ
package main
import (
"fmt"
)
func MyCode(someChannel chan<- string) {
someChannel <- "test1"
fmt.Println("1")
someChannel <- "test2"
fmt.Println("2")
someChannel <- "test3"
fmt.Println("3")
}
func main() {
writeOnly := make(chan<- string, 2) // Make sure the code is writing to channel jsut 2 times
MyCode(writeOnly)
}
But that would be pretty silly technique for unit testing. You're better to create a buffered channel and check its contents.
One of the main reason that people use types (especially in Go) is as a form of documentation. Being able to show that a channel is read-only or write-only, can help the consumer of the API have a better idea of what is going on.