Ensuring a value is retrieved only once - go

Question

I'm developing a Go package to access a web service (via HTTP). Every time I retrieve a page of data from that service, I also get the total of pages available. The only way to get this total is by getting one of the pages (usually the first one). However, requests to this service take time and I need to do the following:
When the GetPage method is called on a Client and the page is retrieved for the first time, the retrieved total should be stored somewhere in that client. When the Total method is called and the total hasn't yet been retrieved, the first page should be fetched and the total returned. If the total was retrieved before, either by a call to GetPage or Total, it should be returned immediately, without any HTTP requests at all. This needs to be safe for use by multiple goroutines. My idea is something along the lines of sync.Once but with the function passed to Do returning a value, which is then cached and automatically returned whenever Do is called.
I remember seeing something like this before, but I can't find it now even though I tried. Searching for sync.Once with value and similar terms didn't yield any useful results. I know I could probably do that with a mutex and a lot of locking, but mutexes and a lot of locking don't seem to be the recommended way to do stuff in go.

Answer 1

General "init-once" solution
In the general / usual case, the easiest solution to only init once, only when it's actually needed is to use sync.Once and its Once.Do() method.
You don't actually need to return any value from the function passed to Once.Do(), because you can store values to e.g. global variables in that function.
See this simple example:
var (
total int
calcTotalOnce sync.Once
)
func GetTotal() int {
// Init / calc total once:
calcTotalOnce.Do(func() {
fmt.Println("Fetching total...")
// Do some heavy work, make HTTP calls, whatever you want:
total++ // This will set total to 1 (once and for all)
})
// Here you can safely use total:
return total
}
func main() {
fmt.Println(GetTotal())
fmt.Println(GetTotal())
}
Output of the above (try it on the Go Playground):
Fetching total...
1
1
Some notes:
You can achieve the same using a mutex or sync.Once, but the latter is actually faster than using a mutex.
If GetTotal() has been called before, subsequent calls to GetTotal() will not do anything but return the previously calculated value, this is what Once.Do() does / ensures. sync.Once "tracks" if its Do() method has been called before, and if so, the passed function value will not be called anymore.
sync.Once provides all the needs for this solution to be safe for concurrent use from multiple goroutines, given that you don't modify or access the total variable directly from anywhere else.
Solution to your "unusal" case
The general case assumes the total is only accessed via the GetTotal() function.
In your case this does not hold: you want to access it via the GetTotal() function and you want to set it after a GetPage() call (if it has not yet been set).
We may solve this with sync.Once too. We would need the above GetTotal() function; and when a GetPage() call is performed, it may use the same calcTotalOnce to attempt to set its value from the received page.
It could look something like this:
var (
total int
calcTotalOnce sync.Once
)
func GetTotal() int {
calcTotalOnce.Do(func() {
// total is not yet initialized: get page and store total number
page := getPageImpl()
total = page.Total
})
// Here you can safely use total:
return total
}
type Page struct {
Total int
}
func GetPage() *Page {
page := getPageImpl()
calcTotalOnce.Do(func() {
// total is not yet initialized, store the value we have:
total = page.Total
})
return page
}
func getPageImpl() *Page {
// Do HTTP call or whatever
page := &Page{}
// Set page.Total from the response body
return page
}
How does this work? We create and use a single sync.Once in the variable calcTotalOnce. This ensures that its Do() method may only call the function passed to it once, no matter where / how this Do() method is called.
If someone calls the GetTotal() function first, then the function literal inside it will run, which calls getPageImpl() to fetch the page and initialize the total variable from the Page.Total field.
If GetPage() function would be called first, that will also call calcTotalOnce.Do() which simply sets the Page.Total value to the total variable.
Whichever route is walked first, that will alter the internal state of calcTotalOnce, which will remember the total calculation has already been run, and further calls to calcTotalOnce.Do() will never call the function value passed to it.
Or just use "eager" initialization
Also note that if it is likely that this total number have to be fetched during the lifetime of your program, it might not worth the above complexity, as you may just as easily initialize the variable once, when it's created.
var Total = getPageImpl().Total
Or if the initialization is a little more complex (e.g. needs error handling), use a package init() function:
var Total int
func init() {
page := getPageImpl()
// Other logic, e.g. error handling
Total = page.Total
}