Let's assume I declared two maps and want to assign it in two different goroutines in error group. I don't perform any reads/write.
Should I protect assign operation with lock or I can omit it?
UPD3: In the Java Concurrency In Practice Brian Goetz's Part I Chapter 3 Shared Objects, mentioned:
Locking is not just about mutual exclusion; it is also memory
visibility. To ensure that all threads see the most up-to-date values
of shared mutable variables, the reading and writing threads must
synchronize on a common lock.
var (
mu sync.Mutex
one map[string]struct{}
two map[string]struct{}
)
g, gctx := errgroup.WithContext(ctx)
g.Go(func() error {
resp, err := invokeFirstService(gctx, request)
if err != nil {
return err
}
mu.Lock()
one = resp.One
mu.Unlock()
return nil
})
g.Go(func() error {
resp, err := invokeSecondService(gctx, request)
if err != nil {
return err
}
mu.Lock()
two = resp.Two
mu.Unlock()
return nil
})
if err := g.Wait(); err != nil {
return err
}
// UPD3: added lock and unlock section
m.Lock()
defer m.Unlock()
performAction(one, two)
UPD: added more context about variables
UPD2: what were my doubts: we have 3 goroutines - parent and two in the error group. there is no guarantee that our parent goroutine shared memory gets the last update after errgroup goroutines complete until we wrap access to shared memory with memory barriers
Group.Wait() blocks until all function calls from the Group.Go() method have returned, so this is a synchronization point. This ensures performAction(one, two) will not start before any writes to one and two are done, so in your example the mutex is unnecessary.
g, gctx := errgroup.WithContext(ctx)
g.Go(func() error {
// ...
one = resp.One
return nil
})
g.Go(func() error {
// ...
two = resp.Two
return nil
})
if err := g.Wait(); err != nil {
return err
}
// Here you can access one and two safely:
performAction(one, two)
If you would access one and two from other goroutines while the goroutines that write them run concurrently, then yes, you would need to lock them, e.g.:
// This goroutine runs concurrently, so all concurrent access must be synchronized:
go func() {
mu.Lock()
fmt.Println(one, two)
mu.Unlock()
}()
g, gctx := errgroup.WithContext(ctx)
g.Go(func() error {
// ...
mu.Lock()
one = resp.One
mu.Unlock()
return nil
})
g.Go(func() error {
// ...
mu.Lock()
two = resp.Two
mu.Unlock()
return nil
})
if err := g.Wait(); err != nil {
return err
}
// Note that you don't need to lock here
// if the first concurrent goroutine only reads one and two.
performAction(one, two)
Also note that in the above example you could use sync.RWMutex, and in the goroutine that reads them, RWMutex.RLock() and RWMutex.RUnlock() would also be sufficient.
In this case, only one goroutine can access the map. I don't think you need a lock.
Related
Background: currently I am investigating the mechanism of distributed lock in order to implement it in our email service to deduplicate in the time the QPS is high.
The main article I am reading is this one https://redis.io/docs/reference/patterns/distributed-locks/#retry-on-failure from redis official.
In this article it mentioned in the retry section that
When a client is unable to acquire the lock, it should try again after a random delay in order to try to desynchronize multiple clients trying to acquire the lock for the same resource at the same time (this may result in a split brain condition where nobody wins)
Also the faster a client tries to acquire the lock in the majority of Redis instances, the smaller the window for a split brain condition (and the need for a retry), so ideally the client should try to send the SET commands to the N instances at the same time using multiplexing.
The fact that when a client needs to retry a lock, it waits a time which is comparably greater than the time needed to acquire the majority of locks, in order to probabilistically make split brain conditions during resource contention unlikely.
I read a few recommended open source implementation of redlock from git, indeed there is delay between retry, copy and pasted here for your convenience.
func (m *Mutex) LockContext(ctx context.Context) error {
if ctx == nil {
ctx = context.Background()
}
value, err := m.genValueFunc()
if err != nil {
return err
}
for i := 0; i < m.tries; i++ {
if i != 0 {
select {
case <-ctx.Done():
// Exit early if the context is done.
return ErrFailed
case <-time.After(m.delayFunc(i)):
// Fall-through when the delay timer completes.
}
}
start := time.Now()
n, err := func() (int, error) {
ctx, cancel := context.WithTimeout(ctx, time.Duration(int64(float64(m.expiry)*m.timeoutFactor)))
defer cancel()
return m.actOnPoolsAsync(func(pool redis.Pool) (bool, error) {
return m.acquire(ctx, pool, value)
})
}()
if n == 0 && err != nil {
return err
}
now := time.Now()
until := now.Add(m.expiry - now.Sub(start) - time.Duration(int64(float64(m.expiry)*m.driftFactor)))
if n >= m.quorum && now.Before(until) {
m.value = value
m.until = until
return nil
}
_, err = func() (int, error) {
ctx, cancel := context.WithTimeout(ctx, time.Duration(int64(float64(m.expiry)*m.timeoutFactor)))
defer cancel()
return m.actOnPoolsAsync(func(pool redis.Pool) (bool, error) {
return m.release(ctx, pool, value)
})
}()
if i == m.tries-1 && err != nil {
return err
}
}
return ErrFailed
}
What I do not understand is that why the existence of the retry can help to avoid brain split?
Intuitively the backoff time(delay) for the retry is used to avoid overloading the redis instance.
I have a slice clientFiles which I am iterating over sequentially and writing it in S3 one by one as shown below:
for _, v := range clientFiles {
err := writeToS3(v.FileContent, s3Connection, v.FileName, bucketName, v.FolderName)
if err != nil {
fmt.Println(err)
}
}
The above code works fine but I want to write in S3 in parallel so that I can speed things up. Does worker pool implementation works better here or is there any other better option here?
I got below code which uses wait group but I am not sure if this is better option here to work with?
wg := sync.WaitGroup{}
for _, v := range clientFiles {
wg.Add(1)
go func(v ClientMapFile) {
err := writeToS3(v.FileContent, s3Connection, v.FileName, bucketName, v.FolderName)
if err != nil {
fmt.Println(err)
}
}(v)
}
Yes, parallelising should help.
Your code should work well after changes regarding usage of WaitGroup. You need to mark work as Done and wait for all goroutines to finish after for-loop.
var wg sync.WaitGroup
for _, v := range clientFiles {
wg.Add(1)
go func(v ClientMapFile) {
defer wg.Done()
err := writeToS3(v.FileContent, s3Connection, v.FileName, bucketName, v.FolderName)
if err != nil {
fmt.Println(err)
}
}(v)
}
wg.Wait()
Be aware that your solution creates N goroutines for N files, which can be not optimal if number of files is very big. In such case use this pattern https://gobyexample.com/worker-pools and try different number of workers to find which works best for you in terms of performance.
given the following variables
wg.Add(4)
pDayResCh := make(chan map[string]map[string]int)
pDayErrCh := make(chan error)
The following code hangs
// Convert the previous, current, and next day prayers and put them in map
go func() {
fmt.Println("executing first go routine") // TODO: Remove this line after debug
data, err := dayPrayerMapConv(previousDayPrayers)
fmt.Printf("first goroutine result %v\n", data) // TODO: Remove this line after debug
if err != nil {
fmt.Printf("first goroutine err != nil %v\n", err)
pDayErrCh <- err
}
fmt.Printf("first goroutine putting data into channel")
pDayResCh <- data
fmt.Printf("first go routine finished") // TODO: Remove this line after debug
wg.Done()
}()
pDayErr := <-pDayErrCh
close(pDayErrCh)
if pDayErr != nil {
return pDayErr
}
fmt.Println("pday err finised")
p.PreviousDayPrayers = <-pDayResCh
close(pDayResCh)
This is the result of the print statement
first goroutine result map[Asr:map[Hour:3 Minute:28] Dhuhr:map[Hour:12 Minute:23] Fajr:map[Hour:5 Minute:32] Isha:map[Hour:7 Minute:5] Maghrib:map[Hour:6 Minute:13]]
first goroutine putting data into channel
So there is data in the data variable that should have been passed into pDayResCh, but it seems to get stuck there, why?
Due to the condition in my goroutine where I check for the error
if err != nil {
fmt.Printf("first goroutine err != nil %v\n", err)
pDayErrCh <- err
}
Because error is nil, that data never gets passed into the channel, thus hangs
Removing the condition and passing the err regardless solves the issue.
The objective of my backend service is to process 90 milllion data and at least 10 million of data in 1 day.
My system config:
Ram 2000 Mb
CPU 2core(s)
what I am doing right now is something like this:
var wg sync.WaitGroup
//length of evs is 4455
for i, ev := range evs {
wg.Add(1)
go migrate(&wg)
}
wg.Wait()
func migrate(wg *sync.WaitGroup) {
defer wg.Done()
//processing
time.Sleep(time.Second)
}
Without knowing more detail about the type of work you need to do, your approach seems good. Some things to think about:
Re-using variables and or clients in your processing loop. For example reusing an HTTP client instead of recreating one.
Depending on how your use case calls to handle failures. It might be efficient to use erroGroup. It's a convenience wrapper that stops all the threads on error possibly saving you a lot of time.
In the migrate function be sure to be aware of the caveats regarding closure and goroutines.
func main() {
g := new(errgroup.Group)
var urls = []string{
"http://www.someasdfasdfstupidname.com/",
"ftp://www.golang.org/",
"http://www.google.com/",
}
for _, url := range urls {
url := url // https://golang.org/doc/faq#closures_and_goroutines
g.Go(func() error {
resp, err := http.Get(url)
if err == nil {
resp.Body.Close()
}
return err
})
}
fmt.Println("waiting")
if err := g.Wait(); err == nil {
fmt.Println("Successfully fetched all URLs.")
} else {
fmt.Println(err)
}
}
I have got the solution. to achieve this much huge processing what I have done is
a limited number of goroutine to 50 and increased the number of cores from 2 to 5.
I read Twelve Go Best Practices and encounter and interesting example on page 30.
func sendMsg(msg, addr string) error {
conn, err := net.Dial("tcp", addr)
if err != nil {
return err
}
defer conn.Close()
_, err = fmt.Fprint(conn, msg)
return err
}
func broadcastMsg(msg string, addrs []string) error {
errc := make(chan error)
for _, addr := range addrs {
go func(addr string) {
errc <- sendMsg(msg, addr)
fmt.Println("done")
}(addr)
}
for _ = range addrs {
if err := <-errc; err != nil {
return err
}
}
return nil
}
func main() {
addr := []string{"localhost:8080", "http://google.com"}
err := broadcastMsg("hi", addr)
time.Sleep(time.Second)
if err != nil {
fmt.Println(err)
return
}
fmt.Println("everything went fine")
}
The programmer mentioned, that happens to the code above:
the goroutine is blocked on the chan write
the goroutine holds a reference to the chan
the chan will never be garbage collected
Why the goroutine is blocked here? The main thread is blocked, until it receive data from goroutine. After it continues the for loop. Not?
Why the errc chan will be never garbage collected? Because I do not close the channel, after goroutine is finished?
One problem I see is that inside broadcastMsg() after goroutines have started:
for _ = range addrs {
if err := <-errc; err != nil {
return err
}
}
If a non-nil error is received from errc, broadcastMsg() returns immediately with that error and does not receive futher values from the channel, which means further goroutines will never get unblocked because errc is unbuffered.
Possible Fixes
A possible fix would be to use a buffered channel, big enough to not block any of the goroutines, in this case:
errc := make(chan error, len(addrs))
Or even if a non-nil error is received from the channel, still proceed to receive as many times as many goroutines send on it:
var errRec error
for _ = range addrs {
if err := <-errc; err != nil {
if errRec == nil {
errRec = err
}
}
}
return errRec
Or as mentioned in the linked talk on slide #33: use a "quit" channel to prevent the started goroutines to remain blocked after broadcastMsg() has completed/returned.
You have a list of two addresses (localhost, google). To each of these you're sending a message (hi), using one goroutine per address. And the goroutine sends error (which may be nil) to errc channel.
If you send something to a channel, you also need something that reads the values from that channel, otherwise it will block (unless it's a buffered channel, but even buffered channels block once their buffer is full).
So your reading loop looks like this:
for _ = range addrs {
if err := <-errc; err != nil {
return err
}
}
If the first address returns an error which is not nil, the loop returns. The subsequent error values are never read from the channel thus it blocks.