After reading the mutex examples on golang.org and stackoverflow, I'm still not sure about the declaration and idiomatic usage with anonymous functions. Therefore I've summarized a few examples.
Are examples A, B and C nearly equivalent or are there major differences that I don't notice?
I would prefer the global example "B". I guess if I'm careful with it, it's probably the simplest solution.
Or is there maybe a better approach to use mutex?
This example on go playground
package main
import (
"fmt"
"sync"
)
type MuContainer struct {
sync.RWMutex
data int
}
var mucglobal = &MuContainer{}
func main() {
// A: Global declaration - working: adds 45
for i := 0; i < 10; i++ {
go func(j int, mucf *MuContainer) {
mucf.Lock()
mucf.data += j
mucf.Unlock()
}(i, mucglobal)
}
// B: Global only - working: adds 45
for i := 0; i < 10; i++ {
go func(j int) {
mucglobal.Lock()
mucglobal.data += j
mucglobal.Unlock()
}(i)
}
// C: Local declaration - working: adds 45
muclocal := &MuContainer{}
for i := 0; i < 10; i++ {
go func(j int, mucf *MuContainer) {
mucf.Lock()
mucf.data += j
mucf.Unlock()
}(i, muclocal)
}
// // D: Pointer to struct - not working: adds 0
// // I guess because it points directly to the struct.
// for i := 0; i < 10; i++ {
// go func(j int, mucf *MuContainer) {
// mucf.Lock()
// mucf.data += j
// mucf.Unlock()
// }(i, &MuContainer{})
// }
for {
mucglobal.RLock()
muclocal.RLock()
fmt.Printf("global: %d / local: %d\n", mucglobal.data, muclocal.data)
if mucglobal.data == 90 && muclocal.data == 45 {
muclocal.RUnlock()
mucglobal.RUnlock()
break
}
muclocal.RUnlock()
mucglobal.RUnlock()
}
}
D is not working because you are creating a new struct for each iteration. In the end, you'll have 10 independent instances of MuContainer.
The first two options are semantically identical. The bottom line for those two is that each goroutine shares the same instance of the object, which happens to be a global var.
The second one is similar with the only difference being the object locked and updated happens to be a local var. Again, the goroutines are working on the same instance of the object.
So these are not really different from each other, and all three have their uses.
Related
For this task i need to find min sum in list of numbers. Then i must print number that have min sum. This must be done with Mutex and WaitGroups. I can't find where is the mistake or why is output different.
Logic: Scanf n and make vector with len(n). Then create funcion for sum of number and forward that function to second where we in one FOR cycle give goroutines function to.
I run this code a few times, and sometimes give different answer for same input.
Input:
3
13
12
11
Output:
Sometimes 12
Sometimes 11
package main
import (
"fmt"
"math"
"runtime"
"sync"
)
var wg sync.WaitGroup
var mutex sync.Mutex
var vector []int
var i int
var n int
var firstsum int
var p int //Temp sum
var index_result int
func sumanajmanjih(broj int) int {
var br int
var suma int
br = int(math.Abs(float64(broj)))
suma = 0
for {
suma += br % 10
br = br / 10
if br <= 0 {
break
}
}
return suma
}
func glavna(rg int) {
var index int
firstsum = sumanajmanjih(vector[0])
for {
mutex.Lock()
if i == n {
mutex.Unlock()
break
} else {
index = i
i += 1
mutex.Unlock()
}
fmt.Printf("Procesor %d radni indeks %d\n", rg, index)
p = sumanajmanjih(vector[index])
if p < firstsum {
firstsum = p
index_result = index
}
}
wg.Done()
}
func main() {
fmt.Scanf("%d", &n)
vector = make([]int, n)
for i := 0; i < n; i++ {
fmt.Scanf("%d", &vector[i])
}
fmt.Println(vector)
brojGR := runtime.NumCPU()
wg.Add(brojGR)
for rg := 0; rg < brojGR; rg++ {
go glavna(rg)
}
wg.Wait()
fmt.Println(vector[index_result])
}
Not a full answer to your question, but a few suggestions to make code more readable and stable:
Use English language for names - glavna, brojGR are hard to understand
Add comments to code explaining intent
Try to avoid shared/global variables, especially for concurrent code. glavna(rg) is executed concurrently, and you assign global i and p inside that function, that is a race condition. Sends all the data in and out into function explicitly as argument or function result.
Mutex easily can lock the code, and it is complicated to debug. Simplify its usage. Often defer mutex.Unlock() in the next line after Lock() is good enough.
This question already has answers here:
How safe are Golang maps for concurrent Read/Write operations?
(8 answers)
Map with concurrent access
(6 answers)
Is it safe to write to a Golang map concurrently if the keys are always unique?
(2 answers)
Closed 10 months ago.
this is a fatal error: concurrent map read and map write demo code:
package main
var Map = make(map[int]int)
func main() {
for i := 0; i < 100000; i++ {
go updateMap()
go readMap(i)
}
}
func readMap(key int) int {
return Map[key]
}
func updateMap() {
for j := 0; j < 1000; j++ {
Map[j] = j
}
}
I don't use the lock or sync.Map, just use tmp var to replace global map, it will not panic, codes like this:
package main
var Map = make(map[int]int)
func main() {
for i := 0; i < 100000; i++ {
go updateMap()
go readMap(i)
}
}
func readMap(key int) int {
return Map[key]
}
func updateMap() {
tmp := map[int]int{}
for j := 0; j < 1000; j++ {
tmp[j] = j
}
Map = tmp
}
Is there any problem with this way to update the global map?
The second way of updating the map will not panic, because the map being updated is a temporary map, but there is still a race condition. After the updateMap function, there is no guarantee when the other goroutine will see the effects of updateMap, if ever.
The correct way is to use a RWMutex for both cases. The readMap should use RLock/RUnlock, and updateMap should use Lock/Unlock. For the second case, if the updateMap the only goroutine that ever updates the map, you can use Rlock/RUnlock during the copy phase, and Lock/Unlock when you assign the map.
I'm creating global counter, which can be shared between goroutines.
Referring to this question, following code may satisfy my needs.
However if there ware lots of concurrent requests, could it happen that the same number is assigned to more than two goroutines?
If so how can I avoid this?
This question is different from the link I pasted, as what I want to know about is how I can avoid duplication using channel counter. if the only possible solution is other implementation like sync.Mutex or atomic, I'll use it. however, according to the link (again), channel seems to be the best option. Any comment or answer really helpful. thanks in advance.
I'm new to multithread coding and also go, might be silly question. sorry for that.
package main
import (
"fmt"
"time"
)
var counter int
var counter_chan chan int
func main() {
counter_chan = make(chan int, 100)
counter = 0
go func() {
for {
select {
case chanc := <-counter_chan:
counter += chanc
fmt.Printf("%d \n", counter)
}
}
}()
for i := 0; i < 10; i++ {
go AddCounter(counter_chan)
}
time.Sleep(time.Second)
fmt.Printf("Total Count is ... %d \n", GetCount())
}
func AddCounter(ch chan int) {
ch <- 1
}
func GetCount() int {
return counter
}
func ResetCount() {
if counter > 8190 {
counter = 0
}
}
-- Edit 05/14 2018
Assume following code is thread-safe for getting and resetting value. Am I right?
package main
import (
"fmt"
"time"
)
var counter int
var addCounterChan chan int
var readCounterChan chan int
func main() {
addCounterChan = make(chan int, 100)
readCounterChan = make(chan int, 100)
counter = 0
go func() {
for {
select {
case val := <-addCounterChan:
counter += val
if counter > 5 {
counter = 0
}
readCounterChan <- counter
fmt.Printf("%d \n", counter)
}
}
}()
for i := 0; i < 10; i++ {
go AddCounter(addCounterChan)
}
time.Sleep(time.Second)
for i := 0; i < 10; i++ {
fmt.Printf("Total Count #%d is ... %d \n", (i + 1), GetCount(readCounterChan))
}
}
// Following two functions will be implemented in another package in real case.
func AddCounter(ch chan int) {
ch <- 1
}
func GetCount(ch chan int) int {
r := <-ch
return r
}
The direct answer to your question is: The code you've pasted updates the counter safely, but doesn't read or reset it safely.
Contrary to the accepted answer in the question you linked to, however, the easiest, most efficient way to implement a shared counter is with the atomic package. It can be used to atomically increment several common types. Example:
var globalCounter *int32 = new(int32)
// .. later in your code
currentCount := atomic.AddInt32(globalCounter, 1)
Use a sync.Mutex to create a counter with add, get and reset operations as shown in the question.
type counter struct {
mu sync.Mutex
n int
}
func (c *counter) Add() {
c.mu.Lock()
c.n++
c.mu.Unlock()
}
func (c *counter) Get() int {
c.mu.Lock()
n := c.n
c.mu.Unlock()
return n
}
func (c *counter) Reset() {
c.mu.Lock()
if c.n > 8190 {
c.n = 0
}
c.mu.Unlock()
}
If the reset function is not needed, then use the sync/atomic.
type counter struct {
n int32
}
func (c *counter) Add() {
atomic.AddInt32(&c.n, 1)
}
func (c *counter) Get() int {
return int(atomic.LoadInt32(&c.n))
}
Go 1.19
The sync/atomic package now includes atomic types, such as atomic.Int32, which you can use to manage a value that can only be accessed atomically.
This basically accomplishes the same thing as having a custom struct with a mutex, or using top-level atomic functions to read and write a "naked" numerical type. Instead of rolling your own, you can simply rely on the standard library.
A simple example:
package main
import (
"fmt"
"sync"
"sync/atomic"
)
// zero value is 0
var counter = atomic.Int32{}
func main() {
wg := &sync.WaitGroup{}
wg.Add(100)
for i := 0; i < 100; i++ {
go func() {
counter.Add(1)
wg.Done()
}()
}
wg.Wait()
fmt.Println(counter.Load())
}
Playground: https://go.dev/play/p/76xM3xXTAM5?v=gotip
I am trying to parallelize an operation in golang and save the results in a manner that I can iterate over to sum up afterwords.
I have managed to set up the parameters so that no deadlock occurs, and I have confirmed that the operations are working and being saved correctly within the function. When I iterate over the Slice of my struct and try and sum up the results of the operation, they all remain 0. I have tried passing by reference, with pointers, and with channels (causes deadlock).
I have only found this example for help: https://golang.org/doc/effective_go.html#parallel. But this seems outdated now, as Vector as been deprecated? I also have not found any references to the way this function (in the example) was constructed (with the func (u Vector) before the name). I tried replacing this with a Slice but got compile time errors.
Any help would be very appreciated. Here is the key parts of my code:
type job struct {
a int
b int
result *big.Int
}
func choose(jobs []Job, c chan int) {
temp := new(big.Int)
for _,job := range jobs {
job.result = //perform operation on job.a and job.b
//fmt.Println(job.result)
}
c <- 1
}
func main() {
num := 100 //can be very large (why we need big.Int)
n := num
k := 0
const numCPU = 6 //runtime.NumCPU
count := new(big.Int)
// create a 2d slice of jobs, one for each core
jobs := make([][]Job, numCPU)
for (float64(k) <= math.Ceil(float64(num / 2))) {
// add one job to each core, alternating so that
// job set is similar in difficulty
for i := 0; i < numCPU; i++ {
if !(float64(k) <= math.Ceil(float64(num / 2))) {
break
}
jobs[i] = append(jobs[i], Job{n, k, new(big.Int)})
n -= 1
k += 1
}
}
c := make(chan int, numCPU)
for i := 0; i < numCPU; i++ {
go choose(jobs[i], c)
}
// drain the channel
for i := 0; i < numCPU; i++ {
<-c
}
// computations are done
for i := range jobs {
for _,job := range jobs[i] {
//fmt.Println(job.result)
count.Add(count, job.result)
}
}
fmt.Println(count)
}
Here is the code running on the go playground https://play.golang.org/p/X5IYaG36U-
As long as the []Job slice is only modified by one goroutine at a time, there's no reason you can't modify the job in place.
for i, job := range jobs {
jobs[i].result = temp.Binomial(int64(job.a), int64(job.b))
}
https://play.golang.org/p/CcEGsa1fLh
You should also use a WaitGroup, rather than rely on counting tokens in a channel yourself.
I am new to Go language and got confused with the following code
package main
import "fmt"
// fibonacci is a function that returns
// a function that returns an int.
func fibonacci() func() int {
previous := 0
current := 1
return func () int{
current = current+previous
previous = current-previous
return current
}
}
func main() {
f := fibonacci
for i := 0; i < 10; i++ {
fmt.Println(f()())
}
}
This code is supposed to print out the Fibonacci Sequence (first 10), but only print out 10 times 1.
but if I change the code to:
func main() {
f := fibonacci()
for i := 0; i < 10; i++ {
fmt.Println(f())
}
}
Then it is working fine. The output is the Fibonacci sequence.
Could any one help me explain this?
Thanks
fibonacci() creates a new fibonacci generator function. fibonacci()() does the same, and then calls it once, returns the result and discards the generator, never to be used again. If you call that in a loop, it'll just keep creating new generators and only using their first value.
If you want more than just the first value, you need to do exactly what you did in your second example. Store the generator itself in a variable and then call the same generator multiple times.
This has to do with how variables are encapsulated in closures after returning the closure.
Consider the following example (live code on play):
func newClosure() func() {
i := 0
fmt.Println("newClosure with &i=", &i)
return func() {
fmt.Println(i, &i)
i++
}
}
func main() {
a := newClosure()
a()
a()
a()
b := newClosure()
b()
a()
}
Running this code will yield something like the following output. I annotated
which line comes from which statement:
newClosure with &i= 0xc010000000 // a := newClosure()
0 0xc010000000 // a()
1 0xc010000000 // a()
2 0xc010000000 // a()
newClosure with &i= 0xc010000008 // b := newClosure()
0 0xc010000008 // b()
3 0xc010000000 // a()
In the example, the closure returned by newClosure encapsulates the local variable i.
This corresponds to current and the like in your code. You can see that a and b
have different instances of i, or else the call b() would've printed 3 instead.
You can also see that the i variables have different addresses. (The variable is already
on the heap as go does not have a separate stack memory, so using it in the closure is
no problem at all.)
So, by generating a new closure you're automatically creating a new context for the
closure and the local variables are not shared between the closures. This is the reason
why creating a new closure in the loop does not get you further.
The equivalent in for your code in terms of this example would be:
for i:=0; i < 10; i++ {
newClosure()()
}
And you've already seen by the output that this will not work.
func fibonacci() func() int return a function literal (closure) that returns an int representing the last generated number in the list.
The first main() method
f := fibonacci
for i := 0; i < 10; i++ {
fmt.Println(f()())
}
f is the generator function, every iteration in the loop invoke f()() that generates a new closure with a new envirement ( previous := 0, current := 1), so , the second invocation returns current which is always equal to 1.
The second main() method
func main() {
f := fibonacci()
for i := 0; i < 10; i++ {
fmt.Println(f())
}
}
f is the closure (not the generator) with an initial envirement ( previous := 0, current := 1), every iteration in the loop invoke f() witch returns current and modify the envirement, so whith the next call, previous will be 1 and current 2.