i have a question on the go memory model.
in the last example:
type T struct {
msg string
}
var g *T
func setup() {
t := new(T)
t.msg = "hello, world"
g = t
}
func main() {
go setup()
for g == nil {
}
print(g.msg)
}
In my opnion,reads and writes of values with a single machine word is a atomic behavior.I try many times to run the test but it is always can be observed.
So please tell me why g.msg is not guarntee to observed? I want to know the reason in detail,please.
Because there are 2 write operations in the launched goroutine:
t := new(T) // One
t.msg = "hello, world" // Two
g = t
It may be that the main goroutine will observe the non-nil pointer assignment to g in the last line, but since there is no explicit synchronization between the 2 goroutines, the compiler is allowed to reorder the operations (that doesn't change the behavior in the launched goroutine), e.g. to the following:
t := new(T) // One
g = t
t.msg = "hello, world" // Two
If operations would be rearranged like this, the behavior of the launched goroutine (setup()) would not change, so a compiler is allowed to to this. And in this case the main goroutine could observe the effect of g = t, but not t.msg = "hello, world".
Why would a compiler reorder the operations? E.g. because a different order may result in a more efficient code. E.g. if the pointer assigned to t is already in a register, it can also be assigned to g right away, without having to reload it again if the assignment to g would not be executed right away.
This is mentioned in the Happens Before section:
Within a single goroutine, reads and writes must behave as if they executed in the order specified by the program. That is, compilers and processors may reorder the reads and writes executed within a single goroutine only when the reordering does not change the behavior within that goroutine as defined by the language specification. Because of this reordering, the execution order observed by one goroutine may differ from the order perceived by another. For example, if one goroutine executes a = 1; b = 2;, another might observe the updated value of b before the updated value of a.
If you use proper synchronization, that will forbid the compiler to perform such rearranging that would change the observed behavior from other goroutines.
Running your example any number of times and not observing this does not mean anything. It may be the problem will never arise, it may be it will arise on a different architecture, or on a different machine, or when compiled with a different (future) version of Go. Simply do not rely on such behavior that is not guaranteed. Always use proper synchronization, never leave any data races in your app.
Related
Both functions are doing the same task which is initializing "Data struct". what are the Pros or Cons of each function? e.g. the function should unmarshal a big JSON file.
package main
type Data struct {
i int
}
func funcp(c chan *Data) {
var t *Data
t = <-c //receive
t.i = 10
}
func funcv(c chan Data) {
var t Data
t.i = 20
c <- t //send
}
func main() {
c := make(chan Data)
cp := make(chan *Data)
var t Data
go funcp(cp)
cp <- &t //send
println(t.i)
go funcv(c)
t = <- c //receive
println(t.i)
}
Link to Go Playground
The title of your question seems wrong. You are asking not about swapping things but rather about whether to send a pointer to some data or a copy of some data. More importantly, the overall thrust of your question lacks crucial information.
Consider two analogies:
Which is better, chocolate ice cream, or strawberry? That's probably a matter of opinion, but at least both with serve similar purposes.
Which is better, a jar of glue or a brick of C4? That depends on whether you want to build something, or blow something up, doesn't it?
If you send a copy of data through a channel, the receiver gets ... a copy. The receiver does not have access to the original. The copying process may take some time, but the fact that the receiver does not have to share access may speed things up. So this is something of an opinion, and if your question is about which is faster, well, you'll have to benchmark it. Be sure to benchmark the real problem, and not a toy example, because benchmarks on toy examples don't translate to real-world performance.
If you send a pointer to data through a channel, the receiver gets a copy of the pointer, and can therefore modify the original data. Copying the pointer is fast, but the fact that the receiver has to share access may slow things down. But if the receiver must be able to modify the data, you have no choice. You must use a tool that works, and not one that does not.
In your two functions, one generates values (funcv) so it does not have to send pointers. That's fine, and gives you the option. The other (funcp) receives objects but wants to update them so it must receive a pointer to the underlying object. That's fine too, but it means that you are now communicating by sharing (the underlying data structure), which requires careful coordination.
var p = &sync.Pool{
New: func() interface{} {
return &serveconn{}
},
}
func newServeConn() *serveconn {
sc := p.Get().(*serveconn)
runtime.SetFinalizer(sc, (*serveconn).finalize)
fmt.Println(sc, "SetFinalizer")
return sc
}
func (sc *serveconn) finalize() {
fmt.Println(sc, "finalize")
*sc = serveconn{}
runtime.SetFinalizer(sc, nil)
p.Put(sc)
}
The above code tries to reuse object by SetFinalizer, but after debug I found finalizer is never called, why?
UPDATE
This may be related:https://github.com/golang/go/issues/2368
The above code tries to reuse object by SetFinalizer, but after debug I found finalizer is never called, why?
The finalizer is only called on an object when the GC
marks it as unused and then tries to sweep (free) at the end
of the GC cycle.
As a corollary, if a GC cycle is never performed during the runtime of your program, the finalizers you set may never be called.
Just in case you might hold a wrong assumption about the Go's GC, it may worth noting that Go does not employ reference counting on values; instead, it uses GC which works in parallel with the program, and the sessions during which it works happen periodically and are triggered by certain parameters like pressure on the heap produced by allocations.
A couple assorted notes regarding finalizers:
When the program terminates, no GC is forcibly run.
A corollary of this is that a finalizer is not guaranteed
to run at all.
If the GC finds a finalizer on an object about to be freed,
it calls the finalizer but does not free the object.
The object itself will be freed only at the next GC cycle —
wasting the memory.
All in all, you appear as trying to implement destructors.
Please don't: make your objects implement the sort-of standard method called Close and state in the contract of your type that the programmer is required to call it when they're done with the object.
When a programmer wants to call such a method no matter what, they use defer.
Note that this approach works perfectly for all types in the Go
stdlib which wrap resources provided by the OS—file and socket descriptors. So there is no need to pretend your types are somehow different.
Another useful thing to keep in mind is that Go was explicitly engineered to be no-nonsense, no-frills, no-magic, in-your-face language, and you're just trying to add magic to it.
Please don't, those who like decyphering layers of magic do program in Scala different languages.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
The following go play example shows in a simplistic way what I have defined. I am passing a map as a copied value to a function (not a reference) as well as there is a recursion in my function which I assume passes by value as well.
https://play.golang.org/p/na6y6Wih4M
// this function has no write operations to dataMap, just reads
// dataMap, in fact, has no write operations since it was copied
func findParentAncestors(ID int, dataMap map[int]Data) []Data {
results := []Data{}
if _, ok := dataMap[ID]; ok {
if parentData, ok := dataMap[dataMap[ID].ParentID]; ok {
results = append(results, parentData)
// recursion
results = append(results, findParentAncestors(parentData.ID, dataMap)...)
}
}
return results
}
PROBLEM: somehow along my program execution, which involves much more data than this example (obviusly), an error "fatal error: concurrent map read and map write" points function findParentAncestors():
main.findParentAncestors(0x39e3, 0xc82013ac90, 0x0, 0x0, 0x0)
/opt/test/src/test.go:17 +0xa6 fp=0xc820269fb8 sp=0xc820269bd0
main.findParentAncestors(0x5d25, 0xc82013ac90, 0x0, 0x0, 0x0)
/opt/test/src/test.go:21 +0x239 fp=0xc82026a3a0 sp=0xc820269fb8
From your example, https://play.golang.org/p/na6y6Wih4M:
// the orignalMap is defined elsewhere in the program (here represented)
originalMap := map[int]Data{}
originalMap[0] = Data{ID: 0, ParentID: -1, Name: "zero"}
originalMap[1] = Data{ID: 1, ParentID: 0, Name: "one"}
originalMap[2] = Data{ID: 2, ParentID: 1, Name: "two"}
// copies the original map from a global location (here represented)
copiedMap := originalMap
// identifies ancestors unsing the copied map
parents := findParentAncestors(2, copiedMap)
This is a misnomer, copiedMap := originalMap, you are not copying the map.
In Go all arguments are passed by value. It's equivalent to assigning each argument to each parameter. For a map, assignment, copiedMap := originalMap, or passing by value, findParentAncestors(2, copiedMap), copies the map descriptor which is a pointer to the map descriptor struct which contains a pointer to the map key-value data. Obviously you have a potential race condition if there are any writes to the map.
You are using go version go1.6.3 linux/amd64, so run the race detector.
Go 1.6 Release Notes
Runtime
The runtime has added lightweight, best-effort detection of concurrent
misuse of maps. As always, if one goroutine is writing to a map, no
other goroutine should be reading or writing the map concurrently. If
the runtime detects this condition, it prints a diagnosis and crashes
the program. The best way to find out more about the problem is to run
the program under the race detector, which will more reliably identify
the race and give more detail.
Command go
Compile packages and dependencies
-race
enable data race detection.
Supported only on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64.
Also, compile and run your program using Go 1.8, the current release of Go, which significantly improves concurrent map misuse.
Go 1.8 Release Notes
Concurrent Map Misuse
In Go 1.6, the runtime added lightweight, best-effort detection of
concurrent misuse of maps. This release improves that detector with
support for detecting programs that concurrently write to and iterate
over a map.
As always, if one goroutine is writing to a map, no other goroutine
should be reading (which includes iterating) or writing the map
concurrently. If the runtime detects this condition, it prints a
diagnosis and crashes the program. The best way to find out more about
the problem is to run the program under the race detector, which will
more reliably identify the race and give more detail.
Go 1.7 beta 1 was released this morning, here is the release notes draft of Go 1.7. A new function KeepAlive was added to the package runtime. The doc of runtime.KeepAlive has given an example:
type File struct { d int }
d, err := syscall.Open("/file/path", syscall.O_RDONLY, 0)
// ... do something if err != nil ...
p := &FILE{d}
runtime.SetFinalizer(p, func(p *File) { syscall.Close(p.d) })
var buf [10]byte
n, err := syscall.Read(p.d, buf[:])
// Ensure p is not finalized until Read returns.
runtime.KeepAlive(p)
// No more uses of p after this point.
The doc of runtime.SetFinalizer has also given an explanation about runtime.KeepAlive:
For example, if p points to a struct that contains a file descriptor
d, and p has a finalizer that closes that file descriptor, and if the
last use of p in a function is a call to syscall.Write(p.d, buf,
size), then p may be unreachable as soon as the program enters
syscall.Write. The finalizer may run at that moment, closing p.d,
causing syscall.Write to fail because it is writing to a closed file
descriptor (or, worse, to an entirely different file descriptor opened
by a different goroutine). To avoid this problem, call
runtime.KeepAlive(p) after the call to syscall.Write.
What confused me is that the variable p has not left its life scope yet, why will it be unreachable? Does that mean that a variable will be unreachable if only there is no use of it in the following code, no matter whether it is in its life scope?
A variable becomes unreachable when the runtime detects that the Go code cannot reach a point where that variable is referenced again.
In the example you posted, a syscall.Open() is used to open a file. The returned file descriptor (which is just an int value) is "wrapped" in a struct. Then a finalizer is attached to this struct value that closes the file descriptor. Now when this struct value becomes unreachable, its finalizer may be run at any moment, and the closing / invalidation / re-using of the file descriptor could cause unexpected behavior or errors in the execution of the Read() syscall.
The last use of this struct value p in Go code is when syscall.Read() is invoked (and the file descriptor p.d is passed to it). The implementation of the syscall will use that file descriptor after the initiation of syscall.Read(), it may do so up until syscall.Read() returns. But this use of the file descriptor is "independent" of the Go code.
So the struct value p is not used during the execution of the syscall, and the syscall blocks the Go code until it returns. Which means the Go runtime is allowed to mark p as unreachable during the execution of Read() (before Read() returns), or even before its actual execution begins (because p is only used to provide the arguments to call Read().
Hence the call to runtime.KeepAlive(): since this call is after the syscall.Read() and it references the variable p, the Go runtime is not allowed to mark p unreachable before Read() returns, because this is after the Read() call.
Note that you could use other constructs to "keep p alive", e.g. _ = p or returning it. runtime.KeepAlive() does nothing magical in the background, its implementation is:
func KeepAlive(interface{}) {}
runtime.KeepAlive() does provide a much better alternative because:
It clearly documents we want to keep p alive (to prevent runs of Finalizers).
Using other constructs such as _ = p might get "optimized" out by future compilers, but not runtime.KeepAlive() calls.
Go's sync package has a Mutex. Unfortunately it's not recursive. What's the best way to implement recursive locks in Go?
I'm sorry to not answer your question directly:
IMHO, the best way how to implement recursive locks in Go is to not implement them, but rather redesign your code to not need them in the first place. It's probable, I think, that the desire for them indicates a wrong approach to some (unknown here) problem is being used.
As an indirect "proof" of the above claim: Would a recursive lock be a common/correct approach to the/some usual situations involving mutexes, it would be sooner or later included in the standard library.
And finally, last but not least: What Russ Cox from the Go development team wrote here https://groups.google.com/d/msg/golang-nuts/XqW1qcuZgKg/Ui3nQkeLV80J:
Recursive (aka reentrant) mutexes are a bad idea.
The fundamental reason to use a mutex is that mutexes
protect invariants, perhaps internal invariants like
"p.Prev.Next == p for all elements of the ring", or perhaps
external invariants like "my local variable x is equal to p.Prev."
Locking a mutex asserts "I need the invariants to hold"
and perhaps "I will temporarily break those invariants."
Releasing the mutex asserts "I no longer depend on those
invariants" and "If I broke them, I have restored them."
Understanding that mutexes protect invariants is essential to
identifying where mutexes are needed and where they are not.
For example, does a shared counter updated with atomic
increment and decrement instructions need a mutex?
It depends on the invariants. If the only invariant is that
the counter has value i - d after i increments and d decrements,
then the atmocity of the instructions ensures the
invariants; no mutex is needed. But if the counter must be
in sync with some other data structure (perhaps it counts
the number of elements on a list), then the atomicity of
the individual operations is not enough. Something else,
often a mutex, must protect the higher-level invariant.
This is the reason that operations on maps in Go are not
guaranteed to be atomic: it would add expense without
benefit in typical cases.
Let's take a look at recursive mutexes.
Suppose we have code like this:
func F() {
mu.Lock()
... do some stuff ...
G()
... do some more stuff ...
mu.Unlock()
}
func G() {
mu.Lock()
... do some stuff ...
mu.Unlock()
}
Normally, when a call to mu.Lock returns, the calling code
can now assume that the protected invariants hold, until
it calls mu.Unlock.
A recursive mutex implementation would make G's mu.Lock
and mu.Unlock calls be no-ops when called from within F
or any other context where the current thread already holds mu.
If mu used such an implementation, then when mu.Lock
returns inside G, the invariants may or may not hold. It depends
on what F has done before calling G. Maybe F didn't even realize
that G needed those invariants and has broken them (entirely
possible, especially in complex code).
Recursive mutexes do not protect invariants.
Mutexes have only one job, and recursive mutexes don't do it.
There are simpler problems with them, like if you wrote
func F() {
mu.Lock()
... do some stuff
}
you'd never find the bug in single-threaded testing.
But that's just a special case of the bigger problem,
which is that they provide no guarantees at all about
the invariants that the mutex is meant to protect.
If you need to implement functionality that can be called
with or without holding a mutex, the clearest thing to do
is to write two versions. For example, instead of the above G,
you could write:
// To be called with mu already held.
// Caller must be careful to ensure that ...
func g() {
... do some stuff ...
}
func G() {
mu.Lock()
g()
mu.Unlock()
}
or if they're both unexported, g and gLocked.
I am sure that we'll need TryLock eventually; feel free to
send us a CL for that. Lock with timeout seems less essential
but if there were a clean implementation (I don't know of one)
then maybe it would be okay. Please don't send a CL that
implements recursive mutexes.
Recursive mutexes are just a mistake, nothing more than
a comfortable home for bugs.
Russ
You could quite easily make a recursive lock out of a sync.Mutex and a sync.Cond. See Appendix A here for some ideas.
Except for the fact that the Go runtime doesn't expose any notion of goroutine Id. This is to stop people doing silly things with goroutine local storage, and probably indicates that the designers think that if you need a goroutine Id you are doing it wrong.
You can of course dig the goroutine Id out of the runtime with a bit of C if you really want to. You might want to read that thread to see why the designers of Go think it is a bad idea.
as was already established, this is a miserable, horrible, awful, and terrible idea from a concurrency perspective.
Anyway, since your question is really about Go's type system, here's how you would define a type with a recursive method.
type Foo struct{}
func (f Foo) Bar() { fmt.Println("bar") }
type FooChain struct {
Foo
child *FooChain
}
func (f FooChain) Bar() {
if f.child != nil {
f.child.Bar()
}
f.Foo.Bar()
}
func main() {
fmt.Println("no children")
f := new(FooChain)
f.Bar()
for i := 0; i < 10; i++ {
f = &FooChain{Foo{}, f}
}
fmt.Println("with children")
f.Bar()
}
http://play.golang.org/p/mPBHKpgxnd