Hi I am using the qml library for go to create UI's. I am trying to learn how to pass information from the UI (qml) to go to then "do something" with.
QML is working if it is just a UI. I can run that fine when I do:
func main() {
if len(os.Args) != 2 {
fmt.Fprintf(os.Stderr, "usage: %s <qml file>\n", os.Args[0])
os.Exit(1)
}
if err := qml.Run(run); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(1)
}
}
func run() error {
engine := qml.NewEngine()
engine.On("quit", func() { os.Exit(0) })
component, err := engine.LoadFile(os.Args[1])
if err != nil {
return err
}
window := component.CreateWindow(nil)
window.Show()
window.Wait()
return nil
}
However when I add some code, to try and "learn" something from the UI I get the run time error:
panic: runtime error: cgo argument has Go pointer to Go pointer
The code I am adding is:
window.On("visibleChanged", func(visible bool) {
if (visible) {
fmt.Println("Width:", window.Int("width"))
}
})
I am running "go version go1.6 darwin/amd64" on OSX El Capitan
Any ideas why? A google suggests this was an error in Go 1.6 Beta, but I am running the latest stable version (installed a couple of days ago).
If it's not a simple fix, can someone explain "why" this is occuring?
The problem is that when C code stores a Go pointer (in this case, a pointer to your callback function), the garbage collector cannot track that pointer in the C code, and may garbage collect the memory that the pointer is pointing to if no Go code is referencing it. This will cause the C code to crash when it attempts to access that memory. All the runtime knows is that the C code retained the pointer (that's why it can panic), but it doesn't know what the C code will do with it later and for how long it will keep it.
To avoid this, the trick used by most libraries was to hold on to a pointer in Go as well (e.g. in a global map), to ensure that the memory is protected from the garbage collector. go-qml uses this trick as well. This trick works, but the compiler and garbage collector have no idea that it does, they cannot verify that you're not making a mistake (e.g. deleting the Go pointer, while the C code still has its pointer).
With Go 1.6, the Go developers decided to be very strict about this, and they no longer allow C code to retain a Go pointer at all. However, if you disable this check, everything will still work in this case, because go-qml implements the trick correctly (it may break in the future however, e.g. if go implements a moving garbage collector).
Here's the issue about it: https://github.com/go-qml/qml/issues/170
Side note: In this specific case, what gets passed to C is a pointer to an interface{}, which itself contains a pointer to the function. That's why you get the error "cgo argument has Go pointer to Go pointer". The reason this isn't allowed is that it's more difficult to protect these pointers from the GC for the duration of the C call, and it's not worth it, so it's forbidden instead (https://github.com/golang/go/issues/12416#issuecomment-136473697).
However, even if this were allowed, the code would still be violating the rule about C code keeping a copy of the Go pointer.
This isn't actually a problem in Go 1.6, since it doesn't implement a moving garbage collector, but the rules were made so that it can be implemented later.
If you're just playing around, I suggest trying with go 1.5.3. Go 1.6 introduced a different set of constraints on pointers to memory when using cgo, a more restrictive set, and it's possible some go packages that were designed for the older version of go are now breaking a go rule or two.
If this is the case, getting the older package to work with go 1.6, where C is allowed to call go closures, could be harder to fix. But I don't have first hand experience with that yet.
Thanks for all the help here. I have written up a beginners tutorial on using QML with Go. It can be viewed here. I will continually update it if I run into any more errors and find fixes. Thanks everyone for your help. QML/GO is an awesome combination.
https://golog.co/blog/article/Using_Go_with_QML_part_1
Related
Going through the standard library, I see a lot functions similar to the following:
// src/database/sql/sql.go
func (dc *driverConn) removeOpenStmt(ds *driverStmt) {
dc.Lock()
defer dc.Unlock()
delete(dc.openStmt, ds)
}
...
func (db *DB) addDep(x finalCloser, dep interface{}) {
//println(fmt.Sprintf("addDep(%T %p, %T %p)", x, x, dep, dep))
db.mu.Lock()
defer db.mu.Unlock()
db.addDepLocked(x, dep)
}
// src/expvar/expvar.go
func (v *Map) addKey(key string) {
v.keysMu.Lock()
defer v.keysMu.Unlock()
v.keys = append(v.keys, key)
sort.Strings(v.keys)
}
// etc...
I.e.: simple functions with no returns and presumably no way to panic that are still deferring the unlock of their mutex. As I understand it, the overhead of a defer has been improved (and perhaps is still in the process of being improved), but that being said: Is there any reason to include a defer in functions like these? Couldn't these types of defers end up slowing down a high traffic function?
Always deferring things like Mutex.Unlock() and WaitGroup.Done() at the top of the function makes debugging and maintenance easier, since you see immediately that those pieces are handled correctly so you know that those important pieces are taken care of, and can quickly move on to other issues.
It's not a big deal in 3 line functions, but consistent-looking code is also just easier to read in general. Then as the code grows, you don't have to worry about adding an expression that may panic, or complicated early return logic, because the pre-existing defers will always work correctly.
Panic is a sudden (so it could be unpredicted or unprepared) violation of normal control flow. Potentially it can emerge from anything - quite often from external things - for example memory failure. defer mechanism gives an easy and quite cheap tool to perform exit operation. Thus not leave a system in broken state. This is important for locks in high load applications because it help not to lose locked resources and freeze the whole system in lock once.
And if for some moment code has no places for panic (hard to guess such system ;) but things evolve. Later this function would be more complex and able to throw panic.
Conclusion: Defer helps you to ensure your function will exit correctly if something “goes wring”. Also quite important it is future-proof - same reply for different failures.
So it’s a food style to put them even in easy functions. As a programmer you can see nothing is lost. And be more sure in a code.
I am currently building a CLI using Go and am trying to disable any backtrace that is produced as a result of a panic. I believe my code has great error handling, but would now like to suppress any panic messages (fairly new to Go).
I currently put in the following in my main.go function (to purposely cause a panic):
var myarr [2]string
myarr[0] = "Foo"
myarr[1] = "Bar"
for i := range myarr {
fmt.Println(myarr[i+1])
}
And I get this as a result:
goroutine 1 [running]:
Bar
panic: runtime error: index out of range [2] with length 2
main.main()
{directory where main.go is located}/main.go:23 +0x207
How can I suppress this error such that anyone with the executable binary file will not be able to see this error?
I've tried utilizing the GOBACKTRACE environment variable when building my binary and setting its value to GOBACKTRACE=none, but this has no effect on other operating systems I've tested on.
I've tried utilizing the GOBACKTRACE environment variable when building my binary and setting its value to GOBACKTRACE=none, but this has no effect on other operating systems I've tested on.
The environment variable is called GOTRACEBACK, not GOBACKTRACE.
Also, you can use debug.SetTraceback("none") to achieve the same effect, although this can still be overridden via the GOTRACEBACK environment variable.
If you use the correct naming, it should work. If it does not work, congratulations: you found a bug in golang and you should likely report it.
As #Burak mentioned you want to use the built-in Go function recover. There's a good Go blog post on all the subtleties of panic and recovery.
If you want to blanket cover your entire application stack, then simply register recover via a defer function at the main level:
func main() {
defer func() {
if r := recover(); r != nil {
fmt.Println("unexpected problem encountered - aborting")
// optionally log `r` to an exception log - so users may email to developer
os.Exit(1)
}
}()
run(os.Args...)
}
https://play.golang.org/p/b8qYnlNZsr5
Going through the standard library, I see a lot functions similar to the following:
// src/database/sql/sql.go
func (dc *driverConn) removeOpenStmt(ds *driverStmt) {
dc.Lock()
defer dc.Unlock()
delete(dc.openStmt, ds)
}
...
func (db *DB) addDep(x finalCloser, dep interface{}) {
//println(fmt.Sprintf("addDep(%T %p, %T %p)", x, x, dep, dep))
db.mu.Lock()
defer db.mu.Unlock()
db.addDepLocked(x, dep)
}
// src/expvar/expvar.go
func (v *Map) addKey(key string) {
v.keysMu.Lock()
defer v.keysMu.Unlock()
v.keys = append(v.keys, key)
sort.Strings(v.keys)
}
// etc...
I.e.: simple functions with no returns and presumably no way to panic that are still deferring the unlock of their mutex. As I understand it, the overhead of a defer has been improved (and perhaps is still in the process of being improved), but that being said: Is there any reason to include a defer in functions like these? Couldn't these types of defers end up slowing down a high traffic function?
Always deferring things like Mutex.Unlock() and WaitGroup.Done() at the top of the function makes debugging and maintenance easier, since you see immediately that those pieces are handled correctly so you know that those important pieces are taken care of, and can quickly move on to other issues.
It's not a big deal in 3 line functions, but consistent-looking code is also just easier to read in general. Then as the code grows, you don't have to worry about adding an expression that may panic, or complicated early return logic, because the pre-existing defers will always work correctly.
Panic is a sudden (so it could be unpredicted or unprepared) violation of normal control flow. Potentially it can emerge from anything - quite often from external things - for example memory failure. defer mechanism gives an easy and quite cheap tool to perform exit operation. Thus not leave a system in broken state. This is important for locks in high load applications because it help not to lose locked resources and freeze the whole system in lock once.
And if for some moment code has no places for panic (hard to guess such system ;) but things evolve. Later this function would be more complex and able to throw panic.
Conclusion: Defer helps you to ensure your function will exit correctly if something “goes wring”. Also quite important it is future-proof - same reply for different failures.
So it’s a food style to put them even in easy functions. As a programmer you can see nothing is lost. And be more sure in a code.
I am running a program with go 1.4 and I am trying to pass a large struct to a go function.
go ProcessImpression(network, &logImpression, campaign, actualSpent, partnerAccount, deviceId, otherParams)
I get this error:
runtime.newproc: function arguments too large for new goroutine
I have moved to pass by reference which helps but I am wondering if there is some way to pass large structs in a go function.
Thanks,
No, none I know of.
I don't think you should be too aggressive tuning to avoid copying, but it appears from the source that this error is emitted when parameters exceed the usable stack space for a new goroutine, which should be kilobytes. The copying overhead is real at that point, especially if this isn't the only time these things are copied. Perhaps some struct either explicitly is larger than expected thanks to a large struct member (1kb array rather than a slice, say) or indirectly. If not, just using a pointer as you have makes sense, and if you're worried about creating garbage, recycle the structs pointed to using sync.Pool.
I was able to fix this issue by changing the arguments from
func doStuff(prev, next User)
to
func doStuff(prev, next *User)
The answer from #twotwotwo in here is very helpful.
Got this issue at processing list of values([]BigType) of big struct:
for _, stct := range listBigStcts {
go func(stct BigType) {
...process stct ...
}(stct) // <-- error occurs here
}
Workaround is to replace []BigType with []*BigType
I have a cli application in Go (still in development) and no changes were made in source code neither on dependencies but all of a sudden it started to panic panic: sync: unlock of unlocked mutex.
The only place I'm running concurrent code is to handle when program is requested to close:
func handleProcTermination() {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
go func() {
<-c
curses.Endwin()
os.Exit(0)
}()
defer curses.Endwin()
}
Only thing I did was to rename my $GOPATH and work space folder. Can this operation cause such error?
Have some you experienced any related problem without having any explanation? Is there a rational check list that would help to find the cause of the problem?
Ok, after some unfruitful debugging sessions, as a last resort, I simply wiped all third party code (dependencies) from the workspace:
cd $GOPATH
rm -rf pkg/ bin/ src/github.com src/golang.org # the idea is to remove all except your own source
Used go get to get all used dependencies again:
go get github.com/yadayada/yada
go get # etc
And the problem is gone! Application is starting normally and tests are passing. No startup panics anymore. It looks like this problem happens when you mv your work space folder but I'm not 100% sure yet. Hope it helps someone else.
From now on, re install dependencies will be my first step when weird panic conditions like that suddenly appear.
You're not giving much information to go on, so my answer is generic.
In theory, bugs in concurrent code can remain unnoticed for a long time and then suddenly show up. In practice, if the bug is easily repeatable (happens nearly every run) this usually indicates that something did change in the code or environment.
The solution: debug.
Knowing what has changed can be helpful to identify the bug. In this case, it appears that lock/unlock pairs or not matching up. If you are not passing locks between threads, you should be able to find a code path within the thread that has not acquired the lock, or has released it early. It may be helpful to put assertions at certain points to validate that you are holding the lock when you think you are.
Make sure you don't copy the lock somewhere.
What can happen with seemingly bulletproof code in concurrent environments is that the struct including the code gets copied elsewhere, which results in the underlying lock being different.
Consider this code snippet:
type someStruct struct {
lock sync.Mutex
}
func (s *someStruct) DoSomethingUnderLock() {
s.lock.Lock()
defer s.lock.Unlock() // This will panic
time.Sleep(200 * time.Millisecond)
}
func main() {
s1 := &someStruct{}
go func() {
time.Sleep(100 * time.Millisecond) // Wait until DoSomethingUnderLock takes the lock
s2 := &someStruct{}
*s1 = *s2
}()
s1.DoSomethingUnderLock()
}
*s1 = *s2 is the key here - it results in a different lock being used by the same receiver function and if the struct is replaced while the lock is taken, we'll get sync: unlock of unlocked mutex.
What makes it harder to debug is that someStruct might be nested in another struct (and another, and another), and if the outer struct gets replaced (as long as someStruct is not a reference there) in a similar manner, the result will be the same.
If this is the case, you can use a reference to the lock (or the whole struct) instead. Now you need to initialize it, but it's a small price that might save you some obscure bugs. See the modified code that doesn't panic here.
For those who come here and didn't solve your problem. Check If the application is compiled in one version of Linux but running in another version. At least in my case it happened.