So I'm receiving a fault when running go test --race my/package/ya. The output looks like this:
unexpected fault address 0x200015a490e0
fatal error: fault
[signal 0xb code=0x1 addr=0x200015a490e0 pc=0x47399ac]
goroutine 1 [running, locked to thread]:
runtime.throw(0x0, 0xc820000180)
/opt/go1.6/src/runtime/panic.go:530 +0x90 fp=0x7fff5fbfeb80 sp=0x7fff5fbfeb68
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/opt/go1.6/src/runtime/asm_amd64.s:1998 +0x1
I've tried catching this fault in a TestMain, but apparently my TestMain doesn't get called. I've tried using -trace to generate a trace file, but the file never is written although the testing file pkg.out is written. I'm not entirely sure where to go from here since the output doesn't really provide many hints. I suppose I can attempt gdb and delve, but haven't had luck with either of those in the past.
I'm wondering if there is something else to consider?
Additional notes: this fault only occurs when the --race flag is present. Go 1.6 darwin/amd64.
(Warming up this really old question)
Simply upgrade to Go 1.20 (see issue on GitHub).
For reference, the errors are (taken from issue):
In version 1.19:
syscall.ptrace: nosplit stack over 792 byte limit
syscall.ptrace<1>
grows 128 bytes, calls syscall.ptrace1<1>
[...]
In version 1.18.5
runtime.fatalthrow: nosplit stack overflow
792 assumed on entry to syscall.ptrace<1> (nosplit)
[...]
Related
we have an application that is mostly Go (1.17) that makes a lot of calls through CGo (GCC 7.5) to CUDA on an ARM processor. We occasionally see panics that look like something has done bad things to the heap in the C side. I tried running the whole application under valgrind, but I get too many messages like
==14869== Thread 1:
==14869== Invalid read of size 8
==14869== at 0x4783AC: runtime.startm (proc.go:2508)
==14869== by 0x47890B: runtime.wakep (proc.go:2584)
==14869== by 0x47CF8F: runtime.newproc.func1 (proc.go:4261)
==14869== by 0x4A476B: runtime.systemstack (asm_arm64.s:230)
==14869== by 0x4A465F: runtime.mstart (asm_arm64.s:117)
==14869== Address 0x1fff0001a8 is on thread 1's stack
==14869== 8 bytes below stack pointer
to see anything useful. I am assuming these are false positives, and the Go runtime is not in fact riddled with undefined behaviour. I can't see a flag to suppress that check. Have I missed it? Is there some other way to investigate this problem? I could write test harnesses in C++ but that will change the use pattern which I suspect is key to the problem.
I build the cgo software to an executable with
go build
which produces an executable by the name of the directory you are in. Let's say it's call "mycgoprog"
I then run valgrind against it with this command:
G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind -v --tool=memcheck --leak-check=full --num-callers=40 --log-file=valgrind.log ./mycgoprog
The valgrind.log then contains the all the indiscretions detected by valgrind in the linked c code.
cmd.StdoutPipe example at go documentation: https://pkg.go.dev/os/exec#example-Cmd.StdoutPipe does not run in playground.
https://play.golang.org/p/ek7-_Xa_bN3
Error:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x7faec2ac4e88, 0x72)
/usr/local/go-faketime/src/runtime/netpoll.go:234 +0x89
internal/poll.(*pollDesc).wait(0xc000074180, 0xc000100000, 0x1)
/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go-faketime/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000074180, {0xc000100000, 0x200, 0x200})
/usr/local/go-faketime/src/internal/poll/fd_unix.go:167 +0x25a
os.(*File).read(...)
/usr/local/go-faketime/src/os/file_posix.go:32
os.(*File).Read(0xc00000e028, {0xc000100000, 0x28, 0xc000060e80})
/usr/local/go-faketime/src/os/file.go:119 +0x5e
encoding/json.(*Decoder).refill(0xc00007e000)
/usr/local/go-faketime/src/encoding/json/stream.go:165 +0x17f
encoding/json.(*Decoder).readValue(0xc00007e000)
/usr/local/go-faketime/src/encoding/json/stream.go:140 +0xbb
encoding/json.(*Decoder).Decode(0xc00007e000, {0x4cf580, 0xc00000c048})
/usr/local/go-faketime/src/encoding/json/stream.go:63 +0x78
main.main()
/tmp/sandbox2294589397/prog.go:24 +0x185
Program exited.
Locally it's running correctly, no deadlock. I am not able to understand why deadlock happens in go playground.
From the documentation on the os.exec package (which is where this example comes from):
Note that the examples in this package assume a Unix system. They may
not run on Windows, and they do not run in the Go Playground used by
golang.org and godoc.org.
That note doesn't provide a reason, but the reason is presumably that allowing user-provided unix commands to run would give a broader attack surface for malicious code. It's not that it's impossible to allow this in a relatively secure way, but there's various tradeoffs which make disallowing os.exec a natural choice.
I am trying to build CockroachDB from sources on a RPi 3, following a couple of posts (this and this).
My RaspberryPi setup:
OS: 2018-06-27-raspbian-stretch
CPU: ARMv7 rev 4(v71)
Mem: 1G + SWAP: 1.5G
Go version: go1.11 linux/arm
I fixed a couple of errors regarding 3rd party libraries while 'make'ing it but now I got stuck on this:
pi#raspberrypi:~/work/go/src/github.com/cockroachdb/cockroach $ make build TAGS='stdmalloc' -j2
GOPATH set to /home/pi/work/go
go build -o cockroach -v -tags 'stdmalloc make arm_linux_gnueabihf_stdmalloc' -ldflags '-X github.com/cockroachdb/cockroach/pkg/build.typ=development -extldflags "" -X "github.com/cockroachdb/cockroach/pkg/build.tag=v2.2.0-alpha.00000000-668-gf76d921f42-dirty" -X "github.com/cockroachdb/cockroach/pkg/build.rev=f76d921f4262356fde83312d0f471262d995f1e0" -X "github.com/cockroachdb/cockroach/pkg/build.cgoTargetTriple=arm-linux-gnueabihf" -X "github.com/cockroachdb/cockroach/pkg/build.utcTime=2018/09/11 12:47:32"' ./pkg/cmd/cockroach
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2372b8]
goroutine 1 [running]:
runtime/internal/atomic.goLoad64(0x3d65bdc, 0x0, 0x5607a20)
/usr/local/go/src/runtime/internal/atomic/atomic_arm.go:124 +0x1c
github.com/cockroachdb/cockroach/pkg/util/humanizeutil.(*BytesValue).String(0x55fb578, 0x5607a01, 0x56d9ea0)
/home/pi/work/go/src/github.com/cockroachdb/cockroach/pkg/util/humanizeutil/humanize.go:109 +0x28
github.com/cockroachdb/cockroach/pkg/cli.(*bytesOrPercentageValue).String(0x57b4920, 0x56d9ea0, 0x7)
/home/pi/work/go/src/github.com/cockroachdb/cockroach/pkg/cli/flags_util.go:492 +0x20
github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflag.(*FlagSet).VarPF(0x54d3a00, 0x25cf778, 0x57b4920, 0x1ef3864, 0xe, 0x0, 0x0, 0x56d2900, 0x169, 0x56d9ea0)
/home/pi/work/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflag/flag.go:778 +0x24
github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflag.(*FlagSet).VarP(0x54d3a00, 0x25cf778, 0x57b4920, 0x1ef3864, 0xe, 0x0, 0x0, 0x56d2900, 0x169)
/home/pi/work/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/pflag/flag.go:786 +0x5c
github.com/cockroachdb/cockroach/pkg/cli.VarFlag(0x54d3a00, 0x25cf778, 0x57b4920, 0x1ef3864, 0xe, 0x0, 0x0, 0x0, 0x0, 0x1fa4462, ...)
/home/pi/work/go/src/github.com/cockroachdb/cockroach/pkg/cli/flags.go:129 +0x6c
github.com/cockroachdb/cockroach/pkg/cli.init.5()
/home/pi/work/go/src/github.com/cockroachdb/cockroach/pkg/cli/flags.go:324 +0x6c0
Makefile:1288: recipe for target 'docs/generated/settings/settings.html' failed
make: *** [docs/generated/settings/settings.html] Error 2
make: *** Deleting file 'docs/generated/settings/settings.html'
Any ideas what might be wrong?
thanks for the question. I was the original author of that blog post. It looks like you're running into an issue related to https://github.com/golang/go/issues/9959. It looks like golang requires 64-bit alignment of 64-bit words when atomically accessed. This is somewhat tricky to enforce across an entire project. Still, we can try to get this working by removing the atomic access on pkg/util/humanizeutil/humanize.go:109 (which doesn't look necessary) and replacing it with return IBytes(*b.val). That should at least get you past this roadblock. There may be other issues you run into after this though. Unfortunately, CockroachDB has never set up CI around a 32-bit ARMv7 build because 32-bit architectures have never been officially supported by the project.
However, a 64-bit ARMv8 build is more actively maintained. In fact, an entire cross-compilation toolchain has been introduced to build CockroachDB for ARMv8 on an x86_64 host. Compilation using this toolchain can be kicked off by running build/builder.sh mkrelease arm64-linux-gnueabi. The resulting binary can be dropped onto a RaspberryPi running a 64-bit OS like https://wiki.debian.org/RaspberryPi3. I actually tried this out last week and ran into no problems.
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 3 years ago.
Improve this question
I've already killed a week trying to solve a mysterious problem in a project of mine and I am out of ideas.
I wrote a Go package intended to play sounds which wraps OpenAL... pretty basic stuff. I got it working on my Xubuntu 14.04 (32-bit), so I booted into Windows (7, also 32-bit) in order to port it... and that's where the problems started.
Whenever I tried to use my audio package, the program crashed with c0000005. I tried to run it through gdb and was surprised to find out that it worked without a problem, it even played my test sound.
Time passed and not knowing what to do, I downloaded the OpenAL Soft source code and started adding printfs - and discovered the exact line it crashed on:
http://repo.or.cz/w/openal-soft.git/blob/HEAD:/Alc/backends/dsound.c#l361
For those lazy to click the link (or if the link stopped working), it's a call to DirectSoundCreate. Running through the debugger again, I saw my prints right before and after the call, and 4 new threads being created between them.
These are the relevant things in the Go file:
package audio
/*
#cgo CFLAGS: -I"../libraries/include"
#cgo windows,386 LDFLAGS: ../libraries/lib/windows/x86/OpenAL32.dll
#cgo windows,amd64 LDFLAGS: ../libraries/lib/windows/x64/OpenAL32.dll
#cgo linux LDFLAGS: -lopenal
#include "audio.h"
*/
import "C"
import (
"errors"
)
var context *C.ALCcontext
func Init() error {
context = C.initAudio() // it crashes on this line
if context == nil {
return errors.New("could not initialize audio")
}
SetActiveListener(NewListener())
return nil
}
And here's the C file actually making the OpenAL calls:
#include "audio.h"
#include <string.h>
#include <stdio.h>
ALCcontext* initAudio() {
ALCdevice* device = alcOpenDevice(NULL); // crashes here
if (device == NULL) {
return NULL;
}
ALCcontext* context = alcCreateContext(device, NULL);
if (!alcMakeContextCurrent(context)) {
return NULL;
}
return context;
}
Last but not least, if I do the exact same thing in pure C (literally just adding main into the file I posted and calling initAudio), it works.
My question is obvious: What the ##!$ is going on?
edit:
Some other things which might be worth mentioning:
I created a new project completely separate from the previous one, in which I have the two files I posted earlier (with different paths for the linker in the Go file, obviously) and the main Go file which does only one thing: call audio.Init. Regardless of which dll I try (I tried the "official" one from http://openal.org, the precompiled version of OpenAL Soft and two versions I compiled myself, one with MinGW and one with Visual Studio), it behaves the same. I also tried to call runtime.LockOSThread at the start of audio.Init, but it didn't help.
Also, I sent the compiled program to my two friends, one running Windows 8 and one with Windows 7 (both 64-bit). The one with Windows 8 said it worked, the one with 7 said it crashed. However, if we tried to compile it from source on the latter machine, with the 64-bit toolchain, it worked.
edit #2:
I just tried to compile OpenAL Soft from source again - first with Visual Studio and then with MinGW - and then link my independent project I mentioned earlier with the library files (OpenAL32.lib from VS and libOpenAL32.dll.a from MinGW) instead of the DLL directly (since that is... the more correct way I guess?)
The result with VS:
It failed during linking, with the following message:
# command-line-arguments
C:\Users\Milan\AppData\Local\Temp\go-build078751523/audiot/audio.a(_all.o): malf
ormed pe file: unexpected flags 0xe0500020 for PE section .text
audiot/audio._Cfunc_initAudio: undefined: _cgo_e0a3be4f138e_Cfunc_initAudio
The result with MinGW:
It succeeded to compile and run, and while it didn't prevent the crash, at least it printed something out:
fatal error: unexpected signal during runtime execution
[signal 0xc0000005 code=0x1 addr=0x420f85 pc=0x42874c]
runtime stack:
invalid spdelta 96655 -1
runtime: unexpected return pc for _cgo_ec587e40eeca_Cfunc_initAudio called from
0x42874800
runtime.throw(0x455dc0)
c:/go/src/pkg/runtime/panic.c:520 +0x71
runtime.sigpanic()
c:/go/src/pkg/runtime/os_windows.c:352 +0x46
invalid spdelta 96655 -1
runtime: unexpected return pc for _cgo_ec587e40eeca_Cfunc_initAudio called from
0x42874800
_cgo_ec587e40eeca_Cfunc_initAudio()
?:0 +0xc
goroutine 16 [syscall]:
runtime.cgocall(0x428740, 0x533f64)
c:/go/src/pkg/runtime/cgocall.c:143 +0xed fp=0x533f58 sp=0x533f2c
audiot/audio._Cfunc_initAudio(0x42e340)
audiot/audio/_obj/_cgo_defun.c:53 +0x37 fp=0x533f64 sp=0x533f58
audiot/audio.Init(0x0, 0x0)
C:/Users/Milan/Desktop/Dropbox/Projekty/Go/src/audiot/audio/at.go:23 +0x
3c fp=0x533f90 sp=0x533f64
main.main()
c:/Users/Milan/Desktop/Dropbox/Projekty/Go/src/audiot/main.go:10 +0x29 f
p=0x533f9c sp=0x533f90
runtime.main()
c:/go/src/pkg/runtime/proc.c:247 +0x11e fp=0x533fd0 sp=0x533f9c
runtime.goexit()
c:/go/src/pkg/runtime/proc.c:1445 fp=0x533fd4 sp=0x533fd0
created by _rt0_go
c:/go/src/pkg/runtime/asm_386.s:101 +0x102
goroutine 17 [syscall]:
runtime.goexit()
c:/go/src/pkg/runtime/proc.c:1445
exit status 2
I also thought it can't hurt to post the specifics of my toolchain:
$ gcc --version
gcc.exe (GCC) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.
$ go version
go version go1.3 windows/386
And let's not forget Visual Studio 2013 Express
I just learnt Go 1.3.1 came out, so I tried updating... and the problem disappeared. (argh!)
I guess it was a compiler bug then. Anyway, thanks to everyone who tried to help, I really appreciate it.
It might be related to Thread Switching:
The solution could be to use LockOSThread
http://golang.org/pkg/runtime/#LockOSThread
You can read more about it here:
https://code.google.com/p/go-wiki/wiki/LockOSThread
Also your Init function has capital I if you are trying to initialize the library it has to be lower case.
My go code errors out with a strack trace like this:
unexpected fault address 0x0
fatal error: fault
[signal 0xb code=0x80 addr=0x0 pc=0x407d50]
goroutine 52246872 [running]:
runtime.throw(0xad6a77)
/usr/local/go/src/pkg/runtime/panic.c:464 +0x69 fp=0xc214d2c1f8
runtime.sigpanic()
/usr/local/go/src/pkg/runtime/os_linux.c:237 +0xe9 fp=0xc214d2c210
hash_lookup(0x671ec0, 0xc21001eed0, 0xc214d2c2d0)
/usr/local/go/src/pkg/runtime/hashmap.c:502 +0x150 fp=0xc214d2c290
runtime.mapaccess(0x671ec0, 0xc21001eed0, 0xc214d2c318, 0xc214d2c328, 0xc214d2c330)
/usr/local/go/src/pkg/runtime/hashmap.c:1004 +0x57 fp=0xc214d2c2c0
runtime.mapaccess2(0x671ec0, 0xc21001eed0, 0x389c, 0x2060009ae, 0xad4dc0, ...)
/usr/local/go/src/pkg/runtime/hashmap.c:1061 +0x5d fp=0xc214d2c308
growmobile.com/rtb/cache.HistogramCheck(0xc2121eafd0, 0xc, 0xeca93c6a0, 0x852a9f8, 0xad4dc0, ...)
/mnt/rtb_server/go/src/growmobile.com/rtb/cache/histogram_cache.go:211 +0x16b fp=0xc214d2c798
growmobile.com/rtb/controller.processBidRequest(0xc2121eafd0, 0xc, 0xeca93c6a0, 0x852a9f8, 0xad4dc0, ...)
/mnt/rtb_server/go/src/growmobile.com/rtb/controller/bidder_logic.go:272 +0x7c2 fp=0xc214d2e928
----- stack segment boundary -----
growmobile.com/rtb/controller.Bid(0x7f7a14250ed8, 0xc2197b3a00, 0xc2157f9dd0)
/mnt/rtb_server/go/src/growmobile.com/rtb/controller/controller.go:96 +0x903 fp=0x7f79f8dd9da0
net/http.HandlerFunc.ServeHTTP(0x800850, 0x7f7a14250ed8, 0xc2197b3a00, 0xc2157f9dd0)
/usr/local/go/src/pkg/net/http/server.go:1220 +0x40 fp=0x7f79f8dd9dc0
net/http.(*ServeMux).ServeHTTP(0xc21001e660, 0x7f7a14250ed8, 0xc2197b3a00, 0xc2157f9dd0)
/usr/local/go/src/pkg/net/http/server.go:1496 +0x163 fp=0x7f79f8dd9df8
net/http.serverHandler.ServeHTTP(0xc21001fa00, 0x7f7a14250ed8, 0xc2197b3a00, 0xc2157f9dd0)
/usr/local/go/src/pkg/net/http/server.go:1597 +0x16e fp=0x7f79f8dd9e38
net/http.(*conn).serve(0xc215d60a00)
/usr/local/go/src/pkg/net/http/server.go:1167 +0x7b7 fp=0x7f79f8dd9f98
runtime.goexit()
/usr/local/go/src/pkg/runtime/proc.c:1394 fp=0x7f79f8dd9fa0
created by net/http.(*Server).Serve
/usr/local/go/src/pkg/net/http/server.go:1644 +0x28b
goroutine 1 [chan receive]:
main.serve()
/mnt/rtb_server/go/src/growmobile.com/rtb/rtb_server/rtb_server.go:71 +0x5f0
main.main()
/mnt/rtb_server/go/src/growmobile.com/rtb/rtb_server/rtb_server.go:86 +0x1d8
At the time it is trying to access a hash which is a struct that maps to a pointer. The hash is a global one and looks like this:
var histogramCache = make(map[model.HistogramKey]*model.HistogramValue)
The access is a very standard access which looks like:
value, ok := histogramCache[key]
I tried to google something similar but could not find it. I do not use cgo or assertions.
This is the go version we are using: go version go1.2 linux/amd64
Thank you, any help would be appreciated.
Maps are not safe for concurrent access. This might very well be the reason why your code is failing.
Use a central goroutine (see this example) and retrieve things using channels or use locking tools from sync.
Reinstalling the goland package fixed this problem for me. Some dependencies had been replaced by another package or were not correctly installed.