I made some changes to a fairly large project of mine today, and now I'm getting some odd behavior. Because I'm a knucklehead, I can't go back and figure out what I did.
But the main thrust of my question is how I should understand the negative line number in the stack trace that is printed. The -1218 below is the one that I mean.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x1 pc=0x80501f2]
goroutine 1 [running]:
server.init() // vv-------------RIGHT HERE
/home/.../debugComponent.go:-1218 +0x282
_/home/.../possessions.init()
/home/.../possessions.go:29 +0x42
_/home/.../pageWrap.init()
/home/.../pageWrap.go:112 +0x47
main.init()
/home/.../main.go:0 +0x3c
goroutine 2 [syscall]:
goroutine 3 [runnable]:
The associated debugComponent.go file is pretty non-essential right now, so I removed it to see what would happen, and the file name just gets replaced with a different one, and a different negative number.
I've had to find plenty of bugs while developing this app, but this one has got me stumped.
If it helps, there's the main.go and then several packages in play. The three files listed above are all different packages, and this seems to be happening during the imports.
I hope you've read this far, because here's the strangest part. If I add this declaration to main.go, the error goes away!
var test = func() int { return 1 }() // Everything is fine now!
Very confusing! It doesn't fix it if I do var test = "foobar". It has to be the invoked func.
Any insight is appreciated, but mostly I'm curious about the -1218 in the trace.
Update
I'm trying to get this down to a small example that reproduces the issue. After working on it I reverted back to my original code, and restarted the machine.
The first time I tried to build and run, two new entries were added to the top of the stack trace. But only the first time.
goroutine 1 [syscall]:
syscall.Syscall()
/usr/local/go/src/pkg/syscall/asm_linux_386.s:14 +0x5
syscall.Mkdir(0x83a2f18, 0x2, 0x2, 0x806255e, 0x83a2f1c, ...)
/usr/local/go/src/pkg/syscall/zerrors_linux_386.go:2225 +0x80
server.init()
So this would be in line with my main question about interpreting stack trace. The -1218 is still there, but now there are these.
The asm_linux_386.s has this at line 14:
MOVL 4(SP), AX // syscall entry
I found the zerrors_linux_386.go too, but there's no line 2225. The file stops long before that line.
It's already reported and accepted as Issue 5243.
Program execution
A package with no imports is initialized by assigning initial values
to all its package-level variables and then calling any package-level
function with the name and signature of
func init()
defined in its source. A package-scope or file-scope identifier with
name init may only be declared to be a function with this signature.
Multiple such functions may be defined, even within a single source
file; they execute in unspecified order.
Within a package, package-level variables are initialized, and
constant values are determined, according to order of reference: if
the initializer of A depends on B, A will be set after B. Dependency
analysis does not depend on the actual values of the items being
initialized, only on their appearance in the source. A depends on B if
the value of A contains a mention of B, contains a value whose
initializer mentions B, or mentions a function that mentions B,
recursively. It is an error if such dependencies form a cycle. If two
items are not interdependent, they will be initialized in the order
they appear in the source, possibly in multiple files, as presented to
the compiler. Since the dependency analysis is done per package, it
can produce unspecified results if A's initializer calls a function
defined in another package that refers to B.
An init function cannot be referred to from anywhere in a program. In
particular, init cannot be called explicitly, nor can a pointer to
init be assigned to a function variable.
If a package has imports, the imported packages are initialized before
initializing the package itself. If multiple packages import a package
P, P will be initialized only once.
The importing of packages, by construction, guarantees that there can
be no cyclic dependencies in initialization.
A complete program is created by linking a single, unimported package
called the main package with all the packages it imports,
transitively. The main package must have package name main and declare
a function main that takes no arguments and returns no value.
func main() { … }
Program execution begins by initializing the main package and then
invoking the function main. When the function main returns, the
program exits. It does not wait for other (non-main) goroutines to
complete.
Package initialization—variable initialization and the invocation of
init functions—happens in a single goroutine, sequentially, one
package at a time. An init function may launch other goroutines, which
can run concurrently with the initialization code. However,
initialization always sequences the init functions: it will not start
the next init until the previous one has returned.
As your program begins execution, it initializes package variables and executes init functions. Adding package variables is going to change the initialization. It looks like the initialization failed in debugComponent.go on something related to server.init(). The negative line number is probably a bug.
Without the source code, it's hard to say more.
Related
In Golang, function which is not starting with an Uppercase is a private function that can not be accessible outside the package then how come main() function from the main package is accessed while starting the application? Is there any special provision made for the call of main()?
main() is a special case. "Exported" and "unexported" are qualities of identifiers used by the compiler to determine if one package is allowed to directly reference an identifier from a different package. But main() isn't being called like a normal function. Your main() is called as the final step in the Go runtime's startup process, which includes starting up the goroutine scheduler and garbage collector, initializing package variables, and calling init() functions (another special case), among other things.
Here's the spec explanation, here's a thorough analysis of runtime intialization, and here's the source of the runtime's main() entrypoint.
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 5 years ago.
Improve this question
The following go play example shows in a simplistic way what I have defined. I am passing a map as a copied value to a function (not a reference) as well as there is a recursion in my function which I assume passes by value as well.
https://play.golang.org/p/na6y6Wih4M
// this function has no write operations to dataMap, just reads
// dataMap, in fact, has no write operations since it was copied
func findParentAncestors(ID int, dataMap map[int]Data) []Data {
results := []Data{}
if _, ok := dataMap[ID]; ok {
if parentData, ok := dataMap[dataMap[ID].ParentID]; ok {
results = append(results, parentData)
// recursion
results = append(results, findParentAncestors(parentData.ID, dataMap)...)
}
}
return results
}
PROBLEM: somehow along my program execution, which involves much more data than this example (obviusly), an error "fatal error: concurrent map read and map write" points function findParentAncestors():
main.findParentAncestors(0x39e3, 0xc82013ac90, 0x0, 0x0, 0x0)
/opt/test/src/test.go:17 +0xa6 fp=0xc820269fb8 sp=0xc820269bd0
main.findParentAncestors(0x5d25, 0xc82013ac90, 0x0, 0x0, 0x0)
/opt/test/src/test.go:21 +0x239 fp=0xc82026a3a0 sp=0xc820269fb8
From your example, https://play.golang.org/p/na6y6Wih4M:
// the orignalMap is defined elsewhere in the program (here represented)
originalMap := map[int]Data{}
originalMap[0] = Data{ID: 0, ParentID: -1, Name: "zero"}
originalMap[1] = Data{ID: 1, ParentID: 0, Name: "one"}
originalMap[2] = Data{ID: 2, ParentID: 1, Name: "two"}
// copies the original map from a global location (here represented)
copiedMap := originalMap
// identifies ancestors unsing the copied map
parents := findParentAncestors(2, copiedMap)
This is a misnomer, copiedMap := originalMap, you are not copying the map.
In Go all arguments are passed by value. It's equivalent to assigning each argument to each parameter. For a map, assignment, copiedMap := originalMap, or passing by value, findParentAncestors(2, copiedMap), copies the map descriptor which is a pointer to the map descriptor struct which contains a pointer to the map key-value data. Obviously you have a potential race condition if there are any writes to the map.
You are using go version go1.6.3 linux/amd64, so run the race detector.
Go 1.6 Release Notes
Runtime
The runtime has added lightweight, best-effort detection of concurrent
misuse of maps. As always, if one goroutine is writing to a map, no
other goroutine should be reading or writing the map concurrently. If
the runtime detects this condition, it prints a diagnosis and crashes
the program. The best way to find out more about the problem is to run
the program under the race detector, which will more reliably identify
the race and give more detail.
Command go
Compile packages and dependencies
-race
enable data race detection.
Supported only on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64.
Also, compile and run your program using Go 1.8, the current release of Go, which significantly improves concurrent map misuse.
Go 1.8 Release Notes
Concurrent Map Misuse
In Go 1.6, the runtime added lightweight, best-effort detection of
concurrent misuse of maps. This release improves that detector with
support for detecting programs that concurrently write to and iterate
over a map.
As always, if one goroutine is writing to a map, no other goroutine
should be reading (which includes iterating) or writing the map
concurrently. If the runtime detects this condition, it prints a
diagnosis and crashes the program. The best way to find out more about
the problem is to run the program under the race detector, which will
more reliably identify the race and give more detail.
Go 1.7 beta 1 was released this morning, here is the release notes draft of Go 1.7. A new function KeepAlive was added to the package runtime. The doc of runtime.KeepAlive has given an example:
type File struct { d int }
d, err := syscall.Open("/file/path", syscall.O_RDONLY, 0)
// ... do something if err != nil ...
p := &FILE{d}
runtime.SetFinalizer(p, func(p *File) { syscall.Close(p.d) })
var buf [10]byte
n, err := syscall.Read(p.d, buf[:])
// Ensure p is not finalized until Read returns.
runtime.KeepAlive(p)
// No more uses of p after this point.
The doc of runtime.SetFinalizer has also given an explanation about runtime.KeepAlive:
For example, if p points to a struct that contains a file descriptor
d, and p has a finalizer that closes that file descriptor, and if the
last use of p in a function is a call to syscall.Write(p.d, buf,
size), then p may be unreachable as soon as the program enters
syscall.Write. The finalizer may run at that moment, closing p.d,
causing syscall.Write to fail because it is writing to a closed file
descriptor (or, worse, to an entirely different file descriptor opened
by a different goroutine). To avoid this problem, call
runtime.KeepAlive(p) after the call to syscall.Write.
What confused me is that the variable p has not left its life scope yet, why will it be unreachable? Does that mean that a variable will be unreachable if only there is no use of it in the following code, no matter whether it is in its life scope?
A variable becomes unreachable when the runtime detects that the Go code cannot reach a point where that variable is referenced again.
In the example you posted, a syscall.Open() is used to open a file. The returned file descriptor (which is just an int value) is "wrapped" in a struct. Then a finalizer is attached to this struct value that closes the file descriptor. Now when this struct value becomes unreachable, its finalizer may be run at any moment, and the closing / invalidation / re-using of the file descriptor could cause unexpected behavior or errors in the execution of the Read() syscall.
The last use of this struct value p in Go code is when syscall.Read() is invoked (and the file descriptor p.d is passed to it). The implementation of the syscall will use that file descriptor after the initiation of syscall.Read(), it may do so up until syscall.Read() returns. But this use of the file descriptor is "independent" of the Go code.
So the struct value p is not used during the execution of the syscall, and the syscall blocks the Go code until it returns. Which means the Go runtime is allowed to mark p as unreachable during the execution of Read() (before Read() returns), or even before its actual execution begins (because p is only used to provide the arguments to call Read().
Hence the call to runtime.KeepAlive(): since this call is after the syscall.Read() and it references the variable p, the Go runtime is not allowed to mark p unreachable before Read() returns, because this is after the Read() call.
Note that you could use other constructs to "keep p alive", e.g. _ = p or returning it. runtime.KeepAlive() does nothing magical in the background, its implementation is:
func KeepAlive(interface{}) {}
runtime.KeepAlive() does provide a much better alternative because:
It clearly documents we want to keep p alive (to prevent runs of Finalizers).
Using other constructs such as _ = p might get "optimized" out by future compilers, but not runtime.KeepAlive() calls.
I’m working with a native linux C binary which has a fairly expensive initialization call which I would like to perform once at application startup. This call should open a bunch of file handles internally for later use. When I call this expensive initialization C function from Go, it completes successfully and correctly opens the files but those handles are open only for the duration of the call to the C function! This means that when I call successive C functions against the same library from Go, the file handles are no longer open and the calls fail. I have verified this using the lsof command. Interestingly, when the initialization call as well as calls to subsequent behavior are composed into a single C function which is then called from Go, the files are opened and remain open, allowing successful completion of all desired functionality.
Is there some kind of undocumented cgo behavior which is “cleaning up”, shutting down, or even leaking file handles or other stateful resources between multiple invocations of C functions from Go? If so, is this behavior configurable? We don’t have access to the source code for this library.
Also, I've verified that this is not related to thread-local storage. Calling runtime.LockOSThread() has no effect and we've verified that the files are closed after control returns from C back to the calling Go code.
Here’s an example of the kind of Go code I’d like to write:
// Go code:
func main() {
C.Initialize()
C.do_stuff() // internal state is already cleaned up! This call fails as a result. :(
}
Here’s an example of a C function that invokes the initialization and behavior all at once. This “wrapping” function is invoked from Go:
// C code:
void DoEverything(void)
{
Initialize();
do_stuff(); // succeeds because all internal state is intact (not cleaned up).
}
Ok, this is a bit embarrassing, but I figured it out. Right after calling initialize(), I was calling defer close(), but it was actually defer fmt.Println(close()). Because arguments to deferred functions are resolved immediately (not deferred), the close function was being invoked before we could invoke any other behavior. The golang blog clearly explains argument resolution to deferred function calls.
First I'd like to point out that I'm using a GNU GCC compiler. I'm using Code::Blocks as my IDE so I don't have to type in all the compiler junk into the Windows DOS command prompt. If I could be more specific about my compiler, what shows up as a line at the bottom of Cod::Blocks when I successfully compile is
mingw32-g++.exe -std=c++11 -g
Anyways, my question involves using the delete operator to release dynamically allocated memory. When I compile this code snippet:
int* x;
x = new int;
delete x;
delete x;
I don't get any warnings or errors or crashes. From the book I'm learning C++ from, releasing a pointer to a dynamically allocated memory chuck can only be done once, then the pointer is invalid. If you use delete on the same pointer again, then there will be problems. However, I don't get this problem.
Likewise, if I pass an object by value to a function, so that it is shallow copied, I get no error if I don't have a copy constructor to ensure deep copy (using raw pointers in the object). This means that when the function returns, the shallow copy goes out of scope, and invokes its destructor (where I'm using delete on a pointer). When int main returns, the original object goes out of scope, its destructor is invoked, and that same shallow copied pointer is deleted. But I have no problems.
I tried finding documentation online about the compiler I'm using and couldn't find any. Does this mean that the mingw32 compiler has some sort of default copy constructor it uses? Thus, I don't have to worry about creating copy constructors?
The compiler documentation is not likely to be helpful in this case: If it exists, it is likely to list exceptions to the C++ spec. It's the C++ spec that you need here.
When you delete the same pointer twice, the result--according to the C++ spec--is undefined. The compiler could do anything at all and have it meet spec. The compiler is allowed to recognize the fault and give an error message, or not recognize the fault and blow up immediately or sometime later. That your compiler appeared to work this time is no indication that the double delete is safe. It could be mucking up the heap in a way that results in a seg fault much later.
If you do not define a copy constructor, C++ defines one for you. The default copy constructor does a memberwise copy.
When you have the same object pointed to by multiple pointers, such as you do, consider using std::smart_ptr.