I worked on languages like PHP and Python but recently I got a Go project to maintain which utilizes several workers and goroutines.
In PHP or Python, we can know about the execution of code as its all synchronous calls. But I am finding it very difficult to understand how the code execution takes place in Go, especially in workers and goroutine. Can anyone suggest me any tools by using which I can get the execution flow(the name of the function being called)?
For example, if there is one API in which there are various goroutines and inside those goroutines, other goroutines are called, so how to know in which sequence these functions are called.
Go runtime trace tool will help you to trace , but you need to add trace option in go code start
package main
import (
"os"
"runtime/trace"
)
func main() {
trace.Start(os.Stderr) //start the trace
defer trace.Stop() // defer to the end
.... //rest of the code
}
Then use ,
go run main.go 2> trace.out
then use the trace tool
go tool trace trace.out.
This link has more info https://blog.gopheracademy.com/advent-2017/go-execution-tracer/
[I]f there is one API in which there are various goroutines and inside those goroutines, other goroutines are called, [...]
There isn‘t.
Related
I have small Golang package which does some work. This work suppose a high amount of errors could be produced and this is OK. Currently all errors are ignored. Yes it may look strange, but visit the link and check the main purpose of package.
I'd like to extend functionality of the package and provide ability to see errors occurred during runtime. But due to lack of software design skills I have some questions with no answers.
At first, I thought to implement logging inside the package using the existing logging (zerolog, zap or whatever else). But, will it be ok for package's users? Because they might want to use other logging packages and would like to modify output format.
Maybe it's possible to provide a way to user to inject it's own logging?
I'd like to achieve the ability to provide easy-configurable way for logging which could be switched on or off on users demands.
Some go lib use logging like this
in your packge definite a logger interface
type Yourlogging interface{
Errorf(...)
Warningf(...)
Infof(...)
Debugf(...)
}
and definite a variable for this interface
var mylogger Yourlogging
func SetLogger(l yourlogging)error{
mylogger = l
}
in your func, you can call them for logging
mylogger.Infof(..)
mylogger.Errorf(...)
you don't need implement the interface, but you can use them who implement this interface
for example:
SetLogger(os.Stdout) //logging output to stdout
SetLogger(logrus.New()) // logging output to logrus (github.com/sirupsen/logrus)
In Go, you will see some libraries implement logging interfaces like other answers have suggested. However, you could completely avoid your packages needing to log if you structured your application differently, for your example.
For example, in your example application you linked, your main application runtime calls idleexacts.Run(), which starts this function.
// startLoop starts workload using passed settings and database connection.
func startLoop(ctx context.Context, log log.Logger, pool db.DB, tables []string, jobs uint16, minTime, maxTime time.Duration) error {
rand.Seed(time.Now().UnixNano())
// Increment maxTime up to 1 due to rand.Int63n() never return max value.
maxTime++
// While running, keep required number of workers using channel.
// Run new workers only until there is any free slot.
guard := make(chan struct{}, jobs)
for {
select {
// Run workers only when it's possible to write into channel (channel is limited by number of jobs).
case guard <- struct{}{}:
go func() {
table := selectRandomTable(tables)
naptime := time.Duration(rand.Int63n(maxTime.Nanoseconds()-minTime.Nanoseconds()) + minTime.Nanoseconds())
err := startSingleIdleXact(ctx, pool, table, naptime)
if err != nil {
log.Warnf("start idle xact failed: %s", err)
}
// When worker finishes, read from the channel to allow starting another worker.
<-guard
}()
case <-ctx.Done():
return nil
}
}
}
The problem here is all of the orchestration of your logic is happening inside of your packages. Instead, this loop should be running in your main application, and this package should provide users with simple actions such as selectRandomTable() or createTempTable().
If the orchestration of code was in your main application and the package only provided simple actions. It would be much easier to return errors to the user as part of the function calls.
It would also make your packages easier for others to reuse because they have simple actions and open users to use them in other ways than you intended.
Trying to understand the flow of goroutines so i wrote this code only one thing which i am not able to understand is that how routine-end runs between the other go routines and complete a single go routines and print the output from the channel at the end.
import(
"fmt"
)
func add(dataArr []int,dataChannel chan int,i int ){
var sum int
fmt.Println("GOROUTINE",i+1)
for i:=0;i<len(dataArr);i++{
sum += dataArr[i]
}
fmt.Println("wRITING TO CHANNEL.....")
dataChannel <- sum
fmt.Println("routine-end")
}
func main(){
fmt.Println("main() started")
dataChannel := make(chan int)
dataArr := []int{1,2,3,4,5,6,7,8,9}
for i:=0;i<len(dataArr);i+=3{
go add(dataArr[i:i+3],dataChannel,i)
}
fmt.Println("came to blocking statement ..........")
fmt.Println(<-dataChannel)
fmt.Println("main() end")
}
output
main() started
came to blocking statement ..........
GOROUTINE 1
wRITING TO CHANNEL.....
routine-end
GOROUTINE 4
wRITING TO CHANNEL.....
6
main() end
Your for loop launches 3 goroutines that invoke the add function.
In addition, main itself runs in a separate "main" goroutine.
Since goroutines execute concurrently, the order of their run is typically unpredictable and depends on timing, how busy your machine is, etc. Results may differ between runs and between machines. Inserting time.Sleep calls in various places may help visualize it. For example, inserting time.Sleep for 100ms before "came to blocking statement" shows that all add goroutines launch.
What you may see in your run typically is that one add goroutine launches, adds up its slice to its sum and writes sum to dataChannel. Since main launches a few goroutines and immediately reads from the channel, this read gets the sum written by add and then the program exists -- because by default main won't wait for all goroutines to finish.
Moreover, since the dataChannel channel is unbuffered and main only reads one value, the other add goroutines will block on the channel indefinitely while writing.
I do recommend going over some introductory resources for goroutines and channels. They build up the concepts from simple principles. Some good links for you:
Golang tour
https://gobyexample.com/ -- start with the Goroutines example and do the next several ones.
I'm learning Go and I'm trying to understand how to properly deal with panics from external packages.
Here is a runnable example, say a package defines the doFoo method. (It's located in the same package here for the sake of the example )
package main
import (
"log"
"net/http"
"sync"
"time"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
)
// Method from External package
func doFoo() {
var wg sync.WaitGroup
wg.Add(1)
// Do some cool async stuff
go func() {
time.Sleep(500)
wg.Done()
panic("Oops !")
}()
}
func router() *mux.Router {
var router = mux.NewRouter().StrictSlash(true)
router.HandleFunc("/doFoo", index).Methods("GET")
return router
}
func main() {
log.Fatal(http.ListenAndServe(":8080", handlers.RecoveryHandler()(router())))
}
func index(w http.ResponseWriter, r *http.Request) {
defer func() {
recover()
w.WriteHeader(http.StatusInternalServerError)
}()
doFoo()
w.WriteHeader(http.StatusOK)
}
Invoking the doFoo method will crash the server, I appreciate that is correct behavior, since the application is now in an undermined state. And it's best to crash and have subsequent requests forwarded to a different processes trough some load balancer.
But, my api server might still be serving other clients, it might be maintaining websockets, and I Might would also want to return a 500 error here.
Coming from nodejs, I am used to the concept of uncaughtException, for handeling uncaptured synchronous exceptions and unhandledRejection to handle uncaptured asynchronous exceptions. These two process constructs give the developer the choice to either crash the program right away ( if it makes sense ), or log the error, return a proper http code, and then maybe shutdown gracefully if needed.
In my online research I find a lot of resources saying, panic's are not like exceptions, they are unusual, you don't need to worry about them. But it seems like it's actually very easy to cause a panic when writing code. It's completely up to the developer to ensure his library does not panic, the human factor is 100% involved here.
This leads me to wonder, do I need to audit the entire code base of every single package I'm going to use, including all the package dependencies as well ? just because I have no means of safeguarding against a missed recover in some external package that will take down my whole server and ruin my user's experience ?
Or is there some strategy I am not aware of that I can fail gracefully when an asynchronous panic occurs in library code ?
I noticed there is graceful shutdown since 1.8, but I can't use this because my program has already crashed.
https://golang.org/pkg/net/http/#Server.Shutdown
There is the gorilla recovery handler, but again, this only protects against synchronous panics.
http://www.gorillatoolkit.org/pkg/handlers#RecoveryHandler
Update:
I am aware that panics are not exceptions. Restating that does not answer the question, panics and exceptions is not what this question is about. This question is about understanding what tools the language may provide to enforce boundaries without bestowing the need to read every single line in the entire package tree onto the developer. If it's not possible in the language then stating that is a valid answer. I just don't know if it is or not.
Panics are not exceptions. Do not treat them like exceptions and you will be fine.
First things first: Package APIs should never panic, they should always return an error except in certain very rare cases, and then they must be clearly documented as to when and why they can panic (regexp.MustCompile is a good example of something that may panic). Any package that panics if it hits an error (and doesn't have a very good reason to do so) is bad, don't use it.
If you do bounds checking, make sure not to acess nil pointers, etc you should never have to worry about panics.
As for recovering panics in a goroutine, unless the goroutine has its own recovery handler you can't.
If the goroutine in from a third party library, don't use that library! If they are lax enough not to check edge cases and/or are lazy enough to just panic on error, why are you using their code? Who knows what other mines it holds?
If the goroutine is your own code, try to eliminate things that can panic, then add a recovery handler to catch the ones you can't prevent if needed.
A Go process is running. I want to
dump a stack trace for each of its goroutines
from the outside, without depending on anything I add to its source code
without killing it.
How can I do that?
This should be easy -- the feature was requested: https://code.google.com/p/go/issues/detail?id=2516 and, according to the conclusion of that thread, implemented. That was over two years ago. But neither the issue thread nor the commit contains any hint as to how to invoke this feature.
The feature request mentioned SIGQUIT as the signal the JVM accepts to invoke the corresponding feature there. But SIGQUIT isn't the answer; on go1.2 at least, SIGQUIT does #1 and #2 but also kills the process.
Someone asked a related question here a while ago: How to dump goroutine stacktraces?
but they didn't clearly ask for #2 or #3, none of the answers meet #2, and they accepted an answer that doesn't meet #2. So this is a different question.
You can set up a handler like that using code something like this:
import (
"fmt"
"os"
"os/signal"
"runtime"
"syscall"
)
func main() {
sigChan := make(chan os.Signal)
go func() {
stacktrace := make([]byte, 8192)
for _ = range sigChan {
length := runtime.Stack(stacktrace, true)
fmt.Println(string(stacktrace[:length]))
}
}()
signal.Notify(sigChan, syscall.SIGQUIT)
...
}
The SIGQUIT signal will now be caught and sent to the given channel. The runtime.Stack function is then used to format the stack trace into a prepared buffer (if it is larger than the buffer, it will be truncated), and then printed.
If you are using net/http you can access the goroutines via the debug handlers. If you look at the following source
http://golang.org/src/pkg/runtime/pprof/pprof.go
You will see the profile, goroutineProfile, on line 62. This profile writes out via writeGoroutine. If writeGoroutine is called with debug >= 2 then it will write out all goroutines.
You should be able to curl http://localhost:<port>/debug/pprof/goroutine?debug=2 to get all goroutines dumped. Unfortunately I didn't see any references to signal handlers that would call that code but you can look at references to how pprof makes use of runtime.Stack in the source above to implement this yourself pretty easily.
I wrote the following program:
package main
import (
"fmt"
)
func processevents(list chan func()) {
for {
//a := <-list
//a()
}
}
func test() {
fmt.Println("Ho!")
}
func main() {
eventlist := make(chan func(), 100)
go processevents(eventlist)
for {
eventlist <- test
fmt.Println("Hey!")
}
}
Since the channel eventlist is a buffered channel, I think I should get at exactly 100 times the output "Hey!", but it is displayed only once. Where is my mistake?
Update (Go version 1.2+)
As of Go 1.2, the scheduler works on the principle of pre-emptive multitasking.
This means that the problem in the original question (and the solution presented below) are no longer relevant.
From the Go 1.2 release notes
Pre-emption in the scheduler
In prior releases, a goroutine that was looping forever could starve out other goroutines
on the same thread, a serious problem when GOMAXPROCS provided only one user thread.
In Go > 1.2, this is partially addressed: The scheduler is invoked occasionally upon
entry to a function. This means that any loop that includes a (non-inlined) function
call can be pre-empted, allowing other goroutines to run on the same thread.
Short answer
It is not blocking on the writes. It is stuck in the infinite loop of processevents.
This loop never yields to the scheduler, causing all goroutines to lock indefinitely.
If you comment out the call to processevents, you will get results as expected, right until the 100th write. At which point the program panics, because nobody reads from the channel.
Another solution is to put a call to runtime.Gosched() in the loop.
Long answer
With Go1.0.2, Go's scheduler works on the principle of Cooperative multitasking.
This means that it allocates CPU time to the various goroutines running within a given OS thread by having these routines interact with the scheduler in certain conditions.
These 'interactions' occur when certain types of code are executed in a goroutine.
In go's case this involves doing some kind of I/O, syscalls or memory allocation (in certain conditions).
In the case of an empty loop, no such conditions are ever encountered. The scheduler is therefore never allowed to run its scheduling algorithms for as long as that loop is running. This consequently prevents it from allotting CPU time to other goroutines waiting to be run and the result you observed ensues: You effectively created a deadlock that can not be detected or broken out of by the scheduler.
The empty loop is usually never desired in Go and will, in most cases, indicate a bug in the program. If you do need it for whatever reason, you have to manually yield to the scheduler by calling runtime.Gosched() in every iteration.
for {
runtime.Gosched()
}
Setting GOMAXPROCS to a value > 1 was mentioned as a solution. While this will get rid of the immediate problem you observed, it will effectively move the problem to a different OS thread, if the scheduler decides to move the looping goroutine to its own OS thread that is. There is no guarantee of this, unless you call runtime.LockOSThread() at the start of the processevents function. Even then, I would still not rely on this approach to be a good solution. Simply calling runtime.Gosched() in the loop itself, will solve all the issues, regardless of which OS thread the goroutine is running in.
Here is another solution - use range to read from the channel. This code will yield to the scheduler correctly and also terminate properly when the channel is closed.
func processevents(list chan func()) {
for a := range list{
a()
}
}
Good news, since Go 1.2 (december 2013) the original program now works as expected.
You may try it on Playground.
This is explained in the Go 1.2 release notes, section "Pre-emption in the scheduler" :
In prior releases, a goroutine that was looping forever could starve
out other goroutines on the same thread, a serious problem when
GOMAXPROCS provided only one user thread. In Go 1.2, this is partially
addressed: The scheduler is invoked occasionally upon entry to a
function.