I am trying to debug the fuzzing part in golang sdk (go/internal/fuzz) with GoLand, with a demo like this:
package awesomeProject1
import "testing"
func fn(s string) int {
var sum int
for _, i := range s {
sum += int(i)
}
return sum
}
func FuzzFn(f *testing.F) {
f.Add("hello")
f.Fuzz(
func(t *testing.T, s string) {
fn(s)
})
}
It works fine when I run or debug it without breakpoint:
=== RUN FuzzFn
fuzz: elapsed: 0s, gathering baseline coverage: 0/8 completed
fuzz: elapsed: 0s, gathering baseline coverage: 8/8 completed, now fuzzing with 6 workers
fuzz: elapsed: 3s, execs: 2041388 (679470/sec), new interesting: 0 (total: 8)
However, when debug with a simple breakpoint at whether f.Add or f.Fuzz, it hangs:
=== RUN FuzzFn
So how can I debug it?
I have tried to set the breakpoint inside the sdk, it doesn't work.
You are likely using Go 1.20 with GoLand 2022.3. This version of GoLand ships with Delve that supports Go versions up to 1.19.
You can either change your project to target Go 1.19 or use the EAP version of GoLand that includes an updated version of Delve.
Related
go version: 1.18.1
suppose i wrote this test file parallel_test.go
package parallel_json_output
import (
"fmt"
"testing"
"time"
)
func TestP(t *testing.T) {
t.Run("a", func(t *testing.T) {
t.Parallel()
for i := 0; i < 5; i++ {
time.Sleep(time.Second)
fmt.Println("a", i)
}
})
t.Run("b", func(t *testing.T) {
t.Parallel()
for i := 0; i < 5; i++ {
time.Sleep(time.Second)
fmt.Println("b", i)
}
})
}
after running go test parallel_test.go -v -json, i got
{"Time":"2022-06-11T02:48:10.3262833+08:00","Action":"run","Package":"command-line-arguments","Test":"TestP"}
{"Time":"2022-06-11T02:48:10.3672856+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP","Output":"=== RUN TestP\n"}
{"Time":"2022-06-11T02:48:10.3682857+08:00","Action":"run","Package":"command-line-arguments","Test":"TestP/a"}
{"Time":"2022-06-11T02:48:10.3682857+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/a","Output":"=== RUN TestP/a\n"}
{"Time":"2022-06-11T02:48:10.3692857+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/a","Output":"=== PAUSE TestP/a\n"}
{"Time":"2022-06-11T02:48:10.3702858+08:00","Action":"pause","Package":"command-line-arguments","Test":"TestP/a"}
{"Time":"2022-06-11T02:48:10.3702858+08:00","Action":"run","Package":"command-line-arguments","Test":"TestP/b"}
{"Time":"2022-06-11T02:48:10.3712858+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"=== RUN TestP/b\n"}
{"Time":"2022-06-11T02:48:10.3712858+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"=== PAUSE TestP/b\n"}
{"Time":"2022-06-11T02:48:10.3722859+08:00","Action":"pause","Package":"command-line-arguments","Test":"TestP/b"}
{"Time":"2022-06-11T02:48:10.373286+08:00","Action":"cont","Package":"command-line-arguments","Test":"TestP/a"}
{"Time":"2022-06-11T02:48:10.373286+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/a","Output":"=== CONT TestP/a\n"}
{"Time":"2022-06-11T02:48:10.374286+08:00","Action":"cont","Package":"command-line-arguments","Test":"TestP/b"}
{"Time":"2022-06-11T02:48:10.374286+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"=== CONT TestP/b\n"}
{"Time":"2022-06-11T02:48:11.3352891+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"b 0\n"}
{"Time":"2022-06-11T02:48:11.3352891+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"a 0\n"}
...
look at this line {"Time":"2022-06-11T02:48:11.3352891+08:00","Action":"output","Package":"command-line-arguments","Test":"TestP/b","Output":"a 0\n"}. this output should be printed by case TestP/a instead of b, but the output messed up the case name in parallel tests.
this problem made reporting tool generate wrong HTML report, IDEs (like GoLand) are effected too and cannot sort parallel output correctly.
i found an issue of it in Github here, but this issue seems had been fixed already in go 1.14.6, however, it still appears in go 1.18.
i wonder what happend and how to deal with it, many thanks.
It makes sense that generic fmt package has little knowledge about currently executed tests in concurrent environment.
Testing package has its own Log method that correctly renders current test:
t.Log("a", i)
I have a production golang code and functional tests for it written not in golang. Functional tests run compiled binary. Very simplified version of my production code is here: main.go:
package main
import (
"fmt"
"math/rand"
"os"
"time"
)
func main() {
rand.Seed(time.Now().UTC().UnixNano())
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
os.Exit(0)
}
if i%2 == 0 {
os.Exit(1)
}
time.Sleep(time.Second)
}
}
I want to build coverage profile for my functional tests. In order to do it I add main_test.go file with content:
package main
import (
"os"
"testing"
)
var exitCode int
func Test_main(t *testing.T) {
go main()
exitCode = <-exitCh
}
func TestMain(m *testing.M) {
m.Run()
// can exit because cover profile is already written
os.Exit(exitCode)
}
And modify main.go:
package main
import (
"flag"
"fmt"
"math/rand"
"os"
"runtime"
"time"
)
var exitCh chan int = make(chan int)
func main() {
rand.Seed(time.Now().UTC().UnixNano())
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
exit(0)
}
if i%2 == 0 {
fmt.Println("status 1")
exit(1)
}
time.Sleep(time.Second)
}
}
func exit(code int) {
if flag.Lookup("test.coverprofile") != nil {
exitCh <- code
runtime.Goexit()
} else {
os.Exit(code)
}
}
Then I build coverage binary:
go test -c -coverpkg=. -o myProgram
Then my functional tests run this coverage binary, like this:
./myProgram -test.coverprofile=/tmp/profile
6507374435908599516
PASS
coverage: 64.3% of statements in .
And I build HTML output showing coverage:
$ go tool cover -html /tmp/profile -o /tmp/profile.html
$ open /tmp/profile.html
Problem
Method exit will never show 100% coverage because of condition if flag.Lookup("test.coverprofile") != nil. So line os.Exit(code) is kinda blind spot for my coverage results, although, in fact, functional tests go on this line and this line should be shown as green.
On the other hand, if I remove condition if flag.Lookup("test.coverprofile") != nil, the line os.Exit(code) will terminate my binary without building coverage profile.
How to rewrite exit() and maybe main_test.go to show coverage without blind spots?
The first solution that comes into mind is time.Sleep():
func exit(code int) {
exitCh <- code
time.Sleep(time.Second) // wait some time to let coverprofile be written
os.Exit(code)
}
}
But it's not very good because will cause production code slow down before exit.
As per our conversation in the comments our coverage profile will never include that line of code because it will never be executed.
With out seeing your full code it is hard to come up with a proper solutions however there a few things you can do to increase the coverage with out sacrificing too much.
func Main and TestMain
Standard practice for GOLANG is to avoid testing the main application entry point so most professionals extract as much functionality into other classes so they can be easily tested.
GOLANG testing framework allows you to test your application with out the main function but in it place you can use the TestMain func which can be used to test where the code needs to be run on the main thread. Below is a small exert from GOLANG Testing.
It is sometimes necessary for a test program to do extra setup or teardown before or after testing. It is also sometimes necessary for a test to control which code runs on the main thread. To support these and other cases, if a test file contains a function: func TestMain(m *testing.M)
Check GOLANG Testing for more information.
Working example
Below is an example (with 93.3% coverage that we will make it 100%) that tests all the functionality of your code. I made a few changes to your design because it did not lend itself very well for testing but the functionality still the same.
package main
dofunc.go
import (
"fmt"
"math/rand"
"time"
)
var seed int64 = time.Now().UTC().UnixNano()
func doFunc() int {
rand.Seed(seed)
var code int
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
code = 0
break
}
if i%2 == 0 {
fmt.Println("status 1")
code = 1
break
}
time.Sleep(time.Second)
}
return code
}
dofunc_test.go
package main
import (
"testing"
"flag"
"os"
)
var exitCode int
func TestMain(m *testing.M) {
flag.Parse()
code := m.Run()
os.Exit(code)
}
func TestDoFuncErrorCodeZero(t *testing.T) {
seed = 2
if code:= doFunc(); code != 0 {
t.Fail()
}
}
func TestDoFuncErrorCodeOne(t *testing.T) {
seed = 3
if code:= doFunc(); code != 1 {
t.Fail()
}
}
main.go
package main
import "os"
func main() {
os.Exit(doFunc());
}
Running the tests
If we build our application with the cover profile.
$ go test -c -coverpkg=. -o example
And run it.
$ ./example -test.coverprofile=/tmp/profile
Running the tests
1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 93.3% of statements in .
So we see that we got 93% coverage we know that is because we don't have any test coverage for main to fix this we could write some tests for it (not a very good idea) since the code has os.Exit or we can refactor it so it is super simple with very little functionality we can exclude it from our tests.
To exclude the main.go file from the coverage reports we can use build tags by placing tag comment at the first line of the main.go file.
//+build !test
For more information about build flags check this link: http://dave.cheney.net/2013/10/12/how-to-use-conditional-compilation-with-the-go-build-tool
This will tell GOLANG that the file should be included in the build process where the tag build is present butNOT where the tag test is present.
See full code.
//+build !test
package main
import "os"
func main() {
os.Exit(doFunc());
}
We need need to build the the coverage application slightly different.
$ go test -c -coverpkg=. -o example -tags test
Running it would be the same.
$ ./example -test.coverprofile=/tmp/profile
We get the report below.
1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 100.0% of statements in .
We can now build the coverage html out.
$ go tool cover -html /tmp/profile -o /tmp/profile.html
In my pkglint project I have declared a package-visible variable:
var exit = os.Exit
In the code that sets up the test, I overwrite it with a test-specific function, and when tearing down a test, I reset it back to os.Exit.
This is a simple and pragmatic solution that works well for me, for at least a year of extensive testing. I get 100% branch coverage because there is no branch involved at all.
I have been trying to dive in to Go (golang) performance analysis, based on articles like https://software.intel.com/en-us/blogs/2014/05/10/debugging-performance-issues-in-go-programs .
However, in the actual profiled programs, the generated CPU profiles have very little information. The go tool either tells that the profile is empty or it has no information about any function calls. This happens on both OS X and Linux.
I generated a minimal example of this situation - I am gathering the profile in a similar manner and facing the same issues in actual programs, too.
Here's the source code for miniprofile/main.go:
package main
import (
"fmt"
"os"
"runtime/pprof"
)
func do_something(prev string, limit int) {
if len(prev) < limit {
do_something(prev+"a", limit)
}
}
func main() {
f, err := os.Create("./prof")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
do_something("", 100000)
}
I am expecting to see a CPU profile that tells that almost all the time has been spent on different recursive calls to do_something.
However, this happens (the minimal app above is called miniprofile) - not very useful:
$ go version
go version go1.6.2 darwin/amd64
$ go install .
$ miniprofile
$ go tool pprof --text prof
1.91s of 1.91s total ( 100%)
flat flat% sum% cum cum%
1.91s 100% 100% 1.91s 100%
Am I doing something in a horribly wrong way?
You're missing the binary argument to pprof:
go tool pprof --text miniprofile prof
in the interests of learning more about Go, I have been playing with goroutines, and have noticed something - but am not sure what exactly I'm seeing, and hope someone out there might be able to explain the following behaviour.
the following code does exactly what you'd expect:
package main
import (
"fmt"
)
type Test struct {
me int
}
type Tests []Test
func (test *Test) show() {
fmt.Println(test.me)
}
func main() {
var tests Tests
for i := 0; i < 10; i++ {
test := Test{
me: i,
}
tests = append(tests, test)
}
for _, test := range tests {
test.show()
}
}
and prints 0 - 9, in order.
now, when the code is changed as shown below, it always returns with the last one first - doesn't matter which numbers I use:
package main
import (
"fmt"
"sync"
)
type Test struct {
me int
}
type Tests []Test
func (test *Test) show(wg *sync.WaitGroup) {
fmt.Println(test.me)
wg.Done()
}
func main() {
var tests Tests
for i := 0; i < 10; i++ {
test := Test{
me: i,
}
tests = append(tests, test)
}
var wg sync.WaitGroup
wg.Add(10)
for _, test := range tests {
go func(t Test) {
t.show(&wg)
}(test)
}
wg.Wait()
}
this will return:
9
0
1
2
3
4
5
6
7
8
the order of iteration of the loop isn't changing, so I guess that it is something to do with the goroutines...
basically, I am trying to understand why it behaves like this...I understand that goroutines can run in a different order than the order in which they're spawned, but, my question is why this always runs like this. as if there's something really obvious I'm missing...
As expected, the ouput is pseudo-random,
package main
import (
"fmt"
"runtime"
"sync"
)
type Test struct {
me int
}
type Tests []Test
func (test *Test) show(wg *sync.WaitGroup) {
fmt.Println(test.me)
wg.Done()
}
func main() {
fmt.Println("GOMAXPROCS", runtime.GOMAXPROCS(0))
var tests Tests
for i := 0; i < 10; i++ {
test := Test{
me: i,
}
tests = append(tests, test)
}
var wg sync.WaitGroup
wg.Add(10)
for _, test := range tests {
go func(t Test) {
t.show(&wg)
}(test)
}
wg.Wait()
}
Output:
$ go version
go version devel +af15bee Fri Jan 29 18:29:10 2016 +0000 linux/amd64
$ go run goroutine.go
GOMAXPROCS 4
9
4
5
6
7
8
1
2
3
0
$ go run goroutine.go
GOMAXPROCS 4
9
3
0
1
2
7
4
8
5
6
$ go run goroutine.go
GOMAXPROCS 4
1
9
6
8
4
3
0
5
7
2
$
Are you running in the Go playground? The Go playground, by design, is deterministic, which makes it easier to cache programs.
Or, are you running with runtime.GOMAXPROCS = 1? This runs one thing at a time, sequentially. This is what the Go playground does.
Go routines are scheduled randomly since Go 1.5. So, even if the order looks consistent, don't rely on it.
See Go 1.5 release note :
In Go 1.5, the order in which goroutines are scheduled has been changed. The properties of the scheduler were never defined by the language, but programs that depend on the scheduling order may be broken by this change. We have seen a few (erroneous) programs affected by this change. If you have programs that implicitly depend on the scheduling order, you will need to update them.
Another potentially breaking change is that the runtime now sets the default number of threads to run simultaneously, defined by GOMAXPROCS, to the number of cores available on the CPU. In prior releases the default was 1. Programs that do not expect to run with multiple cores may break inadvertently. They can be updated by removing the restriction or by setting GOMAXPROCS explicitly. For a more detailed discussion of this change, see the design document.
I'm working on Project Euler questions in order to get used to Go. The question is not about Project Euler, but there is Project Euler specific code in this question that might give away the challenge of a question. "Spoiler Alert" or whatever, but now you know. Here's my file structure:
+ Project Euler
+-+ Go <= GOPATH set here
+-+ src
+-+ util
| +- util.go
|
+- 001.go
+- 002.go
...
+- 023.go
For problem 23, I'm adding a new function SumOfDivisors to util.go (a file with various methods used by multiple problems):
func GetPrimeFactors(val int) map[int]int {
primes := map[int]int{}
init := val
num := 2
for val > 1 {
if (val % num) == 0 {
if num == init {
return nil
}
_, e := primes[num]
if e {
primes[num]++
} else {
primes[num] = 1
}
val /= num
} else {
num++
}
}
return primes
}
func SumOfDivisors(val int) int {
primes := GetPrimeFactors(val)
if primes == nil {
if val == 0 {
return 0
} else {
return 1
}
}
total := 1
for k, v := range primes {
if v > 1 {
n := int((math.Pow(float64(k), float64(v+1)) - 1) / float64(k-1))
total *= n
} else {
n := k + 1
total *= n
}
}
return total - val
}
In order to test this method, I wrote this basic Go inside 023.go:
package main
import (
"fmt"
"util"
)
func main() {
fmt.Println(util.SumOfDivisors(12))
}
I have my GOPATH set to /Project Euler/Go and it builds and runs seemingly fine when I call go run 023.go. "Seemingly fine" meaning there are no errors, warnings, no output other than my code.
What is printing to the screen is 1 when it should be 16. I don't think it's a logic issue because when I copy the function from my util.go into 023.go (and fix the call to GetPrimeFactors to be util.GetPrimeFactors) then the function runs just fine and prints 16 just like it should. I've tried adding fmt.Println("TEST") to util.SumOfDivisors but it won't print those statements out and I don't get errors or anything else. If I change the name of the function in util.go to anything else, even if the main function is 023.go doesn't change, it still builds and runs outputting 1. It's really behaving weird.
Other functions in my util.go file seem to be called just fine.
I'm running Go 1.4.2. What might be causing this kind of behavior? The function works properly locally but not when moved to an external file that's imported and why can't that external file print anything to screen? All this while building just fine.
Use go build -a 023.go
This will rebuild all dependencies that 023.go has and will avoid using old compiled versions of a package. This is one of go's strong suits that enables faster build times but it can cause these types of problems too.
As I mentioned in my comment, you kept building 023.go but you probably didn't run go build util.go to update the util package that 023.go depended on.
The -a option will rebuild all dependencies and you can even add -v to see what it is building and when.
go build -a -v 023.go
OR
go run -a -v 023.go
Run, build, clean, and test have similar flags. Run go help build for more info.
After a little more toying around, I've found that there exists a /Project Euler/pkg folder with a util.a inside it. Apparently a version of my code was built and it's intermediate files were cached.
After deleting the pkg folder, everything fell into place. The mismatched function names became a compiler error, then (after correcting the function names) my util.go fmt.Println calls were starting to print, and my answer was coming out as 16. In the end this wasn't a code solution, it was a caching problem.