Find dead code in Golang monorepo - go

My team has all our Golang code in a monorepo.
Various package subdirectories with library code.
Binaries/services/tools under cmd
We've had it for a while and are doing some cleanup. Are there any tools or techniques that can find functions not used by the binaries under cmd?
I know go vet can find private functions that are unused in a package. However I suspect we also have exported library functions that aren't used either.

UPD 2020: The unused tool has been incorporated into staticcheck.
Unfortunately, v0.0.1-2020.1.4 will probably be the last to support this
feature. Dominik explains that it is because the check consumes a lot of
resources and is hard to get right.
To get that version:
env GO111MODULE=on go get honnef.co/go/tools/cmd/staticcheck#v0.0.1-2020.1.4
To use it:
$ staticcheck --unused.whole-program=true -- ./...
./internal/pkg/a.go:5:6: type A is unused (U1001)
Original answer below.
Dominik Honnef's unused tool might be what you're looking for:
Optionally via the -exported flag, unused can analyse all arguments as a
single program and report unused exported identifiers. This can be useful for
checking "internal" packages, or large software projects that do not export
an API to the public, but use exported methods between components.

Try running go build -gcflags -live. This passes the -live flag to the compiler (go tool compile), instructing it to output debugging messages about liveness analysis. Unfortunately, it only prints when it's found live code, not dead code, but you could in theory look to see what doesn't show up in the output.
Here's an example from compiling the following program stored in dead.go:
package main
import "fmt"
func main() {
if true {
fmt.Println(true)
} else {
fmt.Println(false)
}
}
Output of go build -gcflags -live:
# _/tmp/dead
./dead.go:7: live at call to convT2E: autotmp_5
./dead.go:7: live at call to Println: autotmp_5
If I'm reading this correctly, the second line states that the implicit call to convT2E (which converts non-interface types to interface types, since fmt.Println takes arguments of type interface{}) is live, and the third line states that the call to fmt.Println is live. Note that it doesn't say that the fmt.Println(false) call is live, so we can deduce that it must be dead.
I know that's not a perfect answer, but I hope it helps.

It is a bit dirty, but it works for me.
I had a lot of structs which I did not want to test manually, so I wrote a script that renames the struct then runs all the tests (ci/test.sh) and renames it back if any test failed:
#!/bin/sh
set -e
git grep 'struct {' | grep type | while read line; do
file=$(echo $line | awk -F ':' '{print $1}')
struct=$(echo $line | awk '{print $2}')
sed "s/$struct struct/_$struct struct/g" -i $file
echo "testing for struct $struct changed in file $file"
if ! ./ci/test.sh; then
sed "s/_$struct struct/$struct struct/g" -i $file
fi
done

It's not an open source solution, but it works.
If you guys are using Goland, you should consider using its code-inspections feature, includes useful features:
Reports unused constant
Reports unused exported functions
Reports unused exported types in the main package and in tests
Reports unused unexported functions
Reports global variables that are defined but are never used in code
Reports unused function parameters
Reports unused types
(It looks like the implementation of this feature is black box, jetbrains does not open source this feature)
Go-related detection tools seem to place more emphasis on accuracy, and they want to do their best to minimize error reporting. And using Goland's code-inspections feature may require more self-judgment. :)
Interests: Paid users only, not working for Jetbrains, simply think this feature works well.

A reliable but inelegant method I've used is to rename or comment out functions you suspect might not be used and then recompile everything -- no errors means you didn't need them.
If they are needed, it shows you where these functions are called so it's good for getting familiar with a code base and seeing how things connect.

Related

File is not `gofmt`-ed with `-s`: why is this happening and how to resolve it?

We use a linter (for Golang) that run through a Github Actions workflow every time we open or update a Pull Request on our repository.
It recently started to return the following error:
File is not `gofmt`-ed with `-s` (gofmt)
After what happened in this other PR to the file pkg/api/api/go.
(EDIT: link added to evaluate and eventually reproduce the error)
Evidences:
I would like to understand what was the source of this error, as well as how to resolve it?
Source of the error
It seems this error can be returned when the file is not properly formatted according to Go rules.
For example: If you accidentally used tab indentation rather than spaces.
EDIT: blackgreen's answer gives more accurate details about the source of the error
How to resolve it
You can use the following Go command:
gofmt -s -w <path_to_file>.go
... then commit the code.
Note that in my case: gofmt -w pkg/api/api.go was enough to resolve the problem (without the -s flag, which I found strange as the error specifically asked for the -s).
Source 1 + Source 2
The -s flag in gofmt has nothing to do with formatting. It's about simplifying the code:
Try to simplify code (after applying the rewrite rule, if any).
The warning you see comes from the linter golangci-lint. Since you claim to have fixed the error by running gofmt -w, the presence of the hint "with -s" may be due to this bug: https://github.com/golangci/golangci-lint/issues/513.
The linked issue was fixed in 2019 and released with v1.17.0. You might want to check if your pipeline is using an older version.
Assuming that your file pkg/api/api.go triggered the warning just because it was not formatted, gofmt -w solves the issue because -w overwrites the file:
If a file's formatting is different from gofmt's, overwrite it with gofmt's version.

Which function of V8 engine print informations to stdout?

I add flag “--print-opt-code” to V8 engine, then v8 could print the compiled code to stdout. And now I what to print these information to specific files, then which function should I modify?
Thank you very much~~
Try this: --redirect-code-traces-to=<filename> ;-)
Generally speaking: follow the FLAG_print_opt_code through the source and see what it does.
Depending on where you run V8 (in d8? in Chrome? in node.js?), it might be easiest to just use shell redirection: .../d8 test.js --print-opt-code > output.txt

Why isn't my GNAT's standout file descriptor working?

As part of a little project, I'm writing a shell in Ada. As such, when I was investigating the system calls, I learned that there are three ways to do it.
The POSIX system calls, which are probably the least reliable.
Passing the arguments along to C's system(), which I didn't really want to do, since this was about writing the emulator in Ada and not C.
Using GNAT's runtime libraries.
I chose to go for the last option, considering this to be the most "Ada-like" of the choices. I found a code snippet on RosettaCode here. I copied and pasted it and compiled it after changing the "cmd.exe" to "ls" and removing the second argument definition. However, nothing happens when I run the executable. The shell just goes right back to the prompt. I have tested this on two different computers, one running Fedora 21, the other Debian Jessie. Here's what I've done to test it:
Seen if lacking an arguments string caused it
Checked if any of the file descriptors in GNAT's libraries are mis-named
Redirected both stderr and stdin to stdout just to see if GNAT was dumping them to the wrong FD anyway.
Looked thoroughly through the System.OS_lib library file, and there seems to be no reason.
Googled it, but GNAT's own page on the GCC website is very poorly documented.
For now I'm using the C.Interface system in the preparation of my shell, but I'm dissatisfied with this. I'm new to Ada and have only been tinkering with it for a month or so now, so if there's some kind of Ada wisdom here that would help I'm not in on it.
UPDATE: I have tried running it with absolute path, both to /usr/bin and /bin locations, and it doesn't work. Interestingly, the result code returned by the operating system is 1, but I don't know what that means. A quick search suggests that it's for "all general errors", and another site suggests that it's for "incorrect functions".
I had to tweak the RosettaCode example a little to run /bin/ls on Debian Linux, but it does run as expected...
with Ada.Text_IO; use Ada.Text_IO;
with Gnat.OS_Lib; use Gnat.OS_Lib;
procedure Execute_Synchronously is
Result : Integer;
Arguments : Argument_List :=
( 1=> new String'("-al")
);
begin
Spawn
( Program_Name => "/bin/ls",
Args => Arguments,
Output_File_Descriptor => Standout,
Return_Code => Result
);
for Index in Arguments'Range loop
Free (Arguments (Index));
end loop;
end Execute_Synchronously;
Changes :
my Gnat (FSF Gnat 4.92 from Debian Jessie) warned about System.OS_Lib, recommending Gnat.OS_Lib instead. (Which simply renames System.OS_Lib .... why???
System.OS_Lib comments:
-- Note: this package is in the System hierarchy so that it can be directly
-- be used by other predefined packages. User access to this package is via
-- a renaming of this package in GNAT.OS_Lib (file g-os_lib.ads).
Program name including path.
Arguments. The first time I ran it, it displayed the details of "ls" itself, because it was given its own name as the first argument, so I deleted that to see the current directory instead.
Notes :
the best information ot the available subprograms and their arguments is usually in the package specs themselves in the "adainclude" folder : this is /usr/lib/gcc/x86_64-linux-gnu/4.9/adainclude on my Debian installation, locate system.ads will find yours. The specific files are: s-os_lib.ads for System.OS_Lib which exports Spawn and Standout, and a-textio.ads for Ada.Text_IO.
Standout is not the preferred way of accessing Standard Output : it's a file descriptor (integer), the preferred way would be the Standard_Output function from Ada.Text_IO which returns a File. However there doesn't seem to be an overload for Spawn which takes a File (nor would I expect one in this low level library) so the lower level file descriptor is used here.
Absent a shell, you'll need to search the PATH yourself or specify a full path for the desired executable:
Spawn (
Program_Name => "/bin/ls",
…
);
I have tried running it with absolute path…neither /usr/bin nor /bin locations work.
Use which to determine the full path to the executable:
$ which ls
/bin/ls

Julia: Having a function f() containing the macro #printf, how can I access the output outside f()?

In the Julia NMF package a verbose option provides information on convergence using the #printf macro.
How can I access this output without rewriting the NMF package io?
To rephrase, having a function f() containing the macro #printf, how can I access the output outside f()?
This does seem like useful functionality to have: I would suggest that you file an issue with the package.
However, as a quick hack, something like the following should work:
oldout = STDOUT
(rd,wr) = redirect_stdout()
start_reading(rd)
# call your function here
flush_cstdio()
redirect_stdout(oldout)
close(wr)
s = readall(rd)
close(rd)
s

go test flag: flag provided but not defined

Hi I am using a flag when testing in go:
file_test.go
var ip = flag.String("ip", "noip", "test")
I am only using this in one test file. And it works fine when only testing that one test file, but when I run:
go test ./... -ip 127.0.0.1 alle of the other test file saying: flag provided but not defined.
Have you seen this?
Regards
flag.Parse() is being called before your flag is defined.
You have to make sure that all flag definitions happen before calling flag.Parse(), usually by defining all flags inside init() functions.
If you've migrated to golang 13, it changed the order of the test initializer,
so it could lead to something like
flag provided but not defined: -test.timeout
as a possible workaround, you can use
var _ = func() bool {
testing.Init()
return true
}()
that would call test initialization before the application one. More info can be found on the original thread:
https://github.com/golang/go/issues/31859#issuecomment-489889428
do not call flag.Parse() in any init()
I'm very late to the party; but is this broken (again) on Go 1.19.5?
I hit the same errors reported on this thread and the same solution reported above (https://github.com/golang/go/issues/31859#issuecomment-489889428) fixes it.
I see a call to flags.Parse() was added back in go_test.go in v1.18
https://go.googlesource.com/go/+/f7248f05946c1804b5519d0b3eb0db054dc9c5d6%5E%21/src/cmd/go/go_test.go
I am only just learning Go so it'd be nice to have some verification from people more skilled before I report this elsewhere.
If you get this, when running command via docker-compose then you do incorrect quoting. Eg.
services:
app:
...
image: kumina/openvpn-exporter:latest
command: [
"--openvpn.status_paths", "/etc/openvpn_exporter/openvpn-status.log",
"--openvpn.status_paths /etc/openvpn_exporter/openvpn-status.log",
]
First is correct, second is wrong, because whole line counted as one parameter. You need to split them by passing two separate strings, like in first line.

Resources