For unit testing a Go code base, is there any reasonable alternative to the foo.go and foo_test.go pattern? The only "concern" is having all those extra files in the same place on the filesystem. It might be nice to put the test files in a central place, but that might not work due to the way Go packages work.
When you tell the go command to test a package, e.g.
go test some/path/mypackage
Then the go tool will look for test files in the package's directory. If you put the test files elsewhere, then the go tool will not find them (it will not even look for them) in other folders, so they will not be run / executed.
This argument alone is enough to not put them elsewhere.
Package doc of testing:
To write a new test suite, create a file whose name ends _test.go that contains the TestXxx functions as described here. Put the file in the same package as the one being tested.
Command go: Testing functions:
The 'go test' command expects to find test, benchmark, and example functions in the "*_test.go" files corresponding to the package under test.
Some notes:
The go tool only expects test files to be in the same folder, but you can name them however you like, you just have to use the _test.go suffix. E.g. you may have a foo.go, and you may use my_test.go for the tests. There is also no requirement to have a separate test file for each .go source files, you may put all tests into a single test file, and you may have more test files than source files.
In the test files, you may use the same package name, and then the test files are compiled together with the package so tests have access to everything–including unexported identifiers–in the package (white-box testing). You may use the package name suffixed with _test, in which case the tests in those files will only have access to the package's exported identifiers (black-box testing). Read more about this here: How can I allow one package access to another package's unexported data only when testing?
You shouldn't worry about the number of .go files. To go tool will handle even if you have a thousand .go files in a package. Although in most cases if you have many .go files in a package, that's an indication that the package is doing too much, and it should be broken into multiple, smaller packages.
Related
I successfully used rules_go to build a gRPC service:
go_proto_library(
name = "processor_go_proto",
compilers = ["#io_bazel_rules_go//proto:go_grpc"],
importpath = "/path/to/proto/package",
proto = ":processor_proto",
deps = ["//services/shared/proto/common:common_go_proto"],
)
However, I'm not sure how to import the resulting file in VSCode. The generated file is nested under bazel_bin and under the original proto file path; so to import this, it seems like I would need to write out the entire path (including the bazel_bin part) to the generated Go file. To my understanding, there doesn't seem to be a way to instruct VSCode to look under certain folders that only contain Go packages/files; everything seems to need a go.mod file. This makes it quite difficult to develop in.
For clarity, my directory structure looks something like this:
WORKSPACE
bazel-bin
- path
- to
- generated_Go_file.go
go.mod
go.sum
proto
- path
- to
- gRPC_proto.proto
main.go
main.go should use the generated_Go_file.go.
Is there a way around this?
I don't use Bazel and so cannot help with the Bazel configuration. It's likely there is a way to specify the generated code location so that you can revise this to reflect you preference.
The outline you provide of the generated code, is workable though and a common pattern. Often the generated proto|gRPC code is placed in a module's gen subdirectory.
This is somewhat similar to vendoring where your code incorporates what may often be a 3rd-party's stubs (client|server) into your code. The stubs must reflect the proto(s) package(s) and, when these are 3rd-party, using gen or bazel-bin provide a way to keep potentially multiple namespaces discrete.
You're correct that the import for main.go, could (!) be prefixed with the module name from go.mod (first line) followed by the folder path to the generated code. This is standard go packaging and treats the generated code in a similar way to vendored modules.
Another approach is to use|place the generated code in a different module.
For code generated from 3rd-party protos, this may be preferable and the generated code may be provided by the 3rd-party in a module that you can go get or add to your go.mod.
An example of this approach is Google Well-Known Types. The proto (sources) are bundled with protoc (lib directory) and, when protoc compiles sources that references any of these, the Go code that is generated includes imports that reference a Google-hosted location of the generated code (!) for these types (google.golang.org/protobuf/types/known).
Alternatively, you can replicate this behavior without having to use an external repo. The bazel-bin folder must be outside of the current module. Each distinct module in bazel-bin, would need its own go.mod file. You would include in a require block in your code's go.mod file references to the modules' (one or more) locations. You don't need to publish the module to a external repo but can simply require ( name => path/to/module ) to provide a local reference.
I have a directory view_test with two files: main_test.go and test.go.
Both files start with
main_test.go:
package view_test
import (
test.go:
package view_test
import (
Now when I try to test with go test I get:
found packages view (main_test.go) and view_test (test.go) in /foo/internal/resources/view_test
Seems like there are some unknown assumptions where go starts clobbering together package names. Could someone enlighten me what these are?
The _test suffix in test files is special. From go help test:
Test files that declare a package with the suffix "_test" will be compiled as a separate package, and then linked and run with the main test binary.
This feature exists so you can enforce black box tests. Normally tests are defined in the same package as non-test code, so tests have access to unexported identifiers. If that's not desired (i.e. because you want to make sure that tests only use the public API), tests can be declared in a *_test package instead. It's an exception to the rule that all go files in a directory must declare the same package name.
As pointed out in the comments in your case this leads to a package named "view" in the context of tests, and a package named "view_test " in the context of production code. That's the wrong way around and therefore causes multiple defined packages in the same directory.
Don't use the _test suffix for production code to fix the issue.
I am writing units test. I put all of the test files in other directory. Let's say the folder mypack.
There two files in the folder fun1_test.go and base.go.
The base.go has same common basic functions which are called by fun1_test.go. The base.go looks like:
package mypack_test
import (
.....
)
func Base1() {
// some code
}
The func1_test.go has functions which test func1. The func1_test.go looks like:
package mypack_test
import (
.....
)
func TestFunc1() {
// some code
Base1()
// some code
}
When I use command
go test func1_test.go base.go
There will be an error:
can't load package: package main: found packages mypack (func1_test.go) and mypack_test (base.go)
Can any one tell me why this happend. I know If the change the file base.go to base_test.go. The command will work ok. But I want to know why.
UPDATE:
I notice some of you guys misunderstanding the problem. The problem is not about if the two file need the other packages or one can call one.
The problem is that: If you have two file with same package, the package name looks like xxx_test. But the two files' name are yyy_test.go and zzz.go. When using go test yyy_test.go zzz.go command, there will be an error said they two file not in same packages.
The error are: can't load package: package main: found packages xxx (yyy_test.go) and xxx_test (zzz.go)
You should follow go's best practices.
That is:
Package names should contain only letters with no underscores.
Test file should be name of original file + test like: base.go - base_test.go.
Run test by going to packages directory and running go test.
If you make those changes, your tests should run without any problems.
If you checkout the go help test command there is this:
Test files that declare a package with the suffix "_test" will be
compiled as a separate package, and then linked and run with the main
test binary.
What is happening is that your file yyy_test.go is recognised as a test file because of its _test.go ending. The package defined in that file package xxx_test is considered to be the test version of the xxx package.
See this stack answer for desc: https://stackoverflow.com/a/31443271/6376471
Then, along comes zzz.go which is not recognised as a test file, because it's missing the _test.go suffix, it has a packge xxx_test which is not considered to be a test package.
This means that essentially you are defining the packages xxx_test from zzz.go and xxx from yyy_test.go, even though yyy_test.go actually defines xxx_test, but it's in a test file so handled differently.
Solutions:
Identify zzz.go as a test file by making it zzz_test.go.
Set zzz.go to have the non test package name package xxx instead of package xxx_test.
Source file from Golang's stdlib
File's base directory: ast
Package specified in the file: ast_test ???
Package specified in all other files inside the same directory: ast
From golang.org:
src contains Go source files organized into packages (one package per directory) ...
By convention, packages are given lower case, single-word names; there should be no need for underscores or mixedCaps
... Another convention is that the package name is the base name of its source directory
How is it possible to have multiple packages (here 2) in one folder?
You find another example in src/pkg/go/ast/commentmap_test.go, with the comment:
// To avoid a cyclic dependency with go/parser, this file is in a separate package.
I suppose it allows for an othogonal command like:
go test
That will test parser features while avoiding for that test to be part of the same parser features (since it has been put in a separate package)
From go command man page:
Test files that declare a package with the suffix "_test" will be compiled as a separate package, and then linked and run with the main test binary.
This thread asked the question:
Now that the go tool requires each directory to be one package and doesn't allow to have files with different package names inside the same folder, how is the package keyword useful? It seems like a unnecessary repetition.
Is it required by the compiler or is there any plan to remove it?
The answers hinted at the fact that you can have more than one package in a folder:
The package declaration declares the name of the package.
The language Go doesn't know what a file or a directory is and the import path itself doesn't effect the actual name of the package that is being imported. So the only way the compiler knows what to call the package is the package declaration.
The language doesn't require separate packages to be in separate directories; it is a requirement of the go tool.
Another hypothetical implementation may not have this requirement.
Even this go tool requirement can be bypassed thanks to the "// +build" build tags.
For example, read misc/cgo/gmp or misc/cgo/stdio (some files include // +build ignore)
This repo has 3 go files all begin with "package lumber".
To use this package, I'm supposed to put this in my GOROOT and simply
import lumber
in my program. How do variables and types in this package connect with each other across multiple files? How does the go compiler know which file to begin reading first?
In case I want to read the package, where should I begin reading to understand the package? What exactly is the flow of things here?
To elaborate on jnml's answer:
When you use import "foo/bar" in your code, you are not referring to the source files (which will be located in $GOPATH/src/foo/bar/).
Instead, you are referring to a compiled package file at $GOPATH/pkg/$GOOS_$GOARCH/foo/bar.a. When you build your own code, and the compiler finds that the foo/bar package has not yet been compiled (or is out of date), it will do this for you automatically.
It does this by collating* all the relevant source files in the $GOPATH/src/foo/bar directory and building them into a single bar.a file, which it installs in the pkg directory. Compilation then resumes with your own program.
This process is repeated for all imported packages, and packages imported by those as well, all the way down the dependency chain.
*) How the files are collated, depends on how the file itself is named and what kind of build tags are present inside it.
For a deeper understanding of how this works, refer to the build docs.
No, you're not "supposed to put this in my GOROOT". You're supposed to execute
$ go get github.com/jcelliott/lumber
which will clone the repository into $GOPATH/src/github.com/jcelliott/lumber. Then you can use the package by importing it in your code as
import "github.com/jcelliott/lumber"
About the scoping rules: Declarations and scope