I have a directory structure like this:
Animal/
dog/
chiwawa.go
rabbit.go
Both chiwawa.go and rabbit.go are packaged named: "package animal" at the top of the file. However when I try to use a variable var Food = apple from chiwawa.go in rabbit.go, I get error - undefined: Food.
I don't have any problems when I place chiwawa.go directly under Animal without the dog directory.
Is this intentional in Go? Can I work around this while retaining the package names and directory structure?
How to Write Go Code
Workspaces
The go tool is designed to work with open source code maintained in
public repositories. Although you don't need to publish your code, the
model for how the environment is set up works the same whether you do
or not.
Go code must be kept inside a workspace. A workspace is a directory
hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
The go tool builds source packages and installs the resulting binaries
to the pkg and bin directories.
The src subdirectory typically contains multiple version control
repositories (such as for Git or Mercurial) that track the development
of one or more source packages.
To give you an idea of how a workspace looks in practice, here's an
example:
bin/
hello # command executable
outyet # command executable
pkg/
linux_amd64/
github.com/golang/example/
stringutil.a # package object
src/
github.com/golang/example/
.git/ # Git repository metadata
hello/
hello.go # command source
outyet/
main.go # command source
main_test.go # test source
stringutil/
reverse.go # package source
reverse_test.go # test source
This workspace contains one repository (example) comprising two
commands (hello and outyet) and one library (stringutil).
A typical workspace would contain many source repositories containing
many packages and commands. Most Go programmers keep all their Go
source code and dependencies in a single workspace.
For the Go tools, all the Go source files for a Go package should be in the same directory.
It is intentional. Packages in Go are imported using a directory path. So, "animal" and "animal/dog" are two different packages even if you give them the same package name.
To demonstrate this, I wrote the following example set of files:
~/go/src/test/
├── animal
│ ├── animal.go
│ └── dog
│ └── dog.go
└── test.go
animal/animal.go
package animal
type Species string
animal/dog/dog.go
package animal
// In package "animal" importing a package "animal"
import "test/animal"
type Dog struct {
Species animal.Species
Breed string
}
test.go
package main
import (
"fmt"
"test/animal/dog"
)
func main() {
d := animal.Dog{}
fmt.Printf("%#v", d)
}
Notice that dog.go imports the "test/animal" package and references it as "animal".
Furthermore notice that test.go imports "test/animal/dog" to get access to the "Dog" struct type. If it imports, "test/animal", the error (undefined: animal.Dog) is generated.
Related
I can not find a way to factor out some code from main.go into a local package when using Go modules (go version >= 1.11) outside of $GOPATH.
I am not importing any external dependencies that need to be included into go.mod, I am just trying to organize locally the source code of this Go module.
The file main.go:
package main
// this import does not work
import "./stuff"
func main() {
stuff.PrintBaz()
}
The file ./stuff/bar.go (pretending to be a local package):
package stuff
import "log"
type Bar struct {
Baz int
}
func PrintBaz() {
baz := Bar{42}
log.Printf("Bar struct: %v", baz)
}
The file go.mod (command go mod init foo):
module foo
go 1.12
When executing go run main.go:
If I import "./stuff", then I see build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
If I import "stuff", then I see build command-line-arguments: cannot load stuff: cannot find module providing package stuff.
If I import stuff "./stuff" with a package alias, then I see again: build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
I can not find a way to make local packages work with go modules.
What's wrong with the code above?
How can I import local packages into other Go code inside a project defined with Go modules (file go.mod)?
Module structure
The most common and easiest approach is:
Use a single go.mod per repository, and
Place the single go.mod file in the repository root, and
Use the repository name as the module path declared in the module line in the go.mod
(If you are using a custom import path such as me.io/mymod rather than using a VCS host based import path, then you would use the custom import path instead of the repository name in your go.mod).
For example, if your repo is github.com/my/repo, then you would place a single go.mod in the repo root, with the first line reading module github.com/my/repo. That can be created by cd'ing to the repo root and running go mod init github.com/my/repo. When working with the repo locally, the repo directory can be located wherever is convenient on your filesystem. (If you do not have a repo, see below).
Following this helps you stay on the happy path with modules, and it avoids multiple subtleties.
Russ Cox commented in #26664:
For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.
There is much more about multi-module repositories in the "Multi-module Repositories" FAQ section on the modules wiki. Those 6 or so FAQs in that section should be read in their entirety by anyone considering veering off the recommendation above.
Arranging packages within a module
Once you have set up your go.mod, you can arrange your packages in directories however you see fit in directories underneath the directory containing the go.mod, as well as in the directory with the go.mod. Three good articles about how to arrange your code in packages:
https://rakyll.org/style-packages/
https://medium.com/#benbjohnson/standard-package-layout-7cdbc8391fc1#.ds38va3pp
https://www.goinggo.net/2017/02/design-philosophy-on-packaging.html
Those articles are classics that pre-date the introduction of modules, but the philosophies in them still apply to how to arrange your packages within a module.
Importing other packages in the same module
When importing another package with modules, you always use the full path including the module path. This is true even when importing another package in the same module. For example, if a module declared its identity in its go.mod as module github.com/my/repo, and you had this organization:
repo/
├── go.mod <<<<< Note go.mod is located in repo root
├── pkg1
│ └── pkg1.go
└── pkg2
└── pkg2.go
Then pkg1 would import its peer package as import "github.com/my/repo/pkg2". Note that you cannot use relative import paths like import "../pkg2" or import "./subpkg". (This is part of what OP hit above with import "./stuff").
Modules vs. repositories vs. packages vs. import paths
A Go module is a collection of related Go packages that are versioned together as a single unit. Modules record precise dependency requirements and create reproducible builds.
Summarizing the relationship between repositories, modules, and packages:
A repository contains one or more Go modules (most often exactly one module in the repository root).
Each module contains one or more Go packages.
Each package consists of one or more Go source files that all reside in a single directory.
Each Go source code file:
declares its own package with a package foo statement, which must be consistent for all Go files in a single package directory.
automatically has access to other Go source code in the same package, without explicitly importing its own package.
imports code from another package via an import path supplied in an import statement such as import "github.com/my/repo/pkg1". The import path always starts with the module path of that package, regardless of whether that package is in the same module or a different module.
A single module can be used to build multiple executable binaries, but the func main() {...} entry point for each binary must be in its own package main, with each package main in a distinct directory.
Local-only modules
You can create, build and run modules without ever publishing them or placing them on a VCS server.
A module is defined by a tree of Go source files with a go.mod file in the tree's root directory, so you can pick a directory on your local filesystem to serve as the root of your module and place your go.mod there. Everything described above still applies, except there does not need to be a repo.
For example:
some-directory/
├── go.mod <<<< go.mod at module root declaring 'module example.com/foo'
├── pkg1
│ └── pkg1.go
└── pkg2
└── pkg2.go
If the module path in the first line of that go.mod reads module example.com/foo, then pkg1 would import its peer package as import "example.com/foo/pkg2".
Of course, you can start with a local-only module without a repo, then later start tracking it in a local repo or publish it to a VCS server.
Even for local-only modules, it is a good idea to give the module a globally unique name, such as go mod init my.org/foo, or go mod init github.com/some/repo as if it will be published someday, even if that repo does not yet exist. (If you are in a hurry, it is possible to use a shorter module name without a dot or leading hostname component, such as go mod init temp or go mod init m, but it won’t be fetchable by the go command, and you run the risk of colliding with a current or future standard library package or otherwise seeing confusing error messages later).
If you have multiple local modules, you can point them at each other via replace directives or via the Go 1.18+ workspace feature, though those are more complex workflows than placing everything for a given project in a single module.
Next steps
If you are new to Go modules, it is worthwhile to go through the official "Create a Go Module" tutorial from the Go project. This will likely save you time overall.
If you are stuck right now with an error message related to modules or import paths, there's a very good chance the error will make much more sense after completing the tutorial.
First you have to choose a name for your project and write it to go.mod file. This name belongs to root directory of the project. Each new package you create must be located inside its own subdirectory and its name should match directory name.
go.mod:
module myprojectname
or (preferred way, see #thepudd's answer for details)
module github.com/myname/myproject
Then import your project's packages like:
import myprojectname/stuff
or
import github.com/myname/myproject/stuff
Files of package stuff should be located inside project's stuff directory. You name these files as you like.
Also it's possible to create deeper project structure. For instance, you decided to separate source code files from other ones (like app configs, docker files, static files, etc...). Let's move stuff directory inside pkg, every go file inside pkg/stuff still have stuff package name. To import stuff package just write:
import myprojectname/pkg/stuff
Nothing stops you from creating more levels in the hierarchy like github.com/myuser/myproject/pkg/db/provider/postgresql, where:
github.com/myuser/myproject - project name.
postgresql - package name.
pkg/db/provider/postgresql - path to the package relative to project's root.
You can read more about go modules here: https://github.com/golang/go/wiki/Modules
Check out this repository to get useful information about various patterns used in project organizing: https://github.com/golang-standards/project-layout If you go inside pkg directory you will find out which open source projects use pkg directory in their structure.
This sounds stupid, but I am trying for build my new golang project for a while now and I am stuck with following error
can't load package: package github.com/kuskmen/yamq/cmd/yamq-client: found packages main (main.go) and yamqclient (yamq-client.go) in C:\projects\yamq\cmd\yamq-client
I know this should be straightforward to fix, but I come from .NET and I am still not experienced in Go projects and its dependency resolution model hence the struggle.
My project structure looks like so
/yamq
/cmd
/yamq-client // yamq client application binary
main.go // package main
yamq-client.go // package yamqclient
/yamq-server // yamq server application binary
main.go // package main
yamq-server.go // package yamqserver
go.mod // contains only "module github.com/kuskmen/yamq" for now
... // some library files that will probably be moved to /shared folder
so far so good, when I do go build in outermost directory ( /yamq ) it is building successfully (or at least it is not showing any errors), but when I try to build either yamq-client or yamq-server binaries I get the aforementioned error and every time I try to google it or find something useful I got some old article or answer that dates back 2013-2016 that suggests something about $GOPATH and etc which shouldn't be the case here since I am trying to use go modules.
Help a fellow .NET developer join Go community by explaining him how exactly modules work cause I found this and this useless or at least I am missing the point, thanks in advance!
To follow up from my comment above:
From https://golang.org/doc/code.html:
Go programmers typically keep all their Go code in a single workspace.
A workspace contains many version control repositories (managed by Git, for example).
Each repository contains one or more packages.
Each package consists of one or more Go source files in a single directory.
The path to a package's directory determines its import path.
For your project, I'd do something like this:
$ tree
.
├── clientlib
│ └── lib.go
├── cmd
│ ├── client
│ │ └── main.go
│ └── server
│ └── main.go
├── go.mod
└── serverlib
└── lib.go
5 directories, 5 files
$ cat go.mod
module myproject.com
The module name is arbitrary (could be github.com/yourname/yourproject).
For the server side:
$ cat serverlib/lib.go
package serverlib
import "fmt"
func Hello() {
fmt.Println("Hello from serverlib.Hello")
}
$ cat cmd/server/main.go
package main
import (
"fmt"
"myproject.com/serverlib"
)
func main() {
fmt.Println("Running server")
serverlib.Hello()
}
And now we can build and run it:
$ go build -o server cmd/server/main.go
$ ./server
Running server
Hello from serverlib.Hello
The client side looks symmetrical.
Variations: you could name the .go files in cmd/... by their actual binary names - like server.go and client.go. The package in each is still main, but then go build creates an executable with the file's name (sans the .go) without needing to -o explicitly.
I'd like to use a helper package from Go Cloud Function. The package has some helper logic that can be shared between multiple functions. But, what is the right way to structure the packages so they all work? The package should be in the same project - not published and public as a completely separate package.
I work at Google. The goal for this question is to proactively answer common questions and help developers starting off with Go on GCF.
You can use subpackages with Go modules. Go modules are Go's new dependency management solution - they let you work outside of GOPATH and let you manage the exact versions of each dependency you have.
Modules also let you define a group of Go packages with the same import path prefix. When you're writing a function, this lets you import other packages in your module.
The function you're deploying needs to be at the root of your module.
Here is an example file structure and how packages would be imported:
.
├── cmd
│ └── main.go # Useful for testing. Can import and setup your function.
├── function.go # Can import example.com/foo/helperpackage
├── function_test.go
├── go.mod # module example.com/foo
└── helperpackage
└── helper.go
This setup has your function in function.go and tested by function_test.go. They are in a module named example.com/foo. helperpackage can be imported by function.go using example.com/foo/helperpackage.
This also has a cmd directory, which may be helpful for local testing. You can import example.com/foo and start an HTTP server which registers your function as an HTTP handler. For example:
package main
import (
"log"
"net/http"
"example.com/foo"
)
func main() {
http.Handle("/HelloHTTP", foo.HelloHTTP)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Note: You could use a vendor directory to achieve the same result. But, all of the packages your function imports would need to be in the vendor directory (with the full import path), which works, but is cumbersome to maintain. It's uncommon to copy sub-packages into your vendor directory, so I wouldn't recommend this.
We also ran into this issue. The go.mod approach provided by #TylerBui-Palsulich (https://stackoverflow.com/a/54255486/2307346) didn't work as we also had to download dependencies from a private repository.
From the Google documentation:
If your function's dependencies are hosted in a repository that is not publicly accessible, you must use a vendor directory to fetch your dependencies before deploying your function
(https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)
Given the following package structure:
cloudfunctions.go
sharedpackage
-> shared.go
go.mod
In our go.mod file we defined the following namespace: module some/namespace
The cloudfunctions.go file has the following package definition and imports
package foobar
import (
"bitbucket.org/some/private/dependency"
"some/namespace/sharedpackage"
)
Because of the private dependency we cannot use the go.mod file. Instead we provide a vendor directory.
Warning: If you have both a go.mod file and a vendor directory at the root of your project, the contents of the vendor directory will be ignored when your function is built in the cloud. To ensure that your vendor directory is used, you must exclude the go.mod file from your project's source code prior to deployment. If you are using the gcloud command-line tool, you can ensure that go.mod is not uploaded by using .gcloudignore.
(https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)
gcloud deploy will ignore the vendor directory if it finds go.mod, we fix this with a .gloudignore file with the following content:
go.mod
go.sum
At the end we have the following file structure:
cloudfunctions.go
sharedpackage
-> shared.go
go.mod
vendor
.gcloudignore
The last step, Fixing the cannot find package error
When running the deploy step you will still run into an error similar to:
(gcloud.functions.deploy) OperationError: code=3, message=Build failed: /tmp/sgb/gopath/src/serverlessapp/vendor/some/namespace/cloud_functions.go:5:2: cannot find package "some/namespace/sharedpackage" in any of:...
This is because the go.mod file, containing the module name, is ignored. now the Go compiler no longer knows that some/namespace/sharedpackage refers to the local sharedpackage directory.
We managed to get it working by changing the module name to match the package name:
Change module name in go.mod to module foobar
Change the imports in cloudfunctions.go to: "foobar/sharedpackage":
package foobar
import (
"bitbucket.org/some/private/dependency"
"foobar/sharedpackage"
)
Now the Go compiler is able to detect that foobar/sharedpackage is a subpackage of the foobar package.
I have a go package located on my filesystem (not in the $GOPATH), called bitbucket.org/me/awesome.
~/awesome> tree
.
├── main.go
├── go.mod
├── go.sum
├── subpackageA
│ └── main.go
My go.mod looks like:
module bitbucket.org/me/awesome
require (
... # lots of external dependencies
)
replace bitbucket.org/me/awesome => ./
In main.go in my top-level directory, I call a subpackage like follows:
import "bitbucket.org/me/awesome/subpackageA"
which all seems pretty normal. go get works. However, when I clone this entire repository somewhere else (say in a Docker image) and run go get for the first time, I get errors like:
package bitbucket.org/me/awesome/subpackageA: https://api.bitbucket.org/2.0/repositories/me/awesome?fields=scm: 403 Forbidden,
which means it's not using the local filesystem version of the packages, even though I told it to with the replace directive in the go.mod file.
What am I doing wrong? How do I ensure that subpackages are used from the filesystem instead of attempting to be fetched from the internet?
Go has no (real) notion of "subpackage". All packages are basically treated equal. This means that a replace bitbucket.org/me/awesome does not influence package bitbucket.org/me/awesome/subpackageA as these are two individual, unrelated packages. The folder layout does not introduce a relation of subpackageA to awsome, or the other way around *).
So you need to add an individual replace directive for subpackageA
replace bitbucket.org/me/awesome/subpackageA => ./subpackageA
*) Nitpicking for absolute correctness: Folder layout does have influence for folders named internal (cannot be imported from other projects), for folders named vendor (which may contain vendored packages) and searching for a go.mod file stops at the repo root.
For another approach, you can have go.mod like this:
module awesome
Then call subpackage like this:
import "awesome/subpackageA"
https://golang.org/doc/code.html
I have a project that provides a library (exports some funcs) and also must provide a command-line interface (there must be an executable file).
Example of directory structure:
whatever.io/
myproject/
main.go
myproject.go
The go compiler needs the package main and func main to start execution. My library needs the package myproject where I put stuff on it. This is what the go tool says when I am building another project that tries to import myproject:
main.go:5:2: found packages myproject (myproject.go) and main (main.go) in $GOPATH/src/whatever.io/myproject
So I believe there is no way to do it.
Should I move the library or the CLI to another package?
Just move your packages inside a new folder within the same directory of main.go.
Remember to import the new package from the reference of the $GOPATH.
Example:
user#user:~/p/go/test/so-multipack$ ls -R
.:
a main.go
./a:
a.go
user#user:~/p/go/test/so-multipack$ cat main.go
package main
import (
"../so-multipack/a"
)
func main(){
a.Hello()
}
user#user:~/p/go/test/so-multipack$ cat a/a.go
package a
import (
"fmt"
)
func Hello(){
fmt.Println("hello from a")
}
user#user:~/p/go/test/so-multipack$ go run main.go
hello from a
user#user:~/p/go/test/so-multipack$ go build
user#user:~/p/go/test/so-multipack$ ls
a main.go so-multipack
user#user:~/p/go/test/so-multipack$
Useful link:
go build vs go build file.go
You cannot have two packages per directory, hence the error. So the solution as #Larry Battle said to move your myproject.go to a new directory.
From How to write go code
Go code must be kept inside a workspace. A workspace is a directory
hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
In most cases, no. However, there is an exception for unit tests.
Working Example:
Here are 2 different packages (mypackage and mypackage_test) in 1 directory (mypackage). The compiler will not complain about this.
mypackage folder:
mypackage/
foo.go
foo_test.go
mypackage/foo.go:
package mypackage
func Add(a int, b int) int {
return a + b
}
mypackage/foo_test.go:
package mypackage_test
// Unit tests...
Rules:
The 2 packages must have the following names:
NameOfDirectory.
NameOfDirectory + _test.
The names of the files in the _test package must end with _test.go
If you're receiving a confusing compiler error along the lines of found packages "foo" and "bar", you've probably broken one or more of these rules.
You can't have two golang files in one directory with two packages. So you need to move main.go out of myproject.
the directory structure before move
whatever.io/
go.mod
myproject/
main.go
myproject.go
After move
whatever.io/
go.mod
main.go
myproject/
myproject.go
And you need to change your main.go's import path. If the module name is aaa
Before
import "aaa"
Need change to this
import "aaa/myproject"