I have a pretty simple Go project that I'm trying to restructure so that it follows the normal Go project structure (and so I can run go build).
I currently have two source files, both with package main. All the files, including a few text configuration files that my program needs at runtime.
So right now, it looks like:
<project_name>
- main.go
- source2.go
- config_file.txt
I can run go build when I'm in this directory, and it creates one binary (named <project_name>. This works fine, but I'd like to set this up to better follow the Go standard package structure (specifically so that Intellij IDEA will recognize it as a valid project).
Right now, I have the entire <project_name> directory in Git, and I'd like to keep it that way.
I tried putting the source files in a folder called src, but then go build says there aren't any source files to compile.
How should I structure this?
EDIT:
Figured out the issue with putting stuff in src/: I need to run go build <project_name>.
I'm still wondering if there's a way to set up a project without a global GOPATH. I have all my projects under one folder, with a subfolder for each project (not all the projects are Go project). I'd like to keep that system.
What I want is:
projects/
- project 1/
- src/
- bin/
- pkg/
- project 2/
- src/
- bin/
- pkg/
Then I'd like to be able to run go build <project_name> (while I'm in that project's directory) and have it compile that project. Is that possible?
The "canonical" way to organize your Go code is described in How to Write Go Code. It is explained in a less formal way in this blog post. This concept is kind of contrary to what you have in mind - you have a workspace directory, specified by the GOPATH environment variable, and the source code of all projects resides in subdirectories of the "src" directory of the workspace. You can have multiple workspaces if you specify several directories in GOPATH.
I would suggest you give the "recommended" way to organize your code a chance, maybe it will grow on you. It may seem a bit unusual, but it has its advantages. If you find out you absolutely can't live with it, you can still work around it, e.g. by setting GOPATH dinamically in a script.
As for the Golang IDEA plugin, the last version I tried some time ago didn't yet support this project structure, but newer versions may have changed that. In fact, one of the plugin's authors (dlsniper) has added a comment to the above blog post giving examples of alternative project structures that still use a global GOPATH.
Related
I have an existing project, and I am attempting to add a new Go-based component to it. However, I am having trouble figuring out a directory structure that is amenable to Go and also fits in with the existing structure.
Here is my current project structure (simplified):
content
└── (some other stuff is in here)
db
└── (some other stuff is in here)
Makefile
server
├── go.mod
├── go.sum
├── options.go
├── routes.go
└── server.go
All components of the project are in subdirectories from the main one, and each component has a target in the Makefile to build it. So in this case, I would like to be able to run make serve and have Go build, and then run, the server. (Note that I am not trying to use Make as a Go build system — it is merely a task runner. When you have many projects, having a common language-independent set of commands such as make serve is useful).
Here are the things I have tried so far:
Keeping the Go files immediately inside the server directory (so server/server.go, server/routes.go, etc, as is pictured above), and having the Make task run cd server; go build .. This successfully builds and runs the server. However, any compilation errors will start with ./server.go:, rather than server/server.go:, so I lose the ability to control-click on a filename in my terminal to open the offending line in my editor. I'm accustomed to writing code like this and I'd really like to keep doing it.
Keping the Go files inside the server directory (as above), and having the Make task run go build server. This gives me the error package server is not in GOROOT. Which, to be fair, it isn't, but I don't think putting it in GOROOT is the right thing to do.
Avoiding Go modules entirely, setting GOPATH to $(pwd)/server, putting the Go files in server/src/my-server, and running go build. This gives me the error go.mod file not found in current directory or any parent directory, which I thought was strange, as I was trying to opt out of modules. (I learnt Go after modules were added and it's hard to find documentation on how do things without them; I'm still a little confused by this.)
As above, putting the Go files in server/src/myserver, setting GOROOT to $(pwd)/server. This gives me all kinds of errors such as package errors is not in GOROOT. Again, to be fair, it's not, but I have no idea how to solve that issue.
I cannot do the thing that Go seems to want me to do, which is to have the entirety of my Go code all inside one directory — this is a Go component of an existing project, not a project completely written in Go. Nor do I want to have to go into the server directory in order to do anything Go-related, because none of the other components in any other project have that requirement.
Is there any way for me to get the hierarchy I want with Go?
coming from a java and grails background, (and having written millions of lines of C 30 years ago), I cant see how go can be usable with a fixed gopath on windows.
installing go creates this structure
c:\users\me\go\scr
\pkg
\bin
As you will want to have many go projects it seems they have to be mixed together in the same src/kpg/bin dirs, polluting each other. e.g.
/src/project1/hello.go
/project2/hello.go
/pkg/xx
/bin/hello.exe
Which hello.go will hello.exe run?
Unless I am missing something fundamental, this seems crazy - all completely separate projects are expected to share the same package and bin dirs. It means you dont know which versions of which packages, and which exe files belong to which project, and there is presumably plenty of scope for conflicts. I would have expected a /src, /pkg and /bin for each separate go app (like java or grails or even C), and go path is completely redundant, it can be relative to the current project root (like in grails).
To make matters works, for work, we have to use a different directry, e.g.
d:\work\project3
\project4
\package5
\go_utility6
\go_utility7
So now we have a total of 6 separate directories where go progams live. It is not feasible to change the path every time you switch to working on a different project. I guess the other option is to add the 6 paths to the GOPATH. Presumably, all 7 go projects write to the same pkg and bin dir, which is going to be a disaster.
Is there a tenable solution to this situation (on windows at least)?
If we need to add a PATH to GOPATH for every project, what should the file structure be under each project?
E.g. uner xxx\go_utility6, which is a stand alone command line go app, what should the structure be? does there need to be a src dir in there somewhere? does this dir need gopath to point to it? does it need its own pkg, or should it use the c:\users\me\pkg dir?
UPDATE: When I posted this Go did not have modules support and we built and used a tool called vg. These days the recommended way to go is to use go modules.
I use vg for that, it takes care of keeping separate GOPATH paths per project and it switches automatically when you cd a project.
Your example "which hello.exe" should be used honestly makes not much sense. Two tools with the same name?
Even if both are, let's say, an api, your devops will be happier with more meaningful names.
The bin folder is used for 3rd party tools you install, you so not have to install your project binaries. Except they are tools, but then the name should be meaningful again.
You can get more information about the project structure here: https://golang.org/doc/code.html
Since go 1.8 supports a vendor folder below project folders, it is possible to break the original structure. (imho vendors were not maintainable before 1.8, yes that was crazy)
You might want to use a tool like direnv, which would support your desire to change GOPATH per project.
https://github.com/direnv/direnv
It also has some built in function for adding the current path to the GOPATH.
https://github.com/direnv/direnv/blob/master/stdlib.sh#L355:1
For example GoLang also supports handling multiple GOPATHs and per project GOPATHs. So direnv should also work properly.
In my company we have one go folder right next to our other projects.
Under go/src are our projects. No problem so far, since vendors are in the projects' vendor folders and committed.
The so far best dependency manager I would recommend for go is:
https://github.com/golang/dep
I hope that input helps.
With Go 1.11 Go Modules were introduced. You can use Go Modules to have Go projects outside the GOPATH directory.
Here is an example of how to configure a project using GoModules.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have a go project that is starting to become more complex, and want to lay the filesystem out in such a way to reduce pain.
Are there some good examples out there of what makes sense?
Update May 2013: the official documentation is in the section "Code organization"
Go code must be kept inside a workspace.
A workspace is a directory hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
The go tool builds source packages and installs the resulting binaries to the pkg and bin directories.
The src subdirectory typically contains multiple version control repositories (such as for Git or Mercurial) that track the development of one or more source packages.
bin/
streak # command executable
todo # command executable
pkg/
linux_amd64/
code.google.com/p/goauth2/
oauth.a # package object
github.com/nf/todo/
task.a # package object
src/
code.google.com/p/goauth2/
.hg/ # mercurial repository metadata
oauth/
oauth.go # package source
oauth_test.go # test source
Update July 2014: see "Structuring Applications in Go" from Ben Johnson
That article include tips like:
Separate your binary from your application
combining the main.go file and my application logic in the same package has two consequences:
It makes my application unusable as a library.
I can only have one application binary.
The best way I’ve found to fix this is to simply use a “cmd” directory in my project where each of its subdirectories is an application binary.
camlistore/
cmd/
camget/
main.go
cammount/
main.go
camput/
main.go
camtool/
main.go
Library driven development
Moving the main.go file out of your root allows you to build your application from the perspective of a library. Your application binary is simply a client of your application’s library.
Sometimes you might want users to interact in multiple ways so you create multiple binaries.
For example, if you had an “adder” package that that let users add numbers together, you may want to release a command line version as well as a web version.
You can easily do this by organizing your project like this:
adder/
adder.go
cmd/
adder/
main.go
adder-server/
main.go
Users can install your “adder” application binaries with “go get” using an ellipsis:
$ go get github.com/benbjohnson/adder/...
And voila, your user has “adder” and “adder-server” installed!
Don’t go crazy with subpackages
Usually my project’s types are all very related so it fits better from a usability and API standpoint.
These types can also take advantage of calling unexported between them which keeps the API small and clear.
Group related types and code together in each file. If your types and functions are well organized then I find that files tend to be between 200 and 500 SLOC. This might sound like a lot but I find it easy to navigate. 1000 SLOC is usually my upper limit for a single file.
Organize the most important type at the top of the file and add types in decreasing importance towards the bottom of the file.
Once your application starts getting above 10,000 SLOC you should seriously evaluate whether it can be broken into smaller projects.
Note: that last practice isn't always good:
Sorry I just cant agree with this practice.
Separating type to files helps code management, readability, maintenancability, testability.
It may also ensure single responsibility and the follow of open/closed principle…
The rule for not allowing circular dependency is to force we have a clear structure of the packages.
(Alternative February 2013, regarding src only)
You can find the classic layout illustrated in "GitHub Code Layout":
The app and both libraries live on Github, each in its own repository.
$GOPATH is the root of the project - each of your Github repos will be checked out several folders below $GOPATH.
Your code layout would look like this:
$GOPATH/
src/
github.com/
jmcvetta/
useless/
.git/
useless.go
useless_test.go
README.md
uselessd/
.git/
uselessd.go
uselessd_test.go
README.md
Each folder under src/github.com/jmcvetta/ is the root of a separate git checkout.
That attracted some criticisms though, in this reddit page:
I highly recommend not structuring the repo the way you have, it'll break "go get", which is one of the most useful things about Go.
It's far better to write your code for people who do know Go, since they're most likely to be the ones compiling it.
And for people who don't, they'll at least get a feel for the language.
Put the main package in the root of the repo.
Have the assets in a subdirectory (to keep things neat).
Keep the meat of the code in a subpackage (in case anyone wants to reuse it outside your binary).
Include a setup script in the root of the repo so it's easy to find.
It's still only a two step process to download, build, install, and setup.:
"go get <your repo path>": downloads and installs the go code, with a subdir for the assets
$GOPATH/<your repo path>/setup.sh: distributes the assets to the right place and installs the service
I assume that with 'project' you don't mean a Go package but a software you develop.
Otherwise you can get help here and here.
However it's not so much different to writing packages for Go: Use packages, create a folder for each package and combine those packages in your application.
To build yourself an opinion, you can look at trending Go repositories on github: https://github.com/trending/go. Notable examples are cayley
and zeus.
The most popular scheme is probably to have a main Go file and many modules and submodules in their own directories. In case you have many meta files (doc, license, templates, ...) you may want to put the source code into a subdirectory. That's what I did so far.
There is a recommended approach from the authors of Golang that defines how to layout your code to work best with the go tools and to support source control systems
You should probably also have a look into this repo. It shows a lot of ideas how to structure go applications: https://github.com/golang-standards/project-layout
I'm a beginner in golang, and I was reading the article How to write Go code on the go website. It explains how the typical Go structure contains three folders in the root of the project:
bin/
contains compiled code
pkg/
contains package objects
src/
contains the Go source files
So to learn from other projects I checked out some popular go projects in github, but to my surprise I don't see this src/pkg/bin structure in any of those projects.
What am I missing here? Does anybody know a (preferably simple) project in golang which follows this structure? I think I could learn a lot from reading other people's code.
The structure you reference in your question is not how projects are structured but rather how the local workspace is configured.
https://golang.org/doc/code.html#Workspaces
Typically, you'll set up one workspace in your local machine, for example you would set up your workspace root to be:
$HOME/go
You then point your GOPATH environment variable to $HOME/go to let Go know that's your workspace location.
Inside your workspace you then create the src, bin and pkg folders that the documentation describes.
Inside src is where you checkout (or create your own) project folders.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have a go project that is starting to become more complex, and want to lay the filesystem out in such a way to reduce pain.
Are there some good examples out there of what makes sense?
Update May 2013: the official documentation is in the section "Code organization"
Go code must be kept inside a workspace.
A workspace is a directory hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
The go tool builds source packages and installs the resulting binaries to the pkg and bin directories.
The src subdirectory typically contains multiple version control repositories (such as for Git or Mercurial) that track the development of one or more source packages.
bin/
streak # command executable
todo # command executable
pkg/
linux_amd64/
code.google.com/p/goauth2/
oauth.a # package object
github.com/nf/todo/
task.a # package object
src/
code.google.com/p/goauth2/
.hg/ # mercurial repository metadata
oauth/
oauth.go # package source
oauth_test.go # test source
Update July 2014: see "Structuring Applications in Go" from Ben Johnson
That article include tips like:
Separate your binary from your application
combining the main.go file and my application logic in the same package has two consequences:
It makes my application unusable as a library.
I can only have one application binary.
The best way I’ve found to fix this is to simply use a “cmd” directory in my project where each of its subdirectories is an application binary.
camlistore/
cmd/
camget/
main.go
cammount/
main.go
camput/
main.go
camtool/
main.go
Library driven development
Moving the main.go file out of your root allows you to build your application from the perspective of a library. Your application binary is simply a client of your application’s library.
Sometimes you might want users to interact in multiple ways so you create multiple binaries.
For example, if you had an “adder” package that that let users add numbers together, you may want to release a command line version as well as a web version.
You can easily do this by organizing your project like this:
adder/
adder.go
cmd/
adder/
main.go
adder-server/
main.go
Users can install your “adder” application binaries with “go get” using an ellipsis:
$ go get github.com/benbjohnson/adder/...
And voila, your user has “adder” and “adder-server” installed!
Don’t go crazy with subpackages
Usually my project’s types are all very related so it fits better from a usability and API standpoint.
These types can also take advantage of calling unexported between them which keeps the API small and clear.
Group related types and code together in each file. If your types and functions are well organized then I find that files tend to be between 200 and 500 SLOC. This might sound like a lot but I find it easy to navigate. 1000 SLOC is usually my upper limit for a single file.
Organize the most important type at the top of the file and add types in decreasing importance towards the bottom of the file.
Once your application starts getting above 10,000 SLOC you should seriously evaluate whether it can be broken into smaller projects.
Note: that last practice isn't always good:
Sorry I just cant agree with this practice.
Separating type to files helps code management, readability, maintenancability, testability.
It may also ensure single responsibility and the follow of open/closed principle…
The rule for not allowing circular dependency is to force we have a clear structure of the packages.
(Alternative February 2013, regarding src only)
You can find the classic layout illustrated in "GitHub Code Layout":
The app and both libraries live on Github, each in its own repository.
$GOPATH is the root of the project - each of your Github repos will be checked out several folders below $GOPATH.
Your code layout would look like this:
$GOPATH/
src/
github.com/
jmcvetta/
useless/
.git/
useless.go
useless_test.go
README.md
uselessd/
.git/
uselessd.go
uselessd_test.go
README.md
Each folder under src/github.com/jmcvetta/ is the root of a separate git checkout.
That attracted some criticisms though, in this reddit page:
I highly recommend not structuring the repo the way you have, it'll break "go get", which is one of the most useful things about Go.
It's far better to write your code for people who do know Go, since they're most likely to be the ones compiling it.
And for people who don't, they'll at least get a feel for the language.
Put the main package in the root of the repo.
Have the assets in a subdirectory (to keep things neat).
Keep the meat of the code in a subpackage (in case anyone wants to reuse it outside your binary).
Include a setup script in the root of the repo so it's easy to find.
It's still only a two step process to download, build, install, and setup.:
"go get <your repo path>": downloads and installs the go code, with a subdir for the assets
$GOPATH/<your repo path>/setup.sh: distributes the assets to the right place and installs the service
I assume that with 'project' you don't mean a Go package but a software you develop.
Otherwise you can get help here and here.
However it's not so much different to writing packages for Go: Use packages, create a folder for each package and combine those packages in your application.
To build yourself an opinion, you can look at trending Go repositories on github: https://github.com/trending/go. Notable examples are cayley
and zeus.
The most popular scheme is probably to have a main Go file and many modules and submodules in their own directories. In case you have many meta files (doc, license, templates, ...) you may want to put the source code into a subdirectory. That's what I did so far.
There is a recommended approach from the authors of Golang that defines how to layout your code to work best with the go tools and to support source control systems
You should probably also have a look into this repo. It shows a lot of ideas how to structure go applications: https://github.com/golang-standards/project-layout