Golang project directory structure - go

I have some confusion regarding Golang directory structure.
Base on the book The Way to Go by Ivo Balbaert, project code should be placed into src, and recommends the following directory structure.
├──src/
| ├──main.go
| ├──say/
| | ├──say.go
| | ├──say_test.go
├──bin/
| ├──say
└──pkg/
└──linux_amd64/
└──say.a
but I found that many packages in github.com, have no src directory.
For example:
https://github.com/facebookgo/grace
https://github.com/astaxie/beego
So, I don't know whether src directory is needed.
I have some project, their have inter-dependency.
They are managed in a private GitLab repository.
How can I organized them?

This article by Ben Johnson has guided me on this when I was starting with Go.
It's generally good to start with something like this (assuming you are inside your project directory like $GOPATH/src/myproject:
├──cmd/ -- this is where you compose several packages in to main package
| ├──foo -- an example would be `foo`
| | ├──main.go
├──pkg/ -- this is where put your reusable packages
| ├──pkg1 -- reusable package 1
| ├──pkg2 -- reusable package 2
├──otherpackage1
| ├── ...
├──otherpackage2
| ├── ...
You can have a look at this example from go-kit for this kind of project structure.
Sometimes it will depend on your needs. On our workflow, we are using a hot code reload tool called fresh, so we needed to put the main.go on the project root so that the tool can detect all the file changes and rebuild the source code.
├──app/
| ├──app.go
├──model/ --
| ├──model.go
├──store
| ├──store.go
├──main.go -- this is where the app starts
├──...
On the app.go package, I have something like func Run() error which starts the application. On the main.go, I am just calling the function:
...
func main(){
log.Fatal(app.Run())
}

Now there are new ways to organize Go projects.
The GitHub golang-standards repository says:
This is a basic layout for Go application projects. It represents the
most common directory structure with a number of small enhancements
along with several supporting directories common to any real world
application.
This project layout is intentionally generic and it doesn't try to
impose a specific Go package structure.
Or you can follow this slides:
$GOPATH/
src/
github.com/user/repo/
mypkg/
mysrc1.go
mysrc2.go
cmd/mycmd/
main.go
bin/
mycmd

Here is another project layout sample Simple Go project layout with modules
├── LICENSE
├── README.md
├── config.go
├── go.mod
├── go.sum
├── clientlib
│ ├── lib.go
│ └── lib_test.go
├── cmd
│ ├── modlib-client
│ │ └── main.go
│ └── modlib-server
│ └── main.go
├── internal
│ └── auth
│ ├── auth.go
│ └── auth_test.go
└── serverlib
└── lib.go
Some answer points out the go standard layout, however, there is one issue this is not a standard Go project layout by Russ Cox
There are two problems with this GitHub repo:
it claims to host Go standards and does not, in the sense that these are in no way official standards
the project-layout standard it puts forth is far too complex and not a standard
Regarding "why not tell us the standard Go project layout and we'll update the doc?", that only addresses point 2. If there really were standards, they would be in the main Go project doc tree. The standard for project layout would also be a lot shorter. I appreciate your trying to provide a useful resource, but calling it 'golang-standards' is claiming more than it is.
But for the record, the minimal standard layout for an importable Go repo is really:
Put a LICENSE file in your root
Put a go.mod file in your root
Put Go code in your repo, in the root or organized into a directory tree as you see fit
That's it. That's the "standard".
Update at 11/30/2021
Here are a summary from How to structure Go code
Before we start
doc.go file puts the general description of the package
Readme file a general overview of this project
When you have more documentation to provide put them into the docs folder
For linting use golangci-lint. Enable all linters that seem to be reasonable for your project
The flat structure (single package)
courses/
main.go
server.go
user_profile.go
lesson.go
course.go
When to create a new package?
When you have more than one way of starting your application
When you want to extract more detailed implementation
When you started to add a common prefix to closely related things
Modularisation
Organising by kind
.
├── handlers
│ ├── course.go
│ ├── lecture.go
│ ├── profile.go
│ └── user.go
├── main.go
├── models
│ ├── course.go
│ ├── lecture.go
│ └── user.go
├── repositories
│ ├── course.go
│ ├── lecture.go
│ └── user.go
├── services
│ ├── course.go
│ └── user.go
└── utils
└── stings.go
Organising by components
.
├── course
│ ├── httphandler.go
│ ├── model.go
│ ├── repository.go
│ └── service.go
├── main.go
└── profile
├── httphandler.go
├── model.go
├── repository.go
└── service.go
Clean Architecture
You have 4 layers of your application or module (depending on how big your codebase is): Domain, Application, Ports, Adapters. In some sources, names may differ.

The src directory is not needed and in fact a lot of public repositories do not use this structure.
There is a few different way of organizing your project. If you plan on having your project used by an other repository, like lib. I would recommend using a cmd struct something like this. This would also be the recommended way of doing it if there would be more then one way of starting the application. (multipliable main.go files)
├──cmd/
| ├──(application name)
| | ├──main.go
└──say/
├──say.go
└──say_test.go
Otherwise for example if it is a standalone application. You can place the main.go in the root of the repository.
bin and pkg you can keep in the root and add this to .gitignore. (assuming you are using git)

The book describes the directory structure after checkout. It would have been helpful if the book included the .git directory.
$GOPATH/src is required for imports to work.
├──src/
| ├──.git
| | ├──...
| ├──main.go
| ├──say/
| | ├──say.go
| | ├──say_test.go
├──bin/
| ├──say
└──pkg/
└──linux_amd64/
└──say.a
In practice, main.go would actually be in a path that reflects the remote git repository, for instance
.
├── bin
│   └── say
├── pkg
│   └── linux_amd64
│   └── github.com
│      └── pschultz
│      └── hello-world
│   └── say.a
└── src
   └── github.com
   └── pschultz
   └── hello-world
   ├── .git
   │ └── ...
   ├── main.go
   └── say
   ├── say.go
   └── say_test.go

Related

golang package local does not find

I can not import a package of mine.
I have already used go mod init but it does not work. Content of go.mod:
module github.com/grafeno30/golang_redes/rat
go 1.13
My project has the following structure:
github.com/grafeno30/golang_redes/
├── rat
│   ├── client
│   │   └── main.go
│   ├── server
│   │   ├── core
│   │   │   └── handleConnection
│   │   │   └── establishedConnection.go
│   │   ├── main.go
│   │   └── server
│   ├── go.mod
│   └── go.sum
└── README.md
When I execute go build . in the folder "github.com/grafeno30/golang_redes/rat/server" I get the following error:
# github.com/grafeno30/golang_redes/rat/server
./main.go:14:13: undefined: handleConnection
In summary, I use a function of the module rat/server/core/handleConnection in the main go file rat/server/main.go
Thanks in advance
I finally figured it out
I have had to use: go mod, go mod init, go mod edit -replace and go mod tidy.
Basically I have made a replacement "go mod edit -replace github.com/grafeno30/golang_redes/rat/server/core/handler=./core/handler"
Thank you all!!!

Using go modules with a typical directory structure, can't seem to get it working

So I created a simple test project to demonstrate the issue I'm having in a much larger code base here https://github.com/cleblanc189/test.go
.
├── go.mod
├── main.exe
├── main.go
├── server
│   ├── v1beta1
│   │   ├── foo.go
│   │   └── go.mod
│   └── v1ga
│   ├── foo.go
│   └── go.mod
└── service
├── v1beta1
│   ├── foo.go
│   └── go.mod
└── v1ga
├── foo.go
└── go.mod
I'd like to be able to reference service/v1beta1 from server/v1beta1 but I'm getting this error
go: test.go/src/server/v1beta1#v0.0.0 requires
test.go/src/service/v1beta1#v0.0.0: unrecognized import path "test.go/src/service/v1beta1" (https fetch: Get https://test.go/src/ser
vice/v1beta1?go-get=1: dial tcp: lookup test.go: no such host)
In foo.go from server/v1beta1 I import like this
svc "test.go/src/service/v1beta1"
and have a go.mod file like below;
require test.go/src/service/v1beta1 v0.0.0
replace test.go/src/service/v1beta1 => ../../service/v1beta1
When I try to cd src && go mod tidy I get that error above. Also if anyone knows should I even have a go.mod above the src directory?
Appreciate any help anyone can offer. This is maddening.
Your go.mod has to declare the module path/name, typically like this:
module test.go
Then all the imports in your code will be taken relative to that. When you import something relative to your declared module name, the go tool will automatically pick up the code from the right place locally.
Not sure where your src comes into play though... is everything in folder src? See this post and accompanying code for a complete example you could start from.

Kubernetes client code generator: Can the code exist only locally and not on a repository for the core-generator to work?

I am trying to generate client code using k8s.io/code-generator.
These are the instructions that I am following: https://itnext.io/how-to-generate-client-codes-for-kubernetes-custom-resource-definitions-crd-b4b9907769ba
My question is, does my go module need to be present on a repository or can I simply run the generate-groups.sh script on a go module that is ONLY present on my local system and not on any repository?
I have already tried running it and from what I understand, there needs to be a repository having ALL the contents of my local go module. Is my understanding correct?
You CAN run kubernetes/code-generator's generate-groups.sh on a go module that is only present on your local system. Neither code-generator nor your module needs to be in your GOPATH.
Verification
Cloned kubernetes/code-generator into a new directory.
$HOME/somedir
├── code-generator
Created a project called myrepo and mocked it with content to resemble sample-controller. Did this in the same directory to keep it simple.
somedir
├── code-generator
└── myorg.com
└── myrepo # mock of sample-controller
├── go.mod
├── go.sum
└── pkg
└── apis
└── myorg
├── register.go
└── v1alpha1
├── doc.go
├── register.go
└── types.go
My go.mod looked like
module myorg.com/myrepo
go 1.14
require k8s.io/apimachinery v0.17.4
Ran generate-group.sh. The -h flag specifies which header file to use. The -o flag specifies the output base which is necessary here because we're not in GOPATH.
$HOME/somedir/code-generator/generate-groups.sh all myorg.com/myrepo/pkg/client myorg.com/myrepo/pkg/apis "myorg:v1alpha1" \
-h $HOME/somedir/code-generator/hack/boilerplate.go.txt \
-o $HOME/somedir
Confirmed code generated in correct locations
myrepo
├── go.mod
├── go.sum
└── pkg
├── apis
│   └── myorg
│   ├── register.go
│   └── v1alpha1
│   ├── doc.go
│   ├── register.go
│   ├── types.go
│   └── zz_generated.deepcopy.go
└── client
├── clientset
│   └── versioned
│   ├── clientset.go
│   ├── doc.go
│   ├── fake
│   ├── scheme
│   └── typed
├── informers
│   └── externalversions
│   ├── factory.go
│   ├── generic.go
│   ├── internalinterfaces
│   └── myorg
└── listers
└── myorg
└── v1alpha1
Sources
Go modules support https://github.com/kubernetes/code-generator/issues/57
Documentation or support for Go modules https://github.com/kubernetes/sample-controller/issues/47

Problem with modules and importing folders

I have some problems with how Go importing and modules works.
I have a project with this structure:
My-Project
|- Program 1
|- main.go
|- go.mod
|- Program 2
|- main.go
|- go.mod
|- Support
|- go_file.go
The two programs have different module "definition". I need to include the "Support" folder, which is shared between the two programs. I tried
import "My-Project/Support"
inside Program1's main.go, but this is not working, because Support can’t be resolved. How can I do it?
As given, your Support/go_file.go is not in any module so it's not importable using Go modules.
Either use a single module for all three packages or use three separate modules (i.e. put Support in it's own module). You could also use sub-modules.
Single module:
For a single module you'd do something like:
└── My-Project
├── go.mod
├── Program\ 1
│   └── main.go
├── Program\ 2
│   └── main.go
└── Support
└── go_file.go
And the project level go.mod would be something like:
module github.com/My-Name/My-Project
(substitute with appropriate module name) and then both main packages would import the support package with:
import "github.com/My-Name/My-Project/Support"
Note Go packages normally do not start with capitals like this.
More idiomatic for such a setup with a single repository which mixes packages and commands is a layout that puts commands into a cmd subdirectory (e.g. see the golang.org/x/tools layout). This would look something like this:
└── My-Project
├── cmd
│   ├── program1
│   │   └── main.go
│   └── program2
│   └── main.go
├── go.mod
├── internal
│   └── support
│   └── support.go
└── other-package
└── some-other-package.go
Here the support package is in an internal sub-directory which makes it non-importable from outside My-Project where-as the other-package is importable by anyone.
Again, go.mod would define the module name which sets the prefix to be used for all import statements.
Separate modules:
To use separate modules you'd do something like:
└── My-Project
├── Program\ 1
│   ├── go.mod
│   └── main.go
├── Program\ 2
│   ├── go.mod
│   └── main.go
└── Support
├── go_file.go
└── go.mod
Here My-Project/Program 1/go.mod would be something like:
module "github.com/My-Name/My-Project/Program 1"
require (
github.com/My-Name/My-Project/Support
)
The imports would be the same as for the single module example.
Similarly for Program 2. For Support/go.mod it'd look something like:
module github.com/My-Name/My-Project/Support
For local development you'll probably also want/need a replace directive in each program's go.mod to use your local working copy of the support module (go mod edit -replace github.com/My-Name/My-Project/Support=../Support should add this).

Configure Intellij Idea to import generated files

I'm having trouble to import a generated package with Intellij, however without any changes to the default settings it works on Eclipse.
Here is my architechture:
├── build.properties
├── pom.xml
├── README.md
├── src
│   ├── main
│   │   ├── java
│   │      └── mypackage
│   └── test
└── target
├── classes
│ └── mypackage
| └── generated
├── generated-sources
├── generated-test-sources
├── APP.jar
└── test-classes
I have most of my classes in com.mypackage however some of them are generated in
└── target
├── classes
└── mypackage
named as com.mypackage.generated and i have to use these classes in com.mypackage:
├── src
├── main
├── java
└── mypackage
However intellij cannot resolve symbol generated when I'm doing
import com.mypackage.generated
I tried to make it work by looking at the project structure/modules/dependencies menu but it seems to be for external modules. How can I do this ?
Actually it was very simple, I only was about marking classes as sources root. I don't know why marking generated as sources root did not work.

Resources