Break up go project into subfolders - go

I want to break my project up to subfolders.
I want this code structure:
├── main.go
└── models
└── user.go
Where main.go is:
package main
import (
"fmt"
"./models"
)
func main(){
fmt.Println(User{"new_user"})
}
And user.go is:
package models
type User struct {
Login string
}
But User is not defined in main package and import raise warning "imported and not used".
What am I doing wrong? My project is simple (not such a example but just with few files (controllers and models)) and I want a simple structure.
Maybe I doing it in completely wrong way?
Problem project is here: https://github.com/abonec/go_import_problem

I recently achieved this by using go modules.
Golang introduced preliminary opt-in support for modules as of go v1.11.1 which is intended to completely remove the, frankly, absurd $GOPATH necessity. Not only can you now have versioned dependencies in any normal directory such as ~/development, but you can basically have something that looks like namespaces and sub-directories. You can enable this feature by invoking the go command with the following environment variable: GO111MODULE=on.
Go v1.11.3 expects to enable modules by default and is slated for August 2019.
Here is an example directory structure (that you might find typically in some other languages).
~/Dev/my-app
├── src/
│ ├── one/
│ │ ├── two/
│ │ │ └── two.go
│ │ └── one.go
│ └── zero.go
├── go.mod
└── app.go
The application is called my-app, which will be the module name for app.go. We define this once in go.mod and then each of all the other go files in subdirectories will automatically be importable as if they were namespaced.
Given the above, two.go, assuming it contains a function named Two, will be importable in app.go by using my-app/src/one/two.
Here's what you need to do to achieve this:
go.mod
module my-app
two.go
package two
func Two() string {
return "I'm totally not supposed to be using go modules for this"
}
app.go
package main
import "my-app/src/one/two"
func main() {
two.Two()
}
If you were to place another file within two/, then you would simply use two.TheNewFunc() as long as you made TheNewFunc() available within the new file.
I created a very simple GitHub repo which you can check out as a demonstration.

Your import should be an absolute one:
import "github.com/abonec/go_import_problem/models"
If you don't want to export your project to an external referential, you can do a:
import "go_import_problem/models"
(That is: "the name of your project folder accessible by GOPATH/your package")
See "How to use custom packages in golang?".
And you would use:
models.User
As mentioned in Effective Go:
The importer of a package will use the name to refer to its contents, so exported names in the package can use that fact to avoid stutter.
(Don't use the import . notation, which can simplify tests that must run outside the package they are testing, but should otherwise be avoided.)
kostix adds in the comments:
to reiterate, names of Go packages are always absolute (that is, there's no relative package names, neither with ./ nor with ../ or anything like that) but that names are "anchored" to one of the so-called workspaces listed in $GOPATH.
When Go searches for a package, it looks through workspaces and tries to find a package in each of them, in order.
The search is not recursive.
And no, there's no requirement to encode URLs in package paths -- unless you want to make your package public.

You need to qualify items in in a package by its package name
So
fmt.Println(models.User{"new_user"})

Breaking up a single project into subfolders is not the recommended way of structuring a go project, which is why there is basically no good way to do what you want.
If the project is really large, and too unwieldy to make a single package, consider splitting it into several totally distinct packages, rather than special sub-directory packages. This has the advantage of forcing you to think cleanly about your internal APIs.

The packages are referenced in code in relation to your "go/src" folder
└── go
└── src
└── myAwesomeProject
├── main.go
└── models
└── user.go
So in main.go
package main
import (
"fmt"
"myAwesomeProject/models"
)
Similarly packages can reference each other using the same convention.

You should use your imported objects by it's imported names. For example if you
import "./models"
with struct User you should use it as
models.User

Related

Are there conventions for Go module names in mono-repos with multiple modules?

In a multi-module repository, should a module name (set via the go.mod module directive) follow the conventions of package naming?
E.g. module github.com/org-name/repo-name/path/to/module-dir
I understand that, whatever the module is named, the packages within the module refer to each other using the module name as prefix. But, from outside the module, there seems to problems if the module name is set to something other than the <host><path-within-repo> pattern. get-ing a package included in the module then gives messages about unrecognized import path.
Is there any cause to name a module differently than <host><path-within-repo> ?
There isn't any hard requirement for referencing modules, although it always good practice to use the domain/repo pattern. So, if you want to reference other modules locally that are not in the GOPATH, you can use the replace directive.
https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive
replace also can be used to inform the go tooling of the relative or absolute on->disk location of modules in a multi-module project, such as:
replace example.com/project/foo => ../foo
Let's say we have the following structure:
├── .gitignore
├── pkg1
│   ├── go.mod
│   └── main.go
└── pkg2
├── go.mod
└── utils.go
pkg1/main.go
package main
import (
"fmt"
"local/pkg2"
)
func main() {
fmt.Println(pkg2.Add(1, 2))
}
pkg1/go.mod
module local/pkg1
go 1.12
require local/pkg2 v0.0.0
replace local/pkg2 => ../pkg2
pkg2/utils.go
package pkg2
func Add(a, b int) int {
return a + b
}
pkg2/go.mod
module local/pkg2
go 1.12
Running:
cd pkg1
go run main.go
You get:
3
If you want to be able to go get a module, it should follow the <host>/repo/path/within/repo pattern.
However, I would suggest stepping back and asking if you really do want a multi-module repo. It adds substantial complexity, it is hard to get right, and usually means more work on an on-going basis.
Russ Cox commented in #26664:
For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.
For more details, see "Organize local code in packages using Go modules".

How can I resolve dependencies in nested application binary in Go project?

This sounds stupid, but I am trying for build my new golang project for a while now and I am stuck with following error
can't load package: package github.com/kuskmen/yamq/cmd/yamq-client: found packages main (main.go) and yamqclient (yamq-client.go) in C:\projects\yamq\cmd\yamq-client
I know this should be straightforward to fix, but I come from .NET and I am still not experienced in Go projects and its dependency resolution model hence the struggle.
My project structure looks like so
/yamq
/cmd
/yamq-client // yamq client application binary
main.go // package main
yamq-client.go // package yamqclient
/yamq-server // yamq server application binary
main.go // package main
yamq-server.go // package yamqserver
go.mod // contains only "module github.com/kuskmen/yamq" for now
... // some library files that will probably be moved to /shared folder
so far so good, when I do go build in outermost directory ( /yamq ) it is building successfully (or at least it is not showing any errors), but when I try to build either yamq-client or yamq-server binaries I get the aforementioned error and every time I try to google it or find something useful I got some old article or answer that dates back 2013-2016 that suggests something about $GOPATH and etc which shouldn't be the case here since I am trying to use go modules.
Help a fellow .NET developer join Go community by explaining him how exactly modules work cause I found this and this useless or at least I am missing the point, thanks in advance!
To follow up from my comment above:
From https://golang.org/doc/code.html:
Go programmers typically keep all their Go code in a single workspace.
A workspace contains many version control repositories (managed by Git, for example).
Each repository contains one or more packages.
Each package consists of one or more Go source files in a single directory.
The path to a package's directory determines its import path.
For your project, I'd do something like this:
$ tree
.
├── clientlib
│   └── lib.go
├── cmd
│   ├── client
│   │   └── main.go
│   └── server
│   └── main.go
├── go.mod
└── serverlib
└── lib.go
5 directories, 5 files
$ cat go.mod
module myproject.com
The module name is arbitrary (could be github.com/yourname/yourproject).
For the server side:
$ cat serverlib/lib.go
package serverlib
import "fmt"
func Hello() {
fmt.Println("Hello from serverlib.Hello")
}
$ cat cmd/server/main.go
package main
import (
"fmt"
"myproject.com/serverlib"
)
func main() {
fmt.Println("Running server")
serverlib.Hello()
}
And now we can build and run it:
$ go build -o server cmd/server/main.go
$ ./server
Running server
Hello from serverlib.Hello
The client side looks symmetrical.
Variations: you could name the .go files in cmd/... by their actual binary names - like server.go and client.go. The package in each is still main, but then go build creates an executable with the file's name (sans the .go) without needing to -o explicitly.

Using subpackages with go mod locally

I have a go package located on my filesystem (not in the $GOPATH), called bitbucket.org/me/awesome.
~/awesome> tree
.
├── main.go
├── go.mod
├── go.sum
├── subpackageA
│   └── main.go
My go.mod looks like:
module bitbucket.org/me/awesome
require (
... # lots of external dependencies
)
replace bitbucket.org/me/awesome => ./
In main.go in my top-level directory, I call a subpackage like follows:
import "bitbucket.org/me/awesome/subpackageA"
which all seems pretty normal. go get works. However, when I clone this entire repository somewhere else (say in a Docker image) and run go get for the first time, I get errors like:
package bitbucket.org/me/awesome/subpackageA: https://api.bitbucket.org/2.0/repositories/me/awesome?fields=scm: 403 Forbidden,
which means it's not using the local filesystem version of the packages, even though I told it to with the replace directive in the go.mod file.
What am I doing wrong? How do I ensure that subpackages are used from the filesystem instead of attempting to be fetched from the internet?
Go has no (real) notion of "subpackage". All packages are basically treated equal. This means that a replace bitbucket.org/me/awesome does not influence package bitbucket.org/me/awesome/subpackageA as these are two individual, unrelated packages. The folder layout does not introduce a relation of subpackageA to awsome, or the other way around *).
So you need to add an individual replace directive for subpackageA
replace bitbucket.org/me/awesome/subpackageA => ./subpackageA
*) Nitpicking for absolute correctness: Folder layout does have influence for folders named internal (cannot be imported from other projects), for folders named vendor (which may contain vendored packages) and searching for a go.mod file stops at the repo root.
For another approach, you can have go.mod like this:
module awesome
Then call subpackage like this:
import "awesome/subpackageA"
https://golang.org/doc/code.html

Structuring a multi-executable Go project

I am trying to build a micro-services architecture project in Go. I am currently using Go 1.11 which supports modules, so I put my root directory in an arbitrary chosen directory, outside of GOPATH.
If I am getting the micro-services architecture concepts right, although my microservices need to be independent, they can share dependencies (and I don't see another way to do stuff, is it?)
Below is my directory structure:
.
├── go.mod
├── lambda
│   └── account_create
│   └── main.go
├── readme.md
└── types
├── account.go
├── location.go
├── order.go
├── pricing.go
├── product.go
└── types.go
Now the behavior I expected was to be able to run go build lambda/account_create and get an executable with that functionality so I can supply it to the respective AWS Lambda function.
However, when I run the command, I get:
can't load package: package lambda/account_create: unknown import path "lambda/account_create": cannot find module providing package lambda/account_create
Please explain me why this does not work and give me some advice on how a project like this should look.
Thank you very much!
Edit Here is my lambda/account_create/main.go file contents:
package account_create
import (
"fmt"
"roasted.ro/pkg/types"
)
func main() {
account := types.UserAccount{Name: "Victor"}
fmt.Printf("Welcome %s", account.Name)
}
To answer the latter part of your question (as the first half is more about preference), you need to do:
go build ./lambda/account_create
Notice the ./. go build takes in a package name or a path. If you don't prepend the path with a ./, it assumes you are giving it a package name.
Edit: the package has to be main if you want an executable. According to the docs, you always have to use package main for projects that build an executable:
Executable commands must always use package main.
Edit: I'll add some opinions on layout:
Consider which parts of your project are exported to other go projects. Meaning someone could import your package and expect to gain value and not have their code break when yours changes.
That should be either in a github.com/you/pkg/somepath package or just github.com/you/somepath.
Anything else (other than main packages) should be under a /internal package.
I typically make a /cmd/myexecurable1 /cmd/myexecurable2 and so forth for each executable.

Convert type from a different directory [duplicate]

I want to break my project up to subfolders.
I want this code structure:
├── main.go
└── models
└── user.go
Where main.go is:
package main
import (
"fmt"
"./models"
)
func main(){
fmt.Println(User{"new_user"})
}
And user.go is:
package models
type User struct {
Login string
}
But User is not defined in main package and import raise warning "imported and not used".
What am I doing wrong? My project is simple (not such a example but just with few files (controllers and models)) and I want a simple structure.
Maybe I doing it in completely wrong way?
Problem project is here: https://github.com/abonec/go_import_problem
I recently achieved this by using go modules.
Golang introduced preliminary opt-in support for modules as of go v1.11.1 which is intended to completely remove the, frankly, absurd $GOPATH necessity. Not only can you now have versioned dependencies in any normal directory such as ~/development, but you can basically have something that looks like namespaces and sub-directories. You can enable this feature by invoking the go command with the following environment variable: GO111MODULE=on.
Go v1.11.3 expects to enable modules by default and is slated for August 2019.
Here is an example directory structure (that you might find typically in some other languages).
~/Dev/my-app
├── src/
│ ├── one/
│ │ ├── two/
│ │ │ └── two.go
│ │ └── one.go
│ └── zero.go
├── go.mod
└── app.go
The application is called my-app, which will be the module name for app.go. We define this once in go.mod and then each of all the other go files in subdirectories will automatically be importable as if they were namespaced.
Given the above, two.go, assuming it contains a function named Two, will be importable in app.go by using my-app/src/one/two.
Here's what you need to do to achieve this:
go.mod
module my-app
two.go
package two
func Two() string {
return "I'm totally not supposed to be using go modules for this"
}
app.go
package main
import "my-app/src/one/two"
func main() {
two.Two()
}
If you were to place another file within two/, then you would simply use two.TheNewFunc() as long as you made TheNewFunc() available within the new file.
I created a very simple GitHub repo which you can check out as a demonstration.
Your import should be an absolute one:
import "github.com/abonec/go_import_problem/models"
If you don't want to export your project to an external referential, you can do a:
import "go_import_problem/models"
(That is: "the name of your project folder accessible by GOPATH/your package")
See "How to use custom packages in golang?".
And you would use:
models.User
As mentioned in Effective Go:
The importer of a package will use the name to refer to its contents, so exported names in the package can use that fact to avoid stutter.
(Don't use the import . notation, which can simplify tests that must run outside the package they are testing, but should otherwise be avoided.)
kostix adds in the comments:
to reiterate, names of Go packages are always absolute (that is, there's no relative package names, neither with ./ nor with ../ or anything like that) but that names are "anchored" to one of the so-called workspaces listed in $GOPATH.
When Go searches for a package, it looks through workspaces and tries to find a package in each of them, in order.
The search is not recursive.
And no, there's no requirement to encode URLs in package paths -- unless you want to make your package public.
You need to qualify items in in a package by its package name
So
fmt.Println(models.User{"new_user"})
Breaking up a single project into subfolders is not the recommended way of structuring a go project, which is why there is basically no good way to do what you want.
If the project is really large, and too unwieldy to make a single package, consider splitting it into several totally distinct packages, rather than special sub-directory packages. This has the advantage of forcing you to think cleanly about your internal APIs.
The packages are referenced in code in relation to your "go/src" folder
└── go
└── src
└── myAwesomeProject
├── main.go
└── models
└── user.go
So in main.go
package main
import (
"fmt"
"myAwesomeProject/models"
)
Similarly packages can reference each other using the same convention.
You should use your imported objects by it's imported names. For example if you
import "./models"
with struct User you should use it as
models.User

Resources