Lets say I have an awesome Go binary that denormalizes some data, and I'd like to place it under:
//some/path/denormalize/
Ideally my package name would be denormalize which would allow me to follow the guidelines for "[when] designing a package, consider how the two parts of a qualified identifier work together, not the member name alone". Thus a potential member function might be denormalize.FromX(...). However, if I place the main.go for the denormalize binary under denormalize/ package I can't use the intended package name as now the package would be named main.
Some options I've considered:
Place main.go under denormalize/ and then place the rest of the code under internal/ or api/ or pkg/. Pros: This seems to be what most folks do. Cons: The package name is not meaningful, and it violates the principle of insuring that the two parts of the qualifier work together since api.FromX(...) no longer makes sense. This leads to more verbose names such as api.DenormalizeFromX(...).
Place main.go under denormalize/main/ and the rest of the code under denormalize/. Pros: Package name stays meaningful. Cons: a subdirectory then has a dependency on a parent directory which has some code smell.
Is there other options that I have not considered for where to place the main.go such that it won't force me to use a different package name?
Related
I am new to Golang and grasp the concept of packages fairly well, but have a question about their relationship to folder/directory location.
So I am working on a fairly large project with multiple subdirectories inside the /pkg directory. There are probably 30+ distinct package ___ namespaces declared.
Typically of course all files inside of a directory will have the same package, e.g. /pkg/system/api-client; all files in that directory are declared package apiclient
The question arises when I notice two files of package config; one is in /pkg/config/config.go and the other is /pkg/writer-job/pkg/config/config.go.
Are they part of the same package? If so, what is the convention for this (as it seems quite scattered)? And if not, how do you lexically separate config as two separate packages? I did search the docs but don't see this.
There are two concepts: package name, and import path.
The package name is what you declare with a package as the first statement in a go file. A directory can contain at most one package.
An import path is how you import that package, and shows the location of the package. Once you import it, the declared package name is used to qualify exported identifiers from that package.
In case of conflicting package names, you define an alias for one of them.
import (
"someproject/pkg/config"
writerconfig "someproject/pkg/writer-job/config"
)
Then, config.X refers to the first package, and writerconfig.X refers to the second.
I am developing a go package, which is a little bit complex and thus I want to organize the source code into multiple directories.
However, I don't want the users of the package to have to use too long imports. Anyways, the internal structure of the package isn't their concern.
Thus, my package structure looks so:
subDir1
subSubDir1
subSubDir2
subDir2
subSubDir3
...and so on. All of them have their exported calls.
I would like to avoid that my users have to import
import (
"mypackage/subDir1"
"mypackage/subDir1/subSubDir2"
)
...and so on.
I only want, if they want to use an exported function from my package, they should have access all of them by simply importing mypackage.
I tried that I declare package mypackage in all of the .go files. Thus, I had source files in different directories, but with the same package declaration.
In this case, the problem what I've confronted was that I simply couldn't import multiple directories from the same package. It said:
./src1.go:6:15: error: redefinition of ‘mypackage’
"mypackage/mysubdir1"
^
./src1.go:4:10: note: previous definition of ‘mypackage’ was here
"mypackage"
^
./src1.go:5:15: error: redefinition of ‘mypackage’
"mypackage/mysubdir2"
^
./src1.go:4:10: note: previous definition of ‘mypackage’ was here
"mypackage"
^
Is it somehow possible?
You should not do this in any case, as the language spec allows a compiler implementation to reject such constructs. Quoting from Spec: Package clause:
A set of files sharing the same PackageName form the implementation of a package. An implementation may require that all source files for a package inhabit the same directory.
Instead "structure" your file names to mimic the folder structure; e.g. instead of files of
foo/foo1.go
foo/bar/bar1.go
foo/bar/bar2.go
You could simply use:
foo/foo1.go
foo/bar-bar1.go
foo/bar-bar2.go
Also if your package is so big that you would need multiple folders to even "host" the files of the package implementation, you should really consider not implementing it as a single package, but break it into multiple packages.
Also note that Go 1.5 introduced internal packages. If you create a special internal subfolder inside your package folder, you may create any number of subpackages inside that (even using multiple levels). Your package will be able to import and use them (or to be more precise all packages rooted at your package folder), but no one else outside will be able to do so, it would be a compile time error.
E.g. you may create a foo package, have a foo/foo.go file, and foo/internal/bar package. foo will be able to import foo/internal/bar, but e.g. boo won't. Also foo/baz will also be able to import and use foo/internal/bar because it's rooted at foo/.
So you may use internal packages to break down your big package into smaller ones, effectively grouping your source files into multiple folders. Only thing you have to pay attention to is to put everything your package wants to export into the package and not into the internal packages (as those are not importable / visible from the "outside").
Inside your package source code, you have to differentiate your source directories by renamed imports. You can declare the same package mypackage in all of your source files (even if they are in different directories).
However, while you import them, you should give an induvidual names to the directories. In your source src1.go, import the other directories on this way:
import (
"mypackage"
submodule1 "mypackage/mySubDir"
)
And you will be able to reach the API defined in "mypackage" as mypackage.AnyThing(), and the API defined in mySubDir as submodule1.AnyThing().
The external world (i.e. the users of your package) will see all exported entities in myPackage.AnyThing().
Avoid namespace collisions. And use better understable, intuitive naming as in the example.
Yes, this is doable without any problems, just invoke the Go compiler by hand, that is not via the go tool.
But the best advice is: Don't do that. It's ugly and unnecessarily complicated. Just design your package properly.
Addendum (because the real intention of this answer seems to get lost sometimes, maybe because irony is too subtle): Don't do that!! This is an incredible stupid idea! Stop fighting the tools! Everybody will rightfully hate you if you do that! Nobody will understand your code or be able to compile it! Just because something is doable in theory doesn't mean this is a sensible idea in any way. Not even for "learning purpose"! You probably even don't know how to invoke the Go compiler by hand and if you figure it out it will be a major pita.
I have read the Go Tour and Googled "golang packages" but I have not yet found any advice about best practice in Go for organising moderately sized applications.
If I have an application that conceptually has several distinct parts, perhaps 10^3-10^4 LOC, and I don't intent to create reusable libraries for use in other applications, should all the source code files be package main?
To clarify ...
For example, lets say my program will have the following major chunks:
Something that manages a bunch of persistently stored data
allowing usual create, read, update, delete operations
Something that allows a human to view the stored data
Something that coordinates / mediates between these
Something that periodically fetches data updates from a web-service using SOAP.
So that would be MVC plus a fetcher of data.
From looking around at what people do, I now suspect I should
create $GOPATH/src/myprogramname
in there put some main.go with package main and func main() { ... } in it.
create some subdirectories like
$GOPATH/src/myprogramname/model
$GOPATH/src/myprogramname/view
$GOPATH/src/myprogramname/control
$GOPATH/src/myprogramname/fetch
have the .go files in those subdirectories begin with package fetch, etc. Where the package name always matches the subdirectory name.
my main.go will probably import ( ... "fetch"; "model"; "view"; "control" )
as main.go grows, split it into other reasonably sized .go files named according to purpose.
build the program, including *.go in the above package subdirectories by
cd $GOPATH/src/myprogramname
go build
Is that all I need to do? Is that the properly idiomatic Go way of organising things? Is there more I should know or be thinking of? Is there some canonical webpage or PDF I overlooked and should read to find out this stuff?
In short, I don't want a 10,000 line main.go with everything in it. What are the idiomatic Go principles for organising code into files, subdirectories, packages and any other organisational units corresponding to normal conceptual divisions according to well-known structured-programming and/or OO principles?
You could break down your project into several layers based on the encapsulation level of your functions, i.e. having low-level functions in separate packages and logic functions in your main package. (You could inspire yourself of MVC-like architectures)
Since we don't have any details about your code, it is hard to see what kind of architecture would be best suited.
But in the end your choice will be based on the code simplicity / re-usability balance.
The general "best practice" in Go seems to be having each package provide a type or a service. Most of the packages in the standard library expose one or two types, and functions for working with those types. Some, like net/http and testing, provide a service - not in the "microservices" sense of something executable in itself, but rather a set of functionality related to a specific activity.
I have a short program in Go with the following files part of it.
Program directory structure:
myprogram/
main.go
server.go
routines.go
structs.go
These different files contain different function. The structs.go file contains a list of structure type defined, and used in several files of my program. What I want to do, now is to split my program into package like in this example :
main/
main.go // the main program
server/
server.go // a package imported in main
routines/
routines.go // a package imported in main
My problem, is that I do not know where to put structs.go because it contains structures used in several package, as in the 'main.go' code.
How to share efficiently this structs.go file ? Should I include it (via a symlink to the file) in each of the package I defined, i.e serverand routines and also in main ?
My method may be awkward because I'm a beginner in Go, and in programming generally.
Don't link files across packages, that's bad practice. For one, the code will be duplicated. For another, identifiers will be duplicated meaning to denote the same entities (e.g. type or function), but they will be distinct. E.g. if linked and structs.go would contain a type Response definition, you would have 2 distinct types server.Response and routines.Response giving just more confusion.
One solution would be to put structs.go into its own package, e.g. model, and all other packages relying on it can import it (e.g. your main, server and routines).
In a theoretical example: if package A imports package B and structs.go would be needed in both, then it could also be added to package B. If there would be a package C needing only structs.go, then again it would be wiser to create its own package model (so package C doesn't need to import / know about package B, only the new model package).
Also if noone else will use your package and it is not too complex, it might not worth the hassle to organize it into multiple packages.
It is possible to define a type in one package only and to use it in other packages this way:
package one
type A struct{ B int }
Variant 1:
package two
. import "one"
var name A
Variant 2:
package two
import "one"
type A = one.A
var name A
I would prefer variant 2.
I have some questions on package naming for external Go libraries.
I am interested if using generic names like "text" is considered a good practice? Having in mind that I cannot declare a "nested package" and that the library I am building deals with text processing, is it ok to have the package named "text" or should I stick to the library name as a package name too?
I am building a set of libraries (different projects) and I want to combine them under the same package. Is this also problematic? I am new to the Go community and am still not sure if package pollution is a problem or not (I do not see a problem as long as I import few packages in my code).
The reference on that naming topic is "blog: Package names"
It includes:
Avoid unnecessary package name collisions.
While packages in different directories may have the same name, packages that are frequently used together should have distinct names. This reduces confusion and the need for local renaming in client code. For the same reason, avoid using the same name as popular standard packages like io or http.
Check also your package publishing practice, as it will help disambiguate your "text" package from others.
As illustrated in "An Introduction to Programming in Go / Packages":
math is the name of a package that is part of Go's standard distribution, but since Go packages can be hierarchical we are safe to use the same name for our package. (The real math package is just math, ours is golang-book/chapter11/math)
When we import our math library we use its full name (import "golang-book/chapter11/math"), but inside of the math.go file we only use the last part of the name (package math).
We also only use the short name math when we reference functions from our library. If we wanted to use both libraries in the same program Go allows us to use an alias:
import m "golang-book/chapter11/math"
func main() {
xs := []float64{1,2,3,4}
avg := m.Average(xs)
fmt.Println(avg)
}
m is the alias.
As mentioned in the comments by elithrar, Dave Cheney has some additional tips:
In other languages it is quite common to ensure your package has a unique namespace by prefixing it with your company name, say com.sun.misc.Unsafe.
If everyone only writes packages corresponding to domains that they control, then there is little possibility of a collision.
In Go, the convention is to include the location of the source code in the package’s import path, ie
$GOPATH/src/github.com/golang/glog
This is not required by the language, it is just a feature of go get.