Go version: 1.12.9
Here is the structure of a simple demo project:
So we have a module domain, which contains two modules: activity and person.
I would like to use domain with all nested modules in the main file => modules.go.
I know how to import domain in the main go.mod:
module modules
go 1.12
replace modules/domain v0.0.0 => ./domain
require modules/domain v0.0.0
So after that can use code from domain/domain.go, but I cannot access code from activity and person modules.
Yes, I could manually import nested modules, for example:
(main go.mod):
module modules
go 1.12
replace modules/domain v0.0.0 => ./domain
replace modules/domain/activity v0.0.0 => ./domain/activity
require (
modules/domain v0.0.0
modules/domain/activity v0.0.0
)
but I don't want to manually import all submodules.
How to configure modules to import domain with all submodules?
Yes, I could manually import nested modules [...]
but I don't want to manually import all submodules.
How to configure modules to import domain with all submodules ?
You simply cannot do this.
Some background:
There is no real notion of "sub"-module, all modules are equal.
Same for packages: all(*) packages are equal. If you want to use a package you must import it.
The layout in the filesystem does not imply any kind of technical relation between packages (e.g. net/http/cookiejar is as much related to net/http as it is related crypto/md5: not at all).
Being part of a module doesn't mean much: Modules are just sets of packages versioned together and doesn't add any additional relation between these packages.
The rule is dead simple: If you want to import a package you have to import it.
The fact that there are no magical wildcard imports might seem annoying (see below) but prevents unintended imports: Adding a package doesn't magically import it (and execute its init functions!).
In real live having to import all packages is not that annoying because "normal" Go code doesn't use tiny packages. This is a common mistake for someone indoctrinated by Java/C#/PHP/Node "project architectures" with lots of folders and tiny classes: Don't do that in Go. It is not helping. It often even leads to circular import and hurts.
Same for modules. A Go module is probably not what you think it is. A module is a set of packages versioned together. I doubt there is a reason to provide different versions for package person and package domain at the same time. (It is allowed to have several modules in one repository/project, but having more than one is a very rare case, yours clearly isn't).
Best advice (in order of relevance):
Stop making every package its own module.
Stop making tiny packages.
Stop trying to mimick a source code layout ("architecture") you might be used from other languages.
(*) internal and vendored packages are an exception which does not apply to your problem.
go 1.18 has the support for the new multi-workspace feature by go.work files.
for more info, check golang workspace proposal
I faced a similar situation with Go 1.20. Here is How I solved it.
Package
I have used a repository as package in my Go project. This repository is uses a sub-module.
Here is the .gitmodules file inside my repository my-repository/services:
Change
What I was missing:
After the introduction of Workspace, submodules were treated differently. So now I need to create a go.work file. Needed to be placed at same level as go.mod.
go.work
go 1.20
use ./validator/validate
Usage
Now I want to use this package in my main repo. Imported it as,
This solved my issue. You can find the documentation at: Setting up your workspace
Related
I want to use a function from a parent module for this project https://github.com/eregnier/beuss
however (and for the repository in the current state), all my tries fails.
in my root folder I have the following go.mod file
module github.com/eregnier/beuss
go 1.17
in the ./cmd folder I have the following go.mod file
module beuss/cmd
go 1.17
I tried various combination of go.mod file for the cmd folder like
module github.com/eregnier/beuss/cmd
go 1.17
I also tried some imports from the https://github.com/eregnier/beuss/blob/main/cmd/cmd.go file looking like
import (
"fmt"
"io/ioutil"
"log"
"os"
"github.com/eregnier/beuss"
// The following line does not work either
// "github.com/eregnier/beuss" beuss
)
//try to use parent module functions
func main () {
connPUT, err := beuss.newClient(beuss.MESSAGE_PUT)
}
nothing work.
I tried the following commands desperately
go mod init
go mod tidy
go get github.com/eregnier/beuss
go install github.com/eregnier/beuss
I got the follwing errors :
go get: github.com/eregnier/beuss#v0.0.0-20220101172233-d7ecaadb1d81: parsing go.mod:
module declares its path as: beuss
but was required as: github.com/eregnier/beuss
with not understanding the real issue. I googled errors, I had a look at this which looks intersting https://go.dev/doc/code#Workspaces
In the end, I am just loosing patience for this issue where I already spend lot of time without really understanding the real good practice / why it should be done like this or that.
I am sorry if this is a redundant question. but this topic looks nebulous to me where I just want to resolve a dependency. I guess I am lost between local imports / remote imports / modules and pacakges concepts. I guess I just want an entry point to understand all of this where in javascript a simple require('../code.js') does the trick.
What I would like in the end is to be able to better understand go imports system, and pragmatically, how to solve my linked project dependency so I can use functions from parent "folder" (ideally without namespacing)
As the question was unclear because I did not understand what was wrong, I'll try here to synthetise what worked for me.
I followed andrey dyatlov advice by simply removing the go.mod from my cmd folder and It is possible to make this folder a standalone program without module management. So I can go run / build the command file.
Also I slightly changed imports system and reorganized my code folders by understanding how imports works a bit better (I did not practised go since a while and I am a relatively new commer to go world)
So for this last point, doing imports with github.com/eregnier/beuss does the trick, but I missed to export my "lib" functions by just capitalizing their name. Doing so let me import and use them elsewhere in the code.
For the next person that this question might help, it should be possible to see the state of the project (wip) from the link in the question to see how it works now.
I have this problem:
Module A imports module X v0.1.0
Module B imports module X v0.2.0
I import both A and B in my project.
Golang picks out X v0.2.0 and calls it good. But it is not good. v0.1 and v0.2 are different enough that A breaks and my project doesn't compile.
What options do I have to fix this?
The offical go stance seems to be "developers of X were doing it wrong and should've released a major version after breaking changes". But that doesn't help the current situation.
I can't find discussions of practical solutions.
Further information
The above is a simplification, A and B have a couple more dependencies that also depend on X.
This project needs to be maintainable by a group of people. So updating A and B to new versions is ideally straightforwards.
This is security-critical code so subtle bugs from mismatched dependencies are a concern.
Things I've tried:
Forking A and X, changing the import paths throughout, and updating A's go.mod. It works but makes updates to those modules slow and error prone.
Copy and pasting the required code out of A and X to avoid importing it. Also slow and error prone.
Did a deep dive into creative applications of go mod vendor, but couldn't find a solution.
The ideal solution here is to submit an upstream PR to module A to make it compatible with X v0.2.0, and then update your own module to use the fixed version of module A.
If you need to temporarily slot in a fix to module A while you are waiting for the upstream maintainer to respond to the PR, you can do that using a replace directive. (See https://golang.org/doc/modules/managing-dependencies#tmp_11 for more detail.)
You can, for example, create local copies of the both modules and use replace directives along with a pseudo major versions to import multiple versions of the same repo using different tags:
replace moduleX/v1010 => ./src/moduleX/0.1.0
replace moduleX/v1020 => ./src/moduleX/0.2.0
Then in your project would use either of the versions as needed:
import (
moduleX_010 "moduleX/v1010"
moduleX_020 "moduleX/v1020"
)
Suppose I have a Go project which act as shared library in another Go project. How do I markup Go symbols (consts, structs, vars) that they are used outside this project?
I guess the underlying problem is that I have a hard time knowing which code uses said symbols.
Please note: This is not about semantic versioning, of which I am very well aware and which I use. I know Semver can help to identify breaking changes.
Instead, this is about finding out if I actually break one of my own projects (compared to: This symbol should be unexported or used outside the package). I am thinking of some sort of annotation which don't exist in Go.
As an aside, IntelliJ doesn't know either and marks those symbols as "Unnecessarily exported". Maybe an IntelliJ-centric solution could suffice.
To illustrate my problem:
package sharedlib
import "time"
// MyFavoriteTimeFormat is a blablabla...
const MyFavoriteTimeFormat = Time.RFC3339
package dependingproject
import "github.com/thething/sharedlib"
import "time"
func convertToString(timestamp time.Time) string {
return timestamp.Format(sharedlib.MyFavoriteTimeFormat)
}
When I happily rename MyFavoriteTimeFormate and release it, the code will break in the depending project when it updates the dependency.
Don't export anything until some other package needs it. If another package needs something, then do export, and then you'll know that if something is exported, it is because it is used outside of the package. And do not do breaking changes on exported identifiers. If you really must, then increment major version. Using go modules, that won't break existing other packages, they will continue to use the old version.
If your module is broken down into multiple packages (because it is "big"), and you wish to export something solely for the other packages of your module, then use the internal package concept, so it will still be "unexported" (unimportable) to other modules. For details, see Can I develop a go package in multiple source directories?
In my go.mod file, I have:
require (
// ... editted for brevity
github.com/Liquid-Labs/catalyst-firewrap v2.0.0-prototype.3
// ...
)
When I tried to build another package dependent on this package, I get: invalid module: github.com/Liquid-Labs/catalyst-firewrap should be v0 or v1, not v2 (v2.0.0-prototype.3).
There was a rewrite back before go went modular, and I wasn't tagging stuff back then, so there is no v1... is that the problem? I'm not having any luck finding an explanation of the underlying problem here.
Per the Go modules wiki:
If the module is version v2 or higher, the major version of the module
must be included as a /vN at the end of the module paths used in
go.mod files (e.g., module github.com/my/mod/v2, require github.com/my/mod/v2 v2.0.0)
and in the package import path (e.g.,
import "github.com/my/mod/v2/mypkg").
The same document quotes the rationale from the official Go FAQ:
Packages intended for public use should try to maintain backwards
compatibility as they evolve. The Go 1 compatibility guidelines are a
good reference here: don't remove exported names, encourage tagged
composite literals, and so on. If different functionality is required,
add a new name instead of changing an old one. If a complete break is
required, create a new package with a new import path.
and:
If an old package and a new package have the same import path, the new
package must be backwards compatible with the old package.
I'm having trouble achieving what should be an easy task. I understand the GitHub model for code organization (ie a library repo and an app repo that consumes the library). I think it's fantastic. But I find often that I want mylib to come bundled with a simple executable in a single main.go file. The main.go should be package main and should import mylib. In other words, it should be an exact documentation of how to build an app that consumes this library.
My point is, since it is often enough convenient to provide a simple command line interface that wraps your library, there should be an easy way to do this without having to make another repo, and golang should help.
I'd like something like the following:
$GOPATH/src/github.com/me/mylib
mylib.go
mylib_also.go
main.go
where mylib is the library (package mylib) and main.go is package main and on running go install it generates bin/mylib and pkg/mylib.a.
Either main.go should import "github.com/me/mylib" (if I do that now, I get cyclical import error) or go would understand what's happening since this feature should be built in and the one main.go in the repo generates the exec. Probably requiring the import (and dropping the cyclical error) is the better way.
Right now, I have to do
$GOPATH/src/github.com/me/mylib
mylib/
mylib.go
main.go
So I have to import github.com/me/mylib/mylib which is ridiculous.
In sum, go install should allow the special case of a package and a main which imports the package and provides a simple cli that demonstrates the packages API. The GitHub model promotes two repos, but it should be easy to do simple clis in one!
You can't have multiple packages per folder. Go operates on a package-level, not a file level.
Convention in this case—a binary that consumes your library—is to create a cmd folder with your package main - i.e. as per https://github.com/bradfitz/camlistore/tree/master/cmd or https://github.com/boltdb/bolt
In your case this would be:
$GOPATH/src/github.com/me/mylib
mylib/
mylib.go
README.md
cmd/
your-lib-tool/
main.go
If you just want a convenient, little wrapper, just some command line interface to your lib for demonstration purpose and it is okay if the main program is not built on go get (and not during simple go build and go install) but you are okay by running it via go run main.go or building it manually e.g. with 6g: Just use a build tag (see http://golang.org/pkg/go/build/#hdr-Build_Constraints) in main.go:
// +build ignore
and you won't need a different folder or repo.