I ran into a repo that seems to be a Go module, but there's no go.mod file in it: github.com/confluentinc/confluent-kafka-go.
Is it ok for a go module to have no go.mod file with dependencies, or the authors of that library just didn't migrate to modules yet?
Dependency modules do not need to have explicit go.mod files.
The “main module” in module mode — that is, the module containing the working directory for the go command — must have a go.mod file, so that the go command can figure out the import paths for the packages within that module (based on its module path), and so that it has a place to record its dependencies once resolved.
In addition, any modules slotted in using replace directives must have go.mod files (in order to reduce confusion due to typos or other errors in replacement paths).
However, in general a module that lacks an explicit go.mod file is valid and fine to use. Its effective module path is the path by which it was required, which can be a bit confusing if the same repository ends up in use via multiple paths. Since a module with no go.mod file necessarily doesn't specify its own dependencies, consumers of that module will have to fill in those dependencies themselves (go mod tidy will mark them as // indirect in the consumer's go.mod file).
SHORT SUMMARY OF THE DISCUSSION:
The answer is "No"!
This project contains a set of go packages, but it is not a Go module as it doesn't contain go.mod file (although, it used to be a multi-module repo (Go) previously).
go get can run in both ways: module-aware mode and legacy GOPATH mode (as of Go 1.16).
To read more about this, refer to docs by using the go command:
$ go help gopath-get
and
$ go help module-get
It'd tell about how go get works in both cases.
Also, I noticed that it can download any repository and would treat it as a Go package, even if it contains an arbitrary Python project.
I did a simple test to demonstrate the same:
$ go get github.com/mongoengine/mongoengine
And it surprisingly worked.
Modules are defined by their go.mod file. Without a go.mod file, it is not a module.
See this from the Go Modules Reference
A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.
A module is identified by a module path, which is declared in a go.mod file, together with information about the module's dependencies. The module root directory is the directory that contains the go.mod file.
And
A module is defined by a UTF-8 encoded text file named go.mod in its root directory.
Related
I successfully used rules_go to build a gRPC service:
go_proto_library(
name = "processor_go_proto",
compilers = ["#io_bazel_rules_go//proto:go_grpc"],
importpath = "/path/to/proto/package",
proto = ":processor_proto",
deps = ["//services/shared/proto/common:common_go_proto"],
)
However, I'm not sure how to import the resulting file in VSCode. The generated file is nested under bazel_bin and under the original proto file path; so to import this, it seems like I would need to write out the entire path (including the bazel_bin part) to the generated Go file. To my understanding, there doesn't seem to be a way to instruct VSCode to look under certain folders that only contain Go packages/files; everything seems to need a go.mod file. This makes it quite difficult to develop in.
For clarity, my directory structure looks something like this:
WORKSPACE
bazel-bin
- path
- to
- generated_Go_file.go
go.mod
go.sum
proto
- path
- to
- gRPC_proto.proto
main.go
main.go should use the generated_Go_file.go.
Is there a way around this?
I don't use Bazel and so cannot help with the Bazel configuration. It's likely there is a way to specify the generated code location so that you can revise this to reflect you preference.
The outline you provide of the generated code, is workable though and a common pattern. Often the generated proto|gRPC code is placed in a module's gen subdirectory.
This is somewhat similar to vendoring where your code incorporates what may often be a 3rd-party's stubs (client|server) into your code. The stubs must reflect the proto(s) package(s) and, when these are 3rd-party, using gen or bazel-bin provide a way to keep potentially multiple namespaces discrete.
You're correct that the import for main.go, could (!) be prefixed with the module name from go.mod (first line) followed by the folder path to the generated code. This is standard go packaging and treats the generated code in a similar way to vendored modules.
Another approach is to use|place the generated code in a different module.
For code generated from 3rd-party protos, this may be preferable and the generated code may be provided by the 3rd-party in a module that you can go get or add to your go.mod.
An example of this approach is Google Well-Known Types. The proto (sources) are bundled with protoc (lib directory) and, when protoc compiles sources that references any of these, the Go code that is generated includes imports that reference a Google-hosted location of the generated code (!) for these types (google.golang.org/protobuf/types/known).
Alternatively, you can replicate this behavior without having to use an external repo. The bazel-bin folder must be outside of the current module. Each distinct module in bazel-bin, would need its own go.mod file. You would include in a require block in your code's go.mod file references to the modules' (one or more) locations. You don't need to publish the module to a external repo but can simply require ( name => path/to/module ) to provide a local reference.
go.mod's replace directive is a local configuration option, different developers could have the local module source in different locations.
It just feels wrong including this option in a file that has to be committed to a repo from which others can use the module (be it private or public).
Is there a way to specify this somewhere else than in go.mod?
Example:
https://github.com/Drean64/c64/blob/master/src/go.mod#L5
module github.com/Drean64/c64
go 1.18
replace github.com/Drean64/cpu6502 => ../../cpu6502/src
replace directive temporary solution when you want to use local modules but I prefer to use build flags, below -modfile is good and you can use it while building or running the program.
example : go run -modfile=local.mod main.go
-modfile file
in module aware mode, read (and possibly write) an alternate go.mod
file instead of the one in the module root directory. A file named
"go.mod" must still be present in order to determine the module root
directory, but it is not accessed. When -modfile is specified, an
alternate go.sum file is also used: its path is derived from the
-modfile flag by trimming the ".mod" extension and appending ".sum".
I do use replace directive only when needs a temporary solution.
different developers could have the local module source in different locations.
By using a relative path, you could reference a folder which is a submodule of your main repository project, which means all developer would benefit from the same local replace directive.
Is there a way to specify this somewhere else than in go.mod?
It does not seem to be, the replace directive is linked to go.mod, and:
replace directives only apply in the main module’s go.mod file and are ignored in other modules.
What I essentially have is a mono-repo, which doesn't have a go.mod at the root level. There are multiple directories inside this mono-repo, with each of them having their own go.mod files. I'll refer to them as sub-modules.
Now, I've figured out a way to be able to access the sub-modules independently (versioned) in a completely different codebase. The issue I'm facing now is, to dis-allow the import of the entire mono-repo, using:
go get link.to/mono-repo#commit_id ------> A
and only allow import using :
go get link.to/mono-repo/sub_mod1#v0.x.y
go get link.to/mono-repo/sub_mod2#v0.y.z
The command A is able to fetch the entire repo, and then can be used to access the inner modules. Is there any way to stop this from being allowed?
I tried a few things, like:
Added non-compilable code in a file noCompile.go at the root level. On go get..., the compilation error is printed, but use of the inner modules still works fine.
Added an init() function in the same noCompile.go file, which just calls panic(). This init function is not being executed, as the root directory is never directly accessed, only the inner modules are.
Is there ANY way to achieve what I'm intending to?
Any directory that contains its own go.mod file is excluded from the module in the parent directory. So if you go get link.to/mono-repo#commit_id, that should not contain the packages link.to/mono-repo/sub_mod1 or link.to/mono-repo/sub_mod2 (assuming that they exist and have their own go.mod files).
I suspect that you were observing imports like import "link.to/mono-repo/sub_mod2/some/package/here" to resolve not because of that initial go get, but rather because the go command was resolving the missing imports (and adding the missing dependencies) automatically; see https://golang.org/ref/mod#go-mod-file-updates.
As of Go 1.16 (released today!), most go commands no longer implicitly modify the go.mod file, so the fact that the module at the repo root does not include the contents of nested modules should hopefully be clearer.
I am now using golang 1.13 and use go module.
However, when I import a package (for example, a) that is not opted in go module, in go.sum file there is still two lines. Go module tells us that "Each known module version results in two lines in the go.sum file. The first line gives the hash of the module version's file tree. The second line appends "/go.mod" to the version and gives the hash of only the module version's (possibly synthesized) go.mod file. The go.mod-only hash allows downloading and authenticating a module version's go.mod file, which is needed to compute the dependency graph, without also downloading all the module's source code."
(https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification).
But this package is not a module, and so it does not have a go.mod file? For example, if I import package call "github.com/example/a" that is not a module, in go.sum file, it still has these two lines:
github.com/example/a v0.0.0-20190627063042-31896c4e4162 h1:rSqi2vQEpS+GAFKrLvmxzWW3OGlLI4hANnEf/ib/ofo=
github.com/example/a v0.0.0-20190627063042-31896c4e4162/go.mod h1:tcpxll8wcruwpPpWBbjAsWc1JbLHld/v9F+3rgLIr4c=
My question is, how the second line generated?
go.sum contains the expected cryptographic checksums
of the content of specific module versions. Each time a dependency is
used, its checksum is added to go.sum if missing or else required to match
the existing entry in go.sum.
Every package/module is dependency and every dependency mean to be maintained with checksums in go.sum so whether it is package or module it would be maintained.
The sources would be downloaded in $GOPATH/src directory accordingly.
TRY -
Cause in into the go.sum file are written EVERY dependencies sum hash. The one related to your go.mod file, and the one imported from the modules that you have imported. Try run go mod tidy in order to reduce the imported modules, your go.mod file will contains some //indirect import, that are the one that your imported modules use internally.
Maybe the Golang source code can explain the reason:
func (r *codeRepo) legacyGoMod(rev, dir string) []byte {
// We used to try to build a go.mod reflecting pre-existing
// package management metadata files, but the conversion
// was inherently imperfect (because those files don't have
// exactly the same semantics as go.mod) and, when done
// for dependencies in the middle of a build, impossible to
// correct. So we stopped.
// Return a fake go.mod that simply declares the module path.
return []byte(fmt.Sprintf("module %s\n", modfile.AutoQuote(r.modPath)))
}
code from here: https://github.com/golang/go/blob/acac535c3ca571beeb168c953d6d672f61387ef1/src/cmd/go/internal/modfetch/coderepo.go#L857
↓ VSCode open /usr/local/go/src:
locate to cmd/go/internal/modfetch/coderepo.go, add a breakpoint to func legacyGoMod
locate to cmd/go/internal/modfetch/coderepo_test.go, press F5
wait for a while, stop at the breakpoint
Following the release of Go 1.11, I have been trying to move my repositories to Go modules, by adding a go.mod file at their root.
One of my root libraries my.host/root is in its version 17.0.1, so I wrote in its go.mod file:
module my.host/root/v17
I tagged that version v17.0.1 as documented in the Go modules manual.
When I try to make a new Go project that uses my root library, like:
package main
import root "my.host/root/v17"
func main() {
root.DoSomething()
}
And try to compile it, I get the following error:
go: my.host/root#v0.0.0-20180828034419-6bc78016491a: go.mod has post-v0 module path "my.host/root/v17" at revision 6bc78016491a
I am at loss figuring out why this happens. I explicitly added v17.0.1 in the go.mod file, yet every attempt at go build replaces the entry with a v0.0.0-20180828034419-6bc78016491a version which then fails because at that commit, the go.mod file module entry of my root library indeed ends with a v17, as it should.
For the record, this commit is the same as the tagged v17.0.1 version.
What am I doing wrong here? How can I debug this situation?
I had make two mistakes:
My initial v17.0.0 tag would point to a commit where go.mod did not contain the v17 import path suffix. As a result, it seems Go tooling considers the whole v17 major version as a v0/v1 instead, even if later v17 tags point to a commit with a correct go.mod directive, hence the commit ID "translation".
In my dependent projects, in the go.mod file, I mistakenly specified
require my.host/root v17.0.1 instead of
require my.host/root/v17 v17.0.1.
After fixing both those issues, everything seems back to normal and it works perfectly. I wish the documentation had been clearer about this but I guess this is a good opportunity to make a contribution!
The error I got was: github.com/emicklei/go-restful#v0.0.0-20180531035034-3658237ded10: go.mod has post-v0 module path "github.com/emicklei/go-restful/v2" at revision 3658237ded10
Appending github.com/emicklei/go-restful with v2 like so: github.com/emicklei/go-restful/v2 in my go.mod file fixed it for me.