Building Go module without main file - go

I have a small module that contains some shared code. The module looks like the following :
Shared
├── go.mod
├── go.sum
├── logging
│ └── logger.go
└── db-utils
├── db.go
If I'll try to build the Shared module from inside the directory I'm getting an error that no go file available under this module.
bash-3.2$ go build .
no Go files in /Users/xxxx/go/src/Shared
Is there a way to build a Go module that only has packages inside and doesn't have a main.go file? The motivation here is to use those packages in other go modules that can't access the internet or private repo in order to retrieve the packages.

The go command stores downloaded modules in the module cache as source code, not compiled object files. go build rebuilds the object files on demand and stores them in a separate build cache.
If you want to distribute a non-public module for use with other modules, you have a few options:
You can publish the module to a private repository — typically accessed with credentials stored in a .netrc file — and use the GOPRIVATE environment variable to tell the go command not to try to fetch it from the public proxy and checksum servers.
You can provide a private GOPROXY server or directory containing the source code, and use the GOPROXY environment variable to instruct the go command to use that proxy.
You can publish the source code as a directory tree and use a replace directive in the consumer's go.mod file to slot it into the module graph.

If you only needed to build files in either the logging or db-utils directory, then you could executing the following from the root directory Shared:
go build <INSERT_MODULE_NAME_FROM_GO_MOD>/logging
go build <INSERT_MODULE_NAME_FROM_GO_MOD>/db-utils
I'm not certain if those commands will work if one of the files has a dependency on a file from the other directory.
Another command that will probably build the entire project is this:
go build ./...

Is there a way to build a go module that only has packages inside and doesn't have a main.go file?
No. The input for the build process is a package, not a module. Note how it says [packages] in the CLI documentation of go build.
When building a package leads to multiple packages being compiled, that is merely a consequence of direct and indirect import statements coming from .go-files located in the package you are building.
Note that Go does not support compiling packages to binaries to distribute closed-source libraries or such. This was not always the case, though. See #28152 and Binary-Only packages. The closest which exists to supporting that are plugins, but they are a work in progress and require resolution of symbols at runtime.

Related

should src folder be created in golang go1.17

I am starting off with golang according to some document it mentions $GOPATH default should have bin pkg src directories.
When a I do go get github.com/astaxie/beedb there no src folder created and the source goes and sits in $GOPATH/pkg//mod/github.com/astaxie/beedb#v0.0.0-20141221130223-1732292dfde4/
according to documentation the it should be in src folder
I am using golang go1.17. Has something changed in newer versions or am I refering old doc.
you don't need to create src folder under $GOPATH
With Go modules, Go projects are no longer confined to $GOPATH
Don't confuse the project level /src directory with the /src directory Go used to use for its workspaces, but please note
"Since Go 1.14, module support is considered ready for production use, and all users are encouraged to migrate to modules from other dependency management systems "
please check below links
https://github.com/golang/go/wiki/Modules
There is no need to manually create the sub-folders in $GOPATH. When you run go get, go will handle everything for you. So if there is no src folder, you can just leave it.
The src folder was basically for package management before Go Modules (enabled by default since go 1.13). You don't really need this now.

How to use an alternate go.mod file for local development?

Currently I am working on an API which uses Serverless Framework with Go.
I'm using the Serverless-offline plugin for local testing.
This API depends on a few other repositories (which I also maintain), which I import using the go.mod file.
However I am having a hard time refining my developer workflow.
Currently, if I want to make changes in a repository which this API depends upon, I have to alter the projects go.mod to include replace directives for the purpose of testing, but then I'm having to manually change it back for deployment to production.
Basically I'm looking for a way to include replace directives, which only get applied during local development. How has everyone else dealt with this problem?
Bonus question: Is there any way to run Serverless offline in docker? I'm finding that serverless-offline running on the bare metal is causing inconsistencies between different developers environments.
You can run go commands with an alternate go.mod file with the -modfile option:
From Build commands:
The -modfile=file.mod flag instructs the go command to read (and
possibly write) an alternate file instead of go.mod in the module root
directory. The file’s name must end with .mod. A file named go.mod
must still be present in order to determine the module root directory,
but it is not accessed. When -modfile is specified, an alternate
go.sum file is also used: its path is derived from the -modfile flag
by trimming the .mod extension and appending .sum.
Create a local.go.mod file with the necessary replace directive for development and build, for example, with:
go build -modfile=local.go.mod ./...

Cannot build github.com/jonpchin/gochess - "working directory is not part of a module"

I unwrapped the project and from that project i did go get. Then I received the following errors.
C:\Users\Downloads\gochess-master>go install main.go
main.go:14:2: no required module provides package github.com/dchest/captcha: working directory is not part of a module
main.go:15:2: no required module provides package github.com/go-sql-driver/mysql: working directory is not part of a module
main.go:17:2: no required module provides package github.com/jonpchin/gochess/goforum: working directory is not part of a module
main.go:18:2: no required module provides package github.com/jonpchin/gochess/gostuff: working directory is not part of a module
main.go:20:2: no required module provides package golang.org/x/net/websocket: working directory is not part of a module
Then I went and tried doing go get each of the packages. I try doing the go install main.go and got same error.
then I tried including the repo inside $GOPATH/src/github.com/jonpchin/gochess and tried doing go get from there. I got the same errors.
github.com/jonpchin/gochess is a couple of years old and has not been updated to use go modules. I suspect you are using Go 1.16 under which "Module-aware mode is enabled by default, regardless of whether a go.mod file is present in the current working directory or a parent directory". There are a few ways to address this but given that there is a plan to "drop support for GOPATH mode in Go 1.17" the simplest approach might be to download the code and setup modules yourself. The following process works for me (well I get prompted for a mysql password):
git clone https://github.com/jonpchin/gochess
cd gochess
go mod init github.com/jonpchin/gochess
go get
go build
.\gochess.exe

How can I use a local repository in the vendor in Google Cloud Functions with Go

I'm trying to deploy a Google Cloud Function written in Go.
By doing some research I found out that vendor files are prefered over go.mod so I'm vendoring everything I use (which includes some local dependencies) and ignoring the go.mod/sum files in the .gcloudignore file.
The problem is that after trying to deploy, I get the following error:
go: nimbus#v0.0.0-00010101000000-000000000000: parsing /nimbus/go.mod: open /nimbus/go.mod: no such file or directory; Error ID: 03a1e2f7
nimbus is my local dependency and it has the following structure:
My Function repository has the following structrure:
and my go.mod file is:
module my_function
go 1.13
require nimbus v0.0.0-00010101000000-000000000000
replace nimbus => ../../../nimbus
I've tried this solution https://stackoverflow.com/questions/5441096 already. But it did not fix my issue.
I've tried everything to solve this issue, but nothing seems to work.
If you have a go.mod file and a vendor directory, the vendor directory will be ignored when you deploy your function.
https://cloud.google.com/functions/docs/writing/specifying-dependencies-go
I have used modules when deploying GCP functions in Go. Haven't had any problems. But I can't speak to the preference of using vendor/ instead. It should work, just without the go.mod file.
Turns out the problem was very complicated and I hope Google finds a solution for it asp.
By deploying my function using Cloud Build, It would read from my repository on Google Source, but, by reading from there it would bypass the .gcloudignore file and deploy both the go.mod/sum files and the vendor directory with my local code.
As said in https://stackoverflow.com/a/62050872/10316247:
If you have a go.mod file and a vendor directory, the vendor directory will be ignored when you deploy your function.
So the error would occur because of my go.mod not being able to find local repository.
My solution was to rename my go.mod/sum files so it would not be considered:
When you use golang 1.16 and specify golang 1.16 in the go.mod folder it will instead default to using the vendor files with the --mod=vendor flag set, which will solve this issue.
You'll just need to ensure your module name is formatted correctly (something like example.com/module).

Getting all dependencies of a Go project

For my project I am trying to get all the dependencies and sub dependencies of my project. I need to specific version of each of these dependencies. Not only do I need the dependencies of my project, but the dependencies of the dependencies and so on until the root.
For my project, go list -m all works for everything except indirect dependencies that have not opted into using go.mod files. Right now my workflow is taking an initial batch of repositories, downloading them from git then using “GO111MODULE=on go build ./…”. and “GO111MODULE=on go list -m -json all” to get the list of dependencies. I do not check for go.mod as all of the repositories I am scanning are using go.mod files.
For the list of dependencies that come out of this initial list I have some questions, for files without go.mod files, I used this as a reference: “https://blog.golang.org/using-go-modules”
-Path = Received from go list -m all, it can be GitHub, gopkg, or whatever is used to dl the go package.
Without go.mod
-“GO111MODULE=on go mod init <PATH from parent go.mod>”
-“GO111MODULE=on go build ./…”
-“GO111MODULE=on go mod tidy”
-“GO111MODULE=on go list -m -json all”
-From there I get a list of the dependencies of this module.
With go.mod
-“GO111MODULE=on go build ./…”
-“GO111MODULE=on go mod tidy”
-“GO111MODULE=on go list -m -json all”
Should I be running go build on each dependency that has a go.mod file? For ones without a go.mod file, I understand this should be done, as how else will we populate the go.mod file with the dependencies. But for files with a go.mod file, will I be pulling extra stuff that is not necessarily being used by my project with go build, like test files and other files that might not be used when I am simply importing that project? I understand that its better to get more unused dependencies rather than missing some, but it is getting a bit overwhelming with how massive the amount of dependencies is.
I can try to analyze go.sum file (when you execute go list -u, go.sum was created)
The go command uses the go.sum file to ensure that future downloads of these modules retrieve the same bits as the first download, to ensure the modules your project depends on do not change unexpectedly, whether for malicious, accidental, or other reasons. Both go.mod and go.sum should be checked into version control. (Using Go Modules - Adding a dependency)
go.sum file lists down the checksum (and version tag) of direct and indirect dependency required by the module.
% cat go.sum
...
github.com/bmizerany/perks v0.0.0-20141205001514-d9a9656a3a4b/go.mod h1:ac9efd0D1fsDb3EJvhqgXRbFx7bs2wqZ10HQPeU8U/Q=
github.com/c2h5oh/datasize v0.0.0-20171227191756-4eba002a5eae/go.mod h1:S/7n9copUssQ56c7aAgHqftWO4LTf4xY6CGWt8Bc+3M=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cockroachdb/apd v1.1.0/go.mod h1:8Sl8LxpKi29FqWXR16WEFZRNSz3SoPzUzeMeY4+DwBQ=
...
The phrase “all the dependencies” is unfortunately ambiguous.
go list -m lists all modules whose selected versions are determined by your go.mod file. That is one possible definition of "all the dependencies", but it is a much broader set of modules than I think most people intend when they talk about the “dependencies” of a module.
In practice, go list -test all (without the -m flag) is the broadest set of dependencies I would generally care about: it includes all of the packages listed in import statements within your module (i.e. everything you need in order to run go test ./... within your module), plus all of the packages transitively needed to run go test on those packages.
(In particular, go list -test all is also the set of packages that will be resolved when you run go mod tidy.)

Resources