go build not looking in $GOPATH - go

I have set $GOPATH, and importing some part of my source code which is present in $GOPATH/src.
Package I need to import is written by me and stored in GOPATH/src
I have named it as otelkafkago and it's path is GOPATH/src/otelkafkago
I have one more code base elsewhere on my HardDisk and when I try to compile that, it gives me
main.go:51:2: package otelkafkago is not in GOROOT (/usr/local/go/src/otelkafkago)
I have imported it as
import(
"otelkafkago"
)
Following is my screen capture during build,
kshitijpatil#PNQ-KPATIL checkoutservice % echo $GOPATH
/Users/kshitijpatil/go/
kshitijpatil#PNQ-KPATIL checkoutservice % go build
main.go:51:2: package otelkafkago is not in GOROOT (/usr/local/go/src/otelkafkago)
kshitijpatil#PNQ-KPATIL checkoutservice % env GOPATH=/Users/kshitijpatil/go go build
main.go:51:2: package otelkafkago is not in GOROOT (/usr/local/go/src/otelkafkago)
kshitijpatil#PNQ-KPATIL checkoutservice %

Since Go 1.16, the module-aware mode is enabled by default, regardless of whether a go.mod file is present in the current working directory or a parent directory.
More precisely, the GO111MODULE environment variable now defaults to on. To switch to the previous behavior, set GO111MODULE to auto.
In your case, it would be easier to migrate to go modules, even though setting GO111MODULE to auto might help.
As commented, the first line of your go.mod should be:
module myProject
Your other "otelkafkago" project, it should have its own go.mod with module otelkafkago as first line, and be built first.

Related

How to solve the import-local-package problem: GOPATH is ignored, only GOROOT takes effects

I'm new in GO and has got stuck in environment configuration for hours.
I succeeded to run some test projects, but when I tried to import my custom local packages (say xxx), the "go run" command failed, logging that:
*test/main/main.go:6:2: package test/xxx is not in GOROOT (/usr/local/go/src/test/xxx)*
It is strange that GOPATH seems to be ignored when importing local packages on my ubuntu computer.
go version is go1.16.5 linux/amd64
GOPATH is set using export GOPATH="$HOME/GoProjects"
GOROOT is set using export GOPATH="/usr/local/go"
GoMod is "on" using go env -w GO111MODULE=on
After editing project's .go files, command "go mod init" and "go mod tidy" are typed in the root folder of the project ($GOPATH/src/test), and the project file structure looks like:
/src
/.vscode
-launch.json
/gitbub.com
/test
/main
-main.go (here: import "test/xxx")
/xxx
-xxx.go (here: package xxx)
-go.mod (almost empty - line 1: module test
line 2:
line 3: go 1.16 )
What should I do to fix this problem?
Assuming (not clear from your post) this entire tree is under your $HOME/GoProjects. When using modules; don't put your project inside GOPATH as mixing both GOPATH and modules only leads to problems.
First, set $GOPATH to something like $HOME/go which you can use to go get various tools you want to use across projects (such as e.g. dlv debugger and so on) but do not put your projects in there.
When starting a new project put it outside $GOPATH to $HOME/GoProjects/test (you can retain project layout you have under src but you do not need src there):
GoProjects
/test
/main
-main.go
/xxx
-xxx.go
In project directory (here test) run go mod init NAME where NAME is name of your module. Do not use just test but fully qualified name which will namespace your project to your "domain" (e.g. github.com/yourusername/test). That name will then be used when importing packages from your module. So in main.go you will import xxx package (needs to have package xxx) like this:
import "github.com/yourusername/test/xxx"
See more info about modules here: https://blog.golang.org/using-go-modules
First, stop worrying about GOPATH. That is old stuff, and you don't need to do it any more. Then, make a folder somewhere test (doesn't need to be anything to do with GOPATH, just put in current directory, or wherever you want). Then make test/xxx. Then make test/xxx/xxx.go:
package xxx
const X = 1
Then make folder test/main. Then make test/main/main.go:
package main
import "test/xxx"
func main() {
println(xxx.X)
}
Then go back to test, and do go mod init test. Then go back to test/main,
and do go build. Done.

Can a go module have no go.mod file?

I ran into a repo that seems to be a Go module, but there's no go.mod file in it: github.com/confluentinc/confluent-kafka-go.
Is it ok for a go module to have no go.mod file with dependencies, or the authors of that library just didn't migrate to modules yet?
Dependency modules do not need to have explicit go.mod files.
The “main module” in module mode — that is, the module containing the working directory for the go command — must have a go.mod file, so that the go command can figure out the import paths for the packages within that module (based on its module path), and so that it has a place to record its dependencies once resolved.
In addition, any modules slotted in using replace directives must have go.mod files (in order to reduce confusion due to typos or other errors in replacement paths).
However, in general a module that lacks an explicit go.mod file is valid and fine to use. Its effective module path is the path by which it was required, which can be a bit confusing if the same repository ends up in use via multiple paths. Since a module with no go.mod file necessarily doesn't specify its own dependencies, consumers of that module will have to fill in those dependencies themselves (go mod tidy will mark them as // indirect in the consumer's go.mod file).
SHORT SUMMARY OF THE DISCUSSION:
The answer is "No"!
This project contains a set of go packages, but it is not a Go module as it doesn't contain go.mod file (although, it used to be a multi-module repo (Go) previously).
go get can run in both ways: module-aware mode and legacy GOPATH mode (as of Go 1.16).
To read more about this, refer to docs by using the go command:
$ go help gopath-get
and
$ go help module-get
It'd tell about how go get works in both cases.
Also, I noticed that it can download any repository and would treat it as a Go package, even if it contains an arbitrary Python project.
I did a simple test to demonstrate the same:
$ go get github.com/mongoengine/mongoengine
And it surprisingly worked.
Modules are defined by their go.mod file. Without a go.mod file, it is not a module.
See this from the Go Modules Reference
A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.
A module is identified by a module path, which is declared in a go.mod file, together with information about the module's dependencies. The module root directory is the directory that contains the go.mod file.
And
A module is defined by a UTF-8 encoded text file named go.mod in its root directory.

How do go package paths work when the directory is different?

I am doing some light hacking on the rclone project, and I'm somewhat new to go. I have cloned the repository in my home directory, /home/poundifdef/rclone.
When I look at the file, rclone.go, in the top-level directory, I see the following import statement:
package main
import (
_ "github.com/ncw/rclone/backend/all"
"github.com/ncw/rclone/cmd"
_ "github.com/ncw/rclone/cmd/all"
)
My question is this: nowhere in this code, as far as I can tell, do we specify that the github.com/ncs/rclone/* packages are supposed to refer to the local versions of these directories. Nowhere in my filesystem have I used a directory called github.com/ncw. And yet, running go run rclone.go is able to execute this code.
Where is the path github.com/ncw/rclone being mapped to my local directory, given these packages, as named, do not exist?
github.com/ncw/rclone is a module, because it has a file named go.mod at the repository root. Since you have $GOPATH unset, the experimental module feature is enabled.
Where modules are placed in your filesystem is irrelevant. when constructing import paths, all directory names in the module are appended to the module name. The module name is defined in the first line of go.mod, here "github.com/ncw/rclone". So the package in ./cmd has the import path github.com/ncw/rclone/cmd.
Commands such as go build, go install, go test, etc. download all other required packages automatically to $HOME/go/pkg/mod, unless the -mod=vendor flag is specified, in which case the sources in vendor are used.
Golang has an internal mechanism to resolve import path.
This mechanism relies on the existence of several variables on your computer, GOROOT, GOPATH and for old releases GO15VENDOREXPERIMENT.
When the compiler meet a package path, it performs a look up with those paths
GOROOT/src/[import path]
GOPATH/src/[import path]
cwd/vendor/[import path]
The longest path wins and is selected for further processing.
GOROOT is the path to the stdlib.
GOPATH is the path to your local workspace.
For all the details https://go.googlesource.com/proposal/+/master/design/25719-go15vendor.md
This all depends on the setup of your computer, so carefully check go env output.

golang modules and local packages

I'm trying to understand how to organize my golang project using go1.11 modules. I tried several options, but none of them worked.
I have some code in the main package under the application folder and a local package that the main package uses.
$GOPATH
+ src
+ application/
+ main/
+ main.go
+ otherFileUnderMainPackage.go
+ aLocalPackage/
+ someCode.go
+ someCode_test.go
+ someMoreCode.go
+ someMoreCode_test.go
Files in the main package, imports ../aLocalPackage. When I compile by go build main/*.go it's working.
Then, I ran go mod init application: V.0.9.9 and got the go.mod file, but the build always fails. I always get error about not finding the local package: build application:V0.9.9/main: cannot find module for path _/.../src/application/aLocalPackage. I also tried to place the local package right under src/, place it under main/ etc. but none of these methods worked for me.
What is the way to use modules and local packages?
Thanks.
Relative import paths are not supported in module mode. You will need to update your import statements to use a full (absolute) import path.
You should also choose a module name other than application. Your module path should generally begin with a URL prefix that you control — either your own domain name, or a unique path such as github.com/$USER/$REPO.
I had some problems working with local packages myself.
There are two tricks to make it work:
you run "go build" in the package directory
This compiles the package and places it in the build cache.
This link about code organisation in go explains more.
You can identify where the cache is using:
>go env GOCACHE
/home/<user>/.cache/go-build
Import using a path relative to the project
I puzzled loads over what the correct import path was and finally discovered that go doc or go list will tell you.
>go doc
package docs // import "tools/src/hello/docs"
>go list
tools/src/hello/docs
For example. I have a hello world API project and was using swaggo to generate documentation which it does in a docs sub-directory.
To use it I add an import:
_ "tools/src/hello/docs"
For my case the _ is important as docs is not used directly but we its init() function to be invoked.
Now in hello/main.go I can add "tools/src/hello/docs" and it will import the correct package.
The path is relative to the location of go.mod if you have one.
I have tools/ here as I have a go.mod declaring "modules tools".
Modules are a different kettle of fish - see https://github.com/golang/go/wiki/Modules.
Recent versions of go (1.11 and later) can create a go.mod file which you may use to fix the version of a module that is used and avoid go's crazy default behaviour of just downloading the latest version of any package you import.
I have written a blogpost on how to start your first Go project using modules.
https://marcofranssen.nl/start-on-your-first-golang-project/
In general it boils down to just create a new folder somewhere on your system (doesn't have to be in GOPATH).
mkdir my-project
cd my-project
go mod init github.com/you-user/my-project
This will create the go.mod file. Now you can simply create your project layout and start building whatever you like.
Maybe one of my other blogs can inspire you a bit more on how to do things.
https://marcofranssen.nl/categories/software-development/golang/

Recursive compile files

I just started with Go, and i love it! I have tried to make the structure of my project a bit more manageable, instead of having everything in my main.go
So now i have a structure like this.
src/
-> main.go
-> routes.go
-> handlers/
--> user_handlers.go
But when i try to build this with the following command
go build -v -o ./bin/my_bin ./src/...
I get this error
cannot use -o with multiple packages
But if i make it a flat structure like this
src/
-> main.go
-> routes.go
-> user_handlers.go
It works just fine, all my files got "package main" in the top of them.
What am i doing wrong?
The package name must match the directory name. Moving a source file to a new directory requires that you also change the package name.
foo/foo.go // package foo
foo/bar/bar.go // package bar
foo/bar/qux.go // package bar
The PATH is not relevant in terms of the package name.
Package foo: /some/path/some/where/foo
This allows multiple "foo" packages to be created and imported provided your import specifies the desired location of "foo"
P.S. The convention for package names is lowercase, no punctuation (e.g., no _'s)
It tells you what you did wrong, you can't separate a single package over multiple folders.
You need to set and properly use $GOPATH and properly import your routes/ folder in routes.go.
A simple example of it is:
// routes.go
// the . means you can call imported functions without prefixing them with the package name
import . "full-path-to-routes/-relative-to-$GOPATH"
From https://golang.org/doc/code.html:
The GOPATH environment variable specifies the location of your workspace. It is likely the only environment variable you'll need to set when developing Go code.
To get started, create a workspace directory and set GOPATH accordingly. Your workspace can be located wherever you like, but we'll use $HOME/go in this document. Note that this must not be the same path as your Go installation.
I highly recommend reading Effective Go.

Resources