Local packages and directory structure - go

This is the directory structure for my Go project:
my_project
|
main.go
- my_package
|
- my_package.go
On main.go this works ok:
import (
"my_package/my_package"
)
But when I create a new folder "examples" to test some new functionalities of the package, tha main.go cannot import the package as expected:
my_project
|
main.go
- my_package
|
- my_package.go
- examples
|
- main.go
importing package from parent directory:
import (
"../my_package/my_package"
)
Trying running examples/main.go does not work:
cd examples
go run main.go
build _/mypath/my_project/my_package: cannot find module for path _/mypath/my_project/my_package
Shouldn't I import local packages from parent directories?
Is it always compulsory to have the package modules on subfolders of main.go?
Is there any simple and clear way to organize main/test/debug go programs w/ shared dependencies on local packages?
I've read on golang modules and local packages that maybe I should use abosulte paths for importing packages, but I don't like that option because this is a code meant to be uloaded to a repository, so absolute paths wouldn't work on other implementations.

The way to proceed according to stackoverflow masters (and it works perfectly well indeed) is to import a fake repository and then replace it with a local redirection as stated on How to use a module that is outside of "GOPATH" in another module?
In main.go:
import (
my_package "my_fake_repository/myself/my_project/my_package"
)
In go.mod:
require my_fake_repository/myself/my_project/my_package v0.0.0
replace my_fake_repository/myself/my_project/my_package => ../my_package
It works! Thx a lot!

Related

Can't import local project from other project: cannot find module providing package <package_name>

I have a project I am working on named project1.
Couple of months ago I worked on project2 which contains package named engine and I want to use it inside project1.
These projects are local so I read here how to import local projects and tried it but received an error:
Cannot resolve file `project2`
One interesting thing is that when I type the name of project2 in the import(..) section, Goland identify it as module but after I press on it I received the error that it can't be resolve.
With Goland I have an option to run sync packages of 'project1' but when I pressed on it I also received an error:
project1/pkg/utils imports
project2: cannot find module providing package project2
I also tried to create vendor folder in project1 and copy-paste the whole project2 beneath the vendor folder but it still didn't help.
Any idea why it doesn't being resolve ?
If you have both of your projects under your $GOPATH, you can check out this example for importing projects.
EDIT:
If you are using go modules and want to import local modules, then you can make use of the replace directive.
So, basically you have to add in your go.mod of your Project1 these lines:
require /$module-name-project2/$package-name v0.0.0
replace $module-name-project2/$package-name => ../$localpath-to-project2
More info here
A quick example (both my projects are outside of $GOPATH and using go modules):
Project1 is located under .../go-experiments/project1
main.go:
package main
import "go-experiments/project2/greeting"
func main() {
println("How to greet?")
greeting.English()
}
go.mod:
module go-experiments/project1
require go-experiments/project2/greeting v0.0.0
replace go-experiments/project2/greeting => ../project2
go 1.14
Project2 is located under .../go-experiments/project2
greeter.go:
package greeting
func English() {
println("hi, i am boo")
}
go.mod:
module go-experiments/project2
go 1.14

Go 1.12 modules: local import in non-local import

I'm currently using Go 1.12 modules and really tired about importing.
I'm making the rest-api using gin(web microservices) and gorm(golang orm). Everything still ok while working in golang modules. But getting trouble with local packages importing
The directory tree:
The go.mod:
module github.com/Aragami1408/go-gorm
go 1.12
require (
github.com/gin-gonic/gin v1.4.0
github.com/jinzhu/gorm v1.9.9
github.com/lib/pq v1.1.1
github.com/satori/go.uuid v1.2.0
)
The db.go:
package db
//code below...
The tasks.go:
package task
import (
"../db"
)
But when I run and still get this error:
local import "../db" in non-local package
I've searched a lot on google and nothing helps
From "Do modules work with relative imports like import "./subdir"?"
In modules, there finally is a name for the subdirectory.
If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".
In your case:
import "github.com/Aragami1408/go-gorm/db"
# or maybe
import "go-gorm/db"
This assumes, as commented below by Silvio Lucas, that you have set GO111MODULE=on.
If you are using go modules you can replace your package by a local one using:
go mod edit -replace github.com/username/project=/localpath
then just call
go get github.com/username/project
and everything should work fine.
When migrating to go as "package manager", you may create the file go mod using the command :
go mod init myhost/myrepo/mymodule
Then the file will be created go.mod will be created :
module myhost/myrepo/mymodule
go 1.15
Now you can leverage this file to list your dependencies to other modules :
# i.e: your module mymodule depennds on github.com/gorilla/mux
got get github.com/gorilla/mux
You run it ? then check again the content of go.mod
module myhost/myrepo/mymodule
go 1.15
require (
github.com/gorilla/mux v1.7.4
)
You are happy because you leverage the package manager features and you are managing dependencies like a Boss.
However,....
However, you forget that you need to maintain all go files which imports directories with relative paths.
example :
if you have main.go
package main
import (
"fmt"
"./router" // !! RELATIVE PATH
)
You must migrate also by replacing the relative path by [module-name]/relative-path. in this case, it must become :
package main
import (
"fmt"
"myhost/myrepo/mymodule/router" // !! 💼 No more RELATIVE PATH
)

How do I resolve "cannot find module for path X" importing a local Go module?

In my Go project, I want to break out some generic functionality into a Go module, separate from the main project. I'm doing this outside of GOPATH in keeping with go's future. I don't want to publish the module on GitHub or anywhere else.
All my attempts to import this module into the main project result in:
cannot find module for path X
I've run go mod init X in the module's folder. The contents of its go.mod file is:
module X
Building or installing this module seems to do nothing. I've found no sign of it in $GOPATH/pgk/mod.
I've tried a variety of import statements:
import "X"
import "../x" (relative path to the module directory)
import "../x/X" (path to the directory + module name)
Help!
So you wrote a Go "library" module X which:
you don't want to publish on GitHub or elsewhere
you want to import and use in your project (the "main" module).
Use a replace directive along with require
In your main module's go.mod, add the following lines:
require "X" v0.0.0
replace "X" v0.0.0 => "{local path to the X module}"
The path should point to the root directory of X. It can be absolute or relative.
To import package util from module X:
import "X/util"
(You don't import modules. You import packages from modules.)
Explanation
Go's module functionality is designed for publicly published modules. Normally, a module's name is both its unique identifier and the path to its public repo. When your go.mod declares a module dependency with the require directive, Go will automatically find and retrieve the specified version of the module at that path.
If, for example, your go.mod file contains require github.com/some/dependency v1.2.3, Go will retrieve the module from GitHub at that path. But if it contains require X v0.0.0, "X" isn't an actual path and you will get the error cannot find module for path X.
The replace directive allows you to specify a replacement path for a given module identifier and version. There are many reasons you'd want to do this, such as to test changes to a module before pushing them to the public repo. But you can also use it to bind a module identifier to local code you don't ever intend to publish.
More details in the Go Modules documentation:
Can I work entirely outside of VCS on my local filesystem?
When should I use the replace directive?
Hope this helps.
If you don't want to use Go modules, you don't need to. As of Go v1.13, by default, go modules are used. Therefore, you need to tell explicitly if you don't want to do this.
Example:
main.go:
package main
import (
"fmt"
"./pakk"
)
func main() {
fmt.Println("Hello" + pakk.World())
}
pakk/pakk.go:
package pakk
func World() string {
return " world"
}
In this case, running go run main.go will output
build command-line-arguments: cannot find module for path .../pakk
but running
GO111MODULE=off go run main.go
will output
Hello world
Spent 4 days figuring this out. Very disapointed, but the way to import ./X
is as following:
first issue this magic command:
go mod init poop/strawberry/monkey
then you can EASILLY import your ./X folder:
import (
"poop/strawberry/monkey/X"
)
Folder structure:
./X/X.go
./main.go
./go.mod
contents of go.mod:
module poop/strawberry/monkey
this is the awesome solution from go module creators
https://go.dev/ref/mod#go-mod-init
module path: A path that identifies a module and acts as a prefix for package import paths within the module. For example, "golang.org/x/net"
go mod init api.com
// "api.com" name your app , it's alike swift 's bundle identifier , your can also "whatever.youlike"
myExample
Error ? Go cannot resolve the path to your modules, this is probably from misconfiguration or (not configuring) of the "go.mod" file used for project dependency tracking.
Solution lets assume your project folder looks like below;
/
|-- folderOne/
|-- folderTwo/
|-- folderThree/
|-- main.go
And the main.go script imports the modules folderOne,folderTwo and folderFour's script (folderfour.go) imports the module folderThree.
folderOne:
execute in the commandline:
go mod init github.com/../folderOne (i.e path from github.com folder to folderOnes)
The go mod init command creates a go.mod file to track your code's dependencies
folderTwo:
execute in the commandline:
go mod init github.com/../folderTwo (i.e path from github.com folder to folderTwos)
The go mod init command creates a go.mod file to track your code's dependencies
folderThree:
execute in the commandline:
go mod init github.com/../folderThree (i.e path from github.com folder to folderThrees)
The go mod init command creates a go.mod file to track your code's dependencies
folderFour:
execute in the commandline:
go mod init github.com/../folderThree (i.e path from github.com folder to folderFour)
Go to the folderFours script and import the module folderThree i.e
import "github.com/../folderThree"
in folderfours commandline:
$ go mod edit -replace github.com/{pathto}/folderThree=./folderThree**
then execute: go mod tidy
in your projects root folder execute the command: go mod init github.com/../ProjectRootDirectory (i.e path from github.com folder to ProjectRootDirectory)
then to import the modules, i.e folderThree, folderTwo, folderOne
execute the following at the projects root folder(ie folder with main.go)
$ go mod edit -replace github.com/{pathto}/folderOne=./folderOne
$ go mod edit -replace github.com/{pathto}/folderTwo=./folderTwo
$ go mod edit -replace github.com/{pathto}/folderFour=./folderFour
then execute;
$ go mod tidy
and then
$ go run main.go

How to import local packages in go?

I am new to go and working on an example code that I want to localize.
In the original main.go import statement it was:
import (
"log"
"net/http"
"github.com/foo/bar/myapp/common"
"github.com/foo/bar/myapp/routers"
)
Now I have common and routers package in /home/me/go/src/myapp
So I converted the import statement to:
import (
"log"
"net/http"
"./common"
"./routers"
)
But when I run go install myapp I get these errors:
can't load package: /home/me/go/src/myapp/main.go:7:3: local import "./common" in non-local package
Also, when I use common and routers instead of ./common and ./routers in the import statement, I get:
myapp/main.go:7:3: cannot find package "common" in any of:
/usr/local/go/src/common (from $GOROOT)
/home/me/go/src/common (from $GOPATH)
myapp/main.go:8:2: cannot find package "routers" in any of:
/usr/local/go/src/routers (from $GOROOT)
/home/me/go/src/routers (from $GOPATH)
How can I fix this?
Well, I figured out the problem.
Basically Go starting path for import is $HOME/go/src
So I just needed to add myapp in front of the package names, that is, the import should be:
import (
"log"
"net/http"
"myapp/common"
"myapp/routers"
)
If you are using Go 1.5 above, you can try to use vendoring feature.
It allows you to put your local package under vendor folder and import it with shorter path.
In your case, you can put your common and routers folder inside vendor folder
so it would be like
myapp/
--vendor/
----common/
----routers/
------middleware/
--main.go
and import it like this
import (
"common"
"routers"
"routers/middleware"
)
This will work because Go will try to lookup your package starting at your project’s vendor directory (if it has at least one .go file) instead of $GOPATH/src.
FYI: You can do more with vendor, because this feature allows you to put "all your dependency’s code" for a package inside your own project's directory so it will be able to always get the same dependencies versions for all builds. It's like npm or pip in python, but you need to manually copy your dependencies to you project, or if you want to make it easy, try to look govendor by Daniel Theophanes
For more learning about this feature, try to look up here
Understanding and Using Vendor Folder by Daniel Theophanes
Understanding Go Dependency Management by Lucas Fernandes da Costa
I hope you or someone else find it helpfully
You should have created your package with go mod init e.g. go mod init github.com/my-org/my-package
Now in my-package you have a sub module called utils for example.
main.go
utils
|- randstr.go
And your randstr.go looks like this:
package utils
func RandStr(n int) string {
// TODO: Generate random string....
return "I am a random string"
}
And then anywhere in your project you would use exported functions from the utils package like this, for example in main.go:
package main
import (
"fmt"
// "github.com/my-org/my-package" is the module name at the
// top of your `go.mod`
"github.com/my-org/my-package/utils"
)
func main() {
fmt.Printf("Random string: %s\n", utils.RandStr(20))
}
Import paths are relative to your $GOPATH and $GOROOT environment variables. For example, with the following $GOPATH:
GOPATH=/home/me/go
Packages located in /home/me/go/src/lib/common and /home/me/go/src/lib/routers are imported respectively as:
import (
"lib/common"
"lib/routers"
)
an example:
in ./greetings, do go mod init example.com/greetings
from another module, do go mod edit -replace=example.com/greetings=../greetings
go get example.com/greetings
from the go tutorial
Local package is a annoying problem in go.
For some projects in our company we decide not use sub packages at all.
$ glide install
$ go get
$ go install
All work.
For some projects we use sub packages, and import local packages with full path:
import "xxxx.gitlab.xx/xxgroup/xxproject/xxsubpackage
But if we fork this project, then the subpackages still refer the original one.
Follow instructions here https://go.dev/doc/tutorial/call-module-code
Mainly you need the replace call in your go.mod file.
module example.com/hello
go 1.16
replace example.com/greetings => ../greetings
As in the question, the folder structure is:
/home/me/go/src/myapp
└─ common
└─ routers
So go to myapp dir
cd /home/me/go/src/myapp
Do
go mod init myapp
This will create a go.mod file which lets Go know the name of the module myapp so that when it’s looking at import paths in any package, it knows not to look elsewhere for myapp
Then you can do the following in the code:
import (
"log"
"net/http"
"myapp/common"
"myapp/routers"
)
Now package common and routers gets imported.
Another approach, available since go1.18, is to use a go.work file.
First, the local common package has to be a module, so provide a go.mod file inside the common folder:
module common
go 1.18
You can now create a go.work file in the root of your directory manually or call go work init, then go work use . and finally go work use ./common. It will look like this:
go 1.18
use (
.
./common
)
Finally you can import the package in your code by name
package main
import "common"
Just remember to not commit your go.work files :)
The key is how you name your module in the following command
go mod init <TheNameGiven>
Then refer the modules in the inner folder with,
TheNameGiven/folder
I have found the best solution here... Read More
Try to change the package name with the go mod init command.
So, I have go 1.17, and I have the same import problem. My project directory is $GOPATH/src/myswagger/app-swagger-test. I ran this command into app-swagger-test dir:
go mod init app-swagger-test
go mod tidy
In my new go.mod file the package name is app-swagger-test. For example, this import was wrong:
import (
...
"myswagger/app-swagger-test/internal/generated/restapi"
"myswagger/app-swagger-test/internal/generated/restapi/operations"
)
So I removed go.mod and go.sum. And I ran next commands into app-swagger-test dir:
go mod init myswagger/app-swagger-test
go mod tidy
After that all imports in the project were imported successfully. In the new go.mod file the first line is:
module myswagger/app-swagger-test
Maybe this information is common, but I did not find it. Thanks!

Does it make sense to have two packages in the same directory?

I have a project that provides a library (exports some funcs) and also must provide a command-line interface (there must be an executable file).
Example of directory structure:
whatever.io/
myproject/
main.go
myproject.go
The go compiler needs the package main and func main to start execution. My library needs the package myproject where I put stuff on it. This is what the go tool says when I am building another project that tries to import myproject:
main.go:5:2: found packages myproject (myproject.go) and main (main.go) in $GOPATH/src/whatever.io/myproject
So I believe there is no way to do it.
Should I move the library or the CLI to another package?
Just move your packages inside a new folder within the same directory of main.go.
Remember to import the new package from the reference of the $GOPATH.
Example:
user#user:~/p/go/test/so-multipack$ ls -R
.:
a main.go
./a:
a.go
user#user:~/p/go/test/so-multipack$ cat main.go
package main
import (
"../so-multipack/a"
)
func main(){
a.Hello()
}
user#user:~/p/go/test/so-multipack$ cat a/a.go
package a
import (
"fmt"
)
func Hello(){
fmt.Println("hello from a")
}
user#user:~/p/go/test/so-multipack$ go run main.go
hello from a
user#user:~/p/go/test/so-multipack$ go build
user#user:~/p/go/test/so-multipack$ ls
a main.go so-multipack
user#user:~/p/go/test/so-multipack$
Useful link:
go build vs go build file.go
You cannot have two packages per directory, hence the error. So the solution as #Larry Battle said to move your myproject.go to a new directory.
From How to write go code
Go code must be kept inside a workspace. A workspace is a directory
hierarchy with three directories at its root:
src contains Go source files organized into packages (one package per directory),
pkg contains package objects, and
bin contains executable commands.
In most cases, no. However, there is an exception for unit tests.
Working Example:
Here are 2 different packages (mypackage and mypackage_test) in 1 directory (mypackage). The compiler will not complain about this.
mypackage folder:
mypackage/
foo.go
foo_test.go
mypackage/foo.go:
package mypackage
func Add(a int, b int) int {
return a + b
}
mypackage/foo_test.go:
package mypackage_test
// Unit tests...
Rules:
The 2 packages must have the following names:
NameOfDirectory.
NameOfDirectory + _test.
The names of the files in the _test package must end with _test.go
If you're receiving a confusing compiler error along the lines of found packages "foo" and "bar", you've probably broken one or more of these rules.
You can't have two golang files in one directory with two packages. So you need to move main.go out of myproject.
the directory structure before move
whatever.io/
go.mod
myproject/
main.go
myproject.go
After move
whatever.io/
go.mod
main.go
myproject/
myproject.go
And you need to change your main.go's import path. If the module name is aaa
Before
import "aaa"
Need change to this
import "aaa/myproject"

Resources