My folder structure looks something like this... (say my git repo name is demorepo)
demorepo
|--directory1
|-- (no go.mod at this level)
|--module1
|--package1 --------->--------------->--------------------->----|
|--go.mod (github.com/demorepo/directory1/module1) |
|--go.sum |
|--module2 |
|--module3 |
|--directory2 |
|-- (no go.mod at this level) |
|--newmodule ------<------------<------------------<----------------|
Now, I want to use a function defined in "package1" in my "newmodule"
When I hit
go get <repo_address>/directort1/module1/package1 at "new_module"
it says ....
github.com/<repo>#upgrade found (v0.0.0-20211215055943-92e412ad4a12), but does not contain package github.com/<repo>/directory1/module1/package1
There is a proposal for a Go Workspace File for Go 1.18 which should simplify this task.
Meanwhile, you can use the replace directive in your go.mod file to refer to a module located on a local filesystem.
demorepo/directory1/module1/go.mod:
module github.com/<repo>/directory1/module1
demorepo/directory2/newmodule/go.mod:
module github.com/<repo>/directory2/newmodule
replace github.com/<repo>/directory1/module1 => ../../directory1/module1
Now you can import github.com/<repo>/directory1/module1/package1 in newmodule normally and it'll refer to the local module1.
You might not want the replace directive in the go.mod file itself, and instead make a copy of it e.g. go.mod.local and use it when building your project: go build -modfile go.mod.local . (can also add *.local to .gitignore).
Approach :
If you want to use module1 inside newModule then you should make one new repo for module1 put your logic there push it in github. please make sure you should use appropriate version for the library.
import it as a library and it will work.
Also refer the official docs of module dependency and also check root level dependency.
Official docs : https://go.dev/blog/using-go-modules
Related
I ran into a repo that seems to be a Go module, but there's no go.mod file in it: github.com/confluentinc/confluent-kafka-go.
Is it ok for a go module to have no go.mod file with dependencies, or the authors of that library just didn't migrate to modules yet?
Dependency modules do not need to have explicit go.mod files.
The “main module” in module mode — that is, the module containing the working directory for the go command — must have a go.mod file, so that the go command can figure out the import paths for the packages within that module (based on its module path), and so that it has a place to record its dependencies once resolved.
In addition, any modules slotted in using replace directives must have go.mod files (in order to reduce confusion due to typos or other errors in replacement paths).
However, in general a module that lacks an explicit go.mod file is valid and fine to use. Its effective module path is the path by which it was required, which can be a bit confusing if the same repository ends up in use via multiple paths. Since a module with no go.mod file necessarily doesn't specify its own dependencies, consumers of that module will have to fill in those dependencies themselves (go mod tidy will mark them as // indirect in the consumer's go.mod file).
SHORT SUMMARY OF THE DISCUSSION:
The answer is "No"!
This project contains a set of go packages, but it is not a Go module as it doesn't contain go.mod file (although, it used to be a multi-module repo (Go) previously).
go get can run in both ways: module-aware mode and legacy GOPATH mode (as of Go 1.16).
To read more about this, refer to docs by using the go command:
$ go help gopath-get
and
$ go help module-get
It'd tell about how go get works in both cases.
Also, I noticed that it can download any repository and would treat it as a Go package, even if it contains an arbitrary Python project.
I did a simple test to demonstrate the same:
$ go get github.com/mongoengine/mongoengine
And it surprisingly worked.
Modules are defined by their go.mod file. Without a go.mod file, it is not a module.
See this from the Go Modules Reference
A module is a collection of packages that are released, versioned, and distributed together. Modules may be downloaded directly from version control repositories or from module proxy servers.
A module is identified by a module path, which is declared in a go.mod file, together with information about the module's dependencies. The module root directory is the directory that contains the go.mod file.
And
A module is defined by a UTF-8 encoded text file named go.mod in its root directory.
Just 3 days experience in Go language. Hope an example will be more apt to understand my confusion.
root#freebsd1:/usr/home/arun/go-start/src/test2 # go mod init f1/f2/f3/f4/f5/hello
go: creating new go.mod: module f1/f2/f3/f4/f5/hello
root#freebsd1:/usr/home/arun/go-start/src/test2 #
Here in above example go mod init is creating all these folders(f1/f2/f3/f4/f5/hello)?. I searched a lot, couldn't find any such folders in system. Then what is the significance of this path.
Even though below command will not run if this path is not mentioned as it is
# go install f1/f2/f3/f4/f5/hello
--:EDIT:--
May be this will help someone later ( Just walk through the steps to understand this in a proper way, especially for newbies )
I am planning to create a program 'calculator' and will upload in GitHub later.
I will keep the functions in different packages like sum,multiply etc
first step #go mod init github.com/go-arun/calculator( Don't confuse here , this is just an assumption that, in future I may create a repository in github for this project )
created folder sum(one of the package folder , and created sum.go inside )
See those in by system:
1.
root#debian1:/home/arun/lab# go mod init github.com/go-arun/calculator
go: creating new go.mod: module github.com/go-arun/calculator
root#debian1:/home/arun/lab# cat go.mod
module github.com/go-arun/calculator
go 1.15
2.
root#debian1:/home/arun/lab# cat sum/sum.go
package sum
import "fmt"
func Sum(num1,num2 int)(){
fmt.Println(num1+num2)
}
3.
root#debian1:/home/arun/lab# cat main.go
package main
import(
"github.com/go-arun/calculator/sum"
)
func main(){
n1 := 10
n2 := 10
sum.Sum(n1,n2)
}
4.
root#debian1:/home/arun/lab# tree
.
|-- go.mod
|-- main.go
`-- sum
`-- sum.go
go mod init does not create those folders. You pass the "module path" to go mod init which is recorded in the go.mod file it creates.
The "module path" is the import path prefix corresponding to the module root. The module path and the relative path to the module root together form the complete import path which must be unique in an app.
So for example if your module contains a folder named foo (and a package foo in it), it is imported by a path being modulepath/foo. In your case it would be f1/f2/f3/f4/f5/hello/foo.
It is allowed for moduleA to contain a foo package, and also for moduleB to have a foo package. When used / imported, first would be imported like moduleA/foo the latter like moduleB/foo, so it's unambiguous which one you're importing. The module path is like a namespace.
It's recommended to use a module path that corresponds to a repository you plan or will publish your module to, so when you do, go get will be able to automatically fetch, build and install your module. For example you may choose a module path github.com/bob/hello, so when you publish your module, everyone can get it by simply using import "github.com/bob/hello" in their app.
Also note that you don't need to publish your code to a remote repo before you can build it. But it's still recommended to follow this pattern so you'll have less work to make it work in the future if you decide to publish it. Nothing to lose here.
More in the docs: Command go: Defining a module
Also: How to Write Go Code: Code organization
I can not find a way to factor out some code from main.go into a local package when using Go modules (go version >= 1.11) outside of $GOPATH.
I am not importing any external dependencies that need to be included into go.mod, I am just trying to organize locally the source code of this Go module.
The file main.go:
package main
// this import does not work
import "./stuff"
func main() {
stuff.PrintBaz()
}
The file ./stuff/bar.go (pretending to be a local package):
package stuff
import "log"
type Bar struct {
Baz int
}
func PrintBaz() {
baz := Bar{42}
log.Printf("Bar struct: %v", baz)
}
The file go.mod (command go mod init foo):
module foo
go 1.12
When executing go run main.go:
If I import "./stuff", then I see build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
If I import "stuff", then I see build command-line-arguments: cannot load stuff: cannot find module providing package stuff.
If I import stuff "./stuff" with a package alias, then I see again: build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
I can not find a way to make local packages work with go modules.
What's wrong with the code above?
How can I import local packages into other Go code inside a project defined with Go modules (file go.mod)?
Module structure
The most common and easiest approach is:
Use a single go.mod per repository, and
Place the single go.mod file in the repository root, and
Use the repository name as the module path declared in the module line in the go.mod
(If you are using a custom import path such as me.io/mymod rather than using a VCS host based import path, then you would use the custom import path instead of the repository name in your go.mod).
For example, if your repo is github.com/my/repo, then you would place a single go.mod in the repo root, with the first line reading module github.com/my/repo. That can be created by cd'ing to the repo root and running go mod init github.com/my/repo. When working with the repo locally, the repo directory can be located wherever is convenient on your filesystem. (If you do not have a repo, see below).
Following this helps you stay on the happy path with modules, and it avoids multiple subtleties.
Russ Cox commented in #26664:
For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.
There is much more about multi-module repositories in the "Multi-module Repositories" FAQ section on the modules wiki. Those 6 or so FAQs in that section should be read in their entirety by anyone considering veering off the recommendation above.
Arranging packages within a module
Once you have set up your go.mod, you can arrange your packages in directories however you see fit in directories underneath the directory containing the go.mod, as well as in the directory with the go.mod. Three good articles about how to arrange your code in packages:
https://rakyll.org/style-packages/
https://medium.com/#benbjohnson/standard-package-layout-7cdbc8391fc1#.ds38va3pp
https://www.goinggo.net/2017/02/design-philosophy-on-packaging.html
Those articles are classics that pre-date the introduction of modules, but the philosophies in them still apply to how to arrange your packages within a module.
Importing other packages in the same module
When importing another package with modules, you always use the full path including the module path. This is true even when importing another package in the same module. For example, if a module declared its identity in its go.mod as module github.com/my/repo, and you had this organization:
repo/
├── go.mod <<<<< Note go.mod is located in repo root
├── pkg1
│ └── pkg1.go
└── pkg2
└── pkg2.go
Then pkg1 would import its peer package as import "github.com/my/repo/pkg2". Note that you cannot use relative import paths like import "../pkg2" or import "./subpkg". (This is part of what OP hit above with import "./stuff").
Modules vs. repositories vs. packages vs. import paths
A Go module is a collection of related Go packages that are versioned together as a single unit. Modules record precise dependency requirements and create reproducible builds.
Summarizing the relationship between repositories, modules, and packages:
A repository contains one or more Go modules (most often exactly one module in the repository root).
Each module contains one or more Go packages.
Each package consists of one or more Go source files that all reside in a single directory.
Each Go source code file:
declares its own package with a package foo statement, which must be consistent for all Go files in a single package directory.
automatically has access to other Go source code in the same package, without explicitly importing its own package.
imports code from another package via an import path supplied in an import statement such as import "github.com/my/repo/pkg1". The import path always starts with the module path of that package, regardless of whether that package is in the same module or a different module.
A single module can be used to build multiple executable binaries, but the func main() {...} entry point for each binary must be in its own package main, with each package main in a distinct directory.
Local-only modules
You can create, build and run modules without ever publishing them or placing them on a VCS server.
A module is defined by a tree of Go source files with a go.mod file in the tree's root directory, so you can pick a directory on your local filesystem to serve as the root of your module and place your go.mod there. Everything described above still applies, except there does not need to be a repo.
For example:
some-directory/
├── go.mod <<<< go.mod at module root declaring 'module example.com/foo'
├── pkg1
│ └── pkg1.go
└── pkg2
└── pkg2.go
If the module path in the first line of that go.mod reads module example.com/foo, then pkg1 would import its peer package as import "example.com/foo/pkg2".
Of course, you can start with a local-only module without a repo, then later start tracking it in a local repo or publish it to a VCS server.
Even for local-only modules, it is a good idea to give the module a globally unique name, such as go mod init my.org/foo, or go mod init github.com/some/repo as if it will be published someday, even if that repo does not yet exist. (If you are in a hurry, it is possible to use a shorter module name without a dot or leading hostname component, such as go mod init temp or go mod init m, but it won’t be fetchable by the go command, and you run the risk of colliding with a current or future standard library package or otherwise seeing confusing error messages later).
If you have multiple local modules, you can point them at each other via replace directives or via the Go 1.18+ workspace feature, though those are more complex workflows than placing everything for a given project in a single module.
Next steps
If you are new to Go modules, it is worthwhile to go through the official "Create a Go Module" tutorial from the Go project. This will likely save you time overall.
If you are stuck right now with an error message related to modules or import paths, there's a very good chance the error will make much more sense after completing the tutorial.
First you have to choose a name for your project and write it to go.mod file. This name belongs to root directory of the project. Each new package you create must be located inside its own subdirectory and its name should match directory name.
go.mod:
module myprojectname
or (preferred way, see #thepudd's answer for details)
module github.com/myname/myproject
Then import your project's packages like:
import myprojectname/stuff
or
import github.com/myname/myproject/stuff
Files of package stuff should be located inside project's stuff directory. You name these files as you like.
Also it's possible to create deeper project structure. For instance, you decided to separate source code files from other ones (like app configs, docker files, static files, etc...). Let's move stuff directory inside pkg, every go file inside pkg/stuff still have stuff package name. To import stuff package just write:
import myprojectname/pkg/stuff
Nothing stops you from creating more levels in the hierarchy like github.com/myuser/myproject/pkg/db/provider/postgresql, where:
github.com/myuser/myproject - project name.
postgresql - package name.
pkg/db/provider/postgresql - path to the package relative to project's root.
You can read more about go modules here: https://github.com/golang/go/wiki/Modules
Check out this repository to get useful information about various patterns used in project organizing: https://github.com/golang-standards/project-layout If you go inside pkg directory you will find out which open source projects use pkg directory in their structure.
I'd like to use a helper package from Go Cloud Function. The package has some helper logic that can be shared between multiple functions. But, what is the right way to structure the packages so they all work? The package should be in the same project - not published and public as a completely separate package.
I work at Google. The goal for this question is to proactively answer common questions and help developers starting off with Go on GCF.
You can use subpackages with Go modules. Go modules are Go's new dependency management solution - they let you work outside of GOPATH and let you manage the exact versions of each dependency you have.
Modules also let you define a group of Go packages with the same import path prefix. When you're writing a function, this lets you import other packages in your module.
The function you're deploying needs to be at the root of your module.
Here is an example file structure and how packages would be imported:
.
├── cmd
│ └── main.go # Useful for testing. Can import and setup your function.
├── function.go # Can import example.com/foo/helperpackage
├── function_test.go
├── go.mod # module example.com/foo
└── helperpackage
└── helper.go
This setup has your function in function.go and tested by function_test.go. They are in a module named example.com/foo. helperpackage can be imported by function.go using example.com/foo/helperpackage.
This also has a cmd directory, which may be helpful for local testing. You can import example.com/foo and start an HTTP server which registers your function as an HTTP handler. For example:
package main
import (
"log"
"net/http"
"example.com/foo"
)
func main() {
http.Handle("/HelloHTTP", foo.HelloHTTP)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Note: You could use a vendor directory to achieve the same result. But, all of the packages your function imports would need to be in the vendor directory (with the full import path), which works, but is cumbersome to maintain. It's uncommon to copy sub-packages into your vendor directory, so I wouldn't recommend this.
We also ran into this issue. The go.mod approach provided by #TylerBui-Palsulich (https://stackoverflow.com/a/54255486/2307346) didn't work as we also had to download dependencies from a private repository.
From the Google documentation:
If your function's dependencies are hosted in a repository that is not publicly accessible, you must use a vendor directory to fetch your dependencies before deploying your function
(https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)
Given the following package structure:
cloudfunctions.go
sharedpackage
-> shared.go
go.mod
In our go.mod file we defined the following namespace: module some/namespace
The cloudfunctions.go file has the following package definition and imports
package foobar
import (
"bitbucket.org/some/private/dependency"
"some/namespace/sharedpackage"
)
Because of the private dependency we cannot use the go.mod file. Instead we provide a vendor directory.
Warning: If you have both a go.mod file and a vendor directory at the root of your project, the contents of the vendor directory will be ignored when your function is built in the cloud. To ensure that your vendor directory is used, you must exclude the go.mod file from your project's source code prior to deployment. If you are using the gcloud command-line tool, you can ensure that go.mod is not uploaded by using .gcloudignore.
(https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)
gcloud deploy will ignore the vendor directory if it finds go.mod, we fix this with a .gloudignore file with the following content:
go.mod
go.sum
At the end we have the following file structure:
cloudfunctions.go
sharedpackage
-> shared.go
go.mod
vendor
.gcloudignore
The last step, Fixing the cannot find package error
When running the deploy step you will still run into an error similar to:
(gcloud.functions.deploy) OperationError: code=3, message=Build failed: /tmp/sgb/gopath/src/serverlessapp/vendor/some/namespace/cloud_functions.go:5:2: cannot find package "some/namespace/sharedpackage" in any of:...
This is because the go.mod file, containing the module name, is ignored. now the Go compiler no longer knows that some/namespace/sharedpackage refers to the local sharedpackage directory.
We managed to get it working by changing the module name to match the package name:
Change module name in go.mod to module foobar
Change the imports in cloudfunctions.go to: "foobar/sharedpackage":
package foobar
import (
"bitbucket.org/some/private/dependency"
"foobar/sharedpackage"
)
Now the Go compiler is able to detect that foobar/sharedpackage is a subpackage of the foobar package.
I'm trying to understand how to organize my golang project using go1.11 modules. I tried several options, but none of them worked.
I have some code in the main package under the application folder and a local package that the main package uses.
$GOPATH
+ src
+ application/
+ main/
+ main.go
+ otherFileUnderMainPackage.go
+ aLocalPackage/
+ someCode.go
+ someCode_test.go
+ someMoreCode.go
+ someMoreCode_test.go
Files in the main package, imports ../aLocalPackage. When I compile by go build main/*.go it's working.
Then, I ran go mod init application: V.0.9.9 and got the go.mod file, but the build always fails. I always get error about not finding the local package: build application:V0.9.9/main: cannot find module for path _/.../src/application/aLocalPackage. I also tried to place the local package right under src/, place it under main/ etc. but none of these methods worked for me.
What is the way to use modules and local packages?
Thanks.
Relative import paths are not supported in module mode. You will need to update your import statements to use a full (absolute) import path.
You should also choose a module name other than application. Your module path should generally begin with a URL prefix that you control — either your own domain name, or a unique path such as github.com/$USER/$REPO.
I had some problems working with local packages myself.
There are two tricks to make it work:
you run "go build" in the package directory
This compiles the package and places it in the build cache.
This link about code organisation in go explains more.
You can identify where the cache is using:
>go env GOCACHE
/home/<user>/.cache/go-build
Import using a path relative to the project
I puzzled loads over what the correct import path was and finally discovered that go doc or go list will tell you.
>go doc
package docs // import "tools/src/hello/docs"
>go list
tools/src/hello/docs
For example. I have a hello world API project and was using swaggo to generate documentation which it does in a docs sub-directory.
To use it I add an import:
_ "tools/src/hello/docs"
For my case the _ is important as docs is not used directly but we its init() function to be invoked.
Now in hello/main.go I can add "tools/src/hello/docs" and it will import the correct package.
The path is relative to the location of go.mod if you have one.
I have tools/ here as I have a go.mod declaring "modules tools".
Modules are a different kettle of fish - see https://github.com/golang/go/wiki/Modules.
Recent versions of go (1.11 and later) can create a go.mod file which you may use to fix the version of a module that is used and avoid go's crazy default behaviour of just downloading the latest version of any package you import.
I have written a blogpost on how to start your first Go project using modules.
https://marcofranssen.nl/start-on-your-first-golang-project/
In general it boils down to just create a new folder somewhere on your system (doesn't have to be in GOPATH).
mkdir my-project
cd my-project
go mod init github.com/you-user/my-project
This will create the go.mod file. Now you can simply create your project layout and start building whatever you like.
Maybe one of my other blogs can inspire you a bit more on how to do things.
https://marcofranssen.nl/categories/software-development/golang/