I have a template in package notifications
/notifications
/html/notify.tpl
The issue is that if I call the functions of this package from a different path/package it can't find the file anymore as the program is assuming the path belongs to the package that is calling.
e.g.
ls
main.go
/notifications
/html/notify.template
go run main.go
open html/notify.tpl: no such file or directory
You don't.
You either provide a path through configuration, use a well-known fully qualified path, use a path relative to your working directory (not recommended), or a combination of the three.
Related
Not so familiar with Golang, it's probably a stupid mistake I made... But still, I can't for the life of me figure it out.
So, I got a proto3 file (let's call it file.proto), whose header is as follows:
syntax = "proto3";
package [package_name];
option go_package = "github.com/[user]/[repository]";
And I use protoc:
protoc --go_out=$GOPATH/src --go-grpc_out=$GOPATH/src file.proto
So far so good, I end up with two generated files (file.pb.go and file_grpc.pb.go) inside /go/src/github.com/[user]/[repository]/, and they are defined inside the package [package_name].
Then, the code I'm trying to build has the following import:
import (
"github.com/[user]/[repository]/[package_name]"
)
And I naively thought it would work. However, it produces the following error when running go mod tidy:
go: downloading github.com/[user]/[repository] v0.0.0-20211105185458-d7aab96b7629
go: finding module for package github.com/[user]/[repository]/[package_name]
example/xxx imports
github.com/[user]/[repository]/[package_name]: module github.com/[user]/[repository]#latest found (v0.0.0-20211105185458-d7aab96b7629), but does not contain package github.com/[user]/[repository]/[package_name]
Any idea what I'm doing wrong here? Go version is go1.19 linux/amd64 within Docker (golang:1.19-alpine).
Note: I also tried to only import github.com/[user]/[repository], same issue obviously.
UPDATE:
OK so what I do is that I get the proto file from the git repository that only contains the proto file:
wget https://raw.githubusercontent.com/[user]/[repository]/file.proto
Then I generate go files from that file with protoc:
protoc --go_out=. --go-grpc_out=. file.proto
Right now, in current directory, it looks like:
- directory
| - process.go
| - file.proto
| - github.com
| - [user]
| - [repository]
| - file.pb.go
| - file_grpc.pb.go
In that same directory, I run:
go mod init xxx
go mod tidy
CGO_ENABLED=0 go build process.go
The import directive in process.go is as follows:
import (
"xxx/github.com/[user]/[repository]"
)
Now it looks like it finds it, but still getting a gRPC error, which is weird because nothing changed. I still have to figure out if it comes from the issue above or not. Thanks!
Your question is really a number of questions in one; I'll try to provide some info that will help. The initial issue you had was because
At least one file with the .go extension must be present in a directory for it to be considered a package.
This makes sense because importing github.com/[user]/[repository] would be fairly pointless if that repository does not contain any .go files (i.e. the go compiler could not really do anything with the files).
Your options are:
Copy the output from protoc directly into your project folder and change the package declarations to match your package. If you do this there is no need for any imports.
Copy (or set go_out argument to protoc) the output from protoc into a subfolder of your project. The import path will then be the value of the module declaration in your go.mod plus the path from the folder that the go.mod is in (this is what you have done).
Store the files in a repo (on github or somewhere else). This does not need to be the same repo as your .proto files if you "want it to be agnostic" (note that 2 & 3 can be combined if the generated files will only be used within one code base or the repo is accessible to all users).
Option 1 is simple but its often beneficial to keep the generated code separate (makes it clear what you should not edit and improves editor autocomplete etc).
Option 2 is OK (especially if protoc writes the files directly and you set go_package appropriately). However issues may arise when the generated files will be used in multiple modules (e.g. as part of your customers code) and your repo is private. They will need to change go_package before running protoc (or search/replace the package declarations) and importing other .proto files may not work well.
Option 3 is probably the best approach in most situations because this works with the go tooling. You can create github.com/[user]/goproto (or similar) and put all of your generated code in there. To use this your customers just need to import github.com/[user]/goproto (no need to run protoc etc).
Go Modules/package intro
The go spec does not detail the format of import paths, leaving it up to the implementation:
The interpretation of the ImportPath is implementation-dependent but it is typically a substring of the full file name of the compiled package and may be relative to a repository of installed packages.
As you are using go modules (pretty much the default now) the implementations rules for resolving package paths (synonym of import path) can be summarised as:
Each package within a module is a collection of source files in the same directory that are compiled together. A package path is the module path joined with the subdirectory containing the package (relative to the module root). For example, the module "golang.org/x/net" contains a package in the directory "html". That package’s path is "golang.org/x/net/html".
So if your "module path" (generally the top line in a go.mod) is set to xxx (go mod init xxx) then you would import the package in subfolder github.com/[user]/[repository] with import xxx/github.com/[user]/[repository] (as you have found). If you got rid of the intervening folders and put the files into the [repository] subfolder (directly off your main folder) then it would be import xxx/[repository]
You will note in the examples above that the module names I used are paths to repo (as opposed to the xxx you used in go mod init xxx). This is intentional because it allows the go tooling to find the package when you import it from a different module. For example if you had used go mod init github.com/[user]/[repository] and option go_package = "github.com/[user]/[repository]/myproto";" then the generated files should go into the myproto folder in your project and you import them with import github.com/[user]/[repository]/myproto.
While you do not have to follow this approach I'd highly recommend it (it will save you from a lot of pain!). It can take a while to understand the go way of doing this, but once you do, it works well and makes it very clear where a package is hosted.
I am trying to import a Proto file into another one from a different folder and haven't been successful in doing so. Here's the scenario:
I have a .Proto in folder ....\abc\protos\ProtoA.proto and another one in folder ....\def\protos\ProtoB.proto.
I need ProtoA to import ProtoB but it's in a different folder and using Import "....\def\protos\ProtoB.proto" doesn't work because it doesn't like "...." in the path.
What are the steps i need to follow to import the file in correctly from a different path?
It's confusing and I'm unsure whether it's effectively explained in the docs.
Proto file imports are absolute to the proto package and the package structure must be preserved in the filing system structure.
However, the absolute disk location is only important when using protoc per #Brits comment so that the compiler can find the protos.
So.... Your import for ....\def\protos\ProtoB.proto should reflect the specific package and service or method or message name that you're importing not its disk location (which is what you're using).
Then, when you protoc, you should --proto_path and give (I think absolute not relative) paths to the filing system locations that contain the protos needed to be imported.
Have a look at Any by way of example.
In a proto, you import "google.protobuf.Any", it's package plus the message name.
When you protoc it, Any is often already in the include path but, if it weren't, you'd need to --proto_path=/path/to/foo if foo is the root directory containing google/protobuf/any.proto; the proto file must be in a directory called protobuf in a directory called google for the import to work.
If you're familiar with Golang and GOPATH, this mirrors how Go packages are named by their directory (not file) name and referenced locally by their location being in the GOPATH; it's now different with Go Modules.
I am doing some light hacking on the rclone project, and I'm somewhat new to go. I have cloned the repository in my home directory, /home/poundifdef/rclone.
When I look at the file, rclone.go, in the top-level directory, I see the following import statement:
package main
import (
_ "github.com/ncw/rclone/backend/all"
"github.com/ncw/rclone/cmd"
_ "github.com/ncw/rclone/cmd/all"
)
My question is this: nowhere in this code, as far as I can tell, do we specify that the github.com/ncs/rclone/* packages are supposed to refer to the local versions of these directories. Nowhere in my filesystem have I used a directory called github.com/ncw. And yet, running go run rclone.go is able to execute this code.
Where is the path github.com/ncw/rclone being mapped to my local directory, given these packages, as named, do not exist?
github.com/ncw/rclone is a module, because it has a file named go.mod at the repository root. Since you have $GOPATH unset, the experimental module feature is enabled.
Where modules are placed in your filesystem is irrelevant. when constructing import paths, all directory names in the module are appended to the module name. The module name is defined in the first line of go.mod, here "github.com/ncw/rclone". So the package in ./cmd has the import path github.com/ncw/rclone/cmd.
Commands such as go build, go install, go test, etc. download all other required packages automatically to $HOME/go/pkg/mod, unless the -mod=vendor flag is specified, in which case the sources in vendor are used.
Golang has an internal mechanism to resolve import path.
This mechanism relies on the existence of several variables on your computer, GOROOT, GOPATH and for old releases GO15VENDOREXPERIMENT.
When the compiler meet a package path, it performs a look up with those paths
GOROOT/src/[import path]
GOPATH/src/[import path]
cwd/vendor/[import path]
The longest path wins and is selected for further processing.
GOROOT is the path to the stdlib.
GOPATH is the path to your local workspace.
For all the details https://go.googlesource.com/proposal/+/master/design/25719-go15vendor.md
This all depends on the setup of your computer, so carefully check go env output.
Every Go file starts with package <something>.
As far as I understand - and this is probably where I am missing some information - there are only two possible values for <something>: The name of the directory it is in*, or main. If it is main, all other files in that directory can only have main, too. If it is something else, the project is inconsistent/violating convention.
Now if it is the name of the directory, it's redundant, because the same information is, well, in the name of the directory.
If it is main, it's kind of useless, because as far as I can see there is no way to tell go build to "please build all main packages".
* Because, in other words, one directory is one package.
The name of the package does not have to coincide with the directory name. It is possible to have package foobar in the directory xyz/go-foobar. In this case, xyz/go-foobar becomes an import path, but the package name that you use to quality the identifiers (functions, types etc.) would be foobar.
Here's an example to make it more concrete: I created a test package http://godoc.org/github.com/dmitris/go-foobar (source in https://github.com/dmitris/go-foobar) - you can see from the documentation page, that the import path is "github.com/dmitris/go-foobar" but the package name is foobar, so you would call the function it provides as foobar.Demo() (not go-foobar.Demo()).
A similar real-life example - the import path for the NSQ Messaging platform is "github.com/nsqio/go-nsq" while the package name is "nsq": http://godoc.org/github.com/nsqio/go-nsq. However, for the sake of user-friendliness and simplicity, the standard and recommended practice is to keep the last portions of the import path and the package name being the same whenever possible.
package main is not useless - it tells the Go compiler to create an executable as opposed to a .a library file (with go install or go get; go build discards the compilation result). The executable is named after the directory name in which the package main file or files are placed. Again a concrete example - I made a test program https://github.com/dmitris/go-foobar-client, you install it with go get github.com/dmitris/go-foobar-client and you should get a go-foobar-client executable placed in your $GOPATH/bin directory. It is from the the directory name where the package main file is placed that the Go compiler takes the name of the executable from. The filename of the .go file that contains the main() function is not important - in the example above, we can rename main.go to client.go or something else, but as long as the enclosing directory is called go-foobar-client, that's how the resulting executable will be named.
For an additional accessible and practically oriented reading about Go packages, I recommend Dave Cheney's article "Five suggestions for setting up a Go project" http://dave.cheney.net/2014/12/01/five-suggestions-for-setting-up-a-go-project.
The missing information you "have" is that the package name does not need to be the same as the directory name.
It is perfectly fine to use a package name other than the folder name. If you do so, you still have to import the package based on the directory structure, but after the import you have to refer to it by the name you used in the package clause.
For example, if you have a folder $GOPATH/src/mypck, and in it you have a file a.go:
package apple
const Pi = 3.14
Using this package:
package main
import (
"mypck"
"fmt"
)
func main() {
fmt.Println(apple.Pi)
}
Just like you are allowed to use relative imports but is not advisable, you may use package names other that their containing folder, but this is not advisable also to avoid further misunderstanding.
Note that the specification doesn't even require all files belonging to the same package to be in the same folder (but it may be an implementation requirement). Spec: Package clause:
A set of files sharing the same PackageName form the implementation of a package. An implementation may require that all source files for a package inhabit the same directory.
What's the use of this?
Simple. A package name is a Go identifier:
identifier = letter { letter | unicode_digit } .
Which allows unicode letters to be used in identifiers, e.g. αβ is a valid identifier in Go. Folder and file names are not handled by Go but by the Operating System, and different file systems have different restrictions. There are actually many file systems which would not allow all valid Go identifiers as folder names, so you would not be able to name your packages what otherwise the language spec would allow.
So in one hand not all valid Go identifiers may be valid folder names. And on the other hand, not all valid folder names are valid Go identifiers, for example go-math is a valid folder name in most (all?) file systems, but it's not a valid Go identifier (as identifiers cannot contain the dash - character).
Having the option to use package names different than their containing folders, you have the option to really name your packages what the language spec allows, regardless of the underlying operating and file system, and put it in a folder named anything that the underlying OS and file system allows - regardless of the package name.
I just started with Go, and i love it! I have tried to make the structure of my project a bit more manageable, instead of having everything in my main.go
So now i have a structure like this.
src/
-> main.go
-> routes.go
-> handlers/
--> user_handlers.go
But when i try to build this with the following command
go build -v -o ./bin/my_bin ./src/...
I get this error
cannot use -o with multiple packages
But if i make it a flat structure like this
src/
-> main.go
-> routes.go
-> user_handlers.go
It works just fine, all my files got "package main" in the top of them.
What am i doing wrong?
The package name must match the directory name. Moving a source file to a new directory requires that you also change the package name.
foo/foo.go // package foo
foo/bar/bar.go // package bar
foo/bar/qux.go // package bar
The PATH is not relevant in terms of the package name.
Package foo: /some/path/some/where/foo
This allows multiple "foo" packages to be created and imported provided your import specifies the desired location of "foo"
P.S. The convention for package names is lowercase, no punctuation (e.g., no _'s)
It tells you what you did wrong, you can't separate a single package over multiple folders.
You need to set and properly use $GOPATH and properly import your routes/ folder in routes.go.
A simple example of it is:
// routes.go
// the . means you can call imported functions without prefixing them with the package name
import . "full-path-to-routes/-relative-to-$GOPATH"
From https://golang.org/doc/code.html:
The GOPATH environment variable specifies the location of your workspace. It is likely the only environment variable you'll need to set when developing Go code.
To get started, create a workspace directory and set GOPATH accordingly. Your workspace can be located wherever you like, but we'll use $HOME/go in this document. Note that this must not be the same path as your Go installation.
I highly recommend reading Effective Go.