I'm relatively new to Go, and I wanted some source code that I downloaded. Though, it imports two packages which I do not have, namely these two:
import (
"git.schwanenlied.me/yawning/chacha20.git"
"golang.org/x/crypto/sha3"
)
It there a tool or way in Go that automatically reads the source files in a directory and downloads the packages needed? On the other hand, when I just try to use go get to download them, I get an error that it uses insecure protocol (due to git). Any ideas how to resolve these dependencies?
As far as I know there is no such a tool.
But in your case you may use 'go get', just remove .git from the end of the line.
go get git.schwanenlied.me/yawning/chacha20
same for import statement
import (
"git.schwanenlied.me/yawning/chacha20"
"golang.org/x/crypto/sha3"
)
Have you tried godep? You can read the documentation here
Related
I have my projects that have many packages which import each other and import outside packages. When I make a change to one of my low lever packages, and then push it to git it is fine and works in that section. When I go get it for use in another project that was working perfectly I now get this go get this error:
module declares its path as: github.com/xdg-go/scram
but was required as: github.com/xdg/scram
None of my code uses either of those directly. It looks like it automatically updated some lower external packages and broke things the used to then old import.
How do I either find out the package that is importing the wrong name or stop all auto-updates?
The go.mod file at github.com/xdg/scram declares itself as github.com/xdg-go/scram:
module github.com/xdg-go/scram
go 1.11
require (
github.com/xdg-go/stringprep v1.0.2
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2
)
The go.mod file should be updated to reflect the correct import path.
Unfortunately if this module is for you an indirect dependency, the best fix possible is to update whatever project you import that is directly importing it.
When that is not an option, a solution to this error is to clone the problematic repository locally and use the replace directive in your go.mod file:
module mymodule
replace github.com/xdg/stringprep => ../strprep
go 1.16.2
require (
github.com/divjotarora/mgo v0.0.0-20190308170442-1d451d2a3149
)
where ../strprep is where the code of the required module exists in your local machine, relative to the go.mod file of your project.
The downside of this of course is that you have to replicate this palliative fix wherever you plan to go get your modules.
Note also:
divjotarora/mgo is just a random example of a project that imports one of those packages using their old import path.
I'm using xdg/stringprep as an example because I can't find modules that import xdg/scram instead, but apparently it suffers from the same issue
Beside, you can use:
go mod why <package> to find out why a certain package is listed as a dependency of your project
go mod graph to show the full dependency graph. The output is in <package> <requirement> format
I'm trying to create a microservice-based API in Go, very basic, just to learn some stuff. I have three main goals I have to achieve.
Monorepo.
Be able to use different back-end languages as need arises (ie. one microservice in Go, two in Node.js, one in Python).
Use gRPC.
Currently my structure looks like this (and is stored in ~/Projects/tkg, outside of GOPATH).
Each "service" should be a self-contained application written in a "whatever". As you can see I have a Go service and a React front-end application. Additionally there is a Makefile there that I want to use for building stuff, but I might move to shell scripts, Docker, whatever. Doesn't matter.
So now the question. How can I make generated proto files play well with this setup? I think I don't understand something about Go modules and packages because I cannot set it so articles.go (from cmd) can access the generated api/article.pb.go. How to do it?
// services/articles/go.mod
module tkg/services/articles
go 1.12
require (
github.com/golang/protobuf v1.3.2
google.golang.org/grpc v1.22.1
)
// services/articles/cmd/article.go
package main
import (
pb "tkg/services/articles/api/article"
)
type repository interface {
Create(*pb.Article) (*pb.Article, error)
}
func main() {
}
// services/articles/api/article.proto
syntax = "proto3";
package article;
option go_package = "tkg/services/articles/api/article";
...
// Makefile
build:
protoc services/articles/api/article.proto --go_out=.
I have tried various different package names in go.mod, different go_packages in the proto file, I had tried different protoc commands and paths. I bet this is silly and it's very obvious to someone who is well-versed in Go, but for someone from Node.js backgroud like me the inability to do import "../api/article.pb.go" is infuriating. :(
The error I am getting is: could not import tkg/services/articles/api/article (no parsed files for package tkg/services/articles/api/article). Of course with different values for package names. I've been trying to solve it for two days now.
How would you approach this problem?
If you are generating the .pb.go file in the same directory as the .proto file (recommended), then your import path should read:
import (
pb "tkg/services/articles/api"
)
(not tkg/services/articles/api/article.)
Go packages are collections of .go files in a single directory - each with the same package XYZ first line - where XYZ is the package name. So when importing a package, one uses the package base directory - not including any .go filenames.
Edit: (this was too long to fit into a comment):
I would step back and think about the base-directory of your entire project.
Standard go packages are usually single words like time, sync etc. because they are part of go's standard library. All other packages should have a full internet path. These typically match a git repo address (e.g. "github.com/boltdb/bolt") - but not always e.g. ("gopkg.in/yaml.v2"). This is actually superior to the centrally-hosted NPM package model - as it easily allows for pulling packages from any repo host. Also since repos are cloned to local disk - those same paths can exist on your local disk first (before they've been hosted on the internet, say for development purposes).
So I would suggest naming your local base-directory something like:
github.com/myname/myproj/tkg/services/...
and then ensure you import packages based on this directory structure e.g.
import "github.com/myname/myproj/tkg/services/api"
If a go build does not pick up the generated (.pb.go) code, then there is something up with your GOPATH or if using the new go-modules your go.mod setup.
I have an log package in my project src folder. But when I include the log package from an another package as following, the go seams find the log in the system folder instead of my package.
import ("log")
And seams I cannot using relative path to import log package, because go install give following error:
local import "./log" in non-local package
So how can I let the go using my log package?
You need to add your package inside $GOPATH
So if your package is in
$GOPATH/src/github.com/ZijingWu/awesomeapp/src/
your log package would be in
$GOPATH/src/github.com/ZijingWu/awesomeapp/src/log
then it would be possible to use
import("github.com/ZijingWu/awesomeapp/src/log")
The paths seams a bit strange tho, and maybe you should consider adding the log package as a completely separate repository on github, so it would look something like.
import("github.com/ZijingWu/log")
Just using github as an example here, could of course be placed wherever, also just guessing at your username there. Doesnt need to be in github either you can just place it localy on your computer in the $GOPATH, but then noone else would be able to build your package.
I have imported an app from github which has many imports, spread in several files like:
import (
"log"
"net/http"
"github.com/johndoe/sleekrest/model"
"github.com/johndoe/sleekrest/shared/passhash"
"github.com/johndoe/sleekrest/shared/recaptcha"
"github.com/johndoe/sleekrest/shared/session"
"github.com/johndoe/sleekrest/shared/view"
"github.com/johndoe/csrfbanana"
)
I want to work on the packages on my local path /go/src/myrest, so I'd like to have imports to be like
import (
"log"
"net/http"
"./model"
"./shared/passhash"
"./shared/recaptcha"
"./shared/session"
"./shared/view"
"./csrfbanana"
)
I know I can use bash commands like sed, find, etc to replace the import paths, but I'm wondering if there is an idiomatic way to do so in golang?
There is not an idiomatic way to do this because relative import paths are not idiomatic Go.
Reasons to not use relative import paths
The following is from Organizing Go code:
An import path is the string with which users import a package. It specifies the directory (relative to $GOROOT/src/pkg or $GOPATH/src) in which the package's source code resides.
Sometimes people set GOPATH to the root of their source repository and put their packages in directories relative to the repository root, such as "src/my/package". On one hand, this keeps the import paths short ("my/package" instead of "github.com/me/project/my/package"), but on the other it breaks go get and forces users to re-set their GOPATH to use the package. Don't do this.
The following is from Command go:
Second, if you are compiling a Go program not in a work space, you can use a relative path in an import statement in that program to refer to nearby code also not in a work space. This makes it easy to experiment with small multipackage programs outside of the usual work spaces, but such programs cannot be installed with go install (there is no work space in which to install them), so they are rebuilt from scratch each time they are built. To avoid ambiguity, Go programs cannot use relative import paths within a work space.
Also, you might look at these StackOverflow answers:
zzzz's answer to Golang how to import local packages without go path?
mostafa's answer to Relative import from parent directory
Vendoring
Vendoring became an experimental feature in Go 1.5. As of Go 1.6, vendoring is no longer experimental. For more information on vendoring, see:
Vendor Directories section of Command go.
Go 1.5 Vendor Experiment document by Russ Cox.
This repo has 3 go files all begin with "package lumber".
To use this package, I'm supposed to put this in my GOROOT and simply
import lumber
in my program. How do variables and types in this package connect with each other across multiple files? How does the go compiler know which file to begin reading first?
In case I want to read the package, where should I begin reading to understand the package? What exactly is the flow of things here?
To elaborate on jnml's answer:
When you use import "foo/bar" in your code, you are not referring to the source files (which will be located in $GOPATH/src/foo/bar/).
Instead, you are referring to a compiled package file at $GOPATH/pkg/$GOOS_$GOARCH/foo/bar.a. When you build your own code, and the compiler finds that the foo/bar package has not yet been compiled (or is out of date), it will do this for you automatically.
It does this by collating* all the relevant source files in the $GOPATH/src/foo/bar directory and building them into a single bar.a file, which it installs in the pkg directory. Compilation then resumes with your own program.
This process is repeated for all imported packages, and packages imported by those as well, all the way down the dependency chain.
*) How the files are collated, depends on how the file itself is named and what kind of build tags are present inside it.
For a deeper understanding of how this works, refer to the build docs.
No, you're not "supposed to put this in my GOROOT". You're supposed to execute
$ go get github.com/jcelliott/lumber
which will clone the repository into $GOPATH/src/github.com/jcelliott/lumber. Then you can use the package by importing it in your code as
import "github.com/jcelliott/lumber"
About the scoping rules: Declarations and scope