For example, we can use go get k8s.io/client-go to install the go package but is there a way to figure out the source code URL is actually https://github.com/kubernetes/client-go? Because if I visit k8s.io/client-go directly it shows 404.
How does the go client figure out where the source code is in this example?
Command go: Remote import paths:
Certain import paths also describe how to obtain the source code for the package using a revision control system.
... For code hosted on other servers, import paths may either be qualified with the version control type, or the go tool can dynamically fetch the import path over https/http and discover where the code resides from a <meta> tag in the HTML.
... If the import path is not a known code hosting site and also lacks a version control qualifier, the go tool attempts to fetch the import over https/http and looks for a tag in the document's HTML <head>.
The meta tag has the form:
<meta name="go-import" content="import-prefix vcs repo-root">
The import-prefix is the import path corresponding to the repository root. It must be a prefix or an exact match of the package being fetched with "go get". If it's not an exact match, another http request is made at the prefix to verify the <meta> tags match.
For example in your case the go tool will query https://k8s.io/client-go?go-get=1. Checking it ourselves:
curl https://k8s.io/client-go?go-get=1
Response:
<html><head>
<meta name="go-import"
content="k8s.io/client-go
git https://github.com/kubernetes/client-go">
<meta name="go-source"
content="k8s.io/client-go
https://github.com/kubernetes/client-go
https://github.com/kubernetes/client-go/tree/master{/dir}
https://github.com/kubernetes/client-go/blob/master{/dir}/{file}#L{line}">
</head></html>
As you can see, the response HTML document clearly indicates the code is available at github.com/kubernetes/client-go.
Related
Let's say I have all source code for module hosted on github.com/example/my-module. But I want to create a proxy hosted on example.com/my-module to redirect all requests to github.com/example/my-module and use my custom domain name in imports:
package main
import "example.com/my-module"
instead of:
package main
import "github.com/example/my-module"
According to docs, I can host git repository on example.com/my-module, but it would be overhead for me.
How can I set up package source code proxy for Go on custom domain? Can I just use HTTP reverse proxy for that or it doesn't work?
If you just need it for yourself, or your package's users are open to an extra step to including the package in the go.mod, you could use the "replace" function of the mod file: https://thewebivore.com/using-replace-in-go-mod-to-point-to-your-local-module/
Is it possible to import a module using mercurial with ssh?
I have found very little on using mercurial with go and the little that I found was with http.
The short answer is yes.
The text displayed by go help importpath, or available here, describes how to set import paths so as to imply a particular version control system. Some sites are known in advance:
A few common code hosting sites have special syntax:
[list snipped, but GitHub implies using Git protocol, Launchpad implies Bazaar, and so on]
For code hosted on other servers, import paths may either be qualified with the version control type, or the go tool can dynamically fetch the import path over https/http and discover where the code resides from a <meta> tag in the HTML.
So if you don't have access to or control over this sort of <meta> tag, you should import with an explicitly-specified VCS:
... an import path of the form
repository.vcs/path
specifies the given repository, with or without the .vcs suffix, using the named version control system, and then the path inside that repository.
That is, to tell go get that it must speak Mercurial protocol to host example.com you might use:
import "example.com/me.hg/repo"
or:
import "example.com/me/repo.hg"
where the .hg is what implies the use of Mercurial.
Once you've chosen a specific VCS, things get a little trickier:
When a version control system supports multiple protocols, each is tried in turn when downloading. For example, a Git download tries https://, then git+ssh://.
The source code for the Go VCS importer has the details. Mercurial repository imports try https first, then ssh.
If you can use the <meta> tag, that can provide more detail, so that you can avoid the relatively clumsy .hg in the import path:
If the import path is not a known code hosting site and also lacks a version control qualifier, the go tool attempts to fetch the import over https/http and looks for a <meta> tag in the document's HTML <head>.
If you're implementing the <meta> response to the request page,read all of the rest of this, because this goes on to say:
When using modules, an additional variant of the go-import meta tag is recognized and is preferred over those listing version control systems. That variant uses "mod" as the vcs in the content value, as in:
<meta name="go-import" content="example.org mod https://code.org/moduleproxy">
This tag means to fetch modules with paths beginning with example.org from the module proxy available at the URL https://code.org/moduleproxy. See 'go help goproxy' for details about the proxy protocol.
Here is a quote from a blog post:
Enforce vanity URLs go get supports getting packages by a URL that is
different than the URL of the package's repo. These URLs are called
vanity URLs and require you to serve a page with specific meta tags
the Go tools recognize. You can serve a package with a custom domain
and path using vanity URLs.
For example,
$ go get cloud.google.com/go/datastore
checks out the source code from
https://code.googlesource.com/gocloud behind the scenes and puts it in
your workspace under $GOPATH/src/cloud.google.com/go/datastore.
Given code.googlesource.com/gocloud is already serving this package,
would it be possible to go get the package from that URL? The answer
is no, if you enforce the vanity URL.
To do that, add an import statement to the package. The go tool will
reject any import of this package from any other path and will display
a friendly error to the user. If you don't enforce your vanity URLs,
there will be two copies of your package that cannot work together due
to the different namespace.
package datastore // import "cloud.google.com/go/datastore"
Can someone explain what the last line means?
If you don't enforce your vanity URLs,
there will be two copies of your package that cannot work together due
to the different namespace.
Or demonstrate it in an example?
It's pretty simple.
If your package is hosted at, say github.com/foo/bar, and you have vanity URL foo.com/bar, then someone could do both:
import "github.com/foo/bar"
and
import "foo.com/bar"
This will be problematic if you have different files or packages that import the same package at different paths.
Imagine these two files in the same package:
// foo.go
package foo
import "github.com/foo/bar"
func frobnicate(x *bar.Something) { /* ... */ }
// bar.go
package foo
import "foo.com/bar"
func widget() {
x := *bar.Something{}
frobnicate(x) // compilation error: cannot use x (type foo.com/bar.Something) as type github.com/foo/bar.Something
}
I want to set my go module path to example.com/myrepo instead of github.com/myusername/myrepo such that I am able to import in inside another repository.
for example, if my go.mod looks like this
module example.com/myrepo
go 1.13
how will I make go get example.com/myrepo work?
I am getting the following on go get example.com/myrepo
unrecognized import path "example.com/myrepo" (parse https://example.com/myrepo?go-get=1: no go-import meta tags ())
Given I am the owner of example.com how can I do this?
it is called vanity import paths.
In addition to the common hosting sites (GitHub, Bitbucket, etc) and custom VCS URLs (.git, .hg, etc) known to the go command, this mechanism can be used to point a custom URL to any of the services.
you must be looking for this https://sagikazarmark.hu/blog/vanity-import-paths-in-go/.
Making two different go modules
source.cloud.google.com/me/a
source.cloud.google.com/me/b
With source.cloud.google.com/me/common as a common lib dependency (to share a model)
I'm trying to go get source.cloud.google.com/me/common (even manually wrote it in the go.mod file) but I keep receiving the following error:
package source.cloud.google.com/me/common:
unrecognized import path "source.cloud.google.com/me/common"
(parse https://source.cloud.google.com/me/common?go-get=1: no go-import meta tags ())
I have gcloud set up to be able to use app deploy and create new source repositories. I've tried setting up ssh for google cloud and attempted to use manual credentials. This neither works locally or in the google cloud build service.
I want to do two things:
Be able to go get this dependencsource.cloud.google.com/me/common
Be able to integrate this go get into my App Engine automated build pipeline.
Any help would be appreciated.
Configure repo on https://source.cloud.google.com/
Authorize manual git access https://source.developers.google.com/p/YOUR_PROJECT_ID/r/YOUR_REPO
In this example: https://source.developers.google.com/p/m/r/common
Your common module should go like source.developers.google.com/p/m/r/common.git
Run: go get source.developers.google.com/p/m/r/common.git on the other module
I would try the following steps:
Make sure it has manual git access - You can try a git clone from folder "a" to check if correct git access is in place. Delete it after it gets cloned.
Make sure that you are using HTTPs - looks like you are good in that regards - go1.14 made HTTPs as default for go get's.
Now, coming to the actual problem - looks like your private version control systems isn't sending the required "go-import" meta tag.
For example - refer any github go module, you can see the "go-import" meta tag:
In order to fix it, the VCS server needs to respond with this tag when go get tries to download "common" module
<meta name="go-import" content="source.cloud.google.com/me/common git https:source.cloud.google.com/me/common">
This works:
got get source.developers.google.com/p/YOUR_PROJECT_ID/r/YOUR_REPO.git