Issues with 'go build' on forked repository - go

I build a binary file for a GitHub repo (go code), which works fine. no issues.
I forked that repo, and modified a single line in the HTML file that has nothing to do with GO code, built the binary file for the new forked repo but the binary it generates refers to the original repo code, can't understand why.
I even cleaned all the code using go clean -i all command and manually removed all the installed code, binary files from $home/go/bin and the repo directory, but it still refers to the original repo code instead of new forked code.
Based on the solution suggested by Tobias, I performed the following steps:
After that, I executed go build in that repo directory, but the new binary file still refers to the old code. I even removed the old binary file and generated a new one.

That's a common problem in go. The references system in "location based" so it searches for these files in the "correct" path. Idk if go modules fix this issue, but atleast when not using go modules you'll have to work around it.
You can solve it by
Solution 1
Download the original repository you forked by:
go get http://github.com/awesome-org/tool
Add your fork as remote
git remote add awesome-you-fork http://github.com/awesome-you/tool
You'll have to make changes in the folder of the original downloaded repo and Push and Pull to/from your fork.
git pull --rebase awesome-you-fork
git push awesome-you-fork
Solution 2
Work around go get:
You create the path the original repo would have, but clone your own fork into it. That way you can push & pull to your fork. That may be the better solution
cd $GOPATH
mkdir -p {src,bin,pkg}
mkdir -p src/github.com/awesome-org/
cd src/github.com/awesome-org/
git clone git#github.com:awesome-you/tool.git # OR: git clone https://github.com/awesome-you/tool.git
cd tool/
go get ./...
These Solutions were found here: http://code.openark.org/blog/development/forking-golang-repositories-on-github-and-managing-the-import-path

The problem with a forked copy of a go packages is when the package is really multiple go packages in one repo, the import statements refer to original base repo, ie: github.com/orig/repo.
This is not an issue for repos with only one go package as it never refers to itself.
But if it has multiple, ie: package github.com/orig/repo/A imports github.com/orig/repo/B
And then you fork it as: github.com/fork/repo
Then when the go compiler sees import "github.com/orig/repo/B" in the source, it goes to download the original version and not your fork.
Fortunately, go modules solves this.
Basically, create a go.mod at the top of your forked repo and add:
module github.com/orig/repo
then, the go compiler will assume that you are "orig/repo" regardless of where you actually are checked out from.
so, when orig/repo/A imports orig/repo/B, it will look locally.
If there are other imports you need to override that are outside the main forked repo, you can also force dependencies to come from another place using replace

SOLVED
At first I thought it was problem with a FORK (which is a common problem experienced with fork of Go language repo's), but it turns out, it was Repo specific problem.
One of the dependent libraries had to be reinstalled for the forked repo to work, which was not mentioned in the Original repo docs.
Finally, this link helped: https://github.com/inconshreveable/ngrok/issues/181#issuecomment-65646229, the problem was not generic but repo specific
I followed the below steps from above link to resolve the dependency on go-bindata
go get github.com/jteeuwen/go-bindata
cd $GOPATH/src/github.com/jteeuwen/go-bindata/go-bindata
go build

Related

Can 'go install' be made to work for executables with different names from the git repo?

Go has a nice feature where you can go install <x> and it will download, build and install a binary.
For example, on my local windows PC, go install github.com/goreleaser/goreleaser will find the latest release for goreleaser, download, build and install it to my local binaries path.
I am working on a project where we would like to enable go install, but encounter a problem if the github repo name does not match the executable name. The GitHub CLI itself runs into the exact same problem:
Example:
go install github.com/cli/cli#latest
go: downloading github.com/cli/cli v1.14.0
go: github.com/cli/cli#latest: module github.com/cli/cli#latest found (v1.14.0), but does not contain package github.com/cli/cli
Is there a way to resolve this?
Update: I worked out that I could directly reference the package via it's sub directory. In my particular instance this works: go install github.com/OctopusDeploy/cli/cmd/octopus#latest
This is a bit unpleasant, but works correctly. It doesn't work for the github CLI because their go.mod has a replace directive in it :-(
Question: Can this be made nicer? Is there a way to put some sort of alias or configuration file so that go install github.com/OctopusDeploy/cli#latest can be used instead of go install github.com/OctopusDeploy/cli/cmd/octopus#latest ?
Can this be made nicer? Is there a way to put some sort of alias or configuration file so that go install github.com/OctopusDeploy/cli#latest can be used instead of go install github.com/OctopusDeploy/cli/cmd/octopus#latest ?
No. Dead simple.

Where is the go project working directory when using modules?

I am trying to download, make some tweaks and build a golang project from GitHub. The project's instructions are:
go get github.com/<vendor>/<projectName>
cd src/github.com/<vendor>/<projectName>
go build .
That used to work in the past — before enabling Go Modules.
Now I have GO111MODULE=on (go version go1.15.4 linux/amd64). When running the first command, go downloads the project as a module and all its dependencies.
But then there is no src/github.com/<vendor>/<projectName> folder anymore. Moreover, the is no folder named <projectName> anywhere in the system.
Instead, there is folder pkg/mod/github.com/<vendor> which contains the project folder with weird symbols in its name (exclamation marks etc.) and version identifiers.
How do I get the project folder available for making tweaks and builds?
As pointed by #Volker, good old git clone should be used.
It turns out that it should be used instead of go get github.com/<vendor>/<projectName> (no idea why the project vendor recommends that):
git clone git://github.com/<vendor>/<projectName>
cd <projectName>
go get ./...
# do tweaks here
go build .
If your goal is tweaks, the easiest way it use to use go mod vendor.
https://golang.org/ref/mod#go-mod-vendor
The go mod vendor command constructs a directory named vendor in the main module's root directory that contains copies of all packages needed to support builds and tests of packages in the main module

go get installation from a local fork

I am trying to install a go package from a local directory (basically I checked out and existing package and applied a pending pull request).
$ # Both commands give a similar output
$ go get -u file:///Users/me/workspaces/go/somepackage
$ go get -u /Users/me/workspaces/go/somepackage
unrecognized import path "[...]" (import path does not begin with hostname)
Since go get is downloading then installing, I tried:
$ go install /Users/me/workspaces/go/somepackage
[] cannot import absolute path
Any experienced go user could give a hand?
If you just want to use a different branch (your PR branch), you can continue using the original import path.
Go get the original package first.
Go into the downloaded package in your local file system cd $GOPATH/pkg/<package directory>
From your local package cache, switch to the branch you want to pull from. git checkout <PR branch>
Now go get -u <package>
If the package is available locally, go get update will just pull the latest code from the branch your local package is checked out to.
As far as I know you can't do this with go get, bur you can copy the directory in to GOPATH manually. If you forked example.com/somepackage then copy /Users/me/workspaces/go/somepackage to ~/go/src/example.com/somepackage or ./vendor/example.com/somepackage.
That being said, the best solution is usually to host the forked code somewhere, and then use that import path. Decency tools such as dep and modules support fetching a different origin for packages.
I could be wrong and maybe there is a workaround exists
Go documentantion on cmd/go/#hdr-Remote_import_paths says:
The supported version control systems are:
Bazaar .bzr
Fossil .fossil
Git .git
Mercurial .hg
Subversion .svn
And a later:
For example,
import "example.org/pkg/foo"
will result in the following requests:
https://example.org/pkg/foo?go-get=1 (preferred)
http://example.org/pkg/foo?go-get=1 (fallback, only with -insecure)
So I suppose that go get never looks in filesystem repos.
Need to investigate source code of go get to be sure.
I will be glad if someone will prove that I am incorrect in this question.
UPDATE
Maybe I'm mistaken but I suppose that all abilities of go get to work with VCSs are here: https://github.com/golang/go/blob/master/src/cmd/go/internal/get/vcs.go
And I don't see here a possibility to work with git local repos.

Library dependencies in Go

I've created a library/package in Go and the consensus was that only applications include a vendor folder in their project and libraries don't.
So now I included my package in another (govendor'ed) project and everything worked fine, untill it got to Jenkins and it had to use its local resources, where 2 of the dependencies were missing.
My project readme says all you need to do is go get my project and you're done. But that's not the case in case you're using govendoring.
What should be the approach for my library? Can this be solved, or is this 'problem' just something the end-user has to solve because they use govendor?
This is more of an opinion question I guess, however I'll share what I use.
I use git subtree for vendoring sub repos in my tree then add a //go:generate line to update it later on, for example:
➜ git subtree add --prefix vendor/xxx/yyy/zzz https://github.com/xxx/yyy/zzz master --squash
Then add //go:generate git subtree pull --prefix vendor/xxx/yyy/zzz https://github.com/xxx/yyy/zzz master --squash to one of my library files.
And just run go generate before I make release.
That solves the vendoring issue without the need of any external tools.
Live example: https://github.com/OneOfOne/xxhash/blob/master/xxhash_cgo.go

Any smart method to get exp/html back after Go1?

I've installed the Go release version as root.
Go1 removed all exp/ code.
Is there smart method to get exp/* back after Go1?
(I mean how to install in my local GOPATH?)
[My Solution]
# pull from go repository to $HOME/repo/go
cd $HOME/repo
hg clone https://go.googlecode.com/hg/go
# make symbolic link to your GOPATH(eg. $HOME/go)
cd $HOME/go/src
ln -s $HOME/repo/go/src/pkg/exp .
The exp/html library was incomplete which is why it was removed for Go1.
However if you really want to use it then
go get code.google.com/p/go/src/pkg/exp/html
may install it back for you. If you want a slightly more complete html parser then you might checkout http://code.google.com/p/go-html-transform/ as well it has an html5 parser as well as a css selector based scraping and transformation library.
EDIT: Apparently trying to go get the package that way doesn't really work. It appears the only way to install this is to checkout the go source code and then install from source. This is actually a really quick an painless process if you want to go that route.
Building from source is the way to do this. When you do the hg update step though, note that since the exp tree is not tagged go1, that hg update release won't get it for you. Instead hg update weekly will get it, and is probably what you want.
Edit: Weekly releases were discontinued after Go 1, so hg update weekly will access increasingly stale code. A better strategy is hg update tip, then copy the exp directory or directories of interest somewhere and recompile it with whatever Go version you are using, Go 1.0.1, for example.
Note: with go 1.4 (Q4, 2014), the url for that exp package will change (again):
code.google.com/p/go.exp => golang.org/x/exp
That means now:
go get golang.org/x/exp
See "Go 1.4 subrepo renaming".
Regarding the html package, it is in net/html, so this will become (as commented by andybalholm):
go get golang.org/x/net/html
The exp packages have been moved to different repositories now, to make them easier to install. Now you can install the former exp/html with go get "golang.org/x/net/html".
This answer is outdated.
This is covered in the golang wiki:
https://code.google.com/p/go-wiki/wiki/InstallingExp
% cd $GOPATH/src
% hg clone https://code.google.com/p/go go-exp
requesting all changes
adding changesets
adding manifests
adding file changes
added 13323 changesets with 50185 changes to 7251 files (+5 heads)
updating to branch default
3464 files updated, 0 files merged, 0 files removed, 0 files unresolved
% mv go-exp/src/pkg/exp .
% rm -rf go-exp
% go install exp/...
Then, to use it:
import "exp/proxy"
I tried this a few months ago and it worked pretty well. Also, when I ran go install ... I limited it to only the package I was interested in: go install exp/html (if I recall, correctly).

Resources