"go get -t ./...", Travis CI, and installing all go dependencies - go

In the documentation here: https://docs.travis-ci.com/user/languages/go#Dependency-Management
This is the install dependencies step:
if go version is greater than or equal to 1.2
go get -t ./...
My project looks like this:
root
---- src
---- github.com
---- myProject
---- pkg
the GOPATH is set to root
but I am getting this error:
package github.com/aws/aws-lambda-go/lambda: home/travis/build/path/to/package exists but home/travis/build/path/to/package.git does not - stale checkout?
How do I get all the dependencies to install at once like an npm install in nodejs?

How do I get all the dependencies to install at once?
That's what go get does... it looks like something might be misconfigured or corrupted in your case. You could start by trying some things.
go get installs into the first path in the $GOPATH environment variable. Confirm that it is set to what you want (usually a path ending in a directory named go; subdirs src, pkg, etc. will be created).
If you're going to use ./..., make sure you're calling go get from the right directory.
Check to make sure the git root is in the right place
try using a non-wildcard name/path/address for the package you want to get, instead of ./...
try calling go get without the -t flag
If none of that works, you may be able to solve the problem by deleting the directory (home/travis/build/path/to/package) and trying again -- make sure you aren't deleting any code you worked on, or the git repository/files, unless it's backed up somewhere.
According to Go tools documentation, you should only need to call
go get [packages]
to install the named packages along with their dependencies:
Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.
The ... ellipsis is a wildcard that can expand to match any string. See section on Description of Package Lists:
An import path is a pattern if it includes one or more "..." wildcards, each of which can match any string, including the empty string and strings containing slashes. Such a pattern expands to all package directories found in the GOPATH trees with names matching the patterns.
The ./ means "here": so make sure you're running from the right directory if you're going to use ./...
The -t flag is for downloading the packages required to build the tests:
The -t flag instructs get to also download the packages required to build the tests for the specified packages.
The error you're seeing is related to git. Sometimes the cause is unknown, but it can often be fixed by deleting the directory and starting again (see, e.g., "Error to install golint" or "Correct way to get package")
(You might also find this blog post on configuring travis-ci for Go helpful.)

Related

"go get" command not generating the bin folder when it is run in a shell script

I have installed go package. When I go to the instance(VM) and run the command
go get github.com/linkedin/Burrow from a terminal/cmd it is downloading both "src" & "bin" folder under user home directory. But when I run the same command by setting GOPATH in a shell script, it is only downloading the "src" folder but not generating "bin" folder.
SOURCE_DIR="/opt/burrow"
export GOPATH=$SOURCE_DIR/go
go get $BURROW_REPO
Am I missing anything?
Use go install command (Compile and install packages and dependencies).
That command download src to $GOPATH and build it to $GOBIN.
GOBIN may not be set at the moment.
You can check by go env GOBIN. if its empty so you must set with export GOBIN=$(go env GOPATH)/bin
Also, for calling binary files from your terminal, you need to use go install command. This will create related bin file under GOBIN path.
There's a lot going on here.
The first point of your confusion is that go get does not download neither "src" nor "bin": Go packages are always contain only source code, and they typically does not contain the "src" directory in their file and directory hierarchies.
Instead, these directories are artefacts of the the Go toolchain.
The second point of confusion is that since Go 1.8, the Go toolchain uses a fallback value for the GOPATH environment variable if that is not set, and on Unix-like systems it defaults to the directory named "go" under the "home" directory of the user executing the go command.
If this directory is missing, the toolchain will create it.
Hence my stab at your problem is that you have some sort of permissions problem: when GOPATH is unset, "$HOME/go" is used — with whatever value $HOME expands to for the current user; when you set GOPATH by hand, something prevents creation of the "bin" directory under $GOPATH.
There's another possibility: you also set the GOBIN environment variable, which, when set, overrides the usual location used to install binaries built by go install (and go get).
You might study the output of go help environment to read more on the subject.
In either case, the most sensible path forward is to run go install with the -x command-line option and see where the command tries to put the generated executable image, and why this fails (if at all).
You might study the output of go help install and go help build to read more on the subject.
You might also consider forcing usage of Go modules for your particular case:
running GO111MODULES=on go get github.com/linkedin/Burrow would work like five times faster for your use case.
Be sure to study the output of go help modules and go help mod-get first.
bin folder or src folders are not automatically created for you by the go get command. Here are the standard steps creating a new project in go assuming this the first time you are creating a project in go:
Under your workspace directory, create a project directory, say "project1" bin, src directories.
Set GOPATH, GOBIN:
export GOPATH=~/workspace/project1/
export GOBIN=~/workspace/project1/bin
Now if you need just the source code, do a go get github.com/linkedin/Burrow
or if you need a binary do a go install github.com/linkedin/Burrow
The binary will be stored under ~/workspace/project1/bin and source code under ~/workspace/project1/
Same steps would apply either if your creating your project through a make file or terminal

Installing gota package in go workspace

I'm writing this away from my code so fingers crossed.
I've recently started learning Go from a Python background. I've set up my workspace (Linux Mint OS) so:
GOPATH=$HOME/go
GOROOT=/usr/local/go
Where under $HOME i have a dir called go and 3 subdirs called src, bin and pkg.
I wanted to mess around with some dataframes (I use pandas a lot in Python) so I tried to install gota from github. Only their recommended install command:
go get -u github.com/kniren/gota/dataframe
go get -u github.com/kniren/gota/series
returns an error saying it could find the package in GOPATH or GOROOT. To me this is strange as go get seems like the equivalent to pip install and shouldn't be looking in my path but rather at the URL. I managed to get some files to install but using goget and the github URL of the project:
go get github.com/go-gota/gota/dataframe
go get github.com/go-gota/gota/series
and this built some files under a gonum.org directory in my src dir and a linux_amd64 dir in my pkg directory. So far neither section have the expected path to the libraries and I can't find a suitable method to import. import statements look in gopath's src directory however I assume it should be looking in the pkg directory? Why is this and what's wrong with my env?
The authors of the repository must have migrated to a different repository.
The official repository of these packages is: github.com/go-gota/gota
https://www.github.com/kniren/gota/dataframe
https://www.github.com/kniren/gota/series
These repositories do not exist, this is why your go get commands failed. In fact, trying to navigate to github.com/kniren/gota redirects me to their official repository.

How to find a Go package installed with 'go get -u' when no $GOPATH is defined?

I'm following the gRPC Quickstart tutorial for Go, https://grpc.io/docs/quickstart/go/, and have installed gRPC using the command
go get -u google.golang.org/grpc
I actually haven't defined a GOPATH environment variable:
> echo $GOPATH
which, as I understand it, means that it defaults to ~/go, or in my case /Users/kurt/go.
At the next step, I'd like to build the example by doing
cd $GOPATH/src/google.golang.org/grpc/examples/helloworld
However, I find that the directory doesn't exist, and there is also no google.golang.org directory in /Users/kurt/go/src:
~/g/src> ls *google*
fish: No matches for wildcard '*google*'. See `help expand`.
ls *google*
^
Should the package not be located here? That's what I understand from Where does go get install packages?.
Using Go Modules, you can find 'go get' downloaded files at:
~/go/pkg/mod/cache/download
However, it should be treated like an immutable copy of the source code.
If you want a mutable copy of the source code, you should clone the repository:
git clone https://github.com/grpc/grpc-go
In your example output, you are in ~/g/arc
Go path default would be ~/go/src
I think auto complete bit ya there
In the end, I worked around the problem by cloning https://github.com/grpc/grpc-go which appears to contain the examples/helloworld directory I'm looking for. Still curious to hear where the package downloaded with go get is located.

Get missing imports in batches

When I clone a project written in golang, it is normal that a lot of imports like
'github.com/XXXX' are missing. Is there any way to get these imports in batches by a command? or I am suppose to get them one by one.
You should use go get to get "remote" packages. Quoting from Command go: Download and install packages and dependencies
Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.
You may use the -v flag in all of the following commands, and so you will see what go get is doing under the hood.
You may use the -d flag if you just want to download the packages but you do not want to install them.
The examples use the example remote package github.com/somebody/somepackage, but obviously it works for other packages hosted outside of github.com.
For more information, see the official doc: Command go, or type go help get.
To get a single package with all the dependencies of that package and install them, use
go get github.com/somebody/somepackage
To get a package with all its dependencies, and all other packages rooted at that path (along with their dependencies), and install all of them, use:
go get github.com/somebody/somepackage/...
Quoting from Command go:
An import path is a pattern if it includes one or more "..." wildcards, each of which can match any string, including the empty string and strings containing slashes. Such a pattern expands to all package directories found in the GOPATH trees with names matching the patterns.
To get a package with all its dependencies (and "subpackages") including dependencies of tests, and install all of them, use:
go get -t github.com/somebody/somepackage/...
To update a package you already have, use:
go get -u github.com/somebody/somepackage/...
To fetch dependencies of a package you already have (which is not necessarily from a remote location):
go get path/to/package/name/...
Or go to its folder and then you may use a relative path:
go get ./...
A lot of golang projects now use dependency management so you should look for that first. e.g a Glide.lock (glide) or Gopkg.lock (dep - the way people are moving now) file present in the root of the project.
https://github.com/golang/dep
https://golang.github.io/dep
if dep is used and you have it installed then dep ensure will set the dependencies up for you and make sure you get the versions the author intended
if a project is not using dependency management you can just get the packages with go get ./... but I don't think you will be guaranteed the correct versions (e.g if the author was pinned to a version tag for a dep)
If you run dep init it sets up dep on a project and will attempt to resolve the correct versions, however this doesnt always work if the stars dont align (e.g I have seen issues with dependencies using gopkg.in)
try using go get ./... in root of your project

can't load package: package .: no buildable Go source files

Here is the error message:
% go get
can't load package: package .: no buildable Go source files in /Users/7yan00
% echo $GOPATH
/Users/7yan00/Golang
How would you troubleshoot that error?
Make sure you are using that command in the Go project source folder (like /Users/7yan00/Golang/src/myProject).
One alternative (similar to this bug) is to use the -d option (see go get command)
go get -d
The -d flag instructs get to stop after downloading the packages; that is, it instructs get not to install the packages.
See if that helps in your case.
But more generally, as described in this thread:
go get is for package(s), not for repositories.
so if you want a specific package, say, go.text/encoding, then use
go get code.google.com/p/go.text/encoding
if you want all packages in that repository, use ... to signify that:
go get code.google.com/p/go.text/...
You should check the $GOPATH directory. If there is an empty directory of the package name, go get doesn't download the package from the repository.
For example, If I want to get the github.com/googollee/go-socket.io package from it's github repository, and there is already an empty directory github.com/googollee/go-socket.io in the $GOPATH, go get doesn't download the package and then complains that there is no buildable Go source file in the directory. Delete any empty directory first of all.
Another possible reason for the message:
can't load package: .... : no buildable Go source files
Is when the source files being compiled have:
// +build ignore
In which case the files are ignored and not buildable as requested.This behaviour is documented at https://golang.org/pkg/go/build/
To resolve this for my situation:
I had to specify a more specific sub-package to install.
Wrong:
go get github.com/garyburd/redigo
Correct:
go get github.com/garyburd/redigo/redis
If you want all packages in that repository, use ... to signify that, like:
go get code.google.com/p/go.text/...
you can try to download packages from mod
go get -v all
I had this exact error code and after checking my repository discovered that there were no go files but actually just more directories. So it was more of a red herring than an error for me.
I would recommend doing
go env
and making sure that everything is as it should be, check your environment variables in your OS and check to make sure your shell (bash or w/e ) isn't compromising it via something like a .bash_profile or .bashrc file. good luck.

Resources