I am getting a stuck while trying to create a Terraform provider. I have been following the advice given on https://www.terraform.io/docs/extend/writing-custom-providers.html but when I go to build my binaries via Go go build -o terraform-provider-example I get several missing packages errors.
So I then work my way down the list of missing packages and use go get ... to get those packages installed in my Go libraries.
I get an error indicating that github.com/hashicorp/hcl/v2 cannot be found. I navigate to the location and sure enough it doesn't exist.
Package not available at install time screen shot
Package not available with go get
So I am getting stuck and unable to build these providers. I have looked for a while now trying to find something which describes how to setup the environment for creating providers but have been unsuccessful so far. Can anyone help get me going?
Terraform Core and Terraform provider development requires using the Go toolchain in the new "modules mode", which in current versions of Go is not the default.
The easiest way to ensure you're working in modules mode is to clone the repository you want to work on outside the $GOPATH/src directory. Development outside of GOPATH is only supported in Modules mode, and so the Go toolchain assumes that you intend to use modules mode if you are working in that way.
One reason why Terraform development requires modules mode (though not the only one) is that it has a dependency on github.com/hashicorp/hcl/v2, which is a module path type that is not supported in the old GOPATH mode because previously the Go toolchain was only able to install from the master branch of a particular remote dependency in a Git repository. The module path github.com/hashicorp/hcl/v2 is the Go Modules way to specify using the second major version of HCL, whereas github.com/hashicorp/hcl is the first major version.
In modules mode, it should be sufficient to just run go build -o terraform-provider-example (or, if you prefer, go install) and it will automatically fetch the dependencies to the local modules cache and use them from there. In modules mode, go get is for changing the dependencies recorded in go.mod rather than for installing existing dependencies.
Related
I'm trying to play around with the lightning network. I have cloned the repo, and on disk placed it here (I'm using windows):
C:\Users\hallibut\Documents\GitHub\lnd
I'd like to run any of the tests in itest, lets say testMultiHopPayments. The cli commands I'm using after I cd into the above location is:
go test itest -run testMultiHopPayments
However, I keep getting the error:
package itest is not in GOROOT (C:\Program Files\Go\src\itest)
I've read through the various posts on this error, but I'm still not quite sure why it happens, and it's likely because I don't fully understand the go module (I'm new to go). This article, was probably the best in helping me understand the structure and env variables:
https://golangbyexample.com/workspace-hello-world-golang/
My understanding from the various readings is that whatever directory the go.mod file is in, indicates the module level directory. Prior to version 1.13 there was a required directory and structure, but now that should not be an issue if you're using at least version 1.13 and modules. I'm using 1.17.1. This is somewhat of an assumption or inference, but I believe everything lower in the directory structure is part of a package to be installed as part of the module (and is defined by the package keyword). However, I don't understand why a package with source code within a subdirectory would be missing/throw the aforementioned error. I've also tried running:
go mod install github.com/lightningnetwork/lnd/lntest/itest
That doesn't seem to do anything/has not effect on the error. What am I not understanding about packages? Looking at the go.mod file I observe that itest is not specifically defined anywhere… Not sure if that's required. Also, I assume I've got to run some build process prior? I attempted this with:
go install -v ./...
If you're using VS Code and Go Modules, you need to "Open folder" and point to the cloned repo, to get around that error
I'm trying to create a docker image for use in a build pipeline that has various tools pre-installed for building and testing go projects. One tool we need is golint but I'm struggling to see how to install a specific version of it. The reason I want to lock down the version is to avoid accidental / unwanted / unintended breakages at a later date.
For a start, looking here the versions are not exactly in an easy-to-type format.
Also when I try to use the following command
go get -u golang.org/x/lint/golint#v0.0.0-20181217174547-8f45f776aaf1
I get an error
go: cannot use path#version syntax in GOPATH mode
Googling has so far yielded very few relevant results...
Is what I'm trying to do possible? Many thanks!
You need to be in go module mode to get code of a specific version, since in addition to downloading the code, the version is recorded in the go module file.
The easiest way to do this would be to create an empty directory, run go mod init, which will create a go.mod file.
Then, you can run go get golang.org/x/lint/golint#v0.0.0-20181217174547-8f45f776aaf1, which will add golint at that version to your go.mod file. You can then run go install golang.org/x/lint/golint from within that directory, which will install golint at the version specified into your $GOBIN directory (which defaults to $GOPATH/bin).
I was trying to set up GO Modules in intellij and was trying import a package under GOPATH. When I use Go Modules, it doesnt seem to 'import' the packages from GOPATH. Any ideas on what I could be doing wrong?
Below is a screenshot. Left pic: GoModules, which doesnt recognize the package. Right Pic: Simple GO project, which recognized the packages.
I tried doing sync package, with no luck.
Go version - 1.12.3
.
The two supported modes ("GOPATH mode" and "module-aware mode") are mutually exclusive modes. This means you can't have both, you can't mix modules and GOPATH.
Quoting from Command go: GOPATH and Modules:
When using modules, GOPATH is no longer used for resolving imports. However, it is still used to store downloaded source code (in GOPATH/pkg/mod) and compiled commands (in GOPATH/bin).
And also Command go: Preliminary module support:
For more fine-grained control, the module support in Go 1.11 respects a temporary environment variable, GO111MODULE, which can be set to one of three string values: off, on, or auto (the default). If GO111MODULE=off, then the go command never uses the new module support. Instead it looks in vendor directories and GOPATH to find dependencies; we now refer to this as "GOPATH mode." If GO111MODULE=on, then the go command requires the use of modules, never consulting GOPATH. We refer to this as the command being module-aware or running in "module-aware mode". If GO111MODULE=auto or is unset, then the go command enables or disables module support based on the current directory. Module support is enabled only when the current directory is outside GOPATH/src and itself contains a go.mod file or is below a directory containing a go.mod file.
In module-aware mode, GOPATH no longer defines the meaning of imports during a build, but it still stores downloaded dependencies (in GOPATH/pkg/mod) and installed commands (in GOPATH/bin, unless GOBIN is set).
If you wish to use packages located on your disk, see How to use a module that is outside of "GOPATH" in another module?
I faced this problem, and I use this setting for each project, and it solved my problem.
But I'm still looking for a global GO module configuration.
I'll start with a disclaimer here. I'm running go on Ubuntu 18.0.4 on Windows subsystem.
I have a package in which I was able to run "dep init" successfully from the windows command line. I created a new clone (successful clone, so no network connectivity issues) of the repo in my Ubuntu subsystem and ran "dep init."
This is when I run into problems. Dep just hangs after pulling a single package down. Subsequent calls to dep init do not pull more packages. Running "dep init -v" produces the following:
#> dep init -v
Getting direct dependencies...
Checked 14 directories for packages.
Found 3 direct dependencies.
Take a look at the attached screen capture to see that process threads are spun up but are doing nothing (0% CPU and RAM usage).
Screen Capture with HTOP and my dep pkg folder structure
Things I've tried:
Turning it off and on again.
I've completely cleaned out my dep package folder and re-run dep init. This is how I know that there is just a single package that is actually getting pulled.
Made sure that my ssh keys are up-to-date with github.
dep was the "official experiment." The Go toolchain, as of 1.11, has adopted an approach that sharply diverges from dep. As a result, we are continuing development of dep, but gearing work primarily towards the development of an alternative prototype for versioning behaviour in the toolchain.
Go 1.11 included preliminary support for modules and includes support for vendoring.
You should stop using dep and refer to How to Define a Module to migrate your project.
First, navigate to your source tree, and then follow these instructions:
Create the initial module definition and write it to the go.mod file:
$ go mod init
This step converts from any existing dep Gopkg.lock file or from any of the other nine total supported dependency formats, adding require statements to match the existing configuration.
go mod init will often be able to use auxiliary data (such as VCS meta-data) to automatically determine the appropriate module path, but if go mod init states it can not automatically determine the module path, or if you need to otherwise override that path, you can supply the module path as an optional argument to go mod init, for example:
$ go mod init github.com/my/repo
When I clone a project written in golang, it is normal that a lot of imports like
'github.com/XXXX' are missing. Is there any way to get these imports in batches by a command? or I am suppose to get them one by one.
You should use go get to get "remote" packages. Quoting from Command go: Download and install packages and dependencies
Get downloads the packages named by the import paths, along with their dependencies. It then installs the named packages, like 'go install'.
You may use the -v flag in all of the following commands, and so you will see what go get is doing under the hood.
You may use the -d flag if you just want to download the packages but you do not want to install them.
The examples use the example remote package github.com/somebody/somepackage, but obviously it works for other packages hosted outside of github.com.
For more information, see the official doc: Command go, or type go help get.
To get a single package with all the dependencies of that package and install them, use
go get github.com/somebody/somepackage
To get a package with all its dependencies, and all other packages rooted at that path (along with their dependencies), and install all of them, use:
go get github.com/somebody/somepackage/...
Quoting from Command go:
An import path is a pattern if it includes one or more "..." wildcards, each of which can match any string, including the empty string and strings containing slashes. Such a pattern expands to all package directories found in the GOPATH trees with names matching the patterns.
To get a package with all its dependencies (and "subpackages") including dependencies of tests, and install all of them, use:
go get -t github.com/somebody/somepackage/...
To update a package you already have, use:
go get -u github.com/somebody/somepackage/...
To fetch dependencies of a package you already have (which is not necessarily from a remote location):
go get path/to/package/name/...
Or go to its folder and then you may use a relative path:
go get ./...
A lot of golang projects now use dependency management so you should look for that first. e.g a Glide.lock (glide) or Gopkg.lock (dep - the way people are moving now) file present in the root of the project.
https://github.com/golang/dep
https://golang.github.io/dep
if dep is used and you have it installed then dep ensure will set the dependencies up for you and make sure you get the versions the author intended
if a project is not using dependency management you can just get the packages with go get ./... but I don't think you will be guaranteed the correct versions (e.g if the author was pinned to a version tag for a dep)
If you run dep init it sets up dep on a project and will attempt to resolve the correct versions, however this doesnt always work if the stars dont align (e.g I have seen issues with dependencies using gopkg.in)
try using go get ./... in root of your project