go get -insecure on a corporate network - go

C:\Users\me
> go get -insecure github.com/denisenkom/go-mssqldb
# cd .; git clone https://github.com/denisenkom/go-mssqldb C:\Users\me\Projects\Go\src\github.com\denisenkom\go-mssqldb
Cloning into 'C:\Users\me\Projects\Go\src\github.com\denisenkom\go-mssqldb'...
fatal: unable to access 'https://github.com/denisenkom/go-mssqldb/': SSL certificate problem: unable to get local issuer certificate
package github.com/denisenkom/go-mssqldb: exit status 128
According to go help get this connection should drop down to http? Yes? Do I misunderstand? How do I get this to work w/o https?
PS: I'm not interested in trying to fix https (which on this Win10 image I have no control over anyway) - I already fought that battle with npm and lost...
Edit: I found a passable answer by fixing the global git config. I hate to do it, but needs must... atom.io/go-plus does not seem to pick up this config change, I will ping the author.
C:\Users\me
> git config --global http.sslVerify false

Update Q2 2021: with the newly released Go 1.17 (beta), you now have, for deprecations:
go get
The go get -insecure flag is deprecated and has been removed.
To permit the use of insecure schemes when fetching dependencies, please use the GOINSECURE environment variable.
The -insecure flag also bypassed module sum validation, use GOPRIVATE or GONOSUMDB if you need that functionality.
See go help environment for details.

Related

Why does GO mod tidy not work with bitbucket

I'm attempting to use private repositories as go libraries.
Whenever i try to run go get og god mod tidy, i get this kind of error
>go get bitbucket.org/myworkspace/myRepo
go get bitbucket.org/myworkspace/myRepo: reading https://api.bitbucket.org/2.0/repositories/myworkspace/myRepo?fields=scm: 404 Not Found
I've found multiple suggestions to fix this, with git config insteadOf url reqriting, but it doesn't work, and it all seems to assume that go will clone the library repo via git, and not the api.
My colleague who is running Linux, tried this and it worked, and at no point does it appear to use api.bitbucket.org instead of just bitbucket.org.
I've tried calling https://api.bitbucket.org/2.0/repositories/myworkspace/myRepo?fields=scm via Insomnia, with credentials, and i get the repo back just fine.
Why does go use the bitbucket api on windows, and how can i have it use credentials, so it can find the repo ?
This is due to a change made by Bitbucket (rolling out from June 1st 2022):
Rolling out these changes will break previous versions of Go due to the fact that the go command relies on a 403 response to fetch repositories hosted on Bitbucket Cloud. This means that users who use older versions of Go with private repositories, for example CI/CD builds with Go dependencies, will run into 404 errors.
Go has been updated to support these changes; version 1.18 includes the change but if you are running an earlier version you may need to upgrade to a later minor revision (change is in 1.17.7 and 1.16.14). The relevant Go issue is here (the aim of the change is something different but it resolves the issue).
Why does go use the bitbucket api on windows...
Go was using the API to determine if the Bitbucket repo was using Git or Mercurial (Bitbucket is dropping support for Mercurial).
As mentioned in the comments I've found that the new Git Credential Manager removes the need for the workarounds previously required to access private Bitbuicket repos. Using the credential manager and setting GOPRIVATE was all that was needed..
You can first export the private repository with the command export GOPRIVATE=<remote module name>. Then you can run the command env GIT_TERMINAL_PROMPT=1 go get <remote module name> so that if the credentials are not configured, you get a prompt.

Why does go module ssh custom private repo (non-github) config still request https fetch?

I am using Go modules.
In order to use module version, I cannot use local module. For example:
replace locakpkg => ../localpkg v0.1.0
The above will fail because replacement local path cannot have version so far (go 1.15).
Thus, to make the module version work, I decided to use a private ssh repo.
I did search how to make private ssh repo work for two days.
By following many online articles, I did
git config --global url.user#private.com:.insteadOf https://private.com/
go env -w GOPRIVATE=private.com
I found out go get will always do https fetch to check ssl credential. So I configured a https server properly too.
But in the end, I still get an error message:
unrecognized import path "private.com/foo": reading https://private.com/foo?go-get=1: 404 Not Found
I did google this error and found out this spec https://golang.org/ref/mod#vcs-find which says I have to let the server reply with <meta name="go-import" content="root-path vcs repo-url"> for https fetch request.
If there is a way to use git tag versioning in local module packages, I am OK to use local replace in go.mod instead of configuring a private ssh repo.
If the above point is not possible, how to avoid https fetch when I configure a private ssh repo? I think ssh repo has nothing to do with https protocol.
(I am using go 1.15 at linux. The latest stable version while posting this answer)
I solved the problem and posting here, hopefully, this will help other people one day. I don't find any correct answer by my search online.
In short, the answer is to use .git suffix in all places. Without .git suffix, go mod tidy and go get will use https instead of ssh (git).
At Client:
The file ~/.gitconfig (at linux) if you use /repopath/foo.git path at server:
[url "ssh://user#private.com"]
insteadOf = https://private.com
The file ~/.gitconfig (at linux) if you use ~/repopath/foo.git path at server:
[url "user#private.com:"]
insteadOf = https://private.com/
Execute the following to update ~/.config/go/env at linux:
go env -w GOPRIVATE=private.com
In go.mod, it should use
require private.com/repopath/foo.git v0.1.0
In file.go, it should be
import private.com/repopath/foo.git
At SSH Server
in foo.git/go.mod at private server should have:
module private.com/repopath/foo.git
And make sure the git repo at server has tag version v0.1.0. Don't forget to use git push --tags at client to update the tag version to the server. Without --tags, tag version will not be pushed.
After adding .git suffix to all the required places, go mod tidy and go get will no longer send https request.

Use go get to require dependency from private github repo on WSL 2

I'm trying out WSL 2 on windows 10 and it's gone well so far, but I've been struggling for 2 weeks to make this work, because for some reason go get doesn't use or is not able to make the Git Credentials Manager to prompt for my credentials.
I followed this blog to set up WSL2 with GCM https://www.edwardthomson.com/blog/git_credential_manager_with_windows_subsystem_for_linux.html
And it works very good for most of the daily tasks like cloning, read and write. But when using go get I get this error.
go get <remote github repo>#<latest commit id>
go: <remote github repo> 681dceefc81203e094872401c184d038090d6049 => v0.0.17-0.20200501212733-681dceefc812
go get: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: verifying module: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: reading https://sum.golang.org/lookup/<remote github repo>#v0.0.17-0.20200501212733-681dceefc812: 410 Gone
server response:
not found: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/gopath/pkg/mod/cache/vcs/232ff028cb2fdebd254e30bfc612843483f0fe3fbeb18d5fc8fb4b20f21c9021: exit status 128:
fatal: could not read Username for 'https://github.com': terminal prompts disabled
Already tried ssh-keys and the solutions proposed here go get results in 'terminal prompts disabled' error for github private repo
But the error remains the same, when enabling env GIT_TERMINAL_PROMPT=1 nothing happens, I guess it's because WSL 2 doesn't have the permissions to do that. Anyway I also tried this tool https://github.com/microsoft/Git-Credential-Manager-for-Mac-and-Linux and by setting a variable for plain credentials store, it prompts in the terminal for credentials. But I'm using 2FA because it's required by the organization and the prompt only asks for username and password, so the authentication fails.
So I have to reach out to a mate who is using Mac. He is able to go get the dependency to affect go.mod, make a commit and push the change so I can pull it and continue from there. But of course this is not ideal, and he doesn't have any problem, he uses osxkeychain to manage his git credentials.
Anyone has faced this issue? or know how to solve it? Thank you so much in advance.
Go is not able to understand that certain modules are private and their checksum should not be validated against go's checksum library. The following error comes from that
verifying module: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: reading https://sum.golang.org/lookup/<remote github repo>#v0.0.17-0.20200501212733-681dceefc812: 410 Gone
If possible use at least go 1.13 or, higher. Go had introduced an env variables by name GOPRIVATE, GONOPROXY and GONOSUMDB for managing private modules better. Simplest way to signal to Go that you are importing a private repo is to use GOPRIVATE. Set the pattern of private repos to GOPRIVATE env variable to suppress checksum validation and usage of GOPROXY. Example below avoids checksum for all repos in that hierarchy:
GOPRIVATE=github.com/<your org>/*
Check out answers here and here. You can also do go help module-private for help.

HTTP 407 while attempting to get dependencies using dep

I am trying to work on a Go open source project, on a corporate device. Attempts to address the dependencies via dep ensure command always return 407 PROXY AUTH required. I do have http_proxy and https_proxy environment variable set with values in the format http://user:pasword#proxyname:proxyport. I have also attempted to set the git configs http.proxy. But I do get the same error. What am I missing?
Thanks
This is owing to Git client configuration issues and mismatch between git binary and gnutls. I have had to recompile git binary using openssl, to get this to work.

Unable to resolve "unable to get local issuer certificate" using git on Windows with self-signed certificate

I am using Git on Windows. I installed the msysGit package. My test repository has a self signed certificate at the server. I can access and use the repository using HTTP without problems. Moving to HTTPS gives the error:
SSL Certificate problem: unable to get local issuer certificate.
I have the self signed certificate installed in the Trusted Root Certification Authorities of my Windows 7 - client machine. I can browse to the HTTPS repository URL in Internet Explorer with no error messages.
This blog post by Philip Kelley explained that cURL does not use the client machine's certificate store. I followed the blog post's advice to create a private copy of curl-ca-bundle.crt and configure Git to use it. I am sure Git is using my copy. If I rename the copy; Git complains the file is missing.
I pasted in my certificate, as mentioned in the blog post, I still get the message "unable to get local issuer certificate".
I verified that Git was still working by cloning a GitHub Repository via HTTPS.
The only thing I see that's different to the blog post is that my certificate is the root - there is no chain to reach it. My certificate originally came from clicking the IIS8 IIS Manager link 'Create Self Signed Certificate'. Maybe that makes a certificate different in some way to what cURL expects.
How can I get Git/cURL to accept the self signed certificate?
The problem is that git by default using the "Linux" crypto backend.
Beginning with Git for Windows 2.14, you can now configure Git to use SChannel, the built-in Windows networking layer as the crypto backend. This means that it will use the Windows certificate storage mechanism and you do not need to explicitly configure the curl CA storage mechanism: https://msdn.microsoft.com/en-us/library/windows/desktop/aa380123(v=vs.85).aspx
Just execute:
git config --global http.sslbackend schannel
That should help.
Using schannel is by now the standard setting when installing git for Windows, also it is recommended to not checkout repositories by SSH anmore if possible, as https is easier to configure and less likely to be blocked by a firewall it means less chance of failure.
Open Git Bash and run the command if you want to completely disable SSL verification.
git config --global http.sslVerify false
Note: This solution opens you to attacks like man-in-the-middle attacks.
Therefore turn on verification again as soon as possible:
git config --global http.sslVerify true
I had this issue as well. In my case, I was trying to get a post-receive Git hook to update a working copy on a server with each push. Tried to follow the instructions in the blog you linked to. Didn't work for me as well and overriding the settings on a per-user basis didn't seem to work either.
What I ended up having to do was disable SSL verification (as the article mentions) for Git as a whole. Not the perfect solution, but it'll work until I can figure out a better one.
I edited the Git config text file (with my favorite line-ending neutral app like Notepad++) located at:
C:\Program Files (x86)\Git\etc\gitconfig
In the [http] block, I added an option to disable sslVerify. It looked like this when I was done:
[http]
sslVerify = false
sslCAinfo = /bin/curl-ca-bundle.crt
That did the trick.
NOTE:
This disables SSL verification and is not recommended as a long term solution.
You can disable this per-repository which still isn't great, but localizes the setting.
With the advent of LetsEncrypt.org, it is now fairly simple, automated and free to set up SSL as an alternative to self-signed certs and negates the need to turn off sslVerify.
kiddailey I think was pretty close, however I would not disable ssl verification but rather rather just supply the local certificate:
In the Git config file
[http]
sslCAinfo = /bin/curl-ca-bundle.crt
Or via command line:
git config --global http.sslCAinfo /bin/curl-ca-bundle.crt
I faced this issue as well. And finally got resolved by getting guidance from this MSDN Blog.
Update
Actually you need to add the certificate in git's certificates file curl-ca-bundel.cert that resides in Git\bin directory.
Steps
Open your github page in browser, and click over lock icon in address bar.
In the opened little popup up navigate to 'view certificate' link, it will open a popup window.
In which navigate to certificates tab (3rd in my case). Select the top node that is root certificate. And press copy certificate button in the bottom and save the file.
In file explorer navigate Git\bin directory and open curl-ca-bundle.crt in text editor.
Open the exported certificate file (in step 3) in text editor as well.
Copy all of the content from exported certificate to the end of curl-ca-bundle.crt, and save.
Finally check the status. Please note that backup curl-ca-bundle.crt file before editing to remain on safe side.
An answer to Using makecert for Development SSL fixed this for me.
I do not know why, but the certificate created by the simple 'Create Self Signed Certificate' link in IIS Manager does not do the trick. I followed the approach in the linked question of creating and installing a self-signed CA Root; then using that to issue a Server Authentication Certificate for my server. I installed both of them in IIS.
That gets my situation the same as the blog post referenced in the original question. Once the root certificate was copy/pasted into curl-ca-bundle.crt the git/curl combo were satisfied.
To avoid disabling ssl verification entirely or duplicating / hacking the bundled CA certificate file used by git, you can export the host's certificate chain into a file, and make git use it:
git config --global http.https://the.host.com/.sslCAInfo c:/users/me/the.host.com.cer
If that does not work, you can disable ssl verification only for the host:
git config --global http.https://the.host.com/.sslVerify false
Note : Subjected to possible man in the middle attacks when ssl verification is turned off.
In case of github Repositories (or any none-self-signed certs), choosing below while installing Git-on-windows, resolved the issue.
To completely detail out the summary of all the above answers.
Reason
This problem is occuring because git cannot complete the https handshake with the git server were the repository you are trying to access is present.
Solution
Steps to get the certificate from the github server
Open the github you are trying to access in the browser
Press on the lock icon in the address bar > click on 'certificate'
Go to 'Certification Path' tab > select the top most node in the hierarchy of certificates > click on 'view certificate'
Now click on 'Details' and click on 'Copy to File..' > Click 'Next' > Select 'Base 64 encoded X509 (.CER)' > save it to any of your desired path.
Steps to add the certificate to local git certificate store
Now open the certificate you saved in the notepad and copy the content along with --Begin Certificate-- and --end certificate--
To find the path were all the certificates are stored for your git, execute the following command in cmd.
git config --list
Check for the key 'http.sslcainfo', the corresponding value will be path.
Note: If u can't find the key http.sslcainfo check for Git's default path: C:\Program Files\Git\mingw64\ssl\certs
Now open 'ca-bundle.crt' present in that path.
Note 1 : open this file administrator mode otherwise you will not be able to save it after update. (Tip - you can use Notepad++ for this
purpose)
Note 2 : Before modifying this file please keep a backup elsewhere.
Now copy the contents of file mentioned in step 1 to the file in step 4 at end file, like how other certificates are placed in ca-bundle.crt.
Now open a new terminal and now you should be able to perform operations related to the git server using https.
I've just had the same issue but using sourcetree on windows Same steps for normal GIT on Windows as well. Following the following steps I was able to solve this issue.
Obtain the server certificate tree
This can be done using chrome.
Navigate to be server address.
Click on the padlock icon and view the certificates.
Export all of the certificate chain as base64 encoded files (PEM) format.
Add the certificates to the trust chain of your GIT trust config file
Run "git config --list".
find the "http.sslcainfo" configuration this shows where the certificate trust file is located.
Copy all the certificates into the trust chain file including the "- -BEGIN- -" and the "- -END- -".
Make sure you add the entire certificate Chain to the certificates file
This should solve your issue with the self-signed certificates and using GIT.
I tried using the "http.sslcapath" configuration but this did not work. Also if i did not include the whole chain in the certificates file then this would also fail. If anyone has pointers on these please let me know as the above has to be repeated for a new install.
If this is the system GIT then you can use the options in TOOLS -> options
GIt tab to use the system GIT and this then solves the issue in sourcetree as well.
I have had this issue before, and solve it using the following config.
[http "https://your.domain"]
sslCAInfo=/path/to/your/domain/priviate-certificate
Since git 2.3.1, you can put https://your.domain after http to indicate the following certificate is only for it.
Jan 2021 - Got around this in VS2019 by setting Menu > Git > Settings > Git Global Settings > Cryptographic Network Provider > [Secure Channel] instead of [OpenSSL]
Git SSL certificate problem unable to get local issuer certificate (fix)
PS: Didn't need to set --global or --local http.sslVerify false. I was cloning an Azure DevOps repo which wasn't using any self signed certs.. This seems like an issue with either VS2019 or Git for Windows.. They need to fix it !!
In my case, as I have installed the ConEmu Terminal for Window 7, it creates the ca-bundle during installation at C:\Program Files\Git\mingw64\ssl\certs.
Thus, I have to run the following commands on terminal to make it work:
$ git config --global http.sslbackend schannel
$ git config --global http.sslcainfo /mingw64/ssl/certs/ca-bundle.crt
Hence, my C:\Program Files\Git\etc\gitconfig contains the following:
[http]
sslBackend = schannel
sslCAinfo = /mingw64/ssl/certs/ca-bundle.crt
Also, I chose same option as mentioned here when installing the Git.
Hope that helps!
When using Windows, the problem resides that git by default uses the "Linux" crypto backend. Starting with Git for Windows 2.14, you can configure Git to use SChannel, the built-in Windows networking layer as the crypto backend. To do that, just run the following command in the GIT client:
git config --global http.sslbackend schannel
This means that it will use the Windows certificate storage mechanism and you don't need to explicitly configure the curl CA storage (http.sslCAInfo) mechanism.
One thing that messed me up was the format of the path (on my Windows PC). I originally had this:
git config --global http.sslCAInfo C:\certs\cacert.pem
But that failed with the "unable to get local issuer certificate" error.
What finally worked was this:
git config --global http.sslCAInfo "C:\\certs\\cacert.pem"
solved my problem
git config --global http.sslBackend schannel
Download certificate from this link:
https://github.com/bagder/ca-bundle
Add it to C:\Program Files\Git\bin and C:\Program Files\Git\mingw64\bin
Then try something like: git clone https://github.com/heroku/node-js-getting-started.git
git config --global http.sslVerify false
To fix the especific error SSL certificate problem: unable to get local issuer certificate in git
I had the same issue with Let's Encrypt certificates .
An web site with https we just to need :
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
but git pull says :
fatal: unable to access 'https://example.com/git/demo.git/': SSL certificate problem: unable to get local issuer certificate
To fix it, we need also add:
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
In my case, I had to use different certificates for different git repositories.
Follow steps below (If you have a certificate of your repository, you can read from step 5)
Go to remote repository's site. Ex: github.com, bitbucket.org, tfs.example...
Click Lock icon on the upper left side and click Certificate.
Go to Certification Path tab and double click to .. Root Certificate
Go to Details tab and click Copy to file.
Export/Copy certificate to wherever you want. Ex: C:\certs\example.cer
Open git bash at your local repository folder and type:
$ git config http.sslCAInfo "C:\certs\example.cer"
Now you can use different certificates for each repository.
Remember, calling with the --global parameter will also change the certificates of git repositories in other folders, so you should not use the --global parameter when executing this command.
git config --global http.sslbackend secure-transport
(had to do that after update to Big Sюr)
This works for me. I opened cmd line and ran following command. and pulled again.
git config --global http.sslVerify false
I've had the same problem from Azure DevOps (Visual Studio). Finally I've decided to clone my repo using SSH protocol because of i've prefered it instead of disabling SSL verification.
You only need to generate a SSH Key, you can do it so... SSH documentation
ssh-keygen
And then, import your public key on yout git host (like Azure Devops, Github, Bitbucket, Gitlab, etc.)
I had this error occur when using visual studio. This occurs when you have the Cryptographic Network provider settings set to OpenSSL in the Visual Studio Options window. When I changed the setting to Secure Channel it solved it for me. This setting must have been set for me when I upgraded my VS.
Error
push failed
fatal: unable to access
SSL certificate problem: unable to get local issuer certificate
Reason
After committing files on a local machine, the "push fail" error can occur when the local Git connection parameters are outdated (e.g. HTTP change to HTTPS).
Solution
Open the .git folder in the root of the local directory
Open the config file in a code editor or text editor (VS Code, Notepad, Textpad)
Replace HTTP links inside the file with the latest HTTPS or SSH link available from the web page of the appropriate Git repo (clone button)
Examples:
url = http://git.[host]/[group/project/repo_name] (actual path)
replace it with either
url = ssh://git#git.[host]:/[group/project/repo_name] (new path SSH)
url = https://git.[host]/[group/project/repo_name] (new path HTTPS)
I have resolved the issue by adding below entry in ${HOME}/.gitconfig file
[remote "origin"]
proxy=
In most case it will happen when proxy enabled in your machine so above mentioned entry will fix this problem.
You might have a DNS issue and not a certificate issue, so before you disable SSL verification in your Git shell you should rule out a DNS problem. Cases such as these have been mentioned in Q&A forums such as https-issues-possibly-related-to-dns. If you are using WSL on Windows as your terminal, then you can try running sudo echo nameserver 8.8.8.8 > /etc/resolv.conf and then issue the git commands to see if that makes a difference. This does not seem to be a permanent DNS fix (lasting only the lifetime of your terminal session), but it could help you determine whether it is a DNS issue and not a certificate issue. You could also check this document on configuring your network to use a public DNS. Again, this is only to help you determine if your DNS settings might need adjusting in order to help resolve the certificate issues.
Download and install local certificate. Probably it is published at your company site. For instance, *.cer file.
Right click it and select Install Certificate. 'Certificate Inport Wizard' will appear. Select Local Machine. Press Next, confirm.
Select Place all certificates in the following store, press Browse and select Trusted Root Certification Authorities, OK, Finish.
Also you can check if other applications can fetch, pull or push data. For instance, in Android Studio or probably IDEA you should select in Settings this checkbox: Use credential helper.
I got this error when trying to "clone" the project. One work-around is to just use the "download as zip" on the webpage, which, for me, achieved what I wanted to do.
This might help some who come across this error. If you are working across a VPN and it becomes disconnected, you can also get this error. The simple fix is to reconnect your VPN.

Resources