CMake ExternalProject_Add proxy settings - proxy

I have been quite successfully using CMake to perform builds using the ExternalProject_Add function, but my company recently put in a proxy server... Which has broken the aforementioned build scripts.
The download step fails during the extract phase because the tarball that was downloaded is only the redirect request from the proxy server (at least I think this is what is contained in the tiny tarball it acquires).
I found this post on the CMake mailing-list. I thought maybe if it worked for the file() command it might work for the ExternalProject_Add() command. I set both http_proxy and HTTP_PROXY environment variables, but still received the same error. I have thought about overriding the DOWNLOAD_COMMAND argument with a wget call since this command seems to behave with the proxy settings. However, I wanted to know if there was a better way.
UPDATE 1: I checked the contents of the small tarball, and it does contain HTML; however, it is a notification that Authentication is required. I'm not sure why it is requiring authentication because I haven't had to enter any login information for wget. wget shows the following output:
Resolving webproxy... 10.0.1.50
Connecting to webproxy|10.0.1.50|:80... connected.
Proxy request sent, awaiting response... 200 OK
Download begins here...
UPDATE 2: I have also noticed that both apt-get and svn fail with this new proxy setup, but git does not... svn complains about "Server sent unexpected return value (307 Proxy Redirect)..." Very confusing...
Thanks!

What version of CMake are you using? The file(DOWNLOAD command started using the follow redirect flag in version 2.8.2, introduced by the following commit:
http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ef491f78218e255339278656bf6dc26073fef264
Using a custom DOWNLOAD_COMMAND is certainly a reasonable workaround.

Related

Using Artifactory to proxy Cypress with CYPRESS_DOWNLOAD_MIRROR

Desired behavior
I would like to use CYPRESS_DOWNLOAD_MIRROR pointing toward my artifactory configuration for Cypress and just be able to do npm install and download library AND binary of Cypress
Current behavior
When setting Cypress in artifactory, and downloading it with CYPRESS_DOWNLOAD_MIRROR set toward this tool, the download script find binary files as X.Y.Z and not cypress.zip and fail. Apparently I can't rename binaries in artifactory. It seems to have made it available in a name X.Y.Z instead of cypress.zip
My artifactory admin tell me to do this command before but I can't since it's a post action of Cypress
curl –v « https://artifactory.mycompany.fr/artifactory/remote-download.cypress-generic/desktop/6.8.0?platform=win32&arch=x64 » > cypress.zip
Workaround
For now, I'm using CYPRESS_INSTALL_BINARY to point to a manually uploaded binary in artifactory but it's a pain because I have to separate Linux binary (for CI) and Windows binary (for dev) and if my package is configured with "cypress": "^6.2.1" the npm library will go to 6.2.1 and my binary will stuck to 6.2.0 for example...
Debug logs
Installing Cypress (version: 6.8.0)
× Downloading Cypress
→ Cypress Version: 6.8.0
Unzipping Cypress
Finishing Installation
The Cypress App could not be downloaded.
Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading
Cypress. Read more: https://on.cypress.io/proxy-configuration
Otherwise, please check network connectivity and try again:
URL: https://artifactory.mycompany.fr/artifactory/remote-download.cypress.io/desktop/6.8.0?platform=win32&arch=x64
Error: self signed certificate in certificate chain
Download method
npm
Operating System
Linux
Windows
Other
I'm behind a proxy
I don't really know if it's an artifactory or a Cypress matter but I need help ^^
In addition to accepted answer, it is possible to replace pre-defined 'Query Params' with enabling 'Propagate Query Params'. If set, the query params passed with the request to Artifactory, will be passed on to the remote repo.
Please note, according to JFrog docs, this setting is only available for Generic type repositories.
I was able to make it work on Windows using the following:
I created a generic remote repository, making sure it is pointing to https://download.cypress.io, and under the advanced tab, added the query params: platform=win32&arch=x64 (notice there is a dedicated field for it).
The above is required in order to cache the correct binary based on the OS and arch (you might require a different remote repository with different query params).
I found it on Cypress doc that these query params control the binary type which will be downloaded (so we need to make sure it fits the client os and arch).
In the .npmrc I simply provided the following:
CYPRESS_DOWNLOAD_MIRROR=https://user:myverystrongpassword#myartifactory/artifactory/generic-cypress-windows
I've used this command (on MacOS) to directly pass path to downloaded Cypress.zip file
CYPRESS_INSTALL_BINARY=~/Downloads/cypress.zip yarn add cypress --D

Windows, Git and the SSH url/path of a bare repository

I am trying to set up a git repository on our Windows server.
I installed OpenSSH for Windows and Git for Windows.
The OpenSSH server runs on port 2345 to filter out generic bots.
I created a bare and shared repository on C:\git\protocolrepo.git
I couldn't reach it with the URL:
ssh://User#domain#server.com:2345:C:/git/protocol.git
It seems to me to be a malformed URL but everywhere I look on the web I get this, but it just doesn't work.
As I found out in this thread: https://github.com/PowerShell/Win32-OpenSSH/issues/895, the shell might be the cause, so I set the powershell as the default shell of OpenSSH.
This didn't work either and I still needed to use this oddly formed URL.
So I installed Cygwin and set the Cygwin bash as the default shell, and used a more appropriately formed url:
ssh://User#domain#server.com:2345/cygdrive/c/git/protocol.git
I had a little bit more success with this, but it now states the following error message:
fatal: '/cygdrive/c/git/protocol.git' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
The access rights should be fine as I make use of the same user alltogether, but I'm not 100% sure how this works on Windows.
I'm not sure either how Git/ssh handles this path on Windows.
On a Linux based server this worked just fine with /git/protocol.git path.
Pushing a new branch to a bare repository is something I haven't done in a while either so that also might be done wrong here, but I can't recall it to be an issue.
What would you recommend to debug this issue?
I would start by checking if an interactive SSH session works, before considering using a Git repo SSH URL.
ssh -p 2345 User#domain#server.com
If it does, try and access /C/git/protocol.git in that interactive shell.
If that works, then try for your git commands the URL
ssh://User#domain#server.com:2345/C/git/protocol.git
# or
ssh://User#domain#server.com:2345/c/git/protocol.git

HTTP 407 while attempting to get dependencies using dep

I am trying to work on a Go open source project, on a corporate device. Attempts to address the dependencies via dep ensure command always return 407 PROXY AUTH required. I do have http_proxy and https_proxy environment variable set with values in the format http://user:pasword#proxyname:proxyport. I have also attempted to set the git configs http.proxy. But I do get the same error. What am I missing?
Thanks
This is owing to Git client configuration issues and mismatch between git binary and gnutls. I have had to recompile git binary using openssl, to get this to work.

go get -insecure on a corporate network

C:\Users\me
> go get -insecure github.com/denisenkom/go-mssqldb
# cd .; git clone https://github.com/denisenkom/go-mssqldb C:\Users\me\Projects\Go\src\github.com\denisenkom\go-mssqldb
Cloning into 'C:\Users\me\Projects\Go\src\github.com\denisenkom\go-mssqldb'...
fatal: unable to access 'https://github.com/denisenkom/go-mssqldb/': SSL certificate problem: unable to get local issuer certificate
package github.com/denisenkom/go-mssqldb: exit status 128
According to go help get this connection should drop down to http? Yes? Do I misunderstand? How do I get this to work w/o https?
PS: I'm not interested in trying to fix https (which on this Win10 image I have no control over anyway) - I already fought that battle with npm and lost...
Edit: I found a passable answer by fixing the global git config. I hate to do it, but needs must... atom.io/go-plus does not seem to pick up this config change, I will ping the author.
C:\Users\me
> git config --global http.sslVerify false
Update Q2 2021: with the newly released Go 1.17 (beta), you now have, for deprecations:
go get
The go get -insecure flag is deprecated and has been removed.
To permit the use of insecure schemes when fetching dependencies, please use the GOINSECURE environment variable.
The -insecure flag also bypassed module sum validation, use GOPRIVATE or GONOSUMDB if you need that functionality.
See go help environment for details.

Git not working with remote server

I have git installed and working as expected on my local system or a network share, but have not been able to get it to access the remote bare repository on my server using ssh.
I have installed msysgit and CopSSH on the server and msysgit, putty and tortoisegit on my local system. I am able to open a putty session or use plink successfully -- at least I believe plink is successful, but more on that in a bit.
When I attempt to clone a remote repository I get this:
D:\Temp> git clone uname#server:e:/path/test.git Test
Cloning into Test...
fatal: protocol error: bad line length character: Micr
The error message has been consistent for a while, but at times (from one reinstall attempt to another) the last four characters will be 'Micr', 'Allo' or something else I don't remember right now. In the currently installed state it is 'Micr'.
So the big question is what am I not doing right?
Although I imagine it would be much easier to correct what was wrong if I could get the actual error message instead of the generic one followed by the first four characters of the actual error message. Any idea if that is possible?
First, you have variables like export GIT_TRACE=1 and export GIT_TRACE_PACKET=1 which can help debugging this kind of situation.
Second, this error is often caused by any kind of display done during the ssh session (see for instance this thread), like a .bashrc on the server side with an echo in it.
I have a wrapper set-up around ssh that sets the window title of the shell to the ssh command argument. That is what messes things up!
See also Git FAQ
Other causes include:
using the right ssh
correctly forcing to use git-shell
using --exec-path from the server_arg on the server side
Note: with Git 2.6+ (Q3 2015), see also "GIT_TRACE_PACKFILE".

Resources