Gitlab CI - Failed to register runner - continuous-integration

I've setup my gitlab installation from source, secured it with letsencrypt and deployed it under https://gitlab.mydomain.com. I can access the website and create repositories, etc. but I can't find a way to register a gitlab ci runner for the installation.
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/ci):
https://gitlab.mydomain.com/ci
Please enter the gitlab-ci token for this runner:
xxxxxxxx-xxxxxxxx
Please enter the gitlab-ci description for this runner:
[server]: test
Please enter the gitlab-ci tags for this runner (comma separated):
test
ERROR: Registering runner... failed runner=xxxxxxx
status=couldn't execute POST against https://gitlab.mydomain.com/ci/api/v1/runners/register.json:
Post https://gitlab.mydomain.com/ci/api/v1/runners/register.json:
read tcp [ipv6address]:33518->[ipv6address]:443: read: connection reset by peer
PANIC: Failed to register this runner. Perhaps you are having network problems
My gitlab system is working fine and I really ran out of explanations why there would be a connection reset by peer. When I try to curl the address from the error message directly, it returns a correct response.
curl -v https://gitlab.mydomain.com/ci/api/v1/runners/register.json
* Trying ipv6address...
* Connected to gitlab.mydomain.com (ipv6address) port 443 (#0)
* found 174 certificates in /etc/ssl/certs/ca-certificates.crt
* found 700 certificates in /etc/ssl/certs
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_256_GCM_SHA384
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: mydomain.com (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=mydomain.com
* start date: Wed, 18 May 2016 14:35:00 GMT
* expire date: Tue, 16 Aug 2016 14:35:00 GMT
* issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3
* compression: NULL
* ALPN, server did not agree to a protocol
> GET /ci/api/v1/runners/register.json HTTP/1.1
> Host: gitlab.mydomain.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 405 Method Not Allowed
< Server: nginx
< Date: Sun, 29 May 2016 09:14:09 GMT
< Content-Type: application/json
< Content-Length: 2
< Connection: keep-alive
< Allow: OPTIONS, POST
< Cache-Control: no-cache
< Status: 405 Method Not Allowed

If the runner and gitlab are running on the same host you can get around this problem by entering a the first question the following instead what is given in the docs:
http://gitlab:port
where gitlab is the container name and port the left port number of the container. If you are using gitlab internal ssl certs you specify https instead of http. This always solves this problem when I get it.

For those that are using docker:
The issue its about docker network.
If you try
"$docker container inspect $id"
You are going to see the IPAddress of gitlab container.
Point to that ip adress on first question to works fine.

Problem went away after updating gitlab to 8.8.3 and gitlab-multi-ci-runner to the most recent version.
I also started my gitlab nginx configuration files from scratch.
In the end, I can't tell which change exactly solved the problem.

I had so far many errors and issue starting with Error 404, 403 and endings with problem with post request.
For me, problem seems to be incompatibility between GitLab and ci-runner.
Solution, same on post issue, was install older version of ci-runner:
sudo apt install gitlab-ci-multi-runner=1.11.1

I've solved it by installing gitlab-ci-multi-runner=1.11.1.

Related

Use HAProxy with proxy chain

I'm trying to connect to internet services (specifically Facebook API) from an internal application, and for that I have to go through one of two proxies provided by the security team. If I use one of these proxies with CURL then it works OK:
export http_proxy=http://<user>:<pass>#<proxy_ip>:<port>/
export https_proxy=http://<user>:<pass>#<proxy_ip>:<port>/
curl -v -I https://graph.facebook.com/v7.0/me/messages
* About to connect() to proxy <proxy_ip> port <port> (#0)
* Trying <proxy_ip>...
* Connected to <proxy_ip> (<proxy_ip>) port <port> (#0)
* Establish HTTP proxy tunnel to graph.facebook.com:443
* Proxy auth using Basic with user '<user>'
> CONNECT graph.facebook.com:443 HTTP/1.1
> Host: graph.facebook.com:443
> Proxy-Authorization: Basic <base64>
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=*.facebook.com,O="Facebook, Inc.",L=Menlo Park,ST=California,C=US
* start date: May 14 00:00:00 2020 GMT
* expire date: Aug 05 12:00:00 2020 GMT
* common name: *.facebook.com
* issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
> HEAD /v7.0/me/messages HTTP/1.1
> User-Agent: curl/7.29.0
> Host: graph.facebook.com
> Accept: */*
>
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< WWW-Authenticate: OAuth "Facebook Platform" "invalid_request" "An active access token must be used to query information about the current user."
Facebook replied with HTTP 400 Bad Request but it doesn't matter now, I'm just trying to see that it reaches Facebook correctly, which it does.
Now, thing is, security team provided two proxies, and I should use them in failover mode, meaning after a (preferably configurable) number of timeouts from proxy 1, I should switch to proxy 2. Before I implement this feature in my application, I wanted to see if I can configure HAProxy to handle all this for me (as I'm already using HAProxy on this same server for inbound direction from Facebook, through another proxy).
So, this is my HAProxy configuration:
global
log 127.0.0.1 local2 debug
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
stats socket /etc/haproxy/haproxysock level admin
defaults
log global
option httplog
mode http
timeout connect 5000
timeout client 50000
timeout server 50000
# Incoming messages from Facebook
frontend chatbot_front
bind *:8773 ssl crt /etc/haproxy/haproxy.cer
use_backend chatbot_back
backend chatbot_back
balance roundrobin
server chatbot1 <chatbot_ip1>:<chatbot_port1> check ssl verify none
server chatbot2 <chatbot_ip2>:<chatbot_port2> check ssl verify none
# Outbound messages to Facebook through another proxy
frontend serverproxy_front
bind *:8775 ssl crt /etc/haproxy/haproxy.cer
use_backend serverproxy_back
backend serverproxy_back
balance roundrobin
option httpclose
option forwardfor header X-Client
cookie SERVERID insert indirect nocache
http-request set-header Proxy-Authorization "Basic <base64>"
server serverproxy1 <proxy_ip1>:<port1> check
server serverproxy2 <proxy_ip2>:<port2> check
listen stats
bind *:8774 ssl crt /etc/haproxy/haproxy.cer
mode http
maxconn 5
stats enable
stats refresh 10s
stats realm Haproxy\ Statistics
stats uri /stats
stats auth <user>:<password>
The relevant config is the frontend serverproxy_front and the backend serverproxy_back settings above. I tried to look into tutorials on how to set up HAProxy as forward proxy, and this is what I got. But when I try to use it, it doesn't work:
export http_proxy=http://127.0.0.1:8775
export https_proxy=http://127.0.0.1:8775
curl -v -I https://graph.facebook.com/v7.0/me/messages
* About to connect() to proxy <local_ip> port 8775 (#0)
* Trying <local_ip>...
* Connected to <local_ip> (<local_ip>) port 8775 (#0)
* Establish HTTP proxy tunnel to graph.facebook.com:443
> CONNECT graph.facebook.com:443 HTTP/1.1
> Host: graph.facebook.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
* Proxy CONNECT aborted
* Connection #0 to host <local_ip> left intact
curl: (56) Proxy CONNECT aborted
What am I missing here?
EDIT: fixed typo
I found it. It is an embarrassingly basic issue: the 8775 port on the haproxy was configured as "https", but I was using it as simple "http" for proxy URL...
Changing config to:
frontend serverproxy_front
# bind *:8775 ssl crt /etc/haproxy/haproxy.cer
bind *:8775
And restarting haproxy, it works successfully.

Why does tomcat returns "400 Bad Request" while using url which are similar to "something.com-xyz"

Why does Tomcat version 7.0.88 gives "400 Bad Request" error code if the hostname ends with xyx.com-abc.
For testing purpose let's assume we have the following entry in the hosts file
127.0.0.1 hello.hello.hello-erq
And we try to access this url from curl
curl -v hello.hello.hello-er:8080
We get the following output
* Rebuilt URL to: hello.hello.hello-er:8080/
* Trying 127.0.0.1...
* Connected to hello.hello.hello-er (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: hello.hello.hello-er:8080
> User-Agent: curl/7.49.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Server: Apache-Coyote/1.1
< Transfer-Encoding: chunked
< Date: Thu, 20 Dec 2018 19:53:09 GMT
< Connection: close
<
* Closing connection 0
While using the localhost in the url we get
C:\playground\apache-tomcat-7.0.88-windows-x64\apache-tomcat-7.0.88\bin>curl -v localhost:8080
* Rebuilt URL to: localhost:8080/
* Trying ::1...
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.49.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/html;charset=ISO-8859-1
< Transfer-Encoding: chunked
< Date: Thu, 20 Dec 2018 20:00:07 GMT
<
<!DOCTYPE html>
All the tomcat configurations are the same for both the tests and the same issue is replicated on a vanilla out of the box tomcat server too.
I tried to replicate the same issue on tomcat-8 but both the url's worked fine there. How can i dig deeper and find out the root cause of this issue in 7.0.88 ?
Is there some additional logging which i can enable to get more on this issue ?
Or the only thing i have left is to pull my hairs and upgrade ?
Tomcat was attempting to enforce the domain name specification by refusing your hostname with a hyphen in the TLD. This was deemed a bug in Tomcat and fixed in 7.0.89 (and versions of Tomcat 8.0.x, 8.5.x and 9.0.x released around the same time).
So it seems that all you need is a small version bump.

Configuring Traefik on Windows and http2 support

I started experimenting with Traefik on Windows 10 and I need assistance with some very basic steps. Two are my main stumbling blocks: configuring Traefik natively on Windows 10 and http2 support.
1) Native Traefik on Windows
Following the instructions at https://docs.traefik.io/ I successfully managed to install the docker image of Traefik and see the dashboard on localhost:8080
However, when I try to start Traefik natively as "traefik -c traefik.toml" Traefik seems to "hang" no matter what I put in traefik.toml.
Can anybody show me a minimal Windows 10 configuration (.toml) to achieve what I achieve using the Docker version of Traefik?
2) HTTP2 support
Following the test shown at https://docs.traefik.io/#test-it, I execute this on the command line:
curl -v --http2 -H Host:whoami.docker.localhost http://127.0.0.1
but on the output I see:
< HTTP/1.1 200 OK
< Content-Length: 327
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 05 Jul 2017 18:51:00 GMT
Does it mean that the response back is http 1.1 ?
Any help is much appreciated.

Hostname was NOT found in DNS cache for port 8080 but fine on port 80?

I am testing an API I have made using Springboot from my laptop (192.168.1.217:8080) and I am trying to get a cURL request via SSH from my Raspberry Pi.
Here is the error I am receiving when I try to send the request via port 8080 which it seems to not like:
pi#raspberrypi:~ $ curl -v 192.168.1.217:8080/api
* Hostname was NOT found in DNS cache
* Trying 192.168.1.217...
However cURL does work for the same IP but with port 80:
pi#raspberrypi:~ $ curl -v 192.168.1.217
* Rebuilt URL to: 192.168.1.217/
* Hostname was NOT found in DNS cache
* Trying 192.168.1.217...
* Connected to 192.168.1.217 (192.168.1.217) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.38.0
> Host: 192.168.1.217
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Thu, 30 Mar 2017 17:20:43 GMT
* Server Apache/2.4.23 (Win32) OpenSSL/1.0.2h PHP/5.5.38 is not blacklisted
< Server: Apache/2.4.23 (Win32) OpenSSL/1.0.2h PHP/5.5.38
< X-Powered-By: PHP/5.5.38
< Location: http://192.168.1.217/dashboard/
< Content-Length: 0
< Content-Type: text/html
<
* Connection #0 to host 192.168.1.217 left intact
pi#raspberrypi:~ $
I've tried looking around but to no avail... anybody have any suggestions as to why I cannot find my own hostname in the DNS cache?
Cheers
No, Hostname was NOT found in DNS cache is not the problem. You can clearly see on the next line, in both examples, that after saying that curl is trying to connect to 192.168.1.217. Your problem is that nothing is answering on port 8080 on that IP address (while an Apache server is answering on port 80 there).
If you're getting a long pause and then a timeout rather than a quick "Connection refused", you almost certainly need to open port 8080 in the local firewall on your server machine.
curl without a protocol prefix presumes HTTP port 80.
To use another port, where it does not make that assumption, all you need to do is change your command's URL to be like this:
curl -v http://192.168.1.217:8080/api
Here is a decent article on the subject: Using CURL For Testing Web Applications

Git on Windows not working with remote because of "SSL protocol" errors

tl;dr
Git on Windows stops connecting to github because of mysterious "SSL protocol" errors. Halp!
The Issue
I'm developing on Windows, using a private GitHub repo for source control. When I first boot my system, I'm able to access the remote repo without issue - pull, push, fetch, etc. all work just fine.
After some amount of time(*), this stops, and I get the following error:
fatal: unable to access 'https://github.com/our-team/private-repo.git/': Unknown SSL protocol error in connection to github.com:443
(*) The amount of time seems variable - I've witnessed as little as an hour or two, up to a whole day. Usually after coming back from the system sleeping, it seems to be an issue, but I don't know if it's caused by a time delay or by the system sleeping.
Checking via cURL, I get
λ curl -v "https://github.com/our-team/private-repo.git/"
* Trying 192.30.252.130...
* Connected to github.com (192.30.252.130) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: C:\Program Files (x86)\Git\bin\curl-ca-bundle.crt
CApath: none
* TLSv1.0, TLS handshake, Client hello (1):
* Unknown SSL protocol error in connection to github.com:443
* Closing connection 0
curl: (35) Unknown SSL protocol error in connection to github.com:443
Using set GIT_CURL_VERBOSE=1 with git pull shows similar information. Sometimes it succeeds (see below), but most of the time it fails.
Further Notes
There's a little bit of a sporadic nature to it - sometimes I can get requests to succeed, but once it starts exploding, it's generally broken 9 out of 10 requests or more.
A successful cURL request looks like:
λ curl -v "https://github.com/our-team/private-repo.git/"
* Trying 192.30.252.130...
* Connected to github.com (192.30.252.130) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: C:\Program Files (x86)\Git\bin\curl-ca-bundle.crt
CApath: none
* TLSv1.0, TLS handshake, Client hello (1):
* TLSv1.0, TLS handshake, Server hello (2):
* TLSv1.0, TLS handshake, CERT (11):
* TLSv1.0, TLS handshake, Server finished (14):
* TLSv1.0, TLS handshake, Client key exchange (16):
* TLSv1.0, TLS change cipher, Client hello (1):
* TLSv1.0, TLS handshake, Finished (20):
* TLSv1.0, TLS change cipher, Client hello (1):
* TLSv1.0, TLS handshake, Finished (20):
* SSL connection using TLSv1.0 / AES128-SHA
* Server certificate:
* subject: businessCategory=Private Organization; 1.3.6.1.4.1.311.60.2.1.3=US; 1.3.6.1.4.1.311.60.2.1.2=Delaware; serialNumber=5157550; street=548 4th Street; postalCode=94107; C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com
* start date: 2014-04-08 00:00:00 GMT
* expire date: 2016-04-12 12:00:00 GMT
* subjectAltName: github.com matched
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 Extended Validation Server CA
* SSL certificate verify ok.
> GET /our-team/private-repo.git/ HTTP/1.1
> User-Agent: curl/7.41.0
> Host: github.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: GitHub.com
< Date: Mon, 11 May 2015 15:19:43 GMT
< Content-Type: text/html
< Content-Length: 178
< Location: https://github.com/our-team/private-repo/
< Vary: Accept-Encoding
< X-Served-By: 76f8aa18dab86a06db6e70a0421dc28c
<
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host github.com left intact
The Question
I've googled a good bit on trying to find this (over the course of several weeks, so I don't have links), but most suggestions seem to point at certificate errors or OpenSSL version mismatches / bugs (which wouldn't be sporadic like this AFAIK).
What might be causing this failure, and how can I resolve it?
Relevant Software:
λ git --version
git version 1.9.5.msysgit.1
λ curl --version
curl 7.41.0 (i386-pc-win32) libcurl/7.41.0 OpenSSL/0.9.8zf zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz
Oddly, it turns out that the issue is that the laptop was throttled because of a weak power supply. The docking station I was using was plugged into a low-amp powersupply (3.3 A), which, while it was compatible with the laptop, immediately kicked it into a heavily-throttled mode.
Apparently, this slowed everything down enough that the SSL handshake wasn't able to complete fast enough.
We finally tracked it down after reading a Dell support forum post (http://en.community.dell.com/support-forums/laptop/f/3518/t/19363340) that discussed slowness issues. The solution there was to change the power supply.
I had also experienced this slowness, but I did not think it was related. We swapped to a high-amp power supply for the dock, and everything was fine again, and the SSL errors described above went away.
That looks like an error which could result from the security initiatives taken after the Logjam attack -- weakdh.org --.
That resulted in the suppression of some ciphers accepted in a SSL/TLS transaction.
Note that, as reported in "Cannot communicate securely with peer: no common encryption algorithm(s)", you will be able to pass the right cipher list to curl via git.
Before that, you can also try if the issue persists while using a more recent Git for Windows (like the Git 2.4.1)
Had the same issue. Disabled my wifi connection and switched to cable and everything works again. Btw: Used a Dell in Docking-Station too.

Resources