Use HAProxy with proxy chain - proxy

I'm trying to connect to internet services (specifically Facebook API) from an internal application, and for that I have to go through one of two proxies provided by the security team. If I use one of these proxies with CURL then it works OK:
export http_proxy=http://<user>:<pass>#<proxy_ip>:<port>/
export https_proxy=http://<user>:<pass>#<proxy_ip>:<port>/
curl -v -I https://graph.facebook.com/v7.0/me/messages
* About to connect() to proxy <proxy_ip> port <port> (#0)
* Trying <proxy_ip>...
* Connected to <proxy_ip> (<proxy_ip>) port <port> (#0)
* Establish HTTP proxy tunnel to graph.facebook.com:443
* Proxy auth using Basic with user '<user>'
> CONNECT graph.facebook.com:443 HTTP/1.1
> Host: graph.facebook.com:443
> Proxy-Authorization: Basic <base64>
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=*.facebook.com,O="Facebook, Inc.",L=Menlo Park,ST=California,C=US
* start date: May 14 00:00:00 2020 GMT
* expire date: Aug 05 12:00:00 2020 GMT
* common name: *.facebook.com
* issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
> HEAD /v7.0/me/messages HTTP/1.1
> User-Agent: curl/7.29.0
> Host: graph.facebook.com
> Accept: */*
>
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< WWW-Authenticate: OAuth "Facebook Platform" "invalid_request" "An active access token must be used to query information about the current user."
Facebook replied with HTTP 400 Bad Request but it doesn't matter now, I'm just trying to see that it reaches Facebook correctly, which it does.
Now, thing is, security team provided two proxies, and I should use them in failover mode, meaning after a (preferably configurable) number of timeouts from proxy 1, I should switch to proxy 2. Before I implement this feature in my application, I wanted to see if I can configure HAProxy to handle all this for me (as I'm already using HAProxy on this same server for inbound direction from Facebook, through another proxy).
So, this is my HAProxy configuration:
global
log 127.0.0.1 local2 debug
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
stats socket /etc/haproxy/haproxysock level admin
defaults
log global
option httplog
mode http
timeout connect 5000
timeout client 50000
timeout server 50000
# Incoming messages from Facebook
frontend chatbot_front
bind *:8773 ssl crt /etc/haproxy/haproxy.cer
use_backend chatbot_back
backend chatbot_back
balance roundrobin
server chatbot1 <chatbot_ip1>:<chatbot_port1> check ssl verify none
server chatbot2 <chatbot_ip2>:<chatbot_port2> check ssl verify none
# Outbound messages to Facebook through another proxy
frontend serverproxy_front
bind *:8775 ssl crt /etc/haproxy/haproxy.cer
use_backend serverproxy_back
backend serverproxy_back
balance roundrobin
option httpclose
option forwardfor header X-Client
cookie SERVERID insert indirect nocache
http-request set-header Proxy-Authorization "Basic <base64>"
server serverproxy1 <proxy_ip1>:<port1> check
server serverproxy2 <proxy_ip2>:<port2> check
listen stats
bind *:8774 ssl crt /etc/haproxy/haproxy.cer
mode http
maxconn 5
stats enable
stats refresh 10s
stats realm Haproxy\ Statistics
stats uri /stats
stats auth <user>:<password>
The relevant config is the frontend serverproxy_front and the backend serverproxy_back settings above. I tried to look into tutorials on how to set up HAProxy as forward proxy, and this is what I got. But when I try to use it, it doesn't work:
export http_proxy=http://127.0.0.1:8775
export https_proxy=http://127.0.0.1:8775
curl -v -I https://graph.facebook.com/v7.0/me/messages
* About to connect() to proxy <local_ip> port 8775 (#0)
* Trying <local_ip>...
* Connected to <local_ip> (<local_ip>) port 8775 (#0)
* Establish HTTP proxy tunnel to graph.facebook.com:443
> CONNECT graph.facebook.com:443 HTTP/1.1
> Host: graph.facebook.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
* Proxy CONNECT aborted
* Connection #0 to host <local_ip> left intact
curl: (56) Proxy CONNECT aborted
What am I missing here?
EDIT: fixed typo

I found it. It is an embarrassingly basic issue: the 8775 port on the haproxy was configured as "https", but I was using it as simple "http" for proxy URL...
Changing config to:
frontend serverproxy_front
# bind *:8775 ssl crt /etc/haproxy/haproxy.cer
bind *:8775
And restarting haproxy, it works successfully.

Related

HAProxy and websockets never connect

I thought with HAProxy 2.4 supporting WebSockets over h2, I would finally be able to get websockets over HAProxy to work... But no...
When my browser tries to connect, it instantly gives up on the "connect?transport=webSockets" and tries SSE. This "connect" also is instantly abandoned and yet it "start?transport=SSE" just fine.
Logging in HAProxy says it never even got the requests for connect.
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:18.520] www-httpsStrict~ test_backend/werdc2020 557/0/0/2/558 200 704 - - --VN 2/1/0/0/0 0/0 "GET https://myserver.com/signalr/negotiate?clientProtocol=2.1&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D&_=1637143818090 HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.075] www-httpsStrict~ test_backend/werdc2020 52/0/0/1/52 404 1392 - - --VN 2/1/0/0/0 0/0 "GET https://myserver.com/favicon.ico HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.279] www-httpsStrict~ test_backend/werdc2020 96/0/0/7/103 200 339 - - --VN 3/2/1/1/0 0/0 "GET https://myserver.com/signalr/start?transport=serverSentEvents&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D&_=1637143818091 HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.375] www-httpsStrict~ test_backend/werdc2020 65/0/0/20/84 200 348 - - --VN 3/2/1/1/0 0/0 "POST https://myserver.com/signalr/send?transport=serverSentEvents&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D HTTP/2.0"
If I connect directly on the server it works just fine...
HAProxy config:
global
log 127.0.0.1 local0 debug
log-tag haproxy
maxconn 10000
user haproxy
group haproxy
daemon
nbproc 1
nbthread 8
#cpu-map auto:all 0-1
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 30s
pidfile /var/run/haproxy.pid
tune.ssl.default-dh-param 2048
tune.ssl.cachesize 100000
tune.ssl.lifetime 600
tune.ssl.maxrecord 1460
# intermediate configuration
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:!MD5:!aNULL:!DH:!RC4
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5s
timeout client 1m
timeout client-fin 1m
timeout server 1h
timeout tunnel 1h
frontend stats
bind 0.0.0.0:8080
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth myuser:1111
stats refresh 60s
# Force front server MUST use redirect on HTTPS
frontend www-httpStrict
bind *:80
option splice-auto
# Test URI to see if its a letsencrypt request
acl letsencrypt-acl path_beg /.well-known
acl test hdr_sub(cookie) istest=true
redirect scheme https code 307 if !{ ssl_fc } !letsencrypt-acl
use_backend letsencrypt if letsencrypt-acl
use_backend test_backend if test
default_backend www-backendStrict
frontend www-httpsStrict
bind *:443 tfo ssl crt /etc/ssl/private alpn h2,http/1.1
option forwardfor
option splice-auto
acl test hdr_sub(cookie) istest=true
use_backend test_backend if test
default_backend www-backendStrict
backend letsencrypt
server nginx 127.0.0.1:8888
# this backend require haproxy open SSL tunel to port 443 on webservers
backend www-backendStrict
balance roundrobin
cookie MyWebFarm insert
option forwardfor
option splice-auto
option tcp-smart-connect
option httpchk
http-check connect ssl alpn h2 sni myserver.com
http-check send meth HEAD uri /login.aspx ver HTTP/2 hdr Host myserver.com
http-check expect status 200-399
http-request add-header X-Forwarded-Proto https if { ssl_fc }
retry-on all-retryable-errors
http-request disable-l7-retry if METH_POST
default-server ssl tfo verify none alpn h2,http/1.1 check allow-0rtt
server werdc01 192.168.1.5:443 cookie werc01 check
server werdc2020 192.168.1.6:443 cookie werdc2020 check
backend test_backend
balance roundrobin
cookie MyWebFarm insert
option forwardfor
option splice-auto
option tcp-smart-connect
option httpchk
http-check connect ssl alpn h2 sni myserver.com
http-check send meth HEAD uri /login.aspx ver HTTP/2 hdr Host myserver.com
http-check expect status 200-399
http-request add-header X-Forwarded-Proto https if { ssl_fc }
retry-on all-retryable-errors
http-request disable-l7-retry if METH_POST
default-server ssl verify none alpn h2,http/1.1 check allow-0rtt
server werdc2020 192.168.1.6:443 cookie werdc2020 check
I have the istest cookie set to always hit the test_backend...
Headers sent from browser:
GET /signalr/connect?transport=webSockets&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22catiservice%22%7D%5D&tid=7 undefined
Host: myserver.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: https://myserver.com
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: P1GaqaLxIxBfuXDCtxuQwg==
DNT: 1
Connection: keep-alive, Upgrade
Cookie: _ga=GA1.2.95033637.1627965955; .MyAuth=D758016C45CF1B9BDC38D17DCAF7A3CEE7E528B97352267A1B0D019BFE20964BF2900C655950E126CD7FC08CB421C1A997D1A5FDB0266316B920634FB2C23B63080B271B6331096902B47BF73661240D; MyWebFarm=werdc2020; ASP.NET_SessionId=w1zibnnuht4ohicpntyhn55q; istest=true
Sec-Fetch-Dest: websocket
Sec-Fetch-Mode: websocket
Sec-Fetch-Site: same-origin
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
TE: trailers
Why doesn't HAProxy see the "connect" requests?
Why doesn't it connect with WebSockets?
There is a github issue on this: https://github.com/haproxy/haproxy/issues/162
Although your question isn't very insightful, I'd recommend to strip down your config to the bare minimum to test websocket.
Here's a snippet to consider
option http-server-close
option http-use-htx
acl hdr_connection_upgrade hdr(Connection) -i upgrade
acl hdr_upgrade_websocket hdr(Upgrade) -i websocket
Why doesn't HAProxy see the "connect" requests?
I suppose, just because the CONNECT has not been successful. If you'd use tcpdump you pretty sure would see a request incoming.
Using the explanation from #sgohl this config works (at least until 2.5 fixes the issue):
global
log 127.0.0.1 local0 debug
log-tag haproxy
maxconn 10000
user haproxy
group haproxy
daemon
nbproc 1
nbthread 8
#cpu-map auto:all 0-1
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 30s
pidfile /var/run/haproxy.pid
tune.ssl.default-dh-param 2048
tune.ssl.cachesize 100000
tune.ssl.lifetime 600
tune.ssl.maxrecord 1460
# intermediate configuration
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:!MD5:!aNULL:!DH:!RC4
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
defaults all
log global
mode http
option httplog
option dontlognull
timeout connect 5s
timeout client 1m
timeout client-fin 1m
timeout server 1h
timeout tunnel 4h
option splice-auto
option tcp-smart-connect
balance roundrobin
option forwardfor
retry-on all-retryable-errors
frontend stats from all
bind *:8080
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth Myuser:11111
stats refresh 60s
listen www from all
bind *:80
bind *:443 tfo ssl crt /etc/ssl/private alpn h2,http/1.1 allow-0rtt
# Test URI to see if its a letsencrypt request
acs letsencrypt path_beg /.well-known
http-request redirect scheme https unless { ssl_fc } or letsencrypt
use_backend letsencrypt if letsencrypt
http-response set-header Strict-Transport-Security "max-age=10886400; includeSubDomains; preload;"
use_backend ws if { hdr(Connection) -i upgrade } or { hdr(Upgrade) -i websocket }
default_backend h2
backend letsencrypt
server letsencrypt 127.0.0.1:8888
defaults cg from all
cookie MyWebFarm insert
option httpchk
http-check connect ssl alpn h2 sni myserver.com
http-check send meth HEAD uri /login.aspx ver HTTP/2 hdr Host myserver.com
http-check expect status 200-399
backend h2 from cg
default-server ssl tfo verify none alpn h2 check allow-0rtt
http-request disable-l7-retry if METH_POST
use-server werdc2020 if { hdr_sub(cookie) istest=true }
server werdc01 192.168.1.5:443 cookie werc01
server werdc2020 192.168.1.6:443 cookie werdc2020
backend ws from cg
default-server ssl tfo verify none alpn http/1.1 check allow-0rtt
http-request disable-l7-retry if METH_POST
use-server werdc2020 if { hdr_sub(cookie) istest=true }
server werdc01 192.168.1.5:443 cookie werc01
server werdc2020 192.168.1.6:443 cookie werdc2020

what is https and http tunnel methods in proxy?

This image is ProxyDroid application and I saw some proxy soft wares like these one.
I find some free servers for http method (http proxy the famous one) and find servers for socks 4 and 5 but I cant find any server that support https and http tunnel and in other word I cant understand what are exactly these protocols.
Proxying HTTPS is done with a HTTP proxy by using the CONNECT request. Using this request the HTTP proxy is instructed to create a tunnel to the target server. Inside this tunnel the client can the do the TLS handshake needed for HTTPS:
> CONNECT example.org:443 HTTP/1.0
>
... proxy established TCP connection to example.org:443 ...
< HTTP/1.0 200 Connection established
<
... tunnel to target established
... proxy forwards data between client and target unchanged
<-> TLS handshake
<-> application data protected by TLS
HTTP tunnel is similar. Normally a HTTP request gets proxied by sending a HTTP proxy request:
> GET http://example.org/index.html HTTP/1.0
> Host: example.org
>
... proxy connects to target example.org and forwards request
... then sends response from target server back
< HTTP/1.0 200 ok
< Content-length: ...
< ...
With HTTP tunnel the client instead uses the CONNECT method described above to create a tunnel to the target server and send the request:
> CONNECT example.org:80 HTTP/1.0
>
< HTTP/1.0 200 Connection established
<
... tunnel established, send HTTP request over tunnel and get reply back
> GET /index.html HTTP/1.0
> Host: example.org
> ...
< HTTP/1.0 200 ok
< ...

Hostname was NOT found in DNS cache for port 8080 but fine on port 80?

I am testing an API I have made using Springboot from my laptop (192.168.1.217:8080) and I am trying to get a cURL request via SSH from my Raspberry Pi.
Here is the error I am receiving when I try to send the request via port 8080 which it seems to not like:
pi#raspberrypi:~ $ curl -v 192.168.1.217:8080/api
* Hostname was NOT found in DNS cache
* Trying 192.168.1.217...
However cURL does work for the same IP but with port 80:
pi#raspberrypi:~ $ curl -v 192.168.1.217
* Rebuilt URL to: 192.168.1.217/
* Hostname was NOT found in DNS cache
* Trying 192.168.1.217...
* Connected to 192.168.1.217 (192.168.1.217) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.38.0
> Host: 192.168.1.217
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Thu, 30 Mar 2017 17:20:43 GMT
* Server Apache/2.4.23 (Win32) OpenSSL/1.0.2h PHP/5.5.38 is not blacklisted
< Server: Apache/2.4.23 (Win32) OpenSSL/1.0.2h PHP/5.5.38
< X-Powered-By: PHP/5.5.38
< Location: http://192.168.1.217/dashboard/
< Content-Length: 0
< Content-Type: text/html
<
* Connection #0 to host 192.168.1.217 left intact
pi#raspberrypi:~ $
I've tried looking around but to no avail... anybody have any suggestions as to why I cannot find my own hostname in the DNS cache?
Cheers
No, Hostname was NOT found in DNS cache is not the problem. You can clearly see on the next line, in both examples, that after saying that curl is trying to connect to 192.168.1.217. Your problem is that nothing is answering on port 8080 on that IP address (while an Apache server is answering on port 80 there).
If you're getting a long pause and then a timeout rather than a quick "Connection refused", you almost certainly need to open port 8080 in the local firewall on your server machine.
curl without a protocol prefix presumes HTTP port 80.
To use another port, where it does not make that assumption, all you need to do is change your command's URL to be like this:
curl -v http://192.168.1.217:8080/api
Here is a decent article on the subject: Using CURL For Testing Web Applications

cURL unable to use curl-ca-bundle.crt when accessing SSL

I'm on Windows attempting to use cURL using SSL, but running into certificate issues that I absolutely cannot figure out.
For example, here is an example of what I'm trying to run.
$ curl "https://google.com" --ntlm -v --negotiate -u USERNAME:PASSWORD --proxy "PROXY" --cert "c:\temp\curl-ca-bundle.crt"
* Adding handle: conn: 0x147ce88
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x147ce88) send_pipe: 1, recv_pipe: 0
* About to connect() to proxy PROXY port 8080 (#0)
* Trying 192.168.134.80...
* Connected to PROXY (PROXY_IP) port 8080 (#0)
* Establish HTTP proxy tunnel to google.com:443
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> User-Agent: curl/7.30.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* unable to use client certificate (no key found or wrong pass phrase?)
* Closing connection 0
curl: (58) unable to use client certificate (no key found or wrong pass phrase?)
Attempting to use --cacert instead of --cert yields the following message -
* Adding handle: conn: 0x130cdf8
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x130cdf8) send_pipe: 1, recv_pipe: 0
* About to connect() to proxy PROXY port 8080 (#0)
* Trying 192.168.135.80...
* Connected to PROXY (PROXY_IP) port 8080 (#0)
* Establish HTTP proxy tunnel to google.com:443
> CONNECT google.com:443 HTTP/1.1
> Host: google.com:443
> User-Agent: curl/7.30.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* successfully set certificate verify locations:
* CAfile: c:\temp\curl-ca-bundle.crt
CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use the
-k (or --insecure) option.
The curl-ca-bundle.crt I have is from here, which was update a few minutes ago. I ensured the file is not blocked by Windows.
For what it's worth, I'm behind a corporate proxy and firewall. I have read everything I could find on this issue and don't know what to do next. I do realize I can ignore SSL, but would like to avoid this at all costs.
For what it's worth, I'm behind a corporate proxy and firewall. I have read everything I could find on this issue and don't know what to do next. I do realize I can ignore SSL, but would like to avoid this at all costs.
I would suggest the firewall does SSL interception, i.e. it works as a man-in-the-middle. To inspect encrypted connections it will split it into two encrypted connections, but of course it will not be able to sign the connection between browser and proxy with the original certificate. Thus it will create a new certificate, signed by a CA specific to the firewall. You need to add this CA to your CA bundle or the verification will fail.
For more details you might try to access the target site with a web browser and check the CA which signed the servers certificate or ask your system administrator for details of the SSL interception.

Webrick proxy + cURL returns bad request

I'm going nuts, I have a very simple WEBrickProxy (WEBrick 1.3.1, with Ruby 1.9.3), and I want to try it out with curl. Here is the proxy:
require 'webrick'
require 'webrick/httpproxy'
server = WEBrick::HTTPProxyServer.new(:BindAddress => "localhost", :Port => 8888)
trap('INT') { server.shutdown }
server.start
And here is the cURL command
curl --proxy localhost:8888 http://www.google.de -v
But the curl command always returns a bad request:
* About to connect() to proxy localhost port 8888 (#0)
* Trying ::1...
* connected
* Connected to localhost (::1) port 8888 (#0)
> GET http://www.google.de HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: www.google.de
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 400 Bad Request
< Content-Type: text/html; charset=ISO-8859-1
< Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12)
< Date: Mon, 18 Mar 2013 13:44:27 GMT
< Content-Length: 295
< Connection: close
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
<HEAD><TITLE>Bad Request</TITLE></HEAD>
<BODY>
<H1>Bad Request</H1>
bad URI `http://www.google.de'.
<HR>
<ADDRESS>
WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12) at
23tuxmb.local:8888
</ADDRESS>
</BODY>
</HTML>
* Closing connection #0
curl --version returns on my Mac OS X 10.8
curl 7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz
I can't find out where the error is, and I tried that piece of code a couple of weeks ago, and I can remember that it worked.
The weird thing is, that when I configure my Mac to globally use that proxy (inside the System Settings -> Network -> Advanced -> Proxies -> Web Proxy), everything works. A request with Chrome or any other application is routed through the proxy (and I see this inside the Terminal log of the proxy).
So, can anyone reproduce this issue? Is it a curl bug? Or a Webrick related issue?
EDIT
More information: The output of the ruby script itself when curl tries to connect is
[2013-03-18 17:16:32] ERROR bad URI `http://www.amazon.de'.
localhost - - [18/Mar/2013:17:16:32 CET] "GET http://www.amazon.de HTTP/1.1" 400 286
- -> http://www.amazon.de
thx!
If you add a trailing slash to the URL you request, does it work then? Ie a command line like this:
curl --proxy localhost:8888 http://www.google.de/ -v
(this is a curl bug reported in bug #1206, and fixed in git and next release...)

Resources