I'm using Go 1.6 and want to make a HTTP2-only request over http://.
Attempting to do this currently results in:
Head http://localhost:2076/completed/764c1b6bc55548707507a2dd25570483a7216bf4: http2: unsupported scheme
To force http2, I believe I need http.Client.Transport.TLSConfig.NextProtos set to []string{"h2"}.
What else is required?
You need to use https, not http. The http2 transport doesn't recognize the http scheme.
The HTTP/2.0 by default works on highly secured connections. It uses high quality ciphers. So it can only run on HTTPS connections. Moreover, to make HTTPS connections, you also need to have your SSL enabled and have the required certificates installed.
Related
[![enter image description here][1]][1]
Below image has command
[1]: https://i.stack.imgur.com/aTH9u.png
HTTP/2 is negotiated in one of three ways:
As part of the HTTPS set up when using HTTPS - this is the recommended method
As an upgrade when using HTTP after establishing the initial connection over HTTP/1.1. Note this is rarely used so is due to be deprecated in next release of HTTP/2 spec.
By connecting directly using HTTP/2 when on HTTP if you know for certain it supports HTTP/2 by telling you client this.
You are using HTTP not HTTPS so the first option it not available to you.
So it used the second option. It DID use HTTP/2 - you can see that in the second half of your request. But see my note about deprecation.
If you want ti go straight to HTTP/2 for curl you can use the --http2-prior-knowledge command line option.
I'm trying to set up a proxy server on my local mac.
http - seems to work.
But Safari is not connecting via https.
Did I miss something?
No it doesn't. You need to specify a separate https port and a ssl certificate, as documented in the squid config:
The socket address where Squid will listen for client requests made
over TLS or SSL connections. Commonly referred to as HTTPS.
This is most useful for situations where you are running squid in
accelerator mode and you want to do the TLS work at the accelerator
level.
You may specify multiple socket addresses on multiple lines, each
with their own certificate and/or options.
The tls-cert= option is mandatory on HTTPS ports.
See http_port for a list of modes and options.
http://www.squid-cache.org/Doc/config/https_port/
By design, it is quite hard to intercept https traffic:
When a browser creates a direct secure connection with an origin
server, there are no HTTP CONNECT requests. The first HTTP request
sent on such a connection is already encrypted. In most cases, Squid
is out of the loop: Squid knows nothing about that connection and
cannot block or proxy that traffic.
You also need to load the proxy settings for the browser as a PAC file, otherwise the browsers won't connect or throw a certificate warning:
Chrome The Chrome browser is able to connect to proxies over SSL
connections if configured to use one in a PAC file or command line
switch. GUI configuration appears not to be possible (yet).
More details at
http://dev.chromium.org/developers/design-documents/secure-web-proxy
Firefox The Firefox 33.0 browser is able to connect to proxies over
TLS connections if configured to use one in a PAC file. GUI
configuration appears not to be possible (yet), though there is a
config hack for embedding PAC logic.
There is still an important bug open:
Using a client certificate authentication to a proxy:
https://bugzilla.mozilla.org/show_bug.cgi?id=209312
https://wiki.squid-cache.org/Features/HTTPS
I am trying to relay a stream that is being broadcasted over HTTPS, is there a way to be able to do that? The documentation describes how to broadcast with https using listen-socket which I think is not what I want. All the help would be appreciated
I tried relaying normal http streams and it works. But not with https
I tried doing it both with including https and without in the url
`<relay>
<server>https://streamingurl.com</server>
<port>800</port>
<mount>/f</mount>
<local-mount>/f</local-mount>
<on-demand>0</on-demand>
<relay-shoutcast-metadata>0</relay-shoutcast-metadata>
</relay>`
Unfortunately this is currently not possible.
A good workaround for this problem is to set up a reverse proxy using nginx. I did this to access a https stream over http and icecast2 is able to relay it without issues.
What is the origin server you are trying to relay? Another Icecast or something else?
The -kh fork if Icecast supports SSL and has a lot of extensions and may be able to relay a https stream. (Sorry I'm not more help with that) see https://karlheyes.github.io
You're not supposed to include http or https in the context, just the address.
<relay>
<server>sourceip</server>
<port>443</port>
<mount>/sourcemount</mount>
<local-mount>/localmount</local-mount>
<on-demand>0</on-demand>
<relay-shoutcast-metadata>1</relay-shoutcast-metadata>
</relay>
I just tested that with a -kh branch icecast server, and it worked, BUT I wasn't able to confirm it was actually making a SSL connection; but it is making a connection. The kh fork or Icecast will accept http or https over 80 or 443 (or any other port for that matter).
How we can enable squid to cache web content (let says from firefox) for SSL connection, i mean for https URLs?
Actually SQUID can be used to access HTTPS traffic - it is in essence a man-in-the-middle attack - and there are caveats:
See: http://wiki.squid-cache.org/Features/SslBump
I have not tried cacheing this data yet, so can't say that it will work with absolute certainty. If/when I do, I'll update this post.
SSL encrypts the traffic between server and client so it cannot be read by a middle man. When using Squid as a proxy it simply cannot see the actual content in the traffic and therefore it has no means of caching it. The SSL traffic is just random bits that look different each time even if the same content is transferred multiple times and that is how encryption should work. It simply cannot be cached.
I have no problems getting Firefox (version 23.0.1 on Windows) to route SSL traffic via Squid. In Firefox Network Connection settings I just point SSL Proxy and HTTP Proxy to the same Squid installation.
After that I can successfully access https URLs in Firefox and in Squid's access_log I see entries like these:
1379660084.878 115367 10.0.0.205 TCP_MISS/200 6581 CONNECT www.gravatar.com:443 - DIRECT/68.232.35.121 -
Do you have any details about how it doesn't work for you? Squid has quite complicated possibilities to deny and allow certain types of traffic, so it is possible there is a configuration issue in Squid. Do you get any error messages in Squid's logfiles?
I've seen several examples of writing an HTTP proxy in Ruby, e.g. this gist by Torsten Becker, but how would I extend it to handle HTTPS, aka for a "man in the middle" SSL proxy?
I'm looking for a simple source code framework which I can extend for my own logging and testing needs.
update
I already use Charles, a nifty HTTPS proxy app similar to Fiddler and it is essentially what I want except that it's packaged up in an app. I want to write my own because I have specific needs for filtering and presentation.
update II
Having poked around, I understand the terminology a little better. I'm NOT after a full "Man in the Middle" SSL proxy. Instead, it will run locally on my machine and so I can honor whatever SSL cert it offers. However, I need to see the decrypted contents of packets of my requests and the decrypted contents of the responses.
Just for background information, a normal HTTP proxy handles HTTPS requests via the CONNECT method: it reads the host name and port, establishes a TCP connection to this target server on this port, returns 200 OK and then merely tunnels that TCP connection to the initial client (the fact that SSL/TLS is exchanged on top of that TCP connection is barely relevant).
This is what the do_CONNECT method if WEBrick::HTTPProxyServer.
If you want a MITM proxy, i.e. if you want to be able to look inside the SSL/TLS traffic, you can certainly use WEBrick::HTTPProxyServer, but you'll need to change do_CONNECT completely:
Firstly, your proxy server will need to embed a mini CA, capable of generating certificates on the fly (failing that, you might be able to use self-signed certificates, if you're willing to bypass warning messages in the browser). You would then import that CA certificate into the browser.
When you get the CONNECT request, you'll need to generate a certificate valid for that host name (preferable with a Suject Alt. Name for that host name, or in the Subject DN's Common Name), and upgrade the socket into an SSL/TLS server socket (using that certificate). If the browser accepts to trust that certificate, what you get from thereon on this SSL/TLS socket is the plain text traffic.
You would then have to handle the requests (get the request line, headers and entity) and take it to use it via a normal HTTPS client library. You might be able to send that traffic to a second instance of WEBrick::HTTPProxyServer, but it would have to be tweaked to make outgoing HTTPS requests instead of plain HTTP requests.
Webrick can proxy ssl:
require 'webrick'
require 'webrick/httpproxy'
WEBrick::HTTPProxyServer.new(:Port => 8080).start
from my experience HTTPS is nowhere near "simple". Do you need a proxy that would catch traffic from your own machine? There are several applications, like Fiddler. Or google for alternatives. Comes with everything you need to debug the web traffic.
That blog is no way to write a proxy. It's very easy: you just accept a connection, read one line which tells you what to connect to, attempt the upstream connection, if it fails send the appropriate response and close the socket, otherwise just start copying bytes in both directions, simultaneously, until EOS has occurred in both directions. The only difference HTTPS makes is that you have to speak SSL instead of plaintext.