I have a socks4 server that I use to proxy requests from different kinds of applications (http, ftp, etc..) to their destination.
I know that socks4 does not define a way for applications to authenticate in any way (it specifies only an optional username when client connects to the server but it seems there is no way you can tell the client that is should send that information in order for the server to relay the request)
Is there a way to implement some kind of authentication over socks4 that is understood by the regular applications protocols: http, ftp, and others ?
Read up on how the SOCKS protocol works.
The second byte of a SOCKS v4/4a response is a status value that indicates whether the client's request was accepted or rejected (and why). In this case, you would likely need to use status values 0x5C (request failed because client is not running identd or it is not reachable from the server) and 0x5D (request failed because client's identd could not confirm the user ID string in the request). That would make the request's username field have to contain a valid and recognizable value. This is not true authentication, of course, since a client can run an ident server that reports whatever it wants.
Otherwise, switch to SOCKS v5 instead, which has support for true server-side authentication.
Related
I've read a lot of related topics in the net, but I still don't have an answer to my question.
Is it possible to implement flow described below?
Proxy receive request.
If request is encrypted and proxy cert is trusted then intercept.
If request is not encrypted, then intercept.
If request is encrypted and proxy cert is NOT trusted then pass it through without interception.
This behaviour should be default for all traffic going through the proxy.
It'd be also really nice to be able to get all possible info for passing encrypted requests (src and dst ip addresses etc.). Basically the same info which I can get with fiddler.
Not really. The main problem is that mitmproxy can not know if proxy cert is trusted by the client or not.
In the SSL/TLS protocol client starts with the CLIENT_HELLO and in response the server (in this case motmproxy) sends back the SERVER_HELLO message containing the generated server certificate.
The client now checks if the received server certificate is trusted. If not the connection is terminated. As far as I know the SSL/TLS spec does not define how to do so. Sems clients end back an SSL_ALERT message, other simply drop the connection, and a third group continues the SSL/TLS handshake but have certain internal values set in a way that always let the handshake fail.
There is a mitmproxy script that tries to identify connections that were not successful and then if the client asks for the same domain a second time bypasses interception.
Of course this requires that the client resends requests which is not always the case.
https://github.com/sociam/x-ray/blob/master/mitmproxy/examples/tls_passthrough.py
I am serving content locally, accessible through http://0.0.0.0:4000. That works ok, I get a correct webpage, which contains the following line inside a script:
var socket = io('http://example.com');
i.e. I am referencing an external server. Now my browser shows the followoing error:
GET http://example.com:4000/socket.io/?EIO=3&transport=polling&t=1417447089410-1 net::ERR_CONNECTION_REFUSED
That is, the browser is trying to connect using the same port that it used to get the original page.
Everything works fine when both the SocketIO server and the web server listen on the same port.
Am I missing something? Is this a bug? Is there a workaround? Thank you.
You can read here about how a plain webSocket is initially set up. It all starts with a somewhat standard HTTP GET request, but one that has some special headers set:
GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
The interchange may also allow the host to enforce requests only from web pages on certain origins. While this header can be spoofed from non-web-browser agents (so the server has to be prepared for that), it will likely be correct when the OP is using a real browser (assuming no proxy is modifying it).
If the server accepts the incoming request, it will then return an HTTP response that looks something like this:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
At this point, the socket which used to be an HTTP socket is now a webSocket and both endpoints have agreed that they're going to use the webSocket data format from now on. This initial connection may be followed by some form of authentication or new or existing cookies can also be used in the authentication during the initial HTTP portion of the connection.
socket.io adds some enhancements on top of this by initially requesting a particular path of /socket.io and adding some parameters to the URL. This allows socket.io to negotiate whether it's going to use long polling or a webSocket so there are some exchanges between client/server with socket.io before the above webSocket is initialized.
So, back to your question. The socket.io server simply spies at all incoming web requests on the normal web port (and looks for both it's special path and for special headers to indicate a webSocket initiation rather than a classic HTTP request). So, it runs over the same port as the web server. This is done for a bunch of reasons, all of which provide convenience to the server and server infrastructure since they don't have to configure their network to accept anything other than the usual port 80 they were already accepting (or whatever port they were already using for web requests).
By default in socket.io, the domain and port will default to the same domain and port as the web page you are on. So, if you don't specify one or the other in your connect call, it will use the domain or port from the web page you are on. If you want to use both a different domain and port, then you must specify both of them.
Our web application has a button that is supposed to send data to a server on the local network that in turn prints something on a printer.
So far it was easy: The button triggered an AJAX POST request to http://printerserver/print.php with a token, that page connected to the web application to verify the token and get the data to print and then printed.
However, we are now delivering our web application via HTTPs (and I would rather not go back to HTTP for this) and newer versions of Chrome and Firefox don't make the request to the HTTP address anymore, they don't even send the request to check CORS headers.
Now, what is a modern alternative to the cross-protocol XHR? Do Websockets suffer from the same problem? (A Google search did not make clear what is the current state here.) Can I use TCP Sockets already? I would rather not switch to GET requests either, because the action is not idempotent and it might have practical implications with preloading and caching.
I can change the application on the printerserver in any way (so I could replace it with NodeJS or something) but I cannot change the users' browsers (to trust a self-signed certificate for printerserver for example).
You could store the print requests on the webserver in a queue and make the printserver periodically poll for requests to print.
If that isn't possible I would setup a tunnel or VPN between the webserver and printserver networks. That way you can make the print request from the webserver on the server-side instead of the client. If you use curl, there are flags to ignore invalid SSL certificates etc. (I still suspect it's nicer to introduce a queue anyway, so the print requests aren't blocking).
If the webserver can make an ssh connection to something on the network where the printserver is on, you could do something like: ssh params user#host some curl command here.
Third option I can think of, if printserver can bind to for example a subdomain of the webserver domain, like: print.somedomain.com, you may be able to make it trusted by the somedomain.com certificate, IIRC you have to create a CSR (Certificate Signing Request) from the printserver certificate, and sign it with the somedomain.com certificate. Perhaps it doesn't even need to be a subdomain for this per se, but maybe that's a requirement for the browser to do it client-side.
The easiest way is to add a route to the webapp that does nothing more than relay the request to the print server. So make your AJAX POST request to https://myapp.com/print, and the server-side code powering that makes a request to http://printerserver/print.php, with the exact same POST content it received itself. As #dnozay said, this is commonly called a reverse proxy. Yes, to do that you'll have to reconfigure your printserver to accept (authenticated) requests from the webserver.
Alternatively, you could switch the printserver to https and directly call it from the client.
Note that an insecure (http) web-socket connection on a secure (https) page probably won't work either. And for good reason: generally it's a bad idea to mislead people by making insecure connections from what appears to them to be a secure page.
The server hosting the https webapp can reverse proxy the print server,
but since the printer is local to the user, this may not work.
The print server should have the correct CORS headers
Access-Control-Allow-Origin: *
or:
Access-Control-Allow-Origin: https://www.example.com
However there are pitfalls with using the wildcard.
From what I understand from the question, printserver is not accessible from the web application so the reverse proxy solution won't work here.
You are restricted from making requests from the browser to the printserver by cross-origin-policy.
If wish to communicate with the printserver from an HTTPS page you will need the printserver to expose print.php as HTTPS too.
You could create a DNS A record as a subdomain of your web application that resolves to the internal address of your printserver.
With those steps in place you should be able to update your printserver page to respond with permissive CORS headers which the browser should then respect. I don't think the browser will even issue CORS requests across different protocol schemes (HTTPS vs HTTP) or to internal domains, without a TLD.
I'm building a WebClient library. Now I'm implementing a proxy feature, so I am making some research and I saw some code using the CONNECT method to request a URL.
But checking it within my web browser, it doesn't use the CONNECT method but calls the GET method instead.
So I'm confused. When I should use both methods?
TL;DR a web client uses CONNECT only when it knows it talks to a proxy and the final URI begins with https://.
When a browser says:
CONNECT www.google.com:443 HTTP/1.1
it means:
Hi proxy, please open a raw TCP connection to google; any following
bytes I write, you just repeat over that connection without any
interpretation. Oh, and one more thing. Do that only if you talk to
Google directly, but if you use another proxy yourself, instead you
just tell them the same CONNECT.
Note how this says nothing about TLS (https). In fact CONNECT is orthogonal to TLS; you can have only one, you can have other, or you can have both of them.
That being said, the intent of CONNECT is to allow end-to-end encrypted TLS session, so the data is unreadable to a proxy (or a whole proxy chain). It works even if a proxy doesn't understand TLS at all, because CONNECT can be issued inside plain HTTP and requires from the proxy nothing more than copying raw bytes around.
But the connection to the first proxy can be TLS (https) although it means a double encryption of traffic between you and the first proxy.
Obviously, it makes no sense to CONNECT when talking directly to the final server. You just start talking TLS and then issue HTTP GET. The end servers normally disable CONNECT altogether.
To a proxy, CONNECT support adds security risks. Any data can be passed through CONNECT, even ssh hacking attempt to a server on 192.168.1.*, even SMTP sending spam. Outside world sees these attacks as regular TCP connections initiated by a proxy. They don't care what is the reason, they cannot check whether HTTP CONNECT is to blame. Hence it's up to proxies to secure themselves against misuse.
A CONNECT request urges your proxy to establish an HTTP tunnel to the remote end-point.
Usually is it used for SSL connections, though it can be used with HTTP as well (used for the purposes of proxy-chaining and tunneling)
CONNECT www.google.com:443
The above line opens a connection from your proxy to www.google.com on port 443.
After this, content that is sent by the client is forwarded by the proxy to www.google.com:443.
If a user tries to retrieve a page http://www.google.com, the proxy can send the exact same request and retrieve response for him, on his behalf.
With SSL(HTTPS), only the two remote end-points understand the requests, and the proxy cannot decipher them. Hence, all it does is open that tunnel using CONNECT, and lets the two end-points (webserver and client) talk to each other directly.
Proxy Chaining:
If you are chaining 2 proxy servers, this is the sequence of requests to be issued.
GET1 is the original GET request (HTTP URL)
CONNECT1 is the original CONNECT request (SSL/HTTPS URL or Another Proxy)
User Request ==CONNECT1==> (Your_Primary_Proxy ==CONNECT==> AnotherProxy-1 ... ==CONNECT==> AnotherProxy-n) ==GET1(IF is http)/CONNECT1(IF is https)==> Destination_URL
As a rule of thumb GET is used for plain HTTP and CONNECT for HTTPS
There are more details though so you probably want to read the relevant RFC-s
http://www.ietf.org/rfc/rfc2068.txt
http://www.ietf.org/rfc/rfc2817.txt
The CONNECT method converts the request connection to a transparent TCP/IP tunnel, usually to facilitate SSL-encrypted communication (HTTPS) through an unencrypted HTTP proxy.
I want a bit of clarity on whether HTTPS is stateful or stateless? This is with regards to a RESTful API I built. We were initially using HTTP. Since HTTP essentially works over TCP/IP which is stateless hence HTTP is stateless, but when I switched to HTTPS my API became stateful. I wanted to know whether my conclusion that HTTPS is stateful. is correct or not?
I created my API using a middleware tool called webMethods.
Thanks
TLS/SSL is stateful. The web server and the client (browser) cache the session including the cryptographic keys to improve performance and do not perform key exchange for every request.
HTTP 1 is not stateful. HTTP/2 however defines many stateful components, but the "application layer" still remains stateless.
TL;DR: The transport pipe (TLS) is stateful, original HTTP is not.
Additional note: Cookies and other stateful mechanisms are later additions defined in separate RFC's. They are not part of the original HTTP/1.0 specification, although other stateful mechanisms like caching and HTTP auth are defined HTTP 1.1 RFC and RFC 2617. HTTP 1 is said to be stateless although in practice we use standardized stateful mechanisms. HTTP/2 defines stateful components in its standard and is therefore stateful. A particular HTTP/2 application can use a subset of HTTP/2 features to maintain statelessness.
Theory aside, in practice you use HTTP statefully in your everyday life.
The S in HTTPS is concerned with the transport, not the protocol. The semantics of the HTTP protocol remain the same for HTTPS. As the article about HTTPS on Wikipedia states,
Strictly speaking, HTTPS is not a separate protocol, but refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
And the HTTP protocol is stateless by design, not because it is used most frequently over TCP/IP (nothing stops you to use HTTP over UDP for example).
HTTPS is HTTP over a secure connection.
HTTP is a higher level than a connection.
When connecting to a web server, your connection is (maybe always?) of type TCP/IP. So, in case you are visiting a website via HTTPS, your TCP/IP connection is encrypted.
The data the server and/or client send has not been encrypted by the server and/or client. It is just sent, as it is usually via HTTP, but this time using a connection via TCP/IP that is secured via encryption.
If data were vehicles, and the connexion the highway, then:
- using HTTP would be like the vehicles going on the highway, and everyone can see them;
- using HTTPS would be like the same, but the vehicles go through a tunnel or anything that prevents people not on the highway from seeing them. You can determine there is trafic, but you cannot identify the vehicles, except on both ends of the tunnel.
I believe this is an image close to what happens behind the scene. But I'm no expert. I just hope it helps.
HTTP and HTTPS both are stateless protocols. The S in HTTPS stands for Secure and it refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
Use of JWT tokens or the traditional way of establishing sessions using cookies help us to overcome the problem of HTTP being a stateless protocol, as it enables the server to authenticate the identity of the client, so that you don't need to login every time you click a link to navigate on the web-page.
So For example, when you log in to the website of your bank, it only asks you to enter your login details once. Once you are signed in, you don't need to re-enter them when you navigate to the account settings page, this is because the bank site is able to authenticate your identity using JWT tokens.
JWT tokens are only used on HTTPS and not in HTTP, because the connection is encrypted in HTTPS, so it cannot be intercepted by anyone.
Thus, HTTP and HTTPS both are stateless protocols, but JWT Tokens provides a workaround for it.
I believe HTTPS is a stateful protocol as it contains Session identifier field.This generated by server initially to identify a session with the chosen client.