I need to extend a HTTP recorder to record websocket traffic. HTTP recorder is in C / unix platform.
I know HTTP traffic will be through port 80 or 443, hence I am able to record it. In HTTP message a new port is communicated, and through that port websocket traffic passes through. Since this port is dynamic, and HTTP messages will be different in different cases I can not parse and get port for websocket.
I think of below logic, not sure if that is going to work. Need your suggestion / pointers in this regard.
Make use of iptables
Forward all the traffic to a different port (where my application is listening) using iptable rules
Then my application should receive the data (hope it is possible)
Then my application should receive the destination information (when data is forwarded from iptables to my application, can I get actual destination ip/port).
Assuming, I get actual ip/port, then from my application, I can send data to actual destination.
I know, its a long way to go, will this approach work.
Related
As my knowledge, in normal behavior, client will make random port to connect the SNMP service port, now the client use SNMP trap port (162) to communicate with server.
My questions:
As the client setting, I do not have configure any SNMP setting for the client, why WireShark able to capture SNMP traffic came from the client?
Why the client use SNMP trap port (162) to communicate with server, rather than using random port?
SNMP is usually transmitted over UDP, so there is actually no "connection", and speaking technically the source port doesn't matter. You can just send out datagrams (e.g. traps) without binding to a port.
However, even when run over UDP, SNMP does involve some two-way communication. If you are expecting a response (which a client does if it's sending an SNMP Get or Set request), the only place the other end knows to send it is back where the request came from, i.e. the original IP/port combination. There's no information in a SNMP packet that provides any alternative "return address" information.
So, in order to get a response on a predictable port, you'll send the request datagram from a bound socket. Typically the client will run its own listening "server" on port 162, send requests from there, and then it can receive responses there too. Otherwise you wouldn't see the responses. This also allows us to set up simple firewall rules (though you can often get away without firewall rules for the return path, due to hole punching*).
This is also true for the server, which emits traps and informs on a known, standard, predictable port not only so that you can configure your trap receiver and firewalls in a reliable way, but so that inform responses can be sent back to a known, standard, predictable port that you're listening on.
tl;dr: You can send your requests from an arbitrary port if you like, but it's not very useful.
* My SNMP implementation seemed buggy when the client/receiver only saw traps emitted during the ~15 minutes after it had last poked out some kind of request packet. Subsequent traps seemed to be completely missing. After much debugging on the server end, it turned out that we'd forgotten to open the correct port on the inbound firewall for the client, and were accidentally relying on hole punching, which has a time limit. :D
As for why Wireshark is seeing traffic from an unconfigured SNMP client, well, either your SNMP client actually is configured to send requests, or you're misinterpreting the results. Wireshark doesn't invent traffic. Without a more complete picture of your network setup, software setup, and those packets you're seeing, we could only speculate as to the exact cause of your confusion.
I am trying to capture an old application that didn't honour the system's proxy setting. The only config I can change is the server IP address.
Capturing the packets with Wireshark. Without the Charles reverse proxy, I can see requests after the first three handshake requests.
With the reverse proxy, the connection stuck after the handshake requests.
I notice that when Charles received a request and connecting to somewhere but it will just stuck there:
Following is the config of the reverse proxy (Remote host removed):
Any help, solution and workarounds would be appreciated!
First of all, your app uses neither HTTP nor HTTPS. Studying screen shot of successful connection gives some details on protocol used:
the first message after handhsake is originated by server contrary to common client-server approach, where client is responsible for sending query. This fact is enough to cross out HTTP and HTTPS.
payload data isn't human-readable, so it's a binary protocol.
based on PUSH flags, protocol is much more likely to be message-based rather than stream-based
So client establishes connection, immediately gets some command from server and replies it. Then communication continues. I can't guess exact protocol. Port number might be irrelevant, but even if it's not, there are only few protocols using 4321 port by default. Anyway, it can always be custom private protocol.
I'm not familiar with Charles, but forwarding arbitrary TCP stream is probably covered by its port forwarding feature rather than reverse proxy. However, I don't really see any benefits in sending traffic through Charles in this case, capturing data on your PC should be enough to study details.
If you are looking for traffic manipulation, for arbitrary TCP stream it's not an easy task, but it must be possible. I'm not aware of suitable tools, quick googling shows lots of utils, but some of them looks applicable to text based stream only, so deeper study is required.
Reason for Failure
It may be because you are requesting a local IP address from a remote scope, which Charles proxy doesn't applies. For POS(Proof Of Statement), please refer to the below link
https://www.charlesproxy.com/documentation/faqs/localhost-traffic-doesnt-appear-in-charles/
Solution
So In order to solve the problem for the current scenario, use
http://192.168.86.22.charlesproxy.com/
Note: The url that you request will only be proxied properly by Charles not any other proxy services.
I am using Ruby's net/http library to send RESTful requests to my dev server. I've been having issues correctly parsing the data I'm sending in the request and so I'm not looking into ways to inspect the request I'm sending out through Ruby.
On another question I posted here, asking how JSON data was sent in an http request, I received a comment saying that I could possibly inspect the request by using TCPMon. In order to use TCPMon though to check the request, I need to know which port to look for the request on.
And that brings me here. What port does Ruby (and net/http) use to send out HTTP requests?
Unless otherwise specified, it will make requests to port 80 for http requests, and port 443 for https requests (the standard http/https ports).
However, you'll set up TCPMon to listen to requests on a certain port, and then forward them on to their destination. So if TCPMon is listening on 8080, you'll make the requests to port 8080.
The standard port for HTTP is port 80. If you don't specify a port in the URL, that's what it will use.
There's nothing special about port 80, it's just the standard for HTTP. You can run your server on any numeric port you want, test servers often run on something like 8080 or 8000, but you'll have to ensure your clients are connecting to that port.
Here's a list of standard TCP and UDP ports.
I am trying to tunnel websockets over TCP. I know how to tunnel HTTPS - "Connect URL:port" is where I start. From there, one opens a socket to the target and then just pipe between the client and the target. Where do I start with websockets? is there something similar to a "Connect url:port" to begin with, which I can catch in my http server and then do some upgrade processing?
You sure you want to tunnel WebSocket over TCP? WebSocket runs over TCP.
I think you mean you want to tunnel TCP traffic with WebSocket, i.e., take an existing, traditional Socket-based application and make it work over the web. If that's what you mean, you essentially put your TCP data in a WebSocket frame and on the receiving end you read the WebSocket frame and extract the data. Of course this is easier said than done. You have to make sure you create the WebSocket frame correctly on the sender side (also handling the TCP data stream, which also may be tricky), encrypt the data (its going over the web, right?) and on the receiving side read the WebSocket frame, extract the data from the right parts of the frame. And also you need to check to see if all the data is in one WebSocket frame or multiple frames.
As I said, its not dead simple.
There are several WebSocket libraries out there that may (or may) not handle all of this for you (many do not handle the multiple WebSocket frame situation).
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose of using existing web/HTTP infrastructures? And I think it no longer fits the name WebSocket on non-80 ports.
If I use WebSocket over other ports, why not just use TCP directly? Or is there any special benefits in the WebSocket protocol itself?
And since current WebSocket handshake is in the form of a HTTP UPGRADE request, does it mean I have to enable HTTP protocol on the port so that WebSocket handshake can be accomplished?
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose
of using existing web/HTTP infrastructures? And I think it no longer
fits the name WebSocket on non-80 ports.
You can run a webSocket server on any port that your host OS allows and that your client will be allowed to connect to.
However, there are a number of advantages to running it on port 80 (or 443).
Networking infrastructure is generally already deployed and open on port 80 for outbound connections from the places that clients live (like desktop computers, mobile devices, etc...) to the places that servers live (like data centers). So, new holes in the firewall or router configurations, etc... are usually not required in order to deploy a webSocket app on port 80. Configuration changes may be required to run on different ports. For example, many large corporate networks are very picky about what ports outbound connections can be made on and are configured only for certain standard and expected behaviors. Picking a non-standard port for a webSocket connection may not be allowed from some corporate networks. This is the BIG reason to use port 80 (maximum interoperability from private networks that have locked down configurations).
Many webSocket apps running from the browser wish to leverage existing security/login/auth infrastructure already being used on port 80 for the host web page. Using that exact same infrastructure to check authentication of a webSocket connection may be simpler if everything is on the same port.
Some server infrastructures for webSockets (such as socket.io in node.js) use a combined server infrastructure (single process, one listener) to support both HTTP requests and webSockets. This is simpler if both are on the same port.
If I use WebSocket over other ports, why not just use TCP directly? Or
is there any special benefits in the WebSocket protocol itself?
The webSocket protocol was originally defined to work from a browser to a server. There is no generic TCP access from a browser so if you want a persistent socket without custom browser add-ons, then a webSocket is what is offered. As compared to a plain TCP connection, the webSocket protocol offers the ability to leverage HTTP authentication and cookies, a standard way of doing app-level and end-to-end keep-alive ping/pong (TCP offers hop-level keep-alive, but not end-to-end), a built in framing protocol (you'd have to design your own packet formats in TCP) and a lot of libraries that support these higher level features. Basically, webSocket works at a higher level than TCP (using TCP under the covers) and offers more built-in features that most people find useful. For example, if using TCP, one of the first things you have to do is get or design a protocol (a means of expressing your data). This is already built-in with webSocket.
And since current WebSocket handshake is in the form of a HTTP UPGRADE
request, does it mean I have to enable HTTP protocol on the port so
that WebSocket handshake can be accomplished?
You MUST have an HTTP server running on the port that you wish to use webSocket on because all webSocket requests start with an HTTP request. It wouldn't have to be heavily featured HTTP server, but it does have to handle the initial HTTP request.
Yes - Use 443 (ie, the HTTPS port) instead.
There's little reason these days to use port 80 (HTTP) for anything other than a redirection to port 443 (HTTPS), as certification (via services like LetsEncrypt) are easy and free to set up.
The only possible exceptions to this rule are local development, and non-internet facing services.
Should I use a non-standard port?
I suspect this is the intent of your question. To this, I'd argue that doing so adds an unnecessary layer of complication with no obvious benefits. It doesn't add security, and it doesn't make anything easier.
But it does mean that specific firewall exceptions need to be made to host and connect to your websocket server. This means that people accessing your services from a corporate/school/locked down environment are probably not going to be able to use it, unless they can somehow convince management that it is mandatory. I doubt there are many good reasons to exclude your userbase in this way.
But there's nothing stopping you from doing it either...
In my opinion, yes you can. 80 is the default port, but you can change it to any as you like.