I am looking for a specific kind of proxy that is meant to operate in a rendezvous mode, such that two nodes can make an outgoing connection to the same proxy, send a routing token, and have their packets relayed to each other from that point.
Proxy servers like HAProxy would be perfect but AFAIK they do not offer something like that: the goal of the proxy in this case is to make another outgoing connection and route the packets to that location. In this case, I want two nodes to connect to the proxy, and have their packets relayed between them through the proxy, after sending a routing token that can be used to associate the two nodes.
I could write my own server to perform such type of relaying, but I am wondering if something already exists to do something like this. I am looking for such a solution as a fallback for cases where NAT traversal protocols like ICE/STUN/TURN are not feasible due to a highly restricted network environment that does not allow UDP traffic. The base protocol for the proxy could be TCP, HTTP or WebSocket, which would be easier to allow in a firewall with a simple rule.
Any ideas or recommendations?
I believes SOCKSv5 has everything you are asking for.
two nodes can make an outgoing connection to the same proxy. send a routing token, and have their packets relayed to each other from that point.
The routing token in this case would be the endpoint address and/or the user credentials. I would first look at the supper simple implementation built into the 'ssh' utility, this guide goes over how to get everything set up. If you need something more granular then look into dante.
The only tricky part is when you try to use the user credential option with SOCKv5 as it is not as well supported in browsers, but is possible with addons.
Related
I am trying to find (or write) a caching proxy tool that accepts all traffic from a specific container in my localhost (using Iptables). What I want to do with this traffic is to save it and cache the response, and later, if I see that a request was already sent to a server, return the cached response to the requesting party (and not sending the request to the server again, because a previous similar request was already sent).
Here's a diagram to demonstrate what I'm trying to do:
I'm not sure exactly how big is the problem I'm trying to deal with here. I want to do it for all traffic, including HTTP, TLS and other TCP based traffic (database connections and such). I tried to check mitmproxy, and it seems to deal pretty good with HTTP and the TLS part, but intercepting raw TCP traffic (for databases etc.) is not possible.
Any advices or resources I can use to accomplish that? (Not necessarily in Python). How complex do you think this problem is? Do you think I can find a generic solution?
Thanks in advance!
According to the CoAP specification RFC7252 section 5.7 two types of CoAP proxies exists, namely forward-proxy and reverse-proxy.
Orthogonal to this definition a proxy can be CoAP-CoAP or HTTP-CoAP proxy.
I have the following questions according to these definitions:
What are the use cases where each of these proxy types (forward-proxy, reverse-proxy, CoAP-CoAP and HTTP-CoAP) has to be used?
What exactly does the proxy in those use cases?
Which part of the CoAP message does each proxy need in order to accomplish its tasks? (I assume that proxy looks at some of the CoAP options?)
Where runs the proxy? On the boarder router?
It well be very nice if someone can provide in addition to the answers to the questions above also good reverences and examples.
Some use cases for proxy times are:
Forward proxies: Escaping a restrictive firewall (eg. because outgoing connections are blocked unless they go through a proxy); a special case thereof is accessing resources on CoAP from within a browser, when a CoAP-over-WebSocket server is used as a forward proxy.
Reverse proxies: Roughly the same as for forward proxies but in the reverse direction (ie. if the server is in a firewalled network, there can be a reverse proxy running on the firewall that lets requests in selectively). Additionally, reverse proxies can be used for load balancing on the unconstrained server side.
HTTP-CoAP-proxies: Accessing CoAP devices from programming environments that don't have good CoAP support (eg. if your monitoring system supports polling a HTTP server, a H-C proxy easily allows monitoring your CoAP servers).
CoAP-HTTP proxies: Making services on the big web available to constrained devices (eg. querying a weather forecast). This is often of limited use as those web services don't use data formats suitable for a constrained device.
That barely varies over the cases; apart from the obvious (cross-proxies translating CoAP and HTTP messages back and forth), forward and reverse proxies only differ in whether they react to messages with a Proxy-* option (typically Proxy-Scheme) set (forward proxies), or to those without (reverse proxies).
All proxies need to inspect the full message (ie. all header fields and look at all options). Most header fields (token, message-ID, mtype to some extent) and several options (eg. Observe) are hop-by-hop, ie. the server acts on them and creates new values for the forwarded request. If there is anything in the message the proxy would need to understand but doesn't (ie. unknown Proxy-Unsafe options), it rejects forwarding.
That is fully up to the application, but in most of the common examples, that would be some kind of border router; could be the 6LBR in a 6LoWPAN scenario, or the firewall of a larger setup. In cases of routing between a CoAP-over-WebSocket endpoint and the rest of the CoAP world, the CoAP-over-WebSocket server often runs on the same machine that also serves the web application that uses the socket.
I am trying to capture an old application that didn't honour the system's proxy setting. The only config I can change is the server IP address.
Capturing the packets with Wireshark. Without the Charles reverse proxy, I can see requests after the first three handshake requests.
With the reverse proxy, the connection stuck after the handshake requests.
I notice that when Charles received a request and connecting to somewhere but it will just stuck there:
Following is the config of the reverse proxy (Remote host removed):
Any help, solution and workarounds would be appreciated!
First of all, your app uses neither HTTP nor HTTPS. Studying screen shot of successful connection gives some details on protocol used:
the first message after handhsake is originated by server contrary to common client-server approach, where client is responsible for sending query. This fact is enough to cross out HTTP and HTTPS.
payload data isn't human-readable, so it's a binary protocol.
based on PUSH flags, protocol is much more likely to be message-based rather than stream-based
So client establishes connection, immediately gets some command from server and replies it. Then communication continues. I can't guess exact protocol. Port number might be irrelevant, but even if it's not, there are only few protocols using 4321 port by default. Anyway, it can always be custom private protocol.
I'm not familiar with Charles, but forwarding arbitrary TCP stream is probably covered by its port forwarding feature rather than reverse proxy. However, I don't really see any benefits in sending traffic through Charles in this case, capturing data on your PC should be enough to study details.
If you are looking for traffic manipulation, for arbitrary TCP stream it's not an easy task, but it must be possible. I'm not aware of suitable tools, quick googling shows lots of utils, but some of them looks applicable to text based stream only, so deeper study is required.
Reason for Failure
It may be because you are requesting a local IP address from a remote scope, which Charles proxy doesn't applies. For POS(Proof Of Statement), please refer to the below link
https://www.charlesproxy.com/documentation/faqs/localhost-traffic-doesnt-appear-in-charles/
Solution
So In order to solve the problem for the current scenario, use
http://192.168.86.22.charlesproxy.com/
Note: The url that you request will only be proxied properly by Charles not any other proxy services.
Is there any protocol, API or software in existence that can send data/IM/etc directly from one device to another with no server?
Can you not use HTTP GET/POST/DELETE directly between two devices when their device data is known to the user(s)?
I would very much like to know if there is ANY software/protocols that can do this.
thank you!
The internet is build on the Internet Protocol suite. This suite has 5 different layers of protocols: The physical layer, the link layer, the network layer, the transport layer and the application layer. Each depends on the one before.
If you just use the browser, by default HTTP (application layer) is used, which relies on TCP (transport layer), which relies on IP (v4 or v6, network layer), which relies on ethernet (link layer), which finally relies on the actual cable that's plugged into your computer (for WiFi, the first three are the same but the last two differ if I'm not mistaken).
Now to your question: Is there any protocol, API or software in existence that can send data/IM/etc directly from one device to another with no server?
Yes there is. I suggest you start looking at protocols that are in the application layer. To highlight a few standards next to HTTP(S): FTP is for file transfer, IMAP is for emails clients, SMTP is for email servers and SSH is a secure shell which can also be used to tunnel data through.
For your specific case, I think either FTP (FTPS if you want it over SSL), or SSH can be a solution, but it's hard to know for sure without the specifics.
The only thing that these protocols have in common is that one of the two computers will act like server and the other computer as client. This has as downside that port-forwarding might be necessary.
If you've chosen the protocol you'd like to use, then you're up for the next step, selecting a program that can do that for you. For HTTP(S), I'd recommend Apache. If you're using Linux and chose SSH, then you're in luck, there is a build in SSH server in Linux, you can use that. For other protocols, you might just want to search yourself, as I do not have any suggestions.
I hope this answers most of your questions!
Desirius
In browser context, WebRTC is probably what you are looking for: It allows user to user communications.
https://en.wikipedia.org/wiki/WebRTC
https://webrtc.org/
That is to say, if I have a server listening on 127.0.0.1, and a TCP connection comes in, how can I determine the process id of the client?
Also if there isn't an API for this, where would I be able to extract the information from in a more hackish manner?
(The purpose of this is to modify a local HTTP proxy server to accept or deny requests based on the requesting process.)
Edit: palacsint's answer below led me to find this answer to a similar question which is just what's needed
netstat -a -o
prints it. I suppose they are on the same machine becase you are listening on 127.0.0.1.
The only way to do this is if the connecting process sends some sort of custom headers which contains identifier. This is due to the fact that the networking layer is completely separated from the application layer (hint: OSI MODEL. This way it is possible to write lower layers software without caring what happens above as long as the messages exchanged (read: networking packets) follow a pre-determined format (read: use the same protocol).