That is to say, if I have a server listening on 127.0.0.1, and a TCP connection comes in, how can I determine the process id of the client?
Also if there isn't an API for this, where would I be able to extract the information from in a more hackish manner?
(The purpose of this is to modify a local HTTP proxy server to accept or deny requests based on the requesting process.)
Edit: palacsint's answer below led me to find this answer to a similar question which is just what's needed
netstat -a -o
prints it. I suppose they are on the same machine becase you are listening on 127.0.0.1.
The only way to do this is if the connecting process sends some sort of custom headers which contains identifier. This is due to the fact that the networking layer is completely separated from the application layer (hint: OSI MODEL. This way it is possible to write lower layers software without caring what happens above as long as the messages exchanged (read: networking packets) follow a pre-determined format (read: use the same protocol).
Related
I am looking for a specific kind of proxy that is meant to operate in a rendezvous mode, such that two nodes can make an outgoing connection to the same proxy, send a routing token, and have their packets relayed to each other from that point.
Proxy servers like HAProxy would be perfect but AFAIK they do not offer something like that: the goal of the proxy in this case is to make another outgoing connection and route the packets to that location. In this case, I want two nodes to connect to the proxy, and have their packets relayed between them through the proxy, after sending a routing token that can be used to associate the two nodes.
I could write my own server to perform such type of relaying, but I am wondering if something already exists to do something like this. I am looking for such a solution as a fallback for cases where NAT traversal protocols like ICE/STUN/TURN are not feasible due to a highly restricted network environment that does not allow UDP traffic. The base protocol for the proxy could be TCP, HTTP or WebSocket, which would be easier to allow in a firewall with a simple rule.
Any ideas or recommendations?
I believes SOCKSv5 has everything you are asking for.
two nodes can make an outgoing connection to the same proxy. send a routing token, and have their packets relayed to each other from that point.
The routing token in this case would be the endpoint address and/or the user credentials. I would first look at the supper simple implementation built into the 'ssh' utility, this guide goes over how to get everything set up. If you need something more granular then look into dante.
The only tricky part is when you try to use the user credential option with SOCKv5 as it is not as well supported in browsers, but is possible with addons.
I am trying to capture an old application that didn't honour the system's proxy setting. The only config I can change is the server IP address.
Capturing the packets with Wireshark. Without the Charles reverse proxy, I can see requests after the first three handshake requests.
With the reverse proxy, the connection stuck after the handshake requests.
I notice that when Charles received a request and connecting to somewhere but it will just stuck there:
Following is the config of the reverse proxy (Remote host removed):
Any help, solution and workarounds would be appreciated!
First of all, your app uses neither HTTP nor HTTPS. Studying screen shot of successful connection gives some details on protocol used:
the first message after handhsake is originated by server contrary to common client-server approach, where client is responsible for sending query. This fact is enough to cross out HTTP and HTTPS.
payload data isn't human-readable, so it's a binary protocol.
based on PUSH flags, protocol is much more likely to be message-based rather than stream-based
So client establishes connection, immediately gets some command from server and replies it. Then communication continues. I can't guess exact protocol. Port number might be irrelevant, but even if it's not, there are only few protocols using 4321 port by default. Anyway, it can always be custom private protocol.
I'm not familiar with Charles, but forwarding arbitrary TCP stream is probably covered by its port forwarding feature rather than reverse proxy. However, I don't really see any benefits in sending traffic through Charles in this case, capturing data on your PC should be enough to study details.
If you are looking for traffic manipulation, for arbitrary TCP stream it's not an easy task, but it must be possible. I'm not aware of suitable tools, quick googling shows lots of utils, but some of them looks applicable to text based stream only, so deeper study is required.
Reason for Failure
It may be because you are requesting a local IP address from a remote scope, which Charles proxy doesn't applies. For POS(Proof Of Statement), please refer to the below link
https://www.charlesproxy.com/documentation/faqs/localhost-traffic-doesnt-appear-in-charles/
Solution
So In order to solve the problem for the current scenario, use
http://192.168.86.22.charlesproxy.com/
Note: The url that you request will only be proxied properly by Charles not any other proxy services.
Is there any protocol, API or software in existence that can send data/IM/etc directly from one device to another with no server?
Can you not use HTTP GET/POST/DELETE directly between two devices when their device data is known to the user(s)?
I would very much like to know if there is ANY software/protocols that can do this.
thank you!
The internet is build on the Internet Protocol suite. This suite has 5 different layers of protocols: The physical layer, the link layer, the network layer, the transport layer and the application layer. Each depends on the one before.
If you just use the browser, by default HTTP (application layer) is used, which relies on TCP (transport layer), which relies on IP (v4 or v6, network layer), which relies on ethernet (link layer), which finally relies on the actual cable that's plugged into your computer (for WiFi, the first three are the same but the last two differ if I'm not mistaken).
Now to your question: Is there any protocol, API or software in existence that can send data/IM/etc directly from one device to another with no server?
Yes there is. I suggest you start looking at protocols that are in the application layer. To highlight a few standards next to HTTP(S): FTP is for file transfer, IMAP is for emails clients, SMTP is for email servers and SSH is a secure shell which can also be used to tunnel data through.
For your specific case, I think either FTP (FTPS if you want it over SSL), or SSH can be a solution, but it's hard to know for sure without the specifics.
The only thing that these protocols have in common is that one of the two computers will act like server and the other computer as client. This has as downside that port-forwarding might be necessary.
If you've chosen the protocol you'd like to use, then you're up for the next step, selecting a program that can do that for you. For HTTP(S), I'd recommend Apache. If you're using Linux and chose SSH, then you're in luck, there is a build in SSH server in Linux, you can use that. For other protocols, you might just want to search yourself, as I do not have any suggestions.
I hope this answers most of your questions!
Desirius
In browser context, WebRTC is probably what you are looking for: It allows user to user communications.
https://en.wikipedia.org/wiki/WebRTC
https://webrtc.org/
An XMMP server sends push notifications to a client behind a NAT using a public endpoint( IP + Port) supplied by NAT to client. But how long this endpoint is assigned to this specific client by NAT, what will happen if NAT assigns same endpoint to another client ? How this problem can be solved?
XMPP uses a standard TCP connection. NATs will keep the association for as long as the connection is alive (unless they are horribly broken).
Update: The last part of my statement could have been expanded a bit. Horribly broken NAT implementations do exist. Generally these are a small percentage, but many (most?) popular XMPP clients do ensure they send some kind of keepalive over idle connections.
There are three kinds of keepalive you can use I'll list them here in order of bandwidth/processing requirements:
TCP keepalives are a good lightweight option, especially as once they are enabled, they are automatically handled by the OS. How to enable them will depend on your language and framework, but at the lowest level, you need to enable the SO_KEEPALIVE option on the socket.
There are two problems with TCP keepalives. One is that you can't control them from your application (unless you write platform-specific code). The second problem is that some NAT implementations are so broken that they will ignore TCP keepalives too! But you're hopefully down to a very small percentage now.
So another option is whitespace keepalives. Since these involve data going across the stream, you should be safe from even the broken NATs that ignore keepalives.
Whitespace keepalives simply involve sending the space character (' ') across the XMPP stream at any time it is idle. XML and XMPP allow unlimited whitespace between elements, and it is simply ignored by the recipient.
Finally, you can use fully-fledged XMPP pings (XEP-0199). These involve ending an actual <iq/> 'get' stanza to the server, which then must reply. This ensures data flows in both directions, and should make even the most broken NAT implementations keep your connection alive.
Ok, I should mention that there is an even worse class of NAT. I have seen NATs that will simply 'forget' about your mapping for a range of reasons, including their mapping table being full, or just after a timer. There is nothing you can do to work around these, they don't work with any long-lived TCP connections. The best you could probably do at that point is use BOSH (essentially XMPP over HTTP).
Conclusion: If you are concerned that your application may run behind some of these devices, I suggest something like the following algorithm (exact times may be tweaked, but I recommend these as minimum values):
If you have not sent any data for 60s, send a single space character.
If you have not received any data for 120s, send an XMPP ping to your server.
If the server doesn't reply to the ping within a reasonable amount of time, reconnect.
Because the behaviour of broken NAT devices is beyond any standard protocol specification, it is naturally impossible to devise a perfect solution that will work with all of them, all of the time. You just have to accept that these are a small minority, and none of this matters for working NAT devices (though there are other kinds of network breakages that may make regular keepalives/pings a good idea, depending on the needs of your application).
The Solution is sending keep alive messages to maintain the NAT entry. XMPP whitespace is typically used. Send it eg every Ten minutes to preserve reachability of the nated client.
You have to keep in mind that NAT is no standardized technique. Thus there are different implementations. The provided RFCs in the comment above is from the BEHAVE working group.
In a recent series of question I have asked alot about UDP, boost::asio and c++ in general.
My latest question, which doesn't seem to have an answer here at Stackoverflow, is this:
In a client/server application, it is quite okay to require that the server open a port in any firewall, so that messages are allowed in. However, doing the same for clients is definately not a great user experience.
TCP-connections typically achieve this due to the fact that most routers support stateful packet inspection, allowing response packets through if the original request originated from the local host.
It is not quite clear to me how this would work with UDP, since UDP is stateless, and there is no such thing as "response packets" (to my knowledge). How should I account for this in my client application?
Thanks for any answers!
UDP itself is stateless, but the firewall typically is not. The convention on UDP is that if a request goes out from client:port_A to server:port_B, then the response will come back from server:port_B to client:port_A.
The firewall can take advantage of this. If it sees a UDP request go out from the client, it adds an entry to its state table that lets it recognise the response(s), to allow them in. Because UDP is stateless and has no indication of connection termination, the firewall will typically implement a timeout - if no traffic occurs between that UDP address pair for a certain amount of time, the association in the firewall's state table is removed.
So - to take advantage of this in your client application, simply ensure that your server sends responses back from the same port that it uses to receive the requests.