I want to write a general solution to relay data that comes from different application protocols to a proxy (socks and or/http) in order to do web filtering. Are there any open source projects for windows that do that ?
Check out cntlm. Here's the link:
http://cntlm.sourceforge.net/
It's main selling point is that it does NTLM authentication, but it also works well as a general-purpose proxy. It also has a gateway mode that allows it to proxy network traffic to another proxy server.
Related
HTTP proxy with SSL and DNS support.
I must be lacking some key concepts about proxy-ing because I cannot grasp this. I am looking to run a simply http or https proxy without interfering with SSL. Simply, a fully transparent proxy that can passthrough all the traffic to the browser connected via HTTP or HTTPS proxy without modifying or intercepting any packets. Not able to find any code online or I'm not using the right keywords.
EX. On the browser adding server.someVPN.com:80 on the HTTP proxy field and as soon as you try to visit a website, it prompts for authentication. Then it works perfectly with any domain, any security, any ssl, no further steps needed. Most VPN providers have this.
How's this possible? it even resolves DNS itself. I thought on transparent proxy the dns relies on the client. Preferably looking for a nodeJS solution but any lang works.
Please don't propose any solutions such as SOCKS5 or sock forwarding or DNS overriding or CA based MITM. According to HTTP 1.1 which supports 'CONNECT' this should be easy.
Not looking to proxy specific domains, looking for an all inclusive solution just like most VPN Providers providers.
----Found the answer too quickly, feel free to delete this post/question admins.
The way it works is that the browser knows it is talking to a proxy server, so for example if the browser want to connect to htttp://www.example.com it sends a CONNECT www.example.com:443 HTTP/1.1 to the proxy server, the proxy server resolves wwww.example.com via DNS and then opens a TCP connection to wwww.example.com port 443 and proxies the TCP stream transparently to the client.
I don't know any solution for nodejs. Common proxy servers include Squid, Privoxy and Apache Traffic Server
See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT
Found the solution right after I asked...
This module works perfectly https://github.com/mpangrazzi/harrier
Does exactly what I was asking for.
I heard from somewhere WebSphere server can be configured to allow only a pre-defined set systems (IPs, domains) to access server contents such as WSDL.
Need help with following questions:
Is this supported?
What are the steps to do the same?
Prerequisites?
Thanks!!
Check this post Ban ip for deny access to an app. It describes various methods (configuring transport channels, http server, or developing custom filter). Although it talks about banning access, same technique can be applied to allow access.
Take notice however that using transport channel will apply to all applications and resources available via given port (not just wsdl). So the solution depends on level of granularity you require.
I have setup a windows 2003 ftp server and using chilkat to connect to this ftp inside my customized application. My application is developed in VB6 with ftp support of chilkat. The application works on different places of the city and connects to my ftp. Unable to access ftp and transfer files using the customised application, from some networks like idea netsetter / bsnl. It works perfect on other networks.
Thanks in advance.
Regards,
Sam
This is likely to be a firewall issue at the client end. FTP is often blocked by firewalls.
Just as well, FTP has its problems making it a less than ideal alternative. There are better options such as SFTP or FTPS but support for those is limited in Windows and you'll have to buy both server and client pieces to use one of them.
Fewer firewalls block HTTP and HTTPS though some are finicky enough to block traffic that doesn't look like Web browsing. Stiil, your odds of success go up substantially.
An obvious choice might be to use WebDAV. IIS supports WebDAV and it is pretty easy to write simple WebDAV client logic in VB6 based on one of the many HTTP components available. I'd probably use XmlHttpRequest or WinHttpRequest for that. A search ought to turn up several VB6 classes written to wrap one of them to support WebDAV client operations. You can also buy WebDAV client libraries.
Stick to using HTTPS (which means you need a server cetificate for IIS) and you won't have passwords going over the network in the clear. Even if you use HTTP you'll be no worse off than using FTP, plus it'll work through the vast majority of firewalls except those that specifically block non-browsing HTTP requests.
This could be a firewall configuration on the Client or Server. You're not going to be able to do much about the client, but for the server it may depend on whether your doing Active or Passive FTP connections.
If you are doing Active connections, make sure ports 20 and 21 are open.
If you're doing Passive connections, you may want to check out this article about configuring the PassivePortRange in Server 2003 FTP- http://support.microsoft.com/?id=555022.
I have an app, and it makes an https connection to a server. Is it possible to use something like wireshark or charlesproxy to just see the useragent that it's connecting with? I don't want to see any of the actual data, just the useragent - but I'm not sure if that is encrypted as well? (and if it's worth trying)
Thanks
Is it possible to...
No. Browser first establishes secure connection with server, then use it for transfer all data including requests' data, various headers etc.
Too late for the original inquirer, but the answer is that it may be possible in some cases, depending on application implementation.
You can use fiddler, and by turning on the 'decrypt https traffic' you also have visibility to the HTTPS content in some cases.
What fiddler does (on windows at least) is register itself within the wininet as system proxy. It can also add certificates (requires your approval when you select to decrypt https traffic) and generates on the fly certificates for the accessed domains, thus being MitM.
Applications using this infrastructure will be 'exposed' to this MitM. I ran fiddler and ran a few applications and was able to view https traffic related to office products (winword, powerpoint, outlook) other MS executables (Searchprotocolhost.exe) but also to some non-microsoft products such as apple software update, cisco jabber)
How does Netnanny or k9 Web Protection setup web proxy without configuring the browsers?
How can it be done?
Using WinSock directly, or at the NDIS or hardware driver level, and
then filter at those levels, just like any firewalls soft does. NDIS being the easy way.
Download this ISO image: http://www.microsoft.com/downloads/en/confirmation.aspx?displaylang=en&FamilyID=36a2630f-5d56-43b5-b996-7633f2ec14ff
it has bunch of samples and tools to help you build what you want.
After you mount or burn it on CD and install it go to this folder:
c:\WinDDK\7600.16385.1\src\network\ndis\
I think what you need is a transparent proxy that support WCCP.
Take a look at squid-cache FAQ page
And the Wikipedia entry for WCCP
With that setup you just need to do some firewall configuration and all your web traffic will be handled by the transparent proxy. And no setup will be needed on your browser.
netnanny is not a proxy. It is tied to the host machine and browser (and possibly other applications as well. It then filters all incoming and outgoing "content" from the machine/application.
Essentially Netnanny is a content-control system as against destination-control system (proxy).
Easiest way to divert all traffic to a certain site to some other address is by changing hosts file on local host
You might want to have a look at the explanation here: http://www.fiddlertool.com/fiddler/help/hookup.asp
This is how Fiddler2 achieves inserting a proxy in between most apps and the internet without modifying the apps (although lots of explanation of how-to failing the default setup). This does not answer how NetNanny/K9 etc work though, as noted above they do a little more and may be a little more intrusive.
I believe you search for BrowserHelperObjects. These little gizmos capture ALL browser communication, and as such can either remote ads from the HTML (good gizmo), or redirect every second click to a spam site (bad gizmo), or just capture every URL you type and send it home like all the WebToolBars do.
What you want to do is route all outgoing http(s) requests from your lan through a reverse proxy (like squid). This is the setup for a transparent web proxy.
There are different ways to do this, although I've only ever set it up OpenBSD and Linux; and using Squid as the reverse proxy.
At a high level you have a firewall with rules to send all externally bound http traffic to a local squid server. The Squid server is configured to:
accept all http requests
forward the requests on to the real external hosts
cache the reply
forward the reply back to the requestor on the local lan
You can then add more granular rules in Squid to control access to websites, filter content, etc.
I pretty sure you can also get this functionality in different networking gear. I bet F5 has some products that do some or all of what I described, and probably Cisco as well. There is probably other proxies out there besides Squid that you can use too.
PS. I have no idea if this is how K9 Web Protection or NetNanny works.
Squid could provide an intercept proxy for HTTP and HTTPs ports, without configuring the browsers and it also supports WCCP.