I want to write a program that can monitor all system HTTP/HTTPS protocols used to open the default browser, and block certain ones, automatically changing certain requested URL into another. The process of changing a URL is simple, but the monitoring and blocking part is quite puzzling.
e.g. When clicking on the URL 'https://example.com/asdf.htm', the request will be blocked by the program and the the Windows system will receive the command of 'http://www.example2.org/asdf.htm' instead and the latter instead of the former URL will be opened by the default browser.
I am an amateur developer and student who do not have much experience in solving such problems.
I searched the web and found someone asked a similar question years ago:
https://superuser.com/questions/554668/block-specific-http-request-from-windows
However, I didn't find any useful advice on coding in the page. Maybe I can use an antivirus program to block certain URLs or change the hosts file to block certain URLs but the URL replacement cannot be done. Certainly, changing the hosts to a certain server which redirects certain requests might work but that's too complex. I wish someone can help me solve the problem by giving a simple method on monitoring the Windows system itself. Thanks!
To summarize our conversation in the comments, in order to redirect or restrict traffic, either to sites, either to ports (protocols are actually "mapped" via ports) the main solutions usually are:
a software firewall - keep in mind that SWFW don't usually redirect, they just permit or allow traffic via ports
a hardware firewall (or advanced router, not the commercial ones, but enterprise grade) - they do what you want, but they are very expensive and not worth for a home experiment
a proxy server - this can do what you want
Other alternatives that might or might not work would include editing the hosts file, as you said, but as stated earlier I don't recommend it, it's a system file and if you forget about it, then it can be a hindrance (also keep in mind that normally you should not use a Windows user with admin rights even at home, but that is another story) and a browser extension (which Iwould guess only changes content on pages, not the way a browser works (such as changing URLs).
I think a proxy server is the best pick here. Try it and let me know.
Keep in mind I still recommend you read about networking in order to get a better idea of what you can and can't do in each setup.
Related
My MVC-5 website gets a lot of false registrations, or real and followed by experiments in escalation of privileges. I can write a request filter - but how to block web requests from one particular country? Are there publicly available list of IPs that I can block ... or what? How do people approach solving this issue?
Depending on the country you might have a little difficulty in blocking everything but there are lists out there (such as the one maintained by nirsoft). Usually it's better to block specific IP addresses where the bad behavior is originating by using software that watches for the behavior and dynamically blocks it. That way you're covered regardless of where it originates. Especially since managing IP address blacklists is a real pain. I've made use of IPTables on linux before for this and it works like a charm.
There are some websites which make me waste much time so I want to block them by writing a hook program.
Can i do it?
Can i hook the http request of the OS or the browser's messages?
Well... you could but that would be the complicated way.
By far the easiest mechanism for blocking a site from a single computer is to edit your hosts file and change the DNS name to resolve to your local host.
Start Notepad.exe as administrator
Open the hosts file in c:\windows\system32\drivers\etc
at the bottom add a line like:
127.0.0.1 www.google.com
Save the file.
If you put in the exact thing I did above then you will no longer be able to get to google from any browser on your machine.
If you don't want the easy way, there are 2 other approaches.
One is system wide at which point you need to create a proxy server and modify the system to point to that proxy. Then you can decide what to do with the traffic. This is how Fiddler works and is pretty much the only way to ensure you get almost all of the traffic.
The second is to create browser specific add-on. IE calls it a Browser Helper Object (BHO), I think Firefox just calls it a "Browser Extension", etc. You will have to build a separate one for each browser (IE/FF/Chrome) you want to cover. Google those terms to see how to get started. Beware that doing so is quite complicated and will require you to jump through a lot of hoops for security reasons.
This article is a bit old (2010) but should give you enough information to at least know what you are about to get yourself into: https://msmvps.com/blogs/vcsjones/archive/2010/05/23/writing-a-managed-internet-explorer-extension-part-1.aspx
I'm usually a great debugger when it comes to helping family members with their computer problems, I also would normally post this type of question here, but I'm hoping this community can help me get to the bottom of this.
A family member is having problems with certain websites not loading all of the resources, primarily images is what it appears. I have disabled her Symantec protection in case it was scanning or preventing stuff from loading and have also uninstalled and disabled startup applications she doesn't need.
One example of a file that is not loading on her system is:
http://static.ak.fbcdn.net/rsrc.php/v1/yp/r/kk8dc2UJYJ4.png
I'm assuming this loads for everyone else here.
Any thoughts would be much appreciated. Also she gets a similar issue in IE, Chrome, Firefox.
The first place I'd look is if there's a commercial ad-blocker installed, as I guess it can't be an add-in/extension as different browsers have their own settings.
And it may sound silly, but did you check the hosts file (system32/drivers/etc/hosts)? Is it possible static.ak.fbcdn.net is just being redirected? You might want to try opening the command prompt and just doing ping static.ak.fbcdn.net and confirming her computer's exact behavior.
In my case FB redirects me to a749.g.akamai.net (or 125.56.208.11) and everything works fine.
Minor edit: I'm a bit skeptical that's the cause, as FB serves other stuff from that domain (CSS, JS). Photos and profile pictures seem to come from a different domain. But I'd still be interested in whether the problem occurs when connecting to the resource or displaying it.
Thats probably because your DNS resolves the Akamai CDN server, used by facebook to fetch images, to an IP address that is not reachable from your network. You may want to get the IP address of facebook CDNs used by your computer at the time this happens and contact your network administrator to find the reason behind the IP blockage (may be because of firewall). Other than that, you can try changing your DNS in your system settings which might give you an IP address that works for your network.
PS: I ran into this issue a few weeks ago and have found my findings to be correct.
I'm using the Selenium Client (v 1.2.18) to do automated navigation of retail websites for which there exists no external API. My goal is to determine real-time, site-specific product availability using the "Check Availability" button that exists on a lot of these sites.
In case there's any concern, each of these checks will be initiated by a real live consumer who is actually interested in whether or not something's available at that store. There will be no superfluous requests or other internet badness.
I'm using Selenium's Grid framework so that I can run stuff in parallel and I'm keeping each of the controlled browsers open between requests. The issue I'm experiencing is that I need to perform these checks across a number of different domains, and I won't know in advance which one I will have to check next. I didn't think this would be too big an issue, but it turns out that when a Selenium browser instance gets made, it gets linked to a specific domain and I haven't been able to find any way to change what domain that is. This requires restarting a browser each time a request comes in for a domain we're not already linked to.
Oh, and the reason we're using Selenium instead something more light-weight (eg. Mechanize) is because we need something that can handle JavaScript.
Any help on this would be greatly appreciated. Thanks in advance.
I suppose you are restricted from changing domain because of same origin policy. Did you try using browser with elevated security privileges like iehta for internet explorer and chrome for firefox browsers. While using these modes of browsers, use open method in your tests and pass the URL which you want to open. This might solve your problem.
How could a (Windows) desktop application be created to monitor the amount of time spent on a particular website?
My first idea was to play with the Host file to intercept requests, log, and proxy. This feels a bit clunky; and I suspect my program would look like malware.
I feel like there is a smarter way? Any ideas?
There is a tool similar to what you are looking for called K-9 Web Protection. It is more used for parents to monitor what their kids are up to when hooked up to the internet. I have installed this for my niece's computer with good results and praises as it blocks, content filter, restrict internet times. This may be OTT for your needs but worth a shot as you can see what sites were visited.
The other, is to use a dedicated firewall monitoring solution such as IPCOP which is a Linux based distribution with a sole purpose in providing a proxy, stateful packet inspection (SPI) firewall, Intrusion Detection System (IDS).
Hope this helps,
Best regards,
Tom.
You could do this by monitoring active connections via netstat, or if you need more advanced data you can install The Windows Packet Capture Library and get any data about network use, and inside your desktop app, find network traffic that relates to 'spending time' on a website (which might just be GET requests for you, but I don't know), and record various statistics as required.
Route the traffic through a scriptable proxy and change the browser settings to point to that proxy.