There are some websites which make me waste much time so I want to block them by writing a hook program.
Can i do it?
Can i hook the http request of the OS or the browser's messages?
Well... you could but that would be the complicated way.
By far the easiest mechanism for blocking a site from a single computer is to edit your hosts file and change the DNS name to resolve to your local host.
Start Notepad.exe as administrator
Open the hosts file in c:\windows\system32\drivers\etc
at the bottom add a line like:
127.0.0.1 www.google.com
Save the file.
If you put in the exact thing I did above then you will no longer be able to get to google from any browser on your machine.
If you don't want the easy way, there are 2 other approaches.
One is system wide at which point you need to create a proxy server and modify the system to point to that proxy. Then you can decide what to do with the traffic. This is how Fiddler works and is pretty much the only way to ensure you get almost all of the traffic.
The second is to create browser specific add-on. IE calls it a Browser Helper Object (BHO), I think Firefox just calls it a "Browser Extension", etc. You will have to build a separate one for each browser (IE/FF/Chrome) you want to cover. Google those terms to see how to get started. Beware that doing so is quite complicated and will require you to jump through a lot of hoops for security reasons.
This article is a bit old (2010) but should give you enough information to at least know what you are about to get yourself into: https://msmvps.com/blogs/vcsjones/archive/2010/05/23/writing-a-managed-internet-explorer-extension-part-1.aspx
Related
I want to write a program that can monitor all system HTTP/HTTPS protocols used to open the default browser, and block certain ones, automatically changing certain requested URL into another. The process of changing a URL is simple, but the monitoring and blocking part is quite puzzling.
e.g. When clicking on the URL 'https://example.com/asdf.htm', the request will be blocked by the program and the the Windows system will receive the command of 'http://www.example2.org/asdf.htm' instead and the latter instead of the former URL will be opened by the default browser.
I am an amateur developer and student who do not have much experience in solving such problems.
I searched the web and found someone asked a similar question years ago:
https://superuser.com/questions/554668/block-specific-http-request-from-windows
However, I didn't find any useful advice on coding in the page. Maybe I can use an antivirus program to block certain URLs or change the hosts file to block certain URLs but the URL replacement cannot be done. Certainly, changing the hosts to a certain server which redirects certain requests might work but that's too complex. I wish someone can help me solve the problem by giving a simple method on monitoring the Windows system itself. Thanks!
To summarize our conversation in the comments, in order to redirect or restrict traffic, either to sites, either to ports (protocols are actually "mapped" via ports) the main solutions usually are:
a software firewall - keep in mind that SWFW don't usually redirect, they just permit or allow traffic via ports
a hardware firewall (or advanced router, not the commercial ones, but enterprise grade) - they do what you want, but they are very expensive and not worth for a home experiment
a proxy server - this can do what you want
Other alternatives that might or might not work would include editing the hosts file, as you said, but as stated earlier I don't recommend it, it's a system file and if you forget about it, then it can be a hindrance (also keep in mind that normally you should not use a Windows user with admin rights even at home, but that is another story) and a browser extension (which Iwould guess only changes content on pages, not the way a browser works (such as changing URLs).
I think a proxy server is the best pick here. Try it and let me know.
Keep in mind I still recommend you read about networking in order to get a better idea of what you can and can't do in each setup.
I'm using the Selenium Client (v 1.2.18) to do automated navigation of retail websites for which there exists no external API. My goal is to determine real-time, site-specific product availability using the "Check Availability" button that exists on a lot of these sites.
In case there's any concern, each of these checks will be initiated by a real live consumer who is actually interested in whether or not something's available at that store. There will be no superfluous requests or other internet badness.
I'm using Selenium's Grid framework so that I can run stuff in parallel and I'm keeping each of the controlled browsers open between requests. The issue I'm experiencing is that I need to perform these checks across a number of different domains, and I won't know in advance which one I will have to check next. I didn't think this would be too big an issue, but it turns out that when a Selenium browser instance gets made, it gets linked to a specific domain and I haven't been able to find any way to change what domain that is. This requires restarting a browser each time a request comes in for a domain we're not already linked to.
Oh, and the reason we're using Selenium instead something more light-weight (eg. Mechanize) is because we need something that can handle JavaScript.
Any help on this would be greatly appreciated. Thanks in advance.
I suppose you are restricted from changing domain because of same origin policy. Did you try using browser with elevated security privileges like iehta for internet explorer and chrome for firefox browsers. While using these modes of browsers, use open method in your tests and pass the URL which you want to open. This might solve your problem.
How could a (Windows) desktop application be created to monitor the amount of time spent on a particular website?
My first idea was to play with the Host file to intercept requests, log, and proxy. This feels a bit clunky; and I suspect my program would look like malware.
I feel like there is a smarter way? Any ideas?
There is a tool similar to what you are looking for called K-9 Web Protection. It is more used for parents to monitor what their kids are up to when hooked up to the internet. I have installed this for my niece's computer with good results and praises as it blocks, content filter, restrict internet times. This may be OTT for your needs but worth a shot as you can see what sites were visited.
The other, is to use a dedicated firewall monitoring solution such as IPCOP which is a Linux based distribution with a sole purpose in providing a proxy, stateful packet inspection (SPI) firewall, Intrusion Detection System (IDS).
Hope this helps,
Best regards,
Tom.
You could do this by monitoring active connections via netstat, or if you need more advanced data you can install The Windows Packet Capture Library and get any data about network use, and inside your desktop app, find network traffic that relates to 'spending time' on a website (which might just be GET requests for you, but I don't know), and record various statistics as required.
Route the traffic through a scriptable proxy and change the browser settings to point to that proxy.
Is it possible to somehow mimic the slower download speeds of my website, as if it is hosted on a web server, but from my localhost?
It's hard to test things like Ajax loading icons when the content loads so quick from your local machine so I think this could help me out a lot?
I was wondering if there was a tool that you could use for this or if you'd need to use javascript to add setTimeout or setInterval code?
If you are using Windows, then download the Fiddler tool Fiddler.
This will setup a HTTP proxy that you can use for testing HTTP headers and view all the HTTP traffic within the PC and browser. You can then use the Rules > Performance > Simulate Modem Speeds option to slow the browser's HTTP connection.
There is also a Fiddler Switch plug-in for Firefox available to toggle the Fiddler proxy.
First of all, please never mangle your code with if ( DEBUG ) workSlowly(); statements. You will get bitten by this practice, and it will hurt.
Second of all, use one of the traffic throttling solutions which have already been posted here. I'm going to add a Java-based solution: Sloppy. It's dead easy to set up and runs everywhere where Java runs.
This question may help you out.
Personally, I add something like this at the top of my library file or whatever file always gets included:
if(DEBUG) {
sleep(2);
}
The above is PHP, but most languages are going to have something similar.
If you're on a *nix platform, you could alter the 'nice' level to basically make your process an afterthought behind most others until you achieve a level of 'lag' that is satisfactory to your needs.
One option is (if You are on linux, but other unix system have similar tools) Traffic shaping (google for 'HTB', 'qdics' and 'tc' command)
Second option will be apache_mod_cband module, this is also probably easer to get going.
You could try Charles. It's cross platform - I use it and it works really well.
http://www.charlesproxy.com/
On windows you can use Proxomitron or you can install fiddler (for ie). Both let you set the download speed.
If you use Fiddler 2, you can install this Add-On: Fiddler - Connection Simulator.
It gives you a nice UI to setup Bandwith and different simulation kinds.
I would like to setup a http proxy on my work machine (no admin rights, WinXP) to only allow access to a whitelist of URLs. What would be the easiest solution? I prefer open-source software if possible.
Squid seems to be the de facto proxy. This link describes how to set it up on a windows box: http://www.ausgamers.com/features/read/2638752
Why not use the Content Advisor in IE? You can provide a list of approved sites, anything else is blocked. Or do you want pass-through functionality like a true proxy?
Content advisor will ask for authorization every time a javascript function is called. At least that's my experience right now, and that's how I landed here, after hours of googling.
You are right, however, if the sites in the whitelist don't use javascript intensively and I would suggest that that option be tried first because (and I'm an IT person), it's FAAAAAAAAR easier to set up Content Advisor than a proxy server. Google "noaccess.rat" and you'll come accross articles that tell you how to set up IE using a white-list approach.
Having said this, however, you must be fully aware that Content Advisor can be easily disabled, even without knowing the password. One of my users did it in no time. You can find this in google as well.
Alex