On a multi-tier application, I need to simulate various TCP/IP errors to test some reconnection code. Does anyone know of any tools (Windows based) I can use for this purpose? Thanks.
Scapy allows you to control every aspect of the packets, and randomly modify ("fuzz") the ones you don't want to control. If you're a command-line kind of guy, it's a great tool.
Try netwox (formerly lcrzoex.) If it won't do it, it can't be done. It contains >200 tools.
On FreeBSD, the best tool, by far, is dummynet, "a tool originally designed for testing networking protocols, and since then used for a variety of applications including bandwidth management. It simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects."
On Linux, you will have to use netem. (It seems there is now a port of dummynet but I never tried it.)
More details (in French) in my article.
Clumsy is a good tool for TCP error simulation on Windows. It can simulate (copy-pasted from link above):
Lag, hold the packets for a short period of time to emulate network
lagging.
Drop, randomly discard packets.
Throttle, block traffic for a given time frame, then send them in a single batch.
Duplicate, send cloned packets right after to the original one.
Out of order, re-arrange the order of packets.
Tamper, nudge bits of packet's content.
No tools that I'm aware of, but most of TCP errors can be emulated by a custom LSP filter. This article can get you started writing one
Related
I want to set system wide proxy settings on my windows machine. I know about the settings from Internet Explorer but dont want to do it that way. Is there a way to set up a proxy which will be used by all the applications on my machine(especially firefox, I dont want to have to set Use System proxy Settings in the Firefox options menu)?
In windows, that is the preferred way to set up the proxy settings.
But you can have a look at this for command line options
https://superuser.com/questions/419696/in-windows-7-how-to-change-proxy-settings-from-command-line
How can this be achieved theoretically
I am going to provide a somewhat unusual answer, because I've noticed that this particular 'way' of solving this problem has (for some reason) not crossed people's minds so far.
If you want to really make all apps without exception send internet traffic through your proxy, you are going to have to use a special technology known as TUN/TAP devices.
In short, these are special drivers, which when installed appear to a system as a network adapter (just like your local Ethernet or Wireless card), but they are in fact built in such a way so as to be easy to control from a software level.
Basically, when you install such a driver on the system, the system now regards that device as a fully functional Network Adapter. Therefore, if you now set this network adapter as the default gateway, all apps (without knowing it or being able to prevent it) will automatically pass through it, the same way as all apps pass through a generic Wireless Adapter / Ethernet.
Practical ways of achieving this / How can I use this with proxies?
Now that you have a basic idea of what redirecting system traffic through a TAP/TUN device means, there are a couple of ways of doing this.
Before I start, I really recommend that even if you stray from the suggested resources here, you stick to using OpenVPN's open source TAP device, since it has been extensively tested and confirmed to work on many systems, and is very widely used now (Some basics are available at https://openvpn.net/tuntap, and I trust you should find it embedded in any latest version of OpenVPN, the only files you need are the compiled drivers (.inf), you don't need to have the entirety of OpenVPN installed to use them).
The project that instantly comes to mind when thinking of using SOCKS proxies as the endpoint of a TAP device is badvpn/tun2socks. The project basically does exactly what is outlined here, so I definitely recommend you read the source code, or use it as a standalone utility (If you need some help with usage, I suggest you check out this wiki page.
What if any are the drawbacks of using this approach?
First of all, speaking of compatibility, performance and bugs, there are no drawbacks of using this approach at all, it is if anything more reliable and easier to use then even the ways of doing this provided by the system.
The only two drawbacks I can see at this point would be:
You have to be careful to make sure whatever proxy/intermediate host you are using, it is capable of handling at least the majority of system traffic, because if an app sends incompatible internet traffic, it will still be redirected through the TAP device (that is it's purpose).
The code base may be larger than in other cases
An alternative, 'unclean' way of doing this for Firefox in particular
If you are interested in only setting this proxy for Firefox, there are a couple of unclean ways of doing this: For instance, via the command line. It is, however (in my opinion), a very cheap and dirty way of achieving this, as this does not provide any compatibility whatsoever (basically a hack).
Conclusion
While implementing this may take a while, and the code base may be large:
It is not really possible, through any other means to achieve the same effect as VPNs achieve when they tunnel the entirety of your machine's traffic through the OpenVPN server.
If you want to achieve this kind of behavior, it is recommended that you use the approach outlined above, as it is a lot cleaner then 'alternative' methods of doing so (e.g. Socksifying traffic by intercepting it at a software level)
How do you mimic low bandwidth for testing HTML rendering on slow computers? I'm using Safari 4 on a Mac.
Found the SpeedLimit preference tool mentioned in another SO answer. Works for me and is very unobtrusive to use (no application, no command line, no Java).
You're probably looking for traffic shaping, in MacOS X it's included in the kernel. Take a look at http://www.macgeekery.com/hacks/software/traffic_shaping_in_mac_os_x
If you have an Apple Developer account, look for the "Hardware IO Tools for Xcode" download. There is a Network Link Conditioner pref-pane included just for that with some good presets for
3G (Average/Good/Lossy)
Edge (Average/Good/Lossy)
WiFi (Average/Good/Lossy)
Cable
DSL
Also have a look at this answer
I'm a big fan of using the Charles Web Debugging Proxy. It allows you to throttle your own bandwidth to simulate different connection speeds (and a whole mess of other stuff, too). Highly recommended.
You could set up a local webserver (ie Apache) and use the traffic shaping capabilities thereof.
An article on Apache throttling
I guess most popular webservers support this.
edit:Typos
There is a program by Intrarts called "throttled", and the command-line version seems to be free (price):
throttled is a bandwidth shaping
application for Mac OS X and FreeBSD
which allows you to cap your upstream
bandwidth, prioritize ACK packets, and
keep your download speeds high even
when your server is sending out at
full speed.
...which seems to be a little tangential to your needs, but might be able to be configured to do what you want.
I wrote a post on using waterroof to limit bandwidth a while ago.
It uses ipfw under the covers, but has a nice UI for this kind of thing.
One option is Trickle for Mac.
Alternatively, if you don't have to use Safari, you could use Firefox with the Firefox Throttle plugin.
I couldn't find a suitable title for this. I'm going to express my query with examples.
Consider following softwares:
Process explorer from sysinternals (an advanced task manager)
Resource Manager : resmon.exe (lists each and every fine detail about resource usage about each process).
For me these softwares seems like miracles. I wonder how these are even made. C'mon how a user process can know such fine details about other processes? Who tells this software, what processes are running and what all resources are utilized? Which dlls are used? etc..
Does windows operating system give these software that information? I mean though (obviously the most lower level api) WIN32API. Are there some functions,which on calling return these values
abstractly say:
GetAllRunningProcesses()
GetMemoryUsedByProcess(Process* proc)
etc..
Other similar applications are
network Packet Capture software. How does it get information about all those packets? It clearly sits just infront of the NIC card. How is it possible?
Anti-virus: It scans memory for viruses. Intercepts other processes. Acts like a sandbox for the user application space. How? How??
If its WIN32API. I swear, I'm going to master it.
I don't want to create a multi-threaded application. I want to get information about other multithreaded applications.
I don't want to create a program which communicates using sockets. I want to learn how to learn how to capture all communication packets.
I actually want to work at the lower level. But I don't know, what should I learn. Please guide me in proper direction.
This is really a pretty open-ended question. For things like a list of running processes, look up "PSAPI" or "Toolhelp32". For memory information about a particular process, you can use VirtualQuery.
Capturing network packets is normally done by installing a device driver. If you look, you should be able to find a fair amount about how to write device drivers, though don't expect to create wonders overnight, and do expect to crash your machine a few times in the process (device drivers run in kernel mode, so it's easy for a mistake to crash the machine hard).
I can't say as much with any certainty about anti-virus, because I've never tried to write one. My immediate guess would be that their primary technique is API hooking. There's probably more to it than that, but offhand I've never spent enough time looking at them to know what.
Mark Russinovich's classic, Windows Internals, is the go-to book if you want to get deep in this kind of stuff. I notice that the just-released 5th edition includes Vista. Here's a sample chapter to peek at.
If you like Process Explorer, this is the guy who wrote that, and there are lots of examples using it in the book.
Plus, at 1232 hardcover pages, you can use it to press your clothes.
I'd like to write a packet sniffer and editor for Windows. I want to able to see the contents of all packets entering and leaving my system and possibly modify them. Any language is fine but I'd like it to run fast enough that it won't burden the system.
I've read a little about WinPcap but the documentation claims that you can't use WinPcap to create a firewall because it can't drop packets. What tools will help me write this software?
Been there, done that :-) Back in 2000 my first Windows program ever was a filter hook driver.
What I did was implementing the filter hook driver and writing a userspace application that prepared a filter table on what to allow and what to disallow. When you get around your initial set of blue screens (see below for my debug tip in kernel mode) the filter mode driver is quite easy to use ... it gives each packet to a function you wrote and depending on the return code drops it or lets it pass.
Unfortunatley packets at that level are QUITE raw, fragments are not reassembled and it looks more like the "network card" end of things (but no ethernet headers anymore). So you'll have quite a bad time decoding the packets to filter with that solution.
There also is the firewall hook driver, as discussed in this codeproject article.
If you are on Vista or Server 2008 you'd better have a look at WFP (Windows Filtering Platform) instead, that seems to be the mandated API of the day for writing firewalls.
I don't know about it other than google turing it up some minutes ago when I googled for the filter hook driver.
Update: Forgot the debug tip:
Sysinternals DbgView shows kernel-mode DbgPrint output, and more important - it can also read them from the dump file your last blue screen produced. So sprinkle your code with dbgprint and if it bluescreens just load the dump into dbgview to see what happened before it died ... VERY useful. Using this I managed without having a kernel debugger.
I'm pretty sure you'd need to write a filter driver. http://en.wikipedia.org/wiki/Filter_driver I don't know much more than that :). It would definitely be a C/C++ Win32 app and you'd likely being doing some kernel side work. Start by downloading the DDK and finding some of the sample filter drivers.
If you just want to monitor what goes in and out of IIS, consider an ISAPI filter. Still C/C++ in Win32, but relatively easier than writing a device driver.
C# code to do this is here
I actually did this, several years ago. I'm hazy on the details at this point, but I had to develop a filter/pass-thru/intermediate driver using the Windows DDK. I got a lot of good information from pcausa. Here's a url which points to their product that does this: http://www.pcausa.com/pcasim/Default.htm
If you're doing this for practical reasons, and not just for fun, then you should take a look at Microsoft Network Monitor. The home page talks about the version 3.3 beta, but you can download version 3.2 from the Downloads page. There is also an SDK for NM, and the ability to write parsers for your own network protocols.
There's a question you need to ask which you don't know you need to ask; do you want to know which applications sockets belong to? or are you happy to be restricted to the IP:port quad for a connection?
If you want to know applications, you need to write a TDI filter driver, but that makes handling the receive almost impossible, since you can't block on the receive path.
If you're happy with IP:port, go in at the NDIS level, and I believe you can block on receive to your hearts content.
A word of warning; if you have no prior kernel experience, writing either of these drivers (although TDI is significantly harder) will take about two years, full time.
this:
TdiFw is a simple TDI-Based Open Source Personal Firewall for Windows NT4/2000/XP/2003
http://tdifw.sourceforge.net/
may help you
I am having trouble integrating two products, one of which is mine and they appear not to be talking. So I want to make sure they are communicating correctly. I had a look around for network monitor and found TCP Spy. This works but only shows 1 side of the conversation at a time (it has to run locally) I would ideally like to see both sides at the same time - but you can't run two copies of TCP Spy.
I've hit Sourceforge, but nothing seems to jump out - I'm a Windows developer, I don't have perl installed.
I've found a couple of others which are cripple-ware and totally useless, so I was wondering what do the SO guys use for watching the TCP conversation?
BTW - the 'not-written-here' product is not a browser.
Wireshark is a really good and mature network sniffer. It's been around for years.
Deep inspection of hundreds of protocols, with more being added all the time
Live capture and offline analysis
Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2
Coloring rules can be applied to the packet list for quick, intuitive analysis
Output can be exported to XML, PostScript®, CSV, or plain text
I'm not sure if it does everything you want, but have you seen WireShark and the Microsoft Network Monitor?
Wireshark (previously Ethereal)
Wireshark is an award-winning network protocol analyzer developed by an international team of networking experts.
I use wireshark. Very good and free.
Wireshark, aka Ethereal comes with a fair amount of TCP sniffing functionality.
http://www.wireshark.org/
With respect to using Windows and lacking Perl: Why not try Strawberry Perl? It's a free Perl distribution that's run by the Perl community (specifically Adam Kennedy at the core), is easy to install, and wields the full power of CPAN out of the box.
Strange that I did not see WireShark when I visited SourceForge. The top result of the 60 returned was a bizarre german thing.
Wireshark is great.. but another option would be via PowerShell. I've used the Get-Packet script from Jeff Hicks at Sapien Technologies as a really lightweight packet sniffer. You get custom objects representing your packets and can do whatever filtering you need to via PowerShell. The other script in the pair is Analyze-Packet, which can summarize the results of a packet capture.
I tried Wireshark and Microsoft Network Monitor, but neither detected my (and the program I am trying to communicate with) transfer. If I had a day to sit and configure it I probably could get it working but I just wanted the bytes sent and, more specifically, bytes received.
In the end I found HHD Software's Accurate Network Monitor software which did what I wanted it to, even if it was slight clunky.
Take a look at Tcpdump It is not a full fledged GUI network analyzer (not at all) but it is usable in scripts. Since I am more a Linux person, I use it with Bash and Python, but you should be able to call it from powershell.