I'd like to write a packet sniffer and editor for Windows. I want to able to see the contents of all packets entering and leaving my system and possibly modify them. Any language is fine but I'd like it to run fast enough that it won't burden the system.
I've read a little about WinPcap but the documentation claims that you can't use WinPcap to create a firewall because it can't drop packets. What tools will help me write this software?
Been there, done that :-) Back in 2000 my first Windows program ever was a filter hook driver.
What I did was implementing the filter hook driver and writing a userspace application that prepared a filter table on what to allow and what to disallow. When you get around your initial set of blue screens (see below for my debug tip in kernel mode) the filter mode driver is quite easy to use ... it gives each packet to a function you wrote and depending on the return code drops it or lets it pass.
Unfortunatley packets at that level are QUITE raw, fragments are not reassembled and it looks more like the "network card" end of things (but no ethernet headers anymore). So you'll have quite a bad time decoding the packets to filter with that solution.
There also is the firewall hook driver, as discussed in this codeproject article.
If you are on Vista or Server 2008 you'd better have a look at WFP (Windows Filtering Platform) instead, that seems to be the mandated API of the day for writing firewalls.
I don't know about it other than google turing it up some minutes ago when I googled for the filter hook driver.
Update: Forgot the debug tip:
Sysinternals DbgView shows kernel-mode DbgPrint output, and more important - it can also read them from the dump file your last blue screen produced. So sprinkle your code with dbgprint and if it bluescreens just load the dump into dbgview to see what happened before it died ... VERY useful. Using this I managed without having a kernel debugger.
I'm pretty sure you'd need to write a filter driver. http://en.wikipedia.org/wiki/Filter_driver I don't know much more than that :). It would definitely be a C/C++ Win32 app and you'd likely being doing some kernel side work. Start by downloading the DDK and finding some of the sample filter drivers.
If you just want to monitor what goes in and out of IIS, consider an ISAPI filter. Still C/C++ in Win32, but relatively easier than writing a device driver.
C# code to do this is here
I actually did this, several years ago. I'm hazy on the details at this point, but I had to develop a filter/pass-thru/intermediate driver using the Windows DDK. I got a lot of good information from pcausa. Here's a url which points to their product that does this: http://www.pcausa.com/pcasim/Default.htm
If you're doing this for practical reasons, and not just for fun, then you should take a look at Microsoft Network Monitor. The home page talks about the version 3.3 beta, but you can download version 3.2 from the Downloads page. There is also an SDK for NM, and the ability to write parsers for your own network protocols.
There's a question you need to ask which you don't know you need to ask; do you want to know which applications sockets belong to? or are you happy to be restricted to the IP:port quad for a connection?
If you want to know applications, you need to write a TDI filter driver, but that makes handling the receive almost impossible, since you can't block on the receive path.
If you're happy with IP:port, go in at the NDIS level, and I believe you can block on receive to your hearts content.
A word of warning; if you have no prior kernel experience, writing either of these drivers (although TDI is significantly harder) will take about two years, full time.
this:
TdiFw is a simple TDI-Based Open Source Personal Firewall for Windows NT4/2000/XP/2003
http://tdifw.sourceforge.net/
may help you
Related
Are filter drivers intended to extend system drivers?
Is this their main purpose?
Are they basically just an extra layer that sits between the driver and the user?
This seems overly simple an explanation and I am wondering if I am missing something.
Are there good ways to learn more?
The driver topic is a very advanced one.
To get an overview, you can have a closer look on the Windows Driver Kit (WDK) sides.
If you decide to get into this stuff, then you need a lot of time, frustration resistance and fanaticism.
The first thing you should do (befor you touch the WDK!) is, to start reading a good book.
If you want to develop for windows file system, read Rajeev Nagar's book "Windows NT File System Internals : A Developer's Guide". It's published in 1997, but it's something like the "bible" of NTFS.
For common driver developement you can find books like "Developing Drivers with the Windows Driver Foundation", written by Penny Orwick.
These books describes programming kernel mode software, which is done in C language. So, you should have a good base knowlege on C before you start.
Among others there are the OSR side (www.osr.com) and SysInternals on technet (http://technet.microsoft.com/de-DE/sysinternals), which are truely worth to have a closer look on.
More than the halfe time you spend on reading debug outputs and crash dumps, so it's wise to know what these things are meaning and how to get this information, but there are good books for windows debugging too.
I hope, I was able to give a short overview on the question for the ways to learn more.
In a way yes.
For example, if file system filter driver is for file encryption/compression/security, it is enhancing the file system functionality.
The filter driver does not handle talking to actual devices. They rely on lower level drivers to communicate with device. The filter drivers are add-on to the drivers to implement certain functionality. The active drivers which modify data/request are to enhance vanilla drivers while the passive filter drivers are just pass-throughs without any direct enhancements.
So I think your assumptions are correct.
Will like to hear different views though.
I'm new to SNMP. I have to develop a manager. I've been searching on how to program for a manager. I've come across WinSNMP, SNMP++ and Net-SNMP. The plus point for Net-SNMP is that it can translate the trap content from the MIB (it can parse the MIB). But would Net-SNMP be a good choice of library for developing a manager? I was wondering if it would be a good idea to use snmptrapd as a basis to develop a manager. At the moment I'm focusing on receiving traps but eventually I'll have to program for the others too (Get, Set, ...).
What is the general outline on how to program for a manager? I have been reading on SNMP for weeks but still haven't got a clear understanding of how to go about doing it.
Any help would be much appreciated. Thanks.
Which toolkit you want will greatly depend on your final-end-goals and preferred language (eg, C vs C++). Make sure you get one with SNMPv3 support, which the native windows library doesn't have (but mg-soft and snmp research's implementation does). You'll likely have to pay money, eventually, if you want SNMPv3 support on windows using the winsnmp APIs.
Net-SNMP and SNMP++ are both well respected open libraries. Net-SNMP has pretty much cornered the market in terms of deployment and comes pre-installed or easily available on every OS except windows (where it also works fine, but MS just doesn't package it for you).
If you want to receive traps, Net-SNMP's snmptrapd is certainly an easy place to start and offers some very simply ways to get started through extensible commands (see the snmptrapd.conf man page to help you get started) or via C (see the apps/snmptrapd*.c files for some example C-registrations).
To get started with Net-SNMP and programming outgoing management operations, you should start with the tutorials about writing management applications. If your management application is ever going to talk to multiple hosts at once, make sure you get a toolkit that does asynchronous support (see the second item in the above link).
There is also JDMK 5.1 API from Sun microsyatem, it is very easy to develop the SNMP Agent as well Manager.
On a multi-tier application, I need to simulate various TCP/IP errors to test some reconnection code. Does anyone know of any tools (Windows based) I can use for this purpose? Thanks.
Scapy allows you to control every aspect of the packets, and randomly modify ("fuzz") the ones you don't want to control. If you're a command-line kind of guy, it's a great tool.
Try netwox (formerly lcrzoex.) If it won't do it, it can't be done. It contains >200 tools.
On FreeBSD, the best tool, by far, is dummynet, "a tool originally designed for testing networking protocols, and since then used for a variety of applications including bandwidth management. It simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects."
On Linux, you will have to use netem. (It seems there is now a port of dummynet but I never tried it.)
More details (in French) in my article.
Clumsy is a good tool for TCP error simulation on Windows. It can simulate (copy-pasted from link above):
Lag, hold the packets for a short period of time to emulate network
lagging.
Drop, randomly discard packets.
Throttle, block traffic for a given time frame, then send them in a single batch.
Duplicate, send cloned packets right after to the original one.
Out of order, re-arrange the order of packets.
Tamper, nudge bits of packet's content.
No tools that I'm aware of, but most of TCP errors can be emulated by a custom LSP filter. This article can get you started writing one
I couldn't find a suitable title for this. I'm going to express my query with examples.
Consider following softwares:
Process explorer from sysinternals (an advanced task manager)
Resource Manager : resmon.exe (lists each and every fine detail about resource usage about each process).
For me these softwares seems like miracles. I wonder how these are even made. C'mon how a user process can know such fine details about other processes? Who tells this software, what processes are running and what all resources are utilized? Which dlls are used? etc..
Does windows operating system give these software that information? I mean though (obviously the most lower level api) WIN32API. Are there some functions,which on calling return these values
abstractly say:
GetAllRunningProcesses()
GetMemoryUsedByProcess(Process* proc)
etc..
Other similar applications are
network Packet Capture software. How does it get information about all those packets? It clearly sits just infront of the NIC card. How is it possible?
Anti-virus: It scans memory for viruses. Intercepts other processes. Acts like a sandbox for the user application space. How? How??
If its WIN32API. I swear, I'm going to master it.
I don't want to create a multi-threaded application. I want to get information about other multithreaded applications.
I don't want to create a program which communicates using sockets. I want to learn how to learn how to capture all communication packets.
I actually want to work at the lower level. But I don't know, what should I learn. Please guide me in proper direction.
This is really a pretty open-ended question. For things like a list of running processes, look up "PSAPI" or "Toolhelp32". For memory information about a particular process, you can use VirtualQuery.
Capturing network packets is normally done by installing a device driver. If you look, you should be able to find a fair amount about how to write device drivers, though don't expect to create wonders overnight, and do expect to crash your machine a few times in the process (device drivers run in kernel mode, so it's easy for a mistake to crash the machine hard).
I can't say as much with any certainty about anti-virus, because I've never tried to write one. My immediate guess would be that their primary technique is API hooking. There's probably more to it than that, but offhand I've never spent enough time looking at them to know what.
Mark Russinovich's classic, Windows Internals, is the go-to book if you want to get deep in this kind of stuff. I notice that the just-released 5th edition includes Vista. Here's a sample chapter to peek at.
If you like Process Explorer, this is the guy who wrote that, and there are lots of examples using it in the book.
Plus, at 1232 hardcover pages, you can use it to press your clothes.
I am having trouble integrating two products, one of which is mine and they appear not to be talking. So I want to make sure they are communicating correctly. I had a look around for network monitor and found TCP Spy. This works but only shows 1 side of the conversation at a time (it has to run locally) I would ideally like to see both sides at the same time - but you can't run two copies of TCP Spy.
I've hit Sourceforge, but nothing seems to jump out - I'm a Windows developer, I don't have perl installed.
I've found a couple of others which are cripple-ware and totally useless, so I was wondering what do the SO guys use for watching the TCP conversation?
BTW - the 'not-written-here' product is not a browser.
Wireshark is a really good and mature network sniffer. It's been around for years.
Deep inspection of hundreds of protocols, with more being added all the time
Live capture and offline analysis
Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2
Coloring rules can be applied to the packet list for quick, intuitive analysis
Output can be exported to XML, PostScript®, CSV, or plain text
I'm not sure if it does everything you want, but have you seen WireShark and the Microsoft Network Monitor?
Wireshark (previously Ethereal)
Wireshark is an award-winning network protocol analyzer developed by an international team of networking experts.
I use wireshark. Very good and free.
Wireshark, aka Ethereal comes with a fair amount of TCP sniffing functionality.
http://www.wireshark.org/
With respect to using Windows and lacking Perl: Why not try Strawberry Perl? It's a free Perl distribution that's run by the Perl community (specifically Adam Kennedy at the core), is easy to install, and wields the full power of CPAN out of the box.
Strange that I did not see WireShark when I visited SourceForge. The top result of the 60 returned was a bizarre german thing.
Wireshark is great.. but another option would be via PowerShell. I've used the Get-Packet script from Jeff Hicks at Sapien Technologies as a really lightweight packet sniffer. You get custom objects representing your packets and can do whatever filtering you need to via PowerShell. The other script in the pair is Analyze-Packet, which can summarize the results of a packet capture.
I tried Wireshark and Microsoft Network Monitor, but neither detected my (and the program I am trying to communicate with) transfer. If I had a day to sit and configure it I probably could get it working but I just wanted the bytes sent and, more specifically, bytes received.
In the end I found HHD Software's Accurate Network Monitor software which did what I wanted it to, even if it was slight clunky.
Take a look at Tcpdump It is not a full fledged GUI network analyzer (not at all) but it is usable in scripts. Since I am more a Linux person, I use it with Bash and Python, but you should be able to call it from powershell.