Why there is no GUI kernel debugger like SoftICE, which allows to debug kernel driver on a local machine nowadays? Why remote machine is obligatory for driver debugging in Windows 7 and higher?
An in-system kernel-mode debugger is an extremely complicated software because it must take care of many low-level kernel resources and operations. If kernel internals are changed in the next OS version, the debugger must be updated accordingly. Debugger developers must work together with kernel developers and have access to kernel source code. All that makes in-system debugger development complex and expensive.
And any kernel-mode debugging on the development system is a dangerous and inconvenient process. Even if no FS damages and/or other data losses occur due to a BSOD, booting a development system, starting all required applications to re-create convenient development environment is much longer process than rebooting a dumb target machine (hardware or virtual).
When hardware computers were expensive, there was no efficient remote debugging interface and there were no efficient virtual machine solutions, SoftICE was an acceptable tool. But in the last 15 years, remote kernel debugging in Windows had been greatly improved, so using WinDbg is much more convenient that using SoftICE, even though WinDbg has many flaws and bugs.
Related
In the hypothetical world where it is desirable to disable 64 bit Windows Kernel Patch Protection (which is disabled when running in debug mode), are there any downsides or implications to running a system permanently in debug mode?
Specifically, is performance meaningfully impacted by this? Or are there any specific limitations that users of a debug system may encounter?
What behaviours, apart from KPP, does the kernel or system as a whole alter if booted with Kernel Mode Debugging enabled? Is the answer different on Windows 7 vs Windows 8/8.1 vs Windows 10?
The machine can freeze. Some user mode failures get bumped up to kernel breaks.
The DVD software stops working (DRM protection)
Boot time is impacted. (Waiting to attach debugger)
Don't leave it on if you don't need it
Not seen a difference with different os's
I'm building an IOKit CFPlugin driver for OS X. I'll be working with network data coming in that will be translated to MIDI data. No hardware is involved other than the built-in Airport. I have experience with drivers on Windows machines and firmware but this is my first dip into doing it on the Mac. So far things are going pretty well, but the Apple documentation sez: "For safety reasons, you should not load your driver on your development machine."
I only have one Mac. I really don't want two Macs- sorry, Apple. Should I take this warning seriously? Are there things I need to know?
Thanks, Tom Jeffries
You could also consider running OS X inside a VM as your testbed. It would surely be much more convenient that having a separate boot volume.
The warning is rather poorly worded; what you should consider doing is using a separate boot volume (partition) for trying out your driver, since it's possible to arbitrarily hose your system with your driver.
If you're doing kernel development on any OS that isn't isolated from your main system (via a VM, alternate boot disk, etc.), you're crazy!
What may be a bigger issue is that you can't do any kernel debugging, because the only option for that is to use GDB on a remote OS X system. For this, you may want to consider running OS X in virtualization.
you DEFINITELY want to have some way to recover a fubar kext installation: a bootable external drive or something you can quickly restore from-- this is the main reason for Apple's warning against running in-development-kernel-extensions on your production machine.
Nicholas is right that in order to debug using gdb (the only way in kernel space) you do need two machines. I've never tried using a VM as Coxy suggests: but I guess it's feasible (assuming that you run your kext on the virtual machine and use the real host machine to run gdb).
My preferred method for tracing and debugging in the kernel is kprintf() routed to firewire (aka firewire kprintf (man fwkpfv) ). for this you do need two machines with firewire ports.
finally, being an old computer musician myself, I wonder why you want to program a MIDI synthesizer (or transformer) on the network stack level. my guess is that you would have a much more gratifying experience working in userland (where you can use floating point math...)
if you need some hints or tips, feel free to get in touch...
|K<
from the ADC Kernel Programming Guide
Kernel programming is a black art that
should be avoided if at all possible.
Fortunately, kernel programming is
usually unnecessary. You can write
most software entirely in user space.
Even most device drivers (FireWire and
USB, for example) can be written as
applications, rather than as kernel
code. A few low-level drivers must be
resident in the kernel's address
space, however, and this document
might be marginally useful if you are
writing drivers that fall into this
category.
I'd like to ask anybody who has built a virtualized VS2010 environment in VirtualBox or VMware, which one was able to work out of the box without too much tweaking? Or both need workarounds to get stuff working?
Both are fine as long as you install the respective tools and drivers provided for the guest OS
If you're using VMWare Workstation, you can leverage even more out of the environment by installing Visual Studio on the Host PC, and using the Guest VM for debugging, if your application crashes you can actually rewind back to before the crash and step through your code with the same heap and stack before it crashed!
Basically, I suggest going with VMWare Workstation. It's pretty cheap (assuming you get paid to program) and has many, many awesome features that you'll come to love. If you're a hobbyist/student programmer however, you'll likely find VirtualBox to be a little more functional than the free VMWare Player.
As far as performance goes, Intel and AMD both have shipped chips with hardware virtualization since 2005/2006 respectively. This is called VT-x or AMD-V, and often has to be enabled in the bios on older machines.
Basically this means that your BIOS handles Memory and I/O virtualization on this chip, while specialist drivers (e.g. VMWare Tools) are installed to improve graphics and mouse performance - effectively this means the resulting VM has near native performance with minimal overhead.
Hope that helps!
You can work with a VS2010/Windows virtualized environment with no problems.
I've worked with such combination and I had no problems. Both VMWare and VirtualBox are stable so far since years and Windows OS virtualization works properly.
Obviously, you can have performance loss, because a virtualized OS has more bottle necked access to resources than a host one, but current CPUs from Intel and AMD have advanced virtualization instruction extensions which accelerates virtualization operations.
So... Just go ahead!
I don't know your requirement but there is also a great alternative using Win 7.
You can create a vhd file and boot on the vhd file.
A few steps more, you can create a base vhd file with everything you need, mark it as readonly and create as many differential disk as you want.
The drawback of this method are these ones :
it's a bit tricky to create the base and diff disk, because you have to do it in the setup console of windows setup (but google can help you)
there is a small performance impact on the disk I/O (but lower than the visualization environment)
you can run only one system at a time. In fact, nothing disallow you to install a virtualization software
you can't have your "host" and it's potential tools (corporate email, etc.)
but at least, the performance will be greatly better than a virtualization software.
Is there a debugger which works from a virtual machine's host?
That is, instead of using interrupts inside the machine, I expect this debugger to recognize the virtual machine's OS routines, memory locations etc, and to recognize when the OS is launching a certian EXE. Then I want to be able to set hardware-like breakpoints per process through the host computer. I'll clarify. The virtualized computer and OS would never know that the breakpoint was set or occurred. All debug handling would be done by the host computer which emulates a virtual computer.
This would enable much stronger breakpoint mechanism, for example "break when a certain data is read from the CDROM drive", or "break when a certain file on the disk contains the following byte sequence".
This approach will also, for example, eliminate anti-debugger techniques which are suppose to alter the executable's behavior when running under a debugger. (OTOH it opens up a new area of anti-virtualization techniques which relies upon slight differences between emulated computer and real hardware).
Is there such a product? Does it look like a good idea?
VMware has offered VM debugging plugins for Visual Studio and Eclipse for some time now. It is even possible to record a VM run (which logs input from all devices, allowing to replay the execution of the VM precisely as when it was recorded), then step through the recording with a debugger.
Recent versions of IDA Pro include a debugger interface which, among other setups, can inspect a BOCHS virtual machine.
I'm currently setting up vmware Server 2.0 for kernel debugging with gdb ( see this setup guide ) and someone asked me why not use kvm?
So I ask: kvm vs. vmware for kernel debugging / USB driver development
what are the pros and cons of each?
Driver development? are you working on a driver for a particular piece of hardware? if so, then you probably won't be able to use virtualization, because the virtualized instance won't have access to the new hardware.
For this you will need two machines, one running a remote debugger on the other.
*Edit: * Apparently you're developing a driver for a USB Device? this is one area in particular that a VM actually Can help. These days most VM's have the ability to delegate specific USB devices to a guest OS.
That said, this situation doesn't really offer any benefits over the remote debugger option, because you still need a way to inspect the state of the running or crashed OS, and VM's offer very little assistance in this regard. You might be able to replay saved states from just before a crash.
You might be able to get a bit of traction using UML, which would allow you to do local debugging as on a regular user process, which is a little bit less trouble.
Instead of answering the direct question I'll add another option... Depending on if the kernel in question is a Linux kernel, and what part(s) of it you are working on, you might find that UserModeLinux (included in the 2.6.x source, and available as patch sets for 2.4 and 2.2) may trump both of those options.
As it runs the kernel as a userland process under the host kernel it is easier to attach common debugging tools to. I believe it is very commonly used in the early stages of updates/additions to file-system related code. If you are developing/debugging modules that interact directly with hardware it may be much less use to you though.
Reference links: home,
other
I recently started building GNU Mach/HURD and found the combination of QEmu/KVM to work really quite well.. for the following reasons:
QEmu presents quite a clean environment
Networking has alot of options
I can easily mount the filesystem using a raw device file / loopback
Bottom line is, for kernel work I just want the minimum of functionality to boot and see the result. VMWare is much more for usable virtualization rather than down-and-dirty.
There is however no comparison to booting on a real machine with real hardware. The VM environment can seem like a safety blanket somtimes ... because even my toaster would know what a Realtek RTL8139C was.
If it is a "real hardware" device, of course, vmware will not emulate it, so you won't be able to debug the driver under it (nor will any other virtualisation software, unless you extend one to do so).
Device driver debugging can be done to some extent with a real hardware machine with a normal kernel - although there are obviously things you can't do - like set breakpoints.
It is still possible to attach a debugger to the kernel and inspect stuff. Moreover, traditional printf() debugging is quite possible (printk, anyone), and there are various features in the kernel which make debugging easier. It's possible to build the kernel with various debug options to try to detect pointer problems, memory leaks etc.
By default, the kernel even gives a nice-ish stack trace on the log when it encounters an OOPS or BUG condition (obviously this does not necessarily get written anywhere if the system hangs or crashes). Of course a pointer-out-of-range condition happening inside an interrupt is a recipe for disaster, but you could still get a stack trace on the screen immediately before the panic :)