I'm currently learning about the different modes the Windows operating system runs in (kernel mode vs. user mode), device drivers, their respective advantages and disadvantages and computer security in general.
I would like to create a practical example of what a faulty device driver that runs in kernel mode can do to the system, by for example corrupting memory used for critical OS-processes.
How can I execute my code in kernel mode instead of user mode, directly?
Do I have to write a dummy device driver and install it to do this?
Where can I read more about kernel and user mode in Windows?
I know the dangers of this and will do all of the experiments on a virtual machine running Windows XP only
The "Windows Internals" book is rather shallow on the topic at question.
First I should note that any program also runs in kernel mode (KM). This is due to the fact that - not unlike in unixoid systems - for system calls the calling thread transitions into KM where the kernel itself or one of the drivers services the request and then returns to user mode (UM).
A first step to get started would be to download the latest Windows Driver Kit (WDK) and start reading the documentation. If you want a more digestive book, go for one of these:
Windows NT Device Driver Development - though an old title, many of the basics still apply.
Programming the Windows Driver Model (by Oney) - WDM programming in particular, also covers basics, has some errors (as most books).
Undocumented Windows 2000 Secrets (by Schreiber) - contains plenty of information about all kinds of internals at a more technical level than the book mentioned before.
Undocumented Windows NT - contains a more generic part about internals on a technical level followed by a reference of some native API functions.
Windows NT/2000 Native API - the classic, but it's more of a reference. Nevertheless there are several gems (and examples) in it.
Since you want to use Windows XP, many of the techniques described over at rootkit.com (even from some years ago) should work. They also got plenty of samples.
And as you notice by the name of the referenced website, you are in fact in what I'd call a gray area with that question ;)
It's a simple answer, and as you suspect, you do need to write a device driver in order to run in kernel mode. I'm afraid I don't know of a particularly good reference for kernel mode programming but a quick websearch reveals:
en.wikibooks.org/wiki/Windows_Programming/User_Mode_vs_Kernel_Mode
http://www.netomatix.com/Development/Kernelmode.aspx
http://technet.microsoft.com/en-us/library/cc750820.aspx
http://msdn.microsoft.com/en-us/library/ff553208(VS.85).aspx
You will need a good understanding of Windows Internals:
http://technet.microsoft.com/en-us/sysinternals
and yes they have a book: Windows Internals
http://technet.microsoft.com/en-us/sysinternals/bb963901
http://www.amazon.com/Windows%C2%AE-Internals-Including-Windows-PRO-Developer/dp/0735625301
Basically your questions are all answered in this book (and it even comes with samples and hands-on labs).
Related
I'm building an IOKit CFPlugin driver for OS X. I'll be working with network data coming in that will be translated to MIDI data. No hardware is involved other than the built-in Airport. I have experience with drivers on Windows machines and firmware but this is my first dip into doing it on the Mac. So far things are going pretty well, but the Apple documentation sez: "For safety reasons, you should not load your driver on your development machine."
I only have one Mac. I really don't want two Macs- sorry, Apple. Should I take this warning seriously? Are there things I need to know?
Thanks, Tom Jeffries
You could also consider running OS X inside a VM as your testbed. It would surely be much more convenient that having a separate boot volume.
The warning is rather poorly worded; what you should consider doing is using a separate boot volume (partition) for trying out your driver, since it's possible to arbitrarily hose your system with your driver.
If you're doing kernel development on any OS that isn't isolated from your main system (via a VM, alternate boot disk, etc.), you're crazy!
What may be a bigger issue is that you can't do any kernel debugging, because the only option for that is to use GDB on a remote OS X system. For this, you may want to consider running OS X in virtualization.
you DEFINITELY want to have some way to recover a fubar kext installation: a bootable external drive or something you can quickly restore from-- this is the main reason for Apple's warning against running in-development-kernel-extensions on your production machine.
Nicholas is right that in order to debug using gdb (the only way in kernel space) you do need two machines. I've never tried using a VM as Coxy suggests: but I guess it's feasible (assuming that you run your kext on the virtual machine and use the real host machine to run gdb).
My preferred method for tracing and debugging in the kernel is kprintf() routed to firewire (aka firewire kprintf (man fwkpfv) ). for this you do need two machines with firewire ports.
finally, being an old computer musician myself, I wonder why you want to program a MIDI synthesizer (or transformer) on the network stack level. my guess is that you would have a much more gratifying experience working in userland (where you can use floating point math...)
if you need some hints or tips, feel free to get in touch...
|K<
from the ADC Kernel Programming Guide
Kernel programming is a black art that
should be avoided if at all possible.
Fortunately, kernel programming is
usually unnecessary. You can write
most software entirely in user space.
Even most device drivers (FireWire and
USB, for example) can be written as
applications, rather than as kernel
code. A few low-level drivers must be
resident in the kernel's address
space, however, and this document
might be marginally useful if you are
writing drivers that fall into this
category.
I am aware of windows kernel but new to linux kernel. I just need to know how its done in linux, i.e. the program development.
You can check there (free-electrons.com), it's a good informations source for kernel developement. (specialized in embedded linux, but most of the docs are available for standard development)
You have also the classical Linux Devices Drivers, which is very complete and detailled.
And last but not least, the Linux kernel documentation.
Linux does not have a stable kernel API. This is by design, so you should generally avoid writing kernel code if you can; it is unlikely to remain source-compatible indefinitely, and will definitely NOT be binary-compatible, even between minor releases.
This is less-or-more true for vendor kernels; Redhat etc DO maintain source & binary kernel compatibility between major revisions.
More work is gradually being done in the kernel to reduce the amount of kernel-code required to carry out various tasks, such as driver development (for example, USB drivers can typically be done in userspace with libusb), filesystem development (FUSE) and network filtering (NFQUEUE). However, there are still some cases where you need to; in particular, block devices still need to be in the kernel to be able to be usefully used for boot devices and swap.
I've been using Linux for several years, but never stepped beyond installing from a CD/DVD. If the app manager didn't have what I was looking for in the software, then I was a lost cause.
But right now I'm trying to get a grip around what "Linux" is.
The first word that pops into my head is "kernel". After reading on Wikipedia, I understand that a kernel is software running to give other software (OS + apps) access to hardware (CPU, RAM+++). It also handles memory, but isn't that what the OS is supposed to do (what I remember from OS class)?
Is the Linux distro just a packed list of software?
Take my favorite distro: Fedora. It's now in version 14 and ships with kernel 2.6.35.
Does the kernel come from somewhere central and is the core of every Linux distro? If this is true, then is the Linux distro just a way of making the computer with the kernel more user-friendly to use? In that way, the distro+kernel is the OS because the one without the other is not usable (maybe pure kernel, but who sits on that?).
Pretty much correct. To me, "linux" is just the kernel. But it is pretty common to refer to entire distributions as linux. That is what annoys RMS so much. He maintains it should be called GNU/Linux, as he sees distributions as the linux kernel plus the additional software from the GNU project. This makes sense too but I never use the term GNU/Linux. I am either talking about the kernel linux, or "linux distributions", or a specific distribution.
So yes. A distribution is just the kernel (which may include distribution specific patches) plus all the extra programs that make it usable.
The kernel is a central project, and is nominally the same in each distro, but most distros customize it a bit.
And the extra software doesn't just make the kernel more user friendly, it makes it usable at all. A kernel is just interrupt handlers, device drivers, and system calls. It basically virtualizes the hardware and provides a standard environment for programs to work on.
As far as the phrase "operating system" goes, it can be confusing. Some people may say the kernel IS the operating system, and everything else is either a utility or an application or something else. Other people may say the kernel plus some other packages make up the operating system, but most of the software is not part of the operating system. Others may say all the software in the distro forms part of the operating system.
Linux is the kernel. That's what Linus wrote and that's what the kernel developers continue to work on today. It controls the hardware.
An operating system is something that includes a kernel plus quite a few lower-level "applications" to allow you as a user to do useful stuff with your computer (think file manager, control panel and so on).
A distro (distribution) is an operating system packaged with an absolute massive amount of higher-level applications(a) like DVD authoring tools, web browsers, office suites and so on ad-near-infinitum(b).
Now there are grey areas between kernel/OS and even OS/distro but I think that's a fair starting point for understanding how it hangs together.
(a) Even Windows does this to some extent, with the inclusion of Wordpad, Calculator and Paint, though not to the insanely prolific level that Linux distros extend to - do we really need 472 different file managers? Choice is good, yes, but only up to a point :-)
(b) Yes, I'm aware that "ad-near-infinitum" makes no sense since any finite amount subtracted from the infinite is still infinite. But, if you want mathematical accuracy, you should probably be over at https://math.stackexchange.com :-)
OS is just kernel and Shell which work hand in hand.
Distro is combination of customized shell(s) working on a kernel. This means, for examples - Kali, Ubuntu, fedora, Mint, etc. are different distros which work on Linux kernel.
Shell acts as an interface between the user and the kernel. Shell can be command line interface or Graphic user interface. Bash, sh, Windows GUI are some shells.
Kernel is hub of operating system. It allocates time and memory to programs and handles the filestore etc.
To further explain shell and Kernel suppose you type cd. The shell searches the filestore for the file containing the program cd, and then requests the kernel, through system calls, to execute the program cd on myfile.
To take a simple example - Windows GUI is a Shell, Windows OS a distribution by Microsoft.
Similarly, Ubuntu OS or fedora OS etc. a distro working on various shell using Linux kernel.
Shell or a distro does not make Kernel more user friendly to use but it makes it usable for user.
So now, simply you can say Linux is a kernel.
Linux + shell (Bash, Gnome etc.) is a Linux distro say Ubuntu, Mint, Kali etc. and each of them is a OS.
"kernel" and "shell" are the original terms, as in let's say "core" and "shell". "Shell" is the command interpreter. "Distro" is a term that means a customized shell(s) + specific programs included in that distribution. One distribution might several shells though. From a user perspective this is close to the concept of human language. Is the language that you have to talk to the terminal which will talk to shell. Shell will read it and look for a file within the filestore (still inside the shell/ distro). Once the file (executable) is found, shell sends this to the kernel which does the job (process). Think of a car which will have the same basically unmodified engine over many years but will change its frame/ body. I think I need to stop here...
What is kernel's KMS(kernel mode setting) API?
ModeSetting does refer to the graphic stack. It is the process of setting up the clocks and scanout buffers, initialize the chips, lighting up the displays and so on.
The kernel subsystem responsible for this is the DRM subsystem. It has a userspace library that is developed in lock-step with the kernel part and allows i.e. Xorg access to the userland facing part of the interface (normally called ABI). The hardware-facing side of the kernel interface is usually referred to as the API.
Specifically you can use the 'xrandr' binary to instruct XOrg via the randr-protocol to instruct the kernel to change the mode. That binary is installed alongside the X server and also gives you some information about the graphics card and the current mode.
The DRM ModeSetting API is IOCTL based and the following site gives an technical overview: http://dri.freedesktop.org/wiki/DrmModesetting
Also the documentation in the current linux-3.7 releases is quite improved. To check that out, you have to fetch the latest kernel sources, and then, in the kernel sourcetree do
$ make htmldocs
and then look at the generated file Documentation/DocBook/drm/index.html .
Hth
Mode setting is usually related to Graphics setup.
A reference article dated April 19, 2008 notes,
kernel mode-setting involves moving the mode-setting code for video adapters from the user-space X server drivers into the Linux kernel. This may seem like an uninteresting topic for end-users, but having the mode-setting done in the kernel allows for a cleaner and richer boot process, improved suspend and resume support, and more reliable VT switching (along with other advantages). Kernel mode-setting isn't yet in the mainline Linux kernel nor is the API for it frozen, but Fedora 9 shipping next month will be the first major distribution carrying this initial support. In this article we're looking more closely at kernel mode-setting with the Intel X.Org driver as well as showing videos of kernel-based mode-setting in action.
Here is a Fedora wiki KernelModesetting page.
I'm currently setting up vmware Server 2.0 for kernel debugging with gdb ( see this setup guide ) and someone asked me why not use kvm?
So I ask: kvm vs. vmware for kernel debugging / USB driver development
what are the pros and cons of each?
Driver development? are you working on a driver for a particular piece of hardware? if so, then you probably won't be able to use virtualization, because the virtualized instance won't have access to the new hardware.
For this you will need two machines, one running a remote debugger on the other.
*Edit: * Apparently you're developing a driver for a USB Device? this is one area in particular that a VM actually Can help. These days most VM's have the ability to delegate specific USB devices to a guest OS.
That said, this situation doesn't really offer any benefits over the remote debugger option, because you still need a way to inspect the state of the running or crashed OS, and VM's offer very little assistance in this regard. You might be able to replay saved states from just before a crash.
You might be able to get a bit of traction using UML, which would allow you to do local debugging as on a regular user process, which is a little bit less trouble.
Instead of answering the direct question I'll add another option... Depending on if the kernel in question is a Linux kernel, and what part(s) of it you are working on, you might find that UserModeLinux (included in the 2.6.x source, and available as patch sets for 2.4 and 2.2) may trump both of those options.
As it runs the kernel as a userland process under the host kernel it is easier to attach common debugging tools to. I believe it is very commonly used in the early stages of updates/additions to file-system related code. If you are developing/debugging modules that interact directly with hardware it may be much less use to you though.
Reference links: home,
other
I recently started building GNU Mach/HURD and found the combination of QEmu/KVM to work really quite well.. for the following reasons:
QEmu presents quite a clean environment
Networking has alot of options
I can easily mount the filesystem using a raw device file / loopback
Bottom line is, for kernel work I just want the minimum of functionality to boot and see the result. VMWare is much more for usable virtualization rather than down-and-dirty.
There is however no comparison to booting on a real machine with real hardware. The VM environment can seem like a safety blanket somtimes ... because even my toaster would know what a Realtek RTL8139C was.
If it is a "real hardware" device, of course, vmware will not emulate it, so you won't be able to debug the driver under it (nor will any other virtualisation software, unless you extend one to do so).
Device driver debugging can be done to some extent with a real hardware machine with a normal kernel - although there are obviously things you can't do - like set breakpoints.
It is still possible to attach a debugger to the kernel and inspect stuff. Moreover, traditional printf() debugging is quite possible (printk, anyone), and there are various features in the kernel which make debugging easier. It's possible to build the kernel with various debug options to try to detect pointer problems, memory leaks etc.
By default, the kernel even gives a nice-ish stack trace on the log when it encounters an OOPS or BUG condition (obviously this does not necessarily get written anywhere if the system hangs or crashes). Of course a pointer-out-of-range condition happening inside an interrupt is a recipe for disaster, but you could still get a stack trace on the screen immediately before the panic :)