Part1:
To the linux/unix experts out there, Could you please help me understanding about device drivers. As i understood, a driver is a piece of code that directly interacts with hardware and exposes some apis to access the device. My question is where does this piece of code runs, User space or Kernel space?
I know that code that is executed in kernel space has some extra privileges like accessing any memory location(pls correct if i'm wrong). If we install a third party driver and if it runs in kernel space, wouldn't this be harmful for the whole system? How any OS handles this?
Part2:
Lets take an example of USB device(camera, keyboard..), How the system recognizes these devices? how does the system know which driver to install? How does the driver know the address of the device to read and write the data?
(if this is too big to answer here, pls provide links of some good documentation or tutorials.., I've tried and couldn't find answers for these. pls help)
Part 1
On linux, drivers run in kernel space. And yes, as you state there a significant security implications to this. Most exceptions in drivers will take down the kernel, potentially corrupt kernel memory (with all manner of consequences). Buggy drivers also have an impact on system security, and malicious drivers can do absolutely anything they want.
A trend seen on MacOSX and Window NT kernels is user-space drivers. Microsoft has for some time been pushing the Windows Userspace Driver Framework, and MacOSX has long provided user-space APIs for Firewire and USB drivers, and class-compliant drivers for many USB peripherals. it is quite unusual to install 3rd party kernel-mode device drivers on MacOSX.
Arguably, the bad reputation Windows used to have for kernel panics can be attributed to the (often poor quality) kernel mode drivers that came with just about every mobile phone, camera and printer.
Linux graphics drivers are pretty much all implemented in user-space with a minimal kernel-resident portion, and Fuse allows the implementation of filing systems in user-space.
Part 2
USB, Firewire, MCI (and also PCI-e) all have enumeration mechanisms through which a bus driver can match the device to a driver. In practice this means that all devices expose metadata describing what they are.
Contained within the metadata is a DeviceID, VendorID and a description of functions the device provides and associated ClassIDs. ClassIDs facilitate generic Class Drivers.
Conceptually, the operating system will attempt to find a driver that specifically supports the VendorID and DeviceID, and then fall back to one that supports the ClassID(s).
Matching devices to drivers is a core concept at the heart of the Linux Device Model, and exact matching criteria used for matching is match() function in the specific bus driver.
Once device drivers are bound to a device, it uses the bus-driver (or addressing information given by it) to perform read and writes. In the case of PCI and Firewire, this is a memory mapped IO address. For USB it bus addressing information.
The Linux Documentation tree provides some insight into the design of the Linux Device Model, but isn't really entry-level reading.
I'd also recommend reading Linux Device Driver (3rd Edition)
Related
A friend of mine asked me this question in the class and I could not answer it. He asked:
Since we know kernel controls the physical hardware via device drivers. What if all this functionality is kept inside the device controller itself rather than kernel managing them. What would be the consequences of such scenario? Good or Bad?
I searched online for this question but could not get information about this scenario. May be I'm not googling in the right keyword.
You insight into this will help me getting clearing my concepts.
Please answer.
Thanks.
Your question seems to propose the elimination of the "device driver" by "keeping" "control (of) the physical hardware ... inside the device controller". The premise for this seems to be:
kernel controls the physical hardware via device drivers.
That description of a device driver is something similar to what I've seem for end-user comprehension rather than from a developer's perspective. The end-user is aware of the device, and it is the device driver that takes that abstraction and can control that device down to the specific control bits of each device port.
But a device driver is responsible for mundane housekeeping tasks such as:
maintaining device status and availability;
configuring the device for operation;
managing data flow, setting-up/tearing-down data transfers, copying data between user space and kernel space;
handling interrupts and exceptions.
These tasks are integral to a device driver. These tasks cannot be transferred out of the purview of the kernel driver to a peripheral device.
Sometimes the device driver can only try to manage the device, rather than fully control it, for example, a NIC driver during a packet flood.
There is simply no possibility that you can eliminate a device driver no matter how much of "all this functionality is kept inside the device controller itself". And there would still be control directives/commands issued from the device driver to the peripheral.
The hardware device in question should be a computer peripheral device, not an autonomous robot device. The device should be designed to operate with a computer. Whatever interface there is between processor and device should be suitable for the task. If the peripheral is made more "intelligent", then perhaps the CPU can be unburdened and a high-level command interface can replace low-level sub-operation directives. But only "some" functionality can be transferred to the peripheral, not "all".
I'm looking to fuzz virtual drivers, I've read the other questions about this but they don't really go anywhere. Basically looking to see if there's an obvious tool I've missed and want to know if fuzzing IOCTLs from a windows guest would work? Or if I need to write one in low level eg IN/OUT?
Any tools out there for fuzzing drivers in a windows guest to hit the hypervisor either hyper-v or VMware
There are a number of ways to exercise virtualization code.
First, of course, if you're on Windows, is the IOCTL interface.
Then you should remember that all virtual devices are emulated in some way by some code in the guest OS and in the host OS. So, accessing input devices (keyboard and mouse), video device, storage (disks), network card, communication ports (serial, parallel), standard PC devices (PIC, PIT, RTC, DMA), CPU APIC, etc etc will also exercise virtualization code.
It's also very important to remember that virtualization of the various PC devices (unless we're talking about synthetic devices working over the VMBUS in Windows) is done by intercepting, parsing and emulating/executing instructions that access device memory-mapped buffers and registers and I/O ports. This gives you yet another "interface" to pound on.
By using it you might uncover not only device-related bugs but also instruction-related bugs. If you're interested in the latter, you need to have a good understanding of how the x86 CPU works at the instruction level in various modes (real, virtual 8086, protected, 64-bit), how it handles interrupts and exceptions and you'll also need to know how to access those PC devices (how and at what memory addresses and I/O port numbers).
Btw, Windows won't let you directly access these things unless your code is running in the kernel. You may want to have a non-Windows guest VM for things like this just to avoid overprotective functionality of Windows. Look for edge cases, unusual instruction encodings (including invalid encodings) or unusual instructions for usual tasks (e.g. using FPU/MMX/SSE/etc or special protected-mode instructions (like SIDT) to access devices). Think and be naughty.
Another thing to consider is race conditions and computational or I/O load. You may have some luck exploring in that direction too.
I am doing an OS experiment. Until now, all my code utilized the real mode BIOS interrupts to manipulate the hard disk and floppy. But once my code enables Protected Mode, all the real mode BIOS interrupt service routines won't be available.
I have a feeling that I need to do some hardware drivers now. Am I right? Is this why an OS is so difficult to develop?
I know that hardware is controlled by reading from and writing to certain control or data registers. For example, I know that the "Command Block Registers" of a hard disk range from 0x1F0 to 0x1F7. I am wondering whether the register addresses of so many different hardware devices are consistent on different platforms? Or do I have to detect that before using them? How would I do that?
Since I am not sure about how to read/write a floppy or a hard disk in Protected Mode, I have to use BIOS interrupts to load all my necessary kernel files from the floppy before entering protected mode. What could I do if my kernel file exceeds the real mode 1M space limit?
How do I read/write a hard disk when the CPU is in Protected Mode?
I have a feeling that I need to do some hardware drivers now. Am I right?
Strictly speaking; (and depending on your requirements) "need" may be too strong - in theory you can switch back to real mode to use BIOS functions, or use a virtual8086 monitor, or write an interpreter that interprets the firmware's instructions instead of executing them directly.
However, the BIOS is awful (designed for an "only one thing can happen at a time" environment that is completely unsuitable for modern systems where its expected that all devices are able to do useful work at the same time), and the BIOS is deprecated (replaced by UEFI), and it's hard to call something an OS when it doesn't have control over the hardware (because the firmware still has control of the hardware).
Note that if you do continue using BIOS functions; the state of various pieces of hardware (interrupt controller, PCI configuration space for various devices, any PCI bridges, timer/s, etc) has to match the expectations of the BIOS. What this means is that you will either be forced to accept huge limitations (e.g. never being able to use IO APICs, etc. properly) because it will break BIOS functions used by other pre-existing code, or you will be forced to do a huge amount of work to make the BIOS happy (emulating various pieces of hardware so the BIOS thinks the hardware is still in the state it expects even though it's not).
In other words; if you want an OS that's good then you do need to write drivers; but if you only want an OS that doesn't work on modern computers (UEFI), has severe performance problems ("only one thing can happen at a time"), is significantly harder to improve, doesn't support any devices that the BIOS doesn't support (e.g. sound cards), and doesn't support any kind of "hot-plug" (e.g. plugging in a USB device), then you don't need to write drivers.
Is this why an OS is so difficult to develop?
A bad OS is easy to develop. For example, something that is as horrible as MS-DOS (but not compatible with MS-DOS) could probably be slapped together in 1 month.
What makes an OS difficult to develop is making it good. Things like caring about security, trying to get acceptable performance, supporting multi-CPU, providing fault tolerance, trying to make it more future-proof/extensible, providing a nice GUI, creating well thought-out standards (for APIs, etc), and power management - these are what makes an OS difficult.
Device drivers add to the difficulty. Before you can write drivers you'll need support for things that drivers depend on (memory management, IRQ handling, etc - possibly including scheduler and some kind of communication); then something to auto-detect devices (e.g. to scan PCI configuration space) and try to start the drivers for whatever was detected (possibly/hopefully from file system or initial RAM disk, with the ability to add/unload/replace drivers without rebooting); and something to manage the tree of devices - e.g. so that you know which "child devices" will be effected when you put a "parent device" to sleep (or the "parent device" has hardware faults, or its driver crashes, or the device is unplugged). Of course then you'd need to write the device drivers, where the difficulty depends on the device itself (e.g. a device driver for a NVidia GPU is probably harder to write than a device driver for a RS232 serial port controller).
For storage devices themselves (assuming "80x86 PC") there's about 8 standards that matter (ATA/ATAPI, AHCI and NVMe; then OHCI, UHCI, eHCI and xHCI for USB controllers, then the USB mass storage device spec). However, there is also various RAID controllers and/or SCSI controllers where there's no standard (each of these controllers need their own driver), and some obsolete stuff (floppy controller, tape drives that plugged into floppy controller or parallel port, three proprietary CD-ROM interfaces that were built into sound cards).
Please understand that supporting all of this isn't the goal. The goal should be to provide things device drivers depend on (described above), then provide specifications that describe the device driver interfaces (possibly/hopefully including things like IO priorities and synchronization, and notifications for device/media removal, error handling, etc) so that other people can write device drivers for you. Once that's done you might implement a few specific device drivers yourself (e.g. maybe just AHCI initially - everything else could be left until much later or until someone else writes it).
You don't necessarily HAVE to write drivers. You could drop back into real mode to call the BIOS service, and then hop back into protected mode when you're done. This is essentially how DPMI DOS extenders (DOS4GW, Causeway, etc) work.
The source code for the Causeway DOS extender is public domain, you can look at that for a reference. http://www.devoresoftware.com/freesource/cwsrc.htm
I am trying to complete the picture of how the PC and the OS interacts together. And I am at point, where I am little out of guess when it comes to device drivers.
Please, don´t write things like its too complicated, or you don´t need to know when using high programming laguage and winapi functions. I want to know, it´s for study purposes.
So, the very basic structure of how OS and PC (by PC I mean of course HW) is how I see it is that all other than direct CPU commands, which can CPU do on itself (arithmetic operation, its registers access and memory access) must pass thru OS. Mainly becouse from ring level 3 you cannot use in and out intructions which are used for acesing other HW. I know that there is MMIO,but it must be set by port comunication first.
It was not like this all the time. Even I am bit young to remember MSDOS, I know you could access HW directly, becouse there ws no limitation, no ring mode. So you could to write string to diplay use wheather DOS function, or directly acess video card memory and write it by yourself.
But as OS developed, there is no longer this possibility. But it is fine, since OS now handles all the HW comunication, and frankly it more convinient and much more safe (I would say the only option) in multitasking environment. So nowdays you instead of using int instructions to use BIOS mapped function or DOS function you call dll which internally than handles everything you don´t need to know about.
I understand this. I also undrstand that device drivers is the piece of code that runs in ring level 0, so it can do all the HW interactions. But what I don´t understand is connection between OS and device driver. Let´s take a example - I want to make a sound card make a sound. So I call windows API to acess sound card, but what happens than? Does windows call device drivers to do so?
But if it does call device driver, does it mean, that all device drivers which can be called by winAPI function, must have routines named in some specific way? I mean, when I have new sound card, must its drivers have functions named same as the old one? So Windows can actually call the same function from its perspective? But if Windows have predefined sets of functions requored by device drivers, that it cannot use new drivers that doesent existed before last version of OS came out.
Please, help me understand this mess. I am really getting mad. Thanks.
A Windows device driver is a bit like a DLL: except that instead of an application dynamic linking/loading it, it's the O/S that dynamic links/loads it.
Registry entries tell the O/S what device drivers exist (so that the O/S knows which device drivers to dynamic-link/load).
The device drivers run in ring 0. In ring zero, they (device drivers) don't have access to (can't link to or use) Windows APIs: instead they have access to various NT kernel APIs.
But if it does call device driver, does it mean, that all device drivers which can be called by winAPI function, must have routines named in some specific way? I mean, when I have new sound card, must its drivers have functions named same as the old one? So Windows can actually call the same function from its perspective?
Basically yes. All the device drivers within a given type or class (e.g. all video drivers, or all disk drivers) have a similar API, which is invoked by the O/S (and/or invoked by higher-level drivers, for example disk drivers are used/invoked by file system drivers).
The Windows Device Driver Kit defines the various APIs and includes sample drivers for the various types of device.
But if Windows have predefined sets of functions requored by device drivers, that it cannot use new drivers that doesent existed before last version of OS came out.
The O/S is dynamic-linking to the device driver functions: because device driver APIs are predefined, device drivers are interchangeable as far as the O/S is concerned; new device drivers can be written, provided they support (are backward-compatible with) the standard device driver API.
The dynamic-linking mechanism is very similar to the way in which COM objects or C++ classes implement any predefined pure-abstract interface: a header file in the DDK declares the pure-abstract interface (like virtual functions), device drivers implement these functions, and the O/S loads the drivers and invokes these functions.
The basics:
Please note that this explanation is simplified and sometime only true for most cases and not all.
Most HW devices you will ever encounter will have these basic operations:
Write to memory(or Registers) on them.
Read from memory(or Registers) on them.
This is enough to control the HW, to give it the data it needs, and to get the data you want from it.
These memory areas are mapped by the BIOS and/or the OS to the Physical memory range on your PC (which may in turn be accessed by your driver.)
So we now have two operations READ and WRITE that the device driver knows to do.
In addition, the driver can read and write in a manner that does not involve the cpu. This is called Direct Memory Access (DMA) and usually performed by your HW.
The last type of operation is called INTERRUPTS and is meant for your HW to notify your driver of something that just happend. This is usually done by the HW interrupting the CPU and calling your driver to perform some operation in high priority. For example: an image is ready in the HW to be read by the driver.
A question to maybe some who have worked extensively with WinUSB APIs or user mode USB drivers - can the WinUSB API or a user mode driver be used as a passive observer of USB connections, capturing notification of interrupts, control requests, data transfers, etc. without interfering with other applications (such as iTunes) which would obviously require concurrent access to the device at the same time my application is monitoring the connection and displaying data on it?
Or do you pretty much have to write a kernel-mode filter driver and inject yourself in the USB stack in order to make that happen?
In the past, there have been a few credible options (libusb-win32 and usbsnoop to be specific) though both are built around the old DDK, not the Windows Driver Foundation, and are not really supported on a regular basis any more. I'm hesitant to build something significant around them, as a result.
You must write a filter driver. There is no other way to intercept traffic generated by other devices. And you can use newer versions of the DDK, that is, KMDF for this task. It will make your life easier. But I think it will be cheaper and more effective just to get a hardware bus analyzer.
There is a good overview by OSR. You must signed in to read this, but I strongly advice to sign in anyway if you are going to do any kernel mode development.
The core USB drivers in Windows 7 have ETW trace points, so you can sniff traffic that way (blog post ETW in the Windows 7 USB core stack), but it is a bit kludgy and has some limitations like only tracing the first 32 bytes of each packet.
I have made a USB filter driver (KMDF) that is pretty easy to use (IMHO): http://code.google.com/p/busdog/
USB Sniffer allows you to display, record and analyze all the data transferred between any USB device connected to your PC and applications. Also you can recover transferred data (for some file types).
The resulted project includes such components: Driver, Common library, GUI.
http://tellmeword.com/5zu2z5