The new 1-bit exploit of "all" windows versions uses a bug in the kernel code that handles scrollbars. That got me thinking. Why does windows handle scrollbars in kernel, rather than user mode? Historical reasons? Does any other OS do this?
TL;DR: Microsoft sacrificed security for performance.
Scrollbars are a bit special on Windows. Most scrollbars are not real windows but are implemented as decorations on the "parent" window. This leads us to a more general question; why are windows implemented in kernel mode on Windows?
Lets look at the alternatives:
Per-process in user mode.
Single "master" process in user mode.
Alternative 1 has a big advantage when dealing with your own windows; no context switch/kernel transition. The problem is of course that windows from different processes live on the same screen and somebody has to be responsible for deciding which window is active and coordinate changes when the user switches to a different window. This somebody would have to be a special system process or the kernel because this information cannot be per-process, it has to be stored somewhere global. This dual information design is going to be complicated because the per-process information cannot be trusted by the global window manager. I'm sure there are a ton of other downsides to this theoretical design but I'm not going to spend more time on it here.
Windows NT 3 implemented a variant of alternative 2. The window manager was moved into kernel mode in NT 4 mainly for performance reasons:
...the Window Manager (USER) and Graphics Device Interface (GDI) have
been moved from the Win32 subsystem to the Windows NT Executive. Win32
user-mode device drivers, including graphics display and printer
drivers, have also been moved to the Executive. These changes are
designed to simplify graphics handling, reduce memory requirements,
and improve performance.
...and further down in the same document there are more technical details and justifications:
When Windows NT was first designed, the Win32 environment subsystem
was designed as a peer to the environment subsystems supporting
applications in MS-DOS, POSIX, and OS/2. However, applications and
other subsystems needed to use the graphics, windowing, and messaging
functions in the Win32 subsystem. To avoid duplicating these
functions, the Win32 subsystem was used as a server for graphics
functions to all subsystems.
This design worked respectably for Windows NT 3.5 and 3.51, but it
underestimated the volume and frequency of graphics calls. Having
functions as basic as messaging and window control in a separate
process generated substantial memory overhead from client/server
message passing, data gathering, and managing multiple threads. It
also required multiple context switches, which consume CPU cycles as
well as memory. The volume of graphics support calls per second
degraded the performance of the system. It was clear that a redesign
of this facet in Windows NT 4.0 could reclaim these wasted system
resources and improve performance.
The other subsystems are not that relevant these days but the performance issues remain.
If we look at a simple function like IsWindowVisible then there is not a lot of overhead when the window manager is in kernel mode: The function will execute a couple of instructions in user mode and then switch the CPU to ring 0 where the entire operation (validate the window handle passed in and if valid, retrieve the visible property) is performed in kernel mode. It then switches back to user mode and that is about it.
If the window manager lives in another process then you will at least double the amount of kernel transitions and you must somehow pass the functions input and output to and from the window manager process and you must somehow cause the window manager process to execute while you wait for the result. NT 3 did this by using a combination of shared memory, LPC and a obscure feature called paired threads.
Related
This question is about operating systems in general. Is there any necessary mechanism in implementation of operating systems that impacts flow of instructions my program sends to CPU?
For example if my program was set for maximum priority in OS, would it perform exactly the same when run without OS?
Is there any necessary mechanism in implementation of operating systems that impacts flow of instructions my program sends to CPU?
Not strictly necessary mechanisms (depending on how you define "OS"); but typically there's IRQs, exceptions and task switches.
IRQs are used by devices to ask the OS (their device driver) for attention; and interrupting the flow of instructions your program sends to CPU. The alternative is polling, which wastes a huge amount of CPU time checking if the device needs attention when it probably doesn't. Because applications need to use devices (file IO, keyboard, video, etc) and wasting CPU time is bad; IRQs significantly improve the performance of applications.
Exceptions (like IRQs) also interrupt the normal flow of instructions. They occur when the normal flow of instructions can't continue, either because your program crashed, or because your program needs something. The most common cause of exceptions is virtual memory (e.g. using swap space to let the application have more memory than actually exists so that the application can actually work properly; where the exception tells the OS that your program tried to access memory that has to be fetched from disk first). In general; this also improves performance for multiple reasons (because "can't execute because there's not enough RAM" can be considered "zero performance"; and because various tricks reduce RAM consumption and increase the amount of RAM that can be used for things like caching files which improve file IO speed).
Task switches is the basis of multi-tasking (e.g. being able to run more than one application at a time). If there are more tasks that want CPU time than there are CPUs, then the OS (scheduler) may (depending on task priorities and scheduler design) switch between them so that all the tasks get some CPU time. However; most applications spend most of their time waiting for something to do (e.g. waiting for user to press a key) and don't need CPU time while waiting; and if the OS is only running one task then the scheduler does nothing (no task switches because there's no other task to switch to). In other words, if the OS supports multi-tasking but you're only running one task, then it makes no difference.
Note that in some cases, IRQs and/or tasks are also used to "opportunistically" do work in the background (when hardware has nothing better to do) to improve performance (e.g. pre-fetch, pre-process and/or pre-calculate data before it's needed so that the resulting data is available instantly when it is needed).
For example if my program was set for maximum priority in OS, would it perform exactly the same when run without OS?
It's best to think of it as many layers - hardware & devices (CPU, etc), with kernel and device drivers on top, with applications on top of that. If you remove any of the layers nothing works (e.g. how can an application read and write files when there's no file system and no disk device drivers?).
If you shift all of the functionality that an OS provides into the application (e.g. a statically linked library that can make an application boot on bare metal); then if the functionality is the same the performance will be the same.
You can only improve performance by reducing functionality. For example, if you get rid of security you'll improve performance (temporarily, until your application becomes part of an attacker's botnet and performance becomes significantly worse due to all the bitcoin mining it's doing). In a similar way, you can get rid of flexibility (reboot the computer when you plug in a different USB flash stick), or fault tolerance (trash all of your data without any warning when the storage devices start failing because software assumed hardware is permanently perfect).
OS use kernel mode (privilege mode) and user mode. It seems very reasonable for security reasons. Process cant make any command it wants, only the operation system can make those commands.
On the other hand it take long time all the context switch. change between user to kernel mode and vice versa.
The trap to the operation system take a long time.
I think why the operation system not give the ability to process to run in kernel mode to increase it's performance (this can be very big improve)?
In real time systems this works in the same way?
Thanks.
There are safety and stability reasons, which disallow user-space process to access kernel space functions directly.
Kernel code garantees, that no user-space process(until being executed with root priveleges) can break operating system. This is a vital property of modern OS. Also it is important, that development of user-space apps is much more simple, than kernel modules development.
In case when application needs more perfomance than available for use-space, it is possible to move its code(or part of it) into kernel space. E.g., network protocols and filesystems are implemented as kernel drivers mostly because of perfomance reasons.
Real time applications are more demanding to stability. They also use system calls.
I think there is no sense to do this.
1.) If you want something to be runned in kernel context use kernel module API, what is the problem with that?
2.) Why do you think that it will multiple process speed? Switch between kernel and userspace is just additional registers state save / restore. It will run faster, but i don't think user will even notice it.
How does one programmatically cause the OS to switch off, go away and stop doing anything at all so that a program may have complete control of a PC system?
I'm interested in doing this from both an MS Windows and Linux environments. Any languages or APIs considered.
I want the OS to stop preempting my program, stop its virtual memory management, stop its device drivers and interrupt service routines from running and basically just go away. Then, when my program has had its evil way with the bare metal, I want the OS to come back again without a reboot.
Is this even possible?
With Linux, you could use kexec jump to transfer control completely to another kernel (ie, your program). Of course, with great power comes great responsibility - it is entirely up to you to service interrupts, and avoid corrupting the old kernel's memory. You'll end up having to write your own OS kernel to do this. Also, the transfer of control takes quite some time, as the kernel has to de-initialize all hardware, then reinitialize it when it's time to resume. Since kexec jump was originally designed for hibernation support, this isn't a problem in its original context, but depending on what you're doing, it might be a problem.
You may want to consider instead working within the framework given to you by the OS - just write a normal driver for whatever you're doing.
Finally, one more option would be using the linux Real-Time patchset. This lets you assign static priorities to everything, even interrupt handlers; by running a process with higher priority than anything else, you could suspend /nearly/ everything - the system will still service a small stub for interrupts, as well as certain interrupts that can't be deferred, like timing interrupts, but for the most part the heavy work will be deferred until you relinquish control of the CPU.
Note that the RT patchset won't stop virtual memory and the like - mlockall will prevent page faults on valid pages though, if that's enough for you.
Also, keep in mind that whatever you do, the system BIOS can still cause SMM traps, which cannot be disabled, except by motherboard-model-specific methods.
There are lots of really ugly ways to do this. You could modify the running kernel by writing some trampoline code to /dev/kmem that passes control to your application. But I wouldn't recommend attempting something like that!
Basically, you would need to have your application act as its own operating system. If you want to read data from a file, you would have to figure out where the data lives on disk, and generate your own SCSI requests to talk to the disk drive. You would have to implement your own interrupt handler to get notified when the data is ready. Likewise you would have to handle page faults, memory allocation, etc. Most users feel that this isn't worth the effort...
Why do you want to do this?
Is there something that your application needs to do that the OS won't let it do? Are you concerned with the OS impact on performance? Something else?
If you don't mind shelling out some cash, you could use IntervalZero's RTX to do this for a Windows system. It's a hard realtime subsystem that gets installed on a Windows box as sort of a hack into the HAL and takes over the machine, letting Windows have whatever CPU cycles are left over.
It has its own scheduler and device drivers, but if you run your program at the top RTX priority, don't install any RTX device drivers (or disable interrupts for the duration), then nothing will interrupt it.
It also supports a small amount of interaction with programs on the Windows side.
We use it as a nice way to get a hard realtime box that runs Windows.
coLinux loads CoLinuxDriver into the NT kernel or a colinux.ko into the Linux kernel. It does exactly what you asked – it "unschedules" the host OS, and runs its own code, with its own memory management, interrupts, etc. Then, when it's done, it "reschedules" the host OS, allowing it to continue from where it left off. coLinux uses this to run a modified Linux kernel parallel to the host OS.
Unlike more common virtualization techniques, there are no barriers between coLinux and the bare metal hardware at all. However, hardware and the host OS tend to get confused if the coLinux guest touches anything without restoring it before returning to the host OS.
Not really. Operating Systems are a foundation, and your program runs on top of them. The OS handles memory access, disk writing operations, communications, etc. when your application makes requests, and asking the OS to move out of the way would mean that your program would have to do the OS's job instead.
Not as such, no.
What you want is basically an application that becomes an OS; a severely stripped down Linux kernel coupled with some highly customized and minimized tools might be the way to go for this.
if you were devious, and wanted to avoid alot of the operating system housekeeping you could probably hook yourself into a driver routine. Thinking out aloud, verging on hacking. google how to write root kits.
Yeah dude, you can totally do that, you can also write a program to tell my bank to give you all my money and send you a hot Russian.
I have an application that is using about 100k more of the Desktop Heap in this version then it did last version. Is there a way I can see what is on the Desktop Heap and how big the individual objects are? Using Dheapmon I was able to see what percentage of the heap I was using, but I want more details.
Stolen from a comment on a blog post here
Let me
give a little background on how
desktop heap allocations are made. The
desktop heaps are in kernel mode
virtual address space, so individual
desktop heap allocations have to be
made by a component running in kernel
mode. In particular, win32k.sys is the
only kernel mode component that makes
desktop heap allocations. win32k.sys
in the kernel mode side of Win32, and
it includes both the window manager
(USER) and GDI. It is the window
manager piece of win32k.sys that uses
desktop heap. The functionality of the
window manager is exposed to processes
running in user mode through
user32.dll. It is user32.dll that
exports user mode callable functions
that are implemented in win32k.sys. So
if a process does not load user32.dll,
it will not use desktop heap.
Regarding your question about setting
a breakpoint that will catch desktop
heap allocations... yes, there is such
a function - win32k!DesktopAlloc.
However, this is a kernel mode
function, and to set a breakpoint on
it will require that you use a kernel
debugger.
That sounds all scary complicated to me who has never ventured away from user-mode in Windows.
When I had a similar problem I just put break points all over the startup portion of our application. At each break I would watch the level of allocated handles and what dhelpmon told me. Doing a sort of binary search I started to narrow down where the allocations were happening.
Dheapmon is the only tool I know of for looking directly at the desktop heap, but have you tried looking at your application with a tool like Winspector to look for glaring differences between the two versions (say, some type of control in your application now contains far more windows)? Any chance the application has switched to a newer version of IE? I seem to remember IE7 being much more desktop heap-intensive than IE6...
You can walk the heap using the Win32 API call HeapWalk. You can call GetProcessHeap to get all the heaps available to the process if you need to walk more than just the default heap.
Problem: I have a developers machine (read: fast, lots of memory), but the user has a users machine (read: slow, not very much memory).
I can simulate a slow network using Fiddler (http://www.fiddler2.com/fiddler2/)
I can look at how CPU is used over time for a process using Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx).
Is there any way I can restrict the amount of CPU a process can have, or the amount of memory a process can have in order to simulate a users machine more effectively? (In order to isolate performance problems for instance)
I suppose I could use a VM, but I'm looking for something a bit lighter.
I'm using Windows XP, but a solution for any Windows machine would be welcome. Thanks.
The platform SDK used to come with stress tools for doing just this back in the good old days (STRESS.EXE, CPUSTRESS.EXE in the SDK), but they might still be there (check your platform SDK and/or Visual Studio installation for these two files -- unfortunately I have niether the PSDK nor VS installed on the machine I'm typing from.)
Other tools:
memory: performance & reliability (e.g. handling failed memory allocation): can use EatMem
CPU: performance & reliability (e.g. race conditions): can use CPU Burn, Prime95, etc
handles (GDI, User): reliability (e.g. handling failed GDI resource allocation): ??? may have to write your own, but running out of GDI handles (buggy GTK apps would usually eat them all away until all other apps on the system would start falling dead like flies) is a real test for any Windows app
disk: performance & reliability (e.g. handling disk full): DiskFiller, etc.
AppVerifier has a low-resource simulation feature.
You could also try setting the priority of your process to be very low.
You can run MemAlloc to chew up RAM, possibly a few copies at once.
I found a related question:
Set Windows process (or user) memory limit
The accepted answer for the question has a link to the Windows API's SetProcessWorkingSetSize, so it's not exactly a tool that can limit the amount of memory that a process can use.
In terms of changing the amount of CPU resources a process can use, if you don't mind the granularity of per-core limiting of resources, Task Manager can change the processor affinity of a process.
In Task Manager, right-click a process and select "Set Affinity...", then select the processor cores that the process can be assigned to.
If the development machine has many cores but the user machine only has one, then, rather than allowing the process to run on all the available cores, set the process' processor affinity to only one core.
It has nothing to do with SetProcessWorkingSetSize
Just use internal Win32 kernel apis to restrict CPU Usage