I'm looking for an explanation or good free online resources about the organization of memory and memory management in Windows systems.
You can buy this (old) book, (the actual Windows architecture hasn't really changed much in 10 years) for a penny (used):
Inside Windows NT
Here's the up-to-date version:
MS Windows Internals
There's also Mark's Blog
As far as ONLINE resources go, I've found it to be pretty sparse, though :(
Check out this DotNet Rocks podcast on the Windows memory model
In MSDN, some information is located here: http://msdn.microsoft.com/en-us/library/aa366525(VS.85).aspx
Windows Internals is the canonical reference for mm. It's far better than anything else you'll find on the subject, as Landy Wang doesn't write too much :)
Related
Is there any ultimate Hyper-V Performance Monitor Counters guide from Microsoft I can use to see the official (not blog-posted) information of what each counter does and how it should be used?
It is felt that google doesn't know any, some people say it doesn't even exist.
Not so official but still looks legit: https://blogs.technet.microsoft.com/chrisavis/2013/03/25/performance-management-monitoring-cpu-resources/
Found this (using DuckDuckGo ;) https://learn.microsoft.com/en-us/windows-server/administration/performance-tuning/role/hyper-v-server/detecting-virtualized-environment-bottlenecks
Is this what you are looking for?
Is there some sort of thing like a layout/plan of the windows api? A clear overview of which api depends on another? Or a website where they are linked in a hierarchical way (starting with the most dependable, ending with the core function)
e.g Kerenel32.dll!CreateFile() -> ntdll.dll!ntCreateFIle() -> ntoskrnl32.exe!zwCreateFile()
I can link them all by hand with a dependency walker but it takes much time. So I was just wondering if something like this already existed.
I don't have the answer to your question, but I think an even more interesting road would be to read what Geoff Chappell has found out regarding the Windows Kernel and the Win32 API. I have not read those sections, but so far everything I've read on that website has filled me with wonder and awe at how thoroughly things can be analysed given a disciplined and talented investigator.
You can always take a look at ReactOS:
ReactOS® is a free, modern operating system based on the design of Windows® XP/2003. Written completely from scratch, it aims to follow the Windows-NT® architecture designed by Microsoft from the hardware level right through to the application level.
Simply search for whatever function in the documentation and look at its source. 99% of the time you can just pretend to be looking at the actual Windows source :) Very helpful when playing around with a debugger too.
I am currently in an university operating system class and we are working on the windows kernel, more precisely WRK, the windows research kernel, for our projects. WRK is based off of win2k3 server.
I am however having a real hard time dredging up resources to help learn the basics of OS development, Windows kernel development and just generally getting around the Windows API.
We are using the book Microsoft Internals by Russinovich but I was wondering if any of you had some great resources to recommend to me, whether book, online guides or some old class notes. Thanks!
What specifically are you looking for? Online resources? For that, OSROnline is one of the better websites. Alot of kernel development knowledge is found in the MS and the OSR Mailing lists, that's another place to check that might be better than Stack overflow.
Specifically books, there is the Programming WDM,Developing drivers with KMDF and Advance Windows Debugging. The last specifically will not teach you so much about the kernel and more how to navigate inside it, something you will do quite often if you are writing drivers or researching parts of it.
In order to write drivers, the easiest way is probably to take Windows Driver samples and hack at them, stare the results with windbg and learn more.
microsoft kernel dev? that's just weird. what university are you at?
one of the most interesting things about kernels, in my opinion, is the scheduler algorithms. I'd recommend you check that out.
I can't imagine where you'd start looking for windows stuff though. I did it with the linux kernel and there's a LOT of resources (of course).
http://oreilly.com/catalog/linuxkernel/chapter/ch10.html
The third edition of Tanenbaum's Modern Operating Systems has a chapter devoted to the Vista kernel. I haven't looked into that chapter (I only read the Linux one), but as far as big-picture stuff, it's fantastic. I'm not sure what level of detail you're looking for, but that might be a good resource to check out.
I am trying to investigate the behaviour of the Windows Kernel with respect to Memory Mapped Files / Virtual Memory. Specifically I am interested in determining how frequently the contents of a memory mapped file are flushed (by Windows) to disk and what criterion Windows uses for deciding it is time to do so.
I have done a bit of research online and, apart from the MSDN which deals more with the 'hows and whys' rather than detailing the internal workings, there doesn't appear to be much information. If anyone can point me to any articles or has looked into this before and has some insight I would welcome it.
Thanks.
I suggest reading Microsoft Windows Internals by Mark Russinovich and David Solomon, in fifth edition also with Alex Ionescu.
Memory mapped files are controlled by the modified page writer in the kernel - they're flushed whenever the memory that backs the page needs to be re-used (so it can happen under memory pressure).
The system also keeps track of the number of dirty pages and writes them after a threshold is reached.
You can find more information about in this article about Windows memory management changes for Windows Vista - it doesn't directly answer your question about how the memory manager treats modified pages but you can infer some of the details based on the information in the article.
The internal workings are an implementation detail and subject to change. They're not documented anywhere because Microsoft doesn't want you relying on the details.
I've got to ask: Why do you care? If you have hard real-time requirements, Windows might not be the OS for you...
I would suggest that you obtain a copy of
Microsoft® Windows® Internals
Fourth Edition: Microsoft Windows Server 2003 Windows XP and Windows 2000
There are pdfs floating around in the cyber cloud.
This is the book done by the "sysinternals" crowd who have written some very good windows utils over the years.
http://www.microsoft.com/technet/sysinternals
You will have to read the chapters on
Memory Management
Storage Management
Cache Manager
File Systems
They don't directly address your concern but if you manage to read, digest and grasp the very techical paragraphs you can work it out yourself. However if you can predict why Windows does what it does "You are a better man than most".
Happy brain freeze reading that lot.
Seriously, I've trawled MSDN and only got half answers - what do the columns on the Task Manager mean? Why can't I calculate the VM Usage by enumerating threads, modules, heaps &c.? How can I be sure I am accurately reporting to clients of my memory manager how much address space is left? Are their myriad collisions in the memory glossary namespace?
An online resource would be most useful in the short term, although books would be acceptable in the medium term.
Try the book "Windows Internals" by Mark Russinovich and I think some other guy too. It's pretty good on getting down to the nitty gritty.
Mark Russinovich has written the excellent book Windows Internals. A new edition that covers the Vista and Server 2008 operating systems is currently in the works with David Solomon, so you may want to pre-order that if your questions are about the new Windows operating systems instead of the old ones.
Here is a quick article on Windows Memory Management, which goes into sufficient depth to interpret what you're actually seeing in Task Manager or Process Explorer.