At Microsoft BUILD conference, I saw this figure.
I'm wondering whether the position of Win32 is correct or not. What is the role and position of Win32 API in modern Windows architecture?
My concept of Win32 API is a sort of assembly of all modern Windows architecture. Win32 API (or DDK) bridges "Windows Kernel Services" to the other technologies.
I thought that Internet Explorer/.NET/Silverlight are all build upon Win32 API. Even if a fancy new Metro Apps, I conjecture that Win32 is at the bottom place. However, this figure defies my knowledge.
As you can see, IE/.NET/SL/Metro are sitting horizontally with Win32.
Questions:
Does this figure necessarily mean that Win32 is not the framework of all other modern technologies?
What is "Windows Kernel Services"? How programmers can access them? Isn't it through Win32 API and DDK?
The diagram is wrong.
While there were originally other subsystems (e.g. POSIX) which accessed ntdll.dll (the "Kernel Services" gateway) without going through Win32 (kernel32.dll, user32.dll) these are all now defunct.
All modern application frameworks for Windows are built on Win32. The other subsytems are no longer supported. In some cases (drivers, and boot-time system utilities such as chkdsk) code is written directly against the native API, but these scenarios are very rare.
I'm not very familiar with WinRT, but I think Win32 fits underneath it as well (or it may use a combination of Win32 and direct ntdll.dll services).
According to Sasha Goldshtein,
Next, a C++ Metro application will still load Win32 DLLs such as kernel32 and ntdll. Moreover, the WinRT APIs call into the Win32 DLLs – so they are not a replacement but rather a wrapper, an API flavor, on top of Win32. (Historical note: Windows used to have a feature called “environment subsystems”, which can be roughly described as API flavors. WinRT is not an environment subsystem – it is a library on top of the Win32 environment subsystem.)
The diagram is (presumably) correct, but only for Windows 8. Windows 8 has a significantly different architecture from previous versions.
I believe applications can make requests directly to the kernel, but the interface is mostly undocumented. Device drivers use the kernel directly.
Related
I have a C# class library that references and wraps Win32 native methods. But I can't seem to port this code over to a newer "universal" C# class library. What do I need to do in order to properly marshal these native methods?
Windows 10 Universal Applications run under the Windows Runtime API, a new "native" API for Windows. Applications using this API run in a sandbox which limits their access to devices and operating system services. While Win32 APIs can be accessed like normal, for example using P/Invoke in the case of C# code, the sandbox only permits the use of certain API functions:
Windows Runtime apps can use a subset of the Win32 and COM APIs. This
subset of APIs was chosen to support key scenarios for Windows Runtime
apps that were not already covered by the Windows Runtime, HTML/CSS,
or other supported languages or standards. The Windows App
Certification Kit ensures that your app uses only this subset of the
Win32 and COM API. In a native app, you can call these APIs directly.
In a managed app, you can call them via a Windows Runtime Component,
or via P/Invoke.
...
The documentation for each programming element in the API indicates
whether it can be used in a Windows Runtime app. See the "Applies to"
line at the top of the page.
So for example FormatMessage works with Windows Runtime, but GetNamedPipeInfo doesn't.
If the Win32 API you want to use isn't one of the ones supported by Windows Runtime, I believe your only option is to have your class library target traditional desktop apps.
This article (from the Windows engineering team) says:
WOA [Windows On ARM] will not support any type of virtualization or emulation approach,
and will not enable existing x86/64 applications to be ported or run.
Does that mean I won't even be able to recompile an x86/64 application from source for ARM?
If yes, what exactly prevents me from doing that? My understanding is that the Win32 API is present on Windows 8 ARM.
If no, what do they mean by not being able to port x86/64 applications to ARM?
From the Windows article you link to, the second quoted paragraph is the killer. Porting existing apps is definitely NOT supported.
Developers wishing to target WOA do so by writing applications for the WinRT (Windows APIs for building Metro style apps) using the new Visual Studio 11 tools in a variety of languages, including C#/VB/XAML and Jscript/ HTML5. Native code targeting WinRT is also supported using C and C++, which can be targeted across architectures and distributed through the Windows Store. WOA does not support running, emulating, or porting existing x86/64 desktop apps. Code that uses only system or OS services from WinRT can be used within an app and distributed through the Windows Store for both WOA and x86/64. Consumers obtain all software, including device drivers, through the Windows Store and Microsoft Update or Windows Update.
If we enabled the broad porting of existing code we would fail to deliver on our commitment to longer battery life, predictable performance, and especially a reliable experience over time. The conventions used by today’s Windows apps do not necessarily provide this, whether it is background processes, polling loops, timers, system hooks, startup programs, registry changes, kernel mode code, admin rights, unsigned drivers, add-ins, or a host of other common techniques. By avoiding these constructs, WOA can deliver on a new level of customer satisfaction: your WOA PC will continue to perform well over time as apps are isolated from the system and each other, and you will remain in control of what additional software is running on your behalf, all while letting the capabilities of diverse hardware shine through.
Only a subset of Win32 is supported on Win8 for ARM:
http://msdn.microsoft.com/en-us/library/windows/apps/br205757.aspx
If you try to build an application that uses a Win32 API that's not supported, it won't build because the API won't be in the library. If you try to create your own library to support the APIs, the application still won't be usable because the only way to deploy apps to customers will be through the MS app store, which will no doubt check for API conformance.
From The "Building Windows for the ARM processor architecture" article:
Consumers obtain all software, including device drivers, through the Windows Store and Microsoft Update or Windows Update.
It might be possible to do so on your development machine (but maybe not - I'm really not sure if such a 'hack' will be possible, supported or not), but you certainly won't be able to deploy it in any kind of widespread fashion.
In short, any existing Win32 application will likely need significant work to be ported to Win8 for ARM. It won't be a matter of recompile and fix any errors that pop out to get the application to run on ARM.
They do provide this caveat a little further in your cited article:
Additionally, developers with existing code, whether in C, C++, C#, Visual Basic, or JavaScript, are free to incorporate that code into their apps, so long as it targets the WinRT API set for Windows services.
They've got version of Office and IE available; I'm positive those weren't ground-up reimplementations.
For developing programs for windows, we need windows SDK. I understand that this SDK is what helps to create windows and handle window events and all that. I suppose it also enables us to manipulate with files and registries.
(Does the same SDK is the reason for thread creation and handling?)
All that is fine!
I would like to know what are the files and libraries that come as a part of this SDK. Also does it come when I install the OS or when I install editors like Visual studio? Sometimes I see links to windows SDK separately as such. Is it same as the one that I get when installing Visual Studio or has something more than that.
Base Services:
Provide access to the fundamental resources available to a Windows system.
Included are things like
file systems,
devices,
processes & threads
and error handling.
These functions reside in kernel32.dll on 32-bit Windows.
Advanced Services:
Provide access to functionality that is an addition on the kernel.
Included are things like the
Windows registry
shutdown/restart the system (or abort)
start/stop/create a Windows service
manage user accounts
These functions reside in advapi32.dll on 32-bit Windows.
Graphics Device Interface:
Provides functionality for outputting graphical content to
monitors,
printers
and other output devices.
It resides in gdi32.dll on 32-bit Windows in user-mode. Kernel-mode GDI support is provided by win32k.sys which communicates directly with the graphics driver.
User Interface:
Provides the functionality to create and manage screen windows and most basic controls, such as
buttons and scrollbars,
receive mouse and keyboard input,
and other functionality associated with the GUI part of Windows.
This functional unit resides in user32.dll on 32-bit Windows. Since Windows XP versions, the basic controls reside in comctl32.dll, together with the common controls (Common Control Library).
Common Dialog Box Library:
Provides applications the standard dialog boxes for
opening and saving files,
choosing color and font, etc.
The library resides in comdlg32.dll on 32-bit Windows. It is grouped under the User Interface category of the API.
Common Control Library:
Gives applications access to some advanced controls provided by the operating system. These include things like
status bars,
progress bars,
toolbars
and tabs.
The library resides in comctl32.dll on 32-bit Windows. It is grouped under the User Interface category of the API.
Windows Shell:
Component of the Windows API allows applications to access the
functionality provided by the operating system shell,
as well as change and enhance it.
The component resides in shell32.dll on 32-bit Windows. The Shell Lightweight Utility Functions are in shlwapi.dll. It is grouped under the User Interface category of the API.
Network Services:
Give access to the various networking capabilities of the operating system.
Its sub-components include
NetBIOS,
Winsock,
NetDDE,
RPC and many others.
Internet Explorer web browser APIs:
An embeddable web browser control, contained in shdocvw.dll and mshtml.dll.
The URL monitor service, held in urlmon.dll, which provides COM objects to applications for resolving URLs.
A library for assisting with multi-language and international text support (mlang.dll).
XML support (the MSXML components, held in msxml*.dll).
I have listed only files for 32-bit windows (as that is what many window developers work on)
For more info, please check http://en.wikipedia.org/wiki/Windows_API
EDIT:
Above dlls and all are part of the operating system, not the SDK. The Windows SDK provides import libraries (.lib files) that allow code to dynamically link against these system-provided DLLs.
(This was rightly pointed by Marcelo Cantos. Many thanks to him)
SDK stands for Software Development Kit. It is a big fat collection of headers, libraries, tools and other bits and pieces that developers use to construct software. Microsoft provides many SDK's for their large range of products, and they are not generally deployed to the end-user's desktop. They are usually only installed on developer machines, either as part of a development environment like Visual Studio, or separately (but usually intended for use within VS anyway).
When you talk about the thing that handles windows, threads, etc., you are describing the Windows APIs. The purpose of an SDK is to allow developers to write software that accesses the APIs.
The Windows SDK gives you, as a developer, access to all the services of the Windows platform, including all the things you list.
The SDK is installed as part of the installation of Visual Studio, and usually you'll use the one that came with the compiler, and never have to worry about it. The standalone SDK downloads are there to support two scenarios:
If you use a compiler other that Visual Studio, it may not come with the SDK files, so you can download them separately.
Each new version of Windows includes more features in the API, so to call these new functions you need an updated SDK.
I have a win32 GUI (MFC) application which I need to port to a WIN CE environment. I keep encountering undefined identifiers. What is the best way to deal with this- is there some site where i can get a mapping of some kind between win32 supported features and corresponding wince features (even if they are not supported, the information that they are not is valuable)
I think this is what you are looking for:
MFC Reference for Devices
I was curious as to how does one go about finding undocumented APIs in Windows.
I know the risks involved in using them but this question is focused towards finding them and not whether to use them or not.
Use a tool to dump the export table from a shared library (for example, a .dll such as kernel32.dll). You'll see the named entry points and/or the ordinal entry points. Generally for windows the named entry points are unmangled (extern "C"). You will most likely need to do some peeking at the assembly code and derive the parameters (types, number, order, calling convention, etc) from the stack frame (if there is one) and register usage. If there is no stack frame it is a bit more difficult, but still doable. See the following links for references:
http://www.sf.org.cn/symbian/Tools/symbian_18245.html
http://msdn.microsoft.com/en-us/library/31d242h4.aspx
Check out tools such as dumpbin for investigating export sections.
There are also sites and books out there that try to keep an updated list of undocumented windows APIs:
The Undocumented Functions
A Primer of the Windows Architecture
How To Find Undocumented Constants Used by Windows API Functions
Undocumented Windows
Windows API
Edit:
These same principles work on a multitude of operating systems however, you will need to replace the tool you're using to dump the export table. For example, on Linux you could use nm to dump an object file and list its exports section (among other things). You could also use gdb to set breakpoints and step through the assembly code of an entry point to determine what the arguments should be.
IDA Pro is your best bet here, but please please double please don't actually use them for anything ever.
They're internal because they change; they can (and do) even change as a result of a Hotfix, so you're not even guaranteed your undocumented API will work for the specific OS version and Service Pack level you wrote it for. If you ship a product like that, you're living on borrowed time.
Everybody here so far is missing some substantial functionality that comprises hugely un-documented portions of the Windows OS RPC . RPC (think rpcrt4.dll, lsass.exe, csrss.exe, etc...) operations occur very frequently across all subsystems, via LPC ports or other interfaces, their functionality is buried in the mysticism incantations of various type/sub-type/struct-typedef's etc... which are substantially more difficult to debug, due to the asynchronous nature or the fact that they are destine for process's which if you were to debug via single stepping or what have you, you would find the entire system lockup due to blocking keyboard or other I/O from being passed ;)
ReactOS is probably the most expedient way to investigate undocumented API. They have a fairly mature kernel and other executive's built up. IDA is fairly time-intensive and it's unlikely you will find anything the ReactOS people have not already.
Here's a blurb from the linked page;
ReactOS® is a free, modern operating
system based on the design of Windows®
XP/2003. Written completely from
scratch, it aims to follow the
Windows® architecture designed by
Microsoft from the hardware level
right through to the application
level. This is not a Linux based
system, and shares none of the unix
architecture.
The main goal of the
ReactOS project is to provide an
operating system which is binary
compatible with Windows. This will
allow your Windows applications and
drivers to run as they would on your
Windows system. Additionally, the look
and feel of the Windows operating
system is used, such that people
accustomed to the familiar user
interface of Windows® would find using
ReactOS straightforward. The ultimate
goal of ReactOS is to allow you to
remove Windows® and install ReactOS
without the end user noticing the
change.
When I am investigating some rarely seen Windows construct, ReactOS is often the only credible reference.
Look at the system dlls and what functions they export. Every API function, whether documented or not, is exported in one of them (user, kernel, ...).
For user mode APIs you can open Kernel32.dll User32.dll Gdi32.dll, specially ntdll.dll in dependancy walker and find all the exported APIs. But you will not have the documentation offcourse.
Just found a good article on Native APIS by Mark Russinovich