Some Win32 API function documentation (for example this and this) contains the following note:
Starting with TBD Release Iron, the behavior of this and other NUMA
functions has been modified to better support systems with nodes
containing more that 64 processors. For more information about this
change, including information about enabling the old behavior of this
API, see NUMA Support.
What exactly is the "TBD Release Iron"?
Which Windows versions does it support?
What modifications does the note refer to?
Elsewhere, for example on https://learn.microsoft.com/en-us/windows/win32/procthread/numa-support, it's called
Windows 10 Build 20348
In the same note.
So it looks like the folks at MSDN MS Docs Learn have some mass replacing to do.
As for the actual change, there now are (as is tradition) Ex methods for NUMA that add support for processor groups, allowing you to specify affinity for more applications running on machines with more than 64 logical processors, if I interpret it correctly.
"To Be Determined (TBD) release" means Microsoft has not decided exactly when this feature is going to be shipped. Iron is a branch codename. According to Betawiki, Iron was 21H1 and included Windows Server 2022.
Why Microsoft expects people to keep up with their internal codenames and if the changes have shipped and they just forgot to update the page to use the publuc name or if the feature/change has not shipped yet, I don't know.
The changes as compared with Windows 7 are listed under "Behavior starting with TBD Release Iron" on each page...
Related
we have developed a kernel extension (KEXT) for a virtual file system (VFS) on macOS to integrate our software with external programs like Adobe InDesign or Microsoft Word. Our software and the KEXT are used by many of our customers.
As it looks like KEXTs are deprecated and may be removed completely in future versions of macOS, particularly on Apple Silicon based computers. See e.g. Apple's announcement in its security guide:
"This is why developers are being strongly encouraged to adopt system extensions before kext support is removed from macOS for future Mac computers with Apple silicon"
Therefore we are currently investigating in possible alternatives.
Apple suggests to migrate to System Extensions instead of KEXTs. However, the only VFS related API we found is to implement a File Provider that is based on an NSFileProviderReplicatedExtension.
Unfortunately that NSFileProviderReplicatedExtension has several flaws:
Files can either be in the cloud or downloaded. It is not possible to download/read only a portion of a file. This is a big performance problem for us, since we work with large images (> 1GB). The programs we integrate with typically only read a part of the image, e.g. the embedded preview. The API does not offer a way to access selected blocks of a file (random access file).
The File Provider learns about the file system content via enumerators. So everything that is inside a folder must be enumerated (listed) first. Otherwise it cannot be accessed. However, we cannot enumerate our VFS. Most of the content of our VFS is fully dynamic. It only exists when it is accessed by a client the first time. Such dynamic content also includes dynamic parameters like the client's locale or the size of a box where the image will be placed. Since we do not know those parameters in advance, we cannot enumerate the VFS's content in advance.
This means, an NSFileProviderReplicatedExtension in its current state isn't a replacement for a "real" VFS and therefore cannot be used by us as a replacement for our current VFS KEXT.
My questions:
Will Apple allow kernel extensions also in future versions of (Apple Silicon/M1 based) operating systems? Or is there at least a clear deadline?
If not, what is Apple's officially suggested replacement for KEXT based VFS solutions?
Will the API of an NSFileProviderReplicatedExtension be improved to behave like a "real" file system so that above mentioned flaws will no longer be an issue?
Many thanks for any answers or comments!
Best regards,
Michael
Will Apple allow kernel extensions also in future versions of (Apple Silicon/M1 based) operating systems? Or is there at least a clear deadline?
Apple doesn't really give timelines, and they also occasionally break promises of support.
However, this sort of hard API deprecation and removal usually is done as part of a major release, so you will typically get deprecation notice for it at WWDC one year, users might start seeing deprecation notices when the .0 of the OS release ships at the earliest, and sometimes the .3 or .4 revision. Then you'll typically be told at the next WWDC that the API is blocked in the upcoming release, so by that point you should have implemented a replacement.
If not, what is Apple's officially suggested replacement for KEXT based VFS solutions?
As far as I'm aware, NSFileProviderReplicatedExtension is currently the only one.
Will the API of an NSFileProviderReplicatedExtension be improved to behave like a "real" file system so that above mentioned flaws will no longer be an issue?
Other than via beta SDKs, Apple generally doesn't pre-announce future APIs.
My advice:
File issues for each of the file provider shortcomings you are hitting using Feedback Assistant. (Radar)
File an "enhancement request" feedback issue with Apple for a "real" file system API replacement for the VFS KPI.
If your vfs kext is critical to your business/product, I suggest additionally asking Apple's DTS via a TSI what they recommend for your situation. Reference the feedback IDs of the issues filed, otherwise they will recommend that you file issues.
I am very new to windows system programming. In my project I need to read the registered "ProgramFiles" location for 32/64bit processes.
I have finally come up to two choices: either to use SHGetKnownFolderPath or to read the values for these folders from system registry, but I have some security concerns. Can someone please compare these two methods in terms of security and reliability.
You don't read stuff from undocumented random places in the registry, because that's just an implementation detail of where Windows currently stores that data: it may easily happen that:
in some next version of Windows they'll decide that such data needs to be stored elsewhere;
the data you found just is there only on some configurations (some particular IE versions installed, the machine has not joined Active Directory, there's no folder redirection inplace, ...) - but you can't know it, there's no documentation that guarantees you anything.
The correct way to go is to use the documented interfaces that the OS provides, on which Microsoft explicitly makes promises of compatibility (they promise that a public function that works today - if used according to the documentation - will continue to work tomorrow).
tl;dr: use SHGetKnownFolderPath - or SHGetFolderPath if you want to remain compatible with Windows versions before Vista, which in general is a good thing, given that Windows XP still seems to have more market share than all OS X versions combined.
I want to get an overview of all the programs that are being used and how many versions of this software that is being used. I do not need to know the exact version number (though it would be nice), just be able to say that two things are distinct versions (or builds).
Because I do not know anything about each program, I need this to be done in a generic way. How could this be done?
This is quite a general question, so I'll give you a general answer. You are going to need to do the following:
Enumerate all the processes by calling EnumProcesses().
For each process ID, OpenProcess() to obtain a process handle.
With each process handle call GetModuleFileNameEx() to obtain the process's main executable file name.
Finally call GetFileVersionInfo() and perhaps some of its friends to retrieve the information.
This will give you binary version information rather than marketing versions. For example Windows XP is version 5.1, Windows Vista is 6.0, Windows 7 is version 6.1. If you need marketing versions then that's probably not achievable in a general manner.
I have several questions about the PE "subsystem" field, which may overlap to some extent. To avoid spamming this place with each question separately, I thought I'd ask them together and then re-ask separately anything that doesn't get addressed. Hope this is OK...
I know that IMAGE_SUBSYSTEM_WINDOWS_CUI makes the OS "pre-attach" the process to a console, either of its parent process or creating a new console if necessary. IMAGE_SUBSYSTEM_WINDOWS_GUI doesn't do that.
Are there any other differences between these two in a modern version of Windows? Have there been more in the past?
What about the other values, are they used by Windows merely to reject an EXE, or do they cause Windows to emulate a different API? Is this "emulation" process extensible by end-users or is this hard-baked into the OS?
Yes, pre-attaching a console seems to be the only current difference. If memory serves, that wasn't so much the case with the 16-bit based versions of Windows though (95/98/SE/Me).
Older versions of NT accepted other values for the POSIX and OS/2 subsystems.
In theory, you might be able to write your own subsystem using the NT Native API. Although there's a fair amount of documentation around for that if you look, I'm not at all sure it's sufficient to this task. The Win32 subsystem has always had rather a "special" status anyway, and now that the other subsystems are gone, I'm not at all sure they've even tried to assure that other subsystems can be integrated. On NT 4 (for one example) I'd have said "difficult but almost certainly possible". On a current version of Windows, I'd say there's about even odds that some change to the kernel would prevent it from working -- and much lower than even odds that the currently-available documentation would be up to the task of anybody but MS implementing a subsystem.
I was curious as to how does one go about finding undocumented APIs in Windows.
I know the risks involved in using them but this question is focused towards finding them and not whether to use them or not.
Use a tool to dump the export table from a shared library (for example, a .dll such as kernel32.dll). You'll see the named entry points and/or the ordinal entry points. Generally for windows the named entry points are unmangled (extern "C"). You will most likely need to do some peeking at the assembly code and derive the parameters (types, number, order, calling convention, etc) from the stack frame (if there is one) and register usage. If there is no stack frame it is a bit more difficult, but still doable. See the following links for references:
http://www.sf.org.cn/symbian/Tools/symbian_18245.html
http://msdn.microsoft.com/en-us/library/31d242h4.aspx
Check out tools such as dumpbin for investigating export sections.
There are also sites and books out there that try to keep an updated list of undocumented windows APIs:
The Undocumented Functions
A Primer of the Windows Architecture
How To Find Undocumented Constants Used by Windows API Functions
Undocumented Windows
Windows API
Edit:
These same principles work on a multitude of operating systems however, you will need to replace the tool you're using to dump the export table. For example, on Linux you could use nm to dump an object file and list its exports section (among other things). You could also use gdb to set breakpoints and step through the assembly code of an entry point to determine what the arguments should be.
IDA Pro is your best bet here, but please please double please don't actually use them for anything ever.
They're internal because they change; they can (and do) even change as a result of a Hotfix, so you're not even guaranteed your undocumented API will work for the specific OS version and Service Pack level you wrote it for. If you ship a product like that, you're living on borrowed time.
Everybody here so far is missing some substantial functionality that comprises hugely un-documented portions of the Windows OS RPC . RPC (think rpcrt4.dll, lsass.exe, csrss.exe, etc...) operations occur very frequently across all subsystems, via LPC ports or other interfaces, their functionality is buried in the mysticism incantations of various type/sub-type/struct-typedef's etc... which are substantially more difficult to debug, due to the asynchronous nature or the fact that they are destine for process's which if you were to debug via single stepping or what have you, you would find the entire system lockup due to blocking keyboard or other I/O from being passed ;)
ReactOS is probably the most expedient way to investigate undocumented API. They have a fairly mature kernel and other executive's built up. IDA is fairly time-intensive and it's unlikely you will find anything the ReactOS people have not already.
Here's a blurb from the linked page;
ReactOS® is a free, modern operating
system based on the design of Windows®
XP/2003. Written completely from
scratch, it aims to follow the
Windows® architecture designed by
Microsoft from the hardware level
right through to the application
level. This is not a Linux based
system, and shares none of the unix
architecture.
The main goal of the
ReactOS project is to provide an
operating system which is binary
compatible with Windows. This will
allow your Windows applications and
drivers to run as they would on your
Windows system. Additionally, the look
and feel of the Windows operating
system is used, such that people
accustomed to the familiar user
interface of Windows® would find using
ReactOS straightforward. The ultimate
goal of ReactOS is to allow you to
remove Windows® and install ReactOS
without the end user noticing the
change.
When I am investigating some rarely seen Windows construct, ReactOS is often the only credible reference.
Look at the system dlls and what functions they export. Every API function, whether documented or not, is exported in one of them (user, kernel, ...).
For user mode APIs you can open Kernel32.dll User32.dll Gdi32.dll, specially ntdll.dll in dependancy walker and find all the exported APIs. But you will not have the documentation offcourse.
Just found a good article on Native APIS by Mark Russinovich