All WoW64 apps go through WoW64 emulation layer.
I'd like to know what happen in this layer.(especially, how they can convert address space)
Please give me some important points.
Since you have already posted the diagram it is clear that you know why WOW64 exists. Now to answer your question:
I'd like to know what happen in this layer.
I think you want to know how it is implemented.
Process startup: The loader loads 64-bit user-mode part 'Ntdll.dll' as usual, but also loads 32-bit Ntdll.dll in case the process is for 32-bit execution. It is now the loaders responsibility to initialize using Wow64.dll, which sets up process and thread contexts in 32-bit Ntdll and 'switches the CPU to 32-bit mode' for execution.
System Call: Everything is now running in 32-bit mode, until a system call. We know that system calls go through Ntdll.dll, User32.dll, and Gdi32.dll etc; in this case the 32-bit versions. There is a separate 32-bit version of these libraries located in \Windows\Syswow64 bit folder. These are just stubs that instead of issuing 'native system calls,' actually call in Wow64.dll. Now, it's simple for Wow64.dll to transition to 64-bit mode, convert parameters to their 64-bit counterparts, issue the system call using 64-bit versions, get the result, and reconvert the output to 32-bit. It then transitions CPU back to 32-bit mode and returns the output.
Exception dispatching, user callbacks, file system and registry operations, and I/O is handled in the same way, using hooks somewhere down the line. Read the book prescribed below.
(especially, how they can convert address space).
64-bit address space is a superset of 32-bit address space. Plus, the same pointer (actually PTE) in 32-bit/64-bit isn't used to refer to the whole address space, but there are separate page tables for user-space and for system space.
Please give me some important points.
To know windows, the most important point I can give you is to read 'Windows Internals' -- Russinovich
MSDN says:
WOW64 is the x86 emulator that allows 32-bit Windows-based applications to run seamlessly on 64-bit Windows. WOW64 is provided with the operating system and does not have to be explicitly enabled.
The system isolates 32-bit applications from 64-bit applications, which includes preventing file and registry collisions. Console, GUI, and service applications are supported. The system provides interoperability across the 32/64 boundary for scenarios such as cut and paste and COM. However, 32-bit processes cannot load 64-bit DLLs for execution, and 64-bit processes cannot load 32-bit DLLs for execution.
What specifically do you not understand? Have you already read the Wikipedia article on the WoW64 subsystem? I think you'll find that it provides a fairly comprehensive overview.
And Microsoft provides some additional details here: WOW64 Implementation Details
"especially, how they can convert address space"
the key to that is knowing that amd and intel x64 processors have support for running 32bit and 64bit code side by side. this allows the os (when running native x64) to create a context for a 32bit thread that has 32bit addressing that coexists with the 64bit threads
in a similar way when the host os is running 32bit, it can create 16bit threads for running win16 and dos applications.
note, I don't think when running in the os 64bit that it can create 32bit AND 16bit threads - I guess amd decided that was just too much backwards compatability :-)
Related
Involving the Windows operating system, a lot of things use the number 32, especially with Win32. I see it in everything from system folders:
C:\Windows\System32\
to system files:
C:\Windows\System32\win32k.sys
to Windows app development:
Develop a Win32 Application
The significance of the number 32 and computers makes me think 32-bit processors, but, if this is correct, why is there a need to explicitly mention 32-bit systems?
Googling around brought me to the Win32 API. This is, I presume, the main cause of its frequent use, but that doesn't change my question. The Windows operating system works perfectly fine on 64-bit systems.
Is Windows specialized for 32-bit systems?
Or is this just a historical thing (i.e. Windows and its API were developed before the 64-bit system emerged)?
Before Win32 there was Win16 (although perhaps maybe not using that name), and running 32-bit code was a special feature or it had special requirements, especially your CPU had to be able to do so.
Intel 8086 was a 16-bit CPU with 20-bit address space.
Intel 80286 was a 16-bit CPU with 24-bit address space.
Intel 80386 was full 32-bit, both registers and address space.
I wrote a 32-bit application that launches a 16-bit executable, which loads some 16-bit DLLs to perform some of the application's functions. It runs fine on Windows 7 32-bit, but not on Windows 7 64-bit since that version of the OS does not include the NTVDM.
The 16-bit portion of the code is pretty extensive and would be pretty expensive to port to 32-bit. Also, it uses some 3rd-party 16-bit APIs from a company that is no longer in business; therefore, that code would have to be recreated completely, thereby increasing the cost.
Is there any possible way to simply load the existing 16-bit DLLs directly from the 32-bit application, removing the 16-bit executable completely?
I've looked into thunking, but it doesn't appear that is supported in Windows 7 either.
You have to use an emulator or a virtual machine.
Or, if that is not an option then you can write a primitive emulator yourself, read the machinecode instruction-by-instruction and modify fake-registers and memory accordingly, and when the program calls outward then you will have to generate responses.
No. This is not possible in Windows.
32-bit x86 is a superset of 16-bit x86. Suppose I write a code in 16-bit x86. It should ideally work on system with 32-bit x86 without any hitch. But that is not the case. Compatibility is an issue here. But why exactly? Is it because 32-bit OS installed on 32-bit x86 machine loads the programs differently in the memory and manages the memory differently?
Are different memory-management requirements the real difference between 16-bit and 32-bit applications?
In Windows:
The major problem with running 16bit program in 32bit OS is that most of 16bit programs used to run on Real Mode, which is not supported anymore(by the OS). These modes are fundamentally different and therefore require software emulation. Also since all of the 16bit API stubs, DOS functions, and BIOS calls are not available, programs would not really be able to interact with the operating system, thus making them unusable without some kind of emulation. In case of Windows, NTVDM does all the emulation starting from Windows NT3.1.
Of course, if your program does not require any interaction with the OS, you should be able to run it. In terms of the opcodes and instruction set, it is true 32bit x86 is superset of 16bit x86. It's just that the environment in which the code usually runs on is completely different.
The only one difference between the 32 bit - and the 16 bit addressmode is the meaning and the usage of those operandsize- and addresssize prefixes.
what is meant by 32-bit application?
Operand size prefix in 16-bit mode
There's a related (16bit on 64bit OS) discussion at superuser here.
I decided to start learning assembly a while ago, and so I started with 16-bit assembly, using FASM.
However, I recently got a really new computer running Windows 7 64-bit, and now none of the compiled .COM files that the program assembles work any more. They give an error message, saying that the .COM is not compatible with 64-bit however.
32-bit assemblies still work, however I'd rather start with 16 and work my way up...
Is it possible to run a 16-bit program on windows 7? Or is there a specific way to compile them? Or should I give up and skip to 32-bit instead?
The reason you can't use 16-bit assembly is because the 16-bit subsystem has been removed from all 64-bit versions of Windows.
The only way to remedy this is to install something like DOSBox, or a virtual machine package such as VirtualBox and then install FreeDOS into that. That way, you get true DOS anyway. (NTVDM is not true DOS.)
Personally, would I encourage writing 16-bit assembly for DOS? No. I'd use 32- or even 64-bit assembly, the reason being there are a different set of function calls for different operating systems (called the ABI). So, the ABI for 64-bit Linux applications is different to 32-bit ones. I am not sure if that's the case with Windows. However, I guarantee that the meaning of interrupts is probably different.
Also, you've got all sorts of things to consider with 16-bit assembly, like the memory model in use. I might be wrong, but I believe DOS gives you 64K memory to play with "and that's it". Everything, your entire heap and stack along with code must fit into this space, as I understand it, which makes you wonder how anything ever worked, really.
My advice would be to just write 32-bit code. While it might initially seem like it would make sense to learn how to write 16-bit code, then "graduate" to 32-bit code, I'd say in reality rather the opposite is true: writing 32-bit code is actually easier because quite a few arbitrary architectural constraints (e.g., on what you can use as a base register) are basically gone in 32-bit code.
For that matter, I'd consider it open to substantial question whether there's ever a real reason to write 16-bit x86 code at all. For most practical purposes, it's a dead platform -- for desktop machines it's seriously obsolete, and for embedded machines, you're more likely to see things like ARMs or Microchip PICs. Unless you have a specific target in mind and know for sure that it's going to be a 16-bit x86, I'd probably forget that it existed, just like most of the rest of the world has.
32-bit Windows 7 and older include / enable NTVDM by default. On 32-bit Win8+, you can enable it in Windows Features.
On 64-bit Windows (or any other 64-bit OS), you need an emulator or full virtualization.
A kernel in long mode can't use vm86 mode to provide a virtual 8086 real-mode environment. This is a limitation of the AMD64 / x86-64 architecture.
With a 64-bit kernel running, the only way for your CPU to natively run in 16-bit mode is 16-bit protected mode (yes this exists; no, nobody uses it, and AFAIK mainstream OSes don't provide a way to use it). Or for the kernel to switch the CPU out of long mode back to legacy mode, but 64-bit kernels don't do that.
But actually, with hardware virtualization (VirtualBox, Hyper-V or whatever using Intel VT-x or AMD SVM), a 64-bit kernel can be the hypervisor for an entire virtual machine, whether that VM is running in 16-bit real mode or running a 32-bit OS (like Windows 98 or 2000) which can in turn use vm86 mode to run 16-bit real-mode executables.
Especially on a 64-bit kernel, it's usually easier to just emulate a 16-bit PC entirely (like DOSBOX does), instead of using HW virtualization to running normal instructions natively but trap direct hardware access (in / out, loads/stores to VGA memory, etc.) and int instructions that make DOS system calls / BIOS calls / whatever.
While it's my understanding that there's no fundamental reason a program written for 32-bit hardware / OSs not to run on 64-bit hardware / OSs, in practice, I've found many programs intended for 32-bit versions of Windows that will not work on 64-bit versions of Windows. Examples include a number of popular security utilities (most products from Norton and Check Point's Zone Alarm) and several games (I've been trying to get Grand Theft Auto 4 to run for a few weeks now, but to no avail - of course, that might be related to any number of other problems related to GTA4, but that's neither here nor there).
I've heard that a program's incompatibility might result from something as simple as not wanting to run from the "Program Files (x86)" folder, but what are some of the other reasons? Why would a virus scanner or firewall written for a 32-bit system not run on a 64-bit system? Why would a game not run when everything is theoretically backwards-compatible?
There is a lot of misinformation on this thread.
When a 32-bit application is run on 64-bit windows:
Most of the compatibility problems come when the application tries to install a kernel-mode driver. A 32-bit driver can't be installed on the 64-bit OS. This is amost certainly the problem with a firewall. It's trying to hook into the TCP/IP driver stack.
THERE IS NO EMULATOR! The 32-bit object code is executed by the cpu completely natively at full speed.
There is no support for old 16-bit code. This broke a lot of installers.
Accessing the right folders is generally not a problem. When a 32-bit program opens a file in, say %windir%\system32\, the OS automagically redirects it to %windir%\syswow64. The same for certain parts of the registry. There are a few potential gotchas here, but they're generally along the lines of assuming that various WINAPI Get...Directory() functions return the same strings that they did in Windows 95.
Whether it was compiled 10 years ago or just yesterday, then C/C++ pointers are still 32-bits (4 bytes) and all of the code that just assumed that -- including SendMessage()! -- still works. The 8-byte pointer issue doesn't come into the picture until you start converting to 64-bit compilers.
The best explanation I've found is offered here which basically says 32-bit programs are run on an layer of emulation which doesn't allow the system access you'd get from native programs run in a 64-bit environment:
http://blogs.msdn.com/oldnewthing/archive/2008/12/22/9244582.aspx
I would assume this means that problems with programs like GTA4 come from the layer of emulation not producing the expected results found on a 32-bit native system. This is why you keep seeing Microsoft release compatibility updates all the time.
Here's what the MSDN has to say about the matter:
http://msdn.microsoft.com/en-us/library/bb427430(VS.85).aspx
Drivers are a different story that programs:
http://support.microsoft.com/kb/896456
Zone Alarm uses a special 32-bit driver created by Check Point to do the monitoring. This is probably what's creating the issue with that application. As for Grand Theft Auto 4? I have no idea.
There can be any number of reasons.
Any application which is programmed ad a low level might be expecting 32 bit register. The Zone Alarm driver posted by novatrust is a good example. GTA4 might be using assembly to improve performance at several points which might result on anything or even simply assuming 32 bits on C++. For example take the following code:
struct GPoint
{
int x;
int y;
}
// Array of twenty GPoints
GPoint[] myArr = malloc(20 * sizeof(GPoint);
GPoint* myPointer = myArr;
int index = GetIndexAffectedPoint();
// Invert X and Y for the point
myPointer += 8*index;
swap(myPointer);
I know the example is pretty naive but anyway, in that code you are assuming you're struct is 8 bytes long (4 bytes of the x integer and 4 bytes for the y integer) but in a 64 bit system is actually twice that long so you'll end up swapping the wrong point... things like that happen a lot on low level languages, specially when trying to improve performance...
Security applications are a bad example. They all perform unsupported things against undocumented things. Changes between one 32-bit version of Windows to the next are enough to break them, nevermind moving to 64-bit.
That said, there are some compatibility shims that fixup your code on 32-bit that won't when the app is 64-bit. This is because Microsoft assumes you've tested it on 64-bit.
One resulting gotcha is with .NET applications. When running on a 32-bit system, the exe is jitted to 32-bits - where there are compatibility shims to fix your bugs. If your customer happens to be running on a 64-bit system, the executable will be jitted to 64-bits, where those compatibility shims that were protected you from yourself are no longer present.
Chris Jackson had a nice blog entry about this: Shimming Applications on Windows Vista 64-Bit
The problem is probably drivers. With games, it's probably some kind of slimy DRM scheme. Try getting the no-CD crack for the game so you can run it without DRM.
32-bit to 64-bit Migration Considerations
EDIT: Alternative link
If you are using the file system or the registry, make sure that you access the right folders. As a x86 program you will probably want to access "Program Files (x86)", "SysWOW64", "WOW6232Node" and such folders instead of the x64 ones.
Products with x86 applications like Norton and Check Point's Zone Alarm fail to run their x86 driver, as the driver needs to x64 to be able to ran by the operating system.