So a .exe file is a file that can be executed by windows, but what exactly does it contain? Assembly language that's processor specific? Or some sort of intermediate statement that's recognized by windows which turns it into assembly for a specific processor? What exactly does windows do with the file when it "executes" it?
MSDN has an article "An In-Depth Look into the Win32 Portable Executable File Format" that describes the structure of an executable file.
Basically, a .exe contains several blobs of data and instructions on how they should be loaded into memory. Some of these sections happen to contain machine code that can be executed (other sections contain program data, resources, relocation information, import information, etc.)
I suggest you get a copy of Windows Internals for a full description of what happens when you run an exe.
For a native executable, the machine code is platform specific. The .exe's header indicates what platform the .exe is for.
When running a native .exe the following happens (grossly simplified):
A process object is created.
The exe file is read into that process's memory. Different sections of the .exe (code, data, etc.) are mapped in separately and given different permissions (code is execute, data is read/write, constants are read-only).
Relocations occur in the .exe (addresses get patched if the .exe was not loaded at its preferred address.)
The import table is walked and dependent DLL's are loaded.
DLL's are mapped in a similar method to .exe's, with relocations occuring and their dependent DLL's being loaded. Imported functions from DLL's are resolved.
The process starts execution at an initial stub in NTDLL.
The initial loader stub runs the entry points for each DLL, and then jumps to the entry point of the .exe.
Managed executables contain MSIL (Microsoft Intermediate Language) and may be compiled so they can target any CPU that the CLR supports. I am not that familiar with the inner workings of the CLR loader (what native code initially runs to boot strap the CLR and start interpreting the MSIL) - perhaps someone else can elaborate on that.
I can tell you what the first two bytes in .exe files contain - 'MZ'. i mean the characters 'MZ'.
It actually represents: Mark Zbikowski. The guy who designed the exe file format.
http://en.wikipedia.org/wiki/Mark_Zbikowski
1's and 0's!
This wikipedia link will give you all the info you need on the Portable Executable format used for Windows applications.
An EXE file is really a type of file known as a Portable Executable. It contains binary data, which can be read by the processor and executed (essentially x86 instructions.) There's also a lot of header data and other miscellaneous content. The actual executable code is located in a section called .text, and is stored as machine instructions (processor specific). This code (as well as other parts of the .EXE) are put into memory, and the CPU is sent to it, where it starts executing. (Note that there's much more interfaces actually happening; this is a simplified explanation).
Related
I was reading about it and the part about memory was really confusing. What exactly happens to a DLL after compilation?
Questions that were somewhat bugging me:
Is it loaded into memory only once and all processes that require access to it are given only a pointer to where it is?
When is it loaded? I am sure it isn't just arbitrarily loaded into memory after compilation so is there a special procedure to load it or does Windows load a DLL when a process requires it and keeps it for sharing among other processes?
From the Microsoft docs
Every process that loads the DLL maps it into its virtual address space. After the process loads the DLL into its virtual address, it can call the exported DLL functions.
How does that "mapping" look like? I found that a bit confusing.
I don't know if this is a relevant piece of info but I am specifically interested in custom DLLs (DLLs written by me), not system DLLs
An exe file lists DLLs is wants to link to, so when the loader loads an exe, it loads DLLs that are listed as required, unless they are already loaded. A DLL may run initialization code the first time it loads.
Of course a DLL can be loaded dynamically by name, then it's loaded when the LoadLibrary API call is issued by a program. This is useful to implement dynamically loadable plugins.
Windows keeps a reference counter for each DLL, so when all processes stopped referencing a DLL, either by exiting or explicitly calling FreeLibrary, Windows will unload the DLL, giving it a chance to run any cleanup code.
I was asked such a question in an interview:
In windows, suppose there is an exe which depends on some dlls, when you start
the exe, and then the dependent dlls will be loaded, are these dlls
loaded in kernel mode or user mode?
I am not quite sure about the question, not the mention the answer - could you help to explain?
Thanks.
I'm not an expert about how Windows internally works, but for what i know the correct answer is user mode, simply because only the processes related to your Operative System are admitted in the kernel space http://en.wikibooks.org/wiki/Windows_Programming/User_Mode_vs_Kernel_Mode
Basically if it's not an OS process, it's going to be allocated in the user space.
The question is very imprecise/ambiguous. "In Windows" suggests something but isn't clear what. Likely the interviewer was referring to the Win32 subsystem - i.e. the part of Windows that you usually get to see as an end-user. The last part of the question is even more ambiguous.
Now while process and section objects (in MSDN referred to as MMF, loaded PE images such as .exe and .dll and .sys) are indeed kernel objects and require some assistance from the underlying executive (and memory manager etc) the respective code in the DLL (including that in DllMain) will behave exactly the same as for any other user mode process, when called from a user mode process. That is, each thread that is running code from the DLL will transition to kernel mode to make use of OS services eventually (opening files, loading PE files, creating events etc) or do some stuff in user mode whenever that is sufficient.
Perhaps the interviewer was even interested in the memory ranges that are sometimes referred to as "kernel space" and "user space", traditionally at the 2 GB boundary for 32bit. And yes, DLLs usually end up below the 2 GB boundary, i.e. in "user space", while other shared memory (memory mapped files, MMF) usually end up above that boundary.
It is even possible that the interviewer fell victim to a common misunderstanding about DLLs. The DLL itself is merely a dormant piece of memory, it isn't running anything on its own ever (and yes, this is also true for DllMain). Sure, the loader will take care of all kinds of things such as relocations, but in the end nothing will run without being called explicitly or implicitly (in the context of some thread of the process loading the DLL). So for all practical purposes the question would require you to ask back.
Define "in Windows".
Also "dlls loaded in kernel mode or user mode", does this refer to the code doing the loading or to the end result (i.e. where the code runs or in what memory range it gets loaded)? Parts of that code run in user mode, others in kernel mode.
I wonder whether the interviewer has a clear idea of the concepts s/he is asking about.
Let me add some more information. It seems from the comments on the other answer that people have the same misconception that exists about DLLs also about drivers. Drivers are much closer to the idea of DLLs than to that of EXEs (or ultimately "processes"). The thing is that a driver doesn't do anything on its own most of the time (though it can create system threads to change that). Drivers are not processes and they do not create processes.
The answer is quite obviously User mode for anybody who does any kind of significant application development for windows. Let me explain two things.
DLL
A dynamic link library is closely similar to a regular old link library or .lib. When your application uses a .lib it pastes in function definitions just after compile time. You typically use a .lib to store API's and to modify the functions with out having to rebuild the whole project, just paste new .lib with same name over the old and as long as the interface(function name and parameters) hasn't changed it still works. Great modularity.
A .dll does exactly the same thing however it doesn't require re-linking or any compilation. You can think of a .dll as essentially a .lib which gets compiled to an .exe just the same as applications which use it. Simply put the new .dll which shares the name and function signatures and it all just works. You can update your application simply by replacing .dlls. This is why most windows software consists of .dlls and a few exe's.
The usage of a .dll is done in two ways
Implicit linking
To link this way if you had a .dll userapplication.dll you would have an userapplication.lib which defines all the entry points in the dll. You simply link to the static link library and then include the .dll in the working directory.
Explicit linking
Alernatively you can programmatically load the .dll by first calling LoadLibrary(userapplication.dll) which returns a handle to your .dll. Then GetProcAddress(handle, "FunctionInUserApplicationDll") which returns a function pointer you can use. This way your application can check stuff before attempting to use it. c# is a little different but easier.
USER/KERNEL MODES
Windows has two major modes of execution. User mode and Kernel modes (kernel further divided into system and sessions). For user mode the physical memory address is opaque. User mode makes use of virtual memory which is mapped to real memory spaces. User mode driver's are coincidentally also .dll's. A user mode application typically gets around 4Gb of virtual addressing space to work with. Two different applications can not meaningfully use those address because they are with in context of that application or process. There is no way for a user mode application to know it's physical memory address with out falling back to kernel mode driver. Basically everything your used to programming (unless you develop drivers).
Kernel mode is protected from user mode applications. Most hardware drivers work in the context of kernel mode and typically all windows api's are broken into two categories user and kernel. Kernel mode drivers use kernel mode api's and do not use user mode api's and hence don't user .dll's(You can't even print to a console cause that is a user mode api set). Instead they use .sys files which are drivers and essentially work exactly the same way in user mode. A .sys is an pe format so basically an .exe just like a .dll is like an .exe with out a main() entry point.
So from the askers perspective you have two groups
[kernel/.sys] and [user/.dll or .exe]
There really isn't .exe's in kernel because the operating system does everything not users. When system or another kernel component starts something they do it by calling DriverEntry() method so I guess that is like main().
So this question in this sense is quite simple.
I always thought .dll is working the same way as .so in linux,
but some article says not.
Does windows ensure that memory will contain only one copy the same dll as in linux?
I don't have a clue to check it myself in windows,so I can only ask here.
UPDATE
Anyone knows how do verify this manually?
Here's a short description: DLL Hell, basically in modern Windows it use a technique called Memory Mapping where the DLL is loaded once, if both processes try to load the DLL from the same directory. So to answer your question, it is working the same was as in Linux.
If the DLL can be loaded at the same base virtual address in two processes then there will only be one copy of the DLL in physical memory.
Since Windows does not use position independent code, if a DLL cannot load at its preferred base address it will be rebased and thus not be able to share physical memory with other instances.
Process Monitor and Explorer are supplied an EXE file.
But they include a driver. -Where is it.
By Windows Internals,
Process Monitor works by extracting a file system filter device driver from its executable
image (Procmon.exe) the first time you run it after a boot, installing the driver in memory, and then deleting the driver image from disk.
I would like to know the detail mechanism.
Are there some codes about that? Where can I find them.
Or could you explain me this.
Thanks.
Last time I looked it was just embedded into the executable as a resource. You can use something like Resource Hacker to see it. I guess when the process starts it extracts the driver from the resource section and installs it.
Executable file in Windows may contain among other things "resource" section. It may contain any binary data, which executable may access at run-time.
The trick is to put the whole other executable (the SYS file of the driver for instance) inside an EXE during the link time. Then at runtime the EXE extract this into a SYS file.
Then this driver may be loaded on-the-fly (using SC-manager)
I've always been curious as to how one extension, EXE, can be as versatile as it is in that if you assemble an assembly program, you get an EXE in machine code for your processor but if you compile a C# or other .Net program, you also get an EXE except that it is run in the proper runtime environment. I'm not sure if this is different from OS to OS (I imagine it is), but when an EXE is executed, how is it determined how to execute it?
On a related note, if I were writing my own programming language, how would I tie in my runtime environment into this mechanism?
When compiling a .NET program to an EXE, it's more than just a blob of bytecode (like Java). There's actually native executable created that will load the .NET runtime and hand off the .NET bytecode to it, or display a friendly-ish error message indicating that the framework is not available.
The format is even more flexible than that, as every Windows EXE actually includes a DOS program at the beginning which will display an error ("cannot run in DOS mode") when executed as a DOS program.
You can read more details on the PE format on Wikipedia: http://en.wikipedia.org/wiki/Portable_Executable