I was reading about it and the part about memory was really confusing. What exactly happens to a DLL after compilation?
Questions that were somewhat bugging me:
Is it loaded into memory only once and all processes that require access to it are given only a pointer to where it is?
When is it loaded? I am sure it isn't just arbitrarily loaded into memory after compilation so is there a special procedure to load it or does Windows load a DLL when a process requires it and keeps it for sharing among other processes?
From the Microsoft docs
Every process that loads the DLL maps it into its virtual address space. After the process loads the DLL into its virtual address, it can call the exported DLL functions.
How does that "mapping" look like? I found that a bit confusing.
I don't know if this is a relevant piece of info but I am specifically interested in custom DLLs (DLLs written by me), not system DLLs
An exe file lists DLLs is wants to link to, so when the loader loads an exe, it loads DLLs that are listed as required, unless they are already loaded. A DLL may run initialization code the first time it loads.
Of course a DLL can be loaded dynamically by name, then it's loaded when the LoadLibrary API call is issued by a program. This is useful to implement dynamically loadable plugins.
Windows keeps a reference counter for each DLL, so when all processes stopped referencing a DLL, either by exiting or explicitly calling FreeLibrary, Windows will unload the DLL, giving it a chance to run any cleanup code.
Related
The page Create C/C++ DLLs in Visual Studio says the following in its second paragraph:
A DLL isn't a stand-alone executable. DLLs run in the context of the
applications that call them. The operating system loads the DLL into
an application's memory space. It's done either when the application
is loaded (implicit linking), or on demand at runtime (explicit
linking). DLLs also make it easy to share functions and resources
across executables. Multiple applications can access the contents of a
single copy of a DLL in memory at the same time.
Is the sentence highlighted above correct? How can this statement be reconciled with the first bullet point in Advantages of Dynamic Linking, whose contents I reproduce below:
Multiple processes that load the same DLL at the same base address share a single copy of the DLL in physical memory. Doing this saves
system memory and reduces swapping.
In summary, if there is a single copy of a DLL in memory, how can it be loaded into an application's memory space?
This question already has answers here:
Questions Regarding DLL Loading in a Process Address Space
(2 answers)
Closed 8 years ago.
I'm sure this question has been asked multiple times, and I have read tons of articles, but I still don
t understand it. Where exactly does a DLL get loaded? I am talking about the actual code of the DLL. When I call, lets say, MessageBoxA, where does it jump to? Does the code get loaded into my own process, or does it jump somewhere else(not even sure if this is possible)?
Please enlighten me, this has been bothering me for quite some time, but I just can't seem to figure it out.
DLLs are loaded once per process. Here are some more details:
From Wikipedia:
DLLs execute in the memory space of the calling process and with the same access permissions
As described in this Stack Overflow answer:
A DLL is loaded at most once per process. So, if your DLL is used by multiple other DLLs, it still will exist once per process.
MSDN has plenty more details:
The system maintains a per-process reference count for each DLL. When a thread loads the DLL, the reference count is incremented by one. When the process terminates, or when the reference count becomes zero (run-time dynamic linking only), the DLL is unloaded from the virtual address space of the process.
Like any other function, an exported DLL function runs in the context of the thread that calls it. Therefore, the following conditions apply:
The threads of the process that called the DLL can use handles opened by a DLL function. Similarly, handles opened by any thread of the calling process can be used in the DLL function.
The DLL uses the stack of the calling thread and the virtual address space of the calling process.
The DLL allocates memory from the virtual address space of the calling process.
On Variable Scope:
Variables that are declared as global in a DLL source code file are treated as global variables by the compiler and linker, but each process that loads a given DLL gets its own copy of that DLL's global variables. The scope of static variables is limited to the block in which the static variables are declared. As a result, each process has its own instance of the DLL global and static variables by default.
Before DLLs are loaded, they have to be found. There are specific rules about how Windows searches for DLLs, such as those listed at Dynamic-Link Library Search Order.
In Windows if there are two processes each using the same DLL then apparently each process separately loads the DLL into its address space while in Linux a shared-object is loaded once and mapped into different processes. Can someone explain to me the pros and cons of the Windows approach ?
I'm not sure the difference is so stark. Windows shares everything except the data segment between all users of a DLL by loading the DLL once and mapping the shared parts into each process. However, any global data in the DLL is loaded separately for each process so that processes don't unintentionally share data. I'd be surprised if linux wasn't very similar, otherwise shared libraries could pose significant security risks, not to mention potential reliability issues as well. Here are a couple references:
From stackoverflow:
Are .dll files loaded once for every program or once for all programs?
From wikipedia:
http://en.wikipedia.org/wiki/Dynamic-link_library
I was asked such a question in an interview:
In windows, suppose there is an exe which depends on some dlls, when you start
the exe, and then the dependent dlls will be loaded, are these dlls
loaded in kernel mode or user mode?
I am not quite sure about the question, not the mention the answer - could you help to explain?
Thanks.
I'm not an expert about how Windows internally works, but for what i know the correct answer is user mode, simply because only the processes related to your Operative System are admitted in the kernel space http://en.wikibooks.org/wiki/Windows_Programming/User_Mode_vs_Kernel_Mode
Basically if it's not an OS process, it's going to be allocated in the user space.
The question is very imprecise/ambiguous. "In Windows" suggests something but isn't clear what. Likely the interviewer was referring to the Win32 subsystem - i.e. the part of Windows that you usually get to see as an end-user. The last part of the question is even more ambiguous.
Now while process and section objects (in MSDN referred to as MMF, loaded PE images such as .exe and .dll and .sys) are indeed kernel objects and require some assistance from the underlying executive (and memory manager etc) the respective code in the DLL (including that in DllMain) will behave exactly the same as for any other user mode process, when called from a user mode process. That is, each thread that is running code from the DLL will transition to kernel mode to make use of OS services eventually (opening files, loading PE files, creating events etc) or do some stuff in user mode whenever that is sufficient.
Perhaps the interviewer was even interested in the memory ranges that are sometimes referred to as "kernel space" and "user space", traditionally at the 2 GB boundary for 32bit. And yes, DLLs usually end up below the 2 GB boundary, i.e. in "user space", while other shared memory (memory mapped files, MMF) usually end up above that boundary.
It is even possible that the interviewer fell victim to a common misunderstanding about DLLs. The DLL itself is merely a dormant piece of memory, it isn't running anything on its own ever (and yes, this is also true for DllMain). Sure, the loader will take care of all kinds of things such as relocations, but in the end nothing will run without being called explicitly or implicitly (in the context of some thread of the process loading the DLL). So for all practical purposes the question would require you to ask back.
Define "in Windows".
Also "dlls loaded in kernel mode or user mode", does this refer to the code doing the loading or to the end result (i.e. where the code runs or in what memory range it gets loaded)? Parts of that code run in user mode, others in kernel mode.
I wonder whether the interviewer has a clear idea of the concepts s/he is asking about.
Let me add some more information. It seems from the comments on the other answer that people have the same misconception that exists about DLLs also about drivers. Drivers are much closer to the idea of DLLs than to that of EXEs (or ultimately "processes"). The thing is that a driver doesn't do anything on its own most of the time (though it can create system threads to change that). Drivers are not processes and they do not create processes.
The answer is quite obviously User mode for anybody who does any kind of significant application development for windows. Let me explain two things.
DLL
A dynamic link library is closely similar to a regular old link library or .lib. When your application uses a .lib it pastes in function definitions just after compile time. You typically use a .lib to store API's and to modify the functions with out having to rebuild the whole project, just paste new .lib with same name over the old and as long as the interface(function name and parameters) hasn't changed it still works. Great modularity.
A .dll does exactly the same thing however it doesn't require re-linking or any compilation. You can think of a .dll as essentially a .lib which gets compiled to an .exe just the same as applications which use it. Simply put the new .dll which shares the name and function signatures and it all just works. You can update your application simply by replacing .dlls. This is why most windows software consists of .dlls and a few exe's.
The usage of a .dll is done in two ways
Implicit linking
To link this way if you had a .dll userapplication.dll you would have an userapplication.lib which defines all the entry points in the dll. You simply link to the static link library and then include the .dll in the working directory.
Explicit linking
Alernatively you can programmatically load the .dll by first calling LoadLibrary(userapplication.dll) which returns a handle to your .dll. Then GetProcAddress(handle, "FunctionInUserApplicationDll") which returns a function pointer you can use. This way your application can check stuff before attempting to use it. c# is a little different but easier.
USER/KERNEL MODES
Windows has two major modes of execution. User mode and Kernel modes (kernel further divided into system and sessions). For user mode the physical memory address is opaque. User mode makes use of virtual memory which is mapped to real memory spaces. User mode driver's are coincidentally also .dll's. A user mode application typically gets around 4Gb of virtual addressing space to work with. Two different applications can not meaningfully use those address because they are with in context of that application or process. There is no way for a user mode application to know it's physical memory address with out falling back to kernel mode driver. Basically everything your used to programming (unless you develop drivers).
Kernel mode is protected from user mode applications. Most hardware drivers work in the context of kernel mode and typically all windows api's are broken into two categories user and kernel. Kernel mode drivers use kernel mode api's and do not use user mode api's and hence don't user .dll's(You can't even print to a console cause that is a user mode api set). Instead they use .sys files which are drivers and essentially work exactly the same way in user mode. A .sys is an pe format so basically an .exe just like a .dll is like an .exe with out a main() entry point.
So from the askers perspective you have two groups
[kernel/.sys] and [user/.dll or .exe]
There really isn't .exe's in kernel because the operating system does everything not users. When system or another kernel component starts something they do it by calling DriverEntry() method so I guess that is like main().
So this question in this sense is quite simple.
My question is:
What is the flow of execution of an executable file in WINDOWS? (i.e. What happens when we start a application.)
How does the OS integrate with the application to operate or handle the application?
Does it have a central control place that checks execution of each file or process?
Is the process registered in the execution directory? (if any)
What actually happens that makes the file perform its desired task within WINDOWS environment?
All help is appreciated...
There's plenty going on beyond the stages already stated (loading the PE, enumerating the dlls it depends on, calling their entry points and calling the exe entry point).
Probably the earliest action is the OS and the processor collaborating to create a new address space infrastructure (I think essentially a dedicated TLB). The OS initializes a Process Environment Block, with process-wide data (e.g., Process ID and environment variables). It initializes a Thread Environment Block, with thread-specific data (thread id, SEH root handlers, etc). Once an appropriate address is selected for every dll and it is loaded there, re-addressing of exported functions happens by the windows loader. (very briefly - at compile time, the dll cannot know the address at which it will be loaded by every consumer exe. the actual call addresses of its functions is thus determined only at load time). There are initializations of memory pages shared between processes - for windows messages, for example - and I think some initialization of disk-paging structures. there's PLENTY more going on. The main windows component involved is indeed the windows loader, but the kernel and executive are involved. Finally, the exe entry point is called - it is by default BaseProcessStart.
Typically a lot of preparation happens still after that, above the OS level, depending on used frameworks (canonical ones being CRT for native code and the CLR for managed): the framework must initialize its own internal structures to be able to deliver services to the application - memory management, exception handling, I/O, you name it.
A great place to go for such in depth discussions is Windows Internals. You can also dig a bit deeper in forums like SO, but you have to be able to break it down into more focused bits. As phrased, this is really too much for an SO post.
This is high-level and misses many details:
The OS reads the PE Header to find the dlls that the exe depends on and the exe's entry point
The DllMain function in all linked dlls is called with the DLL_PROCESS_ATTACH message
The entry point of the exe is called with the appropriate arguments
There is no "execution directory" or other central control other than the kernel itself. However, there are APIs that allow you to enumerate and query the currently-running processes, such as EnumProcesses.
Your question isn't very clear, but I'll try to explain.
When you open an application, it is loaded into RAM from your disk.
The operating system jumps to the entry point of the application.
The OS provides all the calls required for showing a window, connecting with stuff and receiving user input. It also manages processing time, dividing it evenly between applications.