Background
I'm maintaining a plugin for an application. I'm Using Visual C++ 2003.
The plugin is composed of several DLLs - there's the main DLL, that's the one that the application loads using LoadLibrary, and there are several utility DLLs that are used by the main DLL and by each other.
Dependencies generally look like this:
plugin.dll -> utilA.dll, utilB.dll
utilA.dll -> utilB.dll
utilB.dll -> utilA.dll, utilC.dll
You get the picture.
Some of the dependencies between the DLLs are load-time and some run-time.
All the DLL files are stored in the executable's directory (not a requirement, just how it works now).
The problem
There's a new requirement - running multiple instances of the plugin within the application.
The application runs each instance of a plugin in its own thread, i.e. each thread calls functions exported by plugin.dll. The plugin's code, however, is anything but thread-safe - lots of global variables etc..
Unfortunately, fixing the whole thing isn't currently an option, so I need a way to load multiple (at most 3) copies of the plugin's DLLs in the same process.
Option 1: The distinct names approach
Creating 3 copies of each DLL file, so that each file has a distinct name. e.g. plugin1.dll, plugin2.dll, plugin3.dll, utilA1.dll, utilA2.dll, utilA3.dll, utilB1.dll, etc.. The application will load plugin1.dll, plugin2.dll and plugin3.dll. The files will be in the executable's directory.
For each group of DLLs to know each other by name (so the inter-dependencies work), the names need to be known at compilation time - meaning the DLLs need to be compiled multiple times, only each time with different output file names.
Not very complicated, but I'd hate having 3 copies of the VS project files, and don't like having to compile the same files over and over.
Option 2: The side-by-side assemblies approach
Creating 3 copies of the DLL files, each group in its own directory, and defining each group as an assembly by putting an assembly manifest file in the directory, listing the plugin's DLLs.
Each DLL will have an application manifest pointing to the assembly, so that the loader finds the copies of the utility DLLs that reside in the same directory. The manifest needs to be embedded for it to be found when a DLL is loaded using LoadLibrary. I'll use mt.exe from a later VS version for the job, since VS2003 has no built-in manifest embedding support.
I've tried this approach with partial success - dependencies are found during load-time of the DLLs, but not when a DLL function is called that loads another DLL.
This seems to be the expected behavior according to this article - A DLL's activation context is only used at the DLL's load-time, and afterwards it's deactivated and the process's activation context is used.
Edit: Works with ISOLATION_AWARE_ENABLED as expected - runtime loading of DLLs uses the original activation context of the loading DLL.
Questions
Got any other options? Any quick & dirty solution will do. :-)
Will ISOLATION_AWARE_ENABLED even work with VS2003? Edit: It does.
Comments will be greatly appreciated.
Thanks!
ISOLATION_AWARE_ENABLED is implemented by the Windows SDK header files and thus probably wont worth with VS2003 at all. However, it is possible to download the latest Windows 7 SDK and use that with VS2003.
You don't need to use MT to link in manifests. Manifests can be embedded as resources in environments that dont have explicit knowledge.
Add the following to a dll's .rc file to embed a manifest. (With a recent enough platform sdk RT_MANIFEST should already be defined):
#define RT_MANIFEST 24
#define APP_MANIFEST 1
#define DLL_MANIFEST 2
DLL_MANIFEST RT_MANIFEST dllName.dll.embed.manifest
Related
I'm maintaining a VB6 application with many COM components (DLLs and OCXs). In order to streamline development and deployment I'd like to use reg-free com. The problem with development is that the application runs within the VB6.EXE instance. How can I trick VB6 to use my (unregistered) components? It is very important for me to not have to go through registering/unregistering components when switching between branches. Generating a .manifest file for VB6 is not out of the question but is there some other, more optimal way, to specify a .manifest file when launching VB6.EXE?
Note: The Activation Context API doesn't seem to help, even if used from within the development environment.
Solutions I've thought:
A utility application that activates a context from a manifest and launches VB6 as a child process (doesn't work; processes don't inherit activation context)
Injecting context activation into the VB6 process at startup (too complicated; must hack the executable to do this)
Hosting VB6 in my own process after activating the right context (can't even find out if this is possible)
Using a VB6 Add-In or other utility that runs within VB6 to activate a context (tried that but it doesn't seem to work)
Update Jan. 16
As suggested by wqw, I did some testing with a VB.exe.manifest. The VB6.exe.manifest worked, with some caveats:
The SxS dll specified in the manifest would not appear in the references window on projects that didn't actually reference the component
On projects that did reference the component it would be shown to reside in the directory according to the following order:
The pathname recorded in the project file (if the file was still present)
A pathname as if it resided in the same folder as the project (vbp)
If the file was not in any of these folders, the project would not compile (just running the code causes an internal compile in VB6) with the message "Can't find project or library".
Obviously, VB6 actualy scans the registry to find COM components and verifies, during compilation, that they exist where they say they exist. I'm not sure what that might mean if I actually want to use VB6.exe.manifest to redirect COM component instantiation. Perhaps having dummy component files at some predefined location might trick VB6 into believing that everything is as it should be, although an entirely different set of components got loaded for use.
Further update:
I did a test on that last assumption and it proved to be false. The component has to actually be there in order for the project to compile. It must even properly load (no dummy, zero-length files accepted!). Now I'm not even sure if the manifest works. That's a more time-consuming test (requires a component with two versions that produce different results, one with the project, and one for the manifest).
Our approach to this problem was to write a build assist program that registered and unregistered components, run the VB6 compiler, and would even rewrite project files with updated GUIDs when interfaces changed. You would hand it a VBG project group and it would do the rest.
I suppose we could also have added a mode that unregistered components when you switched branches.
Are you following the practice of using "compatibility" binaries? You shouldn't use the binary at your build location for compatibility references - you should commit a separate copy to version control and configure your project to consider that the "compatible" version - only change this file when you break interfaces.
I need to build a DLL capable to load other DLLs at runtime; these other DLLs have a rather intimate relationship with the main DLL (this is Python and extensions), so they must have a common runtime. The main DLL must be a single file that can be simply copied on the target machine. The auxiliary DLLs will be placed in a different directory.
So: "common runtime" means no static linking; "single file + simple copy" rules out the shared MS redistributables, especially when coupled with "different directories".
I only see the following options: link all DLLs against msvcrt.dll; embed current msvcrtXX into the main DLL and re-export all its symbols; use the msvcrtXX of the host application. To me the first looks the simplest, because it's a common need and there are many web pages explaining how to go about it. How would you approach this?
I suggest you re-architecture so that your plug-in DLL (the one the host application loads explicitly) contains nothing but proxy functions plus the file management logic. This DLL could use a static runtime, or no runtime at all.
Put the actual functionality (and in particular all code that needs to share a runtime with the Python extensions) in a separate ("primary") DLL and put this DLL, as well as the runtime of your choice, in the same directory as the Python extensions.
The primary DLL and the runtime could be zipped to the plug-in DLL, along with the core extensions. (I presume this is within the scope of the runtime's redistributable license, since it's basically the same as the way most installers work, but you should check for yourself.)
The calls to the plug-in will be slightly less efficient since they have to go through the proxy DLL, but the performance difference probably won't be measurable. :-)
Another option (in my humble opinion, much better) would be to dynamically link - aka. LoadLibrary/GetProcAddress) the dependencies when the main starts. This would allow you to support cases of missing DLLs and/or support your own internal version scheme.
I want to know the exact difference between the dll and exe file.
I don't know why everybody is answering this question in context of .NET. The question was a general one and didn't mention .NET anywhere.
Well, the major differences are:
EXE
An exe always runs in its own address space i.e., It is a separate process.
The purpose of an EXE is to launch a separate application of its own.
DLL
A dll always needs a host exe to run. i.e., it can never run in its own address space.
The purpose of a DLL is to have a collection of methods/classes which can be re-used from some other application.
DLL is Microsoft's implementation of a shared library.
The file format of DLL and exe is essentially the same. Windows recognizes the difference between DLL and EXE through PE Header in the file. For details of PE Header, You can have a look at this Article on MSDN
EXE:
It's a executable file
When loading an executable, no export is called, but only the module entry point.
When a system launches new executable, a new process is created
The entry thread is called in context of main thread of that process.
DLL:
It's a Dynamic Link Library
There are multiple exported symbols.
The system loads a DLL into the context of an existing process.
For More Details: http://www.c-sharpcorner.com/Interviews/Answer/Answers.aspxQuestionId=1431&MajorCategoryId=1&MinorCategoryId=1
http://wiki.answers.com/Q/What_is_the_difference_between_an_EXE_and_a_DLL
Reference: http://www.dotnetspider.com/forum/34260-What-difference-between-dll-exe.aspx
The difference is that an EXE has an entry point, a "main" method that will run on execution.
The code within a DLL needs to be called from another application.
There are a few more differences regarding the structure you could mention.
Both DLL and EXE share the same file structure - Portable Executable, or PE. To differentiate between the two, one can look in the Characteristics member of IMAGE_FILE_HEADER inside IMAGE_NT_HEADERS. For a DLL, it has the IMAGE_FILE_DLL (0x2000) flag turned on. For a EXE it's the IMAGE_FILE_EXECUTABLE_IMAGE (0x2) flag.
PE files consist of some headers and a number of sections. There's usually a section for code, a section for data, a section listing imported functions and a section for resources. Some sections may contain more than one thing. The header also describes a list of data directories that are located in the sections. Those data directories are what enables Windows to find what it needs in the PE. But one type of data directory that an EXE will never have (unless you're building a frankenstein EXE) is the export directory. This is where DLL files have a list of functions they export and can be used by other EXE or DLL files. On the other side, each DLL and EXE has an import directory where it lists the functions and DLL files it requires to run.
Also in the PE headers (IMAGE_OPTIONAL_HEADER) is the ImageBase member. It specifies the virtual address at which the PE assumes it will be loaded. If it is loaded at another address, some pointers could point to the wrong memory. As EXE files are amongst the first to be loaded into their new address space, the Windows loader can assure a constant load address and that's usually 0x00400000. That luxury doesn't exist for a DLL. Two DLL files loaded into the same process can request the same address. This is why a DLL has another data directory called Base Relocation Directory that usually resides in its own section - .reloc. This directory contains a list of places in the DLL that need to be rebased/patched so they'll point to the right memory. Most EXE files don't have this directory, but some old compilers do generate them.
You can read more on this topic # MSDN.
This answer was a little more detailed than I thought but read it through.
DLL:
In most cases, a DLL file is a library. There are a couple of types of libraries, dynamic and static - read about the difference. DLL stands for dynamic link library which tells us that it's a part of the program but not the whole thing. It's made of reusable software components (library) which you could use for more than a single program. Bear in mind that it's always possible to use the library source code in many applications using copy-paste, but the idea of a DLL/Static Library is that you could update the code of a library and at the same time update all the applications using it - without compiling.
For example:
Imagine you're creating a Windows GUI component like a Button. In most cases you'd want to re-use the code you've written because it's a complex but a common component - You want many applications to use it but you don't want to give them the source code You can't copy-paste the code for the button in every program, so you decide you want to create a DL-Library (DLL).
This "button" library is required by EXEcutables to run, and without it they will not run because they don't know how to create the button, only how to talk to it.
Likewise, a DLL cannot be executed - run, because it's only a part of the program but doesn't have the information required to create a "process".
EXE:
An executable is the program. It knows how to create a process and how to talk to the DLL. It needs the DLL to create a button, and without it the application doesn't run - ERROR.
hope this helps....
Both DLL and EXE are Portable Executable(PE) Formats
A Dynamic-link library (DLL) is a library and therefore can not be executed directly. If you try to run it you will get an error about a missing entry point. It needs an entry point (main function) to get executed, that entry point can be any application or exe. DLL binding occurs at run-time. That is why its called "Dynamic Link" library.
An Executable (EXE) is a program that can be executed. It has its own entry point. A flag inside the PE header indicates which type of file it is (irrelevant of file extension). The PE header has a field where the entry point for the program resides. In DLLs it isn't used (or at least not as an entry point).
There are many software available to check header information. The only difference causing both to work differently is the bit in header as shown in below diagram.
EXE file has only single main entry means it is isolated application, when a system launches exe, a new process is created while DLLs have many entry points so when application use it no new process started, DLL can be reused and versioned. DLL reduces storage space as different programs can use the same dll.
Dll v/s Exe
1)DLL file is a dynamic link library which can be used in exe files and
other dll files.
EXE file is a executable file which runs in a separate
process which is managed by OS.
2)DLLs are not directly executable . They are separate files containing functions that can be called by programs and other DLLs to perform computations and functions.
An EXE is a program that can be executed . Ex :Windows program
3)Reusability
DLL: They can be reused for some other application. As long as the coder knows the names and parameters of the functions and procedures in the DLL file .
EXE: Only for specific purpose .
4)A DLL would share the same process and memory space of the calling application while an
EXE creates its separate process and memory space.
5)Uses
DLL: You want many applications to use it but you don't want to give them the source code You can't copy-paste the code for the button in every program, so you decide you want to create a DL-Library (DLL).
EXE: When we work with project templates like Windows Forms Applications, Console Applications, WPF Applications and Windows Services they generate an exe assembly when compiled.
6)Similarities :
Both DLL and EXE are binary files have a complex nested structure defined by the Portable Executable format, and they are not intended to be editable by users.
Two things: the extension and the header flag stored in the file.
Both files are PE files. Both contain the exact same layout. A DLL is a library and therefore can not be executed. If you try to run it you'll get an error about a missing entry point. An EXE is a program that can be executed. It has an entry point. A flag inside the PE header indicates which file type it is (irrelevant of file extension). The PE header has a field where the entry point for the program resides. In DLLs it isn't used (or at least not as an entry point).
One minor difference is that in most cases DLLs have an export section where symbols are exported. EXEs should never have an export section since they aren't libraries but nothing prevents that from happening. The Win32 loader doesn't care either way.
Other than that they are identical. So, in summary, EXEs are executable programs while DLLs are libraries loaded into a process and contain some sort of useful functionality like security, database access or something.
The .exe is the program. The .dll is a library that a .exe (or another .dll) may call into.
What sakthivignesh says can be true in that one .exe can use another as if it were a library, and this is done (for example) with some COM components. In this case, the "slave" .exe is a separate program (strictly speaking, a separate process - perhaps running on a separate machine), but one that accepts and handles requests from other programs/components/whatever.
However, if you just pick a random .exe and .dll from a folder in your Program Files, odds are that COM isn't relevant - they are just a program and its dynamically-linked libraries.
Using Win32 APIs, a program can load and use a DLL using the LoadLibrary and GetProcAddress API functions, IIRC. There were similar functions in Win16.
COM is in many ways an evolution of the DLL idea, originally concieved as the basis for OLE2, whereas .NET is the descendant of COM. DLLs have been around since Windows 1, IIRC. They were originally a way of sharing binary code (particularly system APIs) between multiple running programs in order to minimise memory use.
An EXE is visible to the system as a regular Win32 executable. Its entry
point refers to a small loader which initializes the .NET runtime and tells
it to load and execute the assembly contained in the EXE.
A DLL is visible to the system as a Win32 DLL but most likely without any
entry points. The .NET runtime stores information about the contained
assembly in its own header.
dll is a collection of reusable
functions where as an .exe is an
executable which may call these
functions
An exe is an executible program whereas A DLL is a file that can be loaded and executed by programs dynamically.
● .exe and dll are the compiled version of c# code which are also called as
assemblies.
● .exe is a stand alone executable file, which means it can executed directly.
● .dll is a reusable component which cannot be executed directly and it requires
other programs to execute it.
For those looking a concise answer,
If an assembly is compiled as a class library and provides types for other assemblies to use, then it has the ifle extension .dll (dynamic link library), and it cannot be executed standalone.
Likewise, if an assembly is compiled as an application, then it has the file extension .exe (executable) and can be executed standalone. Before .NET Core 3.0, console apps were compiled to .dll fles and had to be executed by the dotnet run command or a host executable. - Source
Difference in DLL and EXE:
1) DLL is an In-Process Component which means running in the same memory space as the client process. EXE is an Out-Process Component which means it runs in its own separate memory space.
2) The DLL contains functions and procedures that other programs can use (promotes reuability) while EXE cannot be shared with other programs.
3) DLL cannot be directly executed as they're designed to be loaded and run by other programs. EXE is a program that is executed directly.
The major exact difference between DLL and EXE that DLL hasn't got an entry point and EXE does. If you are familiar with c++ you can see that build EXE has main() entry function and DLL doesn't :)
Background
My solution consists of two projects:
A standard Windows application
A DLL which my application does not use directly, but instead injects it into a target process
Basically, from my application's perspective, the only requirement that the DLL must meet is to be present in the working directory of my application. In short, my DLL doesn't export any functions that are of interest to my application.
Question
I would like to strongly couple these two binaries. What are my options aside from manually calling LoadLibrary in my application?
If this is too vague, please let me know.
Edit
Since no one seems to be "answering", I ended up doing what EFraim suggested (see comments).
I exported a dummy function from my DLL and added the DLL's generated *.lib file as an additional dependency in my application's linker property sheet. Now, at run-time, if the DLL's missing, Windows gives a nice error message and terminates execution. As an added bonus, the DLL image is also locked after successful IAT initialization; this prevents things like user deletion.
If you don't expect releasing dll and exe separately, you can add dll into your resourses and unpack it from there on startup.
How exactly do DLL files work? There seems to be an awful lot of them, but I don't know what they are or how they work.
So, what's the deal with them?
What is a DLL?
Dynamic Link Libraries (DLL)s are like EXEs but they are not directly executable. They are similar to .so files in Linux/Unix. That is to say, DLLs are MS's implementation of shared libraries.
DLLs are so much like an EXE that the file format itself is the same. Both EXE and DLLs are based on the Portable Executable (PE) file format. DLLs can also contain COM components and .NET libraries.
What does a DLL contain?
A DLL contains functions, classes, variables, UIs and resources (such as icons, images, files, ...) that an EXE, or other DLL uses.
Types of libraries:
On virtually all operating systems, there are 2 types of libraries. Static libraries and dynamic libraries. In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.
More on static and dynamic libraries:
You don't normally see static libraries though on your computer, because a static library is embedded directly inside of a module (EXE or DLL). A dynamic library is a stand-alone file.
A DLL can be changed at any time and is only loaded at runtime when an EXE explicitly loads the DLL. A static library cannot be changed once it is compiled within the EXE.
A DLL can be updated individually without updating the EXE itself.
Loading a DLL:
A program loads a DLL at startup, via the Win32 API LoadLibrary, or when it is a dependency of another DLL. A program uses the GetProcAddress to load a function or LoadResource to load a resource.
Further reading:
Please check MSDN or Wikipedia for further reading. Also the sources of this answer.
What is a DLL?
DLL files are binary files that can contain executable code and resources like images, etc. Unlike applications, these cannot be directly executed, but an application will load them as and when they are required (or all at once during startup).
Are they important?
Most applications will load the DLL files they require at startup. If any of these are not found the system will not be able to start the process at all.
DLL files might require other DLL files
In the same way that an application requires a DLL file, a DLL file might be dependent on other DLL files itself. If one of these DLL files in the chain of dependency is not found, the application will not load. This is debugged easily using any dependency walker tools, like Dependency Walker.
There are so many of them in the system folders
Most of the system functionality is exposed to a user program in the form of DLL files as they are a standard form of sharing code / resources. Each functionality is kept separately in different DLL files so that only the required DLL files will be loaded and thus reduce the memory constraints on the system.
Installed applications also use DLL files
DLL files also becomes a form of separating functionalities physically as explained above. Good applications also try to not load the DLL files until they are absolutely required, which reduces the memory requirements. This too causes applications to ship with a lot of DLL files.
DLL Hell
However, at times system upgrades often breaks other programs when there is a version mismatch between the shared DLL files and the program that requires them. System checkpoints and DLL cache, etc. have been the initiatives from M$ to solve this problem. The .NET platform might not face this issue at all.
How do we know what's inside a DLL file?
You have to use an external tool like DUMPBIN or Dependency Walker which will not only show what publicly visible functions (known as exports) are contained inside the DLL files and also what other DLL files it requires and which exports from those DLL files this DLL file is dependent upon.
How do we create / use them?
Refer the programming documentation from your vendor. For C++, refer to LoadLibrary in MSDN.
Let’s say you are making an executable that uses some functions found in a library.
If the library you are using is static, the linker will copy the object code for these functions directly from the library and insert them into the executable.
Now if this executable is run it has every thing it needs, so the executable loader just loads it into memory and runs it.
If the library is dynamic the linker will not insert object code but rather it will insert a stub which basically says this function is located in this DLL at this location.
Now if this executable is run, bits of the executable are missing (i.e the stubs) so the loader goes through the executable fixing up the missing stubs. Only after all the stubs have been resolved will the executable be allowed to run.
To see this in action delete or rename the DLL and watch how the loader will report a missing DLL error when you try to run the executable.
Hence the name Dynamic Link Library, parts of the linking process is being done dynamically at run time by the executable loader.
One a final note, if you don't link to the DLL then no stubs will be inserted by the linker, but Windows still provides the GetProcAddress API that allows you to load an execute the DLL function entry point long after the executable has started.
DLLs (dynamic link libraries) and SLs (shared libraries, equivalent under UNIX) are just libraries of executable code which can be dynamically linked into an executable at load time.
Static libraries are inserted into an executable at compile time and are fixed from that point. They increase the size of the executable and cannot be shared.
Dynamic libraries have the following advantages:
1/ They are loaded at run time rather than compile time so they can be updated independently of the executable (all those fancy windows and dialog boxes you see in Windows come from DLLs so the look-and-feel of your application can change without you having to rewrite it).
2/ Because they're independent, the code can be shared across multiple executables - this saves memory since, if you're running 100 apps with a single DLL, there may only be one copy of the DLL in memory.
Their main disadvantage is advantage #1 - having DLLs change independent your application may cause your application to stop working or start behaving in a bizarre manner. DLL versioning tend not to be managed very well under Windows and this leads to the quaintly-named "DLL Hell".
DLL files contain an Export Table which is a list of symbols which can be looked up by the calling program. The symbols are typically functions with the C calling convention (__stcall). The export table also contains the address of the function.
With this information, the calling program can then call the functions within the DLL even though it did not have access to the DLL at compile time.
Introducing Dynamic Link Libraries has some more information.
http://support.microsoft.com/kb/815065
A DLL is a library that contains code
and data that can be used by more than
one program at the same time. For
example, in Windows operating systems,
the Comdlg32 DLL performs common
dialog box related functions.
Therefore, each program can use the
functionality that is contained in
this DLL to implement an Open dialog
box. This helps promote code reuse and
efficient memory usage.
By using a DLL, a program can be
modularized into separate components.
For example, an accounting program may
be sold by module. Each module can be
loaded into the main program at run
time if that module is installed.
Because the modules are separate, the
load time of the program is faster,
and a module is only loaded when that
functionality is requested.
Additionally, updates are easier to
apply to each module without affecting
other parts of the program. For
example, you may have a payroll
program, and the tax rates change each
year. When these changes are isolated
to a DLL, you can apply an update
without needing to build or install
the whole program again.
http://en.wikipedia.org/wiki/Dynamic-link_library
DLL is a File Extension & Known As “dynamic link library” file format used for holding multiple codes and procedures for Windows programs. Software & Games runs on the bases of DLL Files; DLL files was created so that multiple applications could use their information at the same time.
IF you want to get more information about DLL Files or facing any error read the following post.
https://www.bouncegeek.com/fix-dll-errors-windows-586985/
DLLs (Dynamic Link Libraries) contain resources used by one or more applications or services. They can contain classes, icons, strings, objects, interfaces, and pretty much anything a developer would need to store except a UI.
According to Microsoft
(DLL) Dynamic link libraries are files that contain data, code, or resources needed for the running of applications. These are files that are created by the windows ecosystem and can be shared between two or more applications.
When a program or software runs on Windows, much of how the application works depends on the DLL files of the program. For instance, if a particular application had several modules, then how each module interacts with each other is determined by the Windows DLL files.
If you want detailed explanation, check these useful resources
What are dll files , About Dll files