Counter argument to overwrite a DLL - windows

We want to hot reload a DLL. The first idea was to overwrite the .dll file then reload it from the application. However the file is write protected.
Is there a (documented) counter-argument to not remove this file protection and overwrite the file?
If no such counter-argument exist how do we bypass this protection?
EDIT: References found on internet point out mmap and the fact that the DLL may not have been loaded. In my case, the (only exported) method has been called so I can reasonably think that the DLL has been fully loaded.

In our situation we have a program that loaded and use a DLL.
An edit in the source code of this DLL is made, we need to recompile it and have it hot-reloaded by the program. The problem is the DLL is locked against write.
We have another program that watch for any modification on the source file (this is actually a custom script Compiler) and need to compile a DLL (by translating the source script into c++ code and invoking clang).
The solution we have for some month now (and is working pretty well), is to have our Script Compiler to check if the file is protected against write, if so it rename this dll with a temporary name and finally compile.
The Main program catch the change on this dll and is able to reload it.
Extra step At the startup of the compiler, we check if there is any temp dll and try to removed them. If the deletion fails it's probably because the DLL is in use, we ignore it.
Note: We need to move the .dll and .pdb to a temporary file. We delete the .exp, .ilk, and .lib just in case of.

Related

Linking to a DLL with a DEF file instead of a LIB file?

I have learned that you can:
Convert a .DLL file into a .DEF file, which includes its exports
(Edit: This doesn't work with many conventions)
Convert a .DEF file into a .LIB file, which you can use to link to the DLL
Why can't (most) linkers link to a DLL given only a .DEF file, instead of a .LIB file?
Ultimately, the answer here is 'because noone wanted it badly enough and it doesn't really help anything'.
The DEF file is an input file that creates an import lib for the DLL. And then, later, when the DLL is being consumed by another link, the importlib is itself an input. The importlib looks like something special on the outside, but when you look at the inside it's really just a slightly special lib with objects in it.
It totally would be possible to modify the linker to take a def file (or a DLL, for that matter) directly.
But the design centre of the linker is that it takes objects as inputs and outputs a PE executable. So taking a DEF or DLL as an input goes outside the design pattern.
Beyond that it'd be rather pointless - allowing the linker to take a DEF file or DLL as input would neither enable any important new scenarios, nor does leaving this feature out block anything. Converting a DEF file you have (even without the actual DLL) into a usable importlib is the work of a few moments (simply create fake empty function for each DEF entry and link that). So there's no reason to add the ability to link a DEF file directly.
Martyn
In terms of MSVC, .lib files are always static libraries. They get linked in as a compilation unit along with all your compiled .c/.cpp files, so all of the library's code is included in your final executable.
Some .lib files, however, (in particular most of the Windows system ones) merely contain stubs which tell the OS to load the desired DLL at loadtime and then the stubs route function calls to the DLL. But, those stubs are statically linked into your executable. Your program will then use DLLs (and gain all the advantages and disadvantages thereof), but since the named DLL functions it requires are happily located in the .lib (and thus actually located in the executable itself), your code doesn't have to know it's using a DLL (specifically using declspec(dllimport)).
A .def file is merely used as a sort of "settings" or "configuration" file during the creation of a .dll to specify what functions the file should export. It cannot be linked to, as it doesn't really describe anything that the linker understands.
You do not convert a dll to a DEF file. The DEF just indicates which dll functions will be accessible from the outside, exported.
From the docs:
A DLL file has a layout very similar to an .exe file, with one
important difference — a DLL file contains an exports table. The
exports table contains the name of every function that the DLL exports
to other executables. These functions are the entry points into the
DLL; only the functions in the exports table can be accessed by other
executables. Any other functions in the DLL are private to the DLL.
The exports table of a DLL can be viewed by using the DUMPBIN tool
with the /EXPORTS option.
You can export functions from a DLL using two methods:
Create a module definition (.def) file and use the .def file when
building the DLL. Use this approach if you want to export functions
from your DLL by ordinal rather than by name.
Use the keyword __declspec(dllexport) in the function's definition.
When exporting functions with either method, make sure to use the
__stdcall calling convention.
Use the provided link to learn more about exporting from your dll's.
I think you got down voted because your point is not really clear, at least not to me. Also check this. It explains how to choose the export method.
Mehrdad, this isn't always a question of how to LINK to a DLL, as I, personally have NEVER linked a DLL using a .DEF file. What I HAVE done is take someone else's DLL, and very painstakingly constructed a header file, or rather, function prototypes that I could use with
LoadLibrary() in C, Declare Function ... Lib "Foo.dll" Alias "OrdinalName" in VB, and
[DllImport()] in C#.
Of course, this is RARELY done, as if you are using a DLL for something, normally you have permission to do so, and the authors provide the .lib's, and the headers to go with the binary DLL file.
I've never done the exact techniques you speak of, by converting a .DEF info a .LIB, etc... But, I suppose it would be easy to take a lib, or the DLL itself and export .DEF from it. Now, THAT I actually HAVE done, in a project where the DLL code was built with a vbScript that took code from the main project, and created an API out of all the existing, compiled, and tested code. This level of complication was only done because I had no idea what functions were going to BE in the DLL, as the main project could change at any time, so a static .DEF file would have never worked. So, I had to build the DLL once, capture the dimpbin /exports, undecorate the functions, and then build the .DEF file, and re-link the DLL.
If you find yourself in that type of situation, perhaps you need to re-think your original designs, and fix the problem from there...
As for .LIB files, USUALLY you'd only NEED those for static linkage, but they are also used when the .H file is available, often making debugging a just a little nicer...

registration free COM without copying the dll to the same directory as the executable

How can I consume a registration free COM object without copying the dll to the same directory as the executable? Is this possible?
I would like a complete working sample or a detailed step-by-step guide for my exact scenario.
For example, I have the following components:
CppTestRunner.exe - vc++ console application
CppTests.dll - vc++ mfc dll
CsCOMServer.dll - c# COM visible dll
I do not want to modify the exe or its directory, like adding an external manifest.
CppTests.dll is in a different directory than the exe.
It is okay if CsCOMServer.dll is in the same directory as CppTests.dll.
CppTestRunner.exe calls LoadLibrary on CppTests.dll which works fine.
After embedding manifests and using the windows activation context apis, I was able to copy CsCOMServer.dll to the exe directory to verify that CppTests.dll can create an instance of CsCOMServer.dll without issues, but again, I am trying to avoid this step.
Currently, with CsCOMServer.dll in the same directory as CppTests.dll, and without CsCOMServer.dll in the exe directory, I am getting the following error when I call CoCreateInstance:
Error = 0x80070002 The system cannot find the file specified.
I have used sysinternals process monitor to track down the paths CppTestRunner.exe looks in, etc.
I notice that depending on the things I try to do, sometimes CppTestRunner.exe tries to read CsCOMServer.dll.config in the same directory as CppTests.dll, but no matter what I try to put in there, I was not able to get past the error.
Also depending on other things I try to do, CppTestRunner.exe tries to read CsCOMServer.dll in the same directory as CppTests.dll but for some reason, I still get the same error.
Anyone know of working code with this exact scenario?
As you can probably tell, this is part of some unit\integration tests.
As a simple workaround, I am currently copying CppTestRunner.exe to the same directory as CppTests.dll and running it from there, but would like to avoid this step.
Thank you.
.NET looks for managed registration-free COM first in the GAC, then in the executable's folder. It uses the activation context to read activation metadata, but not to determine the location of the files themselves.
You can save the DLL in a sub Folder, let say DLLFolder\YourDll.dll, and in the manifest file of yourDll, write the filename like
<file name = "DLLFolder\YourDll.dll">
But you need to keep the manifest file (of the DLL) in the location where the EXE is saved.

Are .OCA files necessary for program execution?

In the system32 directory I saw an .OCX file with a corresponding .OCA file.
I had thought .OCA files are used only by Visual Basic. Are they therefore unnecessary for program execution and could be removed?
If they are unnecessary, why would there be an .OCA in the system32 folder in the first place?
.OCA files serve as a cache of the extended type library information for its corresponding .OCX file.
If you delete an .OCA file for a control VB recognizes and uses, VB will recreate the .OCA file when you load a project requiring the control. The recreation process takes a little time but comes with no penalty otherwise.
Last reviewed: April 4, 1996
Article ID: Q149429
SUMMARY For every custom control file (.OCX) that Visual Basic uses,
there is an accompanying .OCA file of the same file name. For example,
GRAPH16.OCX has an accompanying .OCA file called GRAPH16.OCA. An .OCA
file is a binary file that functions as both an extended type library
file and a cache for the custom control file.
MORE INFORMATION
A type library is a file or component within another
file that contains OLE Automation standard descriptions of exposed
objects, properties, and methods. The actual working type library for
a custom control used in Visual Basic is a combination of the type
library of the control itself and the additional properties provided
by the framework that wrap the control.
Some of the properties of the control are provided by the framework
and some by the control itself. Programmatically, the properties from
the framework and the control all appear as properties of the control.
In order for these properties to appear, Visual Basic creates an
extended type library when the control is loaded into the toolbox.
Because the process of reading the control's type library and creating
the extended type library is time consuming, Visual Basic caches the
extended type library information into an OCA file.
If you delete the OCA file for a control Visual Basic recognized,
Visual Basic will re-create the .OCA file when you load a project
requiring the control. This re-creation process comes with a time
penalty.
(http://support.microsoft.com/kb/149429)
So, don't worry about including them when you deploy your application.
Sorry to resurrect a zombie thread, but I want to summarize to make sure I understand this.
An OCA file is only needed during compilation of a program and if it is missing, VB will create what it needs so all that is lost is time during compiling. Unless you have really sloppy programming and another homemade dll or ocx needs a particular oca.
If an oca is shipped with a product, it should be able to be safely deleted.
No, they are not necessary for it to execute but they are necessary for the program to run properly (if the program required the file in the first place).
They never need to be deployed with a finished program for it to run.
This could be a dumb attempt to answer, but you could simply rename the file and see if the application quits working. If so, those files are necessairy.
DNA-science is about the same:
Remove a gene and see what stopped "working". De facto, this gene is related to / necessairy for that body-part or whatever.
Re: No, they are not necessary for it to execute but they are necessary for the program to run properly (if the program required the file in the first place).
Since execute and run mean the same thing - what you are saying is that:
it's not required for it to run but it's required for it to run properly??
or alternately
it's not required for it to execute but it's required for it to execute properly??
I am suspecting that the *.OCA files are not required EXCEPT for compiling a program using VB and if they don't exist when you load a project then VB will create any missing OCA files automatically in order to speed up future load or compile operations in VB.
I just deleted EVERY OCA file off my system (I'm a brave man when I'm using a cloned virtual machine to do this).
I then rebooted and ran my software and it worked just fine without ANY OCA files on my entire system. Now it might take a bit longer to load/compile my programs next time as vb will regenerate the OCA files but I'm using an i7 with 16Gb of RAM and an SSD so who cares!

Difference between .dll and .exe?

I want to know the exact difference between the dll and exe file.
I don't know why everybody is answering this question in context of .NET. The question was a general one and didn't mention .NET anywhere.
Well, the major differences are:
EXE
An exe always runs in its own address space i.e., It is a separate process.
The purpose of an EXE is to launch a separate application of its own.
DLL
A dll always needs a host exe to run. i.e., it can never run in its own address space.
The purpose of a DLL is to have a collection of methods/classes which can be re-used from some other application.
DLL is Microsoft's implementation of a shared library.
The file format of DLL and exe is essentially the same. Windows recognizes the difference between DLL and EXE through PE Header in the file. For details of PE Header, You can have a look at this Article on MSDN
EXE:
It's a executable file
When loading an executable, no export is called, but only the module entry point.
When a system launches new executable, a new process is created
The entry thread is called in context of main thread of that process.
DLL:
It's a Dynamic Link Library
There are multiple exported symbols.
The system loads a DLL into the context of an existing process.
For More Details: http://www.c-sharpcorner.com/Interviews/Answer/Answers.aspxQuestionId=1431&MajorCategoryId=1&MinorCategoryId=1
http://wiki.answers.com/Q/What_is_the_difference_between_an_EXE_and_a_DLL
Reference: http://www.dotnetspider.com/forum/34260-What-difference-between-dll-exe.aspx
The difference is that an EXE has an entry point, a "main" method that will run on execution.
The code within a DLL needs to be called from another application.
There are a few more differences regarding the structure you could mention.
Both DLL and EXE share the same file structure - Portable Executable, or PE. To differentiate between the two, one can look in the Characteristics member of IMAGE_FILE_HEADER inside IMAGE_NT_HEADERS. For a DLL, it has the IMAGE_FILE_DLL (0x2000) flag turned on. For a EXE it's the IMAGE_FILE_EXECUTABLE_IMAGE (0x2) flag.
PE files consist of some headers and a number of sections. There's usually a section for code, a section for data, a section listing imported functions and a section for resources. Some sections may contain more than one thing. The header also describes a list of data directories that are located in the sections. Those data directories are what enables Windows to find what it needs in the PE. But one type of data directory that an EXE will never have (unless you're building a frankenstein EXE) is the export directory. This is where DLL files have a list of functions they export and can be used by other EXE or DLL files. On the other side, each DLL and EXE has an import directory where it lists the functions and DLL files it requires to run.
Also in the PE headers (IMAGE_OPTIONAL_HEADER) is the ImageBase member. It specifies the virtual address at which the PE assumes it will be loaded. If it is loaded at another address, some pointers could point to the wrong memory. As EXE files are amongst the first to be loaded into their new address space, the Windows loader can assure a constant load address and that's usually 0x00400000. That luxury doesn't exist for a DLL. Two DLL files loaded into the same process can request the same address. This is why a DLL has another data directory called Base Relocation Directory that usually resides in its own section - .reloc. This directory contains a list of places in the DLL that need to be rebased/patched so they'll point to the right memory. Most EXE files don't have this directory, but some old compilers do generate them.
You can read more on this topic # MSDN.
This answer was a little more detailed than I thought but read it through.
DLL:
In most cases, a DLL file is a library. There are a couple of types of libraries, dynamic and static - read about the difference. DLL stands for dynamic link library which tells us that it's a part of the program but not the whole thing. It's made of reusable software components (library) which you could use for more than a single program. Bear in mind that it's always possible to use the library source code in many applications using copy-paste, but the idea of a DLL/Static Library is that you could update the code of a library and at the same time update all the applications using it - without compiling.
For example:
Imagine you're creating a Windows GUI component like a Button. In most cases you'd want to re-use the code you've written because it's a complex but a common component - You want many applications to use it but you don't want to give them the source code You can't copy-paste the code for the button in every program, so you decide you want to create a DL-Library (DLL).
This "button" library is required by EXEcutables to run, and without it they will not run because they don't know how to create the button, only how to talk to it.
Likewise, a DLL cannot be executed - run, because it's only a part of the program but doesn't have the information required to create a "process".
EXE:
An executable is the program. It knows how to create a process and how to talk to the DLL. It needs the DLL to create a button, and without it the application doesn't run - ERROR.
hope this helps....
Both DLL and EXE are Portable Executable(PE) Formats
A Dynamic-link library (DLL) is a library and therefore can not be executed directly. If you try to run it you will get an error about a missing entry point. It needs an entry point (main function) to get executed, that entry point can be any application or exe. DLL binding occurs at run-time. That is why its called "Dynamic Link" library.
An Executable (EXE) is a program that can be executed. It has its own entry point. A flag inside the PE header indicates which type of file it is (irrelevant of file extension). The PE header has a field where the entry point for the program resides. In DLLs it isn't used (or at least not as an entry point).
There are many software available to check header information. The only difference causing both to work differently is the bit in header as shown in below diagram.
EXE file has only single main entry means it is isolated application, when a system launches exe, a new process is created while DLLs have many entry points so when application use it no new process started, DLL can be reused and versioned. DLL reduces storage space as different programs can use the same dll.
Dll v/s Exe
1)DLL file is a dynamic link library which can be used in exe files and
other dll files.
EXE file is a executable file which runs in a separate
process which is managed by OS.
2)DLLs are not directly executable . They are separate files containing functions that can be called by programs and other DLLs to perform computations and functions.
An EXE is a program that can be executed . Ex :Windows program
3)Reusability
DLL: They can be reused for some other application. As long as the coder knows the names and parameters of the functions and procedures in the DLL file .
EXE: Only for specific purpose .
4)A DLL would share the same process and memory space of the calling application while an
EXE creates its separate process and memory space.
5)Uses
DLL: You want many applications to use it but you don't want to give them the source code You can't copy-paste the code for the button in every program, so you decide you want to create a DL-Library (DLL).
EXE: When we work with project templates like Windows Forms Applications, Console Applications, WPF Applications and Windows Services they generate an exe assembly when compiled.
6)Similarities :
Both DLL and EXE are binary files have a complex nested structure defined by the Portable Executable format, and they are not intended to be editable by users.
Two things: the extension and the header flag stored in the file.
Both files are PE files. Both contain the exact same layout. A DLL is a library and therefore can not be executed. If you try to run it you'll get an error about a missing entry point. An EXE is a program that can be executed. It has an entry point. A flag inside the PE header indicates which file type it is (irrelevant of file extension). The PE header has a field where the entry point for the program resides. In DLLs it isn't used (or at least not as an entry point).
One minor difference is that in most cases DLLs have an export section where symbols are exported. EXEs should never have an export section since they aren't libraries but nothing prevents that from happening. The Win32 loader doesn't care either way.
Other than that they are identical. So, in summary, EXEs are executable programs while DLLs are libraries loaded into a process and contain some sort of useful functionality like security, database access or something.
The .exe is the program. The .dll is a library that a .exe (or another .dll) may call into.
What sakthivignesh says can be true in that one .exe can use another as if it were a library, and this is done (for example) with some COM components. In this case, the "slave" .exe is a separate program (strictly speaking, a separate process - perhaps running on a separate machine), but one that accepts and handles requests from other programs/components/whatever.
However, if you just pick a random .exe and .dll from a folder in your Program Files, odds are that COM isn't relevant - they are just a program and its dynamically-linked libraries.
Using Win32 APIs, a program can load and use a DLL using the LoadLibrary and GetProcAddress API functions, IIRC. There were similar functions in Win16.
COM is in many ways an evolution of the DLL idea, originally concieved as the basis for OLE2, whereas .NET is the descendant of COM. DLLs have been around since Windows 1, IIRC. They were originally a way of sharing binary code (particularly system APIs) between multiple running programs in order to minimise memory use.
An EXE is visible to the system as a regular Win32 executable. Its entry
point refers to a small loader which initializes the .NET runtime and tells
it to load and execute the assembly contained in the EXE.
A DLL is visible to the system as a Win32 DLL but most likely without any
entry points. The .NET runtime stores information about the contained
assembly in its own header.
dll is a collection of reusable
functions where as an .exe is an
executable which may call these
functions
An exe is an executible program whereas A DLL is a file that can be loaded and executed by programs dynamically.
● .exe and dll are the compiled version of c# code which are also called as
assemblies.
● .exe is a stand alone executable file, which means it can executed directly.
● .dll is a reusable component which cannot be executed directly and it requires
other programs to execute it.
For those looking a concise answer,
If an assembly is compiled as a class library and provides types for other assemblies to use, then it has the ifle extension .dll (dynamic link library), and it cannot be executed standalone.
Likewise, if an assembly is compiled as an application, then it has the file extension .exe (executable) and can be executed standalone. Before .NET Core 3.0, console apps were compiled to .dll fles and had to be executed by the dotnet run command or a host executable. - Source
Difference in DLL and EXE:
1) DLL is an In-Process Component which means running in the same memory space as the client process. EXE is an Out-Process Component which means it runs in its own separate memory space.
2) The DLL contains functions and procedures that other programs can use (promotes reuability) while EXE cannot be shared with other programs.
3) DLL cannot be directly executed as they're designed to be loaded and run by other programs. EXE is a program that is executed directly.
The major exact difference between DLL and EXE that DLL hasn't got an entry point and EXE does. If you are familiar with c++ you can see that build EXE has main() entry function and DLL doesn't :)

Can the DllMain of an .exe be called?

My question is not exactly the same as this one (it's not theoretical, there is only a main thread without message loop, InitInstance and ExitInstance are no fitting calls).
I'm using a console app without message loop; this app loads an exe with the LoadLibrary function, so that it can use its exported functions. Bad news: the DllMain function of the exe is not called (and I verified the symbols tables, using a def file, DllMain appears correctly); the doc says it's called if the loaded module is a DLL (too bad).
What are the conditions (if they exist) which could lead to the execution of the exe's DllMain function when LoadLibrary is called (and maybe again when FreeLibrary is called)?
Best regards
The most obvious condition is that the process calling LoadLibrary() explicitly gets GetProcAddress("DllMain") and then calls it.
The conditions are:
1) Binary being loaded was compiled as DLL (when using gcc/ld it means using --shared option; If you use --shared, resulting file will be a dll, and will not run, see below)
2) IMAGE_FILE_DLL is set in PE file header of the binary file being loaded. If it is set, file is a dll, and Windows linker will call its DllMain() function for you when it links this file to your program (doesn't matter how it's linked - LoadLibrary() at runtime or -llibraryname at compile time). For that the file must also satisfy (1). But with this flag the binary file being loaded will not be runnable. If IMAGE_FILE_DLL is not set, DllMain() will not be called when file is loaded into your program.
Compiling dll with --shared and then manually removing IMAGE_FILE_DLL from its header (i.e. by using hex editor) will not work - when you run it, only DllMain() will be executed, and fdwReason will be a non-defined number (0x28ffd4 on my machine).
Update
All DLL and EXE files on Windows are PE files, the difference is how they are linked, and which flags are set in their headers. That is why i write file being loaded, not dll being loaded.
The last paragraph also describes the scenario where you compile the file as dll, and then turn it into exe by messing with its header. It doesn't work.
Naming has nothing to do with it (you can choose any name, and with some pexports+dlltool tinkering you can create an import library for an .exe file and be able to link it as -lexenamewithoutextension
To clarify:
if you compile it without --shared:
IMAGE_FILE_DLL will not be set in it, it will be runnable, but DllMain() will NOT be called when you link it.
if you compile it with --shared:
IMAGE_FILE_DLL will be set in it, it will NOT be runnable, but DllMain() will be called when you link it.
if you compile it without --shared, then switch on the IMAGE_FILE_DLL flag in it manually:
it will NOT be runnable anymore, and no DllMain() will be called when you link it.
if you compile it with --shared, then switch off the IMAGE_FILE_DLL flag in it manually:
it will be runnable, but DllMain() will be executed instead of main(), and DllMain() will NOT be called when you link it.
Indeed the Name "DllMain" of the function is ignored by Windows completely (the old Windows API documentation of Windows NT 3.x stated this explicitly).
When a DLL is loaded not the function DllMain() but the function that is located at the file's entry point is called.
Of course the linker will create the DLL file in a way that DllMain() is this function.
However for EXE files the entry function (that will call WinMain()) is located at the entry point.
So it is obvious that Windows cannot call this function when loading the EXE file as DLL.
Completing the good answer of MSalters:
So, then, call the "fake" DllMain with DLL_XXX_ATTACH just after LoadLibrary and with DLL_XXX_DETACH just before FreeLibrary, and make manually the other calls.
Another following implementation would be to build and load an interface DLL which could callback automatically the EXE on its fake DllMain (I don't know if it could work); but it may be more complicated than just manually calling the fake DllMain in a number of cases. (can't LoadLibrary in a DllMain)

Resources