I'm building a windows application that uses a third-party DLL. In order to bundle everything into one executable, I have embedded said DLL as a go-bindata asset into my binary.
Currently I drop the embedded DLL to the local file system when the application runs for the first time and then load it with syscall.NewLazyDLL(). That works fine so far!
I was wondering, though, if anyone knows a solution that would allow me to load the DLL directly from the in-memory byte-slice that the go-bindata asset provides, without first dropping the DLL as a file to the local filesystem.
I am currently looking at Go's runtime.syscall_loadlibrary() method in syscall_windows.go that syscall.NewLazyDLL calls at some point but don't really have an idea where to go from here...
I found this question and this github repository that seem to deal with a similar issue, but all related to C/C++/C#.
Thoughts?
Related
I am beginer in windows programming, and trying to understand use of DLL and lib in an existing project implementation. From my understanding LIB and dll are used to reuse the validated code and reduce the size of exe, by loading them on need basis. Although there are quite a few articles on internet explained about this, but I am looking for any example with diagram(How is the flow from Application programme to Lib/dll and vice versa). I am getting lots of error related to inappropriate setting of lib in resource and C/C++ configuration in my project, but due to my limited understanding of the calling between app-lib-dll i am unable to fix them.
Looking forward for some quick refrence documents/links/video to understand these concepts.
Lib files and DLL files provide ways to include code in your process. For a lib file this inclusion is done by the linker to add code to your exe. For a DLL file the inclusion is done by the operating system when your exe is loaded into memory - before the program starts. So by the time your program starts all the code is in your process memory and calling a lib function or calling a DLL function is done the same way, or almost the same way, that calls to your own functions are done. So the "flow" that you are asking about is not significantly different for a lib or DLL. A call is a call, no matter how the called function got into memory.
Okay... here is the deal (and I wish I could provide a code example, but this is a problem at the library level and not limited to a specific function/file):
I am creating a common logging dll to be used throughout all applications in my organization. As part of the logger, we are using a logging framework names 'NLog.' The .dll project has several projects and is merged into one .dll using 'ILMerge.' I, of course, compile it on my machine and make the .dll and relevant configurations available to the other application(s) that will be using it. When the .dll is used in another application, it runs just fine on my machine and on almost every other machine that is being used in my organization. However, on one of the other developer's machines, the .dll does not work correctly. The only way that I have been able to get it to work correctly on his machine is to actually re-compile the .dll on his machine.
Has anyone else encountered this problem? I have written like a billion .dll's and have NEVER encountered this issue.
Thank you for helping! If you need more information, let me know.
Brett
Are all dependent libraries having exactly the same versions on your machine and on your colleagues machine? Sounds like a version conflict.
I have a VS2010 C# project, that references a large set of native .dll's (a commercial java runtime). These file are referenced as 'Content' files in the project, since the need to be copied with the project.
The code in these libraries is called using PInvoke, there is no assembly reference.
Every time I compile the solution, the Visual Studio testing framework tries to load all the referenced dll files, expecting to find .net assemblies which may contain unit tests. Since the are no .net assemblies, the following exception is thrown:
Error loading some.dll: Unable to load the test container 'e:\some.dll' or one of its dependencies. If you build your test project assembly as a 64 bit assembly, it cannot be loaded. When you build your test project assembly, select "Any CPU" for the platform. To run your tests in 64 bit mode on a 64 bit processor, you must change your test settings in the Hosts tab to run your tests in a 32 bit process. Error details: Could not load file or assembly 'file:///e:\some.dll' or one of its dependencies. The module was expected to contain an assembly manifest.
This takes a whole lot of time, and I would like to tell Visual Studio to not try to load these files.
How can I tell Visual Studio to stop trying to load these files?
Correct me if I got this wrong:
You are including the P/Invoke target binaries in to the VS solution because you want the binaries to be copied over to the target directory when the solution is built. You want this because the project will execute from the target directory as soon as the VS solution is built. Correct?
Often times VS packages (both default and 3rd party) try to get smart about the solution content and will follow certain triggers (which are difficult to contain and control by ourselves) and load the solution and project content in their own ways. Fighting the battle in this area has poor ROI than employing a simpler work around (below).
While I can't provide you with an authoritative answer on how to tell VS's test package to not load all binaries, I suggest removing such binaries from the project as 'content' and leave them in your source control where they are today. Add a post-build task that will copy the said binaries over to the target. This will still give you the same result as it is working today but, takes those binaries out of reach for the test probes.
You must check out configuration settings by just right clicking on your solution name and click on "Configuration Manager"
It will open a pop up window for Configuration Manager.
Check not for the platform your projects are using it is better to choose any CPU.
Hope this can help.Give it a try:)
Because thats what your exeception says as you have quoted
Thanks
I tried to repro this issue and found that the root cause is that you have set your test project to be compiled as !AnyCpu. Is there any particular reason why you would want this for managed test code?
So unless you change this you will continue to see this message.
If you want to continue using this configuration for your test project you would need to update your .testsettings file as suggested in the message.
Sorry if this seems remedial. I am including it for the sake of completeness.
General library behavior
A library can be referenced either in the project file (and so the compiler injects to code to load the references) or dynamically at runtime with LoadLibrary() or PInvoke calls. When a referenced library is loaded, a function at the entry point is run can in turn load any libraries it depends on. When loading the library, there is a well-known set of paths that Windows will search, including %WINDIR%\Assembly and the current directory. There's a lot of good conceptual information on Wikipedia about this. I recommend reading it.
Possible Root Causes
I can't tell from your question if you are having trouble building the application, building the tests, or executing either. Generally I would not expect PInvoke to cause compile errors.
Error during app build: VS generally will show you that you have a reference to a DLL it can't find. However, you may be missing a DLL that is needed to satisfy all the dependencies. To resolve, just add the reference to the missing DLL. (This is the simplest issue, so I'm guessing this isn't what you're seeing.)
Error during test build: Since your test will reference your application/library, it also needs to have the same reference. Usually the easiest way to ensure you are getting everything is to remove all references and add a reference to the project you are testing. It's possible you some additional libraries are necessary for some tests, but not your app/lib itself. These need to be added separately.
Error during app execution: This can happen when starting the application, or later when an call to the external library is made if late binding is used.
Error during test execution: This can happen the same as with app execution. However, tests can also be "partially built" to only execute a small number of tests. In these cases, some files may not be copied. Using the [DeploymentItem()] attribute, you can specify that a test requires the presence of certain files in the test or app/lib project to function. MSDN describes how this can be done.
Resolution
For #1 & #2 the solution lies in adjusting the references in the project.
For #3 & #4, it may get trickier. There is a similar question to yours regarding Windows Mobile here which you may find useful, especially referring to using dumpbin to list out library dependencies. You can also use procmon from SysInternals to monitor file access during compile or load to see which files are not found. Then you can either include the missing file, or remove the library referencing it.
Good luck. Hope this helps.
To my astonishment I found VB6 code that uses Declare statements to define functions in a .dll that lives in the Program Folder without it being registered on Windows. This seems like a supersimple way to avoid the .dll hell without having to resort to using Side by side manifests. Can I read some more about this somewhere? Are there snags?
The Declare statement is used to do "just in time" binding to non-ActiveX DLLs. Until your program "touches" a Declared entrypoint no attempt is made to load the library.
It basically has nothing at all to do with the topic of DLL Hell.
Muddled thinking can even lead people to plop ActiveX DLLs "next to" the EXE which actually can result in DLL Hell because people who tend to do this also use poor techniques for installing and uninstalling applications.
Poorly designed application A deployment plops a commonly shared DLL or OCX next to the EXE.
Poorly designed application is run, the VB6 runtime can't find the classes in the registry, does a DLL Search using Windows heuristics, immediately locates the DLL next to the EXE and calls its self-registration entrypoint.
Innocent, properly designed applications B, C, D are later installed that use the same DLL/OCX and their installers find the library already registered.
Poorly designed application A is uninstalled, typically by simply deleteing its folder in Program Files.
Applications B, C, and D (and any future applications using the library) are now broken - due to orphaned component registration pointing to a non-existant library.
Moral of the story:
Never, never, never put DLLs "next to" your VB6 application on installation. If you have private DLLs that are not shared with other applications even then put them into a libs, etc. folder under the application folder. Possible exception might be non-COM DLLs, such as those you expect to use Declare with.
There is also a great deal of misunderstanding about manifests, of which there are multiple kinds. The ones you are probably thinking of are application and assembly manifests.
These can be used for selecting among different versions of a library installed Side by Side, or they can be used to isolate applications and assemblies which is the part that has bearing on DLL Hell.
Of course application manifests can be used to specify quite a few other things about how Windows should run the application.
Windows searches in a well-documented sequence of folders for LoadLibrary (which VB6 uses behind the scenes to resolve Declare declarations). Since the first location on the list of search folders is the app's own folder, your discovery makes perfect sense.
It doesn't resolve the "DLL hell" issue for the most part, though. It can't work for system DLLs, for instance, because Windows preloads most of them. Also, if a DLL is already loaded into memory, Windows may use that copy of the DLL (not sharing data, but code can be reused).
That's part of the reason that manifests were created; they allow an application to strictly define required versions of system DLLs in order to provide certain functionality. VB6's technique is old fashioned (just like VB6).
How exactly do DLL files work? There seems to be an awful lot of them, but I don't know what they are or how they work.
So, what's the deal with them?
What is a DLL?
Dynamic Link Libraries (DLL)s are like EXEs but they are not directly executable. They are similar to .so files in Linux/Unix. That is to say, DLLs are MS's implementation of shared libraries.
DLLs are so much like an EXE that the file format itself is the same. Both EXE and DLLs are based on the Portable Executable (PE) file format. DLLs can also contain COM components and .NET libraries.
What does a DLL contain?
A DLL contains functions, classes, variables, UIs and resources (such as icons, images, files, ...) that an EXE, or other DLL uses.
Types of libraries:
On virtually all operating systems, there are 2 types of libraries. Static libraries and dynamic libraries. In windows the file extensions are as follows: Static libraries (.lib) and dynamic libraries (.dll). The main difference is that static libraries are linked to the executable at compile time; whereas dynamic linked libraries are not linked until run-time.
More on static and dynamic libraries:
You don't normally see static libraries though on your computer, because a static library is embedded directly inside of a module (EXE or DLL). A dynamic library is a stand-alone file.
A DLL can be changed at any time and is only loaded at runtime when an EXE explicitly loads the DLL. A static library cannot be changed once it is compiled within the EXE.
A DLL can be updated individually without updating the EXE itself.
Loading a DLL:
A program loads a DLL at startup, via the Win32 API LoadLibrary, or when it is a dependency of another DLL. A program uses the GetProcAddress to load a function or LoadResource to load a resource.
Further reading:
Please check MSDN or Wikipedia for further reading. Also the sources of this answer.
What is a DLL?
DLL files are binary files that can contain executable code and resources like images, etc. Unlike applications, these cannot be directly executed, but an application will load them as and when they are required (or all at once during startup).
Are they important?
Most applications will load the DLL files they require at startup. If any of these are not found the system will not be able to start the process at all.
DLL files might require other DLL files
In the same way that an application requires a DLL file, a DLL file might be dependent on other DLL files itself. If one of these DLL files in the chain of dependency is not found, the application will not load. This is debugged easily using any dependency walker tools, like Dependency Walker.
There are so many of them in the system folders
Most of the system functionality is exposed to a user program in the form of DLL files as they are a standard form of sharing code / resources. Each functionality is kept separately in different DLL files so that only the required DLL files will be loaded and thus reduce the memory constraints on the system.
Installed applications also use DLL files
DLL files also becomes a form of separating functionalities physically as explained above. Good applications also try to not load the DLL files until they are absolutely required, which reduces the memory requirements. This too causes applications to ship with a lot of DLL files.
DLL Hell
However, at times system upgrades often breaks other programs when there is a version mismatch between the shared DLL files and the program that requires them. System checkpoints and DLL cache, etc. have been the initiatives from M$ to solve this problem. The .NET platform might not face this issue at all.
How do we know what's inside a DLL file?
You have to use an external tool like DUMPBIN or Dependency Walker which will not only show what publicly visible functions (known as exports) are contained inside the DLL files and also what other DLL files it requires and which exports from those DLL files this DLL file is dependent upon.
How do we create / use them?
Refer the programming documentation from your vendor. For C++, refer to LoadLibrary in MSDN.
Let’s say you are making an executable that uses some functions found in a library.
If the library you are using is static, the linker will copy the object code for these functions directly from the library and insert them into the executable.
Now if this executable is run it has every thing it needs, so the executable loader just loads it into memory and runs it.
If the library is dynamic the linker will not insert object code but rather it will insert a stub which basically says this function is located in this DLL at this location.
Now if this executable is run, bits of the executable are missing (i.e the stubs) so the loader goes through the executable fixing up the missing stubs. Only after all the stubs have been resolved will the executable be allowed to run.
To see this in action delete or rename the DLL and watch how the loader will report a missing DLL error when you try to run the executable.
Hence the name Dynamic Link Library, parts of the linking process is being done dynamically at run time by the executable loader.
One a final note, if you don't link to the DLL then no stubs will be inserted by the linker, but Windows still provides the GetProcAddress API that allows you to load an execute the DLL function entry point long after the executable has started.
DLLs (dynamic link libraries) and SLs (shared libraries, equivalent under UNIX) are just libraries of executable code which can be dynamically linked into an executable at load time.
Static libraries are inserted into an executable at compile time and are fixed from that point. They increase the size of the executable and cannot be shared.
Dynamic libraries have the following advantages:
1/ They are loaded at run time rather than compile time so they can be updated independently of the executable (all those fancy windows and dialog boxes you see in Windows come from DLLs so the look-and-feel of your application can change without you having to rewrite it).
2/ Because they're independent, the code can be shared across multiple executables - this saves memory since, if you're running 100 apps with a single DLL, there may only be one copy of the DLL in memory.
Their main disadvantage is advantage #1 - having DLLs change independent your application may cause your application to stop working or start behaving in a bizarre manner. DLL versioning tend not to be managed very well under Windows and this leads to the quaintly-named "DLL Hell".
DLL files contain an Export Table which is a list of symbols which can be looked up by the calling program. The symbols are typically functions with the C calling convention (__stcall). The export table also contains the address of the function.
With this information, the calling program can then call the functions within the DLL even though it did not have access to the DLL at compile time.
Introducing Dynamic Link Libraries has some more information.
http://support.microsoft.com/kb/815065
A DLL is a library that contains code
and data that can be used by more than
one program at the same time. For
example, in Windows operating systems,
the Comdlg32 DLL performs common
dialog box related functions.
Therefore, each program can use the
functionality that is contained in
this DLL to implement an Open dialog
box. This helps promote code reuse and
efficient memory usage.
By using a DLL, a program can be
modularized into separate components.
For example, an accounting program may
be sold by module. Each module can be
loaded into the main program at run
time if that module is installed.
Because the modules are separate, the
load time of the program is faster,
and a module is only loaded when that
functionality is requested.
Additionally, updates are easier to
apply to each module without affecting
other parts of the program. For
example, you may have a payroll
program, and the tax rates change each
year. When these changes are isolated
to a DLL, you can apply an update
without needing to build or install
the whole program again.
http://en.wikipedia.org/wiki/Dynamic-link_library
DLL is a File Extension & Known As “dynamic link library” file format used for holding multiple codes and procedures for Windows programs. Software & Games runs on the bases of DLL Files; DLL files was created so that multiple applications could use their information at the same time.
IF you want to get more information about DLL Files or facing any error read the following post.
https://www.bouncegeek.com/fix-dll-errors-windows-586985/
DLLs (Dynamic Link Libraries) contain resources used by one or more applications or services. They can contain classes, icons, strings, objects, interfaces, and pretty much anything a developer would need to store except a UI.
According to Microsoft
(DLL) Dynamic link libraries are files that contain data, code, or resources needed for the running of applications. These are files that are created by the windows ecosystem and can be shared between two or more applications.
When a program or software runs on Windows, much of how the application works depends on the DLL files of the program. For instance, if a particular application had several modules, then how each module interacts with each other is determined by the Windows DLL files.
If you want detailed explanation, check these useful resources
What are dll files , About Dll files