We have a huge program. Approximately 250 DLLs. Of these, we have approximately 45 COM DLLs and 20+ .NET assemblies that have ComVisible classes in them.
Now we want to work on an XCopy deployable version... WHich means that we need to use Isolated COM. I have been experimenting with smaller projects and have the basic mechanics down of how it works.
Now the problem is that after having built the manifest for one application, it won't load. I used sxsTrace and it gives a useless error message.
Last four lines of sxstrace log:
INFO: Parsing Manifest File c:\src\v13\Debug-Win32\sta_net_DMModel.DLL.
INFO: Manifest Definition Identity is sta_net_DMModel,processorArchitecture="x86",version="1.0.0.0".
ERROR: Activation Context generation failed.
End Activation Context Generation.
The only thing useful is the name of the last DLL manifest it was trying to load. It was a .NET assembly that had a bunch of ComVisible classes in it that are not necessary at startup. I commented it out of the manifest and then the program loaded. But, as I looked at it, I noticed it had nearly 500 clrClass definitions in the manifest for the DLL. I had another DLL that had about a third of that number. I re-included the original DLL in my application manifest but commented out the ~170 clrClass DLL in the application manifest. When I did that, the application loaded to success.
So, it appears to me that there is some kind of limit on the maximum number of clrClass type information that can be loaded from the manifests for Isolated COM. It seems to be a cumulative thing because if I comment out either DLL, then it loads. But if I leave both in, then it won't load.
Is there a documented limit to the amount of type information that the manifest loaders can process?
Is there a registry setting I could change, or some constant I could add to my manifest to bump up the limit? I have no idea what the limit is, but there seems to be one.
Running depends.exe gives this:
Error: The Side-by-Side configuration information for "c:\src\v13\debug-win32\STATIST.EXE" contains errors. The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail (14001).
I eventually found the problem. It was not that there were too many clrClass'es in the manifests. I used ProcMon and PerfView to try and see the problem, but they didn't show much.
Eventually, I just started editing on the manifests and deleting and undeleting clrClass entries from the manifest until I found the problem. I sort of did a binary search...delete half and see if it goes away. If not delete the other half and see if the problem goes away. Then repeat with the problem section and keep halving until the offending clrClass (or clrClasses) is found.
To make a long story short, I found that two different clrClass'es in the two DLLs I was having problems with had the same CLSID. The manifest loader was erroring out because there were COM visible classes in the two DLLs that had duplicate CLSIDs. Somewhere, someone had made a mistake with copy and paste or something. However, the error message from the sxstrace utility did not specify what the error was. It just gave the generic error. To triple check, I wrote a utility to check the CLSID, ProgID, and class names of all the COM visible classes in the two libraries. I eventually found that there were 3 CLSIDs that were duplicates.
So, if anyone ever looks at this in the future, they can see that duplicate CLSIDs (or ProgIDs?) in the clrClass declarations in the .NET assemblies might be their problem.
Related
I am trying to get a C# Visual Studio 2019/MSBuild job to build on a Jenkins build server. I know that my file paths are too long, so I have enabled Long File Paths in the Group Policy Editor (and verified that it has persisted in the registry editor after a server restart).
However, now I am getting the following error "ALINK: fatal error AL1065: File name ... is too long or invalid".
A quick google search led me to this page for Alchemy Software. However I have no idea what Alchemy Software is and why it is being used in the build process and why it is failing. (Although for the last point, I'm guessing that the Alchemy Software .dll is not "Long Address Aware", which I believe is necessary for an application to take advantage of Long Paths in Windows. But since I can't locate any .dll or .exe associated with this software, I can't be certain.)
Does anyone know why my build is still failing with this error, what Alchemy Software is, and how to get it to take advantage of Long Paths in Windows?
P.S. And please, no comments about how I should restructure my file paths to be shorter. I have tried doing that but it's impractical for this application. And anyway, it keeps popping up and is becoming a whack-a-mole situation, so I'd really rather fix the root cause rather than constantly putting band-aids everywhere.
I am going to set this as an answer since, after Hans Passant's very helpful comments and subsequent research, I think it's pretty definitive that this can mostly only be worked around, not resolved. (As possible exception will be discussed at the end of this answer.)
As stated in those comments, this error originates from a linker module called al.exe that is utilized by MSBuild for certain project configurations (more on that later). This linker can be found in a couple places, but for me it was being called from C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools.
MSBuild can handle long file paths as long as Windows is configured to do so (by enabling long paths in the Group Policy Editor or by modifying the registry directly). However this al.exe module cannot. And as far as I can tell, there's no way to force it to do so. So if your build tool chain requires this al.exe module to be used, you're kind of SOL.
For my particular situation, for a job in Jenkins that was failing because my paths were too long, I worked around this by changing the workspace of my job. So now instead of the default of something like D:\Jenkins\workspaces\[product]\Releases\[product_version], I changed it to D:\j\dd0_1c, which is an encoding that makes sense to me. This shortening of the folder path avoids any subsequent file paths from exceeding the limit of 260 characters. It's not a satisfying solution, but it works for my particular situation.
I did mention that there was a possible exception to all of this: if you can get MSBuild to avoid using al.exe altogether, then you can avoid this error.
I don't know all of the scenarios or workflows in which MSBuild utilizes this module, but I do know that it does get utilized when your application has localized resources, and somehow, some way, MSBuild uses al.exe when it is generating those resources. This was exactly my scenario, and I found this page and this page describing how you can reconfigure your localization projects such that MSBuild does not utilize al.exe. I did try the steps described in these pages and was able to verify that I no longer got this ALINK error from al.exe. However I never got my project to fully build since this reconfiguration caused other build errors to crop up. So in the end, for the sake of expediency (and because it was cleaner than performing a major refactor of my code), I went with the Jenkins workspace workaround.
However, it is interesting to note that you can get MSBuild to avoid using al.exe as long as your project is conducive to the solution given in those two links. So hopefully, if someone runs into this same issue, they might have more success than I in utilizing this method.
I'm maintaining a VB6 application with many COM components (DLLs and OCXs). In order to streamline development and deployment I'd like to use reg-free com. The problem with development is that the application runs within the VB6.EXE instance. How can I trick VB6 to use my (unregistered) components? It is very important for me to not have to go through registering/unregistering components when switching between branches. Generating a .manifest file for VB6 is not out of the question but is there some other, more optimal way, to specify a .manifest file when launching VB6.EXE?
Note: The Activation Context API doesn't seem to help, even if used from within the development environment.
Solutions I've thought:
A utility application that activates a context from a manifest and launches VB6 as a child process (doesn't work; processes don't inherit activation context)
Injecting context activation into the VB6 process at startup (too complicated; must hack the executable to do this)
Hosting VB6 in my own process after activating the right context (can't even find out if this is possible)
Using a VB6 Add-In or other utility that runs within VB6 to activate a context (tried that but it doesn't seem to work)
Update Jan. 16
As suggested by wqw, I did some testing with a VB.exe.manifest. The VB6.exe.manifest worked, with some caveats:
The SxS dll specified in the manifest would not appear in the references window on projects that didn't actually reference the component
On projects that did reference the component it would be shown to reside in the directory according to the following order:
The pathname recorded in the project file (if the file was still present)
A pathname as if it resided in the same folder as the project (vbp)
If the file was not in any of these folders, the project would not compile (just running the code causes an internal compile in VB6) with the message "Can't find project or library".
Obviously, VB6 actualy scans the registry to find COM components and verifies, during compilation, that they exist where they say they exist. I'm not sure what that might mean if I actually want to use VB6.exe.manifest to redirect COM component instantiation. Perhaps having dummy component files at some predefined location might trick VB6 into believing that everything is as it should be, although an entirely different set of components got loaded for use.
Further update:
I did a test on that last assumption and it proved to be false. The component has to actually be there in order for the project to compile. It must even properly load (no dummy, zero-length files accepted!). Now I'm not even sure if the manifest works. That's a more time-consuming test (requires a component with two versions that produce different results, one with the project, and one for the manifest).
Our approach to this problem was to write a build assist program that registered and unregistered components, run the VB6 compiler, and would even rewrite project files with updated GUIDs when interfaces changed. You would hand it a VBG project group and it would do the rest.
I suppose we could also have added a mode that unregistered components when you switched branches.
Are you following the practice of using "compatibility" binaries? You shouldn't use the binary at your build location for compatibility references - you should commit a separate copy to version control and configure your project to consider that the "compatible" version - only change this file when you break interfaces.
I have a VS2010 C# project, that references a large set of native .dll's (a commercial java runtime). These file are referenced as 'Content' files in the project, since the need to be copied with the project.
The code in these libraries is called using PInvoke, there is no assembly reference.
Every time I compile the solution, the Visual Studio testing framework tries to load all the referenced dll files, expecting to find .net assemblies which may contain unit tests. Since the are no .net assemblies, the following exception is thrown:
Error loading some.dll: Unable to load the test container 'e:\some.dll' or one of its dependencies. If you build your test project assembly as a 64 bit assembly, it cannot be loaded. When you build your test project assembly, select "Any CPU" for the platform. To run your tests in 64 bit mode on a 64 bit processor, you must change your test settings in the Hosts tab to run your tests in a 32 bit process. Error details: Could not load file or assembly 'file:///e:\some.dll' or one of its dependencies. The module was expected to contain an assembly manifest.
This takes a whole lot of time, and I would like to tell Visual Studio to not try to load these files.
How can I tell Visual Studio to stop trying to load these files?
Correct me if I got this wrong:
You are including the P/Invoke target binaries in to the VS solution because you want the binaries to be copied over to the target directory when the solution is built. You want this because the project will execute from the target directory as soon as the VS solution is built. Correct?
Often times VS packages (both default and 3rd party) try to get smart about the solution content and will follow certain triggers (which are difficult to contain and control by ourselves) and load the solution and project content in their own ways. Fighting the battle in this area has poor ROI than employing a simpler work around (below).
While I can't provide you with an authoritative answer on how to tell VS's test package to not load all binaries, I suggest removing such binaries from the project as 'content' and leave them in your source control where they are today. Add a post-build task that will copy the said binaries over to the target. This will still give you the same result as it is working today but, takes those binaries out of reach for the test probes.
You must check out configuration settings by just right clicking on your solution name and click on "Configuration Manager"
It will open a pop up window for Configuration Manager.
Check not for the platform your projects are using it is better to choose any CPU.
Hope this can help.Give it a try:)
Because thats what your exeception says as you have quoted
Thanks
I tried to repro this issue and found that the root cause is that you have set your test project to be compiled as !AnyCpu. Is there any particular reason why you would want this for managed test code?
So unless you change this you will continue to see this message.
If you want to continue using this configuration for your test project you would need to update your .testsettings file as suggested in the message.
Sorry if this seems remedial. I am including it for the sake of completeness.
General library behavior
A library can be referenced either in the project file (and so the compiler injects to code to load the references) or dynamically at runtime with LoadLibrary() or PInvoke calls. When a referenced library is loaded, a function at the entry point is run can in turn load any libraries it depends on. When loading the library, there is a well-known set of paths that Windows will search, including %WINDIR%\Assembly and the current directory. There's a lot of good conceptual information on Wikipedia about this. I recommend reading it.
Possible Root Causes
I can't tell from your question if you are having trouble building the application, building the tests, or executing either. Generally I would not expect PInvoke to cause compile errors.
Error during app build: VS generally will show you that you have a reference to a DLL it can't find. However, you may be missing a DLL that is needed to satisfy all the dependencies. To resolve, just add the reference to the missing DLL. (This is the simplest issue, so I'm guessing this isn't what you're seeing.)
Error during test build: Since your test will reference your application/library, it also needs to have the same reference. Usually the easiest way to ensure you are getting everything is to remove all references and add a reference to the project you are testing. It's possible you some additional libraries are necessary for some tests, but not your app/lib itself. These need to be added separately.
Error during app execution: This can happen when starting the application, or later when an call to the external library is made if late binding is used.
Error during test execution: This can happen the same as with app execution. However, tests can also be "partially built" to only execute a small number of tests. In these cases, some files may not be copied. Using the [DeploymentItem()] attribute, you can specify that a test requires the presence of certain files in the test or app/lib project to function. MSDN describes how this can be done.
Resolution
For #1 & #2 the solution lies in adjusting the references in the project.
For #3 & #4, it may get trickier. There is a similar question to yours regarding Windows Mobile here which you may find useful, especially referring to using dumpbin to list out library dependencies. You can also use procmon from SysInternals to monitor file access during compile or load to see which files are not found. Then you can either include the missing file, or remove the library referencing it.
Good luck. Hope this helps.
Background
I'm maintaining a plugin for an application. I'm Using Visual C++ 2003.
The plugin is composed of several DLLs - there's the main DLL, that's the one that the application loads using LoadLibrary, and there are several utility DLLs that are used by the main DLL and by each other.
Dependencies generally look like this:
plugin.dll -> utilA.dll, utilB.dll
utilA.dll -> utilB.dll
utilB.dll -> utilA.dll, utilC.dll
You get the picture.
Some of the dependencies between the DLLs are load-time and some run-time.
All the DLL files are stored in the executable's directory (not a requirement, just how it works now).
The problem
There's a new requirement - running multiple instances of the plugin within the application.
The application runs each instance of a plugin in its own thread, i.e. each thread calls functions exported by plugin.dll. The plugin's code, however, is anything but thread-safe - lots of global variables etc..
Unfortunately, fixing the whole thing isn't currently an option, so I need a way to load multiple (at most 3) copies of the plugin's DLLs in the same process.
Option 1: The distinct names approach
Creating 3 copies of each DLL file, so that each file has a distinct name. e.g. plugin1.dll, plugin2.dll, plugin3.dll, utilA1.dll, utilA2.dll, utilA3.dll, utilB1.dll, etc.. The application will load plugin1.dll, plugin2.dll and plugin3.dll. The files will be in the executable's directory.
For each group of DLLs to know each other by name (so the inter-dependencies work), the names need to be known at compilation time - meaning the DLLs need to be compiled multiple times, only each time with different output file names.
Not very complicated, but I'd hate having 3 copies of the VS project files, and don't like having to compile the same files over and over.
Option 2: The side-by-side assemblies approach
Creating 3 copies of the DLL files, each group in its own directory, and defining each group as an assembly by putting an assembly manifest file in the directory, listing the plugin's DLLs.
Each DLL will have an application manifest pointing to the assembly, so that the loader finds the copies of the utility DLLs that reside in the same directory. The manifest needs to be embedded for it to be found when a DLL is loaded using LoadLibrary. I'll use mt.exe from a later VS version for the job, since VS2003 has no built-in manifest embedding support.
I've tried this approach with partial success - dependencies are found during load-time of the DLLs, but not when a DLL function is called that loads another DLL.
This seems to be the expected behavior according to this article - A DLL's activation context is only used at the DLL's load-time, and afterwards it's deactivated and the process's activation context is used.
Edit: Works with ISOLATION_AWARE_ENABLED as expected - runtime loading of DLLs uses the original activation context of the loading DLL.
Questions
Got any other options? Any quick & dirty solution will do. :-)
Will ISOLATION_AWARE_ENABLED even work with VS2003? Edit: It does.
Comments will be greatly appreciated.
Thanks!
ISOLATION_AWARE_ENABLED is implemented by the Windows SDK header files and thus probably wont worth with VS2003 at all. However, it is possible to download the latest Windows 7 SDK and use that with VS2003.
You don't need to use MT to link in manifests. Manifests can be embedded as resources in environments that dont have explicit knowledge.
Add the following to a dll's .rc file to embed a manifest. (With a recent enough platform sdk RT_MANIFEST should already be defined):
#define RT_MANIFEST 24
#define APP_MANIFEST 1
#define DLL_MANIFEST 2
DLL_MANIFEST RT_MANIFEST dllName.dll.embed.manifest
Background
My solution consists of two projects:
A standard Windows application
A DLL which my application does not use directly, but instead injects it into a target process
Basically, from my application's perspective, the only requirement that the DLL must meet is to be present in the working directory of my application. In short, my DLL doesn't export any functions that are of interest to my application.
Question
I would like to strongly couple these two binaries. What are my options aside from manually calling LoadLibrary in my application?
If this is too vague, please let me know.
Edit
Since no one seems to be "answering", I ended up doing what EFraim suggested (see comments).
I exported a dummy function from my DLL and added the DLL's generated *.lib file as an additional dependency in my application's linker property sheet. Now, at run-time, if the DLL's missing, Windows gives a nice error message and terminates execution. As an added bonus, the DLL image is also locked after successful IAT initialization; this prevents things like user deletion.
If you don't expect releasing dll and exe separately, you can add dll into your resourses and unpack it from there on startup.