There have been many posts on this topic, but I have yet to find the "real" solution.
How does one manage their dependency tree (both compile time and runtime) using MSBuild project files (i.e. Visual Studio project files via project and file references)?
It is well known that project references from child projects will not be copied to an application bin directory if there is no compile time reference, even if there is a runtime dependency, and even if copy-local=true. Hence, any loosely coupled component will not be copied over.
The hack to solve this problem is to include the dependency in the parent project with copy-local=true. However, this basically destroys your dependency tree as you no longer know where the dependency is and ultimately, as your app grows and morphs, you end up with a version of DLL hell. Your parent project ends up with 10s to 100s of dlls, most of which are runtime dependencies of dlls in child projects.
Another hack is to write a custom targets file and call it from every project file: http://blog.alexyakunin.com/2009/09/making-msbuild-visual-studio-to.html. But surely there is a better option. This is such a bread and butter thing. Java devs never have to deal with such trivial issues.
From what I can gather, the Microsoft way to solve this problem is to register every dependency in the GAC for every dev, test and production machine. But this is stupid and annoying. I won't bother giving this option and educated rebuttal.
Avoiding the GAC option, how could one use MSBuild to manage a dependency tree that includes runtime only dependencies? How does Microsoft do it? Surely they don't run custom targets files like the one in the link above.
I hope someone from an enterprise .NET background can step up and offer some real advice on this. Otherwise I'm just going to have to rewrite all my build scripts in NAnt (shudder).
Thanks All.
UPDATE
In response to some comments, the following is a practical example of the issue from my current project.
The app is a Web Application project that exposes a suite of WCF services. It has an external domain DLL containing the external service classes and an internal domain DLL containing internal service POCOs, domain objects and DAOs. There is a separate integration DLL containing interfaces (DTOs) for all the internal domain classes that allows us to completely decouple the external and internal domains. The whole thing is wired up with Spring.net. I hope this is clear, let me know if you need more clarification.
My current build process is to use MSBuild to generate a deployment package for the web application (in TFS Build). So while the whole solution is built initially, only the output from the web application gets packaged. Therefore, the Web Application is treated as the dependency root and I expect that any loosely coupled child references should get copied over on build if they are set to 'copy-always=true'.
So the Web Application contains a reference to the external domain DLL which contains a reference to the internal domain DLL which contains many references to 3rd party libraries and various indirect and loosely coupled dependencies required by the 3rd party libraries.
The problem occurs when there is a 3rd party dependency in the internal domain DLL e.g. oracle.dataaccess which is required by NHibernate at runtime. Even when I set 'copy-always=true' on these DLLs, they do not get copied to the Web App package. The only way I can include them in the package is to add these DLLs to the Web App's references. I don't want to do this because I no longer have a meaningful dependency tree.
I hope this makes the issue clearer. Please let me know if anything is unclear. It's hard to describe this sort of stuff.
If anyone is also having a similar issue, please speak up and share your experience.
I really want to give you a better answer but unfortunately you didn't put enough information about your solution/projects and your dependencies, so I will try to give you several ideas and I hope one of them works.
The easiest thing to do as you said is to set up a separate folder with all of your dependencies and create target file that will copy them to your bin folder. If you have dependencies that are not changing frequently that might work. If another team from your company is building them and they change frequently, this approach is not good.
Another simple approach - if you're referencing your dependencies from your solution only you can change the build path, so that they build directly into the bin folder of your main project. This way you don't have to reference them directly.
Use NuGet. You have a separate team producing loosely coupled dependencies it may make a sense to set up local NuGet repository and use it for that http://juristr.com/blog/2012/04/using-nuget-to-distribute-our-company/
I hope that helps.
Related
I sure am not the first person to wonder, yet it is really hard to find information on my question.
At the company I work at, the servers run WebSphere 8.5. In order to cut down installation time and memory usage, we created some shared libraries, where we have copy-pasted the common dependencies of our applications. Now you can see, that these can easily cause big problems when changing versions.
We have a Nexus Repository however, where every dependency our applications use can be reached. Since in our gradle projects, we have those dependencies listed anyway, it only seems logical for WAS to download and put these dependencies on the classpath of the application at startup (or whenever it wants to).
Is there a way to tell WAS where to look for these dependencies (our repo server) and what files to download? Is there a better solution to the problem?
There's no way to do this inside of the server at runtime. The best you can hope for is setting up the shared libraries during your deployment, single-sourced with your applications declared dependencies.
On my current project we are using Nuget to bring in dependencies on things like NHibernate, we use Ninject as our IoC container (although that's not an important detail of the problem I'm trying to solve) and we use Ninject's scanning functionality for establishing our bindings.
This all works fine but the problem that I'm facing is that I've gone through the trouble of making my Data Access Layer be exposed to the Application Layer in a Presistence Ignorant fashion but Ninject needs to have access to the NHibernate Dlls at scan time. If I wasn't using Nuget to pull in dependencies then this could easily be solved with a post build step to copy the dependencies from wherever I had decided to put them in my source tree.
However, with using Nuget my understanding is that the path to Dlls could change in the package directory as dependencies are upgraded. I definitely don't want to solve this problem by using Nuget in my Coposite Root projects (services, UX, etc...) and so I'm trying to figure out a clean way to get the dlls that I need at scanning time available in my Composite Root projects execution directories.
My need is pretty basic but I could not find any clean answer to it: I simply need to be able to distribute a resource in a multi-module project.
Let us consider for example the LICENSE file, which I hereby assume to be the same for all modules. I prefer not to manually copy it into each and every module because the file could change over time. I also prefer not to statically link to resources (even if using relative paths) outside the project folder, because the modular structure can possibly change too.
Is there any plugin that can be used to robustly guarantee that each module is given the required file? It would be equally acceptable for such copy to be obtained by exploiting the POM of the parent project or directly performed by the super project in the modular hierarchy.
you could use the assembly and the dependency plugins.. did you stumble over that link?
http://www.sonatype.com/people/2008/04/how-to-share-resources-across-projects-in-maven/
it describes that option ..its from 2008, but maven is around for quite some time.. so I guess its more or less up to date
edit regarding comment
Another option is the maven-remote-resources-plugin.
For a more detailed example see:
http://maven.apache.org/plugins/maven-remote-resources-plugin/examples/sharing-resources.html
Since their intro speaks actually for itself, I quote (maven.apache.org)
This plugin is used to retrieve JARs of resources from remote repositories, process those resources, and incorporate them into JARs you build with Maven. A very common use-case is the need to package certain resources in a consistent way across your organization: at Apache it is required that every JAR produced contains a copy of the Apache license and a notice file that references all used software in a given project.
I have an msbuild project which builds a SLN file from visual studio which holds all the projects in (about 70+ project), and a lot of the projects are dependent on each other meaning they need to be build in order - sometimes a developer forgets to set the build order manually in visual studio in the solution file causing the msbuild on a clean solution to fail as something gets built out of order/cant find a dll.
Is there a way for msbuild to take all projects and work out the dependencies and build the projects in order, if there is how do i do this? using an MSBuild task? With current tries it seems to just build in the order it reads the projects in - if i pass in a list of project files+paths.
Currently the only way i can think to solve this is a external app which scans the proj files and references and then manually creates a solution each time.. but this seems overkill for such a simple thing.
Anyone solved / seen this before?
How are you calling MSBuild? If you point MSBuild to the solution file, it should be able to work out the dependencies. If you point it to individual project files, then it won't be able to resolve any project references.
If you don't use project references you can still control the dependency order in a solution by using the "Project Dependencies" dialog to manually set the dependencies.
While Project Dependencies are hard to maintain and not shared across .sln files, Project References are honoured and do dictate the order consistently - see the ResolveReferences task in Microsoft.Common.targets.
ASIDE: A 'friend of mine' may 'during a refactoring' have accidentally stubbed out their Build Task and it's DependsOnTargets linkage to the Microsoft.Common.targets ResolveReferences task and ended up with ProjectReferences not being honoured in ways that sound like the question here. If you read some of the posts, you might get the idea that it's all mad shaky - it's not; the shaky bits are the Project dependencies, not the Project references.
See this excellent MSDN Blog article by Dan Moseley that really explains the topic, including some useful workaround strategies. (via this mildly related issue with building xUnit.net).
If all of your dependent projects are in the solution and you are using Project references, Visual Studio should manage the dependencies for your and build in order of that dependency list.
It sounds like you are not using project references. I always recommend project references.
This is an old question but the issue was most likely that projects in the solution used direct references to dependent DLLs (Add Reference > select Browse tab > select dependent DLL) instead of using project references (Add Reference > select Projects tab > select dependent project). With direct references, Visual Studio can't figure out the dependency chain. You must tell it by right clicking on the solution node and select Properties. Pick Common Properties > Project Dependencies to set the required projects. Mr. Klaus is correct but I wanted to document how to fix this issue.
While it is correct that MSBuild should observe the build order when you use project dependencies there is one caveat. It doesn't at present observe the reverse build order when building the clean target (as I have blogged about here). For regular build however it works nicely as described by others here.
I am using Msbuild 4 found at c:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe
It seems to solve the problem.
There is no Microsoft tool that will examine all the dependencies of your 70+ projects and generate a solution file with dependencies clearly declared for you.
You have to do that on your own by using 2 different methods:
Manually specify a dependency, for the solution, in visual studio.
Specify a project reference in the project file itself.
If you don't want to do that, then you will have to swallow the medicine and accept that you will to use an external tool to do that for you. Yes it's clunky but it can be made to work. If you check in your solution file to your source control you can mitigate these problems. As long as you have an active solution file to work with.
I at one point didn't, and I had 600+ projects in the build. So I wrote a tool (years ago) that would automate 99% of this work. It uses the .NET MSBuild API's to read the msbuild files (no recreating the wheel here with xml api's). It then examines outputs and inputs and generate a dependency tree which I can then do a few things with it:
Spit out a solution file.
Do a dependency sort (also a topological sort in academia), and spit out those projects in order they should be built (for a non-parallel type of build, which can be useful sometimes).
print out all sorts of diagnostic information about dependencies.
The only limitation I have seen with the tool is with a few crazy COM dependencies which are pretty sketchy anyways. Which I added a super simple work-around.
I'm using Castle Windsor to do some dependency injection, specifically I've abstracted the DAL layer to interfaces that are now being loaded by DI.
Once the project is developed & deployed all the .bin files will be in the same location, but for while I'm developing in Visual Studio, the only ways I can see of getting the dependency injected project's .bin file into the startup project's bin folder is to either have a post-build event that copies it in, or to put in a manual reference to the DAL project to pull the file in.
I'm not totally thrilled with either solution, so I was wondering if there was a 'standard' way of solving this problem?
Could you set the build output path of the concrete DAL project to be the bin folder of the dependent project?
Mike: Didn't think of that, that could work, have to remember to turn off copy-local for any libraries / projects that are common between them