I sure am not the first person to wonder, yet it is really hard to find information on my question.
At the company I work at, the servers run WebSphere 8.5. In order to cut down installation time and memory usage, we created some shared libraries, where we have copy-pasted the common dependencies of our applications. Now you can see, that these can easily cause big problems when changing versions.
We have a Nexus Repository however, where every dependency our applications use can be reached. Since in our gradle projects, we have those dependencies listed anyway, it only seems logical for WAS to download and put these dependencies on the classpath of the application at startup (or whenever it wants to).
Is there a way to tell WAS where to look for these dependencies (our repo server) and what files to download? Is there a better solution to the problem?
There's no way to do this inside of the server at runtime. The best you can hope for is setting up the shared libraries during your deployment, single-sourced with your applications declared dependencies.
Related
I've inherited a few maven projects which have added a /dependencies directory to capture Java jar libraries that aren't part of the project war and must be installed by a DevOps into a Tomcat installation.
The libraries in this directory seem to fall into four categories:
"provided" scope libraries,
downstream dependencies of those provided libraries, and
discoverable implementations of api jars
"mystery" libraries, i.e., not available in an external repository, and maybe unsure where they ever came from.
Is there a strategy to get Maven to help manage these dependencies and perhaps fetch them for external install?
There are probably several strategies to choose from.
Number one: leave it as it is. If it works and the build is reproducible (on different environments) that seems one valid solution.
The "mystery" part of the build might not be more of an issue for new people working with it.
I think it is valid to create an own maven module to be delivered to the infrastructure team. This module can contain the jars in the /dependencies folder.
What you would need to do is create a pom.xml and add all dependencies currently in that directory (of course not the transitive ones). The magic ones would need to go in a repository proxy (nexus, artifactory, ...). If you don't have a maven repository yet: you want one! (its easy to setup and it does help a lot!)
I would then use the assembly plugin or some ant task to build the zip do be delivered. So the infrastructure team is able to just unzip / copy the files where they need to be. This step can then even be scripted (so the upload / unzip is done through SSH or something like that).
This is probably only one way to do it. I would assume to resolve the jar's in the /dependencies directory may be a bit of a pain.
The advantage is obviously that you document and simplify the management of those libraries. I would also assume if you update some of them it is easier across branches to merge since there are no binary files around. So it may be worth the effort.
I have a GWT webapp split into two Maven projects where one is a dependency to the other. Each time I change something in the dependency and I'm running webapp in hosted mode I have to rebuild the subproject and restart hosted mode for changes to apply. It takes a lot of time so I'd like to ask you if there is any way to make GWT using "live" version of the dependency?
There are 2 cases:
for server-side code, assuming you use the DevMode's embedded server, rebuilding the app and then refreshing the server should be enough
for client-side code, AFAICT, you have to use the source and output directories of the dependency module rather than the JAR containing them (GWT will load the source from the classpath, but apparently it'll only see the modified sources if it comes from a folder rather than a JAR; at least that's what I found in my tests). This goes against The Maven Way™ but the only solution so far is to use a special profile that will import the sources of the dependency project as sources of the project you're running. You can see examples of that in my archetypes.
There's actually a bug opened for the gwt-maven-plugin, MGWT-332, to do that automatically when running a reactor build. I also mused about what's really needed, for the forthcoming official gwt-maven-plugin (rewritten from scratch, independent from the CodeHaus Mojo plugin).
If your dependency does not come from a reactor build, then you're out on your own: you chose to make it totally distinct, so that's how it'll behave: you'll have to release it (even a snapshot) each time you make a change to it, and use the new version in your app (which means re-launching the DevMode).
This can be circumvented by running DevMode on your own, without the help of the gwt-maven-plugin. You're left on your own managing the classpath though (using the Google Plugin for Eclipse, I suppose you could simply edit the launch configuration to add the source folders of your dependency project to the classpath, before the classpath provided by Maven, that would reference the JAR).
Remove the dependent other application jar file from the primary application lib folder under webapp.
Eclipse should then resolve the dependency using the other project in the workspace if you have added it to your primary application classpath.
As GWT build takes ages, we invested some money in a JRebel license. We have two separate Eclipse projects for our back-end and our GWT front-end. JRebel reloads the classes automatically and I never need to restart my local server while writing code. It proved to be a wonderful time saver. Definitely worth the investment.
There have been many posts on this topic, but I have yet to find the "real" solution.
How does one manage their dependency tree (both compile time and runtime) using MSBuild project files (i.e. Visual Studio project files via project and file references)?
It is well known that project references from child projects will not be copied to an application bin directory if there is no compile time reference, even if there is a runtime dependency, and even if copy-local=true. Hence, any loosely coupled component will not be copied over.
The hack to solve this problem is to include the dependency in the parent project with copy-local=true. However, this basically destroys your dependency tree as you no longer know where the dependency is and ultimately, as your app grows and morphs, you end up with a version of DLL hell. Your parent project ends up with 10s to 100s of dlls, most of which are runtime dependencies of dlls in child projects.
Another hack is to write a custom targets file and call it from every project file: http://blog.alexyakunin.com/2009/09/making-msbuild-visual-studio-to.html. But surely there is a better option. This is such a bread and butter thing. Java devs never have to deal with such trivial issues.
From what I can gather, the Microsoft way to solve this problem is to register every dependency in the GAC for every dev, test and production machine. But this is stupid and annoying. I won't bother giving this option and educated rebuttal.
Avoiding the GAC option, how could one use MSBuild to manage a dependency tree that includes runtime only dependencies? How does Microsoft do it? Surely they don't run custom targets files like the one in the link above.
I hope someone from an enterprise .NET background can step up and offer some real advice on this. Otherwise I'm just going to have to rewrite all my build scripts in NAnt (shudder).
Thanks All.
UPDATE
In response to some comments, the following is a practical example of the issue from my current project.
The app is a Web Application project that exposes a suite of WCF services. It has an external domain DLL containing the external service classes and an internal domain DLL containing internal service POCOs, domain objects and DAOs. There is a separate integration DLL containing interfaces (DTOs) for all the internal domain classes that allows us to completely decouple the external and internal domains. The whole thing is wired up with Spring.net. I hope this is clear, let me know if you need more clarification.
My current build process is to use MSBuild to generate a deployment package for the web application (in TFS Build). So while the whole solution is built initially, only the output from the web application gets packaged. Therefore, the Web Application is treated as the dependency root and I expect that any loosely coupled child references should get copied over on build if they are set to 'copy-always=true'.
So the Web Application contains a reference to the external domain DLL which contains a reference to the internal domain DLL which contains many references to 3rd party libraries and various indirect and loosely coupled dependencies required by the 3rd party libraries.
The problem occurs when there is a 3rd party dependency in the internal domain DLL e.g. oracle.dataaccess which is required by NHibernate at runtime. Even when I set 'copy-always=true' on these DLLs, they do not get copied to the Web App package. The only way I can include them in the package is to add these DLLs to the Web App's references. I don't want to do this because I no longer have a meaningful dependency tree.
I hope this makes the issue clearer. Please let me know if anything is unclear. It's hard to describe this sort of stuff.
If anyone is also having a similar issue, please speak up and share your experience.
I really want to give you a better answer but unfortunately you didn't put enough information about your solution/projects and your dependencies, so I will try to give you several ideas and I hope one of them works.
The easiest thing to do as you said is to set up a separate folder with all of your dependencies and create target file that will copy them to your bin folder. If you have dependencies that are not changing frequently that might work. If another team from your company is building them and they change frequently, this approach is not good.
Another simple approach - if you're referencing your dependencies from your solution only you can change the build path, so that they build directly into the bin folder of your main project. This way you don't have to reference them directly.
Use NuGet. You have a separate team producing loosely coupled dependencies it may make a sense to set up local NuGet repository and use it for that http://juristr.com/blog/2012/04/using-nuget-to-distribute-our-company/
I hope that helps.
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)
Trying to build Spring-based application one needs to figure out all necessary dependencies the application will have.
For example, I was using HibernateTemplate, and each time I run the application the ClassNotFound exception comes out. So I “google” for jars that contains this particular class, after search mvnrepository to find appropriate artifact. Always confuse about which version to use.
And it’s happened again and again, and only after few hours and few dozens of dependencies added the application become runnable.
But even after that, I tried to use my app. on different computer with slightly different parameters, and slf4j class not found error appeared, even after testing extensively on the developer machine, still some dependencies missing.
Now it works fine, but I want to distribute my application, and not sure if on another system there will no dependency missing.
So, what is the best practice to determine all necessary dependencies not only at design time but in runtime too? Is there any tool for that?
How one can manage versions confusion, when there are dozens of dependencies each with its own version?
They both resolve dependencies, so you keep a dependency file and it does all the heavy lifting of making sure everything is included in your builds. Use the full spring dependency list. I had problems with SLF4J too.
http://mvnrepository.com/artifact/org.springframework/spring-full/1.2.8