I've inherited a few maven projects which have added a /dependencies directory to capture Java jar libraries that aren't part of the project war and must be installed by a DevOps into a Tomcat installation.
The libraries in this directory seem to fall into four categories:
"provided" scope libraries,
downstream dependencies of those provided libraries, and
discoverable implementations of api jars
"mystery" libraries, i.e., not available in an external repository, and maybe unsure where they ever came from.
Is there a strategy to get Maven to help manage these dependencies and perhaps fetch them for external install?
There are probably several strategies to choose from.
Number one: leave it as it is. If it works and the build is reproducible (on different environments) that seems one valid solution.
The "mystery" part of the build might not be more of an issue for new people working with it.
I think it is valid to create an own maven module to be delivered to the infrastructure team. This module can contain the jars in the /dependencies folder.
What you would need to do is create a pom.xml and add all dependencies currently in that directory (of course not the transitive ones). The magic ones would need to go in a repository proxy (nexus, artifactory, ...). If you don't have a maven repository yet: you want one! (its easy to setup and it does help a lot!)
I would then use the assembly plugin or some ant task to build the zip do be delivered. So the infrastructure team is able to just unzip / copy the files where they need to be. This step can then even be scripted (so the upload / unzip is done through SSH or something like that).
This is probably only one way to do it. I would assume to resolve the jar's in the /dependencies directory may be a bit of a pain.
The advantage is obviously that you document and simplify the management of those libraries. I would also assume if you update some of them it is easier across branches to merge since there are no binary files around. So it may be worth the effort.
Related
I'm working on a project that has a reliance on a significant number of 3rd party jars; these jars contain both proprietary classes and custom/patched versions of existing libraries. I'm trying to figure out how to tie these jars into the maven architecture so as to be CI friendly.
My initial idea was to create an uberjar of all these libraries, and add that one uberjar to the maven repository, however I have not been able to figure out how to take this set of standalone jar files (that are not dependencies) to merge like that. There exists lots of documentation on how to uberjar a project and its dependencies, but not standalone jars.
I feel like i'm missing something basic, or maybe there is a better way. Any recommendations are welcome.
You would be better off to proceed as follows:
Set up a repository manager.
Write a shell script that runs mvn deploy:deploy-file for each of your jars, generating the GAV with a simple algorithm.
treat them like anything else.
To elaborate on Step 2:
G:A:V - my-company-name:name-of-jar:version-based-on-todays-date.
In my multi-module Maven project, suppose I have two modules, car and horse. They both depend on a JAR file, transport.jar, a file not available in any online Maven repositories. As such, I need to find a way to make these modules depend on a file found somewhere in the project folder structure.
From what I understand, the default Maven solution would be to manually register the JAR file in the local repository. While this would work on a development machine, it breaks on the build server, which clears its local repository before each build.
I've been searching online on how to do this on and off for a while and found some helpful things, but nothing that completely works.
For instance, a common answer is to add a dependency to the file using <scope>system</scope>. However, not only do others claim that it's extremely bad practice to do so, it also doesn't work on the build server. (On a side note, I would also like to point out that using absolute paths to the JAR is also out of the question due to, again, it being built on several different machines.)
A more useful method I found was to define a local repository in the POM file, pointing towards the path file:${project.basedir}/lib. (Such as in this article) Unfortunately, if I place the JAR and repository definition in the car POM, I cannot successfully add a dependency to the JAR in horse. I've tried both with and without an additional reference to car in horse, as well as defining a second repository in horse, pointing to file:${project.basedir}/../car/lib. This problem would also remain if I tried to make a third module, transport-lib, specifically for wrapping the JAR dependency.
I could most likely add the JAR file to both modules and define two separate module-local repositories, but I really don't want to unless I have to due to the need to keep the two (often updated) JARs in sync etc.
So, my question is as follows: Can someone give me a confirmed-to-work method to have two modules depend on the same JAR file inside the project, given the parameters and restrictions mentioned?
Best solution is to use a repository manager like Archiva, Artifactory or Nexus and install that artifact into the repository manager. Afterwards you can use this artifact directly in your pom files without any issue.
Don't use the scope system, cause it will cause other problem after a release for other etc.
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)
We are trying to migrate our builds from ant to maven. Project I am working for is using ant since ages. Scripts are real complex where build artifact is a zip file having a definite directory structure. Build creates about 108 unique jars and packages them to this zip file along with many config files and other 3rd party jars.
We need this zip file to be same as now even after migration to Maven. I am just a learner on Maven as of now. My question to you guys is that if there is a way using which I can use one pom.xml to produce more that one jar file, providing list of jars and includes, excludes packages for each.
I googled and found that in case we need multiple jars from one project but different packages, we can do so by placing one pom.xml at each package (jar will be created from this package) and bonding all using dependency management.
But this does not solves my problem, as having 100+ pom.xmls does not seems to be a good idea.
Hope I have made my question clear. Please suggest if there is a way out.
Adding to Udo's answer here's another Sonatype blog posting with a diagram explaining the Maven anti-pattern of generating multiple jars from a single Maven project.
Both articles recommend adopting a modular structure to your code rather than fighting Maven's approach of decoupling large projects into a set of interrelated sub-modules.
Incremental publish approach
Start with looking at the arifacts you actually plan to share. Your mail suggests that the only file you're actually publishing is a large zip file containing 100+ jars and other files?
You could invoke the Maven command-line tool to publish this zip to the Maven repository:
mvn deploy:deploy-file \
-Durl=$REPO_URL \
-DrepositoryId=$REPO_ID \
-DgroupId=org.myorg \
-DartifactId=myproj \
-Dversion=1.2.3 \
-Dpackaging=zip \
-Dfile=myproj.zip
This approach can also be used to publish jars and POMs (containing dependencies). Eventually you'll be overwhelmed with the number of POMs to maintain.... AT which time it would be simpler to restructure the building of that jar into a sub-module.
Alternative to switching build technology
It's very difficult to walk away from a legacy ANT build. These often contain complex, custom and difficult to reproduce build logic. For such projects I recommend using Apache ivy to externalise 3rd party dependencies and share artifacts with other projects (who might be using Maven).
To that end I wrote an ant2ivy script for generating an initial ivy setup, based on the jars that already exist in the ANT project's directory (normally committed alongside the source).
Using ivy doesn't get you away from the fact that it's a good idea to create project sub-modules. However it does enable you to modernise your ANT build.
Update
Yes, there are Maven ANT tasks available. I don't use them because they based on Maven 2. I'm disappointed that we're still waiting for their Maven 3 replacement aether-ant-tasks (Only available from GitHub). Ivy is still the no1 choice for integrating non-Maven clients with a Maven repository
Well you can generate multiple jars out of one project.
Its not really considered best practice. Look at the supplied article and decide it yourself.
In the includes your are not limited to packages, however this surely makes it easier. :)
I'd like to make sure I'm not including duplicate libraries in my Maven project. I'm pulling in quite a few dependencies from a few different repositories including main, Spring's repo, and JBoss' repository. How can I see a graph of the dependencies so I can make sure I'm not depending on multiple versions of a single library?
It would be kind of a bummer if I was including something like org.springframework:spring-core:3.0.1.RELEASE and org.springframework:org.springframework.spring-core:3.0.5.RELEASE which actually do show up as different dependencies, as weird as it sounds, because Spring's repository names things weirdly. What can I do to make sure I don't have duplicate dependencies?
The dependency:tree maven plugin should do what you want and show the tree.
Alternatively force Maven2 to copy dependencies into target/lib to copy the the artifacts in the target/lib directory which should also show duplicates but you would still need to grep all the POMs in the ~/.m2/repository to see which library is pulling in which version manually and that would be time consuming for large local repositories.
Duplicate libraries can be a problem, but duplicate classes can exist even if no libraries are duplicated. JBoss Tattletale can analyze a set of jar files and produce a report which lists all duplicated classes. There’s a Maven plugin for running the report - see http://docs.jboss.org/tattletale/userguide/1.2/en-US/html/maven.html
If you'd like to get a graphical, searchable representation of the dependency tree (including all modules from your project, transitive dependencies and eviction information), check out UpdateImpact: https://app.updateimpact.com (free service).
Using the search, you can find out how many times a given library is imported, by what modules and in which versions, as well as verify if older versions of a library are evicted by newer ones.
Disclaimer: I'm one of the developers of the site