We have a custom plugin that runs during the initialize phase of our Maven build, and sets certain properties to the project. We also have a dependency, whose version relies on those properties. Our build is failing because the dependency version is invalid, because the project attempts to resolve the dependency before the plugin runs. Is there any way around this? Potential solutions (that I don't know how to implement):
Somehow allow the plugin to run before the project really even kicks off/attempts to resolve dependencies
delay the dependency resolution until later in the lifecycle
Are either of these possible?
Edit: Following a couple questions on context I'll provide some.
For a couple of different reasons, maintaining versions in a large parent (aggregator) pom file was becoming a big problem for us, so it actually started to make sense for us to define them in an external service. The plugin reads in the current versions off of the service, and sets them to properties. A couple of these properties would be used as dependency versions.
Anticipating the comments to be -> "you should just go back to using the large aggregator", we can.... but it is definitely as a last resort.
Edit 2: relevant part of POM per request:
<dependencies>
<dependency>
<groupId>com.myCompany</groupId>
<artifactId>myDep</artifactId>
<version>${myDep.major.version}.${myDep.minor.version}</version>
<type>ear</type>
</dependency>
</dependencies>
Traditionally myDep.major.version and myDep.minor.version we're defined in our parent pom.xml, but now we want to move them to the service we're using.
Related
Our team has multiple projects; most projects are just libraries. Let's assume for simplicity that the libraries don't depend on each other and there is a single project that uses them, e.g.:
Project Main:
Project Lib-A:
X (3rd-party library)
Project Lib-B:
X (3rd-party library)
To avoid surprises in 'Main', we'd like to make sure that all our own projects are using the same versions of the 3rd-party libraries, so that e.g. both 'Lib-A' and 'Lib-B' are built and tested with the same version of library X.
To achieve this we use a parent pom with <dependencyManagement> section detailing the versions of all relevant 3rd-party libraries and also their transitive dependencies. This parent pom is inherited by all the projects, i.e. 'Main', 'Lib-A', and 'Lib-B' from the above example. Then each child pom would only use <dependency> without specifying any version. We also have maven enforcer plugin's dependencyConvergence rule to make sure we have not missed any library conflict in any of our projects.
The problem: increasing the version of X: a developer of 'Lib-A' increases a version of X from 1.0 to 2.0. So he changes X's version in the parent pom, increases the version of parent, releases the parent pom, and notifies guys from 'Main' that they should now use a new parent. The situation becomes like this:
Main - inherits from Parent:2.0 and depends on:
Lib-A:2.0 - inherits from Parent 2.0 and depends on X:2.0
Lib-B:1.0 - inherits from Parent 1.0 and depends on X:1.0
X:2.0 (taken from Parent:2.0 <dependencyManagement> section)
Everything, including 'Main', builds fine, 'maven enforcer plugin' does not detect any conflict because the version of X is clearly specified in the Parent:2.0 from which 'Main' inherits. So we release 'Main'.
Ooops.... Lib-B has never been built with X:2.0. It has great unit tests that would uncover the problem, but we never tried this. We forgot to update Lib-B, try it with X:2.0 and release it. Still 'Main' has been built without problems and maven enforcer plugin has never complained.
Question: we need maven to detect that there are dependencies that inherit from the same artifact but different major versions and fail the build.
In our case the build had to fail since 'Main' and 'Lib-A' inherit from Parent:2.0, but 'Lib-B' inherits from Parent:1.0.
My solution so far (a hack): in addition to inheriting, add an explicit dependency on the parent pom to all out projects (i.e. 'Main', 'Lib-A', and 'Lib-B'):
<dependency>
<artifactId>Parent</artifactId>
<type>pom</type>
<version>${project.parent.version}</version>
</dependency>
Then use <bannedDependencies> rule of maven enforcer plugin to ban other major Parent versions (we could also use its <dependencyConvergence/> rule if we want to fail even on minor Parent version conflicts).
Is there a less hacky and cumbersome way to fail on conflicting major versions of parent pom?May be our entire approach to managing maven dependencies is wrong, what is the recommended way then?
Update:
Tried writing my own rule for maven-enforcer-plugin as suggested by #JF Mayer and described here, before giving up. Reasons:
First, the parent pom information is not available from the dependencies, at least not from the nodes built by maven's DependencyGraphBuilder
OK, I've added my parent poms as explicit dependencies to the children and tried to use this DependencyGraphBuilder to detect dependencies on the parent with different major versions. No way! As could be seen with mvn dependency:tree that also uses this class, DependencyGraphBuilder does not provide all the dependencies so it can't be used to detect dependencies conflicts. That's why the <dependencyConvergence> maven enforcer rule is using a super-deprecated DefaultDependencyTreeBuilder that has been even deleted from the GitHub and everywhere else - not a good choice for a trouble-free custom solution.
For completeness, my own poor-man's solution:
Add an explicit dependency of type pom to the parent to every project so that maven-enforcer-plugin's <dependencyConvergence> rule would detect conflicting parent versions. No big deal with this one as we only add this section once and forget about it:
<dependency>
<groupId>${project.parent.groupId}</groupId>
<artifactId>${project.parent.artifactId}</artifactId>
<version>${project.parent.version}</version>
<type>pom</type>
</dependency>
<dependencyConvergence> will fail a build on 'Main' even on 'increment' differences in the parent versions, e.g. 1.0.1 and 1.0.2. In this case, the developer of 'Main' can decide that it's OK to build it despite the parent version conflict, because it's insignificant (this was my original question). So he builds 'Main' with some special profile that excludes the dependency on the parent: mvn -P I-know-what-I-am-doing deploy.
I am not very happy with this solution because of the step 2 that requires the developers of 'Main' to build it with a special profile in case of a parent versions conflict. I'd prefer a solution that always fails on major parent version conflicts but ignores insignificant differences in the parent pom versions automatically, I just don't know how to achieve this.
I think what you are looking for is Reactor Module Convergence enforcer:
https://maven.apache.org/enforcer/enforcer-rules/reactorModuleConvergence.html
Its not the same as the dependency convergence rule.
This question already has answers here:
Maven : Should I keep or remove declared dependencies that are also transitives dependencies?
(2 answers)
Closed 7 years ago.
I'm a newbie in maven and I have a project that has directly dependencies with several libraries, but if I declare only one dependency in my pom.xml the project compiles and runs perfectly. It is becasuse this library have other dependencies which are automatically imported and contain my directly dependencies.
Is it recommended to add all dependencies in the pom.xml despite transitive dependencies?
What version of a dependency should I use? The highest possible version?
No, when there are transitive dependencies which are resolved properly you don't need to specify them explicitly in the pom.xml. Thus your pom is kept small and tidy.
You should use the highest stable version of dependencies in your new projects.
However there are cases when you need a different version (in most cases higher) of a transitive dependency to be used. In that case you specify the transitive dependency with the higher version in a <dependencyManagement> tag. For example if we have:
<dependencies>
<dependency> <!-- has transitive dependency of com.artifact2 v.1.0 -->
<groupId>com.group1</groupId>
<artifactId>com.artifact1</artifactId>
<version>1.2</version>
</dependency>
</dependencies>
If we want to specify explicitly that we need the new version of com.artifact2 which is 2.0 then we add to the pom these lines:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.group1</groupId>
<artifactId>com.artifact2</artifactId>
<version>2.0</version>
</dependency>
</dependencies>
</dependencyManagement>
It is almost always better to use you exactly those dependencies you need. Bundled dependencies often contain more then what you need. You can however, by adding exclusions in your pom.xml, make the dependencies contain only the libraries that you actually need. Often more then one 3th party library uses same common libraries (for example logging dependencies are found in a lot of dependencies). Those libraries will then cause a conflict which can be problematic at some web containers. If you are using eclipse, open your pom in Dependency Hierarchy and see how it goes from there...
Versions depend of several factors. First important thing is always to pick a RELEASE versions (unless for example when one is explicitly required to pick some newest beta, containing the newest features you have been waiting on for 3 months). Second is to figure out which newest versions of different libraries and frameworks you use can successfully work together. Newer versions are usually preferred cause they are developed last, meaning: more developer support, more probable to work with other state-of-the-art frameworks.
It's a long story. My current place uses Ant for builds. They create about 20 different foundation class jars that are used in multiple projects. Originally, the projects would check in the particular versions of the various jars they needed and then never updated them. That meant that each application had incompatible jars with other projects and our servers. Hilarity ensued.
To handle this, I immediately brought in a Maven repository, and integrated Ivy into our Ant builds. No more checking in jars. Instead, you fetch the correct version from the Maven repository. Originally, I expected the developers to keep the version numbers in the ivy.xml up to date, but they never did. Instead, the Ivy integration and setup depends upon an external Subversion project. This allowed me to integrate Ivy with minimal changes to the old build.xml files. I added a ivy.version.properties file to that external ivy project and maintain the version numbers of various jars in there. There's a corporate wide version number.
The various projects use the ${corporate.version} property for our foundation jars version numbers. When I update that ivy.version.properties file, all projects get updated with the right version number for our foundation classes. We use <ivy:makepom> to generate a pom.xml for our projects and use this to deploy our jars and wars into our Maven repository.
The result: I no longer have to worry about the developers keeping the version numbers of their projects in sync. I, as the release engineer handle that by updating that one ivy.version.properties file. All projects are in sync.
Now, we're moving to Maven, and I want to be able to do the same thing. I doubt developers will remember to update their pom.xml with the correct version numbers, so I want to read that in from another file, and use that.
There are two issues: One is that Maven first reads in the version number of a project before it executes any goal. No version number in the pom.xml, no version number for the generated jar.
Even if I manage to get by that first step, we have the fact that the pom.xml has no version number in it for foundation classes. When Maven pulls down the pom.xml to get the dependencies, it can't figure out which revision.
Yes, I could put that into a corporate pom.xml and have that as a parent project for all of the other projects. We already have a parent project to help set up various aspect of all projects. I could put a foundation class version number in there. However, that means that the developers now have to update the parent project's version number with each release. If developers can't be trusted to update the version number of their project with each release, what makes you think they'll do that with the parent's version for each release?
I know other people must have run into a similar issue. How do you handle this?
I could have an Ant script that generates the pom.xml from a template pom.xml, but that seems a bit silly.
I was wondering if is it possible for Maven to generate a pom.xml on the fly and then to use that to continue the executing the right goal. For example, I type in mvn package, and Maven will take a template.pom.xml file, fill in the missing version numbers to generate a generated.pom.xml file, then execute mvn package on that generated pom.
Or, is there a better way to solve this issue? Basically, I need to control the version number of our releases across all projects. This way, all projects are using the same version of our foundation classes. Also, I control some other versions of other jars (like log4j) this way. I've done this with Ant and Ivy, and now I want to move to Maven.
I think the best option is to create a pom.xml with all the dependencies to your packages in its and then import it in your developer project paren pom.xml using import
So, in the project parent POM:
<properties>
<corporate.version>...</corporate.version>
<properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.mycompany.libs</groupId>
<artifactId>foundation<artifactId>
<version>${corporate.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
...
</dependencies>
</dependencyManagement>
So for new set of foundation libraries, you deploy a new pom.xml with all the versions in it and update the ${corporate.version} in the relevant parent pom.xml file. You can even have when the versions are not yet fixed define a -SNAPSHOT artifact with these version numbers.
for more information, see: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
We actually have the same kind of problem and I know that there is trade-off between "lazy programmers" who never update their poms and the stability aspect: Changing the versions of the foundation jars "suddenly" which may cause unexpected build fails.
If you really want the behaviour you describe, I would put the desired dependencies into a SNAPSHOT pom and use it either as a parent pom or import it as a BOM. In this way, you can change the content and everybody pulls in the new content with the next update.
I've decided that this a stupid idea and should never have been done. It's the developers' responsibility to update their own damn POM and make sure they're pulling the correct version. This is the way it really should be and developers should simply learn to do it (what's the technical term? ...oh yeah) the Correct Way.
I originally implemented this because the company had something like this (which wasn't working) and I had problems getting the developers to do what should be their job. And, the developers liked it because I was now officially responsible when something goes wrong rather than them. It's what you get when you do someone else's job.
I've convinced the company to stop trying to keep all our core jars (the jars used in our other projects) and our releases at the same version number. Instead, every project will have its own versioning. Jars will version only when there's an actual change and not when we force an upgrade to the version. And, the developers must know what versions of the various jars they depend upon.
Use the maven-release-plugin. This plugin can stamp a version number on all pom.xml files. The plugin also understands version control and knows how to tag a maven project for release (or branching).
We have a big and deep module structure with three "real" layers of inheritance and three to four layers of module aggregation.
I am fine with the real inheritance: a company wide super-pom, a product-wide parent and a customer-wide parent for customizations of the product.
But I observe that also the "empty" aggregating modules are defined as parents of their submodules.
If the whole concept is the same as in OO, this makes no sense to me if the aggregating modules (they are empty besides their submodules) add no specific configuration to the pom.
Is there any other reason (perhaps operational) why this could be useful?
Sidenote: Introduction to the pom is not clear in this respect: term "parent" is not clear: it can mean super-pom (=parent in inheritance tree) or aggregating pom (=parent in filesystem...)
There is a great problem in the definition of these concepts in the Maven community. You are right with stating that a parent-pom is inheritance and module-pom is composition. But unfortunately the separation of these two concepts have no history in Maven. The Maven documentation says clearly that it is better to separate them. This would result in this ideal structure
app-api/
app-impl/
app-war/
app-parent/
pom.xml
Where pom.xml is the module pom.
But almost every project you will see in the open source sector does not distinguish between them.
This has a reason: Many plugins do not distiguish between them as well. And even Maven itself assumes another setting: If <relativePath> is not set Maven assumes that the parent is at ... So every project has to point to <relativePath>../app-parent</relativePath> as parent.
The most popular plugin that has huge problems with the mentioned structure is the maven-release-plugin! You will face strange problems when you do not follow the assumption that the module-pom is the parent-pom.
The worst bug is that the release-plugin cannot replace version-properties in your parent and fails because of SNAPSHOT-Dependencies. Your parent perhaps will contain something like this
<properties>
<app-api.version>1.1-SNAPSHOT</app-api.version>
<app-impl.version>1.2-SNAPSHOT</app-impl.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>myorg</groupId>
<artifactId>app-api</artifactId>
<version>${app-api.version}</version>
</dependency>
<dependency>
<groupId>myorg</groupId>
<artifactId>app-impl</artifactId>
<version>${app-impl.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
Normally while releasing these properties would automatically be set to their release versions 1.1 and 1.2. But the release-plugin (tested up to version 2.2.2) fails with a message that the module cannot be released with snapshot dependencies.
If you only "skip" some module-poms this might not be a problem when you define <relativePath> correctly. But you have to try and expect some bugs in Maven plugins. Besides these bugs you are totally right to seperate the poms and I would give it a try if you have the time to make a sandboxed release.
The short answer IIUC is, you don't inherit from your aggregation-only poms. You do when your aggregator is the same as your parent, as in the canonical maven structure. But if you separate aggregator from parent, don't inherit from the aggregator
The inheritance of a parent pom is usefull if you have properties that are valid for several projects. From what I understood you already got that part right ;-)
What you call aggregating modules has a valid practical use too. You can group several submodules into a single pom. If you want to build your project you mostly don't want to build every module sperately but join them into one single application. This can be done by such a pom that references all the submodules as such. It can't be done with just the concept of a parent pom as it knows nothing of the other modules.
Adding an eclipse perspective here :
when you create maven module within a maven project using eclipse IDE's wizard, by default, the new module will created as child of top level project and also top level project will be an aggregation of child.
I have read how to take a jar file that I have and add it to the dependency system in maven in this link
But what i want is, I have a set of 30 to 40 jar files that I have to add to the dependency system. Do i need to add all the jar files by using the
mvn install:install-file -DgroupId=com.stackoverflow... -DartifactId=yourartifactid... -Dversion=1.0 -Dpackaging=jar -Dfile=/path/to/jarfile,
followed by,
2.
<dependencies>
...
<dependency>
<groupId>com.stackoverflow...</groupId>
<artifactId>artifactId...</artifactId>
<version>1.0</version>
</dependency>
...
</dependencies>
or is there a simple way by which i can wrap up all the dependent jar files into one maven project using mvn:install, to which my main project can be made dependent?
Thanks in advance.
If most of the 30-40 jar files you are referring to are standard open source libraries, then you would not need to install them to your local repository manually. It is also possible that many of them are transitive, i.e. one dependency in turn pulling in another. In this case, you would need to only add as dependency the primary dependencies. This will in turn pull in the transitive ones.
If most of the dependencies are proprietary and you manage them yourself, then chances are you have them in your source code management system in some specific folder, say lib. If so, you can skip the install step and just specify dependencies using <system> scope (refer here).
You could define a single maven project (as pom type) to specify all the dependencies and have your main project dependant on it, if that simplifies things.
The first answer in this link really solved my doubt..
"Projects should explicitly list all their dependencies and not rely on inheriting them (IMHO). It means you have to declare your logger a few times but it will save you pain later. (You can of course use a separate POM project to group together dependencies that are related and hence usually specified together, like the hibernate example in your link). If you want to centralize versions of dependencies, you can put a dependencyManagement section in the parent POM, which means you still declare the dependency in the child project, but the version comes from the parent, thus ensuring consistency. Children that don't declare the dependency don't end up with it at all."
So the practice should be to create separate POM (Child) project for all the dependencies, and if we want, we can wrap them up using aggregation to a Parent-POM which can be referred singly in our project. But we have to however create a child project separately for each dependent JAR. Bit time taking process, but seems to be the only answer.