I've got a project with Maven in which one subproject (A) wants to depend on another subproject (B) which uses "pom" packaging.
If I do this the straightforward way, where A specifies a dependency on B with <type>pom</type>, things work perfectly if I do "mvn install", but if I run any phase earlier than install, such as mvn compile or mvn package, then it fails while trying to build A: it goes looking for B's pom in the repository, and doesn't find it.
I don't really want this pom in the repository, because it's part of our active source code and changes frequently.
For all the jar-packaged projects we build, it seems to work fine to keep them out of the repository, build with mvn package, and Maven knows how to find all the dependencies in the source and build trees it manages without resorting to the repository; however for the pom-packaged project it always wants to go to the repository.
A couple things I learned while trying to understand this:
Maven best practices encourage you to use pom-packaged projects to group dependencies, but with the added step of "mvn install" on the POM project
Maven lifecycle documentation says "a project that is purely metadata (packaging value is pom) only binds goals to the install and deploy phases"; maybe this is why the POM project is invisible as a dependency target unless I invoke the install phase? I tried binding the compiler plugin to the compile phase and this didn't seem to help.
Is there a way that I can specify the POM subproject as a dependency of another subproject in the same parent project, without installing the POM project to the repository?
It isn't purely a question of which goals are bound to which lifecycle phases for POM projects. If it were, then binding the "package" goal would solve the problem.
When building a multi-module project, Maven reads the POMs of all modules to determine dependencies between modules, so that it can build the depended-upon modules before the depending modules. It's able to achieve this even when running the "package" goal (such that the depended-upon modules are not yet in the local repository).
Therefore, the code that constructs the classpath for builds must be managing several cases, notably:
extra-project jar dependency, where it looks for the POM in the local repository, handles its dependencies, and adds the POM's jar to the classpath
extra-project pom dependency, where it looks for the POM in the local repository and handles its dependencies
intra-project jar dependency, where it looks for the POM within the project tree, handles its dependencies, and adds that module's target/classes folder to the classpath
intra-project pom dependency, where for some reason it doesn't look for the POM within the project tree, and therefore doesn't handle it's dependencies.
Notice the asymmetry in the last two cases, as compared to the first two.
I can see two solutions to your problem. One is to file a bug report, or rather a request to change the behaviour (since it's obviously intentional), perhaps only for the case of intra-project dependencies on multi-module projects. Or indeed propose a patch. But since the behaviour is intentional, you might meet a refusal. In the best of cases, you're in for a long wait. (I'd vote for your bug report though - I've been stung by that same behaviour, in a different context.)
The other solution is simply to run an install on your project. I don't really understand why you don't want the POM project in your repository: if needs be, you can use a snapshot repository, where it doesn't matter if things change frequently, to avoid polluting your main repository.
Configuring maven-install-plugin to run during the compile phase, and copy the relevant pom.xml to the repository, seems to accomplish what I wanted as far as Maven itself is concerned, though m2eclipse still is not happy (it throws "failed to read artifact descriptor" errors with no additional description for the pom.xml that has a dependency on the other POM project).
Related
I have a question about how the Maven dependency resolving mechanism is working in a multi module project.
Normally I only use 'mvn clean install' when I build my multi module projects and my assumption was that if any module in the project needs a previous module, dependency will be resolved by going local repository and loading the corresponding 'jar'.
For project internal reason, I have to use 'mvn clean compile,' this command naturally does not create any 'jar' while 'install' is not there. So here I started wondering, how the dependency resolution for a multi module project works, while jar' is not created but project still able to see the changes from the previous builds. Does the target directories used for dependency management?
Or for 'mvn clean compile' target directory used but for 'mvn clean install' the local repository.
Can anybody explain me how the dependency resolution works in a 'multi module' project.
Thx for answers.....
I think you will understand better if you look at https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
There is a life cycle in the process of building the jar. The compile target will compile the code and create a complete classes folder in your target directory. This target will resolve all your dependencies in your poms and download any dependencies to your local repo, not already there.
The install target will create the jar from the classes directory and install it in your local repository.
I really think you will need to run the install target to get anything useful.
Maven is made of separate components.
There a component that deal with a given module and among other things try to get its dependencies. It ALWAYS get the dependencies from the local repository, eventually after having downloaded such dependencies. If the dependencies are not there and can't be dowloaded, it will fail. Eventually the module will create its own artifact that it will publish to the local repo.
Then there a compoment that when you ask it to build several maven modules, for example calling mvn at the root of a project does order the various module use the dependencies to find the best ordering for the build so that if a given module depend on another, it will be build after the module it depend on. It then call the previous compoent I described, building each module in order.
In all cases, a given module dependencies are always taken from the local repo. The expectation is that the previous modules that were built before actually pushed their artifact to the local repo typically with the mvn install but you could force it do it at any step thanks to proper configuration (may not be a good idea).
In all case if the previous component jar was not built and put into the repo, there no way such jar can be added in the classpath for the next module to be compiled.
Doing compile only on multiple projets isn't going to be any useful.
I have a parent project(has its own pom.xml) in which I import the child project as a jar with its own pom.xml.
In the parent pom.xml I have specified my child jar as a dependency - this gets resolved, but i want maven to resolve the dependencies required by my child jar.
My Use case to replicate :
When I include spring-web-mvc.jar the transitive dependencies are resolved automatically.
I have a similar requirement where I include my child.jar into a main framework project and expect the transitive dependencies to get resolved (Notw: the child.jar is not hosted it is packaged as jar and present on the local file system)
Current Structure:
Child Project:
|----/src/main/java
|----/src/main/resources
|----child-pom.xml
>This child project will be a jar as dependency in the parent project
Parent Project
|----/src/main/java
|----/src/main/resources
|----/src/main/webapp/WEB-INF/lib/child.jar
|---- parent-pom.xml
The problem:
When i create a war from parent project i want all the dependecy including transitive ones to show in WEB-INF lib.
Currently this is not happening.
First when talking of parent/ child Maven projects normaly you have your"childs" specified as modules in a common parent project which itself is beeing packed with the packaging type pom rather than make them a dependency of the parent project.
When it comes to the dependencies of your "childs" or generally "dependencies of your dependencies" those are called transitive dependencies and are pretty well explained in the official documentation found here: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
The resolving of those transitive dependencies is one of Mavens core strenghts and guaranteed by default unless they lead to conflicts that make the build fail.
Two things to help here are having a closer look into the enforcer plugin (http://maven.apache.org/enforcer/maven-enforcer-plugin/) and the shader plugin (http://maven.apache.org/plugins/maven-shade-plugin/) .... well and the official documentation of corse (reading the whole thing takes less than a day - then supports you further for specific topics whereas we gladly further support you too if you already have pom.xml files and you are stuck somewhere.
While the enforcer-plugin covers certain conflicts regarding different versions of the same artefacts the shader plugin will just pack everything you specified to a single jar for reverseengineering (its not the normal use case but i sometimes use it that way if i am not absolutly sure what ends in my final archives).
Also worth a look at is the dependency-plugin already available in the maven distributions - mvn dependency:tree -Dverbose will give you pretty detailed information on the resolved dependencies (and probably version conflicts).
I'm trying to build a p2 repository from Tycho feature artifacts which are deployed in a remote Maven repository, without having to install the artifacts into the local Maven repository first (as in Tycho fails to resolve reference from product to eclipse-feature from a different reactor build), and without having to build all features and the repository together in a single reactor build.
Background
I have a multi-module Tycho project that builds several Eclipse plugins and features.
So that I can build each module separately - and so that I can reference OSGI artifacts in our Nexus Maven repository - I have enabled <pomDependencies>consider</pomDependencies> in my target platform, and added Maven dependencies between the modules or to the repository artifacts as usual with <dependency/> elements.
This works well - I can build the features or run the plugin tests without their dependant plugins being either in my local Maven repository or in the same reactor build. For example, when I run mvn test on a plugin test project, the relevant dependencies will be downloaded from Nexus and Tycho will happily resolve the Import-Packages in my manifest against these, build everything and run the tests. So far so good.
I would like to generate a p2 repository from these features so that I can install them in Eclipse from an update site, and the advertised way to do this is with the eclipse-repository packaging type. But here the plan falls down - Tycho doesn't seem to be able to resolve feature dependencies when building repositories in the same way as it can resolve plugin dependencies when building features. All attempts yield:
[ERROR] Cannot resolve project dependencies:
[ERROR] Software being installed: my.eclipse.repository raw:0.0.1.'SNAPSHOT'/format(n[.n=0;[.n=0;[-S]]]):0.0.1-SNAPSHOT
[ERROR] Missing requirement: my.eclipse.repository raw:0.0.1.'SNAPSHOT'/format(n[.n=0;[.n=0;[-S]]]):0.0.1-SNAPSHOT requires 'my.prj.eclipse.project.feature.feature.group 0.0.0' but it could not be found
There are two ways I have successfully built the p2 repository:
As part of the same reactor build. If I make the eclipse-repository a module within the Tycho multi-module project, and build the whole project at once with e.g. mvn verify, the features are resolved fine. But I don't want to do this. I would prefer to build modules individually. This means our CI can have an indicator for each module, and we can immediately see what module tests have failed in; it gives us opportunities for parallelising builds; and we avoid having to be constantly running builds on modules that haven't changed. It would be a shame to have to use a monolithic Maven build.
If I install the Tycho project into my local Maven repository, by running mvn install on the dependency. But I don't want to do this either, because this would mean the build is inherently irreproducable, as it would be sensitive to the state of the local repository. Our CI is currently set up to maintain a Maven repository per job and to completely wipe it at the start of execution, to shield us from this potential messiness.
So my question is: is there a third way? Is there any way I can get the Tycho plugin responsible for building eclipse-repository packaging types to download features from a remote Maven repository? Or any other way I can build the p2 repository from plugins that have been individually built and deployed to the Maven repository?
Things I've tried include:
specifiying the Maven feature depedencies as both jar and eclipse-feature
explicitly adding the features to the target platform, like
...
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<dependency-resolution>
<extraRequirements>
<requirement>
<type>eclipse-feature</type>
<id>my.prj.eclipse.project.feature</id>
<versionRange>0.0.0</versionRange>
</requirement>
...
The closest thing I've found to a decent solution is have a multi-module Tycho project that just contains the repository and features.
feature-project
|- feature1 (eclipse-feature)
|- feature2 (eclipse-feature)
|- repository (eclipse-repository)
Building this works - all plugins added to the top-level POM are downloaded from Nexus, available for inclusion in each feature and included in the generated repository.
However this is far from ideal because I can no longer store my features logically alongside my plugins; they need to be in separate project hierarchies. Attempting to build the features and repository separately, like with mvn clean verify -pl :feature1,feature2,repository, fails presumably due to Bug 380152.
Is there a better way? Any help would be gratefully received.
Many thanks
(As an aside: building the repository with mvn clean verify -Dtycho.localArtifacts=ignore will succeed if the features are present in the local Maven repository, and won't show you the warning that artifacts are being resolved from the local repo... is this a bug?)
I am pretty impressed by your thorough analysis. You've almost got everything covered which is possible with the current Tycho version (0.22.0) - except for the solution which is so unintuitive that I wouldn't have expected anyone to be able to guess it (see below). Note however that there is a small fix required to also make the solution work for SNAPSHOT artifacts.
But first, I'd like to provide some technical (and historical) background for what you have observed:
pomDependencies=consider only works for plug-ins: The use case for this functionality was to allow referencing plug-ins (or more precisely OSGi bundles) from Maven repositories. So when the flag is set and the project has dependencies to JARs, Tycho will check if they are OSGi bundles, generate the p2 metadata for them on-the-fly, and add them to the target platform. There is no similar support for feature JARs because these usually don't exist in Maven repositories.
But what about Tycho-built projects? These may deploy into Maven repositories! Yes, this is true, and this is why I tried to extend the pomDependencies concept to allow for what you are trying to do. The idea was that every time Tycho considers a POM dependency for the target platform, it also checks if the p2 index files ...-p2metadata.xml and ...-p2artifacts.xml exist. However this turned out to infer a massive performance penalty because it generally takes very long for a Maven repository server to figure out that an artifact does not exist. So the remote download was disabled, and replaced with a look-up in the local Maven repository. In this way, two Tycho builds could set -Dtycho.localArtifacts=ignore and would still be able to exchange the artifacts specified in the POM via the local Maven repository.
Knowing these implementation details, we get to the following solution: Instead of only adding a POM dependency from the repository to the feature artifact, you also need to add dependencies to the p2metadata and p2artifacts files. Example:
<dependencies>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
<classifier>p2metadata</classifier>
<type>xml</type>
</dependency>
<dependency>
<groupId>myproject</groupId>
<artifactId>myproject.feature</artifactId>
<version>0.1.0-SNAPSHOT</version>
<classifier>p2artifacts</classifier>
<type>xml</type>
</dependency>
</dependencies>
This makes Maven also download these p2 index files, so Tycho recognizes the main artifact as Tycho artifact. In this way, you can also get an eclipse-feature into the target platform via POM dependencies - at least almost: With 0.22.0, the repository build passes, but the feature.jar artifact is missing. I already debugged this issue, and it is easy to fix.
Obviously the syntax with three <dependency> elements for every actual dependency is not nice. It should be possible to boil this down to a single p2artifacts element - but this is more work. In case you are interested in this feature, you could open an enhancement request in Tycho's issue tracker.
According to the Maven lifecycle, mvn install will "install the package into the local repository, for use as a dependency in other projects locally". The local repository then stores all the jars that I downloaded remotely.
My modules have dependencies with other modules. When I run mvn package, nothing is stored in my local repository, but the dependencies appear to be fulfilled. So how does Maven handle the inter-module dependencies? Does Maven refer to the jars of each module from the built target directories or does it fetch them from another location?
Corey,
You are correct, going strictly by Maven docs implies mvn compile on:
parent_pom/
subA/
pom.xml
subB/
pom.xml # depends on subA
should fail since subA hasn't been pushed out to the local repo.
What's happening under the hood is that Maven uses the reactor to trick the build into looking into target dir of earlier submodules on the same build.
Beyond the scope of this particular question, the maven-reactor-plugin is one of the most opaque parts of Maven, but also one of the most powerful if you master it. You would do well to read up on it.
Hope that helps.
It depends on the phase you're executing. Before compile, Maven will fail, since there are no classes compiled. Between compile and package, the target/classes is used. For package and later, the target/artifactId-version.jar is used.
After working with Maven for a while, I am thrilled by the many features that Maven brings into the build architecture, particularly the dependency management. However, I have run into one issue again and again - how Maven resolves dependencies between multi-module projects. I am wondering if this is the big flaw of the current Maven implementation and/or if there is any satisfactory workaround.
Let's say I have a multi-module Maven project. The Parent pom contains three modules -- moduleA (jar), moduleB (jar), and moduleC(war). B depends on A and C depends on B. Simple enough? Now that I want to run the mvn dependency:go-offline at the parent project, which is supposed to resolve all the dependencies and bring them into the local .m2 directory. It fails because Maven complains that it cannot solve dependency for moduleA when it is acting on moduleB. Because all these modules belong to one groupId, I even try to use -DexcludeGroupIds=x.y.z to exclude these module dependencies, but it still fails at the same point.
I understand why Maven is complaining - moduleA is not built yet and thus there is no moduleA:jar artifact in my local or internal repository when go-offline goal is executed. But IMHO the plugin should treat these inter-module dependencies differently. In this case, it should simply ignore it. One might argues that I can simply do mvn clean install, which will install moduleA:jar into the local repository. After that, running mvn dependency:go-offline will work for sure. But that workaround defeats the purpose of this go-offline goal. This plugin allows us to resolve and pull dependencies into our local repository without building the whole project. I used dependency:copy-dependencies goal in another case and it has the same issue.
I also ran into similar issue in other scenarios: "mvn clean generate-source" could not resolve dependencies. When I ran mvn clean compile, everything works fine, but when I ran mvn clean generate-source, it fails because Maven cannot resolve inter-module dependency. In that case, the was caused by #requiresDependencyResolution in the antrun plugin.
Since both antrun plugin and dependency plugin are very popular in the Maven world, I am sure I am not the only one who have run into this issue. Anyone finds any solution/workaround?
Maven has the concept of a "reactor" where artifacts that have been built in a single run (e.g. maven package) are available for dependency resolution during the build. For example, if your dependency graph yields the build order moduleA moduleB moduleC, and you do mvn package, Maven will build moduleA, package its artifact and add it to the reactor, then build moduleB, package it and add it to the reactor, then the same for moduleC. This means moduleB has access to moduleA's artifact for dependency resolution, and moduleC has access to moduleA and moduleB. This only works if artifacts are actually built, i.e. when you run the package goal.
The problem is that when you don't run the package goal because you're not interested in the artifacts (as for your dependency:go-offline example), artifacts for modules that have been processed don't get built and thus not added to the reactor. I find this annoying as well; I think Maven should look at the POM files in its list of modules to build and look there as well; but it doesn't.
In short, the solution to your problem is to do mvn package dependency:go-offline . This will not install artifacts in your local repository (which I believe is very bad practice) but it will put them in the reactor for the duration of the build, meaning that Maven will be able to resolve dependencies from your moduleB to the moduleA that has already been built. The downside is every module will be tested and packaged, which is a lot of work when all you wanted is to do dependency:go-offline.
Either way, hope this helps.
This has been finally been resolved with Maven Dependency Plugin version 3.1.2.
You can make sure it's used by pinning the version in your pom.xml:
<build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.1.2</version>
</plugin>
</plugins>
</pluginManagement>
</build>
I have created a JIRA ticket with a sample project here:
https://issues.apache.org/jira/browse/MDEP-516
Please vote for it.
You explained why it doesn't work so you understand the issues. The problem for you is that it stops when it can't find A.jar but that will only happen when you get to building B. So there is a sort of, sometimes useful strategy.
You have to mess with A by itself. Just build A. Use your plan of loading dependencies and then building it.
Once it builds, you can move on to doing the same thing with B and then C. Step by step.
One thing to remember here is that its sometimes ok to build B with an old snapshot of A in the local repo. You only need the new snapshot of A build in the repo if there are signature changes or new stuff required by B.
There are some discussions here too: Maven Modules + Building a Single Specific Module
One final not that usually these sort of questions come up when people have builds that take too long. There are several ways to make builds go faster:
Get faster hardware. The build computer, the disk storage or the network speed are typical components that are cheaper to upgrade than waste the time taken in slow builds.
Make the build go faster by not building stuff that doesn't need rebuilding. (For example, I had a build that rebuilt all the generated code every time. I added some stuff into the build that kept it from doing that except when dependencies to the generated code changed.)
Speed up the tests. Sometimes this means breaking the tests into two parts. Part 1 is fast tests and part 2 is slow tests. Run the fast tests on every build and the slow tests before any checkin of code or release of artifacts.
Break a multi-module build into 2 or more separate builds and use human intelligence to decide when to rebuild things. This works well when some jars are stable and don't change much any more.
Fill in your own method to make the build go faster.
I doubt such functionality would ever be possible with Maven. Whilst your projects share a common parent and depend upon each other, Maven cannot possibly know where to find these projects in order to build them. It also cannot determine whether the projects just need to be built, or whether you've specified the wrong version number for your dependency.
That's going to be supported by dependency:go-offline goal starting from Maven Dependency Plugin v3.1.2. Related jira ticket MDEP-204 and patch 23b7ca8790ae14175ed8e3a20c75c6274efe5ad8 with fix.