It seems that including direct dependencies with provided scope is well understood. It also appears that including transitive dependencies with runtime scope is also easily accomplished.
But how can I include a dependency two levels of indirection away?
Example:
A --> B --> C
Where A depends on B (compile scope) and B depends on C (provided scope).
I want A to retrieve C (eg: download the jar locally), be it via assembly descriptor or maven-dependency-plugin:copy-dependencies or some other mechanism.
I've tried seemingly every combination of options for both of the aforementioned plugins. Neither approach covers this scenario. They both get B (even if B is changed to a provided dependency), and any compile scope dependencies of B, but not provided dependencies of B.
I suppose that I'm trying to do something similar to a shaded representation of my project but without unpacking dependencies.
Naturally I don't want to have to enumerate all of B's dependencies in A's pom - I'd like to retrieve (and then package) all dependencies implicitly and recursively.
You won't be able to do that. It is not a limitation of the maven-assembly-plugin, but the way Maven considers transitive dependencies. A transitive dependency that is of scope provided will be omitted, always (refer to this table in the documentation).
There is an open bug about this (MNG-2205) but I don't think it will be fixed anytime soon. This really is intended behaviour because provided dependencies, as per the name, as supposed to be provided at runtime.
Although Tunaki is quite right, I found a workable solution that's better than nothing, using a less well-known plugin (NOTE: only works with Maven <= 3.0.X)
This specifies two dependency blocks. The first pulls in normal compile deps including transitive, it's the same as using copy-dependencies. The second block specifically mentions B in addition to the normal pom dependency declaration (unfortunately, but at least both dependency mentions are in the same pom) and requests its provided deps:
<plugin>
<groupId>com.github.goldin</groupId>
<artifactId>copy-maven-plugin</artifactId>
<version>0.2.5</version>
<executions>
<execution>
<id>get-provided-dependencies</id>
<phase>generate-resources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<resources>
<resource>
<targetPath>${project.build.directory}/lib</targetPath>
<dependencies>
<dependency>
<includeScope>compile</includeScope>
</dependency>
<dependency>
<groupId>some.group</groupId>
<artifactId>artifact_i_call_B</artifactId>
<version>1.0</version>
<includeScope>provided</includeScope>
</dependency>
</dependencies>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
Docs for this plugin may only exist in the archive at this point, not sure what happened to it:
http://web.archive.org/web/20130826193436/http://evgeny-goldin.com/wiki/Copy-maven-plugin
Related
I am wondering if there is a maven enforcer rule or something similar to check my project for any 'opened' (not fixed) version in project (transitive) dependencies.
I would like to archive a stable reproducible build with maven, but I cannot guarantee this if a dependency of mine e.g. declares an open-ended version range for one of its dependencies.
A new release of that transitive dependencies would change the output of my 'otherwise' untouched build.
I haven't found any property or enforcer rule which fits this requirement.
Does anybody know how such a requirement can be done with maven?
Best bet would be to take the mvn dependency:list and fix all those versions in <dependencyManagement>
No transitive dependency way
To have a reproducible build, you should probably fix version of all your direct and indirect dependencies in dependencyManagement.
This will allow :
to fix version for range version dependencies
to avoid dependency-convergence issue.
To not forget flatten all your dependencies in dependencyManagement you can use banTransitiveDependencies rules from maven-enforcer-plugin.
If you have lot of dependency this could be painful to manage but maybe you can create a script to generate dependencyManagement section from mvn dependency:list
I created a new feature request for maven-dependency-plugin about this : https://issues.apache.org/jira/browse/MDEP-811
(See also : https://stackoverflow.com/a/35849405/5088764)
Fix range version only ?
Solution above works, but ideally we want to only fix version for :
convergence issue.
range version dependency.
You can add rules for dependency-convergence but AFAIK there is no kind of noRangeVersion rules.
I created a new feature request for maven-enforcer-plugin about this : https://issues.apache.org/jira/browse/MENFORCER-427
But waiting maybe this is possible to create your own rule : https://maven.apache.org/enforcer/enforcer-api/writing-a-custom-rule.html
(or maybe somebody already do that ?)
dependency-lock-maven-plugin ?
I didn't test this but maybe dependency-lock-maven-plugin could help to solve this issue.
See : https://stackoverflow.com/a/54580971/5088764
Short story
it is (still) "possible" to determine whether there are transitive dependencies with open version range:
...
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-assertions-generator</artifactId>
<version>2.1.0</version>
</dependency>
...
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<!-- version makes sense -->
<version>2.6</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
...
% mvn -X assembly:single -Dassembly.dryRun=true| grep 'setting version to'
[DEBUG] org.assertj:assertj-core:jar:2.9.1:compile
(setting version to: 2.9.1 from range: [2.1.0,2.99.0])
Long story
maven 3 had adopted Aether project and, unfortunately, there is no option to intercept or influence on dependency resolution process, basically "project object model" provides information about direct dependencies, but exhaustive information about transitive dependencies is hidden behind aether, that is the reason why you didn't find desired functionality among maven plugins.
I succeeded to get some relevant information from maven-assembly-plugin just because it's old versions are still compatible with modern maven, so, technically it is still possible to implement a plugin with required functionality or even take advantage of gmavenplus-plugin and write groovy scriptlet:
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.13.1</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<bindAllProjectProperties>true</bindAllProjectProperties>
<scripts>
<script><![CDATA[
def resolver = session.container.lookup(org.apache.maven.artifact.resolver.ArtifactResolver.class)
def artifacts = resolver.resolveTransitively(
project.dependencyArtifacts,
project.artifact,
project.managedVersionMap,
session.getLocalRepository(),
project.remoteArtifactRepositories,
null
).artifacts.findAll {
it.versionRange && it.versionRange.restrictions
&& !it.versionRange.recommendedVersion
&& (it.versionRange.restrictions.size() > 1
|| it.versionRange.restrictions[0].lowerBound
|| it.versionRange.restrictions[0].upperBound
)
}.each {
log.error("Found bad guy: $it -> $it.versionRange")
}
]]></script>
</scripts>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>3.0.9</version>
<type>pom</type>
<scope>runtime</scope>
</dependency>
</dependencies>
</plugin>
% mvn initialize
[INFO] Scanning for projects...
...
[INFO] Using plugin classloader, includes GMavenPlus and project classpath.
[INFO] Using Groovy 3.0.9 to perform execute.
Found bad guy: org.assertj:assertj-core:jar:2.9.1:compile -> [2.1.0,2.99.0]
UPD.
The idea of locking versions of all transitive dependencies in dependencyManagement from my perspective seems to be wrong. At first glance it looks attractive to run mvn dependency:list and put all it's output into dependencyManagement, no doubts, at next mvn package we will get the same artifact, but we also need to think about consequences of such "solution":
output of mvn dependency:list is far from ideal: maven tries to do it's best, but it relies on numbers mentioned in version tag and knowns nothing about compatibility, bugs and security issues - we should not blindly trust that output, instead we always need to check everything manually, and the problem is maven answers the question What will we get? when the actual question is Why did we get that?.
By locking versions of transitive dependencies in dependencyManagement we are taking responsibility for the things which we do not actually manage, the question is: how we are going to update versions of dependencies if we have locked versions of their dependencies? Why do we think we know better what to do than the developer of dependency?
The project has the following structure:
FatJarRootPom has dependencies for X and Y.
FatJarA and FatJarB all declare FatJarRootPom as their parent so they get its dependencies X and Y shaded into their JAR along with their own dependencies.
The issue is that, Shade Plugin will leave reference to parent in the dependency-reduced-pom.xml so FatJarA's pom will still explicitly depend on X and Y despite having them inside JAR already.
Flatten Plugin can get rid of parent so it would be optimal to run Shade Plugin on flattened pom. It will process all dependencies and then remove them from dependency-reduced-pom.xml. However, when I configure Flatten Plugin, Shade Plugin will not pick up its output and will still produce incorrect pom.
I have found this answer which suggests changing the invocation order of shade and flatten so that shade comes first, but it won't work since dependencies from FatJarRootPom will still be in the pom, which I am trying to avoid.
Maybe there is some other way of just removing parent section from dependency-reduced-pom.xml without flatten plugin?
Update: I have tried changing my FatJarRootPom to a POM dependency FatJarPom with transitive dependencies listed in it, and amusingly, Maven Shade Plugin will specifically skip POM dependency and will not remove it from dependency reduced POM, even as it would mean that all shaded dependencies will absolutely appear as dependencies of fat-jar artifact.
I have overlooked that Flatten Plugin has "fatjar" mode which will strip all dependencies. It is a very coarse grained approach, but it should do in my use case.
So the solution is placing
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
<version>1.2.7</version>
<executions>
<execution>
<id>flatten</id>
<phase>package</phase>
<goals>
<goal>flatten</goal>
</goals>
<configuration>
<updatePomFile>true</updatePomFile>
<flattenMode>fatjar</flattenMode>
</configuration>
</execution>
...
after the corresponding Shade Plugin section.
I have several Maven projects, say a,b,c, inheriting from a single parent (let's call it parent), and also being modules (of a different project than parent, let's call it super).
These projects all have a pom packaging. Each of these projects has specific configuration, but they also have a common part. To be more speficic, each project two JMeter test configuration files: one specialized for the given project, and another one that is common and identical for all projects.
The problem is - how should I configure the POMs so this common config file is shared among the projects?
A workaround would be to merge all of them into super, and use profiles. However, in this case, I would have to do a separate build for each configuration manually (whereas now I can just build super).
There are similar questions, like this one, but they deal with the jar plugin, which is not relevant for this case.
Structure, for reference:
POM Inheritance:
parent
|
-------------
| | |
a b c
File structure:
super
|
|-a
|
|-b
|
|-c
I have used the maven-remote-resources-plugin for a similar purpose. Create a separate resources project (com.company:resourceProj) of type jar. Put the JMeter resource files in /src/main/resources.
/src/main/resources/common.properties (your filenames obviously)
/src/main/resources/a.properties
etc.
Follow the directions in the example to create the bundle.
Now, add this config to your parent POM (in a testing profile if you want):
<properties>
<shared.resources.dir>${project.build.directory}/shared-resources</shared.resources.dir>
</properties>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-remote-resources-plugin</artifactId>
<executions>
<execution>
<id>load-resources</id>
<phase>initialize</phase>
<goals>
<goal>process</goal>
</goals>
<configuration>
<resourceBundles>
<resourceBundle>com.company:resourceProj:version</resourceBundle>
</resourceBundles>
<attached>false</attached>
<outputDirectory>${shared.resources.dir}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Now, tell Maven these are test resources. If your test resource elements are consistent across the modules, this can go in the parent too, if they are different it goes in the module POM. (In my experience with Maven 3 resources defined in a child project take precedence over the parent's; they aren't merged.)
<testResources>
<testResource>
<directory>${shared.resources.dir}</directory>
<includes>
<include>common.properties</include>
<include>${module.file}.properties</include>
</includes>
</testResource>
<!-- any other test resources here -->
</testResources>
In the child module, define the resources module property (this is module a):
<properties>
<module.file>a</module.file>
</properties>
Adapt this to meet your use case.
---- Edit ----
If the configuration is placed into a parent POM, the parent POM may fail to build depending on what configuration is provided by the child. When we are building the shared base/parent projects we don't want to require that all of the properties that should be provided by child projects (inheriters) are defined. So we activate this profile when building the shared projects to bypass anything that only applies to children.
To do this, add an empty file pom-packaging.marker to the parent project's basedir. Then add this profile to the parent POM. When the parent project is built, Maven will find the marker file, enable the profile, and disable all of the executions included in the profile. When a child project is built, the marker file doesn't exist, so the configuration in the main part of the POM will take effect.
I've used this technique with the Enforcer plugin as well - the parent defines the enforcer rules that should be applied to projects inheriting from the parent, but cannot satisfy the rules when it is built. If the plugin provides a "skip" property, you may enable that in this profile instead of using phase = none in plugin configuration.
<profile>
<id>pom-packaging</id>
<activation>
<file>
<exists>pom-packaging.marker</exists>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-remote-resources-plugin</artifactId>
<executions>
<execution>
<id>load-resources</id>
<phase>none</phase> <!-- disables this execution -->
</execution>
</executions>
</plugin>
.... other plugin executions here ....
</plugins>
</build>
</profile>
The idea with import scope dependencies is that you can put shared resources into a separate project, which is then imported by a number of other ones; I was thinking you could include your shared config file in this way.
You create a new project with packaging pom (maybe at the same level as the parent?), and then include it in the parent's dependencyManagement section with scope import. Each of your child projects can then receive it by inheritance. It might seem like overkill to make an entire project for just a single file, but I wouldn't have a problem with that.
I haven't actually tried this with a tree of pom-packaged projects, so you might have to play around a bit, but the approach I think is sound. There's a (very extensive) example here:
Importing Dependencies
We have many modules in our project and we want to enable test-jar creation for some of those. I tried adding maven jar plugin to the parent pom's pluginmanagement
<plugins>
<pluginManagement>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
But, now it is creating test-jar for all the modules.
As per documenation: Plugin Management contains plugin elements in much the same way, except that rather than configuring plugin information for this particular project build, it is intended to configure project builds that inherit from this one. However, this only configures plugins that are actually referenced within the plugins element in the children. The children have every right to override pluginManagement definitions.
It should not create test-jar for the modules we haven't included in, but somehow it does. None of my modules has maven-jar-plugin added in its build module.
To stop it creating test-jar for all, I can add it only in the module I am interested in.
Is there any other better solution to achieve this?
Normally, what your describing shouldn't be possible. If I had to guess, I would say that since your packaging type is jar configuration for the maven-jar-plugin gets inserted into your effective pom. Then, since the configuration is there, it also uses the configuration from the plugin management section. Running mvn help:effective-pom might provide some insight into whether or not this is true.
I would also make sure that no parent in your hierarchy contains any concrete configuration for the maven-jar-pom that would be inherited by children. If the child inherits a concrete section, I think that will also trigger it to include the plugin management configuration.
I currently have a Java testing library which is built with Maven, and distributed as a jar. My project depends on a very common library (Objectweb ASM), and I've experienced problems where an earlier and incompatible version of ASM is already on the classpath. Thus, I've started usings the jarjar-maven-plugin to create jar, repackaging ASM internally where it cannot conflict with another version of ASM.
This executes fine, and my library can be pulled in as a dependency with no problem.
However, because my project has compile-scope dependencies on ASM, whenever a client project adds my library, the transitive dependencies are all pulled in as well. So, hypothetically, if they use a particular version of ASM, and they also add the version I depend on to the classpath, they have undefined behaviour. I'd like to avoid this situation, and allow clients to depend on the JarJar'd artifact without having Maven pulling down the transitive dependencies both unnecessarily and potentially dangerously.
How do I create a JarJar'd artifact which users can depend on without pulling transitive dependencies?
I found a solution to this problem by ditching the jarjar-maven-plugin, and reverting to the maven-shade-plugin. This allows repackaging classes within your own namespace, setting the main class of the jar, and crucially, rewriting the generated pom to not include the compile time dependencies which are now bundled.
The part of my pom.xml which acheived this is:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
<createDependencyReducedPom>true</createDependencyReducedPom>
<relocations>
<relocation>
<pattern>org.objectweb.asm</pattern>
<shadedPattern>${repackage.base}.org.objectweb.asm</shadedPattern>
</relocation>
</relocations>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${package.base}.my.MainClass</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
The important parts of this configuration are:
shadedArtifactAttached which when set to false, means the shaded jar will replace the main artifact that would normally be produced. This defaults to false but it's worth pointing out.
createDependencyReducedPom which when set to true means that when the shaded jar is deployed or installed, the pom.xml which is deployed will not include the compile-scope dependencies which have been repackaged into the jar.
relocation these elements configure how files within the dependencies are repackaged into the shaded jar. In the above example any class whose canonical name begins with org.objectweb.asm will be moved to ${package.base}.org.objectweb.asm, and thus when packaged in the jar will have the equivalent file path within the jar.
With this configuration, when my project is deployed, when clients declare a compile-scope dependency on my project, it only pulls in the shaded jar, and no transitive dependencies.
Consider trying the maven-shade-plugin instead, which allows all sorts of fine control.
Perhaps setting the <optional> attribute will work in your case. Specifying something like the following in your java testing library pom.
<dependencies>
<dependency>
<groupId>asm.group</groupId>
<artifactId>asm</artifactId>
<version>x.y</version>
<optional>true</optional>
</dependency>
...
</dependencies>