maven copy-dependencies includeGroupIds and all transitive elements to those - maven

I try to figure out how to copy all dependencies to an explicit version and all the required dependencies.
For Example: My project requires version 3 of a third party lib, called foobar.
I want to copy the version 3 libraries to a folder named lib-foobar-${foobar.version}.
In this folder are those jars which are required to use foobar in version 3. That means the jar itself and all dependent jars which are declared in the foobar pom.
I currently use the org.codehaus.mojo:maven-dependency-plugin:2.1 with goal copy-dependencies in phase package.
My configuration is:
<configuration>
<outputDirectory>${project.build.directory}/lib-foobar-${foobar.version}</outputDirectory>
<includeGroupIds>com.foobar</includeGroupIds>
<excludeTransitive>false</excludeTransitive>
<excludeScope>test</excludeScope>
<includeScope>compile</includeScope>
</configuration>
I don't want to list all allowed and not allowed lib's because a step to a newer version takes place every month.
Are there any other tools which can do that or is there any dodge for that?

Big thanks to user944849.
It was very helpful to me for finding out the best solution in that case.
For everyone who is interested in my solution, here it comes:
At first I changed the plugin to maven-assemby-plugin and added a new assembly file
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<dependencies>
<dependency>
<groupId>com.foobar</groupId>
<artifactId>foobar-parent</artifactId>
<version>${foobar.version}</version>
<type>pom</type>
</dependency>
</dependencies>
<executions>
<execution>
<id>copy-foobar</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/assemble/foobar-libs.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
And the assembly file looks like:
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>standalone</id>
<formats>
<format>dir</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<includes>
<include>org.foobar</include>
</includes>
<outputDirectory>/lib-foobar-${foobar.version}</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<useTransitiveFiltering>true</useTransitiveFiltering>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
</assembly>
The key was the <useTransitiveFiltering> element, which resolves all transitive libs based on the included libs.

Related

In Maven, can I pack JEE project in Zip archive together SQL scripts?

Hello everyone first of all,
Suppose I have a JEE project contents some modules and a 'sql' folder inside:
I build all modules to a 'integration-version.ear' archive file.
There is the content of my target:
Using Maven, can I build something like this:
There is the β€˜3.15’ folder includes some SQL files from the same folder of my project.
Is it possible? What is configuration of Maven needed?
Note:
The original question was like: "In Maven, can I pack JEE project in RAR archive together SQL scripts?" but now the question is changed because RAR is a proprietary format and each tools that create RAR files must be commercially licensed.
Solution:
Using build-helper-maven-plugin
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>parse-version</goal>
</goals>
<configuration>
<propertyPrefix>parsedVersion</propertyPrefix>
</configuration>
</execution>
...
Using maven-assembly-plugin
pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version>
<inherited>false</inherited>
<configuration>
<descriptors>
<descriptor>${project.basedir}/../assembly.xml</descriptor>
</descriptors>
</configuration>
...
assembly.xml
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>release</id>
<baseDirectory>${file.separator}</baseDirectory>
<formats>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.basedir}/../sql/migrates/${parsedVersion.majorVersion}.${parsedVersion.minorVersion}</directory>
<outputDirectory>/scripts-${parsedVersion.majorVersion}.${parsedVersion.minorVersion}</outputDirectory>
<includes>
<include>**/*.sql</include>
</includes>
</fileSet>
</fileSets>
<dependencySets>
<dependencySet>
<outputFileNameMapping>${build.finalName}.${packaging}</outputFileNameMappin>
<includes>
<include>*:ear</include>
</includes>
</dependencySet>
</dependencySets>
</assembly>
There are 2 Maven plugins which can do that: maven-assembly-plugin and maven-rar-plugin
If you want to use maven-rar-plugin, i would create a new module with packaging rar and configure the plugin.
I personally think maven-assembly-plugin is easier to use.

How to package multi module in a zip file?

I have a multi module project.
Each module is packaged in a jar/war file in his own /target folder.
I want to take every module jar/war files and some config files, and put everything in a zip.
How to do that? I try with assembly plugin, but all i've managed to do so far is a zip of each module in their respective /target folder, so pretty useless ^^
First is to create a separate module let us call it dist add to the parent as new module and add all modules which you like to see in the resulting zip file as dependencies including the type war, jaretc.
The <packaging>pom</packaging> should be given cause this module does not contain any java code which needed to be compiled etc. We only want to create zip file which contains the given dependencies.
<project ..>
<packaging>pom</packaging>
<dependencies>
<dependency>
<groupId>module-1</groupId>
<artifactId>module-1-artifact</artifactId>
<version>${project.version}</version>
<type>war</type>
</dependency>
<dependency>
<groupId>module-2</groupId>
<artifactId>module-2-artifact</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>make-bundles</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>proj1-assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Add the following descriptor:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>dist-assembly</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>false</useProjectArtifact>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
</assembly>
The dependencies are needed to be sure the order of execution is correctly calculated by Maven.
If you need supplemental configuration in that zip file you can add those files into src/main/config for example and add fileSets parts in the assembly descriptor.

Maven building jar: merge duplicate resources from dependencies

I'm getting this problem:
I have a project with spring-boot. It works perfectly on eclipse, but I need to generate a jar with dependencies, so I add the maven-assembly-plugin configuration to the pom.
Some spring dependencies has in the META-INF a file called spring.schemas, and I need to merge all spring.schemas into one (spring-context, spring-beans, etc)
I tried this solution using maven-shade-pluggin and AppendingTransformer, and it merges all spring.schemas perfectly... But it has an issue, when I execute the jar, it fails with:
java.lang.IllegalStateException: Unable to open nested entry 'lib/spring-boot-starter-batch-1.2.4.RELEASE.jar'.
It has been compressed and nested jar files must be stored without compression.
Please check the mechanism used to create your executable jar file
So, shade plugin compress the jar, and spring-boot doesn't like it, and there's no way to turn off the compression in shade. I manually copy the shade spring.schemas generated by shade and I put it in the maven-assembly-pluggin generated and uncompressed jar with dependencies. It works.
Then I tried to include the generated spring.schemas into my resources folder, but it's allways overrided by the spring.schemas from spring-context.
I also tried this other solution to exclude spring.schemas from dependency jar using a descriptor XML in assembly, but it doesn't work:
<assembly
xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">
<id>distribution</id>
<formats>
<format>jar</format>
</formats>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<includes>
<include>org.springframework:spring-context</include>
</includes>
<unpack>true</unpack>
<unpackOptions>
<excludes>
<exclude>**/spring.schemas</exclude>
</excludes>
</unpackOptions>
</dependencySet>
<dependencySet>
<outputDirectory>/</outputDirectory>
<excludes>
<exclude>org.springframework:spring-context</exclude>
</excludes>
<unpack>true</unpack>
</dependencySet>
</dependencySets>
</assembly>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>org.springframework.batch.core.launch.support.CommandLineJobRunner</mainClass>
</manifest>
<compress>false</compress>
</archive>
<descriptors>
<descriptor>src/main/assembly/distribution.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
This are my dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</dependency>
<!-- MySql 5.5 Connector -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.35</version>
</dependency>
</dependencies>
Any ideas? Thanks in advance
You can use maven-assembly-plugin.
Add the plugin to your pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.1</version>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptors>
<descriptor>fat-jar.xml</descriptor>
</descriptors>
</configuration>
</plugin>
Add a fat-jar.xml descriptor next to your pom.xml:
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
<id>fat</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
<containerDescriptorHandlers>
<containerDescriptorHandler>
<handlerName>metaInf-spring</handlerName>
</containerDescriptorHandler>
</containerDescriptorHandlers>
</assembly>
Notice the metaInf-spring handler. It will merge any META-INF duplicate files having names starting with the spring word.
The reference documentation: https://maven.apache.org/plugins/maven-assembly-plugin/examples/single/using-container-descriptor-handlers.html
Instead of using maven-assembly-plugin and maven-shade-pluggin, I would try to use Spring Boot's own spring-boot-maven-plugin that can be used to create an executable β€˜fat’ JAR. See this reference documentation page, section "73.2 Create an executable JAR with Maven".
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
Or, if you don't use the spring-boot-starter-parent POM:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.2.5.RELEASE</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>

Assembly plugin generating 2 jars. One with classfiles. One with dependencies

I am having trouble building a jar using the assembly plugin. I have a bunch of system jar files that I want bundled alongside class files of the project.
I was looking to do something like:
final
- lib
- system.jar
- system1.jar
- com
- servlet
-- etc
I have tried to use the assembly plugin but it generates 2 jars. One with just the lib with my jars and another with the class files.
I have read through the assembly docs but I must be missing something.
Here is the build element I have in my pom.xml
<dependency>
<groupId>jackson-mapper-asl</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.9.4</version>
<scope>system</scope>
<systemPath>${project.basedir}/lib/jackson-mapper-asl-1.9.4-sc1.jar</systemPath>
</dependency>
//... etc
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<executions>
<execution>
<id>final-jar</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
This is my assembly file (assembly.xml)
<assembly>
<id>distribution</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<useProjectArtifact>false</useProjectArtifact>
<useTransitiveDependencies>true</useTransitiveDependencies>
<unpack>false</unpack>
<scope>system</scope>
<outputDirectory>lib</outputDirectory>
<includes />
</dependencySet>
</dependencySets>
</assembly>
Any idea how I can merge these jars together?
You would usually have a much simpler layout, such as:
final
- lib
-system.jar
-system1.jar
-project.jar
I.e. the project itself is a jar too, and included in the lib. In this case just add <useProjectArtifact>true/>useProjectArtifact> to the dependencySet.
If you really, really want to have classes copied in directly, try adding a fileset to the assembly, copying from target/classes into your root location
<fileSets>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>

How can I make the test jar include dependencies in Maven?

I have a project with src/main/java and src/test/java structure, and I managed to use maven-jar-plugin to build a jar of the test branch. However, I want to package the test jar so that all the dependencies are resolved. Is there a way I can tell maven-jar-plugin to include the dependencies??
Thanks!
Frank
I had a similar problem with integration tests I need to run on Hadoop. Our integration tests are located in the test folder of a separate integration test module, so what is required is a test-jar-with-dependencies to make our life easier.
I'm using the assembly plugin as mentioned by Michael-O. My assembly descriptor is located in src/main/assembly/test-jar-with-dependencies.xml and is a modification of the standard jar-with-dependencies descriptor that's part of the plugin:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>test-jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<!-- we're creating the test-jar as an attachement -->
<useProjectAttachments>true</useProjectAttachments>
<unpack>true</unpack>
<scope>test</scope>
</dependencySet>
</dependencySets>
</assembly>
This assembly relies on the test-jar being created as part of the module build. So I added the following to the module's pom.xml:
<!-- create a complete jar for testing in other environments -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptors>
<descriptor>src/main/assembly/test-jar-with-dependencies.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
You can do this: Create a jar assembly with the assembly plugin, have the dependencies unpacked, pack a new test jar and attach it to the reactor. You're done.
The descriptor for the packaging could look like this.
In a similar situation I ended up moving my test code to a separate jar and made it depend on the original one. You can use an aggregator project to ensure that tests are run when you build the main jar.
<dependency>
<groupId>me.wener.xxx</groupId>
<artifactId>xxx-core</artifactId>
<version>${xxx.version}</version>
<type>test-jar</type>
<!-- <scope>test</scope> -->
</dependency>
I use this to include the test jar.the important line is <type>test-jar</type>.I am not sure this is what you need.
3 years ago, but may help others.At least, it helped me. :-)
to include a test-jar dependency in your assembly specify the include filter of the assembly debendencySet as bellow:
...
<dependencySet>
<outputDirectory>/</outputDirectory>
<includes>
<include>*:jar:*</include>
<include>*:test-jar:*</include>
</includes>
</dependencySet>
...
The following worked for Maven 3
POM.XML
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
<phase>test-compile</phase>
</execution>
</executions>
</plugin>
ASSEMBLY FILE
<dependencySet>
<outputDirectory>demo/test-lib</outputDirectory>
<includes>
<!--test only dependencies (like netty)-->
<include>io.netty:netty-all</include>
<!-- the actual test jar-->
<include>${project.groupId}:${project.artifactId}:test-jar</include>
</includes>
<useProjectAttachments>true</useProjectAttachments>
<scope>test</scope>
</dependencySet>

Resources