How to use Maven assembly plugin with multi module maven project - maven

I am new to maven and spent ~3 days in generating the zip file with assembly plugin refering to http://www.petrikainulainen.net/programming/tips-and-tricks/creating-a-runnable-binary-distribution-with-maven-assembly-plugin/ My project is multi module, so I also referred to Managing multi-module dependencies with Maven assembly plugin
Still I have several inefficiencies. Below is assembly.xml (which I inherited from 1st link)
<assembly>
<id>bin</id>
<!-- Generates a zip package containing the needed files -->
<formats>
<format>zip</format>
</formats>
<!-- Adds dependencies to zip package under lib directory -->
<dependencySets>
<dependencySet>
<!-- Project artifact is not copied under library directory since it is
added to the root directory of the zip package. -->
<useProjectArtifact>false</useProjectArtifact>
<outputDirectory>lib</outputDirectory>
<unpack>false</unpack>
</dependencySet>
</dependencySets>
<moduleSets>
<moduleSet>
<!-- Enable access to all projects in the current multimodule build! <useAllReactorProjects>true</useAllReactorProjects> -->
<!-- Now, select which projects to include in this module-set. -->
<includes>
<include>com.XX:YY-main</include>
</includes>
<!--<binaries> <outputDirectory>YY-main/target</outputDirectory> <unpack>false</unpack>
</binaries> -->
</moduleSet>
</moduleSets>
<fileSets>
<!-- Adds startup scripts to the root directory of zip package. The startup
scripts are located to src/main/scripts directory as stated by Maven conventions. -->
<fileSet>
<directory>${project.build.scriptSourceDirectory}</directory>
<outputDirectory>conf</outputDirectory>
<includes>
<include>*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory></outputDirectory>
<includes>
<include>log4j.properties</include>
</includes>
</fileSet>
<!-- adds jar package to the root directory of zip package -->
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory></outputDirectory>
<includes>
<include>*with-dependencies.jar</include>
</includes>
</fileSet>
</fileSets>
Below is the pom.xml (primary or main)
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration> <descriptors>
<descriptor>YY-main/src/main/assembly/assembly.xml</descriptor>
<!-- <descriptor>DummyAssembly.xml</descriptor> -->
</descriptors>
</configuration>
</plugin>
Questions:
1)Since assembly.xml is referred in primary pom, all submodules also get zipped. How can I specify only certain submodules to be zipped, while all others get ignored. I tried to move YY-main/src/main/assembly/assembly.xml from main pom to child/submodule pom and have dummyassembly.xml in main which does nothing. But that is not working (possibly because dummyassembly.xml is missing some required lines). tried few things, but nothing seem to work
2)Also in assembly.xml (dependencyset), it is copying all the libraries to "lib" folder. How can I avoid this. again I tried few things (exclusion..) based on http://maven.apache.org/plugins/maven-assembly-plugin/assembly.html#class_dependencySet but nothing worked.
Can some one provide me with specific statements that I should change in my pom or assembly files to address 1 and 2-thanks

The basic thing you should change is to create a separate module where you do the packaging which will look like this.
+-- root (pom.xml)
+--- mod-1 (pom.xml)
+--- mod-2 (pom.xml)
+--- mod-assembly (pom.xml)
The pom in the mod-assembly will look like this:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.test.parent</groupId>
<artifactId>root</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>dist</artifactId>
<packaging>pom</packaging>
<name>Packaging Test : Distribution</name>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>module-one</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>module-two</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>make-bundles</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>proj1-assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
That will solve the problem with running maven-assembly-plugin in every child and the problem with the dummy descriptor. Here you can find a real example of such structure.
Furthermore having the modules in one folder and the dependencies in an other folder you can use a assembly descriptor like this:
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<useProjectArtifact>false</useProjectArtifact>
<useTransitiveDependencies>true</useTransitiveDependencies>
<outputDirectory>lib</outputDirectory>
<unpack>false</unpack>
<excludes>
<exclude>${project.groupId}:*:*</exclude>
</excludes>
</dependencySet>
</dependencySets>
<moduleSets>
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<binaries>
<outputDirectory>modules</outputDirectory>
<unpack>false</unpack>
</binaries>
</moduleSet>
</moduleSets>

I think the post How to Assemble Multi-Module Project with Maven might answer your question with an example.
My answers to your questions:
1) You should add the Assembly plugin in the parent pom within the build and pluginManagement tags to reuse it in the modules; the post mentioned above is a good example on how to do it. You can also take a look at 8.6 Best Practise chapter, topic 8.6.2, of the online book 'Maven: The Complete Reference'.
2) The exclusion tag might be tricky. Again, take a look at 8.5 Controlling the Contents of an Assembly chapter of the online book 'Maven: The Complete Reference'. For example, the subtopic 8.5.4, about dependencySets mentions how to fine tune dependency includes and excludes for dependencySets; again, the subtopic 8.5.5, now about moduleSets, shows how to use it in that context.

Related

Seperate the jars using maven assembly plugin

I need to separate the jars into different folders.
Inside my project, I have several modules, i.e: module1, module2, and assembly.
By using maven assembly plugin, I would like to put generated jar from modules1 and module2 into target/modules and all the dependencies into target/dependencies.
How can I achieve this requirement?
Thanks
Based on the fact that your project is structured like :
project
|- module1
|- module2
|- assembly
|- pom.xml
|- src
|- assembly
|- bin.xml
assembly should depend on module1 and module2; set these dependencies in assembly/pom.xml :
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>module1</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>module2</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
You must also add maven-assembly-plugin in assembly/pom.xml :
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<descriptors>
<descriptor>src/assembly/bin.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
assembly:single is bound on package phase, to create the assembly when running mvn package.
Finally, define assembly/src/assembly/bin.xml as follows :
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.0.0 http://maven.apache.org/xsd/assembly-2.0.0.xsd">
<id>bin</id>
<formats>
<format>dir</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>modules</outputDirectory>
<includes>
<include>${project.groupId}:*:*</include>
</includes>
<excludes>
<exclude>${project.groupId}:${project.artifactId}:*</exclude>
</excludes>
</dependencySet>
<dependencySet>
<useTransitiveDependencies>true</useTransitiveDependencies>
<outputDirectory>dependencies</outputDirectory>
<excludes>
<exclude>${project.groupId}:*:*</exclude>
</excludes>
</dependencySet>
</dependencySets>
</assembly>
format defines the format you want for the assembly (here a directory, but could be tar.gz, zip, ...)
first dependencySet defines a folder modules/ where all artifacts from the same groupId will be put. Here you can also control if you want only some of the artifacts (for example if you want only module1). assembly JAR is excluded from this folder, as this module is only used to create the assembly
second dependencySet defines a folder dependencies/ where all dependencies (and transitive dependencies) will be put. Dependency here means artifact with a different groupId, as states the excludes clause
mvn package will then generate the assembly (in assembly/target/ folder), named assembly-${project.version}-bin, with the structure you want.

Maven package phase: how to depend on package artifacts from other projects

I have this Maven project structure:
-- top
-- a
produces a.jar and a-capsule-fat.jar
-- b
produces b.jar and b-capsule-fat.jar
-- pkg
produces all.tar.gz, which contains a-capsule.jar and b-capsule.jar
I am using the capsule-maven-plugin to build fat jars in a couple of projects, as shown above. Normally capsule is run during the package phase. I then want to assemble the capsule jars into a tar.gz for deployment purposes. I am using the maven-assembly-plugin in project pkg to make the tar.
But the maven-assembly-plugin also normally runs during the package phase, and it's running before the capsule jars are created.
Can I specify a assembly dependency or ordering that will force maven to create the capsule jars first? Alternatively I could build the assembly in a later phase, but there are no really suitable later ones (in install? there is no post-package).
POST-ANSWER: I am including some pieces of the working code for posterity:
dependencies in pkg/pom.xml:
<dependency>
<groupId>thegroup</groupId>
<artifactId>a</artifactId>
<version>theVersion</version>
<type>jar</type>
<classifier>capsule-fat</classifier>
</dependency>
<dependency>
<groupId>thegroup</groupId>
<artifactId>b</artifactId>
<version>theVersion</version>
<type>jar</type>
<classifier>capsule-fat</classifier>
</dependency>
assembly plugin settings in pkg/pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>build-tar</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptors>
<descriptor>src/main/assembly/pkg.xml</descriptor>
</descriptors>
</configuration>
</plugin>
pkg.xml (referenced above):
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>ustc-archive-pkg</id>
<formats>
<format>tar.gz</format>
</formats>
<dependencySets>
<dependencySet>
<includes>
<include>*:jar:capsule-fat</include>
</includes>
</dependencySet>
</dependencySets>
</assembly>
Make sure the definition for the capsule-maven-plugin appears before the maven-assembly-plugin. When there are executions bound to the same phase, Maven uses the order of the plugin definitions in the POM to break the tie.
---- edit ----
Make sure the dependencies on a and b include a classifier:
<dependency>
<groupId>theGroup</groupId>
<artifactId>a</artifactId>
<version>theVersion</version>
<classifier>capsule-fat</classifier>
</dependency>
See if that does it.

Including an unpacked War in the Assembly

I have a project which builds a war (no problem). And, that war needs to be packaged with a few shell scripts. Because that war contains properties files that vary from site-to-site, we can't simply install the war as is, but munge the properties files in it. That's what the shell scripts do.
I'd like to package my war in my assembly as a unpacked war. I see <unpacked> in the Assembly Descriptor, but I haven't been able to get that to work.
Here's my first bin.xml where I just packed the war as is. This works fine:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/assembly/scripts</directory>
<includes>
<include>deploy.sh</include>
<include>lock_build.sh</include>
<include>description.sh</include>
<include>url-encode.pl</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.build.directory}/${project.artifactId}-${project.version}</directory>
<outputDirectory>${project.artifactId}</outputDirectory>
</fileSet>
</fileSets>
</assembly>
Here's my first attempt at unpacked:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/assembly/scripts</directory>
<includes>
<include>deploy.sh</include>
<include>lock_build.sh</include>
<include>description.sh</include>
<include>url-encode.pl</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
<moduleSets>
<moduleSet>
<includes>
<include>{$project.groupId}:${project.artifactId}:war</include>
</includes>
<binaries>
<includes>
<include>${project.build.directory}-${project.artifactId}-${project.version}.${project.packaging}</include>
</includes>
<outputDirectory>${project.artifactId}</outputDirectory>
<unpack>true</unpack>
</binaries>
</moduleSet>
</moduleSets>
</assembly>
Here's my last try at getting the unpacked war:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/assembly/scripts</directory>
<includes>
<include>deploy.sh</include>
<include>lock_build.sh</include>
<include>description.sh</include>
<include>url-encode.pl</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
<moduleSets>
<moduleSet>
<binaries>
<attachmentClassifier>war</attachmentClassifier>
<outputDirectory>${project.artifactId}</outputDirectory>
<unpack>true</unpack>
<includeDependencies>true</includeDependencies>
<dependencySets>
<dependencySet/>
</dependencySets>
</binaries>
</moduleSet>
</moduleSets>
</assembly>
In each of these last two attempts, I am only packaging the scripts and the war isn't coming over.
I know I could use the ${project.build.directory}/${project.artifactId}-${project.version} directory which contains almost all of the files in the war, but it doesn't contain my MANIFEST.MF entries which includes information linking the war back to a particular Jenkins build and Subversion revision.
What do I need to do to include an unpacked war into my assembly?
Another Attempt
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/assembly/scripts</directory>
<includes>
<include>deploy.sh</include>
<include>lock_build.sh</include>
<include>description.sh</include>
<include>url-encode.pl</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
<dependencySets>
<dependencySet>
<outputDirectory>${project.artifactId}</outputDirectory>
<unpack>true</unpack>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
</assembly>
When I ran this, I got:
[INFO] ------------------------------------------------------------------------
[INFO] Building My Project 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-assembly-plugin:2.5.2:single (default-cli) # myproj ---
[INFO] Reading assembly descriptor: src/assembly/bin.xml
[WARNING] Cannot include project artifact: \
com.vegicorp:myproj:war:1.0.0; \
it doesn't have an associated file or directory.
[INFO] Building zip: ~/workdir/trunk/myproj/target/archive/myproj.zip
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
Inside the zip are all the jars in the war nice and unpacked, but not my unpacked war.
I know I should be able to add the unpacked war into my assembly. I see that unpack option, and I know it works for other dependencies. However, it looks like I can only access it via a <dependencySet> or a <moduleSet>. I should be able to specify my project as its own module. There must be something I am doing wrong.
Spleen Vent
This is the big thing I hate about Maven: Maven does a great job hiding things from you which is nice because it prevents you from doing stuff you shouldn't. I hate it when developers build Ant build.xml files because most developers don't understand how to do a build, and the build.xml becomes an unreadable mess. With Maven, this isn't an issue. Just configure your project, and Maven will take care of this for you.
But sometimes Maven is like a black box with a bunch of levers and buttons. You sit there pushing and pulling levers and buttons trying to figure out how to get it to do something you want to do. I spend way too much of my time trying to help developers to configure their Maven projects. They want to do something a little different like use hibernate or build source from WSDL files, and have no idea how to get Maven to do what they want.
One developer describes it as a self driving car which can't quite go where you want. You may even see the destination out the window, but you can't figure out how to manipulate the car's destination to get you there.
What I want to do should be easy. I just want to create an assembly, and instead of using the packed war file, I want it unpacked.
In Ant, this can be accomplished in a single task. In Maven, it's a mysterious process. I think I'm close, I am probably missing one little configuration parameter that will make it all work, but I've already spent hours working on this.
What I ended up doing
I used the maven-dependency-plugin to unpack my war. I wasn't sure whether this would work because I didn't want the dependency plugin downloading the war, but it seems to understand that when I specify my war, I am talking about the current build, and nothing is downloaded. (I don't even have the war in our Maven repo).
I also had to upgrade Maven from 2.x to 3.x because I need to make sure that the maven-dependency-plugin ran before the maven-assembly-plugin. Since both run in the packaging phase of the build, Maven 2.x can't guarantee the order the plugins run in.
Once I used that plugin, all I had to do was specify the directory where I unpacked the war in my assembly plugin, and it was included in my zip.
I still want to know how to use the <unpack> entity in the assembly plugin itself instead of having to use another plugin to unpack my war. If anyone can tell me how to do the unpack in the assembly file itself, I'll mark that as the correct answer.
pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<id>unpack</id>
<phase>package</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/unwar/${project.artifactId}</outputDirectory>
<artifactItems>
<artifactItem>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
<type>war</type>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.2</version>
<configuration>
<finalName>${project.artifactId}</finalName>
<appendAssemblyId>false</appendAssemblyId>
<outputDirectory>${project.build.directory}/archive</outputDirectory>
<descriptors>
<descriptor>src/assembly/bin.xml</descriptor>
</descriptors>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
bin.xml (My Assembly)
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<fileSets>
<fileSet>
<directory>${project.basedir}/src/assembly/scripts</directory>
<fileMode>0755</fileMode>
<lineEnding>lf</lineEnding>
<includes>
<include>deploy.sh</include>
<include>lock_build.sh</include>
<include>description.sh</include>
<include>url-encode.pl</include>
</includes>
<outputDirectory>/</outputDirectory>
</fileSet>
<fileSet>
<directory>${project.build.directory}/unwar</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
</assembly>
The ACTUAL Answer
Thanks to khmarbaise's link (see the comment below), I copied that project's assembly plugin and it almost worked. Like my attempt, it unpacked all the runtime jars, and not just the one I wanted. However, it also unpacked my war too.
There were only two differences between that link's answer and my attempt:
They included <useProjectArtifact>true</useProjectArtifact> and I didn't. However, this defaults to true. Removing it still allowed it to work.
They included a / in front of the directory name in the <outputDirectory>. Removing this made no difference.
This meant that it now matched what I had previously tried. So, why was it working this time.
Turns out, I was testing the assembly changes by simply running mvn assembly:single. After all, why do the whole build and repackage when I am simply trying to get the assembly to work. When you run just mvn assembly:single -- even though everything is already packaged, you get this error:
[WARNING] Cannot include project artifact: \
com.vegicorp:myproj:war:1.0.0; \
it doesn't have an associated file or directory.
And your war isn't unpacked. However, if you put the assembly into the package phase, and then run mvn package, everything works out just nifty.
I then spent time trying to just get my war and not all the associated runtime stuff with it. I used <includes/> to do this, but because I have a war and not a jar, I had to include a classifier in my <include>.
At last, I have everything working. I have this in my assembly:
<dependencySets>
<dependencySet>
<outputDirectory>${project.artifactId}</outputDirectory>
<unpack>true</unpack>
<scope>runtime</scope>
<includes>
<include>${project.groupId}:${project.artifactId}:*:${project.version}</include>
</includes>
</dependencySet>
</dependencySets>
And as long as I run mvn package, it works.
This is way better than what I had with the maven-dependency-plugin. Now, all of the information having to do with the assembly is in the assembly XML file.

Generating OSGi bundles distribution with maven-assembly-plugin

I have a multi-module project, where each module is packaged as an OSGi bundle using the Apache Felix maven-bundle-plugin. The whole project is built using a parent POM that lists the above-mentioned modules. Some modules contain configuration resources (e.g. .properties files) that should not be jarred inside the bundles for deployment but rather externalized in a dedicated config folder. My goal is to create a distribution folder (possibly, a zip file) that would look something like this:
my-app-distribution
/bundles
module1-bundle.jar
module2-bundle.jar
etc.
/conf
external1.properties
external2.properties
etc.
where the properties files under the /conf directory are hand-picked files from the individual modules' /target folders. The reason the .properties files need to be picked up from the target folders vs. the src folders is that I am using Maven resource filtering, and the source property files contain ${..} placeholders for environment-specific values. Those placeholders are properly resolved during the build process - per build profiles - and the target/ folders contain actual environment-specific values.
I've done such distribution file manipulations many times - for distributions with executable JARs, etc. In this case I wanted to use the "moduleSets" configuration of the assembly descriptor - it is easy to pull all binaries/jars into a single distribution folder using moduleSet/binary descriptor. It is also easy to exclude certain files from being packaged into an OSGi bundle - in the maven-bundle-plugin. The only issue I am stuck with is creating the /conf distribution folder and collecting the necessary properties files there. I have tried to use "fileSets" inside the "moduleSet/sources" descriptor to include only specific files from **/target of each module, but that didn't seem to work.
Anyone have a suggestion/advice? There's got to be an easy way. Or should I not use at all?
Thanks,
CV
#PetrKozelka I am not sure that extracting configuration files specific to different bundles into a separate module is a good idea. The whole point of OSGi is for bundles to be independent and potentially reusable - both in development and distributions. It only makes sense that - in the source code - the functionality implementation and related configuration files are grouped together. For a particular distribution though I might need to extract some of the files - if there is a requirement for admins to have control of certain parameters. That may be different for a different distribution/application. The assembly configuration may change, but the bundles/sources would stay the same. Also, each bundle may potentially be developed and used separately, not all bundles have to always be part of the same uber project - as you seem to assume. What you are suggesting seems to fall into the same old category of packaging enterprise applications by the type of artifacts (e.g. "model", "services", "dataaccess", "config" etc.), not by functional domain/features. Such approach works ok within a single application/project, but fails on the enterprise level where there is often a need to reuse subsets of vertical components (split by functional domains).
To your point of being dependent on the file layout in the modules, I agree that there should be no such dependency. Files could be hand-picked by their explicit name or naming convention - per very specific distro requirements. (Which is exactly the case I am facing.)
I have actually figured out how to do it more or less elegantly. Posting the solution below in case someone else is looking to solve a similar problem.
SUMMARY
I am using the maven-assembly-plugin to extract the binaries (bundle JARs) from the individual modules and package them in the <my-distribution-folder>/bundles directory. In each module where some resource files should be externalized, I consolidate such files under the /src/main/resources/external directory, and use maven-resources-plugin to copy those resources during the packaging phase to the auto-generated directory in my dedicated distribution module that contains the assembly.xml descriptor file and is also built as part of the top project. I use maven-clean-plugin in the parent POM to clear the contents of the distribution staging directory during the CLEAN phase of the top-level project build.
MAVEN CONFIGURATION
Inside each bundle's module POM that contains resources that need to be externalized I add the following resource management configuration:
<build>
<defaultGoal>install</defaultGoal>
<!--
enable resource filtering for resolving ${...} placeholders with environment-specific values
exclude any files that must be externalized
-->
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
<excludes>
<exclude>external/*.*</exclude>
</excludes>
</resource>
</resources>
...
<plugins>
<!-- Copies contents of resources/external to dedicated folder defined by property in parent -->
<!-- externalized resources will be packaged according to assembly instructions -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>
${project.parent.basedir}/${externalizableResourcesStageDir}
</outputDirectory>
<resources>
<resource>
<directory>src/main/resources/external</directory>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
<!-- builds a JAR file for this bundle -->
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<configuration>
<instructions>
<Bundle-SymbolicName>${project.groupId}.${project.artifactId}</Bundle-SymbolicName>
<Import-Package>*</Import-Package>
<Export-Package>
${project.groupId}.thismodulepackage*;version=${project.version}
</Export-Package>
</instructions>
</configuration>
</plugin>
</plugins>
</build>
where externalizableResourcesStageDir is a property defined in the top/parent POM. In the project, I include a special distribution module with the following structure:
distribution
/ext-resources (target auto-generated dir for external resources from modules)
/src
/assemble
assembly.xml (assembly descriptor)
The assembly.xml file looks like this:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2
http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>bin</id>
<!-- generate a ZIP distribution -->
<formats>
<format>zip</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<baseDirectory>/</baseDirectory>
<moduleSets>
<moduleSet>
<!-- Enable access to all projects in the current multi-module build -->
<useAllReactorProjects>true</useAllReactorProjects>
<!-- select projects to include-->
<includes>
<include>myGroupId:myModuleArtifactId1</include>
<include>myGroupId:myModuleArtifactId2</include>
...
</includes>
<!-- place bundle jars under /bundles folder in dist directory -->
<binaries>
<outputDirectory>${artifactId}/bundles</outputDirectory>
<unpack>false</unpack>
</binaries>
</moduleSet>
</moduleSets>
<!-- now take files from ext-resources in this module and place them into dist /conf subfolder-->
<fileSets>
<fileSet>
<directory>ext-resources</directory>
<outputDirectory>${artifactId}/conf/</outputDirectory>
<includes>
<include>*</include>
</includes>
</fileSet>
</fileSets>
</assembly>
The distribution module's POM would look like this:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>myGroupId</groupId>
<artifactId>parentArtifactId</artifactId>
<version>...</version>
</parent>
<groupId>myGroupId</groupId>
<artifactId>distribution</artifactId>
<version>...</version>
<packaging>pom</packaging>
<name>Distribution</name>
<description>This module creates the <MyProject> Distribution Assembly</description>
<url>http:...</url>
<!-- NOTE: These dependency declarations are only required to sort this project to the
end of the line in the multi-module build.
-->
<dependencies>
<dependency>
<groupId>myGroupId</groupId>
<artifactId>myModuleArtifactId1</artifactId>
<version>${project.version}</version>
</dependency>
...
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>dist-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/assemble/assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
The parent POM would list all the bundle modules, plus the distribution module and also define the assembly plugin:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>myGroupId</groupId>
<artifactId>myParentId</artifactId>
<version>...</version>
<packaging>pom</packaging>
<properties>
...
<!-- directory where build may place any sub-modules' resources that should be externalized -->
<!-- those resources may be picked up by maven-assembly-plugin and packaged properly for distribution -->
<externalizableResourcesStageDir>
esb-distribution/ext-resources
</externalizableResourcesStageDir>
</properties>
<!-- all project modules (OSGi bundles + distribution) -->
<modules>
<module>bundle-module1</module>
<module>bundle-module2</module>
...
<module>distribution</module>
</modules>
<dependencyManagement>
<dependencies>
...
</dependencies>
</dependencyManagement>
<build>
<pluginManagement>
<plugins>
<!--
Cleans contents of the folder where the externalized resources will be consolidated
Each module adds its own external files to the distribution directory during its own build
-->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<id>clean-ext-resources</id>
<phase>clean</phase>
</execution>
</executions>
<configuration>
<filesets>
<fileset>
<directory>${externalizableResourcesStageDir}</directory>
<includes>
<include>*.*</include>
</includes>
<followSymlinks>false</followSymlinks>
</fileset>
</filesets>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.3</version>
<configuration>
<descriptors>
<descriptor>src/assemble/assembly.xml</descriptor>
</descriptors>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
NOTE: We've also made sure that the externalized resource files are excluded from being packaged inside the individual bundle JARs (see the resources section of the module POM.) The resulting unzipped distribution will look like this:
my-app-distribution
/bundles
module1-bundle.jar
module2-bundle.jar
etc.
/conf
external1.properties
external2.properties
etc.

Maven overwrite resource file in dependency

I have two Maven modules, A and B. A is a dependency of B. Both modules have a resource file named default.properties located in src/main/resources. I need to keep the filenames the same and the location of the file the same in both projects because both A and B are using code which expects the file to be named and located where it is. When building B, A's default properties is in the final jar. I wish to have B's properties when I build B. How can I do this?
I know this is 3 years old but I had the same problem and this is the closest question I found, but still without correct answer so maybe someone will find it useful.
Example maven-assembly descriptor based on jar-with-dependencies (fixes overriding of log4j.properties by dependencies):
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>false</useProjectArtifact>
<unpack>true</unpack>
<unpackOptions>
<excludes>
<exclude>log4j.properties</exclude>
</excludes>
</unpackOptions>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
<fileSets>
<fileSet>
<directory>${project.build.outputDirectory}</directory>
<outputDirectory>/</outputDirectory>
</fileSet>
</fileSets>
</assembly>
The key is to provide different rules for dependencies and the actual project (top of hierarchy). Those can be split by using <useProjectArtifact>false</useProjectArtifact> and providing separate rules in fileSets for the project. Otherwise none of log4j.properties would be packed, including the top one.
Ok, Maven Resources Plugin and Assembly plugin did not cut it, so I dug some more.
It seems this is doable with Maven Shade plugin.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<!-- Main class -->
<mainClass> <!-- fully qualified package and class name --> </mainClass>
<manifestEntries>
<Class-Path>.</Class-Path>
</manifestEntries>
</transformer>
</transformers>
<filters>
<filter>
<artifact>org.something:SomeDependency</artifact>
<excludes>
<exclude>*.properties</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
So, inside the <configuration> ... </configuration> -tags I've defined two things: a transformer-implementation that takes care of modifying the jar-manifest to be runnable and use the current directory as classpath root, and excluding all the files ending with .properties from inside of dependency org.something:SomeDependency.
The actual filtering part is where you can exclude the files you don't want to end up in the final jar built by shade. You can exclude the files from all the dependencies and the current project using <artifact>*:*</artifact> inside the defined <filter>, or you can select only certain dependency using <artifact>dependcyGroupId:dependencyArtifact</artifact>, for example <artifact>junit:junit</artifact>, or even using wildcards for one or the other (<artifact>*:junit</artifact>). The excluded files are then defined inside the <excludes>...</excludes> -tags. Again, you can use exact filenames or wildcards. This should get you going with your current problem, although I'd suggest reading the documentation from the plugin-site, because shade can do a lot more than this.

Resources