Maven shaded jar used as external project dependency - maven

I have used maven shade plugin in my project to relocate all dependency jar classes under one package e.g., org.shade.*
When I try to use that shaded jar in other application as maven dependency it pulls dependency jar's.
My expectation is when uber/shaded jar included as maven dependency it should not pull any other dependent class jar, Since already those classes are repackaged within shaded jar.

The classic scenario is:
A project producing an uber-jar has its own dependencies (dependency elements in its pom.xml file) which then are packaged together in one uber-jar as Maven artifact
When using this uber-jar as a dependency (dependency element) of another project, Maven would then inspect its <artifact>-<version>.pom file (published together with the final artifact into the Maven repository), which is basically a renamed copy of its original pom.xml file, where dependencies (dependency element) were declared (exactly the dependencies packaged into the uber-jar).
Since you already have them packed, you would then like to ignore the .pom file (and its dependencies element), for that you need to add exclusions as following:
<dependency>
<groupId>com.sample</groupId>
<artifactId>something-uber</artifactId>
<version>some-version</version>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
Note: the feature above is only available since Maven 3.2.1.
As such, you are making clear to Maven you don't want any transitive dependency and Maven dependency mediation would then not trigger them.
As a side note: this is not a good practice to have an uber-jar as dependency of a project: it will just make maintenance harder since you can't control transitive dependencies via dependencyManagement or dependencies order of the dependent project. As such you will always need to re-pack the uber jar whenever a dependency (one of its transitive one) would need maintenance (change version and so on) and have much less control on the dependent project (again, harder maintenance).

As the pom.xml on your source project that produces the uber jar declares transitive dependencies, if you include it as dependency into an external project then Maven will try to get these ( even if these are already included on the uber jar ).
I share a solution that let you avoid to explicitly exclude all transitive dependencies including it into an external project as explained by #A_DiMatteo on his solution ( I agree also with him about to avoid to use uber jar as dependency if not strictly necessary to do it for some reason ). As result then you should be able to include your uber jar dependency without using exclusions element as follow:
<dependency>
<groupId>com.sample</groupId>
<artifactId>something-uber</artifactId>
<version>some-version</version>
</dependency>
Premise: my goal was to provide both uber ( without transitive dependency declared on pom ) and thin jars on my repository. So my solution "A" is based on this scenario and currently has the limit that the shaded jar is uploaded 2 times on repository.
To provide only the uber jar see "B" solution below
For a possible solution for "A" limit see the UPDATE section at the end
A) Provide both thin and uber jars on repository
1- On your source project configure on your something module pom.xml the following maven-shade-plugin as follow:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>something-uber-${project.version}</finalName>
</configuration>
</execution>
</executions>
</plugin>
Then use the build-helper-maven-plugin plugin to attach the new artifact with "uber" classifier on module:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.2</version>
<executions>
<execution>
<id>attach-artifacts</id>
<phase>package</phase>
<goals>
<goal>attach-artifact</goal>
</goals>
<configuration>
<artifacts>
<artifact>
<file>
${project.build.directory}/something-uber-${project.version}.jar
</file>
<classifier>uber</classifier>
</artifact>
</artifacts>
</configuration>
</execution>
</executions>
</plugin>
This will generate as results of the maven install phase the following two jars on target/ directory:
40K something-0.1.0.jar
7M something-uber-0.1.0.jar
WARN: Executing then the maven deploy phase both jars will be uploaded on repository! The target here should be to upload only the thin jar skipping deploy for shaded jar in order to leave it as a local artifact ( See the UPDATE section at the end for a possible fix )
2- Create then another module on your source project named something-uber adding following dependencies and plugin:
<dependencies>
<dependency>
<groupId>com.sample</groupId>
<artifactId>something</artifactId>
<version>${project.version}</version>
<classifier>uber</classifier>
<exclusions>
<exclusion>
<artifactId>*</artifactId>
<groupId>*</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Note the followings on including the dependency:
the classifier should be equals to uber ( the one you've specified attaching new artifact using build-helper-maven-plugin on first module )
the exclusions are specified.
Executing at the end the maven deploy phase on this module the shaded jar will be uploaded on repository and you we'll be able to add it as dependency into an external project as follow:
<dependency>
<groupId>com.sample</groupId>
<artifactId>something-uber</artifactId>
<version>0.1.0</version>
</dependency>
B) Provide only uber jar on repository
Starting from solution "A" if you want to avoid to provide the thin jar on repository you should avoid on point 1 to specify finalName on maven-shade-plugin configuration and so avoid also the build-helper-maven-plugin plugin as there isn't a new artifact to attach.
Doing this, deploying the module you'll have only the uber jar on target/ as default one ( without classifier ):
7M something-0.1.0.jar
You should also skip the upload otherwise you'll have also here two fat jars uploaded ( something-0.1.0.jar & something-uber-0.1.0.jar ).
To do this add the following plugin on the same module:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
At the end on point 2 just avoid to specify the classifier on adding the dependency as follow:
<dependencies>
<dependency>
<groupId>com.sample</groupId>
<artifactId>something</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<artifactId>*</artifactId>
<groupId>*</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
UPDATE: Skip first shaded jar upload on A) solution
After searching for a solution for a while without success I decided to fork the maven-deploy-plugin plugin from GitHub and work for a new feature in order to skip shaded jar created on first module adding the plugin configured as follow:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>3.0.0-SNAPSHOT</version>
<configuration>
<skipAttachedArtifacts>
<artifact>
<groupId>com.sample</groupId>
<artifactId>something</artifactId>
<version>${project.version}</version>
<packaging>jar</packaging>
<classifier>uber</classifier>
</artifact>
</skipAttachedArtifacts>
</configuration>
</plugin>
Currently using the maven-deploy-plugin plugin all artifacts are excluded from deploy while target here is to exclude only a specific one. On my fork I've introduced the "skipAttachedArtifacts" configuration parameter in order to specify attached artifacts to exclude from deploy.
Here is the link on my forked project on GitHub:
https://github.com/gregorycallea/maven-deploy-plugin
Here the link instead to the pull request I've submitted on apache plugin project:
https://github.com/apache/maven-deploy-plugin/pull/3

Related

MulitModule project dependency management

I am working on the project with the project structure as
root
lib
xyz.jar
modules
module1
module2
Now I want to include the xyz.jar in the module1 but owing to the multimodule structure of the project, I am not able to add the jar directly through maven.
I tried using
<plugin>
<groupId>org.commonjava.maven.plugins</groupId>
<artifactId>directory-maven-plugin</artifactId>
<version>0.1</version>
<executions>
<execution>
<id>directories</id>
<goals>
<goal>highest-basedir</goal>
</goals>
<phase>initialize</phase>
<configuration>
<property>multi.module.project.root.dir</property>
</configuration>
</execution>
</executions>
</plugin>
and then using
<dependency>
<groupId>org.abc.com</groupId>
<artifactId>xyz</artifactId>
<version>0.1.0</version>
<scope>system</scope>
<systemPath>${multi.module.project.root.dir}/lib/xyz.jar</systemPath>
</dependency>
but throws error to specify absolute path of jar.
next I tried using the de
pendency management in parent pom by specifying
<dependency>
<groupId>org.abc.com</groupId>
<artifactId>xyz</artifactId>
<version>0.1.0</version>
<scope>system</scope>
<systemPath>${pom.basedir}/lib/xyz.jar</systemPath>
</dependency>
This resolves the dependency but I am not able to use the dependency in the child pom of module1
How can I solve this
The documentation of the directory-maven-plugin you are using explicitly says about the variables produced by the plugin:
They will be useful ONLY IN PLUGIN CONFIGURATIONS, NOT DURING POM INTERPOLATION.
This means you cannot use it like what you are doing.
As an alternative, I would consider referring to this SO answer.

Is it possible to create a WAR and a JAR of same maven project

I have maven project which needs to be exposed as a webservice for 3rd party client but same project needs to be used as a JAR between internal modules.
Is this thing possible using different maven profiles in a single pom.
You can use the configuration in the maven-war-plugin via:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.6</version>
<configuration>
<archiveClasses>true</archiveClasses>
<attachClasses>true</attachClasses>
</configuration>
</plugin>
After you have configured that way you can use the generated jar file as a usual dependency except for the classifier:
<dependency>
<groupId>myGroup</groupId>
<artifactId>myArtifact</artifactId>
<version>myVersion</myVersion>
<classifier>classes</classifier>
</dependency>

Exclude Java package from dependency jar

I want to use jar from third party vendor. But in this jar I have old version of Java package org.osgi.framework I need to find some way to exclude the package from the main project. Something like this:
<dependency>
<groupId>com.ibm</groupId>
<artifactId>com.ibm.ws.admin.client</artifactId>
<version>8.5.0</version>
<exclusions>
<exclusion>org.osgi.framework</exclusion>
</exclusions>
<type>jar</type>
</dependency>
Can you recommend some solution?
Although a better solution would be to re-pack the dependency (without the unwanted package) with a classifier (as described in this answer) and publish it on your enterprise Maven repository (or install it into your local Maven cache, if it's a personal project), below is a different solution which should also suit your needs.
You could have a multi-module Maven project, having a module with just this dependency and in it you could use the Maven Shade Plugin and its filters property as explained in its official example.
As per documentation, the filters element:
Archive Filters to be used. Allows you to specify an artifact in the form of a composite identifier as used by artifactSet and a set of include/exclude file patterns for filtering which contents of the archive are added to the shaded jar
In your case, the following configuration should apply the filter:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>com.ibm:com.ibm.ws.admin.client</artifact>
<excludes>
<exclude>org/osgi/framework/**</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
The generated jar from the package phase should not contain that package any longer. As part of the Maven output you should see:
[INFO] --- maven-shade-plugin:2.4.3:shade (default) # test-checksum ---
[INFO] Including com.ibm:com.ibm.ws.admin.client:jar:8.5.0 in the shaded jar.
[INFO] Replacing original artifact with shaded artifact.
You can verify the content of the generated jar, the filtered package should not be there.
Then, the output of this module will have the "new"/filtered jar you were looking for. Then the consumer module would just need to have a dependency on this module and as such have the filter applied.
An example of such a multimodule project would be:
+ aggregator/parent project
- filtered-dependency-module (applying the shade filter)
- consumer-module (having dependency on the filtered module)
Update
Further note: in the module which applies the filter, you should declare the dependency as optional so that the consumer module doesn't bring it in transitively again.
<dependencies>
<dependency>
<groupId>com.ibm</groupId>
<artifactId>com.ibm.ws.admin.client</artifactId>
<version>8.5.0</version>
<optional>true</optional>
</dependency>
</dependencies>
Optional doesn't affect the module itself, only the consumer one. And the Shade plugin will keep on working (I re-tested it, just in case).

Add external library .jar to Spring boot .jar internal /lib

I have an external .jar that cannot be imported from public repositories using pom.xml, it's sqljdbc41.jar.
I can run the project locally from my IDE, and everything will work. I referenced the library after downloading it like so:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
<scope>system</scope>
<systemPath>${basedir}/lib/sqljdbc41.jar</systemPath>
</dependency>
When I run mvn clean package to create my .jar file and try to run the created .jar, a mistake will pop up, which mentions the SQL Server references are not valid. I then extracted my .jar file and true enough, everything that is referenced in the pom.xml file properly gets downloaded and added, however, my SQL Server does not.
I can, in a very hacky way* just manually add the sqljdbc41.jar to my /lib folder after it's been compiled as a .jar, and it'll work, however that seems highly unoptimal. What would be a better approach?
*Opening the .jar file with Winrar, going to the /lib folder, manually selecting my sqljdbc41.jar file, then make sure to select the No Compression option bottom left where Winrar gives you compression options, in case you find this by Google and no one answered.
you can set 'includeSystemScope' to true.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
You could install the sqljdbc41.jar in your local repository :
mvn install:install-file -Dfile=path/to/sqljdbc41.jar -DgroupId=com.microsoft.sqlserver -DartifactId=sqljdbc41 -Dversion=4.1 -Dpackaging=jar
And then declare the dependency as a standard dependency :
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
</dependency>
If you use a remote artifact repository (nexus, archiva...) you also need to deploy the artifact on this repository. You can find more here : https://maven.apache.org/guides/mini/guide-3rd-party-jars-remote.html
Another way, you can put it into the resources folder, such as resources/lib/xxx.jar, then config the pom.xml like this:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc41</artifactId>
<version>4.1</version>
<scope>system</scope>
<systemPath>${basedir}/src/main/resources/lib/sqljdbc41.jar</systemPath>
</dependency>
In Spring Boot: I also faced similar issue and below code helped me.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>1.5.7.RELEASE</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
</plugins>
</build>
It works for me:
project {root folder}/libs/ojdbc-11.2.0.3.jar
pom.xml:
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc</artifactId>
<version>11.2.0.3</version>
<scope>system</scope>
<systemPath>${basedir}/libs/ojdbc-11.2.0.3.jar</systemPath>
</dependency>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
In my case, the fault was providing a version number without "dot" in tag:
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1</version>
<systemPath>${basedir}/src/main/resources/lib/tools.jar</systemPath>
</dependency>
This one works:
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<scope>system</scope>
<version>1.8</version>
<systemPath>${basedir}/src/main/resources/lib/tools.jar</systemPath>
</dependency>
When Spring-Boot projects are used with maven or gradle plugins they packaged the applicaiton by default as executable jars.
These executable jars cannot be used as dependency in any another Spring-Boot project because the executable jar add classes in BOOT-INF/classes folder. This means that they cannot be found when the executable jar is used as a dependency because the dependency jar will also have the same class path structure as shown below.
If we want to use project-A as a maven dependency in project-B then we must have two artifacts. To produce the two artifacts, one that can be used as a dependency and one that is executable, a classifier must be specified. This classifier is applied to the name of the executable archive, leaving the default archive for use as a dependency.
To configure a classifier of exec in Maven, you can use the following configuration:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<classifier>exec</classifier>
</configuration>
</plugin>
</plugins>
</build>
So the MAJIC WORD here is <classifier>exec</classifier> this will create a jar structure as below and then it could easily be conusmed by spring-boot project as maven dependency jar on class path.
The above plugin need to be add in project-A pom that is going to be used as dependency in project-B. Same is explained in spring documentation section 16.5. as well.
In order to work through the local repository, the target .jar file that we will work with must be in the s2 folder. Several methods can be used for this:
The file can be taken manually and put in the relevant place (not
preferred). The same process can be done by installing it via the
console.
Relevant Remote URL is written in the .pom file dependencies and
automatically places it in the s2 folder when Intellij is refreshed
(validate) in the IDE used.
The same process can be done by addressing the .pom file dependencies via the centeral repository.
Attention: ComponentScan should not be forgotten for the related jar work on SpringBot.

Maven WAR dependency

I am writing a project for acceptance testing and for various reasons this is dependent on another project which is packaged as a WAR. I have managed to unpack the WAR using the maven-dependency-plugin, but I cannot get my project to include the unpacked WEB-INF/lib/*.jar and WEB-INF/classes/* to be included on the classpath so the build fails. Is there a way to include these files into the classpath, or is there a better way of depending on a WAR?
Many thanks.
There's another option since maven-war-plugin 2.1-alpha-2. In your WAR project:
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>2.1.1</version>
<configuration>
<attachClasses>true</attachClasses>
</configuration>
</plugin>
This creates a classes artifact which you can use in the acceptance tests project with:
<dependency>
<groupId>your-group-id</groupId>
<artifactId>your-artifact-id</artifactId>
<version>your-version</version>
<classifier>classes</classifier>
</dependency>
Indeed, by design, Maven doesn't resolve transitive dependencies of a war declared as dependency of a project. There is actually an issue about that, MNG-1991, but it won't be solved in Maven 2.x and I'm not sure that I don't know if overlays allow to workaround this issue. My understanding of the suggested solution is to duplicate the dependencies, for example in a project of type pom.
(EDIT: After some more digging, I found something interesting in this thread that I'm quoting below:
I have been helping out with the development of the AppFuse project over
the last month where we make heavy use of the war overlay feature in the
Maven war plugin. It is a really nifty feature!
To get max power with war overlays I have developed the Warpath plugin
that allows projects to use war artifacts as fully fledged dependencies.
In brief:
1) The contents of the /WEB-INF/classes directory in the war dependency
artifacts can be included in the project's classpath for normal compile,
etc tasks.
2) Transitive dependencies from the war dependency artifacts become
available for use by other plugins, e.g. compile and ear - so no more
having to include all the dependencies when creating skinny wars!
The plugin has now been actively used in the AppFuse project for the
last few months, and I feel it is at a point where it is both usable and
stable.
Would the war plugin team be interested in including the warpath
functionality inside the war plugin? It would seem to be the most
natural place to host it.
So, I don't have any experience with it, but the maven warpath plugin actually looks nice and simple and is available in the central repo. To use it,include the following plugin configuration element in your pom.xml file:
[...]
<build>
<plugins>
<plugin>
<groupId>org.appfuse</groupId>
<artifactId>maven-warpath-plugin</artifactId>
<version>1.0-SNAPSHOT</version>
<extensions>true</extensions>
<executions>
<execution>
<goals>
<goal>add-classes</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
And add the war dependencies you want included in the classpath as warpath type dependencies:
[...]
<dependencies>
<dependency>
<groupId>org.appfuse</groupId>
<artifactId>appfuse-web</artifactId>
<version>2.0</version>
<type>war</type>
</dependency>
<dependency>
<groupId>org.appfuse</groupId>
<artifactId>appfuse-web</artifactId>
<version>2.0</version>
<type>warpath</type>
</dependency>
</dependencies>
[...]
Both the war and warpath dependency types are needed: the war type is used by the Maven war plugin to do the war overlay, the warpath type is used by the Warpath plugin to determine the correct list of artifacts for inclusion in the project classpath.
I'd give it a try.)
Use overlays. First, your test project need to have also packaging war.
Declare dependency of war project you want to test:
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>your-project-arftifactId</artifactId>
<version>${project.version}</version>
<type>war</type>
<scope>test</scope>
</dependency>
then configure maven-war-plugin overlay:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webResources>
<resource>
<directory>${basedir}/src/main/webresources</directory>
<filtering>true</filtering>
</resource>
</webResources>
<overlays>
<overlay/>
<overlay>
<groupId>your.group</groupId>
<artifactId>your-project-artifactId</artifactId>
</overlay>
</overlays>
</configuration>
</plugin>
In the above example in test project I overwrite webresources configuration files (like conxtext etc.).
EDIT: This solution wasn't tested with Maven 3.
Good point, Justin. That got me actually solving my problem, namely: including a war into an assembly AND including all its transitive dependencies.
I could not duplicate the war-dependency as 'jar' as you suggested since the assembly plugin would not find a jar referenced by that groupId/artefactId, but
duplicating the war-dependency as type pom
works!
The war and its transitive dependencies are not included in the assembly.
To exclude the (now also appearing) pom file I had to add an exclude element like this:
<excludes>
<exclude>*:pom</exclude>
</excludes>
into my assembly.xml file.
I think this could also be a workaround for the original question of this thread.
If you list the dependency on the war project as a jar dependency it seems to pickup the required jars/resources. I'm using Maven 2.2 + m2eclipse.

Resources