Remove test dependencies from deployed POM - maven

I have a fairly typical pom.xml which builds a jar:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>mygroup</groupId>
<artifactId>my-lib</artifactId>
<version>1.0.0</version>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>...</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.6.2</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>true</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I thought it'd be nice to remove the test dependencies (junit-jupiter and its dependencies) from the copy of the POM which is deployed with the jar, just to avoid imposing them on users of the jar. After all, test code isn't included in the deployed jar, so it shouldn't matter to users of the jar how the tests are written.
I figured this would be a common use case for maven-shade-plugin. But this use case doesn't seem to be mentioned in its documentation. And I wasn't able to make the shade plugin remove the junit-jupiter dependency from the reduced POM.
Is there a straightforward way to remove dependencies from the deployed POM? Am I worrying about nothing?
I saw this question, but it seems to be about removing test dependency contents from the uber jar. In my case, I'm not actually creating an uber jar. I'm just trying to use the shade plugin for its ability to rewrite the POM.

If you want to remove unnecessary parts from the deployed POM, you can use the flatten maven plugin:
https://www.mojohaus.org/flatten-maven-plugin/flatten-mojo.html
One of the features is to remove the test dependencies.

Related

Not include src/lib/*.jar in war file

I want when run
mvn verify
to include all files from /lib/ folder to war archive in folder 'web-inf/lib`
So here my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.myhost</groupId>
<artifactId>sailero</artifactId>
<name>myapp</name>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.7.0</version>
</dependency>
<dependency>
<groupId>com.host</groupId>
<artifactId>some-lib</artifactId>
<scope>system</scope>
<version>0.0.1</version>
<systemPath>${project.basedir}/lib/some-lib-0.0.1.jar</systemPath>
</dependency>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<configuration>
<packagingIncludes>lib/*.jar</packagingIncludes>
</configuration>
</configuration>
</plugin>
in console:
mvn verify
But in war NOT INCLUDE some-lib-0.0.1.jar
in war in \target\myapp-1.0-SNAPSHOT.war\WEB-INF\lib\ -> not exist "some-lib-0.0.1.jar", but all other dependencies included.
Maven war plugin automatically copies all the project dependencies into WEB-INF/lib. So if your war needs a dependency just put the relevant GAV into the dependency section of this module.
Example:
In your pom, you have a dependency on:
<dependency>
<groupId>com.auth0</groupId>
<artifactId>java-jwt</artifactId>
<version>3.7.0</version>
</dependency>
So it will be automatically included appear in WEB-INF/lib of your WAR.
Now it doesn't work like this with dependencies in scope system and this is a root cause of the issue here.
Long story short, this question has been already asked/answered in SO.
Bottom line, I suggest getting rid of system dependency and placing it at least in the local repo or ideally in some proxy like Nexus or Artifactory. But of course, you're welcome to test other approaches suggested in the provided link.
This fix problem:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<webResources>
<resource>
<directory>lib</directory>
<targetPath>WEB-INF/lib</targetPath>
</resource>
</webResources>
</configuration>
</plugin>
Here correct approach:
first install "some-lib-0.0.1.jar" in local maven repo:
mvn install:install-file -Dfile=myproject\lib\some-lib-0.0.1 -DgroupId=com.host -DartifactId=some-lib -Dversion=0.0.1 -Dpackaging=jar
and second in pom.xml
<dependency>
<groupId>com.host</groupId>
<artifactId>some-lib</artifactId>
<version>0.0.1</version>
</dependency>
Also I found another approach:
Create a “lib” folder under your project like this: “\src\main\webapp\WEB-INF\lib”
Copy needed “jars” etc that you want included inside your WAR bundle folder.
Invoke your maven build as you normally do. I use “mvn install”, which creates builds the war file

Is there a way to specify to push complete jar folder on both driver and executors?

Is there any way to specify complete folder path of the jars to be pushed on driver as well as executor like --jars in spark-submit, which excepts comma separated jar names with full path. But it's tedious work if we do have too many jars to be pushed on both driver as well as executor.
Question : Is there a way to specify to push complete jar folder on both driver
and executors?
Yes you can make uber jar which is self contained distribution with all depedencies packed inside.
sample if you are using maven, you can use maven shade plugin or assembly plugin for this. below is shade example.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.maventest</groupId>
<artifactId>mytest</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>mytest</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>2.3</version>
<scope>compile</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<finalName>uber-${artifactId}-${version}</finalName>
</configuration>
</plugin>
</plugins>
</build>
</project>
If you are using sbt see this
your spark submit will look like ....
spark-submit [PATH_TO_YOUR_UBER_JAR]/[YOUR_UBER_JAR].jar
Further reading for example Googles article : Managing Java dependencies for Apache Spark applications
Running spark on yarn you have to be able to set spark.yarn.archive or spark.yarn.jars in spark-defaults.conf configuration file.
spark.yarn.archive is intended for distribution of the archive with all the jars you need on your executors.
spark.yarn.jars is for separate jars.
You may find more information in the official docs.

random csv data jmeter maven setup

I would like to integrate blazemeter random CSV data plugin set into my jmeter maven project. I am new to both maven and jmeter 3.2, but I have got a project built.
However there is no information on how I can setup the random CSV data plugin and how it should be setup in the configuration of the POM file.
So far, I have added the dependencies for the plugin, but there is no documentation on configuration within the pom file.
Dependency:
<dependency>
<groupId>kg.apc</groupId>
<artifactId>jmeter-plugins-cmn-jmeter</artifactId>
<version>0.5</version>
</dependency>
<dependency>
<groupId>org.apache.jmeter</groupId>
<artifactId>ApacheJMeter_components</artifactId>
<version>4.0</version>
</dependency>
<dependency>
<groupId>kg.apc</groupId>
<artifactId>jmeter-plugins-emulators</artifactId>
<version>0.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.blazemeter</groupId>
<artifactId>jmeter-plugins-random-csv-data-set</artifactId>
<version>0.6</version>
</dependency>
Is there a way I can configure the plugin in the verify stage or do I configure the plugin by adding in the properties.user and set the values within the in the pom file.
This is not how you should use dependencies in your JMeter Maven project, you should add any required JMeter Plugins into configuration/jmeterExtensions section like:
<configuration>
<jmeterExtensions>
<artifact>com.blazemeter:jmeter-plugins-random-csv-data-set:0.6</artifact>
<artifact>kg.apc:jmeter-plugins-emulators:0.4</artifact>
<artifact>kg.apc:jmeter-plugins-cmn-jmeter:0.5</artifact>
</jmeterExtensions>
<downloadExtensionDependencies>false</downloadExtensionDependencies>
</configuration>
Full pom.xml just in case:
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>mvn-jmeter</artifactId>
<version>1.0-SNAPSHOT</version>
<name>maven-jmeter-demo</name>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<groupId>com.lazerycode.jmeter</groupId>
<artifactId>jmeter-maven-plugin</artifactId>
<version>2.7.0</version>
<executions>
<execution>
<id>jmeter-tests</id>
<phase>verify</phase>
<goals>
<goal>jmeter</goal>
</goals>
</execution>
</executions>
<configuration>
<jmeterExtensions>
<artifact>com.blazemeter:jmeter-plugins-random-csv-data-set:0.6</artifact>
<artifact>kg.apc:jmeter-plugins-emulators:0.4</artifact>
<artifact>kg.apc:jmeter-plugins-cmn-jmeter:0.5</artifact>
</jmeterExtensions>
<downloadExtensionDependencies>false</downloadExtensionDependencies>
</configuration>
</plugin>
</plugins>
</build>
</project>
More information:
Adding jar's to the /lib/ext directory
Five Ways To Launch a JMeter Test without Using the JMeter GUI

Spring boot with maven multi module project

I have a maven multi module project designed like the first answer in following SO post:
Multi-module maven with Spring Boot
Now I want a common maven module that can contain some models to be used by multiple microservices. If I make this common project as a child of the first level parent pom (so that all dependencies injected by boot like jpa, jackson etc are available to common), then STS/Spring is detecting it as a boot application and complains about no Main class on maven build.
Can someone suggest how I can achieve this?
Current Code:
parent pom.xml: (Only relevant parts included)
<project>
<name>...</name>
<groupId>...</groupId>
<artifactId>...</artifactId>
<packaging>pom</packaging>
<version>...</version>
<parent>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-parent</artifactId>
<version>Brixton.M3</version>
<relativePath />
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
child (common module) pom.xml (only relevant parts), not to be boot app:
<project>
<artifactId>...</artifactId>
<version>...</version>
<name>...</name>
<parent>
<groupId>...</groupId>
<artifactId>...</artifactId>
<version>...</version>
</parent>
</project>
I don't have all the details regarding your project but my best guess is that the spring-boot-maven-plugin is defined on the parent (or you are using the spring-boot-starter-parent in your root pom). This effectively ask the build to package your module as a Spring Boot app (which is not what you want).
STS probably looks for that hint to figure out if a module contains a Spring Boot application or not. Maybe it would be nicer if it looks for a main class annotated with #EnableAutoConfiguration (or SpringBootApplication).
You can fix the problem easily (from the build side) by specifying the skip property of the repackage goal
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
If STS still picks up the module as a Spring Boot app, I'd create an issue in their tracker
Normally, Spring Boot won't start a web container if it's not present in the module.
I would suggest you to analyse your dendencies using the command
mvn dependency:tree
One more brute-force way of ensuring is use this configuration in your application.properties
spring.main.web-environment=false
Here are two ways to fix this:
You can add the skip property like #Stephane Nicoll mentioned. However, this will completely ignore the test cases inside that module. https://docs.spring.io/spring-boot/docs/current/maven-plugin/examples/it-skip.html
Another option is to add a classifier property to make a separate executable jar out of this module. https://docs.spring.io/spring-boot/docs/current/maven-plugin/examples/repackage-classifier.html
<executions>
<execution>
<id>repackage</id>
<goals>
<goal>repackage</goal>
</goals>
<configuration>
<classifier>exec</classifier>
</configuration>
</execution>
This fix will make sure the dependent module get its required jar and the source module will still be an executable one.

Getting Started with Maven + Jaxb project + IntellijIdea

I am complete new to IntellijIdea and i am looking for some step-by-step process to set up a basic project.
My project depends on Maven + Jaxb classes so i need a Maven project so that when i compile this project, the JAXB Objects are generated by Maven plugins. Now i started like this
I created a blank project say MaJa project
Added Maven Module to it
Added following settings in POM.XML
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>MaJa</groupId>
<artifactId>MaJa</artifactId>
<version>1.0</version>
<dependencies>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
</dependency>
<dependency>
<groupId>com.sun.xml.bind</groupId>
<artifactId>jaxb-impl</artifactId>
<version>2.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxb2-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>xjc</goal>
</goals>
</execution>
</executions>
<configuration>
<schemaDirectory>${basedir}/src/main/resource/api/MaJa</schemaDirectory>
<packageName>com.rimt.shopping.api.web.ws.v1.model</packageName>
<outputDirectory>${build.directory}</outputDirectory>
</configuration>
</plugin>
</plugins>
</build>
</project>
First of all, is it right settings ?
I tried clicking on Make/Compile 'MaJa' from Project > Right Click Menu and it didn't do anything.
I will be looking forward to yoru replies.
You must click not on Make/Compile 'MaJa'
1) You must choose one of maven Build Lifecycle phases here (not less then Compile).
2) Set path to maven in settings.
3) Add version for jaxb-api artifact
I add shiporder.xsd to directory /src/main/resource/api/MaJa and java classes were generated well
[jaxb2:xjc]
Generating source...
parsing a schema...
compiling a schema...
com\rim\shopping\api\web\ws\v1\model\ObjectFactory.java
com\rim\shopping\api\web\ws\v1\model\Shiporder.java

Resources