Maven multi-module web application (Hot swap) - maven

I have a multi module web application.
The structure is:
Parent
|
| - - Child 1
| - - Child 2
| - - Web
And the web project depends on the two child modules
When using my IDE to build my project I was used to the IDE building the classes in the WEB-INF/classes folder. This was nice as the web server noticed the new classes and either restarted or hot deployed these files. With maven it seems that I have to package the whole thing from scratch every time.
I would like to find a way in maven such that i can avoid running mvn:clean mvn:install mvn:war:inplace. Instead I would like a mvn:comile, and then the stuff just there.
I hope you understand what I mean. Testing the web app is extremely slow when you always have to build all the project jars and run som war command before things are updated.
The web apps pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
...
</parent>
<artifactId>web</artifactId>
<packaging>war</packaging>
<dependencies>
<dependency>
<groupId>bla.bla.bla</groupId>
<artifactId>bla_child1</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>bla.bla.bla</groupId>
<artifactId>bla_child2</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.0.4.v20130625</version>
<configuration>
<scanIntervalSeconds>30</scanIntervalSeconds>
<webApp>
<contextPath>/blabla</contextPath>
</webApp>
</configuration>
</plugin>
</plugins>
</build>
</project>

A possible solution to your problem could be to build a skinny war with the maven-war-plugin (or at least exclude child1 and child2 if you have other dependencies as well) and add the target/classes folders of the child-projects to your classpath. This could be done with the Build Helper Maven Plugin's add-resource goal.
Disclaimer: I didn't try this myself but in theory it could work :)

This is my own temporary answer to this problem.
It would be nice if it could be done with maven, but I haven't been able to find a solution. Therefore I have bound an ant target to the compile phase of the web module.
This is probably not a solution that can be used anywhere, but it works in my case.
web/pom.xml:
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>copy-modules</id>
<phase>compile</phase>
<configuration>
<target>
<delete dir="${project.basedir}\src\main\webapp\WEB-INF\classes"/>
<copy todir="${project.basedir}\src\main\webapp\WEB-INF\classes">
<fileset dir="${main.basedir}\modules\child-1\target\classes">
<include name="**/*.class"/>
<include name="**/*.xml"/>
</fileset>
<fileset dir="${main.basedir}\modules\child-2\target\classes">
<include name="**/*.class"/>
<include name="**/*.xml"/>
</fileset>
</copy>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
Now the jetty sees everytime I do a compile, which is much faster than install, war and the stuff that was needed before.

Related

NetBeans 12.6, Spring Boot / Maven, Cisco AXL Schema - Background scanning of projects

Often related to Background scanning of projects people complain that this happens when NetBeans is started.
I have a Spring Boot 2.6.x with Maven 3.8.2 project, using Cisco AXL Schema 12.5.
With Apache CXF from this AXL Schema a lot of Java source code files are generated.
When I do a Clean and Build on my project afterwards immediatly the Background scanning of projects starts.
And it takes most of the times recently quite long.
I see for example that it scans also
netbeans-12.6/webcommon/jsstubs/corestubs.zip
Why should it scan this too when building my project?
But most of the time, although it shows 100% scanning done, it spends in the folder where the generated Java source code files are
<project folder>/target/generated/cxf
There are 2282 generated Java source code files.
I'm not sure if NetBeans hangs or really scans these files, it shows 100% scanning so it should be done.
Often it takes too long so I have to terminate NetBeans from the console. After a restart of NetBeans the Background scanning for projects starts and takes much shorter time, but this is annoying.
What can I do about it?
When I start NetBeans from console I do only ./netbeans. Is there a difference if starting NetBeans with sudo ./netbeans?
Here is how my project folder/file structure looks like, maybe I don't use correctly:
First of all I extracted the AXL Schema next to my src folder
<project folder>
-> schema
-> 12.5
AXLAPI.wsdl
AXLEnums.xsd
AXLSoap.xsd
-> src
-> main/...
-> test/...
In pom.xml I use
...
<build>
...
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>3.4.5</version>
<executions>
<execution>
<id>generate-sources</id>
<phase>generate-sources</phase>
<configuration>
<sourceRoot>${project.build.directory}/generated/cxf</sourceRoot>
<wsdlOptions>
<wsdlOption>
<wsdl>${basedir}/schema/12.5/AXLAPI.wsdl</wsdl>
<wsdlLocation>classpath:schema/12.5/AXLAPI.wsdl</wsdlLocation>
<extraargs></extraargs>
</wsdlOption>
</wsdlOptions>
</configuration>
<goals>
<goal>wsdl2java</goal>
</goals>
</execution>
</executions>
</plugin>
...
<resources>
...
<resource>
<directory>./</directory>
<includes>
<include>schema/**</include>
</includes>
</resource>
<resource>
<directory>target/generated/cxf</directory>
<includes>
<include>**/*.java</include>
</includes>
</resource>
...
</resources>
</build>
...
Maybe this pom.xml setup is not correct, and that's why the Background scanning for projects works wrongly.
When I look into the resulting war file after build, I see
WEB-INF
-> classes
-> com/cisco/axl/api/_12
-> schema/12.5
and there are artifacts which might not belong there.
For example in com/cisco/axl/api/_12 there are not only the class files but all related generated Java source code files (all 2282).
And perhap schema/12.5 shouldn't be in the war file too.
I tried this pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>cxf</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>3.4.5</version>
<executions>
<execution>
<id>generate-sources</id>
<phase>generate-sources</phase>
<configuration>
<wsdlOptions>
<wsdlOption>
<wsdl>src/main/resources/wsdl/CustomerService.wsdl</wsdl>
</wsdlOption>
</wsdlOptions>
</configuration>
<goals>
<goal>wsdl2java</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>7.0</version>
</dependency>
</dependencies>
</project>
(Latest cxf-codegen-plugin, no additional config, wsdl file from here, java ee dependencies)
Running mvn clean install ("Clean and Build Project"...takes less than 5 seconds with this wsdl), and gets us:
..this nice picture (resolution of generated sources! grouped by provider (cxf)...we could have more).
Conclusion
Netbeans is mature regarding "generated sources". (As long as they are in target/generated-sources/<provider>;).
With "generated projects" (maven/gradle e.g. openapi-plugin) on the other hand, I experienced (netbeans) issues...and had to externalize/"source control" the "generated stuff" (/project!).
Don'ts
build>resources>resource>directory>. This will(try to) package your project root (additionally) to target/classes!! (This may confuse any IDE.)
...>resource>directory>target, for similar reasons esp. in Netbeans.
Hints
When we want the schemas & definitons to reside in (packaged) class path, we place them within src/main/resources. Otherwise: outside.
We add <resources/> to our <build/>, only when we decided so/know what we do/don't create "circles" (with existing maven-defaults), notto "trick netbeans"! (this is out-dated;)
Update:
I updated same project with this wsdl (axl-demo/schema).
It generated 1647 classes.
Netbeans took a while to scan:
I increased memory:
{nb_home}/etc/netbeans.conf:
netbeans_default_options="-J-Xmx4g ..."
(thx to: How to assign more memory to Netbeans? , ...)
(Restarted netbeans,) drank coffee
But then (once scan was completed): Still "nice picture", we can import/declare/use the generated classes:
Some Tweaks
..Yoda added to the build:
moved cxf-execution to profile:
<profiles>
<profile>
<id>gen</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
...
to activate it (only) with mvn install -Pgen (or in netbeans (Project>Properties>Run>) Configuration(drop-down)).
applied this: How to protect auto-generated sources during clean package in maven? like:
<build>
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<excludeDefaultDirectories>true</excludeDefaultDirectories>
<filesets>
<fileset>
<directory>${project.build.directory}</directory>
<excludes>
<exclude>generated-sources/**</exclude>
<exclude>classes/com/cisco/**</exclude>
</excludes>
</fileset>
</filesets>
</configuration>
</plugin>
</plugins>
</build>
I don't agree with "recommended solution"! For "thousands of" classes that b/rarely change, who wants to clean & re-generate them "hundreds" times/day?
Which accelerates us from (mvn -Pgen clean install):
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 44.515 s
To "Project>Clean and Build" (mvn clean install):
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 4.494 s

Groovy and Maven: Stuck in compiling and testing a very simple

I am stuck trying to compile and test a extremely simple project. It's a beginner project in order to understand how all of this works, and I am currently stucked.
My main objective is to understand how to handle resources files that are located outside of the standard folder structure.
I have a main class, with two methods. One load a resource file which is on the standard folder structure (src\main\resources). Another one load a resource which is in a custom folder, outside of the standard structure (resources).
There is one junit file that simply verify that the resource is correctly loaded.
It works fine with IntelliJ. I simply declared the resources folder as resources folders and that's it.
Now with maven ..... actually I can't even compile with gmaven-plus. Nor run the test. So I did not even bother to declare the custom folder as a resource in the pom.xml file.
I based my pom.xml file based on an existing pom we have at work and from stuff I read on the web. There's no way I can make it work.
Here is a link to a 7zip file with my project, if one could put me on the right track, I would be grateful.
https://www.dropbox.com/s/jvn32ll5xfvjfwd/GroovyExample.7z?dl=0
Here is the pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>groupId</groupId>
<artifactId>Example</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.13</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>5.4.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.8.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
<configuration>
<sources>
<source>
<directory>src/main/groovy</directory>
<includes>
<include>**/*.groovy</include>
</includes>
</source>
</sources>
<testSources>
<testSource>
<directory>src/test/groovy</directory>
<includes>
<include>**/*.groovy</include>
</includes>
</testSource>
</testSources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M1</version>
<configuration>
<failIfNoTests>true</failIfNoTests>
<testSourceDirectory>str/test/</testSourceDirectory>
<includes>
<include>**/*Test*.*</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</project>
Here is the output:
Unable to get Groovy version from InvokerHelper or GroovySystem,
trying jar name.
Failed to execute goal
org.codehaus.gmavenplus:gmavenplus-plugin:1.8.1:compile (default) on
project Example: Execution default of goal
org.codehaus.gmavenplus:gmavenplus-plugin:1.8.1:compile failed.
The 2.4 groovy-all POMs do not include Groovy as a dependency, because they are a POM for an uber-jar, rather than a POM that describes all the Groovy module jars. Because of this, GMavenPlus is unable to find the Groovy jar to use for compilation. The <type>pom</type> works for Groovy 2.5, and 3.0, but not 2.4. So for your use case, simply delete the <type>pom</type> (or replace it with the default of <type>jar</type>). This was the way Groovy was often included back before 2.5, so the groovy-all POMs of 2.5 and 3.0 were added to ease the transition. See https://groovy-lang.org/releasenotes/groovy-2.5.html#Groovy2.5releasenotes-Packaging.

Properties defined in the properties section of the POM are not seen in external ant build file

I'm using maven-antrun-plugin in my pom.xml with external ant file.
It's said in plugin's document:
All of the properties available to Maven are also available in the
target configuration. However, you may want to call an external Ant
build script using the ant task. To avoid name conflicts, only a
subset of the properties are passed to the external Ant build. These
include all properties defined in the properties section of the POM.
It also includes prefixed versions of some of the commonly used Maven
properties.
So here's my pom, where I define "test.prop" property:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>testant</groupId>
<artifactId>testant</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<test.prop>TestPropValue</test.prop>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<dependencies>
<dependency>
<groupId>ant</groupId>
<artifactId>optional</artifactId>
<version>1.5.4</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>generate-index-properties</id>
<phase>install</phase>
<configuration>
<tasks>
<!--<property name="test.prop" value="${test.prop}"/>-->
<ant antfile="build.xml">
<target name="echo-prop"/>
</ant>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
And just trying to echo this property in build.xml:
<project default="test">
<target name="echo-prop">
<echo>${test.prop}</echo>
</target>
</project>
This is what I get:
echo-prop:
[echo] ${test.prop}
So property is not resolved as it should, according to the doc.
And it works fine only in case if I uncomment line with explicit property declaration under "tasks" tag.
Could you please help me in understanding, what am I doing wrong?
Thank you!
You should specify a version for the antrun-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.8</version><!--$NO-MVN-MAN-VER$-->
(...)
</plugin>
ant by default passes all properties of the parent to an underlying "ant"-call. Unless you define inheritAll=false, all properties are passed.
In this case, you should have a look at the effective pom. There a very ancient version of the antrun-plugin is defined.
As soon as you switch to a recent one, the example code works.

How does maven decide when to use the target folder for classpath

I have a question regarding how maven calculates the classpath during building. Specifically, what controls when the "target/classes" is used and when the "jar" from a repository (local/remote) is used.
I have a project on version 1.0.0-SNAPSHOT where the artifacts have NOT been installed/deployed so there is no "jar" in some repository (remote or local) to resolve them. I want to run "generate-sources" WITHOUT installing locally (no 'mvn install' run).
The structure looks like this:
parent-prj
parent-prj/sub-prj
parent-prj/gen-src-prj <--- This depends on 'sub-prj'
When I run "mvn -am -pl parent-prj/gen-src-prj generate-sources" in order to just generate some java files, it does not work:
[ERROR] Failed to execute goal on project gen-src-prj: Could
not resolve dependencies for project
mygrp:gen-src-prj:jar:1.0.0-SNAPSHOT:
Could not find artifact
mygrp:sub-prj:jar:1.0.0-SNAPSHOT -> [Help 1]
Using debug output and adding "dependency:build-classpath" I can confirm that maven ignores the presence of "sub-prj" in the reactor and looks for a "jar" somewhere which it can't find. Yet the project is printed in the reactor summary:
[INFO] Reactor Summary:
[INFO]
[INFO] parent-prj ..................................... SUCCESS [ 0.625 s]
[INFO] sub-prj ........................................ SUCCESS [ 0.018 s]
[INFO] gen-src-prj .................................... FAILURE [ 0.040 s]
The interesting thing I noticed is that running the compile goal works fine! This uses sub-prj/target/classes (as shown by dependency:build-classpath) and has no trouble generating the sources and even compiling them: "mvn -am -pl parent-prj/gen-src-prj compile"
So here are the points I want to understand:
Why does the compile goal work but the generate-sources doesn't work?
At what point does maven decide to use the output folder of previous projects on the reactor classpath instead of looking for a jar?
Is there a way for generate-sources to run directly as I want it EVEN WITHOUT having its dependencies resolved?
Regarding (3) my generation tool is a utility invoked by:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
The tool reads some XML in src/main/resources and generates Java files and does NOT need anything in its class-path (so there is no need for maven to resolve it).
Also note that I would be interested to understand (1) and (2) even if a solution for (3) is provided.
EDIT: Per comment request, adding full example
parent-prj/pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>mygrp</groupId>
<artifactId>parent-prj</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>sub-prj</module>
<module>gen-src-prj</module>
</modules>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.9</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
parent-prj/sub-prj/pom.xml
<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>mygrp</groupId>
<artifactId>parent-prj</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>sub-prj</artifactId>
</project>
parent-prj/gen-src-prj/pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>mygrp</groupId>
<artifactId>parent-prj</artifactId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<artifactId>gen-src-prj</artifactId>
<dependencies>
<dependency>
<groupId>mygrp</groupId>
<artifactId>sub-prj</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<includeProjectDependencies>false</includeProjectDependencies>
<includePluginDependencies>true</includePluginDependencies>
<mainClass>uk.co.real_logic.sbe.SbeTool</mainClass>
<systemProperties>
<systemProperty>
<key>sbe.output.dir</key>
<value>${project.build.directory}/generated-sources/java</value>
</systemProperty>
<systemProperty>
<key>sbe.validation.warnings.fatal</key>
<value>true</value>
</systemProperty>
</systemProperties>
<arguments>
<argument>${project.build.resources[0].directory}/Examples.xml</argument>
</arguments>
<workingDirectory>${project.build.directory}/generated-sources/java</workingDirectory>
</configuration>
<dependencies>
<dependency>
<groupId>uk.co.real-logic</groupId>
<artifactId>sbe-tool</artifactId>
<version>1.7.10</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>target/generated-sources/java/</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
EDIT: Armed with the knowledge from the answers I have come up with this workaround that allows one to achieve the desired behaviour. I list the dependencies in a profile that is active by default, then use another profile to run generate-sources with no dependencies active, like follows:
parent-prj/gen-src-prj/pom.xml
<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>mygrp</groupId>
<artifactId>sub-prj</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>excludeDependency</id>
<dependencies>
</dependencies>
</profile>
</profiles>
To generate sources with above, use: mvn -PexcludeDependency generate-sources
Maven can reference only output generated in current Session (during currently executing shell command). It uses the most "mature" place to look for the "output":
If compile is run - the classes end up in the target/classes dir, thus other modules can reference that
If package is run - then target/*.jar is created and this jar file ends up in the classpath instead
If install is run - then jar file ends up in the local repository - which is what ends up on the classpath
So there are 3 factors that impede your task:
maven-exec-plugin requires dependency resolution (as pointed out by #mondaka)
Your module1 references module2
generate-sources is run before the compilation. Thus module2 is not yet prepared to be used as a dependency.
So if you want to do it your way - you'll have to run at least compile phase each time you use anything from the Default Lifecycle. Or you could write your own plugin that doesn't require dependency resolution.
This problem is related to an open maven bug:
https://issues.apache.org/jira/browse/MNG-3283
The issue says: "The problem only occurs when a plugin binds itself to the
generate-sources phase and has #requiresDependencyResolution".
I have checked that exec-maven-plugin Mojo have indeed requiresDependencyResolution = ResolutionScope.TEST. You can see that on https://github.com/mojohaus/exec-maven-plugin/blob/master/src/main/java/org/codehaus/mojo/exec/ExecJavaMojo.java
Then, your only option is to use compile or process-classes phases. This is a Major open bug from 2007...

Which dependencies do I have to include in a Maven Apache Storm Project?

I am pretty new to Apache Storm and Maven projects, so I tried to follow this "tutorial" (which from my point of view is not a tutorial at all):
http://storm.apache.org/releases/current/Creating-a-new-Storm-project.html
There is a huge pom.xml referenced (https://github.com/apache/storm/blob/v1.1.1/examples/storm-starter/pom.xml) which shall be used as basis for a new project. I tried to figure out what to copy into my project pom.xml. So I decided to start with the apache core dependency first. My pom.xml looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>groupId</groupId>
<artifactId>StormTest</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>1.1.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.5.0</version>
<executions>
<execution>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<includeProjectDependencies>true</includeProjectDependencies>
<includePluginDependencies>false</includePluginDependencies>
<classpathScope>compile</classpathScope>
<mainClass>${storm.topology}</mainClass>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<appendAssemblyId>false</appendAssemblyId>
<finalName>StormTest-1.0-SNAPSHOT_dep</finalName>
<archive>
<manifest>
<mainClass>de.arphi.bi.WordCountTopology</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>${basedir}/resources</directory>
<filtering>false</filtering>
<includes>
<include>log4j2.xml</include>
</includes>
</resource>
</resources>
</build>
</project>
This even works when it is about runing maven and building / packaging a jar. The outcome are two jar files (a small one without dependencies and a bigger one with dependencies). I cannot run the smaller one because it says "main manifest not found". But I can run the bigger one by executing the following command:
storm -jar StormTest-1.0-SNAPSHOT_dep.jar
Actually the runs on my locally installed apache storm 1.1.1 (I have some System.out.printlns ...) but I get an exception:
2018-01-02 21:38:31,864 main ERROR Unable to create file C:\Users\Artur\Desktop\Bi\apache-storm-1.1.1\logs/access-web-${sys:daemon.name}.log java.io.IOException: Die Syntax für den Dateinamen, Verzeichnisnamen oder die Datenträgerbezeichnung ist falsch
at java.io.WinNTFileSystem.canonicalizeWithPrefix0(Native Method)
at java.io.WinNTFileSystem.canonicalizeWithPrefix(WinNTFileSystem.java:451)
at java.io.WinNTFileSystem.canonicalize(WinNTFileSystem.java:422)
at java.io.File.getCanonicalPath(File.java:618)
at java.io.File.getCanonicalFile(File.java:643)
at org.apache.logging.log4j.core.util.FileUtils.makeParentDirs(FileUtils.java:134)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:573)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:554)
at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:112)
at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:114)
at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.getFileManager(RollingFileManager.java:155)
at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:131) at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:60)
at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:122)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:952)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:892)
at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:884)
at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:508) at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:232)
at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:244)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:545)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:617)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:634)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:229)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:152)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:122)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:43)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.apache.storm.topology.BasicBoltExecutor.<clinit>(BasicBoltExecutor.java:28)
at org.apache.storm.topology.TopologyBuilder.setBolt(TopologyBuilder.java:215)
at de.arphi.bi.WordCountTopology.main(WordCountTopology.java:22)
It says something about the syntax for creating a directory is wrong. And I know that it is about logging. I played arroung with different other dependencies (log4j, slf4j) and tried ecen to exclude dependecies without any success. I cannot get rid of this error.
Any ideas? I think that I am missing a dependency or that I have to exclude some parts of my pom.xml. But since I am not an Maven expert it is really hard for me to figure out what I have to adapt here.
I agree that storm-starter has gotten pretty big, and we should maybe have more of a minimal example.
First you should set the storm-core dependency to "provided" scope. When you deploy the topology to Storm, your jar will use the storm-core jar present in the Storm installation, so you shouldn't also put it in your fat jar.
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>1.1.1</version>
<scope>provided</scope>
</dependency>
When your topology runs on Storm, it will use the Log4j2 configuration in the log4j2/worker.xml file in your Storm installation. You shouldn't include your own log4j2.xml. If you need to set specific log levels, you can either modify worker.xml or use the CLI as described at http://storm.apache.org/releases/1.1.1/dynamic-log-level-settings.html.
Other than that your pom looks fine. I don't know why you have exec-maven-plugin in there (Edit: I see it's also in storm-starter, I think it's a leftover from when it was possible to run storm-starter in local mode. You shouldn't need it), and I might replace maven-assembly-plugin with the shade plugin, but I'd expect your topology to work regardless.
Thank Stig Rohde Døssing. Finally I found the origin for my exception base on your hint regarding the log4j2/worker.xml. The issue was not in the worker.xml, but in the cluster.xml which is located in the same directory.
When reading my Exception shown here, you can see that Java complains about creating a system path ("access-web-${sys:daemon.name}.log"). I found the placeholder sys:daemon.name in the cluster.xml and replaced it with something static like "access-web-mysysdaemonname.log". That fixed the issue. I have no idea why this placeholder could not be resolved by the system while there was no trouble with other placeholders.
Thanks for the hints. Topic can be closed.

Resources