Process spawned by exec-maven-plugin blocks the maven process - maven

I am trying to execute the following scenario using maven :
pre-integration-phase : Start a java based application using a main class (using exec-maven-plugin)
integration-phase : Run the integration test cases (using maven-failsafe-plugin)
post-integration-phase: Stop the application gracefully (using exec-maven-plugin)
Here is pom.xml snip:
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>launch-myApp</id>
<phase>pre-integration-test</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<arguments>
<argument>-DMY_APP_HOME=/usr/home/target/local</argument>
<argument>-Djava.library.path=/usr/home/other/lib</argument>
<argument>-classpath</argument>
<classpath/>
<argument>com.foo.MyApp</argument>
</arguments>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<forkMode>always</forkMode>
</configuration>
</plugin>
</plugins>
If I execute mvn post-integration-test, my application is getting started as a child process of the maven process, but the application process is blocking the maven process from executing the integration tests which comes in the next phase. Later I found that there is a bug (or missing functionality?) in maven exec plugin, because of which the application process blocks the maven process. To address this issue, I have encapsulated the invocation of MyApp.java in a shell script and then appended “/dev/null 2>&1 &” to spawn a separate background process. Here is the snip (this is just a snip and not the actual one) from runTest.sh:
java - DMY_APP_HOME =$2 com.foo.MyApp > /dev/null 2>&1 &
Although this solves my issue, is there any other way to do it? Am I missing any argument for exec-maven-plugin?

You can use the async configuration parameter to achieve what you require. Also see asyncDestroyOnShutdown which will shutdown the app on JVM exit by default.
https://www.mojohaus.org/exec-maven-plugin/exec-mojo.html#async
If set to true the child process executes asynchronously and build execution continues in parallel.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>launch-myApp</id>
<phase>pre-integration-test</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<arguments>
<argument>-DMY_APP_HOME=/usr/home/target/local</argument>
<argument>-Djava.library.path=/usr/home/other/lib</argument>
<argument>-classpath</argument>
<classpath/>
<argument>com.foo.MyApp</argument>
</arguments>
<async>true</async>
</configuration>
</plugin>

(Unix only)
Use exec:exec and start a script.
This script should call
java -DMY_APP_HOME =$2 com.foo.MyApp > /dev/null 2>&1 &
first, and check the startup before the script finishes.
Did the program crash in 2 seconds?
When your java app needs some time to start you can add a script that waits for a complete startup.
retrymax=30
retry=0
RETURNCODE=1
while [ ${retry} -lt ${retrymax} ]; do
todo_your_test
RETURNCODE=$?
if [ ${RETURNCODE} -eq 0 ]; then
echo MyApp started
break
else
(( retry = retry + 1 ))
sleep 2
fi
done
exit ${RETURNCODE}

I am using the https://github.com/bazaarvoice/maven-process-plugin plugin while this issue is still open https://github.com/mojohaus/exec-maven-plugin/issues/18

There's a possible solution in this thread using Ant: Possible solution here: Maven and Exec: forking a process?
The same disadvantage as in your current solution is that it's platform dependent as well. Your current solution will only run in Unix/Linux-like environments, while the Ant-based solution is starting an instance of cmd, which only works on Windows. Using Maven profiles, one could create a configuration that uses the OS it's running on to determine which command shell to start, of course.

Related

Passing vm arguments to exec-maven-plugin

Guys I have upgraded my project to Java 17 that's why I need these lines to VM to my program works without error.
=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED
My project is working but I have another sub project which I use for testing purposes.It is run on JavaFX and it need previous arguments to work correctly.
I have tried adding as CommandLineArgs and arguments but still arguments are not passing through to vm and giving errors.
<profile>
<id>smoke</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy jars for tests</id>
<phase>generate-resources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeProjectDependencies>false</includeProjectDependencies>
<includePluginDependencies>false</includePluginDependencies>
<stripVersion>true</stripVersion>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<classpathScope>test</classpathScope>
<includeProjectDependencies>false</includeProjectDependencies>
<includePluginDependencies>false</includePluginDependencies>
<executable>${env.HOME}/java/bin/java.exe</executable>
<workingDirectory>${project.build.directory}</workingDirectory>
<arguments>
<argument>-javaagent:${env.HOME}/wildfly/standalone/lib/ext/agent.jar</argument>
<argument>-classpath</argument>
<argument>lib/*;classes</argument>
<argument>-DJBOSS_HOME=${JBOSS_HOME}</argument>
<argument>-DJAVA_HOME=${env.HOME}/java</argument>
<argument>-Duser.home=${env.USERPROFILE}</argument>
<argument>-Djava.library.path=${JBOSS_HOME}/bin</argument>
<argument>-DTESTING_FROM_COMMANDLINE=true</argument>
<mainClass>com.test.GenerateSmoke</mainClass>
<commandlineArgs>-Dexec.args=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED</commandlineArgs>
</arguments>
</configuration>
</plugin>
</plugins>
</build>
</profile>
If you have any idea please let me know.Thanks
I had a similar issue when running tests with the maven-surefire-plugin.
In my case I added it like this into the pom:
<configuration>
<argLine>${argLine} --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED</argLine>
</configuration>
Maybe it is the same in your case.
The plugins configuration options are documented here: https://www.mojohaus.org/exec-maven-plugin/exec-mojo.html
commandlineArgs or -Dexec.args are both an alternative option to override the entire executed command line you specify. These can not be used together with (other) arguments.
So you either define the entire java command line to execute as:
arguments in the pom, or
commandlineArgs in the pom, or
exec.args as an environment variable (so mvn exec:exec -Dexec.args=...)
You should be able to add the arguments to your current plugin configuration like so:
<configuration>
<arguments>
<argument>...</argument>
<argument>--add-opens</argument>
<argument>java.base/java.lang=ALL-UNNAMED</argument>
<argument>--add-opens</argument>
<argument>java.base/java.util=ALL-UNNAMED</argument>
<argument>...</argument>
</arguments>
</configuration>
Depending on what you run these arguments might need to be inserted in a specific place, so the order of these arguments matters.
That is where you can experiment with substituting the ... in my example with your current arguments.
As a test i've ran a java -jar with these options, and that works. I've enabled the maven argument -X for debug logging.
[DEBUG] Executing command line: [some\path\jdk-17.0.2\bin\java.exe, --add-opens, java.base/java.lang=ALL-UNNAMED, --add-opens, java.base/java.util=ALL-UNNAMED, -jar, my.jar]
And if i make an "accidental" typo in my add-opens we see that these arguments are indeed parsed by the java process:
[DEBUG] Executing command line: [some\path\jdk-17.0.2\bin\java.exe, --add-typo, java.base/java.lang=ALL-UNNAMED, --add-opens, java.base/java.util=ALL-UNNAMED, -jar, my.jar]
Unrecognized option: --add-typo
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

gradle - How to start spring boot server before integarion test - so easy with maven

We are switching from maven to gradle. In maven we are starting the Spring Boot server in "pre-integration-test" phase with the use of the spring-boot-mvane-plugin as documented here https://docs.spring.io/spring-boot/docs/current/maven-plugin/usage.html
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
</configuration>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.github.eirslett</groupId>
<artifactId>frontend-maven-plugin</artifactId>
<executions>
<execution>
<id>run e2e tests</id>
<phase>integration-test</phase>
<goals>
<goal>npm</goal>
</goals>
<configuration>
<arguments>run verify</arguments>
</configuration>
</execution>
</executions>
</plugin>
I didn't find a way to fork a process in gradle. The gradle Spring Boot Plugin doesn't help either.
How can I achieve this in gradle?
I got it work with gradle-spawn-plugin, but it looks very complicated to me
backend/build.gradle
plugins {
id "com.wiredforcode.spawn" version "0.8.2"
}
task startServer(type: SpawnProcessTask, dependsOn: 'bootJar') {
command "java -jar ${projectDir}/build/libs/project-${version}.jar --spring.profiles.active=integration"
ready 'Started'
}
task stopServer(type: KillProcessTask)
frontend/build.gradle
task verify(dependsOn: ':backend:startServer', type: NpmTask) {
// Copy from 'dist'
inputs.files(fileTree('tests/e2e'))
dependsOn 'npmBuild'
args = ['run', 'verify']
}
check.dependsOn verify
verify.finalizedBy ':backend:stopServer'
But pay attention to some quirks: If you are logging to the console you get some problems. First everything looks fine. But after some minutes of testing the logging does not work anymore.
Many threads are blocked like this:
"http-nio-127.0.0.1-8082-exec-2" #27 daemon prio=5 os_prio=0 cpu=462,00ms elapsed=531,88s tid=0x00007f1442a32000 nid=0x76d5 waiting on condition [0x00007f136f021000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base#11.0.7/Native Method)
- parking to wait for <0x0000000454d81218> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(java.base#11.0.7/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base#11.0.7/AbstractQueuedSynchronizer.java:885)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base#11.0.7/AbstractQueuedSynchronizer.java:917)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base#11.0.7/AbstractQueuedSynchronizer.java:1240)
at java.util.concurrent.locks.ReentrantLock.lock(java.base#11.0.7/ReentrantLock.java:267)
at ch.qos.logback.core.OutputStreamAppender.writeBytes(OutputStreamAppender.java:197)
at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:231)
at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:102)
at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:84)
at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:51)
at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:270)
at ch.qos.logback.classic.Logger.callAppenders(Logger.java:257)
at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:383)
at ch.qos.logback.classic.Logger.info(Logger.java:579)
I don't know exactly why but it seems to me that gradle-spawn-plugin is something doing with STDOUT. Maybe because of searching for the "ready" string. And as we do not log to console in production it is best not to do it when integration testing. So I disabled logging to console and everything works fine again.

How to stop Maven Execution

I have a mock server which I need to start in Jenkins before running my Automation tests and then stop the mock server after my tests have run.
This mock server is a maven project and is using exec-maven-plugin. I am able to start this server by running the command mvn exec:java. However I am unable to stop this maven project through Jenkins. I have read about it and most answers tell to find the Process ID of this Maven project and then kill it. Is there a cleaner and easier way to stop this project ?
You need to start and stop your sever using goals which are part of the maven life cycle.
An example taken from this answer
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<id>tomcat-run</id>
<goals>
<goal>run-war-only</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<fork>true</fork>
</configuration>
</execution>
<execution>
<id>tomcat-shutdown</id>
<goals>
<goal>shutdown</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
The key to this issue is indeed that you don't know the processId. Consider using exec:exec instead with async. You'll see that asyncDestroyOnShutdown is set to true by default, meaning it'll shut down once Maven is shutting down.
If you are running linux:
ps aux |grep <project-name> and then kill -9 <process id>

pax-provision without gogo shell

I recently started with osgi. I'm using pax tools to generate and deploy my bundles. I'm trying to create a console application, but everytime I launch pax-provision command gogo shell is fired. I also want to know what is the difference between pax-provision and pax-run ( I don't use pax-run ).
I turned off the gogo shell by editing the pom.xml in the base directory of the projec, by adding --noConsole to configuration :
<plugin>
<groupId>org.ops4j</groupId>
<artifactId>maven-pax-plugin</artifactId>
<version>1.5</version>
<configuration>
<provision>
<param>--platform=felix</param>
<param>--noConsole</param>
</provision>
</configuration>
<executions>
<execution>
<id>ide-support</id>
<goals>
<goal>eclipse</goal>
</goals>
</execution>
</executions>
</plugin>

Launching a windows batch script using Maven exec plugin blocks the build even though the script uses "start"

I am trying to perform integration tasting of the deployment of my application on the top of a custom container. Since my container is custom, I cannot use Maven Cargo plugin to setup the container.
My container:
Has to be started though a proper bat file, which is in the path of the machine where the tests are run.
Can be manually closed since I have a single maven module containing all my integration tests, even if one day I would like to know how to shut the process down after my tests are completed.
My problem is that I have to run my container in a different process, because it needs to keep running while my tests are performed. Furthermorely, I have an API in my tests that let me wait for the container to be ready (a sort of lookup with timeout).
I have added the following lines to my pom.xml
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.1</version>
<executions>
<execution>
<phase>pre-integration-test</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>scripts/start-agent.bat</executable>
</configuration>
</plugin>
This will call a script, which contains only
start call gs-agent.bat
However the mvn exec plugin gets stucked and my tests are not run. According to what is suggested in How do I run a batch file from my Java Application? , I have modified my pom.xml as the following:
<configuration>
<executable>cmd</executable>
<arguments>
<argument>/C</argument>
<argument>start</argument>
<argument>gs-agent.bat</argument>
</arguments>
</configuration>
But this does not seem to solve the issue:
exec plugin is not able to do this, and I found the issue for it, too: http://jira.codehaus.org/browse/MEXEC-87 (link now dead due to codehaus rampdown, can be found from web archive)
In the jira issue linked above, there is a mention and a link of a fork for exec plugin that would have the functionality.
Other than that, I think you'll need to use an antrun-plugin for the time being.
Here's an example taken from working configuration and run with mvn verify. This needs to be in the <plugins>, not <pluginManagement> (exec could reside in pluginmanagement just fine).
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>pre-integration-test</phase>
<configuration>
<target>
<exec executable="cmd.exe"
spawn="true">
<arg value="/c"/>
<arg value="D:\myfolder\test.bat"/>
</exec>
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
Note that spawn="true" is key here if you don't want the execution to block, like specified in the question. If you do want it to block and see the output immediately, set it to false.
See this question: How do I run a batch file from my Java Application?
Windows Batch Files are not executable. They are scripts that are run by the cmd executable.
More Information
Exec plugin source code reveals that Apache Commons Executor is used to actually execute the command line.
There is a lot of reading you can do here, i.e. in the Apache Commons Executor documentation and their JIRA issues, but the short version is: this isn't a problem with "Maven," it's a problem with the platform-dependent nature of executing an exec() command.
I've tackled this sort of problem before, and the solution I always devise is to deconstruct the .bat script into its actual commands and launch it directly from the exec plugin, rather than calling the script.
The Fallowing Code works for me
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.3</version>
<executions>
<execution>
<phase>generate-sources</phase>
<configuration>
<tasks>
<exec dir="${project.basedir}" executable="Script.bat" failonerror="true">
</exec>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
Source https://maven.apache.org/guides/mini/guide-using-ant.html
In my case I had trouble with npm and ng.. the underlying problem was exec-maven-plugin was trying to execute the sh scripts (which had no extensions).
Renaming
C:\Users\User\AppData\Roaming\npm\ng to C:\Users\User\AppData\Roaming\npm\ng.sh
and
C:\apps\nodejs\npm to C:\apps\nodejs\npm.sh
solved the problem.
If you want to use exec-maven-plugin here is an example to run a batch script with arguments. E.g. to run something like:
.\test.bat arg1 arg2 arg3
add the following to pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>exec-test</id>
<phase>prepare-package</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>cmd</executable>
<workingDirectory>./</workingDirectory>
<arguments>
<argument>/c</argument>
<argument>start</argument>
<argument>""</argument>
<argument>exec.bat</argument>
<argument>arg1</argument>
<argument>arg2</argument>
<argument>arg3</argument>
</arguments>
</configuration>
</plugin>

Resources