Jenkins maven projet build failed on fork during test - maven

I'm building maven project on Jenkins (version 2.263.4, not the last but not so far).
It's a git project with pipeline configuration and webhook from bitbucket when push is done on the project.
So when a developper create a branch and another one a pull request, i have 2 build in parallel. Domain test running in about 45min.
And sometimes(not always), i have this error :
Command was /bin/sh -c cd /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain && /jenkins/tools/hudson.model.JDK/jdk11/jdk-11.0.2+9/bin/java -Xmx1500M -jar /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain/target/surefire/surefirebooter15630998754365374145.jar /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain/target/surefire 2021-05-04T12-39-54_622-jvmRun1 surefire5846464826560071036tmp surefire_13691263758446667285tmp
Error occurred in starting fork, check output in log
14:46:13 [ERROR] Process Exit Code: 1
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
If i re run the buildin error, it work.
I'm using maven-surefire 2.22.2.
I try to configure it following many thread on stackoverfow, but nothgin work. I try with forkMode none or something else, it work but i loose coverage because jaccoco need fork to work. So i'm stuck.
What can i do to help me to debug this problem?
I don't understand why build failed when another one is building because for me Jenkins work on different working folder, no? so why the problem appear when i'have multiple build in parallel?
thanks for help

I find an hs_err_pid.log in jenkins near source code and i see out of memory! i see that my test was launch with -Xmx1500Mo and problem occured when i launched multiple branch/pr, so jenkins use all of the memory and failed all build during fork...i decrease -Xmx and all seems to work.

Related

Jhipster project runs fine on Mac, but fails to run on Linux

I have a JHipster project that includes Spring Boot and React Native. I run it OK using ./mvnw as the guide specifies, however, when I put the entire project on Ubuntu, it fails to run and the log is as follows:
ERROR Failed to execute goal
com.github.eirslett:frontend-maven-plugin:1.8.0:npm (webpack build dev)
on project galley-1: Failed to run task: 'npm run webpack:build' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 2 (Exit value: 2) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.8.0:npm (webpack build dev)
Some points to note are that: I use sudo, and I already changed the webpack space size to be as big as 9000, in package.json.
Just to answer my own question, I finally find a way to run this:
On mac, I package my project using 'mvn -Pprod package -DskipTests' (sorry there were some tests that didn't pass at that time, I didn't get time to fix those at that time), and then on Linux:
nohup sudo java -jar *.jar &
It will do the trick. But if anyone gets any better idea, please share.

Maven release branch fails with error=7, Argument list too long

We have recently migrated our multimodule(each module with a pom.xml) repository from SVN to GIT.SCM connection in all pom.xml files updated from svn to git.
When i am trying to create a branch from git repository using maven release plugin it fails with the below error
"Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5:branch (default-cli) on project intgr: An error is occurred in the checkin process: Exception while executing SCM command. Error while executing command. Error while executing process. Cannot run program "/bin/sh" (in directory "/u01/remote-agent-home/xml-data/build-dir/IN-CDBFG-JOB1/mahirepository"): error=7, Argument list too long -> [Help 1]"
On examining logs i understand below command generated this error.
"[INFO] Checking in modified POMs... [INFO] Executing: /bin/sh -c cd /u01/remote-agent-home/xml-data/build-dir/IN-CDBFG-JOB1/mahirepository && git add -- intgr-parent/pom.xml archetypes/pom.xml and 1000 more pom files "
Is there a way i can workaround or fix this issue?
Maven version : Apache Maven 3.0.4
JDK version : 1.7.0_01
Maven command :
mvn -s ./settingsBamboo.xml --batch-mode release:branch -DbranchName=${bamboo.branchname} -DupdateWorkingCopyVersions=true -DsuppressCommitBeforeBranch=true -DautoVersionSubmodules=true -DremoteTagging=false -DupdateBranchVersions=false -X
I am currently facing this problem as well, but I got a bit further on this.
The issue has nothing to do with Java or Maven, it's a Linux limitation.
using getconfig MAX_ARGS you can find the max amount of bytes you can use as arguments.
Large maven commands can easily go beyond this, and the amount of bytes differs largely per distro.
Check the amount of bytes you can use, and try to get your command below that.
You could for example move into the build dir first, and then run the command relative.
Another solution would be to increase the amount of usable space, but I have so far not gotten that working.
If anyone does or you have fixed this yourself, let me know.

How to improve the build failure reporting of a prallel gradle build?

We use a parallel gradle build in our ci server. But finding the cause of the build failure is sometimes a pain, because one has to dig through the gradle output to find the real problem. I would like to improve that.
For example one of the gradle tasks uses yarn. If this task fails gradle reports the following in the jenkins console:
10:27:27 * What went wrong:
10:27:27 Execution failed for task ':foo-web:yarn'.
10:27:27 > Process 'command '/home/jenkins/workdir/workspace/foo/foo-web/.gradle/yarn/bin/yarn'' finished with non-zero exit value 1
Then one has to scroll up a lot of lines to find the output of yarn due to parallel tasks being finished, just to find out yarn could not download a dependency.
Can we change the gradle build failure reporting so that it will append the error output of the failed task at the end of the build output to the failure report? Or is there some way to get gradles failure reporting and the output of the failed task more closely together?

Maven wrapper in jenkins

I'm trying to test build a maven based project in jenkins.
https://github.com/tonsV2/Lift-Log-Backend
However I get the following error.
[Lift Log Backend] $ /bin/sh -xe /tmp/hudson4180120395829748105.sh
+ ./mvnw clean
Error: Could not find or load main class org.apache.maven.wrapper.MavenWrapperMain
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Any clues?
Seems like you have spaces in your Jenkins job name. Try to rename your job from "Lift Log Backend" to "Lift-Log-Backend"
I had the same issue, after renaming the Jenkins job, the error was gone.

Maven surefirebooter

I'm using Jenkins to build my projects on Maven 3.
There is ~15 jobs defined. They are running onCommit or by cron expression all day long (mostly once per hour). Some jobs are using Sonar (XXXDailyBuild not).
Sometime I'm facing error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.4.1:clean (default-clean) on project xxx: Failed to clean project: Failed to delete E:\CI\data\jobs\XXXDailyBuild\workspace\XXX\xxx\target\surefire\surefirebooter8004371362793469171.jar -> [Help 1]
It is caused by locked Jenkins workspace by active java process of surefirebooter8004371362793469171.jar
Jenkins job runs with 'Execute concurrent builds if necessary' disabled.
Any other job/process do not use 'XXXDailyBuild' workspace.
I'm trying to figure out what's going on.
Is there any way to dump running 'surefirebooter8004371362793469171.jar' process to analyze their heap?
Problem has been investigated:
Our application runs JUnit tests by surefirebooter maven plugin jar
Every tests need fresh models state
During models initialization old models state are disposed and fresh are initialized
Some models starts threads ie. Directory Watcher
The root PROBLEM is:
1. when some test fails in step 3. old models state aren't disposed
and this way some threads aren't stopped (duplicated on every test fail)
It was easy to check it using jvisualvm tool.
Finally, after maven test run process stays active forever.
Hope this analysis help someone!
Don't know the root cause and resolution. However, you can kill the java.exe from Task Manager (in Windows). This java.exe holds a lock on surefirebooter.jar file. Killing java.exe will Release that lock and you will be able to run the Maven Build.

Resources