I'm using Jenkins to build my projects on Maven 3.
There is ~15 jobs defined. They are running onCommit or by cron expression all day long (mostly once per hour). Some jobs are using Sonar (XXXDailyBuild not).
Sometime I'm facing error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.4.1:clean (default-clean) on project xxx: Failed to clean project: Failed to delete E:\CI\data\jobs\XXXDailyBuild\workspace\XXX\xxx\target\surefire\surefirebooter8004371362793469171.jar -> [Help 1]
It is caused by locked Jenkins workspace by active java process of surefirebooter8004371362793469171.jar
Jenkins job runs with 'Execute concurrent builds if necessary' disabled.
Any other job/process do not use 'XXXDailyBuild' workspace.
I'm trying to figure out what's going on.
Is there any way to dump running 'surefirebooter8004371362793469171.jar' process to analyze their heap?
Problem has been investigated:
Our application runs JUnit tests by surefirebooter maven plugin jar
Every tests need fresh models state
During models initialization old models state are disposed and fresh are initialized
Some models starts threads ie. Directory Watcher
The root PROBLEM is:
1. when some test fails in step 3. old models state aren't disposed
and this way some threads aren't stopped (duplicated on every test fail)
It was easy to check it using jvisualvm tool.
Finally, after maven test run process stays active forever.
Hope this analysis help someone!
Don't know the root cause and resolution. However, you can kill the java.exe from Task Manager (in Windows). This java.exe holds a lock on surefirebooter.jar file. Killing java.exe will Release that lock and you will be able to run the Maven Build.
Related
I'm facing a very strange behavior on my laptop...
I have a maven project (loaded with Visual Studio Code) building a WAR file to be deployed on a remote server.
Project compilation worked properly since yesterday but today, I'm only able to launch only once the clean install...until I reboot my laptop...
here is the command launched (the second time which cause the error):
mvn clean
....
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:3.1.0:clean (default-clean) on project my-backend: Failed to clean project: Failed to delete M:\dev\my-backend\target\my-backend\WEB-INF\lib\plexus-utils-3.3.0.jar -> [Help 1]
I'm trying to delete the target folder manually but It seems that there is another program who access this folder but I do not have anything running on my laptop, no server, no IDE (I Stopped Visual Studio Code).
I even tried to close every windows (even GitBash used to launch mvn commands) but I'm still getting this lock file issue....the only solution is to reboot my laptop !
since yesterday I didn't change anything on my pom.xml file...I'm really stucked...
EDIT 1:
I also checked with other Maven Projects I have and I'm getting the same result. After reboot I can build (mvn clean and mvn install) without any problem but at the end of the build, Windows is adding a lock on the files target/my-backend/WEB-INF/lib
Even if I try to unlock the lib folder (with UnlockIT for instance) the unlock cannot be performed and there is no process handling the lock...
any idea why I'm getting this lock and how to remove it ?
I'm building maven project on Jenkins (version 2.263.4, not the last but not so far).
It's a git project with pipeline configuration and webhook from bitbucket when push is done on the project.
So when a developper create a branch and another one a pull request, i have 2 build in parallel. Domain test running in about 45min.
And sometimes(not always), i have this error :
Command was /bin/sh -c cd /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain && /jenkins/tools/hudson.model.JDK/jdk11/jdk-11.0.2+9/bin/java -Xmx1500M -jar /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain/target/surefire/surefirebooter15630998754365374145.jar /jenkins/workspace/a-hub_test-plantage-jenkins_PR-1/server/project-core/project-core-domain/target/surefire 2021-05-04T12-39-54_622-jvmRun1 surefire5846464826560071036tmp surefire_13691263758446667285tmp
Error occurred in starting fork, check output in log
14:46:13 [ERROR] Process Exit Code: 1
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669)
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
14:46:13 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
If i re run the buildin error, it work.
I'm using maven-surefire 2.22.2.
I try to configure it following many thread on stackoverfow, but nothgin work. I try with forkMode none or something else, it work but i loose coverage because jaccoco need fork to work. So i'm stuck.
What can i do to help me to debug this problem?
I don't understand why build failed when another one is building because for me Jenkins work on different working folder, no? so why the problem appear when i'have multiple build in parallel?
thanks for help
I find an hs_err_pid.log in jenkins near source code and i see out of memory! i see that my test was launch with -Xmx1500Mo and problem occured when i launched multiple branch/pr, so jenkins use all of the memory and failed all build during fork...i decrease -Xmx and all seems to work.
The problem:
Bamboo executes old unit tests that don't exist my current develop branch which causes a build error.
The situation that causes this problem:
After a big refactoring process of my maven java project, where I basically moved, modified and renamed every file, I committed my changes to my remote repository.
That triggered my bamboo build plan, to start the build process.
The git code checkout seems to work, but the next step, running the unit tests, fails!
Looking in the log file I see that an old, no more existing java Unit test class gets executed and of course fails because of NullPointerExceptions.
Things I tried to fix this problem
A. Remove caches in the Administration section
I went to Bamboo->Administration->Repository Settings and selected
the cache of my project and deleted it.
I started the build plan again
BUILD ERROR ! Same problem
B. Delete the cache directory in the file system
Start a RDP session on the bamboo server
stop bamboo
go to D:\bamboo-home_64\xml-data\build-dir_git-repositories-cache
delete all files in this folder
start bamboo
start the build plan again
BUILD ERROR! same problem
Meta info
bamboo version: 6.1.0 build 60103 - 18 Jul 17
I don't know what I can do to fix this..
There's Clean working directory task. Add it as first task to your Job and see if it solves the issue.
Let's pretend that we have a jenkins job called functional-tests. I am looking for a way to configure this job, so that when it fails due to one or more of the tests failing, on the next run (e.g. triggered by a commit) it only runs the failed tests.
Now if this run was successful, it automatically triggers another build of the job to run all the tests.
However if the run was unsuccessful, it will continue to only run the tests that were failed in the previous run.
I think the only thing I need, is to pass the state of the previous build as well as the tests that were failed to the current job.
We are using Maven and Surefire for the build.
I have a Hudson server that runs maven 3 jobs to compile Java code. In one instance, a particular job's build log indicates that the compilation and unit tests all ran successfully, and the build completed in a successful state and another chained job was called, all indicated that Hudson believed the job to be successful. The log shows:
20:44:11 [INFO] BUILD SUCCESS
20:44:11 [INFO] ------------------------------------------------------------------------
20:44:11 [INFO] Total time: 1:35:43.774s
20:44:11 [INFO] Finished at: Mon Mar 24 20:44:11 CDT 2014
20:44:40 [INFO] Final Memory: 51M/495M
20:44:40 [INFO] ------------------------------------------------------------------------
20:44:42 channel stopped
20:44:42 [locks-and-latches] Releasing all the locks
20:44:42 [locks-and-latches] All the locks released
20:44:43 Archiving artifacts
20:45:33 Updating JIRA-1234
20:45:33 Description set: 1.23.567
20:45:33 Labelling Build in Perforce using ${BUILD_TAG}
20:45:33 [jobname] $ /usr/local/bin/p4 -s label -i
20:45:33 Label 'hudson-jobname-45' successfully generated.
20:45:33 [DEBUG] Skipping watched dependency update; build not configured with trigger: jobname #45
20:45:33 Finished: SUCCESS
However, the Hudson job page shows a "red ball" and that job run is listed as "Last failed build (#45)" on the job page. When I look at the hudson#hudson:~hudson/jobs/jobname/builds/45/build.xml file, there is a line that says
<result>FAILURE</result>
Assuming this was where the final result was captured, I changed this to
<result>SUCCESS</result>
and reloaded the page, but the red ball is still showing on the job page for that instance. I have not restarted the server to attempt to re-read the info from disk.
To be fair, there were some environmental issues around this build. Some hours later, I got a message that more than one Hudson server instance was running against the same disk image and confirmed this with
ps -ef | grep hudson.war
on the server console, showing two running processes. The job page says this run "Took 0 ms", even though the log says "Total time: 1:35:43.774s". This is Hudson ver. 2.2.1 running on "CentOS release 5.4 (Final)".
My questions:
What are the criteria for each of the job statuses? (Stable, Failed, Unstable, Aborted and Disabled statuses)?
When and where is that data captured in the running server, and can it be modified?
You have too many questions in one post.
As for "What are the criteria for each of the job statuses? (Stable, Failed, Unstable, Aborted and Disabled statuses)?"
Disabled is when a project/job is disabled from configuration (done by user with permissions). This is not a run status.
The rest are Job Run statuses:
Aborted is when a run has been cancelled/aborted. This happens when a user (with permissions) clicks the red cross button to cancel a running build. I believe SCM checkout failure also causes aborted status (but not too sure about that)
Unstable is a special status that can be applied to the job run. Usually this is done by job configuration (such as Maven) or through plugins such as Text-finder plugin. I am not aware of a way to induce unstable status through command line. Maybe through a groovy script, like plugins do. Most of the times, unstable is set by the job configuration itself, indicating failed test.
Stable and Failed are the direct result of Build Step's exit code. If the build step, such as Execute Shell, exits with exit code 0, the build step is considered success. Anything other than 0 is considered failed. If during execution of a build step, the process exits with anything other than 0, it is considered failed. If there are multiple build steps (Free-style project jobs), the last Build Step's exit code marks the status of the whole run. Once again, any Build Step in between that exits with anything other than 0 will mark the build failed.
Post-build actions, such as Text-finder plugin can also change the status of the job run.
You are not supposed to be tinkering with job history files to change the status of previous jobs. If you need to change the job result, use post-build actions such as Text-finder plugin, but even then, it only allows to downgrade the result.
Here's a one liner to printing the commit SHA1 in a jenkins job using groovy post-build script:
println "git rev-parse HEAD".execute().text
You can save this to a variable or export it as a build parameter.