How does Hudson/Jenkins determine job result status? - maven

I have a Hudson server that runs maven 3 jobs to compile Java code. In one instance, a particular job's build log indicates that the compilation and unit tests all ran successfully, and the build completed in a successful state and another chained job was called, all indicated that Hudson believed the job to be successful. The log shows:
20:44:11 [INFO] BUILD SUCCESS
20:44:11 [INFO] ------------------------------------------------------------------------
20:44:11 [INFO] Total time: 1:35:43.774s
20:44:11 [INFO] Finished at: Mon Mar 24 20:44:11 CDT 2014
20:44:40 [INFO] Final Memory: 51M/495M
20:44:40 [INFO] ------------------------------------------------------------------------
20:44:42 channel stopped
20:44:42 [locks-and-latches] Releasing all the locks
20:44:42 [locks-and-latches] All the locks released
20:44:43 Archiving artifacts
20:45:33 Updating JIRA-1234
20:45:33 Description set: 1.23.567
20:45:33 Labelling Build in Perforce using ${BUILD_TAG}
20:45:33 [jobname] $ /usr/local/bin/p4 -s label -i
20:45:33 Label 'hudson-jobname-45' successfully generated.
20:45:33 [DEBUG] Skipping watched dependency update; build not configured with trigger: jobname #45
20:45:33 Finished: SUCCESS
However, the Hudson job page shows a "red ball" and that job run is listed as "Last failed build (#45)" on the job page. When I look at the hudson#hudson:~hudson/jobs/jobname/builds/45/build.xml file, there is a line that says
<result>FAILURE</result>
Assuming this was where the final result was captured, I changed this to
<result>SUCCESS</result>
and reloaded the page, but the red ball is still showing on the job page for that instance. I have not restarted the server to attempt to re-read the info from disk.
To be fair, there were some environmental issues around this build. Some hours later, I got a message that more than one Hudson server instance was running against the same disk image and confirmed this with
ps -ef | grep hudson.war
on the server console, showing two running processes. The job page says this run "Took 0 ms", even though the log says "Total time: 1:35:43.774s". This is Hudson ver. 2.2.1 running on "CentOS release 5.4 (Final)".
My questions:
What are the criteria for each of the job statuses? (Stable, Failed, Unstable, Aborted and Disabled statuses)?
When and where is that data captured in the running server, and can it be modified?

You have too many questions in one post.
As for "What are the criteria for each of the job statuses? (Stable, Failed, Unstable, Aborted and Disabled statuses)?"
Disabled is when a project/job is disabled from configuration (done by user with permissions). This is not a run status.
The rest are Job Run statuses:
Aborted is when a run has been cancelled/aborted. This happens when a user (with permissions) clicks the red cross button to cancel a running build. I believe SCM checkout failure also causes aborted status (but not too sure about that)
Unstable is a special status that can be applied to the job run. Usually this is done by job configuration (such as Maven) or through plugins such as Text-finder plugin. I am not aware of a way to induce unstable status through command line. Maybe through a groovy script, like plugins do. Most of the times, unstable is set by the job configuration itself, indicating failed test.
Stable and Failed are the direct result of Build Step's exit code. If the build step, such as Execute Shell, exits with exit code 0, the build step is considered success. Anything other than 0 is considered failed. If during execution of a build step, the process exits with anything other than 0, it is considered failed. If there are multiple build steps (Free-style project jobs), the last Build Step's exit code marks the status of the whole run. Once again, any Build Step in between that exits with anything other than 0 will mark the build failed.
Post-build actions, such as Text-finder plugin can also change the status of the job run.
You are not supposed to be tinkering with job history files to change the status of previous jobs. If you need to change the job result, use post-build actions such as Text-finder plugin, but even then, it only allows to downgrade the result.

Here's a one liner to printing the commit SHA1 in a jenkins job using groovy post-build script:
println "git rev-parse HEAD".execute().text
You can save this to a variable or export it as a build parameter.

Related

How to improve the build failure reporting of a prallel gradle build?

We use a parallel gradle build in our ci server. But finding the cause of the build failure is sometimes a pain, because one has to dig through the gradle output to find the real problem. I would like to improve that.
For example one of the gradle tasks uses yarn. If this task fails gradle reports the following in the jenkins console:
10:27:27 * What went wrong:
10:27:27 Execution failed for task ':foo-web:yarn'.
10:27:27 > Process 'command '/home/jenkins/workdir/workspace/foo/foo-web/.gradle/yarn/bin/yarn'' finished with non-zero exit value 1
Then one has to scroll up a lot of lines to find the output of yarn due to parallel tasks being finished, just to find out yarn could not download a dependency.
Can we change the gradle build failure reporting so that it will append the error output of the failed task at the end of the build output to the failure report? Or is there some way to get gradles failure reporting and the output of the failed task more closely together?

Get NOT_BUILT build result in Jenkins when building maven project with selective dependencies

I have a multi-component maven project in Jenkins. This project has a Send Files over FTP Post-Build step. I have set my mvn goals to build just my desired component; not all of them:
clean install -pl component-x,component-y -P develop -X
All the dependencies in my project are built successfully;
[INFO] component-x ............................ SUCCESS [ 5.026 s]
[INFO] component-y ............................ SUCCESS [ 16.912 s]
but the Jenkins says:
FTP: Current build result is [NOT_BUILT], not going to run.
EDIT 1:
Yes, I have read this issue. People suggestions include:
Do it manually.
Use Execute Shell instead.
But there were no solution for How to do it manually.
BTW I have an FTP server which I want to put files on; it's not possible for me to use Execute Shell.
Looks like because of this bug https://issues.jenkins-ci.org/browse/JENKINS-16240
Either you can set the status to success manually or use execute shell for ftp instead of the plugin.
You can run a post build groovy script manager.buildSuccess()
See here
The problem is, that the Publish over FTP plugin checks if the build was successful. Unstable is accepted, too, but NOT_BUILT isn't. If the build is not considered successful, the plugin refuses to run.
It is arguable, if that is an expected behavior. The user might want to transfer files, even if the build was not successful. Besides, the NOT_BUILT obviously refers only to the last build step, not to the overall result which is still SUCCESS.
There is an issue filed under JENKINS-55816.
I've created a patch that does not check for the build result which can be downloaded from here (use at your own risk, with no warranty whatsoever).

Jenkins execute shell

I have a jenkins job which is having 5 build scripts configured in job execute shell. Now I have a flag in script#1 if it's false then I don't want to run remaining 4 build scripts and directly exit then build results should be pass.
But now even after checking script#1 flag is false it's exiting from script#1 and executing remaining 4 build scripts then all the 4 scripts are failing due to dependency with script#1 finally my jenkins build status is failed.
I want after checking script#1 job could exit and directly jenkins job status should be "pass"
Is there anyway in jenkins itself to control this flow? without writing any code in scripts?
Jenkins error :
------------
23:37:16 Nothing to build
23:37:16 SRV_SKIP_UNNECESSARY_BUILDS is set to 1
23:37:16 Exiting now...
23:37:16 +++/usr/atria/bin/cleartool rmview -f -tag swrel_sre_icx_test_ashok_diffreportvu_20180118_233612
23:37:30 SRE_FYI: No rebuild necessary for sre_icx_test_ashok since previous full build[EnvInject] - Injecting environment variables from a build step.
23:37:30 [EnvInject] - [ERROR] - The given properties file path '/var/tmp/jenkins/workspace/Test/sre_icx_test_ashok/srv_env_strip' doesn't exist.
23:37:30 [EnvInject] - [ERROR] - Missing file path was resolved from pattern '${WORKSPACE}/srv_env_strip' .
23:37:30 [EnvInject] - [ERROR] - Problems occurs on injecting env vars as a build step: java.io.IOException: remote file operation failed: /var/tmp/jenkins/workspace/Test/sre_icx_test_ashok at hudson.remoting.Channel#1c0be696:l42-up-ecbld-01: java.io.IOException: The given properties file path '/var/tmp/jenkins/workspace/Test/sre_icx_test_ashok/srv_env_strip' doesn't exist.
23:37:30 Build step 'Inject environment variables' changed build result to FAILURE
23:37:30 Build step 'Inject environment variables' marked build as failure
23:37:30 Set build name.
23:37:30 Unrecognized macro 'SRV_BUILD_LABEL' in '#22 - ${SRV_BUILD_LABEL}'
23:37:30 Archiving artifacts
23:37:30 SSH: Current build result is [FAILURE], not going to run.
23:37:30 [description-setter] Description set: ${SRV_BUILD_DIR}
23:37:30 Notifying upstream projects of job completion
23:37:30 Finished: FAILURE
In theory, you could do so by acquiring a Jenkins plugin called "Fail The Build" and change your job so that:
The first build step writes a flagvar=value to a file which can later be sourced by another script, and then exits with a failure. The failure will skip out of running subsequent build steps and go straight to the post-build actions.
Then you can set in the post-build "Execute a set of scripts->Build steps->Execute shell". Set it to execute steps only if build fails, and then in the first step have it source the first build step's file written to see if it was an abort or an actual failed build. If a failed build, exit again with an error - otherwise let it go to the next post-build build step which calls Fail The Build to change the job status back to Success.
I have not tried this to see if it actually works.

Team City Build Fails when "the build process exit code is not zero" Is Unchecked

I have several build steps that need to run regardless of (test failure, in this case).
However when tests in one step fail, the whole build fails and quits. To overcome this problem I unchecked the "build process exit code is not zero".
After this the build fails more quickly at step 2, which is install of grunt-cli
Is there a (better) way to have my build continue, even on a non-zero exit from a previous step? I tried muting, but this is not what I was hoping for.
TeamCity Enterprise 9.1.7 (build 37573)
If this is a commandline build step, you can add "exit 0" to force the exit code to always be 0. Since you are reporting tests, you will still see the failed tests in TeamCity.
When you configure the build stteps, you can set an execution condition of Execution policy to: Even if some of previous steps failed
You can also set to: Always, even if build stop command was issued if you want to do some cleanup on your agent after the execution of your tests.

Maven surefirebooter

I'm using Jenkins to build my projects on Maven 3.
There is ~15 jobs defined. They are running onCommit or by cron expression all day long (mostly once per hour). Some jobs are using Sonar (XXXDailyBuild not).
Sometime I'm facing error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.4.1:clean (default-clean) on project xxx: Failed to clean project: Failed to delete E:\CI\data\jobs\XXXDailyBuild\workspace\XXX\xxx\target\surefire\surefirebooter8004371362793469171.jar -> [Help 1]
It is caused by locked Jenkins workspace by active java process of surefirebooter8004371362793469171.jar
Jenkins job runs with 'Execute concurrent builds if necessary' disabled.
Any other job/process do not use 'XXXDailyBuild' workspace.
I'm trying to figure out what's going on.
Is there any way to dump running 'surefirebooter8004371362793469171.jar' process to analyze their heap?
Problem has been investigated:
Our application runs JUnit tests by surefirebooter maven plugin jar
Every tests need fresh models state
During models initialization old models state are disposed and fresh are initialized
Some models starts threads ie. Directory Watcher
The root PROBLEM is:
1. when some test fails in step 3. old models state aren't disposed
and this way some threads aren't stopped (duplicated on every test fail)
It was easy to check it using jvisualvm tool.
Finally, after maven test run process stays active forever.
Hope this analysis help someone!
Don't know the root cause and resolution. However, you can kill the java.exe from Task Manager (in Windows). This java.exe holds a lock on surefirebooter.jar file. Killing java.exe will Release that lock and you will be able to run the Maven Build.

Resources