Share environment variables from pipeline to downstream job - maven

I have two Maven projects in Jenkins, which download source code and then build, using mvn clean install and a pipeline that combines these two steps in a build stage.
The Jenkins maven plugin, expects to find an environment variable for the location of the maven repository. Namely, ${env.MAVEN_REPO}.
My goal is to define this variable in the pipeline and so that downstream maven build jobs use it.
This is my pipeline:
pipeline {
agent any
environment {
MAVEN_REPO = "C:\\Users\\Chris\\development\\repositories\\maven"
}
stages {
stage('Build') {
steps {
echo MAVEN_REPO
echo env.MAVEN_REPO
echo "${env.MAVEN_REPO}"
build job: 'projectA'
build job: 'projectB'
}
}
}
}
All three echo's above, print the already set value.
Now, in the downstream job, I have created "pre step > Execute Windows batch command", which contains:
echo MAVEN_REPO is %MAVEN_REPO%
echo List of variables:
SET
The problem is that MAVEN_REPO variable does not have a value.
Finally, printing all environment variables simply verifies that no MAVEN_REPO variable is defined.
I have even trying supplying the variable as job parameter, like so:
pipeline {
agent any
environment {
MAVEN_REPO = "C:\\Users\\Chris\\development\\repositories\\maven"
}
stages {
stage('Build') {
steps {
echo MAVEN_REPO
echo env.MAVEN_REPO
echo "${env.MAVEN_REPO}"
build job: 'projectA', parameters: [[$class: 'StringParameterValue', name: 'MAVEN_REPO', value: 'C:\\Users\\Chris\\development\\repositories\\maven']]
build job: 'projectB'
}
}
}
}
but still the variable does not have a value when trying to print it using the downstream job's pre-step phase and running this bat command echo MAVEN_REPO is %MAVEN_REPO%.
Note: My Jenkins instance currently runs on Windows 10.

Related

How can we trigger Jenkins build in iterative method

I have two jobs in Jenkins upstream and downstream.
When I trigger upstream job, below files has to be renamed as package.xml and deploy to downstream in iterative way. How can I make this done with shell script.
Any Idea?
pkg1.xml
pkg2.xml
pkg3.xml
i'm not sure what you're trying to do with the second part (maybe file another more detailed question), but here's how to rename files:
pipeline {
agent { label 'docker' }
stages {
stage('build') {
steps {
# you don't need to create these files.
# this was just for my testing.
sh 'touch pkg1.xml pkg2.xml pkg3.xml'
sh "rename 's/pkg/package/' *"
}
}
}
}

Statement execution order not followed in gradle?

I need to transfer file names from Android application variants to Maven publication artifacts. The construct in my build.gradle is:
publishing {
publications {
maven(MavenPublication) {
groupId android.defaultConfig.applicationId
artifactId 'apk'
version = project.ext.version
artifacts = {
def list = []
android.applicationVariants.all { variant ->
variant.outputs.each { output ->
list.add (output.outputFile.absolutePath)
println "Output " + output.outputFile.absolutePath + " list " + list
}
}
println "To deploy " + list
list.iterator()
}
}
}
}
No artifacts are ever deployed with this script, and it produces the output that looks like a raise of machines:
To deploy []
Output B:\ox\app\build\outputs\apk\app-debug-1.0.apk list [B:\ox\app\build\outputs\apk\app-debug-1.0.apk]
Output B:\ox\app\build\outputs\apk\app-release-1.0.apk list [B:\ox\app\build\outputs\apk\app-debug-1.0.apk, B:\ox\app\build\outputs\apk\app-release-1.0.apk]
So all files were nicely iterated through, but for some reason the last print statement runs before the terminal print statement, and all construct just returns the empty list.
Why does this happen, and how to fix this? I need to set the list of iterated files and the artifacts property for Maven publication.
Gradle is a declarative language, so order of statements is not important. The builds are executed in following way:
1) script is evaluated in order it's written. That's how Gradle find out about what tasks you have in your script and which plugins to load. Not that tasks themselves are not executed at that point.
2) Then Gradle constructs a DAG of all the tasks that need to be executed. Each task can declare dependencies which are the tasks that must be executed beforehand.
3) After DAG is constructed, the tasks are run. The tasks are run according to their declared dependencies. Tasks with no dependencies are run first, possibly in parallel. The tasks that had dependencies are scheduled to run after all their predecessors have finished. Again if possible Gradle will run things in parallel.
3a) Tasks themseleves are run in following order. First doFirst {} blocks are executed, then the task body and after that the doLast {} blocks.
So in your case if you want to compose the list of artifacts, you should enclose your iterator into doFirst {} block.

How can I use a variable value from a project inside of a Exec task in gradle?

I am trying to write an exec task in gradle that would use a variable set on the project.
Something like so:
task upload(type: Exec) {
executable "echo"
args version
}
This always gives me "unspecified"
If I do a task like this
task upload << {
println version
}
It will print the value of the version variable
How can I use the value of version inside an Exec task ?
Thanks for your help Mark Vieira
I found that writing my own task that calls the exec task works the way I need it to.
Like so
task upload << {
exec {
executable "echo"
args version
}
}

How to mark a build unstable in Jenkins when running shell scripts

In a project I'm working on, we are using shell scripts to execute different tasks. Some are sh/bash scripts that run rsync, and some are PHP scripts. One of the PHP scripts is running some integration tests that output to JUnit XML, code coverage reports, and similar.
Jenkins is able to mark the jobs as successful / failed based on exit status. In PHP, the script exits with 1 if it has detected that the tests failed during the run. The other shell scripts run commands and use the exit codes from those to mark a build as failed.
// :: End of PHP script:
// If any tests have failed, fail the build
if ($build_error) exit(1);
In Jenkins Terminology, an unstable build is defined as:
A build is unstable if it was built successfully and one or more publishers report it unstable. For example if the JUnit publisher is configured and a test fails then the build will be marked unstable.
How can I get Jenkins to mark a build as unstable instead of only success / failed when running shell scripts?
Modern Jenkins versions (since 2.26, October 2016) solved this: it's just an advanced option for the Execute shell build step!
You can just choose and set an arbitrary exit value; if it matches, the build will be unstable. Just pick a value which is unlikely to be launched by a real process in your build.
It can be done without printing magic strings and using TextFinder. Here's some info on it.
Basically you need a .jar file from http://yourserver.com/cli available in shell scripts, then you can use the following command to mark a build unstable:
java -jar jenkins-cli.jar set-build-result unstable
To mark build unstable on error, you can use:
failing_cmd cmd_args || java -jar jenkins-cli.jar set-build-result unstable
The problem is that jenkins-cli.jar has to be available from shell script. You can either put it in easy-to-access path, or download in via job's shell script:
wget ${JENKINS_URL}jnlpJars/jenkins-cli.jar
Use the Text-finder plugin.
Instead of exiting with status 1 (which would fail the build), do:
if ($build_error) print("TESTS FAILED!");
Than in the post-build actions enable the Text Finder, set the regular expression to match the message you printed (TESTS FAILED!) and check the "Unstable if found" checkbox under that entry.
You should use Jenkinsfile to wrap your build script and simply mark the current build as UNSTABLE by using currentBuild.result = "UNSTABLE".
stage {
status = /* your build command goes here */
if (status === "MARK-AS-UNSTABLE") {
currentBuild.result = "UNSTABLE"
}
}
you should also be able to use groovy and do what textfinder did
marking a build as un-stable with groovy post-build plugin
if(manager.logContains("Could not login to FTP server")) {
manager.addWarningBadge("FTP Login Failure")
manager.createSummary("warning.gif").appendText("<h1>Failed to login to remote FTP Server!</h1>", false, false, false, "red")
manager.buildUnstable()
}
Also see Groovy Postbuild Plugin
In my job script, I have the following statements (this job only runs on the Jenkins master):
# This is the condition test I use to set the build status as UNSTABLE
if [ ${PERCENTAGE} -gt 80 -a ${PERCENTAGE} -lt 90 ]; then
echo WARNING: disc usage percentage above 80%
# Download the Jenkins CLI JAR:
curl -o jenkins-cli.jar ${JENKINS_URL}/jnlpJars/jenkins-cli.jar
# Set build status to unstable
java -jar jenkins-cli.jar -s ${JENKINS_URL}/ set-build-result unstable
fi
You can see this and a lot more information about setting build statuses on the Jenkins wiki: https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CLI
Configure PHP build to produce xml junit report
<phpunit bootstrap="tests/bootstrap.php" colors="true" >
<logging>
<log type="junit" target="build/junit.xml"
logIncompleteSkipped="false" title="Test Results"/>
</logging>
....
</phpunit>
Finish build script with status 0
...
exit 0;
Add post-build action Publish JUnit test result report for Test report XMLs. This plugin will change Stable build to Unstable when test are failing.
**/build/junit.xml
Add Jenkins Text Finder plugin with console output scanning and unchecked options. This plugin fail whole build on fatal error.
PHP Fatal error:
Duplicating my answer from here because I spent some time looking for this:
This is now possible in newer versions of Jenkins, you can do something like this:
#!/usr/bin/env groovy
properties([
parameters([string(name: 'foo', defaultValue: 'bar', description: 'Fails job if not bar (unstable if bar)')]),
])
stage('Stage 1') {
node('parent'){
def ret = sh(
returnStatus: true, // This is the key bit!
script: '''if [ "$foo" = bar ]; then exit 2; else exit 1; fi'''
)
// ret can be any number/range, does not have to be 2.
if (ret == 2) {
currentBuild.result = 'UNSTABLE'
} else if (ret != 0) {
currentBuild.result = 'FAILURE'
// If you do not manually error the status will be set to "failed", but the
// pipeline will still run the next stage.
error("Stage 1 failed with exit code ${ret}")
}
}
}
The Pipeline Syntax generator shows you this in the advanced tab:
I find the most flexible way to do this is by reading a file in the groovy post build plugin.
import hudson.FilePath
import java.io.InputStream
def build = Thread.currentThread().executable
String unstable = null
if(build.workspace.isRemote()) {
channel = build.workspace.channel;
fp = new FilePath(channel, build.workspace.toString() + "/build.properties")
InputStream is = fp.read()
unstable = is.text.trim()
} else {
fp = new FilePath(new File(build.workspace.toString() + "/build.properties"))
InputStream is = fp.read()
unstable = is.text.trim()
}
manager.listener.logger.println("Build status file: " + unstable)
if (unstable.equalsIgnoreCase('true')) {
manager.listener.logger.println('setting build to unstable')
manager.buildUnstable()
}
If the file contents are 'true' the build will be set to unstable. This will work on the local master and on any slaves you run the job on, and for any kind of scripts that can write to disk.
I thought I would post another answer for people that might be looking for something similar.
In our build job we have cases where we would want the build to continue, but be marked as unstable. For ours it's relating to version numbers.
So, I wanted to set a condition on the build and set the build to unstable if that condition is met.
I used the Conditional step (single) option as a build step.
Then I used Execute system Groovy script as the build step that would run when that condition is met.
I used Groovy Command and set the script the following
import hudson.model.*
def build = Thread.currentThread().executable
build.#result = hudson.model.Result.UNSTABLE
return
That seems to work quite well.
I stumbled upon the solution here
http://tech.akom.net/archives/112-Marking-Jenkins-build-UNSTABLE-from-environment-inject-groovy-script.html
In addition to all others answers, jenkins also allows the use of the unstable() method (which is in my opinion clearer).
This method can be used with a message parameter which describe why the build is unstable.
In addition of this, you can use the returnStatus of your shell script (bat or sh) to enable this.
For example:
def status = bat(script: "<your command here>", returnStatus: true)
if (status != 0) {
unstable("unstable build because script failed")
}
Of course, you can make something with more granularity depending on your needs and the return status.
Furthermore, for raising error, you can also use warnError() in place of unstable(). It will indicate your build as failed instead of unstable, but the syntax is same.
The TextFinder is good only if the job status hasn't been changed from SUCCESS to FAILED or ABORTED.
For such cases, use a groovy script in the PostBuild step:
errpattern = ~/TEXT-TO-LOOK-FOR-IN-JENKINS-BUILD-OUTPUT.*/;
manager.build.logFile.eachLine{ line ->
errmatcher=errpattern.matcher(line)
if (errmatcher.find()) {
manager.build.#result = hudson.model.Result.NEW-STATUS-TO-SET
}
}
See more details in a post I've wrote about it:
http://www.tikalk.com/devops/JenkinsJobStatusChange/
As a lighter alternative to the existing answers, you can set the build result with a simple HTTP POST to access the Groovy script console REST API:
curl -X POST \
--silent \
--user "$YOUR_CREDENTIALS" \
--data-urlencode "script=Jenkins.instance.getItemByFullName( '$JOB_NAME' ).getBuildByNumber( $BUILD_NUMBER ).setResult( hudson.model.Result.UNSTABLE )" $JENKINS_URL/scriptText
Advantages:
no need to download and run a huge jar file
no kludges for setting and reading some global state (console text, files in workspace)
no plugins required (besides Groovy)
no need to configure an extra build step that is superfluous in the PASSED or FAILURE cases.
For this solution, your environment must meet these conditions:
Jenkins REST API can be accessed from slave
Slave must have access to credentials that allows to access the Jenkins Groovy script REST API.
If you want to use a declarative approach I suggest you to use code like this.
pipeline {
stages {
// create separate stage only for problematic command
stage("build") {
steps {
sh "command"
}
post {
failure {
// set status
unstable 'rsync was unsuccessful'
}
always {
echo "Do something at the end of stage"
}
}
}
}
post {
always {
echo "Do something at the end of pipeline"
}
}
}
In case you want to keep everything in one stage use catchError
pipeline {
stages {
// create separate stage only for problematic command
stage("build") {
steps {
catchError(stageResult: 'UNSTABLE') {
sh "command"
}
sh "other command"
}
}
}
}
One easy way to set a build as unstable, is in your "execute shell" block, run exit 13
You can just call "exit 1", and the build will fail at that point and not continue. I wound up making a passthrough make function to handle it for me, and call safemake instead of make for building:
function safemake {
make "$#"
if [ "$?" -ne 0 ]; then
echo "ERROR: BUILD FAILED"
exit 1
else
echo "BUILD SUCCEEDED"
fi
}

What is the difference between these task definition syntaxes in gradle?

A)
task build << {
description = "Build task."
ant.echo('build')
}
B)
task build {
description = "Build task."
ant.echo('build')
}
I notice that with type B, the code within the task seems to be executed when typing gradle -t - ant echoes out 'build' even when just listing all the various available tasks. The description is also actually displayed with type B. However, with type A no code is executed when listing out the available tasks, and the description is not displayed when executing gradle -t. The docs don't seem to go into the difference between these two syntaxes (that I've found), only that you can define a task either way.
The first syntax defines a task, and provides some code to be executed when the task executes. The second syntax defines a task, and provides some code to be executed straight away to configure the task. For example:
task build << { println 'this executes when build task is executed' }
task build { println 'this executes when the build script is executed' }
In fact, the first syntax is equivalent to:
task build { doLast { println 'this executes when build task is executed' } }
So, in your example above, for syntax A the description does not show up in gradle -t because the code which sets the description is not executed until the task executed, which does not happen when you run gradle -t.
For syntax B the code that does the ant.echo() is run for every invocation of gradle, including gradle -t
To provide both an action to execute and a description for the task you can do either of:
task build(description: 'some description') << { some code }
task build { description = 'some description'; doLast { some code } }

Resources