List in place sorting in Jenkins Pipelines - sorting

I am trying to sort a list of objects in Jenkins pipelines. I'm getting different results running code below locally or within Jenkins:
pipeline {
agent any
stages {
stage('default'){
steps {
script {
#NonCPS
def nonCpsTest = {
def list = [
['CreationDate': '200'],
['CreationDate': '300'],
['CreationDate': '100'],
]
def rval = list.sort { it['CreationDate'] }
echo "Rval=$rval"
echo "List=$list"
}
nonCpsTest()
}
}
}
}
}
When I execute this script locally using groovy shell (groovysh) result is
groovy:000> list = [[ 'CreationDate':200 ], [ 'CreationDate':300 ], [ 'CreationDate':100 ]]
===> [[CreationDate:200], [CreationDate:300], [CreationDate:100]]
groovy:000> rval = list.sort { it['CreationDate'] }
===> [[CreationDate:100], [CreationDate:200], [CreationDate:300]]
groovy:000> list
===> [[CreationDate:100], [CreationDate:200], [CreationDate:300]]
groovy:000> list == rval
===> true
While on the Jenkins server I'm getting following
[Pipeline] {
[Pipeline] stage
[Pipeline] { (default)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Rval=300
[Pipeline] echo
List=[[CreationDate:200], [CreationDate:300], [CreationDate:100]]
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Is Jenkins workflow making list immutable in anyway, or overriding sort method, and if so, how to do in place list sorting within the Jenkins pipeline code?

the problem that you declared nonCpsTest as a variable and it references to closure, so #NonCPS does not work in this case
the following variant works fine:
#NonCPS
def nonCpsTest() {
def list = [
['CreationDate': '200'],
['CreationDate': '300'],
['CreationDate': '100'],
]
def rval = list.sort{ it['CreationDate'] }
echo "Rval=$rval"
echo "List=$list"
}
node{
nonCpsTest()
}

Related

Jenkins File Environment variables

I have the below Jenkins pipeline for my project where I am setting a variable COLLECTOR_TOKEN which will be set via shell script.
I want to handle a negative scenario where suppose if the shell script fails I send and exit code 1, which in return fails the entire pipeline (this is what we are expecting) but the issue is in the logs we cant see the proper error message, so it will hard to debug.
pipeline {
agent { label 'maven' }
options { timeout(time: 300, unit: 'MINUTES') }
environment {
def COLLECTOR_TOKEN = sh(script: "chmod -R 777 *; hello.sh moshin", returnStdout: true).trim()
}
}
Exception : We can just see the below exception.
I want to handle the below exception and throw proper message. how can we achieve it?
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
at org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:266)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:263)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at stepFinalise.call(stepFinalise.groovy:3)
at WorkflowScript.run(WorkflowScript:56)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(ModelInterpreter.groovy:137)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:756)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:395)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:393)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:755)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2030)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2015)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2056)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:745)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executePostBuild(ModelInterpreter.groovy:723)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:107)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:89)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
I got the solution :
As I was using returnStdout with shell script and the result for that shell script was not always the correct output sometime I use to get some exception which was not getting handled in the Jenkins files. So I had to handle the exception of the shell script by sending a error message instead of exit 1. And then in pipeline stage step we had to validate whether we got the token or we got exception (based on the message we sent from shell script).
If exception then stop the build.
Below code is just for your reference.
moshin.sh file:
#!/bin/bash
if [[ $1 == "moshin" ]]; then
echo "moshin"
else
echo "not_moshin"
fi
Jenkins Pipeline:
pipeline {
agent { label 'maven' }
environment {
COLLECTOR_TOKEN = sh(script: "chmod -R 777 *; moshin.sh moshin" , returnStdout: true).trim()
COLLECTOR_TOKEN_1 = sh(script: "chmod -R 777 *; moshin.sh not_moshin" , returnStdout: true).trim()
}
stages {
stage('PASS') {
steps {
script {
if (env.COLLECTOR_TOKEN == "moshin") {
println "hello moshin"
} else {
println "Unable to fetch COLLECTOR_TOKEN"
currentBuild.result = 'FAILED'
sh "exit 1"
}
}
}
}
stage('FAIL') {
steps {
script {
if (env.COLLECTOR_TOKEN_1 == "moshin") {
println "hello moshin"
} else {
println "Unable to fetch COLLECTOR_TOKEN"
currentBuild.result = 'FAILED'
sh "exit 1"
}
}
}
}
}
}
Build output:
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ chmod -R 777 Jenkinsfile moshin.sh
+ moshin.sh not_moshin
[Pipeline] sh
+ chmod -R 777 Jenkinsfile moshin.sh
+ moshin.sh moshin
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (PASS)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
hello moshin
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (FAIL)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Unable to fetch COLLECTOR_TOKEN
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE

Bad substitution when passing parameter to shell script in Jenkins

I'm attempting to pass value of a variable by setting the stdOut of a shell script.
However in The console of Jenkins displays of followings:
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Feature Segunda)
[Pipeline] echo
${params.Segunda}
[Pipeline] sh
/var/jenkins_home/workspace/pruebaParametrizada#tmp/durable-71aead85/script.sh: 1: /var/jenkins_home/workspace/pruebaParametrizada#tmp/durable-71aead85/script.sh: Bad substitution
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] echo
Building finished successfully
I use to escape the quotes, nothing works. I get a bad substitution error. I've also tried without double quotes.
If I hardcode in the shell script arguments, it runs fine.
pipeline {
agent any
parameters {
string(defaultValue: "", description: '', name: 'One')
string(defaultValue: "", description: '', name: 'Two')
}
stages {
stage('Git Checkout') {
steps {
git credentialsId: 'personal-github', url: 'https:xxx'
}
}
stage('Maven Compile') {
steps {
sh 'mvn clean compile'
}
}
stage('Test One') {
steps {
//ERROR
sh 'mvn test -Dcucumber.options="-t #${params.One}"'
//This Works
//sh 'mvn test -Dcucumber.options="-t #One"'
}
}
}
post {
always {
echo 'Building finished successfully'
cucumber failedFeaturesNumber: -1,
failedScenariosNumber: -1,
failedStepsNumber: -1,
fileIncludePattern: '**/*.json',
jsonReportDirectory: 'target/cucumber/',
pendingStepsNumber: -1,
reportTitle: 'test features',
skippedStepsNumber: -1,
sortingMethod: 'ALPHABETICAL',
undefinedStepsNumber: -1
}
}
}
I found the way to pass parameters as follows`
sh """mvn test -Dcucumber.options='-t #${params.One}'"""
Thanks to all, for responses.

Jenkins declarative pipeline get environment variable at post stage

I am getting runtime value in build stage stage which I stored in an environment variable . I saved that to env.cfg file under WORKSPACE .
Now I am trying to get that value in post pipeline step to be used in email communication. I tried load method but it did not work
Any help ?
post {
always {
echo $SNAPSHOT / /this always comes null
}
}
This is the way you can access an environment variable across the pipeline
pipeline {
agent any;
environment {
MESSAGE="Hello World"
}
stages {
stage('one') {
steps {
echo "${env.MESSAGE}"
sh "echo $MESSAGE"
script {
print env.MESSAGE
}
}
}
}
post {
success {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
failure {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
}
}
but as per your scenario let say I have a file called .env with the content below in the current Jenkins job WORKSPACE and I want to read and make this env variable in the pipeline.
.env
export SNAPSHOT=1.0.0
export MESSAGE='Hello World'
export MESSAGE_FROM_ENV_FILE='Hello From .env file'
your pipeline should look like
scripted pipeline
node {
stage('one') {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
declarative pipeline
pipeline {
agent any;
stages {
stage('build') {
steps {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
post {
success {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
You need a global variable:
SNAPSHOT = ""
println "SNAPSHOT is ${SNAPSHOT}"
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
println "SNAPSHOT is ${SNAPSHOT}"
SNAPSHOT = "Modified"
println "SNAPSHOT is now ${SNAPSHOT}"
}
}
}
}
post {
always {
echo "${SNAPSHOT}"
}
}
}

Jenkinsfile setting stageResult variable based on conditions

I'm trying to script a Jenkinsfile in Declarative Syntax, requirement here is if set environment variables are not a match then pipeline stage should be depict as "Aborted" but the build status could be either Aborted or unstable, on Jenkins online documentation and from Snippet Generator did see the following style
catchError(buildResult: 'ABORTED', stageResult: 'ABORTED') {
// some block
}
catchError does not serve my purpose as it is appropriate for use when the script inside the stage does not return true or if there is an execution error, although default when condition in Jenkins Declarative Syntax is appropriate, Jenkins does not allow setting stage result to 'ABORTED'
when {
expression {
SCM_BRANCH_NAME ==~ /(master|QA)/
}
expression{
ENVIRONMENT ==~ /(QA)/
}
allOf{
environment ignoreCase: true,name: 'PRODUCT_NAME' , value: 'PRODUCT-1'
}
}
Please view the Sample Jenkinsfile below using if and else format
Sample Jenkinsfile
pipeline {
agent {
node {
label 'master'
}
}
environment{
ENVIRONMENT = "QA"
PRODUCT_NAME = "PRODUCT-1"
SCM_BRANCH_NAME = "master"
}
stages{
stage('Testing-1') {
when{
expression{
ENVIRONMENT ==~ /(QA)/
}
}
steps {
script {
if (PRODUCT_NAME == PRODUCT-1){
sh """
echo "Reached Here ${ENVIRONMENT} - ${PRODUCT_NAME} - ${SCM_BRANCH_NAME}"
// do testing for product 1
"""
}
else{
stageResult = 'ABORTED'
echo "PRODUCT not Available for Testing"
}
}
}
}
stage('Testing-2'){
steps{
sh '''
echo "Reached Second Stage"
'''
}
}
}
}
Any suggestions of how to implement the scenario that if conditions are not met set the stageResult as Abort, any suggestions either with a plugin or with a sample notation script is greatly appreciated
Thanks
Below snippet worked for me. You can use catch error block and throw the error from that block when some condition met. You can also catch the exception in the post stage failure section.
pipeline {
agent {
node {
label 'master'
}
}
environment{
ENVIRONMENT = "QA"
PRODUCT_NAME = "PRODUCT-1"
SCM_BRANCH_NAME = "master"
}
stages{
stage('Testing-1') {
when{
expression{
ENVIRONMENT ==~ /(QA)/
}
}
steps {
script {
catchError(buildResult: 'FAILURE', stageResult: 'ABORTED'){
if (PRODUCT_NAME != PRODUCT-1) {
error ("PRODUCT not Available for Testing")
}
sh """
echo "Reached Here ${ENVIRONMENT} - ${PRODUCT_NAME} - ${SCM_BRANCH_NAME}"
// do testing for product 1
"""
}
}
}
post {
failure {
echo "Something Failed and error has been catched"
}
}
}
stage('Testing-2'){
steps{
sh '''
echo "Reached Second Stage"
'''
}
}
}

Jenkins Shared Libraries: is it possible to pass arguments to shell scripts imported as 'libraryResource'?

I have the following setup:
(Stripped out) Jenkinsfile:
#Library('my-custom-library') _
pipeline {
agent any
stages {
stage('Example') {
steps {
printHello name: 'Jenkins'
}
}
}
}
my-custom-library/resources/com/org/scripts/print-hello.sh:
#!/bin/bash
echo "Hello, $1"
my-custom-library/vars/printHello.groovy:
def call(Map parameters = [:]) {
def printHelloScript = libraryResource 'com/org/scripts/print-hello.sh'
def name = parameters.name
//the following line gives me headaches
sh(printHelloScript(name))
}
I expect Hello, Jenkins, but it throws the following exception:
groovy.lang.MissingMethodException: No signature of method:
java.lang.String.call() is applicable for argument types:
(java.lang.String) values: [Jenkins]
Possible solutions: wait(), any(), wait(long),
split(java.lang.String), take(int), each(groovy.lang.Closure)
So, is it possible to do something like described above, without mixing Groovy and Bash code?
Yes, check out withEnv
The example they give looks like;
node {
withEnv(['MYTOOL_HOME=/usr/local/mytool']) {
sh '$MYTOOL_HOME/bin/start'
}
}
More applicable to you:
// resources/test.sh
echo "HI here we are - $PUPPY_DOH --"
// vars/test.groovy
def call() {
withEnv(['PUPPY_DOH=bobby']) {
sh(libraryResource('test.sh'))
}
}
Prints:
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] libraryResource
[Pipeline] sh
+ echo HI here we are - bobby --
HI here we are - bobby --
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
Using that, you can pass it in using a scoped named variable, something like
def call(Map parameters = [:]) {
def printHelloScript = libraryResource 'com/org/scripts/print-hello.sh'
def name = parameters.name
withEnv(['NAME=' + name]) { // This may not be 100% syntax here ;)
sh(printHelloScript)
}
// print-hello.sh
echo "Hello, $name"

Resources