Bad substitution when passing parameter to shell script in Jenkins - bash

I'm attempting to pass value of a variable by setting the stdOut of a shell script.
However in The console of Jenkins displays of followings:
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Feature Segunda)
[Pipeline] echo
${params.Segunda}
[Pipeline] sh
/var/jenkins_home/workspace/pruebaParametrizada#tmp/durable-71aead85/script.sh: 1: /var/jenkins_home/workspace/pruebaParametrizada#tmp/durable-71aead85/script.sh: Bad substitution
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] echo
Building finished successfully
I use to escape the quotes, nothing works. I get a bad substitution error. I've also tried without double quotes.
If I hardcode in the shell script arguments, it runs fine.
pipeline {
agent any
parameters {
string(defaultValue: "", description: '', name: 'One')
string(defaultValue: "", description: '', name: 'Two')
}
stages {
stage('Git Checkout') {
steps {
git credentialsId: 'personal-github', url: 'https:xxx'
}
}
stage('Maven Compile') {
steps {
sh 'mvn clean compile'
}
}
stage('Test One') {
steps {
//ERROR
sh 'mvn test -Dcucumber.options="-t #${params.One}"'
//This Works
//sh 'mvn test -Dcucumber.options="-t #One"'
}
}
}
post {
always {
echo 'Building finished successfully'
cucumber failedFeaturesNumber: -1,
failedScenariosNumber: -1,
failedStepsNumber: -1,
fileIncludePattern: '**/*.json',
jsonReportDirectory: 'target/cucumber/',
pendingStepsNumber: -1,
reportTitle: 'test features',
skippedStepsNumber: -1,
sortingMethod: 'ALPHABETICAL',
undefinedStepsNumber: -1
}
}
}

I found the way to pass parameters as follows`
sh """mvn test -Dcucumber.options='-t #${params.One}'"""
Thanks to all, for responses.

Related

Jenkins File Environment variables

I have the below Jenkins pipeline for my project where I am setting a variable COLLECTOR_TOKEN which will be set via shell script.
I want to handle a negative scenario where suppose if the shell script fails I send and exit code 1, which in return fails the entire pipeline (this is what we are expecting) but the issue is in the logs we cant see the proper error message, so it will hard to debug.
pipeline {
agent { label 'maven' }
options { timeout(time: 300, unit: 'MINUTES') }
environment {
def COLLECTOR_TOKEN = sh(script: "chmod -R 777 *; hello.sh moshin", returnStdout: true).trim()
}
}
Exception : We can just see the below exception.
I want to handle the below exception and throw proper message. how can we achieve it?
Error when executing always post condition:
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
at org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:266)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:263)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at stepFinalise.call(stepFinalise.groovy:3)
at WorkflowScript.run(WorkflowScript:56)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.delegateAndExecute(ModelInterpreter.groovy:137)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:756)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:395)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.catchRequiredContextForNode(ModelInterpreter.groovy:393)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:755)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2030)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2015)
at com.cloudbees.groovy.cps.CpsDefaultGroovyMethods.each(CpsDefaultGroovyMethods:2056)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy:745)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.runPostConditions(ModelInterpreter.groovy)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelInterpreter.executePostBuild(ModelInterpreter.groovy:723)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:107)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:89)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at jdk.internal.reflect.GeneratedMethodAccessor516.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
I got the solution :
As I was using returnStdout with shell script and the result for that shell script was not always the correct output sometime I use to get some exception which was not getting handled in the Jenkins files. So I had to handle the exception of the shell script by sending a error message instead of exit 1. And then in pipeline stage step we had to validate whether we got the token or we got exception (based on the message we sent from shell script).
If exception then stop the build.
Below code is just for your reference.
moshin.sh file:
#!/bin/bash
if [[ $1 == "moshin" ]]; then
echo "moshin"
else
echo "not_moshin"
fi
Jenkins Pipeline:
pipeline {
agent { label 'maven' }
environment {
COLLECTOR_TOKEN = sh(script: "chmod -R 777 *; moshin.sh moshin" , returnStdout: true).trim()
COLLECTOR_TOKEN_1 = sh(script: "chmod -R 777 *; moshin.sh not_moshin" , returnStdout: true).trim()
}
stages {
stage('PASS') {
steps {
script {
if (env.COLLECTOR_TOKEN == "moshin") {
println "hello moshin"
} else {
println "Unable to fetch COLLECTOR_TOKEN"
currentBuild.result = 'FAILED'
sh "exit 1"
}
}
}
}
stage('FAIL') {
steps {
script {
if (env.COLLECTOR_TOKEN_1 == "moshin") {
println "hello moshin"
} else {
println "Unable to fetch COLLECTOR_TOKEN"
currentBuild.result = 'FAILED'
sh "exit 1"
}
}
}
}
}
}
Build output:
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ chmod -R 777 Jenkinsfile moshin.sh
+ moshin.sh not_moshin
[Pipeline] sh
+ chmod -R 777 Jenkinsfile moshin.sh
+ moshin.sh moshin
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (PASS)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
hello moshin
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (FAIL)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Unable to fetch COLLECTOR_TOKEN
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE

How to trigger a multiple run in a single pipeline job of jenkins?

I have a pipeline job which run with below pipeline groovy script,
pipeline {
parameters{
string(name: 'Unique_Number', defaultValue: '', description: 'Enter Unique Number')
}
stages {
stage('Build') {
agent { node { label 'Build' } }
steps {
script {
sh build.sh
}
}
stage('Deploy') {
agent { node { label 'Deploy' } }
steps {
script {
sh deploy.sh
}
}
stage('Test') {
agent { node { label 'Test' } }
steps {
script {
sh test.sh
}
}
}
}
I just trigger this job multiple times with different unique ID number as input parameter. So as a result i will have multiple run/build for this job at different stages.
With this, i need to trigger a multiple run/build to be promote to next stage (i.e., from build to deploy or from deploy to test) in this pipeline job as a one single build instead of triggering each and every single run/build to next stage. Is there any possibility?
I was also trying to do the same thing and found no relevant answers. May this help to someone.
This will read a file that contains the Jenkins Job name and run them iteratively from one single job.
Please change below code accordingly in your Jenkins.
pipeline {
agent any
stages {
stage('Hello') {
steps {
script{
git branch: 'Your Branch name', credentialsId: 'Your crendiatails', url: ' Your BitBucket Repo URL '
##To read file from workspace which will contain the Jenkins Job Name ###
def filePath = readFile "${WORKSPACE}/ Your File Location"
##To read file line by line ###
def lines = filePath.readLines()
##To iterate and run Jenkins Jobs one by one ####
for (line in lines) {
build(job: "$line/branchName",
parameters:
[string(name: 'vertical', value: "${params.vertical}"),
string(name: 'environment', value: "${params.environment}"),
string(name: 'branch', value: "${params.aerdevops_branch}"),
string(name: 'project', value: "${params.host_project}")
]
)
}
}
}
}
}
}
You can start multiple jobs from one pipeline if you run something as:
build job:"One", wait: false
build job:"Two", wait: false
Your main job starts children pipelines and children pipelines will run in parallel.
You can read PipeLine Build Step documentation for more information.
Also, you can read about the parallel run in declarative pipeline
Here you can find a lot of examples for parallel running

Jenkins Shared Libraries: is it possible to pass arguments to shell scripts imported as 'libraryResource'?

I have the following setup:
(Stripped out) Jenkinsfile:
#Library('my-custom-library') _
pipeline {
agent any
stages {
stage('Example') {
steps {
printHello name: 'Jenkins'
}
}
}
}
my-custom-library/resources/com/org/scripts/print-hello.sh:
#!/bin/bash
echo "Hello, $1"
my-custom-library/vars/printHello.groovy:
def call(Map parameters = [:]) {
def printHelloScript = libraryResource 'com/org/scripts/print-hello.sh'
def name = parameters.name
//the following line gives me headaches
sh(printHelloScript(name))
}
I expect Hello, Jenkins, but it throws the following exception:
groovy.lang.MissingMethodException: No signature of method:
java.lang.String.call() is applicable for argument types:
(java.lang.String) values: [Jenkins]
Possible solutions: wait(), any(), wait(long),
split(java.lang.String), take(int), each(groovy.lang.Closure)
So, is it possible to do something like described above, without mixing Groovy and Bash code?
Yes, check out withEnv
The example they give looks like;
node {
withEnv(['MYTOOL_HOME=/usr/local/mytool']) {
sh '$MYTOOL_HOME/bin/start'
}
}
More applicable to you:
// resources/test.sh
echo "HI here we are - $PUPPY_DOH --"
// vars/test.groovy
def call() {
withEnv(['PUPPY_DOH=bobby']) {
sh(libraryResource('test.sh'))
}
}
Prints:
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] libraryResource
[Pipeline] sh
+ echo HI here we are - bobby --
HI here we are - bobby --
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
Using that, you can pass it in using a scoped named variable, something like
def call(Map parameters = [:]) {
def printHelloScript = libraryResource 'com/org/scripts/print-hello.sh'
def name = parameters.name
withEnv(['NAME=' + name]) { // This may not be 100% syntax here ;)
sh(printHelloScript)
}
// print-hello.sh
echo "Hello, $name"

how to keep process running after the stage is finished for declarative jenkins pipeline

pipeline {
agent none
stages {
stage('Server') {
agent{
node {
label "xxx"
customWorkspace "/home/xxx/server"
}
}
steps {
sh 'node server.js &'
//start server
}
}
stage('RunCase') {
agent{
node {
label 'clientServer'
customWorkspace "/home/xxx/CITest"
}
}
steps{
sh 'start test'
sh 'run case here'
}
}
}
}
I create above Jenkins pipeline. What I want to do is:
1. start server at server node.
2. start test at test node.
However, I found the server process will be closed when second stage start.
So how to keep server start until my second stage testing work is finished. I try to use &, still not working. It seems it will kill all process I started at first stage.
One solution is to try to start the two stages in "parallel"-mode. For more informations see this two files: parallel-declarative-blog jenkins-pipeline-syntax. But be carefull, because it is not ensured, that the first stage starts before the second one starts. Maybe you need a waiting time for your tests. Here is an example Jenkinsfile:
pipeline {
agent none
stages {
stage('Run Tests') {
parallel {
stage('Start Server') {
steps {
sh 'node server.js &'
}
}
stage('Run Tests) {
steps {
sh 'run case here'
}
}
}
}
}
}
Another solution would be to start the node server in the background. For this you can try different tools, like nohup or pm2.

List in place sorting in Jenkins Pipelines

I am trying to sort a list of objects in Jenkins pipelines. I'm getting different results running code below locally or within Jenkins:
pipeline {
agent any
stages {
stage('default'){
steps {
script {
#NonCPS
def nonCpsTest = {
def list = [
['CreationDate': '200'],
['CreationDate': '300'],
['CreationDate': '100'],
]
def rval = list.sort { it['CreationDate'] }
echo "Rval=$rval"
echo "List=$list"
}
nonCpsTest()
}
}
}
}
}
When I execute this script locally using groovy shell (groovysh) result is
groovy:000> list = [[ 'CreationDate':200 ], [ 'CreationDate':300 ], [ 'CreationDate':100 ]]
===> [[CreationDate:200], [CreationDate:300], [CreationDate:100]]
groovy:000> rval = list.sort { it['CreationDate'] }
===> [[CreationDate:100], [CreationDate:200], [CreationDate:300]]
groovy:000> list
===> [[CreationDate:100], [CreationDate:200], [CreationDate:300]]
groovy:000> list == rval
===> true
While on the Jenkins server I'm getting following
[Pipeline] {
[Pipeline] stage
[Pipeline] { (default)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Rval=300
[Pipeline] echo
List=[[CreationDate:200], [CreationDate:300], [CreationDate:100]]
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Is Jenkins workflow making list immutable in anyway, or overriding sort method, and if so, how to do in place list sorting within the Jenkins pipeline code?
the problem that you declared nonCpsTest as a variable and it references to closure, so #NonCPS does not work in this case
the following variant works fine:
#NonCPS
def nonCpsTest() {
def list = [
['CreationDate': '200'],
['CreationDate': '300'],
['CreationDate': '100'],
]
def rval = list.sort{ it['CreationDate'] }
echo "Rval=$rval"
echo "List=$list"
}
node{
nonCpsTest()
}

Resources