Jenkins pipeline : using variable created in shell 1, in shell 2 - shell

In my pipeline I've 2 stages, and they both call a different shell script (the way they're called is some custom code as we have different pipelines interacting with each other) :
stage('first') {
steps {
script {
stageErrors.add(launchPortalStep.launchStepWithVerification('shell',"chmod +x ./script1.sh && ./script1.sh"))
}
}
}
stage('second') {
steps {
script {
stageErrors.add(launchPortalStep.launchStepWithVerification('shell',"chmod +x ./script2.sh && ./script2.sh"))
}
}
}
The thing is, in script2 I need to use a variable that is set in script1, the only ways I've think of that could work would mean pretty much rewriting everything and that's not a possibility.
Script1 must return either 0 or 1, so it's not possible to return the value and stock in an environment variable that would be then passed as a parameter in script2.
Scripts are also way too long to have them entirely in the pipeline using sh commands (that would be adding at least 700 lines).
What can I do ?

You should use a properties file (it can be even a temp file) to pass the needed variables between the stages.
You should write the needed variables into a properties file (in VAR=VAL format) in the shell script which is called in First step (sh 'echo "MY_ENV_VAR=test_var" > test.prop').
You should read the properties file in the Second step with the readProperties function which is part of the Pipeline Utility Steps plugin def props = readProperties file: 'test.pro). Then set the needed variables as environment variables (env.MY_ENV_VAR = props.MY_ENV_VAR) so you will be able to use the variables in your second shell script as environment variables (sh 'echo MY_ENV_VAR = ${MY_ENV_VAR}').
Complete example code:
pipeline {
agent { label 'MISC' }
stages{
stage('First'){
steps {
sh 'echo "MY_ENV_VAR=test_var" > test.prop'
}
}
stage('Second'){
steps {
script {
// readProperties is a step in Pipeline Utility Steps plugin
def props = readProperties file: 'test.prop'
// Assuming the key name is MY_ENV_VAR in properties file.
env.MY_ENV_VAR = props.MY_ENV_VAR
sh 'echo MY_ENV_VAR = ${MY_ENV_VAR}'
}
}
}
}
}
Output:
[Pipeline] stage
[Pipeline] { (First)
[Pipeline] sh
+ echo MY_ENV_VAR=test_var
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Second)
[Pipeline] script
[Pipeline] {
[Pipeline] readProperties
[Pipeline] sh
+ echo MY_ENV_VAR = test_var
MY_ENV_VAR = test_var
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage

Related

sh command is not executing in jenkins pipeline project

While executing the sh command in jenkins pipeline project, we are getting an an error like below and sh command is not working.
Error message:
[Pipeline] sh
Warning: JENKINS-41339 probably bogus PATH=/bin/sh:/usr/atria/bin:/usr/atria/bin:$PATH; perhaps you meant to use ‘PATH+EXTRA=/something/bin’?
process apparently never started in /var/lib/jenkins/workspace/QSearch_pipelineTest#tmp/durable-6d5deef7
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
[Pipeline] }
Below is the jenkins pipeline script code:
pipeline {
agent any
environment {
DATE = "December 17th"
}
stages {
stage("Env Variables") {
environment {
NAME = "Alex"
}
steps {
echo "Today is ${env.DATE}"
echo "My name ${env.NAME}"
echo "My path is ${env.PATH}"
script {
env.WEBSITE = "phoenixNAP KB"
env.PATH = "/bin/sh:$PATH"
}
echo "This is an example for ${env.WEBSITE}"
echo "My path ${env.PATH}"
**sh 'env'**
withEnv(["TEST_VARIABLE=TEST_VALUE"]) {
echo "The value of TEST_VARIABLE is ${env.TEST_VARIABLE}"
}
}
}
}
Below is the output of jenkins build job:
...
[Pipeline] echo
My path /bin/sh:/usr/atria/bin:/usr/atria/bin:$PATH
**[Pipeline] sh
Warning: JENKINS-41339 probably bogus PATH=/bin/sh:/usr/atria/bin:/usr/atria/bin:$PATH; perhaps you meant to use ‘PATH+EXTRA=/something/bin’?
process apparently never started in /var/lib/jenkins/workspace/QSearch_pipelineTest#tmp/durable-6d5deef7
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)**
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
**ERROR: script returned exit code -2**
Finished: FAILURE

Why is Jenkins pipeline returning -#tmp/durable-56090643/script.sh 4: FSUM7728 bad ${} modifier for shell command?

I have this stage in my Jenkins pipeline that runs a command and stores the output in a variable. I'm trying to get the id number from the stored string but getting the error bad ${} modifier. Should have printed 00062100. It works correctly in the console.
stage('test') {
agent {node 'test'}
steps{
sh "string=$(onetstat -a -P 1111)"
sh "echo ${string:6:8}"
}
}
output from the command("BUILD 00062100 Listen")
**Update:**
stage('server2') {
agent {node 'test'}
steps{
sh '''
var="$(onetstat -a -P 1111)"
echo ${var:6:8}
'''
}
}
**log of the run**
[Pipeline] sh
+ + onetstat -a -P 1111
+ 1<TMP> /tmp/shGgcEdAGgA
var=
BUILDER8 00069B50 Listen
Local Socket: 127.0.0.1..1111
Foreign Socket: 0.0.0.0..0
/Build#tmp/durable-a93a2921/script.sh 3: FSUM7728 bad ${} modifier
There are two misunderstandings in your example. When you use double quotes in the Jenkinsfile, you construct a Groovy String that substitutes variables (defined using $ sign) with associated values (or expressions.)
Another misunderstanding is creating a bash variable in one sh step and accessing it in another sh step. It won't work that way. Each sh step runs in its own shell process, and any local variable created in one shell cannot be accessed in another.
You can solve both issues. Firstly, you need to replace double quotes with single quotes in sh step. Secondly, you need to define shell script in a single sh step. You can use Groovy multiline string for that (triple quotes.) Consider the following example:
pipeline {
agent any
stages {
stage("Test") {
steps {
// Below code prints nothing
sh 'something="BUILD 00062100 Listen"'
sh 'echo ${something:6:8}'
// Below code prints 00062100
sh '''
something="BUILD 00062100 Listen"
echo ${something:6:8}
'''
}
}
}
}
Output:
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] sh
+ something='BUILD 00062100 Listen'
[Pipeline] sh
+ echo
[Pipeline] sh
+ something='BUILD 00062100 Listen'
+ echo 00062100
00062100
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Used this solution to get the id from the command output
var=$(onetstat -a -P 1111)
var=$(echo $var | cut -b 6-10)

How to save the commands history of a Jenkins build run?

I wrote a declarative Jenkins pipeline and would like to track the CLI commandos executed by Jenkins. To do this, I added a stage and the step sh 'history -a' in it:
pipeline {
options {
...
}
agent {
node {
...
}
}
stages {
stage('Build') {
steps {
sh 'hostname'
sh 'pwd'
...
}
}
...
stage('History') {
steps {
sh 'history -a'
}
}
}
post {
...
}
}
But that is not working:
Console Output
...
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Tear Down)
[Pipeline] sh
+ history -a
/path/to/project-root#tmp/durable-66ba15cc/script.sh: 1: history: not found
[Pipeline] }
...
Other Linux commands like hostname, ls, or pwd are working fine.
Why does history run into an error? How to store the shell commands called by Jenkins in the context of a pipeline?
That specific error that you are getting I think it is only because the agent where you are running sh, does not have the history cmd available - history: not found
If you can store the sh commands... If you want only the sh commands, I think you need to write to a file that you create at the beginning, where you write to everytime you have a sh step, or you can just use the pipeline log file (the output console).
You can find here a thread about the location of the pipeline or build logs, in case it helps.

Jenkins pipeline to exit if shell script fails in any stage

I have jenkins pipeline jobs which runs shell scripts internally. even though the shell scripts fails job will show as passed only.
My Pipeline:
stage('Code Checkout') {
timestamps {
step([$class: 'WsCleanup'])
echo "check out======GIT =========== on ${env.gitlabBranch}"
checkout scm
}
}
stage("build") {
sh 'sh script.sh'
}
}
catch(err){
currentBuild.result = 'FAILURE'
emailExtraMsg = "Build Failure:"+ err.getMessage()
throw err
}
}
}
LOG:
+ sh script.sh
$RELEASE_BRANCH is empty
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
It looks like your script returns with zero status code. Otherwise it would throw an exception as described in sh step description. The problem may be that exit status of sh sctipt.sh is the exit status of last executed command and your script may do something after error happens (e.g. echo something before exit). The simplest and brutal way to make sure every error is returned is to use put set -e at the top of your script.
You don't need any catch to have this functionality (I mean fail on script error) unless you want to do some extra operations in case of error. But if you do, then you should enclose script execution in try clause:
stage("build") {
try {
sh 'sh script.sh'
}
catch (err) {
currentBuild.result = 'FAILURE'
emailExtraMsg = "Build Failure:"+ err.getMessage()
throw err
}
}

Multiple variables in Jenkins shell

I try to print two variables in Jenkins shell (one of which is global one) . When I print them independently on shell for each it works, however when I try both variables on single line it fails. See the output, seems like a crop after the first variable .
I've tried to print two local variables, and it seems working. However I need the global one
#!/usr/bin/env groovy
def START
node ('master') {
// options{
// timestamps()
// }
stage("one") {
script{
START = sh(script: 'date --utc +%FT%T', returnStdout: true)
}
stage("two") {
def END = sh(script: 'date --utc +%FT%T', returnStdout: true)
sh "echo start $START"
sh "echo end $END"
sh "echo $START and $END"
}
}
}
+ date --utc +%FT%T
[Pipeline] sh
+ echo start 2019-08-01T14:48:08
start 2019-08-01T14:48:08
[Pipeline] sh
+ echo end 2019-08-01T14:48:09
end 2019-08-01T14:48:09
[Pipeline] sh
+ echo 2019-08-01T14:48:08
2019-08-01T14:48:08
+ and 2019-08-01T14:48:09
/var/jenkins_home#tmp/durable-979e1b9e/script.sh: 2: /var/jenkins_home#tmp/durable-979e1b9e/script.sh: and: not found
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE
sh is a dedicated command of jenkins-groovy. The first two work because $START/$END are the final string and doesn't try to replace something else.
sh "echo ${START} and ${END}" writing the variables like this will limit the GString and convert only the correct part and wont try to convert the "and" also.
For more have a look at this examples http://grails.asia/groovy-gstring-examples

Resources