Jenkins Pipeline, use an Env Var within emailext plugin - jenkins-pipeline

My Pipeline is generating a dynamic recipient list based on each Job execution.I'm trying to use that list which I set it as a Variable, to use in the 'To' section of the emailext plugin, the Problem is that the Content of the variable is not resolved once using the mailext part.
pipeline {
agent {
label 'master'
}
options {
timeout(time: 20, unit: 'HOURS')
}
stages {
stage('Find old Projects') {
steps {
sh '''
find $JENKINS_HOME/jobs/* -type f -name "nextBuildNumber" -mtime +1550|egrep -v "configurations|workspace|modules|promotions|BITBUCKET"|awk -F/ '{print $6}'|sort -u >results.txt
'''
}
}
stage('Generate recipient List') {
steps {
sh '''
for Project in `cat results.txt`
do
grep "mail.com" $JENKINS_HOME/jobs/$Project/config.xml|grep -iv "Ansprechpartner" | awk -F'>' '{print $2}'|awk -F'<' '{print $1}'>> recipientList.txt
done
recipientList=`sort -u recipientList.txt`
echo $recipientList
'''
}
}
stage('Generate list to Shelve or Delete') {
steps {
sh '''
for Project in `cat results.txt`
do
if [ -f "$JENKINS_HOME/jobs/$Project/nextBuildNumber" ]; then
nextBuildNumber=`cat $JENKINS_HOME/jobs/$Project/nextBuildNumber`
if [ $nextBuildNumber == '1' ]; then
echo "$JENKINS_HOME/jobs/$Project" >> jobs2Delete.txt
echo "$Project" >> jobList2Delete.txt
else
echo "$JENKINS_URL/job/$Project/shelve/shelveProject" >> Projects2Shelve.txt
echo "$Project" >> ProjectsList2Shelve.txt
fi
fi
done
'''
}
}
stage('Send email') {
steps {
emailext to: 'admin#mail.com',
from: 'jenkins#mail.com',
attachmentsPattern: 'ProjectsList2Shelve.txt,jobList2Delete.txt',
subject: "This is a subject",
body: "Hello\n\nAttached two lists of Jobs, to archive or delete,\nPlease Aprove or Abort the Shelving / Delition of the Projects:\n${env.JOB_URL}\n\nBlue Ocean:\n${env.RUN_DISPLAY_URL}\n\nyour Team"
}
}
stage('Aprove or Abort') {
steps {
input message: 'OK to Shelve and Delete projects? \n Review the jobs list (Projects2Shelve.txt, jobs2Delete.txt) sent to your email', submitter: 'someone'
}
}
stage('Shelve or Delete') {
parallel {
stage('Shelve Project') {
steps {
withCredentials([usernamePassword(credentialsId: 'XYZ', passwordVariable: 'PA', usernameVariable: 'US')]) {
sh '''
for job2Shelve in `cat Projects2Shelve.txt`
do
curl -u $US:$PA $job2Shelve
done
'''
}
}
}
stage('Delete Project') {
steps {
sh '''
for job2Del in `cat jobs2Delete.txt`
do
echo "Removing $job2Del"
done
'''
}
}
}
}
}
post {
success {
emailext to: "$recipientListTest",
from: 'jenkins#mail.com',
attachmentsPattern: 'Projects2Shelve.txt,jobs2Delete.txt',
subject: "This is a sbject",
body: "Hallo\n\nAttached two lists of Jobs which archived or deleted due to inactivity of more the 400 days\n\n\nyour Team"
}
}
}

I figured out that the only way would be to add a script part as part of the post section, together with a variable Definition outside of the Pipeline block:
post {
success {
script {
RECIPIENTLIST = sh(returnStdout: true, script: 'cat recipientListTest.txt')
}
emailext to: "${RECIPIENTLIST}",
from: 'jenkins#mail.com',
attachmentsPattern: 'Projects2Shelve.txt,jobs2Delete.txt',
subject: "MY SUBJECT",
body: "MY BODY"
}

when you execute a sh command, you cannot reuse the variables that you set within that command. You need to do something like this:
on top you your pipeline file to make this variable global
def recipientsList
then execute your shell command and retrieve the output
recipientsList = sh (
script: '''for Project in `cat results.txt`
do
grep "mail.com" $JENKINS_HOME/jobs/$Project/config.xml|grep -iv "Ansprechpartner" | awk -F'>' '{print $2}'|awk -F'<' '{print $1}'>> recipientList.txt
done
recipientList2=`sort -u recipientList.txt`
echo $recipientList2
''',
returnStdout: true
).trim()
Now in your email you can use the variable $recipientList...
I renamed your bash variable to recipientList2 to avoid confusion.
EDIT: I don't know what you want to obtain, but consider using some default recipients provided by emailext:
recipientProviders: [ developers(), culprits(), requestor(), brokenBuildSuspects(), brokenTestsSuspects() ],

Related

Run a set of linux commands using Jenkinsfile in Jenkins

I had to create a jenkins job to automate certain tasks that will perform certain operations like Updating the public site, Changing public version to latest public release, Updating Software on public site and Restarting Server these include certain operations such as copy files to a tmp folder, log in to a an on-prem server, go to the folder and unzip the file etc.
I have created the jenkinsfile as follows:
pipeline {
options {
skipDefaultCheckout()
timestamps()
}
parameters {
string(name: 'filename', defaultValue: 'abc', description: 'Enter the file name that needs to be copied')
string(database: 'database', defaultValue: 'abc', description: 'Enter the database that needs to be created')
choice(name: 'Run', choices: '', description: 'Data migration')
}
agent {
node { label 'aws && build && linux && ubuntu' }
}
triggers {
pollSCM('H/5 * * * *')
}
stages {
stage('Clean & Clone') {
steps {
cleanWs()
checkout scm
}
}
stage('Updating the public site'){
steps{
sh "scp ./${filename}.zip <user>#<server name>:/tmp"
sh "ssh <user>#<server name>"
sh "cp ./tmp/${filename}.zip ./projects/xyz/xyz-site/"
sh "cd ./projects/xyz/xyz-site/ "
sh "unzip ./${filename}.zip"
sh "cp -R ./${filename}/* ./"
}
stage('Changing public version to latest public release') {
steps {
sh "scp ./${filename}.sql.gz <user>#<server name>:/tmp"
sh "ssh <user>#<server name>"
sh "mysql -u root -p<PASSWORD>"
sh "show databases;"
sh "create database ${params.database};"
sh "GRANT ALL PRIVILEGES ON <newdb>.* TO 'ixxyz'#'localhost' WITH GRANT OPTION;"
sh "exit;"
sh "zcat tmp/${filename}.sql.gz | mysql -u root -p<PASSWORD> <newdb>"
sh "db.default.url="jdbc:mysql://localhost:3306/<newdb>""
sh "ps aux|grep monitor.sh|awk '{print "kill "$2}' |bash"
}
}
stage('Updating Software on public site') {
steps {
sh "scp <user>#<server>:/tmp/abc<version>_empty_h2.zip"
sh "ssh <user>#<server name>"
sh "su <user>"
sh "mv tmp/<version>_empty_h2.zip ./xyz/projects/xyz"
sh "cd xyz/projects/xyz"
sh "cp latest/conf/local.conf <version>_empty_h2/conf/"
}
}
stage('Restarting Server') {
steps {
sh "rm latest/RUNNING_PID"
sh "bash reload.sh"
sh "nohup bash monitor.sh &"
}
}
}
}
Is there a way I can dynamically obtain the zip filename in the root folder? I used ${filename}.zip , but it doesn't seem to work.
Also, is there a better way to perform these operations using jenkins? Any help is much appreciated.
You could write all your steps in one shell script for each stage and execute under one stage.
Regarding filename.zipeither you can take this as a parameter and pass this value to your stages. OR You can also use find command as a shell command or shell script to find .zip files in a current directory. find <dir> -iname \*.zip find . -iname \*.zip .
Example:
pipeline {
options {
skipDefaultCheckout()
timestamps()
}
parameters {
string(name: 'filename', defaultValue: 'abc', description: 'Enter the file name that needs to be copied')
choice(name: 'Run', choices: '', description: 'Data migration')
}
stage('Updating the public site'){
steps{
sh "scp ./${params.filename}.zip <user>#<server name>:/tmp"
...
}
}
}
For executing script at a certain location based on your question , you could use dir with path where your scripts are placed.
OR you can also give the path directly sh label: 'execute script', script:"C:\\Data\\build.sh"
stage('Your stage name'){
steps{
script {
// Give path where your scripts are placed
dir ("C:\\Data") {
sh label: 'execute script', script:"build.sh <Your Arguments> "
...
}
}
}
}

Jenkins declarative pipeline get environment variable at post stage

I am getting runtime value in build stage stage which I stored in an environment variable . I saved that to env.cfg file under WORKSPACE .
Now I am trying to get that value in post pipeline step to be used in email communication. I tried load method but it did not work
Any help ?
post {
always {
echo $SNAPSHOT / /this always comes null
}
}
This is the way you can access an environment variable across the pipeline
pipeline {
agent any;
environment {
MESSAGE="Hello World"
}
stages {
stage('one') {
steps {
echo "${env.MESSAGE}"
sh "echo $MESSAGE"
script {
print env.MESSAGE
}
}
}
}
post {
success {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
failure {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
}
}
but as per your scenario let say I have a file called .env with the content below in the current Jenkins job WORKSPACE and I want to read and make this env variable in the pipeline.
.env
export SNAPSHOT=1.0.0
export MESSAGE='Hello World'
export MESSAGE_FROM_ENV_FILE='Hello From .env file'
your pipeline should look like
scripted pipeline
node {
stage('one') {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
declarative pipeline
pipeline {
agent any;
stages {
stage('build') {
steps {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
post {
success {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
You need a global variable:
SNAPSHOT = ""
println "SNAPSHOT is ${SNAPSHOT}"
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
println "SNAPSHOT is ${SNAPSHOT}"
SNAPSHOT = "Modified"
println "SNAPSHOT is now ${SNAPSHOT}"
}
}
}
}
post {
always {
echo "${SNAPSHOT}"
}
}
}

For loop in Jenkins Pipeline

I stuck with for loop condition in Pipeline
pipeline {
agent any
stages{
stage ('Showing Working Space') {
when {
anyOf {
environment name: 'Test', value: 'ALL'
environment name: 'Test', value: 'IMAGE'
}
}
steps {
sh "echo Display ${Var1}"
script{
sh 'for service in (echo "$Var1"|sed "s/,/ /g");do echo $service; done'
}
}
}
}
}
Getting error like " syntax error near unexpected token `('"
Var1 = has multiple values
Need to execute the "For loop" to pass the values to another script
Please help on this
I believe what you want is
pipeline {
agent any
stages {
stage('Showing Working Space') {
when {
anyOf {
environment name: 'Test', value: 'ALL'
environment name: 'Test', value: 'IMAGE'
}
}
steps {
sh "echo Display ${Var1}"
script {
sh 'for service in $(echo "$Var1"|sed "s/,/ /g"); do echo $service; done'
}
}
}
}
}
In essence, replace service in (echo with service in $(echo (note the $).

variables not getting populated in sh of jenkins pipeline

I have the below code and username,pwd and modulename from previous stages are not getting populated when I'm doing curl in sh script.
Please let me know what do I need to fix it
def USERNAME
def PASSWORD
def MODULE_NAME
node {
try {
stage('userAuth') {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'ID-Creds',
usernameVariable: 'user', passwordVariable: 'pwd']]) {
USERNAME="$user"
PASSWORD="$pwd"
echo "$USERNAME:$PASSWORD" //This is fine
}
}
stage('readPOM') {
def pom = readMavenPom file: 'pom.xml'
MODULE_NAME = pom.module
echo "$MODULE_NAME" //This is printing fine
}
stage('do curl') {
def revision = sh(script: '''
AUTH="$USERNAME:$PASSWORD"; //Not getting populated getting empty
RespInfo=$(curl -u $AUTH "https://host/apis/${MODULE_NAME}/deployments"); //Not getting populated getting empty for modulename
currntRev=$(jq -r .revision[0].name <<< "${RespInfo}");
echo $currntRev
''',returnStdout: true).split()
}
}
catch (e) {
throw e
} finally {
}
}
You have to use double quotes (""") to apply string interpolation:
def revision = sh(script: """
AUTH=\"$USERNAME:$PASSWORD\"; //Not getting populated getting empty
A easier way is to concat two strings as following:
def revision = sh(
returnStdout: true,
script: '''
AUTH="$user:$pwd";
RespInfo=$(curl -u "$AUTH" "https://host/apis/''' + MODULE_NAME + '''/deployments");
currntRev=$(jq -r .revision[0].name <<< "${RespInfo}");
echo $currntRev
'''
).split()
Note: string wrapped in """ will be expanded, but not when wrapped in '''
USERNAME and PASSWORD are Groovy variable, when they are wrapped in """ or "", Groovy executor will expand them before script be executed.
user and pwd are Shell variable, we should use Shell variable when use '''

How to re-use pre process jenkins/groovy in each test

Im using the following code to run our voter , currently I’ve one target which is called Run Tests
Which use exactly the same steps as the last (lint) , currently I duplicate it which I think is not a good solution ,
Is there is nice way to avoid this duplication and done it only once as per-requisite process ?
I need all the steps until the cd to the project
The only difference is one target I run
go test ...
and the second
go lint
All steps before are equal
#!/usr/bin/env groovy
try {
parallel(
'Run Tests': {
node {
//————————Here we start
checkout scm
def dockerImage = 'docker.company:50001/crt/deg:0.1.3-09’
setupPipelineEnvironment script: this,
measureDuration(script: this, measurementName: 'build') {
executeDocker(dockerImage: dockerImage, dockerWorkspace: '/go/src') {
sh """
mkdir -p /go/src/github.com/ftr/myGoProj
cp -R $WORKSPACE/* /go/src/github.com/ftr/MyGoProj
cd /go/src/github.com/ftr/MyGoProj
//————————Here we finish and TEST
go test -v ./...
"""
}
}
}
},
‘Lint’: {
node {
//————————Here we start
checkout scm
def dockerImage = 'docker.company:50001/crt/deg:0.1.3-09’
setupPipelineEnvironment script: this,
measureDuration(script: this, measurementName: 'build') {
executeDocker(dockerImage: dockerImage, dockerWorkspace: '/go/src') {
sh """
mkdir -p /go/src/github.com/ftr/myGoProj
cp -R $WORKSPACE/* /go/src/github.com/ftr/MyGoProj
cd /go/src/github.com/ftr/MyGoProj
//————————Here we finish and LINT
go lint
"""
}
}
)
}
}
}
You can use function and pass Go arguments:
try {
parallel(
'Run Tests': {
node {
checkout scm
runTestsInDocker('test -v ./...')
}
},
'Lint': {
node {
checkout scm
runTestsInDocker('lint')
}
}
)
}
def runTestsInDocker(goArgs) {
def dockerImage = 'docker.company:50001/crt/deg:0.1.3-09'
setupPipelineEnvironment script: this,
measureDuration(script: this, measurementName: 'build') {
executeDocker(dockerImage: dockerImage, dockerWorkspace: '/go/src') {
sh """
mkdir -p /go/src/github.com/ftr/myGoProj
cp -R $WORKSPACE/* /go/src/github.com/ftr/MyGoProj
cd /go/src/github.com/ftr/MyGoProj
go ${goArgs}
"""
}
}
}
Update
If some actions can be separated out of runTestsInDocker they probably should be.
For example setupPipelineEnvironment step. I don't know exact logic but maybe it can be run once before running test.
node {
stage('setup') {
setupPipelineEnvironment script: this
}
stage ('Tests') {
parallel(
'Run Tests': {
node {
checkout scm
runTestsInDocker('test -v ./...')
}
},
'Lint': {
node {
checkout scm
runTestsInDocker('lint')
}
}
)
}
}
def runTestsInDocker(goArgs) {
def dockerImage = 'docker.company:50001/crt/deg:0.1.3-09'
measureDuration(script: this, measurementName: 'build') {
executeDocker(dockerImage: dockerImage, dockerWorkspace: '/go/src') {
sh """
mkdir -p /go/src/github.com/ftr/myGoProj
cp -R $WORKSPACE/* /go/src/github.com/ftr/MyGoProj
cd /go/src/github.com/ftr/MyGoProj
go ${goArgs}
"""
}
}
}
Note
If you are running parallel on remote agents you must remember that setup performed on master may be not aviailable on remote slave.

Resources