variables not getting populated in sh of jenkins pipeline - shell

I have the below code and username,pwd and modulename from previous stages are not getting populated when I'm doing curl in sh script.
Please let me know what do I need to fix it
def USERNAME
def PASSWORD
def MODULE_NAME
node {
try {
stage('userAuth') {
withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: 'ID-Creds',
usernameVariable: 'user', passwordVariable: 'pwd']]) {
USERNAME="$user"
PASSWORD="$pwd"
echo "$USERNAME:$PASSWORD" //This is fine
}
}
stage('readPOM') {
def pom = readMavenPom file: 'pom.xml'
MODULE_NAME = pom.module
echo "$MODULE_NAME" //This is printing fine
}
stage('do curl') {
def revision = sh(script: '''
AUTH="$USERNAME:$PASSWORD"; //Not getting populated getting empty
RespInfo=$(curl -u $AUTH "https://host/apis/${MODULE_NAME}/deployments"); //Not getting populated getting empty for modulename
currntRev=$(jq -r .revision[0].name <<< "${RespInfo}");
echo $currntRev
''',returnStdout: true).split()
}
}
catch (e) {
throw e
} finally {
}
}

You have to use double quotes (""") to apply string interpolation:
def revision = sh(script: """
AUTH=\"$USERNAME:$PASSWORD\"; //Not getting populated getting empty

A easier way is to concat two strings as following:
def revision = sh(
returnStdout: true,
script: '''
AUTH="$user:$pwd";
RespInfo=$(curl -u "$AUTH" "https://host/apis/''' + MODULE_NAME + '''/deployments");
currntRev=$(jq -r .revision[0].name <<< "${RespInfo}");
echo $currntRev
'''
).split()
Note: string wrapped in """ will be expanded, but not when wrapped in '''
USERNAME and PASSWORD are Groovy variable, when they are wrapped in """ or "", Groovy executor will expand them before script be executed.
user and pwd are Shell variable, we should use Shell variable when use '''

Related

Jenkins read json file with multiple list value jsonsurper or readjson

I want to be able ro read json format based on parameter value selected in choice. e.g If dev is selected, it should select (dev1,dev2,dev3) and loop through each selected in json through the node. what is important now is to get the value in json to a file and then I call call it from file into the node
error:
groovy.lang.MissingMethodException: No signature of method: net.sf.json.JSONObject.$() is applicable for argument types: (org.jenkinsci.plugins.workflow.cps.CpsClosure2) values: [org.jenkinsci.plugins.workflow.cps.CpsClosure2#7e1eb88f]
Possible solutions: is(java.lang.Object), any(), get(java.lang.Object), get(java.lang.String), has(java.lang.String), opt(java.lang.String)
script in pipeline
#!/usr/bin/env groovy
node{
properties([
parameters([
choice(
name: 'environment',
choices: ['','Dev', 'Stage', 'devdb','PreProd','Prod' ],
description: 'environment to choose'
),
])
])
node () {
def myJson = '''{
"Dev": [
"Dev1",
"Dev2",
"Dev3"
],
"Stage": [
"Stage1",
"Stage2"
],
"PreProd": [
"Preprod1"
],
"Prod": [
"Prod1",
"Prod2"
]
}''';
def myObject = readJSON text: myJson;
echo myObject.${params.environment};
// put the list of the node in a file or in a list to loop
}
Using Pipeline Utility Steps, it can be easier:
Reading from string:
def obj = readJSON text: myjson
Or reading from file:
def obj = readJSON file: 'myjsonfile.json'
Now you can get the element and iterate the list:
def list = obj[params.environment]
list.each { elem ->
echo "Item: ${elem}"
}
Reference: https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace
Let me make it simple.
To read the json file you need to download it from git or wherever you stored it. Let's assume git in this case. Once the json file is downloaded then you want to access the content of json file in your code. Which can be done by this code.
import groovy.json.JsonSlurperClassic
def downloadConfigFile(gitProjectURL, jsonnFileBranch) {
// Variables
def defaultBranch = jsonnFileBranch
def gitlabAdminCredentials = 'admin'
def poll = false
def jenkinsFilePath = 'jenkins.json'
// Git checkout
git branch: defaultBranch, credentialsId: gitlabAdminCredentials, poll: poll, url: gitProjectURL
// Check if file existed or not
def jenkinsFile = fileExists(jenkinsFilePath)
if (jenkinsFile) {
def jsonStream = readFile(jenkinsFilePath)
JsonSlurperClassic slurper = new JsonSlurperClassic()
def parsedJson = slurper.parseText(jsonStream)
return parsedJson
} else {
return [:]
}
}
Now we have the entire json file parsed using above function.
Now you can call the function and read the value in a global variable.
stage('Download Config') {
jsonConfigData = downloadConfigFile(gitProjectURL, jsonnFileBranch)
if (jsonConfigData.isEmpty()) {
error err_jenkins_file_does_not_exists
} else {
println("jsonConfigData : ${jsonConfigData}")
}
Now you can access the value of json file or say variable like this.
def projectName = jsonConfigData.containsKey('project_name') == true ? jsonConfigData['project_name'] : ''
You can access any thing if its child node in similar way. I hope it helps you.

Pass a value to a shell script from jenkins pipeline

How to pass values to a shell script from jenkins during the runtime of the pipeline job.
I have a shell script and want to pass the values dynamically.
#!/usr/bin/env bash
....
/some code
....
export USER="" // <--- want to pass this value from pipeline
export password="" //<---possibly as a secret
The jenkins pipeline executes the above shell script
node('abc'){
stage('build'){
sh "cd .."
sh "./script.sh"
}
}
You can do something like the following:
pipeline {
agent any
environment {
USER_PASS_CREDS = credentials('user-pass')
}
stages {
stage('build') {
steps {
sh "cd .."
sh('./script.sh ${USER_PASS_CREDS_USR} ${USER_PASS_CREDS_PSW}')
}
}
}
}
The credentials is from using the Credentials API and Credentials plugin. Your other option is Credentials Binding plugin where it allows you to include credentials as part of a build step:
stage('build with creds') {
steps {
withCredentials([usernamePassword(credentialsId: 'user-pass', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
// available as an env variable, but will be masked if you try to print it out any which way
// note: single quotes prevent Groovy interpolation; expansion is by Bourne Shell, which is what you want
sh 'echo $PASSWORD'
// also available as a Groovy variable
echo USERNAME
// or inside double quotes for string interpolation
echo "username is $USERNAME"
sh('./script.sh $USERNAME $PASSWORD')
}
}
}
Hopefully this helps.

Jenkins Pipeline, use an Env Var within emailext plugin

My Pipeline is generating a dynamic recipient list based on each Job execution.I'm trying to use that list which I set it as a Variable, to use in the 'To' section of the emailext plugin, the Problem is that the Content of the variable is not resolved once using the mailext part.
pipeline {
agent {
label 'master'
}
options {
timeout(time: 20, unit: 'HOURS')
}
stages {
stage('Find old Projects') {
steps {
sh '''
find $JENKINS_HOME/jobs/* -type f -name "nextBuildNumber" -mtime +1550|egrep -v "configurations|workspace|modules|promotions|BITBUCKET"|awk -F/ '{print $6}'|sort -u >results.txt
'''
}
}
stage('Generate recipient List') {
steps {
sh '''
for Project in `cat results.txt`
do
grep "mail.com" $JENKINS_HOME/jobs/$Project/config.xml|grep -iv "Ansprechpartner" | awk -F'>' '{print $2}'|awk -F'<' '{print $1}'>> recipientList.txt
done
recipientList=`sort -u recipientList.txt`
echo $recipientList
'''
}
}
stage('Generate list to Shelve or Delete') {
steps {
sh '''
for Project in `cat results.txt`
do
if [ -f "$JENKINS_HOME/jobs/$Project/nextBuildNumber" ]; then
nextBuildNumber=`cat $JENKINS_HOME/jobs/$Project/nextBuildNumber`
if [ $nextBuildNumber == '1' ]; then
echo "$JENKINS_HOME/jobs/$Project" >> jobs2Delete.txt
echo "$Project" >> jobList2Delete.txt
else
echo "$JENKINS_URL/job/$Project/shelve/shelveProject" >> Projects2Shelve.txt
echo "$Project" >> ProjectsList2Shelve.txt
fi
fi
done
'''
}
}
stage('Send email') {
steps {
emailext to: 'admin#mail.com',
from: 'jenkins#mail.com',
attachmentsPattern: 'ProjectsList2Shelve.txt,jobList2Delete.txt',
subject: "This is a subject",
body: "Hello\n\nAttached two lists of Jobs, to archive or delete,\nPlease Aprove or Abort the Shelving / Delition of the Projects:\n${env.JOB_URL}\n\nBlue Ocean:\n${env.RUN_DISPLAY_URL}\n\nyour Team"
}
}
stage('Aprove or Abort') {
steps {
input message: 'OK to Shelve and Delete projects? \n Review the jobs list (Projects2Shelve.txt, jobs2Delete.txt) sent to your email', submitter: 'someone'
}
}
stage('Shelve or Delete') {
parallel {
stage('Shelve Project') {
steps {
withCredentials([usernamePassword(credentialsId: 'XYZ', passwordVariable: 'PA', usernameVariable: 'US')]) {
sh '''
for job2Shelve in `cat Projects2Shelve.txt`
do
curl -u $US:$PA $job2Shelve
done
'''
}
}
}
stage('Delete Project') {
steps {
sh '''
for job2Del in `cat jobs2Delete.txt`
do
echo "Removing $job2Del"
done
'''
}
}
}
}
}
post {
success {
emailext to: "$recipientListTest",
from: 'jenkins#mail.com',
attachmentsPattern: 'Projects2Shelve.txt,jobs2Delete.txt',
subject: "This is a sbject",
body: "Hallo\n\nAttached two lists of Jobs which archived or deleted due to inactivity of more the 400 days\n\n\nyour Team"
}
}
}
I figured out that the only way would be to add a script part as part of the post section, together with a variable Definition outside of the Pipeline block:
post {
success {
script {
RECIPIENTLIST = sh(returnStdout: true, script: 'cat recipientListTest.txt')
}
emailext to: "${RECIPIENTLIST}",
from: 'jenkins#mail.com',
attachmentsPattern: 'Projects2Shelve.txt,jobs2Delete.txt',
subject: "MY SUBJECT",
body: "MY BODY"
}
when you execute a sh command, you cannot reuse the variables that you set within that command. You need to do something like this:
on top you your pipeline file to make this variable global
def recipientsList
then execute your shell command and retrieve the output
recipientsList = sh (
script: '''for Project in `cat results.txt`
do
grep "mail.com" $JENKINS_HOME/jobs/$Project/config.xml|grep -iv "Ansprechpartner" | awk -F'>' '{print $2}'|awk -F'<' '{print $1}'>> recipientList.txt
done
recipientList2=`sort -u recipientList.txt`
echo $recipientList2
''',
returnStdout: true
).trim()
Now in your email you can use the variable $recipientList...
I renamed your bash variable to recipientList2 to avoid confusion.
EDIT: I don't know what you want to obtain, but consider using some default recipients provided by emailext:
recipientProviders: [ developers(), culprits(), requestor(), brokenBuildSuspects(), brokenTestsSuspects() ],

Jenkins custom pipeline and how to add property to be set in jenkinsfile

I'm trying to create a custom pipeline with groovy but I can't find anywhere on the web where it is discussed how to add a property that can be set in the jenkinsfile. I'm trying to add a curl command but need the URL to be set in the jenkinsfile because it will be different for each build.
Can anyone explain how that should be done or links where it has been discussed?
Example Jenkinsfile:
msBuildPipelinePlugin
{
curl_url = "http://webhook.url.com"
}
custom pipeline groovy code:
def response = sh(script: 'curl -i -X POST -H 'Content-Type: application/json' -d '{"text","Jenkins Info.\nThis is more text"}' curl_url, returnStdout: true)
Thanks
If you want to specify the URL as a string during every build, you can do either of the following:
Declarative Pipeline
Use the parameters {} directive:
pipeline {
agent {
label 'rhel-7'
}
parameters {
string(
name: 'CURL_URL',
defaultValue: 'http://www.google.com',
description: 'Enter the URL for file download'
)
}
stages {
stage('download-file') {
steps {
echo "The URL is ${params.CURL_URL}"
}
}
}
}
Scripted Pipeline
Use the properties([parameters([...])]) step:
parameters([
string(
name: 'CURL_URL',
defaultValue: 'http://www.google.com',
description: 'Enter the URL for file download'
)
])
node('rhel-7') {
stage('download-file') {
echo "The URL is ${params.CURL_URL}"
}
}
You can choose to leave the values of defaultValue and description empty.
Job GUI
Either of the above syntax will be rendered in the GUI as:
I got it to work using
//response is just the output of the curl statement
def response = ["curl", "-i", "-v", "-X", "POST", "--data-urlencode", "payload={\"text\":\"message body\"}", "curl url goes here"].execute().text
Thanks

Can't read a file parm in jenkins pipeline

example :
pipeline{
agent any
stages{
stage('Parse CSV'){
steps {
script{
def fileToParse = readFile(params.FileLocation)
}
echo fileToParse
}
}
}
}
I configured the job from the GUI, the file location parameter is called FileLocation. I uploaded a file and tried to read it. When I try to access params.FileLocation it returns null, as if it doesn't recognise it.
Your problem is with the variable scope. You def the variable in the script {} block scope, then try to use it outside of it. One easy fix is to def the variable outside the pipeline {} block at the global level. Or, just use the params.FileLocation in your echo statement.
def fileToParse
pipeline{
agent any
stages{
stage('Parse CSV'){
steps {
script{
fileToParse = readFile(params.FileLocation)
}
echo fileToParse
echo params.FileLocation
}
}
}
}
file param is not supported and it is removed from documentation as well.
https://issues.jenkins-ci.org/browse/JENKINS-27413
Check available parameters: https://jenkins.io/doc/book/pipeline/syntax/#parameters

Resources