Is there a way to reuse node in different stage without copy/pasting the agent every time?
I have below agent docker in 5 of my stages, it's making file rather big.
pipeline {
agent {
node {
label 'centos'
}
}
stages {
stage('3rd party Scan') {
when {
beforeAgent true
allOf {
triggeredBy 'TimerTrigger'
branch 'master'
}
}
agent {
docker {
reuseNode true
image NODE_DOCKER_IMAGE
args '-u root'
}
}
steps {
script {
sh 'npm ci'
scan.run_scan();
}
}
}
stage('Install') {
when {
beforeAgent true
not { expression { ... } }
}
agent {
docker {
reuseNode true
image NODE_DOCKER_IMAGE
args '-u root'
}
}
steps {
sh 'git clean -fxd'
sh 'npm ci'
}
}
stage('Version') {
agent {
docker {
reuseNode true
image NODE_DOCKER_IMAGE
args '-u root'
}
}
steps {...
}
}
}
}
A stages clause is valid under stage so this might work:
pipeline {
agent {
node {
label 'centos'
}
}
stages {
stage('All Docker Stages') {
agent {
docker {
image NODE_DOCKER_IMAGE
args '-u root'
}
}
stages {
stage('3rd party Scan') {
when {
beforeAgent true
allOf {
triggeredBy 'TimerTrigger'
branch 'master'
}
}
steps {
script {
sh 'npm ci'
scan.run_scan();
}
}
}
stage('Install') {
when {
beforeAgent true
not { expression { ... } }
}
steps {
sh 'git clean -fxd'
sh 'npm ci'
}
}
stage('Version') {
steps {
Related
I want to skip Build stage if AMI already exists using declarative syntax.
stage('Build') {
environment {
AMI = sh(returnStdout: true, script: 'aws ec2 describe-images').trim()
}
when {
expression { AMI = null }
}
steps {
sh 'packer build base.json -machine-readable'
}
}
But when I'm running this pipeline I get groovy.lang.MissingPropertyException: No such property: AMI for class: groovy.lang.Binding
At the same time scripted pipeline works perfectly fine
stage('Build') {
steps {
script {
env.AMI = sh(returnStdout: true, script: 'aws ec2 describe-images').trim()
if (env.AMI == '') {
sh 'packer build base.json -machine-readable'
}
}
}
}
}
I'd really love to switch to the declarative pipelines just stuck with this condition. Any help is really appreciated. Thanks
I tried a lot things without any luck
when {
expression {
return AMI.isEmpty()
}
}
when {
not {
expression {
AMI == ''
}
}
when {
not {
expression { env.AMI }
}
}
Nothing works. I suspect it is somehow related to env variable association through sh
You can do something like this.
pipeline {
agent any
stages {
stage('Build') {
when {
expression {
return isAMIAvailable()
}
}
steps {
sh 'packer build base.json -machine-readable'
}
}
}
}
def isAMIAvailable() {
AMI = sh(returnStdout: true, script: 'aws ec2 describe-images').trim()
return AMI == null
}
With #ycr help I was able to build it!
Just in case here is the whole thing
pipeline {
agent any
environment {
ENV = 'dev'
}
stages {
stage('Build') {
when { not { expression { return isAMIAvailable() } } }
steps {
sh 'packer build base.json -machine-readable'
}
}
}
}
def isAMIAvailable() {
AMI = sh(returnStdout: true, script: "aws ec2 describe-images --owners self --filters 'Name=name,Values=base-${ENV}-1' --query 'Images[*].[Name]' --output text").trim()
if (AMI == '') {
return AMI == null
}
return AMI
}
I want add a condition on all stages:
pipeline {
agent { label 'unix' }
options {
buildDiscarder(logRotator(numToKeepStr: '5', artifactNumToKeepStr: '5'))
}
when {branch 'master' }
stages {
}
}
I find any workaround with add when on each steps but I looking for a solution with only one when
stage('master-branch-stuff') {
when {
branch 'master'
}
steps {
echo 'run this stage - ony if the branch = master branch'
}
}
It can be done this way
declarative pipeline
pipeline {
agent any;
stages {
stage('build') {
when { branch 'master' }
stages {
stage('compile') {
steps {
echo 'compile'
}
}
stage('test') {
steps {
echo 'test'
}
}
stage('package') {
steps {
echo 'package'
}
}
}
}
}
}
scripted pipeline
def branch
node {
stage('checkout') {
def myScm = checkout scm
branch = myScm['branch']
}
if(branch == 'master') {
stage('compile') {
echo "Compile"
}
stage('test') {
echo "Test"
}
stage('package') {
echo "Package"
}
} else {
echo "don't do anything"
}
}
If you want to use pure declarative pipeline, no.
Per their doc, https://www.jenkins.io/doc/book/pipeline/syntax/#when, when is only allowed "inside a stage directive."
stage {
when { ... }
...
}
However, you can add stages within a script block. So you can do something like
stage {
when { ... }
steps {
script {
stage("scripted stage 1") { ... }
stage("scripted stage 2") { ... }
}
}
}
Keep in mind that everything inside of script (including the stages) will follow scripted pipeline rules
I have a script similar to this:
pipeline {
agent {
docker {
label 'dev'
image 'node:12-alpine'
args '-p 3000:3000'
}
}
environment {
HOME = '.'
}
stages {
stage('clone repo') {
steps {
git(
url: '...',
credentialsId: '...',
branch: 'master'
)
}
}
stage('install dependency packages') {
steps {
sh 'npm install'
}
}
stage('build prod ready enviroment') {
steps {
sh 'npm run build'
}
}
stage('deploy') {
agent { node { label 'dev' } }
steps {
sh "cp -rf ./build/* /opt/www_folder/"
}
}
}
}
Now everything works fine except deploy stage which just hangs up building process. If I run only last stage (deploy) separately without other stages it works fine. I think there is a conflict with a docker agent but I don't know how to fix it.
I'm not sure if it is the best answer but I managed to fix my problem with this script:
pipeline {
agent none
environment {
HOME = '.'
}
stages {
stage('clone repo') {
agent { node { label 'dev' } }
steps {
git(
url: '...',
credentialsId: '...',
branch: 'master'
)
}
}
stage('install and build') {
agent {
docker {
label 'dev'
image 'node:12-alpine'
args '-p 3000:3000'
}
}
steps {
sh 'npm install'
sh 'npm run build'
}
}
stage('deploy') {
agent { node { label 'dev' } }
steps {
sh "rm -rf /opt/www_folder/*"
sh "cp -rf ./build/* /opt/www_folder/"
}
}
}
}
I have something like this:
def projects = [
"foo",
"bar",
"foobar",
"etc",
]
pipeline {
agent any
stages {
stage('lint') {
parallel {
stage('lint: foo') {
steps {
sh "lint foo"
}
}
stage('lint: bar') {
steps {
sh "lint bar"
}
}
stage('lint: foobar') {
steps {
sh "lint foobar"
}
}
}
}
}
}
I didn't really want to repeat myself there, is there a way I can "generate" the stage code for each project?
I was able to do something similar, without the parallel part with this:
pipeline {
agent any
stages {
stage('initialize') {
steps {
script {
for (int i = 0; i < projects.size(); i++) {
stage("lint: ${projects[i]}") {
sh "lint ${project}"
}
}
}
}
}
}
}
Any help appreciated.
For scripted pipeline you may use something like:
def projects = [
"foo",
"bar",
"foobar",
"etc",
]
Map branches = [:]
projects.each { String project ->
branches[project] = {
stage(project) {
node {
// I assume we need to checkout something, e.g. using git or checkout step...
// git ...
sh "lint ${project}"
}
}
}
}
parallel branches
I have a Jenkinsfile for a declarative pipeline that uses docker agents. A number of the steps use a docker agent and it is a bit repetitive adding the same agent for these steps
e.g.
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
//Do something
}
}
stage('Stage 2') {
agent {
docker { image 'jenkins/jenkins-slave:latest'
reuseNode true
registryUrl 'https://some.registry/'
registryCredentialsId 'git'
args '-v /etc/passwd:/etc/passwd -v /etc/group:/etc/group -e HOME=${WORKSPACE}'
}
}
steps {
//Do something
}
}
stage('Stage 3') {
steps {
//Do something
}
}
stage('Stage 4') {
agent {
docker { image 'jenkins/jenkins-slave:latest'
reuseNode true
registryUrl 'https://some.registry/'
registryCredentialsId 'git'
args '-v /etc/passwd:/etc/passwd -v /etc/group:/etc/group -e HOME=${WORKSPACE}'
}
}
steps {
//Do something
}
}
}
}
Is there any way to template the agent (or write my own) so that I could do something like the following
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
//Do something
}
}
stage('Stage 2') {
agent {
my-docker
}
steps {
//Do something
}
}
stage('Stage 3') {
steps {
//Do something
}
}
stage('Stage 4') {
agent {
my-docker
}
steps {
//Do something
}
}
}
}
So that I do not have to repeatly write the same agent and possibly I can reuse it in all my Dockerfiles