I have a pipeline for building my android application. In some stages, I have shell scripts. when I run the stages one by one(by commenting others) everything works fine, but when I run them all together I see strange behavior.
It seems that the shell scripts are running in parallel !!
here is my jenkinsfile :
pipeline{
agent any
stages{
stage("Clean"){
agent{
node{
label 'master'
customWorkspace getMainDirectory()
}
}
steps{
sh """#!/bin/bash
rm -rf Corona
rm -rf native-corona-android
cd ..
cp -a TemplateWorkspace/. ${getCoronaBranch()}-${getNativeBrach()}
"""
}
}
stage("pull native repo"){
agent{
node{
label 'master'
customWorkspace getNativeProjectPath()
}
}
steps{
echo "pulling native"
git(
url: nativeRepositoryAddress,
credentialsId: credentialsId,
branch: getNativeBrach()
)
echo "pulling done"
}
}
stage("pull corona repo"){
agent{
node{
label 'master'
customWorkspace getCoronaProjectPath()
}
}
steps{
echo "pulling corona"
git(
url: coronaRepositoryAddress,
credentialsId: credentialsId,
branch: getCoronaBranch()
)
echo "pulling done"
}
}
stage("build"){
environment {
docDir = getMainDirectory()
ANDROID_HOME = getAndroidSDKLocation()
}
agent{
node{
label 'master'
customWorkspace getNativeProjectPath()
}
}
steps{
sh """#!/bin/bash
./gradlew clean
./gradlew changeFiles --stacktrace --no-daemon
./gradlew assembleDebug --stacktrace --no-daemon
"""
}
}
stage("move build files"){
agent{
node{
label 'master'
customWorkspace getGradleBuildLocation()
}
}
steps{
sh """#!/bin/bash
yes | cp -rf * ../../../../JenkinsBuilds/${getOutputFolder()}/
"""
}
}
}
}
I just want to run steps synchronous(and of course shell scripts), What is my problem?
Here is what I see:
In "Clean" step the folders get deleted and a fresh copy of Template folders get copied to work directory. Steps "pull native repo" and "pull corona repo" do the jobs they should do. but in step "build" I can see that a part of "native-corona-android" files is gone and "gradlew" script is deleted. I have also seen situations where the whole "native-corona-android" folder gets deleted. Then I thought that the script in "Clean" step is called again.
Thanks
Unless you are using parallel directive all steps should be running synchronously.
What behavior do you observe? Which steps are running parallel?
Related
Following is my Machine details
Specification
Version
OS
Windows-11
Java
11
Gradle
7.3
I had setup of shell on Jenkins as well.
And Jenkins code snippet is as follows:
#!/usr/bin/env groovy
/* groovylint-disable CompileStatic, DuplicateStringLiteral, UnnecessaryGString, VariableName */
import java.text.SimpleDateFormat
pipeline {
agent any
stages {
stage('build-and-package') {
steps {
script {
sh "git update-index --chmod=+x gradlew"
sh "chmod +x gradlew"
sh "./gradlew clean assemble --stacktrace"
//sh "gradle clean build"
sh "ls -lrt"
sh "ls -lrt ./build/libs/"
}
}
}
stage('docker-build-and-push') {
steps {
//Code Commented
}
}
stage('helm-deploy') {
steps {
script {
//Code Commented
}
}
}
}
}
I have tried to provide full access rights to Jenkins workspace
Error Screen shot
The Jenkins is installed at Ubuntu. The Jenkins workspace is at /var/lib/jenkins/workspace. The github repo is at https://github.com/garyyang6/scripts. There are three scripts in the repo, build.sh, deploy.sh and quality.sh. The pipeline groovy script is as follow. After I ran the groovy script in Jenkins, it complains build.sh: not found.
At /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub, I found the scripts, build.sh deploy.sh quality.sh. The folder /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub#tmp is empty.
pipeline {
agent any
stages {
stage('Git-Checkout') {
steps {
echo 'Checking out from Git Repo';
git 'https://github.com/garyyang6/scripts.git'
}
}
stage('Build') {
steps {
echo "Building the checked out project";
sh 'build.sh'
}
}
stage('Unit-Test') {
steps {
echo "Running JUnit Tests";
}
}
stage('Quality-Gate') {
steps {
echo "Verifying Quality Gates";
sh 'quality.sh'
}
}
stage('Deploy') {
steps {
echo "Deploying to Stage Environment for more tests";
sh 'deploy.sh'
}
}
}
}
Erors:
Running on Jenkins in /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub
/var/lib/jenkins/workspace/Scripted_Pipeline_GitHub#tmp/durable-9a8651c4/script.sh: 1: build.sh: not found
How to convert declarative pipeline as below to scripted pipeline?
I have this syntax for declarative pipeline and I would like to use the dockerfile in my jenkinsfile which is in scripted (node () ) pipeline.
pipeline {
agent { dockerfile true }
stages {
stage('Test') {
steps {
sh 'node --version'
sh 'svn --version'
}
}
}
}
You can try like this:
node {
checkout scm
def customImage = docker.build("my-image:${env.BUILD_ID}")
customImage.inside {
sh 'node —-version'
sh 'svn --version'
}
}
The build() method builds the Dockerfile in the current directory by default. This can be overridden by providing a directory path containing a Dockerfile as the second argument of the build() method, for example:
node {
checkout scm
def customImage = docker.build("my-image:${env.BUILD_ID}", "./dockerfiles/test")
customImage.inside {
sh 'node —-version'
sh 'svn --version'
}
}
I have a Jenkins pipeline which is running fine but it depends upon JDK and maven installed tools. There were few instances in the past that these JDK and maven tool's name was changed(e.g. Maven 3.6.2 -> Maven 3.6.3 and it results in my pipeline failure.
stage ("build") {
withMaven(jdk: 'Java SE 8u221', maven: 'Maven 3.6.3', tempBinDir: '') {
sh 'mvn clean package jib:dockerBuild verify'
}
}
I want my pipeline to be independent of what tools are installed. So I rewrite my Jenkins pipeline like below to provide docker image of maven(since JDK is bundled with it)
pipeline {
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
}
stages {
stage('Checkout') {
steps {
git branch: "master", url: "repo url", credentialsId: 'id'
}
}
stage ("build") {
steps {
sh 'mvn clean package jib:dockerBuild verify'
}
}
}
}
But now I am getting an error Failed to execute goal com.google.cloud.tools:jib-maven-plugin:2.3.0:dockerBuild (default-cli) : Build to Docker daemon failed, perhaps you should make sure Docker is installed and you have correct privileges to run it
It seems that docker daemon is not visible after I provided a maven docker image.
I did solve this by adding docker agent inside of my maven docker image
pipeline {
agent any
stages {
stage('build Dockerfile') {
steps {
sh '''echo "FROM maven:3-alpine
RUN apk add --update docker openrc
RUN rc-update add docker boot" >/var/lib/jenkins/workspace/Dockerfile'''
}
}
stage('run Dockerfile') {
agent{
dockerfile {
filename '/var/lib/jenkins/workspace/Dockerfile'
args '--user root -v $HOME/.m2:/root/.m2 -v /var/run/docker.sock:/var/run/docker.sock'
}
}
steps {
sh 'docker version'
sh 'mvn -version'
sh 'java -version'
}
}
}
}
I'm trying to do Continuous Integration for the Jhipster project using Jenkins CI. In Jenkins I created as a pipeline project and tried to build with the following Jenkinsfile,
node {
stage('checkout') {
checkout scm
}
// uncomment these 2 lines and edit the name 'node-4.6.0' according to what you choose in configuration
// def nodeHome = tool name: 'node-4.6.0', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'
// env.PATH = "${nodeHome}/bin:${env.PATH}"
stage('check tools') {
sh "node -v"
sh "npm -v"
sh "bower -v"
sh "gulp -v"
}
stage('npm install') {
sh "npm install"
}
stage('clean') {
sh "mvnw clean"
}
stage('backend tests') {
try {
sh "mvnw test"
} catch(err) {
throw err
} finally {
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
}
stage('frontend tests') {
try {
sh "gulp test"
} catch(err) {
throw err
} finally {
step([$class: 'JUnitResultArchiver', testResults: '**/target/test-results/karma/TESTS-*.xml'])
}
}
stage('packaging') {
sh "mvnw package -Pprod -DskipTests"
}
}
It works fine upto npm install, on clean stage it get failure with error mvnw: command not found. I have tried with mvn clean it works but mvnw command is not working.
Also I have tried with './mvnw clean', this tries to download something from maven and fails because of connection timeout.
Any help would be appreciated.