I'm trying to do Continuous Integration for the Jhipster project using Jenkins CI. In Jenkins I created as a pipeline project and tried to build with the following Jenkinsfile,
node {
stage('checkout') {
checkout scm
}
// uncomment these 2 lines and edit the name 'node-4.6.0' according to what you choose in configuration
// def nodeHome = tool name: 'node-4.6.0', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'
// env.PATH = "${nodeHome}/bin:${env.PATH}"
stage('check tools') {
sh "node -v"
sh "npm -v"
sh "bower -v"
sh "gulp -v"
}
stage('npm install') {
sh "npm install"
}
stage('clean') {
sh "mvnw clean"
}
stage('backend tests') {
try {
sh "mvnw test"
} catch(err) {
throw err
} finally {
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
}
stage('frontend tests') {
try {
sh "gulp test"
} catch(err) {
throw err
} finally {
step([$class: 'JUnitResultArchiver', testResults: '**/target/test-results/karma/TESTS-*.xml'])
}
}
stage('packaging') {
sh "mvnw package -Pprod -DskipTests"
}
}
It works fine upto npm install, on clean stage it get failure with error mvnw: command not found. I have tried with mvn clean it works but mvnw command is not working.
Also I have tried with './mvnw clean', this tries to download something from maven and fails because of connection timeout.
Any help would be appreciated.
Related
Following is my Machine details
Specification
Version
OS
Windows-11
Java
11
Gradle
7.3
I had setup of shell on Jenkins as well.
And Jenkins code snippet is as follows:
#!/usr/bin/env groovy
/* groovylint-disable CompileStatic, DuplicateStringLiteral, UnnecessaryGString, VariableName */
import java.text.SimpleDateFormat
pipeline {
agent any
stages {
stage('build-and-package') {
steps {
script {
sh "git update-index --chmod=+x gradlew"
sh "chmod +x gradlew"
sh "./gradlew clean assemble --stacktrace"
//sh "gradle clean build"
sh "ls -lrt"
sh "ls -lrt ./build/libs/"
}
}
}
stage('docker-build-and-push') {
steps {
//Code Commented
}
}
stage('helm-deploy') {
steps {
script {
//Code Commented
}
}
}
}
}
I have tried to provide full access rights to Jenkins workspace
Error Screen shot
The Jenkins is installed at Ubuntu. The Jenkins workspace is at /var/lib/jenkins/workspace. The github repo is at https://github.com/garyyang6/scripts. There are three scripts in the repo, build.sh, deploy.sh and quality.sh. The pipeline groovy script is as follow. After I ran the groovy script in Jenkins, it complains build.sh: not found.
At /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub, I found the scripts, build.sh deploy.sh quality.sh. The folder /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub#tmp is empty.
pipeline {
agent any
stages {
stage('Git-Checkout') {
steps {
echo 'Checking out from Git Repo';
git 'https://github.com/garyyang6/scripts.git'
}
}
stage('Build') {
steps {
echo "Building the checked out project";
sh 'build.sh'
}
}
stage('Unit-Test') {
steps {
echo "Running JUnit Tests";
}
}
stage('Quality-Gate') {
steps {
echo "Verifying Quality Gates";
sh 'quality.sh'
}
}
stage('Deploy') {
steps {
echo "Deploying to Stage Environment for more tests";
sh 'deploy.sh'
}
}
}
}
Erors:
Running on Jenkins in /var/lib/jenkins/workspace/Scripted_Pipeline_GitHub
/var/lib/jenkins/workspace/Scripted_Pipeline_GitHub#tmp/durable-9a8651c4/script.sh: 1: build.sh: not found
I have a Jenkins pipeline which is running fine but it depends upon JDK and maven installed tools. There were few instances in the past that these JDK and maven tool's name was changed(e.g. Maven 3.6.2 -> Maven 3.6.3 and it results in my pipeline failure.
stage ("build") {
withMaven(jdk: 'Java SE 8u221', maven: 'Maven 3.6.3', tempBinDir: '') {
sh 'mvn clean package jib:dockerBuild verify'
}
}
I want my pipeline to be independent of what tools are installed. So I rewrite my Jenkins pipeline like below to provide docker image of maven(since JDK is bundled with it)
pipeline {
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
}
stages {
stage('Checkout') {
steps {
git branch: "master", url: "repo url", credentialsId: 'id'
}
}
stage ("build") {
steps {
sh 'mvn clean package jib:dockerBuild verify'
}
}
}
}
But now I am getting an error Failed to execute goal com.google.cloud.tools:jib-maven-plugin:2.3.0:dockerBuild (default-cli) : Build to Docker daemon failed, perhaps you should make sure Docker is installed and you have correct privileges to run it
It seems that docker daemon is not visible after I provided a maven docker image.
I did solve this by adding docker agent inside of my maven docker image
pipeline {
agent any
stages {
stage('build Dockerfile') {
steps {
sh '''echo "FROM maven:3-alpine
RUN apk add --update docker openrc
RUN rc-update add docker boot" >/var/lib/jenkins/workspace/Dockerfile'''
}
}
stage('run Dockerfile') {
agent{
dockerfile {
filename '/var/lib/jenkins/workspace/Dockerfile'
args '--user root -v $HOME/.m2:/root/.m2 -v /var/run/docker.sock:/var/run/docker.sock'
}
}
steps {
sh 'docker version'
sh 'mvn -version'
sh 'java -version'
}
}
}
}
I am trying to create Jenkins Pipeline for one of my automation job. I created Jenkins file. Code specified below:
pipeline {
agent any
def mvn_version = 'MavenTest'
withEnv( ["PATH+MAVEN=${tool mvn_version}/bin"] ) {
//sh "mvn clean package"
}
stages {
stage('Git checkout') { // for display purposes
steps {
git branch: "ReportTest", url: 'https://github.abc.com/vsing136/testWDM.git'
sh "mvn clean verify"
}
}
stage('Stage 1') {
steps {
echo 'Hello world!'
}
}
}
}
post {
always {
emailext body: "Build URL: ${BUILD_URL}",
subject: "$currentBuild.currentResult-$JOB_NAME",
to: 'vabc1#example.com'
}
}
Screenshot with my job configuration specified below:
I am not sure what am I doing wrong but I am getting error for this configuration -
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 4: Tool type "maven" does not have an install of "Maven 3.3.9" configured - did you mean "Maven"? # line 4, column 11.
maven 'Maven 3.3.9'
^
1 error
Following code worked for me:
agent any
tools {
maven 'MavenTest'
}
When working with jenkins 2 (declarative) pipelines and maven I always have a problem with how to organize things within the pipeline to make it resusable and flexible.
On the one side I would like to seperate the pipepline into logical stages like:
pipeline
{
stages
{
stage('Clean') {}
stage('Build') {}
stage('Test') {}
stage('Sanity check') {}
stage('Documentation') {}
stage('Deploy - Test') {}
stage('Selenium tests') {}
stage('Deploy - Production') {}
stage('Deliver') {}
}
}
On the other hand I have maven which runs with
mvn clean deploy site
Simply I could split up maven to
mvn clean
mvn deploy
mvn site
But the 'deploy' includes all lifecycle phases from
validate
compile
test
package
verify
install
deploy
So I saw a lot of pipline examples which do things like
sh 'mvn clean compile'
and
sh 'mvn test'
which results in repeating the validate and compile step a second time and waste "time/resources" in this way.
This could be resolved with doing a
sh 'mvn surefire:test'
instead of running the whole lifecycle again.
So my question is - which is the best way to get a good balance between the jenkins pipline stages and the maven lifecycle?
For me I see two ways:
Split up the maven lifecycles to as much pipeline stages as possible - which will result in better jenkins user feedback (see which stage fails etc.)
Let maven do everything and use the jenkins pipeline only to work with the results of maven (i.e. analyzing unit test results etc.)
Or did I missunderstand something in the CI/CD practice?
Two month later I think I have a well balanced Jenkins pipeline script that is not complete, but works stable on windows and linux. It avoids pitfalls of other examples I have seen.
Jenkinsfile
pipeline
{
agent any
tools
{
maven 'Maven3'
jdk 'JDK8'
}
options
{
buildDiscarder(logRotator(numToKeepStr: '4'))
skipStagesAfterUnstable()
disableConcurrentBuilds()
}
triggers
{
// MINUTE HOUR DOM MONTH DOW
pollSCM('H 6-18/4 * * 1-5')
}
stages
{
stage('Clean')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode clean'
}
else
{
bat 'mvn --batch-mode clean'
}
}
}
}
stage('Build')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode compile'
}
else
{
bat 'mvn --batch-mode compile'
}
}
}
}
stage('UnitTests')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode resources:testResources compiler:testCompile surefire:test'
}
else
{
bat 'mvn --batch-mode resources:testResources compiler:testCompile surefire:test'
}
}
}
post
{
always
{
junit testResults: 'target/surefire-reports/*.xml'
}
}
}
stage('Sanity check')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode checkstyle:checkstyle pmd:pmd pmd:cpd com.github.spotbugs:spotbugs-maven-plugin:spotbugs'
}
else
{
bat 'mvn --batch-mode checkstyle:checkstyle pmd:pmd pmd:cpd com.github.spotbugs:spotbugs-maven-plugin:spotbugs'
}
}
}
}
stage('Packaging')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode jar:jar'
}
else
{
bat 'mvn --batch-mode jar:jar'
}
}
}
}
stage('install local')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode jar:jar source:jar install:install'
}
else
{
bat 'mvn --batch-mode jar:jar source:jar install:install' // maven-jar-plugin falseCreation default is false, so no doubled jar construction here, but required for maven-install-plugin internal data
}
}
}
}
stage('Documentation')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode site'
}
else
{
bat 'mvn --batch-mode site'
}
}
}
post
{
always
{
publishHTML(target: [reportName: 'Site', reportDir: 'target/site', reportFiles: 'index.html', keepAll: false])
}
}
}
stage('Deploy test')
{
steps
{
script
{
if (isUnix())
{
// todo
}
else
{
bat returnStatus: true, script: 'sc stop Tomcat8'
sleep(time:30, unit:"SECONDS")
bat returnStatus: true, script: 'C:\\scripts\\clean.bat'
bat returnStatus: true, script: 'robocopy "target" "C:\\Program Files\\Apache Software Foundation\\Tomcat 9.0\\webapps" Test.war'
bat 'sc start Tomcat8'
sleep(time:30, unit:"SECONDS")
}
}
}
}
stage('Integration tests')
{
steps
{
script
{
if (isUnix())
{
sh 'mvn --batch-mode failsafe:integration-test failsafe:verify'
}
else
{
bat 'mvn --batch-mode failsafe:integration-test failsafe:verify'
}
}
}
}
}
}
Hopefully this is interesting for other developers outside there.
I will update this here when I significantly improve it over time.
For those who also wish to see a maven pom along with a Jenkinsfile please have a look at my small example project at github: TemplateEngine
I think there is no right answer, but the following example worked for us.
stage('Build and Unit Test') {
mvn clean deploy -> with unit tests, without integration tests, deploy local
deploy local:
You can define in a maven profile the distributionManagement like:
<distributionManagement>
<repository>
<id>localFile</id>
<url>file:target/repository/</url>
</repository>
<snapshotRepository>
<id>localFile</id>
<url>file:target/repository/</url>
</snapshotRepository>
</distributionManagement>
}
stage('Pre Integration Tests') {
The binaries are now in target/repository.
From there you can use the binaries as you like.
Copy them to a server, deploy them on an application server, etc.
}
stage('Integration Tests') {
maven failsafe:integration-test failsafe:verify
Already all tests are compiled, just execute them and verify the result.
}
stage('Deploy to Binary Repository (Nexus, Artifactory, etc)') {
Now if everything is ok, finally upload the Binaries.
For that we use wagon-maven-plugin
So from target/repository the files are uploaded to the Binary Repository.
}
So to wrap this up:
Fail fast. If a unit test has errors -> fail the build.
Only build once. Use the same binaries for test, deployment/integration test,
upload to repository, etc.
With that the stages are logical units,
which gives you enough feedback where to look for errors.