I am looking for help with with our Jenkins Pipeline setup. I had a Jenkins pipeline job working just fine, where the groovy script was checked out from a Perforce stream (in stage "Declarative: Checkout SCM") and then run. The script itself performs, at its core, a p4 sync and a p4 reconcile.
pipeline {
agent {
node {
customWorkspace "workspaces/MY_WORKSPACE"
}
}
stages {
stage('Sync') {
steps {
script {
p4sync(
charset: 'none',
credential: '1',
format: "jenkins-${NODE_NAME}-MY_WORKSPACE",
populate: syncOnly(force: false, have: true, modtime: false, parallel: [enable: false, minbytes: '1024', minfiles: '1', threads: '4'], pin: '', quiet: true, revert: true),
source: streamSource('//depot/STREAM')
)
}
}
}
stage('Reconcile') {
steps {
script {
withCredentials([usernamePassword(credentialsId: '1', passwordVariable: 'SVC_USER_PW', usernameVariable: 'SVC_USER_NAME')]) {
bat label: 'P4 reconcile', true, script:
"""
p4 -c "%P4_CLIENT%" -p "%P4_PORT%" -u ${SVC_USER_NAME} -P ${SVC_USER_PW} -s reconcile -e -a -d -f "//depot/STREAM/some/folder/location/*.file"
"""
}
}
}
}
}
}
Due to an exterior requirement, we decided to move all our pipeline script files to a separate depot on the same Perforce server and changed the pipeline script checkout accordingly.
Now, the pipeline script checkout step ("Declarative: Checkout SCM") will create a new workspace called jenkins-NODE_NAME-buildsystems (for the pipeline script depot //buildsystems) which will use the same local workspace root directory D:\some\path\workspaces\MY_WORKSPACE on the build node as the actual workspace jenkins-NODE_NAME-MY_WORKSPACE, created and synced in the first pipeline step by p4sync. This means that Perforce creates two workspaces with the same local workspace root directory (which can cause all sorts of problems in itself). In addition, in the pipeline script, the P4 environment variable P4_CLIENT points to the wrong workspace jenkins-NODE_NAME-buildsystems (so the reconcile won't work), which should only have been used by the pipeline script checkout, not by the pipeline itself.
Which brings me to my question. How can I separate the workspaces of the pipeline script checkout and of the p4sync in the pipeline script? In the pipeline I can specify a customWorkspace, but not in the Jenkins configuration for the pipeline script checkout, and the latter weirdly seems to follow that customWorkspace statement, maybe because jenkins-NODE_NAME-MY_WORKSPACE had already been opened by Perforce on the node...?
Any hints are much appreciated.
Thanks,
Stefan
Related
I am new to jenkins and have build a pipeline project. In one of the stage , I build a docker image and in the next stage , I execute container-structure-test on the docker image . The test cases results can be viewed in the console output.
What I want is in the build summary page , it has a link from where I can directly view the test case results in the logs and need not to go through the complete console output. Since its not junit test cases , I could not find any straight out of the box jenkins plugin.
I came across console log parser plugin but I'm not sure how to use in jenkins declarative pipeline project . I see this option in free style project under post build action but no such option is available in pipeline project.
Could someone suggest me how I can make use of this plugin in pipeline builds to address my usecase.
You can write out the container running log to a file, then publish this file as report.
stage('Test') {
steps {
script {
out = sh(returnStdout: true,
script: '''
docker run ......
'''
)
writeFile text: out, file: 'test.log'
publishHTML([
allowMissing: true, alwaysLinkToLastBuild: false,
includes: 'test.log', keepAll: false,
reportDir: '.', reportFiles: 'test.log',
reportName: 'HTML Report'
])
}
}
}
Here is official documentation mentioning which method you should use:
https://www.jenkins.io/doc/pipeline/steps/log-parser/
You can use it anywhere, but probably you would like to use it at the end of the pipeline in the post phase.
Example:
pipeline {
// some stages here
post {
always {
logParser ([
projectRulePath: 'path/to/rules/file/on/the/node',
parsingRulesPath: '',
showGraphs: true,
unstableOnWarning: true,
useProjectRule: true
])
}
}
}
I'm generating a Jenkins pipeline using a script. This pipeline is then loaded and executed. This works fine on any linux node and with sh statements etc.
I have now encountered an issue executing this on a windows node with the powershell and bat steps:
The checkout scm works fine, but any steps with the powershell or bat steps hang indefinitely. Copying the data from the generated file and replaying everything in a single Jenkinsfile works as expected.
An example of what is running (excluding the library):
"Main" Jenkinsfile:
def pod_label = "my_linux_node"
node(pod_label) {
stage("Checkout") {
checkout scm
}
stage("Pipeline generation") {
// genarate pipeline and save in "gen_pipeline.groovy"
}
stage("Run Pipeline") {
pipeline = load "gen_pipeline.groovy"
pipeline.run_pipeline()
}
}
Script1:
def run_pipeline(){
node('my_win_node') {
checkout scm
bat "echo foobar"
}
}
return this
I'm using "checkout build script from scm" option, paired with lightweight checkout.
I would like to add repository polling to that.
This is Jenkinsfile that I use:
pipeline {
agent any
triggers {
pollSCM('H/1 * * * *')
}
stages {
stage('Checkout') {
steps {
checkout([
$class : 'GitSCM',
branches : [[name: 'master']],
userRemoteConfigs : [[url: 'file:///home/my-secret-home/workspace/pipeline-test']]])
}
}
stage('Echo!') {
steps {
sh 'echo TEST'
}
}
}
}
Although job is running, git polling log tries to convince me that 'Polling has not run yet.'
Is configuring such behavior possible at all?
No, it doesn't work.
With lightweight checkout, the mapping to the remote branches are lost, so git doesn't know where to look for further updates.
You can also confirm this by running git pull on the local repository. It returns:
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.
git pull <remote> <branch>
If you wish to set tracking information for this branch you can do so with:
git branch --set-upstream-to=origin/<branch> master
Is it possible to use parameters to allow users to pass a git sha to a multi branch pipeline while defaulting to the head of the branch? Also I would only need to this function for the master branch.
I'm using ...
Jenkinsfile in code
Jenkins Declarative Pipeline
I was able to do this with declarative pipelines with the following...
pipeline {
options {
skipDefaultCheckout()
}
...
steps {
script {
if (GIT_REVISION=='HEAD') {
checkout scm
} else {
checkout([$class: 'GitSCM',
branches: [[name: "${params.GIT_REVISION}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'XXXXXXX', url: 'git#github.com:xxxxx/xxxxx.git']]
])
}
...
}
}
}
Yes, this is possible, but I guess you have to use scripted pipelines instead of declarative ones.
If the current branch is the master, you configure a parameter for this build (as this isn't super intuitive, I wrote a blog article a while ago). params.INPUT_REVISION for example would then store the given revision and you can set default to HEAD or fallback to it, if the parameter is not yet specified (e.g. for the first run).
You supply this revision to the checkout(scm) step as a parameter so that it doesn't checkout the current master branch, but the specified revision.
I wrote a Jenkins pipeline which git clones a repository, builds feature branches code and if everything is finished successfully it should merge the branch to master.
here's the relevant code:
stage ('Merge to master') {
if(currentBuild.result == 'SUCCESS') {
bat """
cd %workspace%
echo "BRANCH_NAME: %BRANCH_NAME%"
echo "Env.BRANCH_NAME: %env.BRANCH_NAME%
git checkout master
REM git merge %GIT_BRANCH%
"""
}
}
The code which is responsible for the cloning:
stage ('Checkout SCM') { // This stage is responsible to clone the repository into Jenkins's workspace
checkout([$class: 'GitSCM', branches: [[name: '*/feature/*']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '99f978af-XXXX-XXXX-8147-2cf8f69ef864', url: 'http://TFS_SERVER:8080/tfs/DefaultCollection/PC_International/_git/repo-name']]])
}
My problem is that it seems like the BRANCH_NAME variable is not set, I've tried accessing it by %BRANCH_NAME% , %env.BRANCH_NAME%, %GIT_BRANCH% and %env.GIT_BRANCH% but to no avail.
In Jenkins build log, it looks like that:
C:\Program Files (x86)\Jenkins\workspace\Ensure>echo "BRANCH_NAME: "
"BRANCH_NAME: "
C:\Program Files (x86)\Jenkins\workspace\Ensure>echo "Env.BRANCH_NAME:
"Env.BRANCH_NAME:
C:\Program Files (x86)\Jenkins\workspace\Ensure>echo "GIT_BRANCH: "
"GIT_BRANCH: "
C:\Program Files (x86)\Jenkins\workspace\Ensure>echo "Env.GIT_BRANCH: env.GIT_BRANCH"
"Env.GIT_BRANCH: env.GIT_BRANCH"
Any idea what I'm doing wrong?
You need to interpolate the string value using the Groovy placeholders ${}, i.e. ${env.BRANCH_NAME}
This will replace the variable with the string literal before the batch script is executed. The %% notation is for interpolating windows environment variables, but env.BRANCH_NAME is Jenkins environment variable not a windows environment variable.
This worked for me:
"manage jenkins" -> "Configure System" ... scroll down to "Global Properties" and check the checkbox for "Environmental Variables".
Add your variable (i.e. Name: MY_BRANCH, value: "*/master")
Restart Jenkins
Go to your build then you can use the variable in your build configs as ${MY_BRANCH}