I have a Jenkins scripted pipeline set up where I execute a number of Maven builds. I want to treat one of them as non-fatal if the root cause is a known one.
I have tried to achieve that by inspecting the Exception's message, e.g.
try {
sh "mvn -U clean verify sonar:sonar ${sonarcloudParams}"
} catch ( Exception e ) {
if ( e.getMessage().contains("not authorized to run analysis")) {
echo "Marking build unstable due to missing SonarCloud onboarding. See https://cwiki.apache.org/confluence/display/SLING/SonarCloud+analysis for steps to fix."
currentBuild.result = 'UNSTABLE'
}
}
The problem is that the exception's message is not the one from Maven, but instead "script returned exit code 1".
There is no further information in e.getCause().
How can I access the cause of the Maven build failure inside my scripted pipeline?
You can get the command output, then parse it containers specific message.
def output = sh(
script: "mvn -U clean verify sonar:sonar ${sonarcloudParams}",
returnStdout: true
).trim()
echo "mvn cmd output: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
// parse jenkins job build log
def logUrl = env.BUILD_URL + 'consoleText'
def cmd = "curl -u \${JENKINS_AUTH} -k ${logUrl} | tail -n 50"
def output = sh(returnStdout: true, script: cmd).trim()
echo "job build log: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
One option is to inspect the last log lines using
def sonarCloudNotEnabled = currentBuild.rawBuild.getLog(50).find {
line -> line.contains("not authorized to run analysis")
}
However, this does not work by default. On the Jenkins instance I'm using it errors out with
Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild. Administrators can decide whether to approve or reject this signature.
Related
I am trying to create a Continuous Integration between BitBucket and Salesforce using Jenkins and I am having trouble with the Scratch Org creation. The Jenkinsfile I BELIEVE is set up correctly. Here it is:
node {
def SF_JENKINSUSER = env.SF_JENKINS_USER
def SF_USERNAME = env.SF_JENKINS_USER + '.' + env.SF_DEV
def SF_URL = env.SF_TESTURL
def SF_PROD = env.SF_PRODURL
def SF_DEV_HUB = env.SF_DEVHUB
stage('Checkout Source') {
checkout scm
}
withEnv(["HOME=${env.WORKSPACE}"]) {
withCredentials([string(credentialsId: 'SF_CONSUMER_KEY_BIND', variable: 'SF_CONSUMER_KEY'), file(credentialsId: 'SERVER_KEY_CREDENTALS_ID', variable: 'server_key_file')]) {
stage('Authorize DevHub Org') {
try {
rc = command "sfdx force:auth:jwt:grant -r ${SF_PROD} -i ${SF_CONSUMER_KEY} -u ${SF_JENKINSUSER} -f ${server_key_file} --setdefaultdevhubusername -a ${SF_DEV_HUB}"
if ( rc != 0 ) {
echo '========== ERROR: ' + rc
error 'Salesforce org authorization failed.'
}
else {
command "sfdx force:org:list"
echo '========== LOGGED IN =========='
}
}
catch (err) {
echo "========== DEVHUB AUTHORIZATION FAILURE: ${err} =========="
}
}
// Create a new scratch org to test the repo
stage('Create Test Scratch Org') {
try {
rc = command "sfdx force:org:create -s -f config\\project-scratch-def.json -a TestScratch -w 10 -d 1"
if (rc != 0) {
error 'Salesforce test scratch org creation failed.'
}
}
catch (err) {
echo "========== SCRATCH ORG CREATION FAILURE: ${err} =========="
}
}
}
} }
def command(script) {
if ( isUnix() ) {
return sh(returnStatus: true, script: script);
}
else {
return bat(returnStatus: true, script: script);
}
Apologies about the formatting there. Now, the results of this I cannot figure out. It says the connected status of the orgs are JwtGrantFailure and it's looking for a server.key file instead of the scratch json file in the command line. Here are the pertinent parts of the output from this job:
E:\DevOps_Root\JENKINS\workspace\TestingCIPipeline2>sfdx force:org:list
=== Orgs
ALIAS USERNAME ORG ID CONNECTED STATUS
(D) DevHub sa.jenkins#[...].com 00D300000000UicEAE JwtGrantError
No active scratch orgs found. Specify --all to see all scratch orgs
[Pipeline] echo
========== LOGGED IN ==========
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Create Test Scratch Org)
[Pipeline] isUnix
[Pipeline] bat
E:\DevOps_Root\JENKINS\workspace\TestingCIPipeline2>sfdx force:org:create -s -f config\project-scratch-def.json -a TestScratch -w 10 -d 1
ERROR running force:org:create: ENOENT: no such file or directory, open
'E:\DevOps_Root\JENKINS\workspace\Pipe#tmp\secretFiles\e0ab232f-1958-42d1-b3bb-aed5e00a562f\server.key'
[Pipeline] echo
========== SCRATCH ORG COMMAND FAILURE: 1
Why would the job be looking for the server.key file when I have already run the withCredentials successfully? What am I missing here?
Any insights would be greatly appredciated.
Ok so .. this one confounded me for a while, but I finally got the script to create a scratch org.
I logged into the Jenkins Virtual Server through Remote Desktop Connection, opened windows explorer, navigated to the Jenkins User .sfdx folder and deleted the following files:
alias.json
key.json
user#domain.json
stash.json
After I did that, I made some updates in the Jenkinsfile and pushed the changes up to the repository. The job ran, and the Scratch Org was created.
My new issue is trying to figure out how to have the same job run again because we will have multiple repos working this single job.
Anyway, I hope this helps some of you out who are facing the same issue.
I'm building several android apps in a docker image using gradle and a bash script. The script is triggered by jenkins, which runs the docker image.
In the bash script I gather information about the successes of the builds. I want to pass that information to the groovy script of the jenkinsfile.
I tried to create a txt file in the docker container, but the groovy script in the jenkinsfile can not find that file.
This is the groovy script of my jenkinsfile:
script {
try {
sh script:'''
#!/bin/bash
./jenkins.sh
'''
} catch(e){
currentBuild.result = "FAILURE"
} finally {
String buildResults = null
try {
def pathToBuildResults="[...]/buildResults.txt"
buildResults = readFile "${pathToBuildResults}"
} catch(e) {
buildResults = "error receiving build results. Error: " + e.toString()
}
}
}
In my jenkins.sh bash script I do the following:
[...]
buildResults+=" $appName: Build Failed!" //this is done for several apps
echo "$buildResults" | cat > $pathToBuildResults //this works I checked, if the file is created
[...]
The file is created, but groovy cannot find it. I think the reason is, that the jenkins script does not run inside the docker container.
How can I access the string buildResults of the bash script in my groovy jenkins script?
One option that you have in order to avoid the need to read the results file is to modify your jenkins.sh script to print the results to the output instead of writing them to a file and then use the sh step to capture that output and use it instead of the file.
Something like:
script {
try {
String buildResults = sh returnStdout: true, script:'''
#!/bin/bash
./jenkins.sh
'''
// You now have the output of jenkins.sh inside the buildResults parameter
} catch(e){
currentBuild.result = "FAILURE"
}
}
This way you are avoiding the need to handle the output files and directly get the results you need, which you can then parse and use however you need.
I am running a shell script inside a docker container via jenkins groovy pipeline script. The bash script sets some environment variables and then executes unit tests. The stdout of these unit test execution is dumped to a text file.
I later copy this text file outside of the container for usage.
Here is the shell script:
#/bin/bash
source /root/venv/bin/activate
export PYTHONPATH=/foo/bar
cd unit_tests
rm -f results.txt
python tests.py >> results.txt
My pipeline script is as follows:
stage('Run Unit Tests') {
steps {
sh '''
docker-compose -f ./dir1/docker-compose-test.yml up -d
docker cp /supporting_files/run_unit_tests.sh container_1:/foo/bar/
docker exec container_1 /bin/bash run_unit_tests.sh
docker cp container_1:/foo/bar/unit_tests/results.txt .
'''
}
}
stage('Reporting') {
steps {
//steps for reporting
}
}
The problem is whenever any test fails, the results.txt has the appropriate text about failures and their stack. But the pipeline stop executing saying
[Pipeline] }
ERROR: script returned exit code 1
Because of this I am not able to execute next steps of parsing the results.txt file and reporting the results.
How do I make the pipeline execute next stage ?
I tried some things like:
1. Using catchError:
stage('Run Unit Tests') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh '''
//Running the commands above
'''
}
}
}
Using try:
try{
stage('Run Unit Tests') {
sh '''
//Executing tests
'''
}
} catch(e) {
echo e.toString()
}
But both of them does not help.
Also the shell script simply dumps the stdout of running tests into a text file so I don't understand why an exit code 1 should be returned as the operation itself does not fail. I saw the text file later, it had the correct failures and error counts with stack.
In my jenkins pipeline i use the "Execute shell command " to run my gradle build script.
Now i want to check if the build has failed in which case i would like to read the console output, store it in a string and publish it to a slack channel.
The code that i have tried goes as follows :
try {
for (int i = 0 ; i < noOfComponents ; i++ ){
component = compileProjectsWithPriority[i]
node {
out = sh script: "cd /home/jenkins/projects/${component} && ${gradleHome}/bin/gradle build", returnStdout: true}
}
}
catch (e){
def errorSummary = 'Build failed due to compilation error in '+"${component}"+'\n'+"${out}"
slackSend (channel: '#my_channel', color: '#FF0000', message: errorSummary)
}
However it does not even execute the shell script and also the console output is null. What is the right approach to do this.
Thanks in advance
The sh command in Jenkins pipelines may not work with shell built-ins like cd. Perhaps try using dir, as below:
node {
dir("/home/jenkins/projects/${component}") {
out = sh script: "${gradleHome}/bin/gradle build", returnStdout: true
}
}
All commands within { and } for a dir will execute with the specified directory as their working directory. This will overcome any problems that may exist with the cd shell built-in.
Generally, to get the artifact of the latest successful build, I do a wget on the below URL:
http://jenkins.com/job/job_name/lastSuccessfulBuild/artifact/artifact1/jenkins.txt
Is there a way, I can do a wget on lastSuccessfulBuild and get a build_id like below?
build_id=`wget http://jenkins.p2pcredit.local/job/job_name/lastSuccessfulBuild`
Yes, there is a way and it is pretty straightforward:
$ build_id=`wget -qO- jenkins_url/job/job_name/lastSuccessfulBuild/buildNumber`
$ echo $build_id
131 # that's my build number
I think the best solution is using groovy with zero dependencies.
node {
script{
def lastSuccessfulBuildID = 0
def build = currentBuild.previousBuild
while (build != null) {
if (build.result == "SUCCESS")
{
lastSuccessfulBuildID = build.id as Integer
break
}
build = build.previousBuild
}
println lastSuccessfulBuildID
}
}
You do not need specify jenkins_url or job_name etc to get last successful build id.
Then you could use it easily in all Jenkinsfile in repositories without useless configurations.
Tested on Jenkins v2.164.2
I find very useful querying permalinks file inside Jenkins workspace.
This allows you, to not only get the last successful build, but also other builds Jenkins considers relevant.
You can see it's content adding this line in Build section, in Execute Shell panel:
cat ../../jobs/$JOB_NAME/builds/permalinks
For example, in my case:
+ cat ../../jobs/$JOB_NAME/builds/permalinks
lastCompletedBuild 56
lastFailedBuild 56
lastStableBuild 51
lastSuccessfulBuild 51
lastUnstableBuild -1
lastUnsuccessfulBuild 56
From there, you would want to parse the number of the last successful build, or any other provided by permalinks, you can do this running:
lastSuccesfulBuildId=$(cat ../../jobs/$JOB_NAME/builds/permalinks | grep lastSuccessfulBuild | sed 's/lastSuccessfulBuild //')
If you want the DisplayName of the last successful job and not build number:
curl --user <username>:<tokenOrPassword> https://<url>/job/<job-name>/lastSuccessfulBuild/api/json | jq -r '.displayName'
Or in groovy
def buildName = Jenkins.instance.getItem('jobName').lastSuccessfulBuild.displayName
Pipeline script solution :
import groovy.json.JsonSlurper
def jResponse = httpRequest "https:/<yourjenkinsjoburlpath>/lastSuccessfulBuild/buildNumber"
def json = new JsonSlurper().parseText(jResponse.content)
echo "Status: ${json}"
jenkins console output:
HttpMethod: GET
URL: https://***/lastSuccessfulBuild/buildNumber
Sending request to url: https://***/lastSuccessfulBuild/buildNumber
Response Code: HTTP/1.1 200 OK
Success code from [100‥399]
[Pipeline] echo
Status: 20
To get the last successful build number using curl:
curl --user userName:password https://url/job/jobName/api/xml?xpath=/*/lastStableBuild/number
to get the job build number simply do:
def build_Number = Jenkins.instance.getItem('JobName').lastSuccessfulBuild.number