I have a Jenkins freestyle job (yeah I know..) that sometimes needs to wait for another job with the same parameter to finish if it's running (or not run at all if it failed).
Using curl and the Jenkins API, is it possible to query a certain job and get the status of the last build where certainParam=certainValue?
(The reason I'm asking how to do that with curl is because it doesn't seem to be possible to do in freestyle job and the job can't be migrated to pipeilnes yet.. It seems like curl would be the east way..)
Thanks ahead!
So far I know there's no direct way to achieve it.
I wrote recursive script that search by the values on each build from the very last until match the information.
It prints every build url and the result of the query.
Dependencies
jq - install "jq.x86_64", Command-line JSON processor
Script
#!/bin/bash
user="admin"
pwd="11966e4dd8c33a730abf88e98fb952ebc3"
builds=$(curl -s --user $user:$pwd localhost:8080/job/test/api/json)
urls=$(echo $builds | jq '.builds[] | .url')
while read -r url
do
url=$(echo $url | sed -nr 's/"([^|]*)"/\1/p')
# get the build log
build=$(curl -s --user $user:$pwd "${url}api/json")
# transform the log in a simple structure
build=$(echo $build | jq '.result as $result | .actions[].parameters[]? | select(.name == "certainParam") | {(.name): .value, "result": $result}')
# check if the parameter value and the build result are the expected
result=$(echo $build | jq 'if .certainParam == "certainValue" and .result == "SUCCESS" then true else false end')
# print the result of each build
echo "url=${url}api/json;result=$result"
if [[ $result == true ]]
then
break
fi
done <<< $urls
Result
sh jenkins.sh
url=http://localhost:8080/job/test/12/api/json;result=false
url=http://localhost:8080/job/test/11/api/json;result=true
I have something like this on a Jenkinsfile (Groovy) and I want to record the stdout and the exit code in a variable in order to use the information later.
sh "ls -l"
How can I do this, especially as it seems that you cannot really run any kind of groovy code inside the Jenkinsfile?
The latest version of the pipeline sh step allows you to do the following;
// Git committer email
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
Another feature is the returnStatus option.
// Test commit message for flags
BUILD_FULL = sh (
script: "git log -1 --pretty=%B | grep '\\[jenkins-full]'",
returnStatus: true
) == 0
echo "Build full flag: ${BUILD_FULL}"
These options where added based on this issue.
See official documentation for the sh command.
For declarative pipelines (see comments), you need to wrap code into script step:
script {
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
}
Current Pipeline version natively supports returnStdout and returnStatus, which make it possible to get output or status from sh/bat steps.
An example:
def ret = sh(script: 'uname', returnStdout: true)
println ret
An official documentation.
quick answer is this:
sh "ls -l > commandResult"
result = readFile('commandResult').trim()
I think there exist a feature request to be able to get the result of sh step, but as far as I know, currently there is no other option.
EDIT: JENKINS-26133
EDIT2: Not quite sure since what version, but sh/bat steps now can return the std output, simply:
def output = sh returnStdout: true, script: 'ls -l'
If you want to get the stdout AND know whether the command succeeded or not, just use returnStdout and wrap it in an exception handler:
scripted pipeline
try {
// Fails with non-zero exit if dir1 does not exist
def dir1 = sh(script:'ls -la dir1', returnStdout:true).trim()
} catch (Exception ex) {
println("Unable to read dir1: ${ex}")
}
output:
[Pipeline] sh
[Test-Pipeline] Running shell script
+ ls -la dir1
ls: cannot access dir1: No such file or directory
[Pipeline] echo
unable to read dir1: hudson.AbortException: script returned exit code 2
Unfortunately hudson.AbortException is missing any useful method to obtain that exit status, so if the actual value is required you'd need to parse it out of the message (ugh!)
Contrary to the Javadoc https://javadoc.jenkins-ci.org/hudson/AbortException.html the build is not failed when this exception is caught. It fails when it's not caught!
Update:
If you also want the STDERR output from the shell command, Jenkins unfortunately fails to properly support that common use-case. A 2017 ticket JENKINS-44930 is stuck in a state of opinionated ping-pong whilst making no progress towards a solution - please consider adding your upvote to it.
As to a solution now, there could be a couple of possible approaches:
a) Redirect STDERR to STDOUT 2>&1
- but it's then up to you to parse that out of the main output though, and you won't get the output if the command failed - because you're in the exception handler.
b) redirect STDERR to a temporary file (the name of which you prepare earlier) 2>filename (but remember to clean up the file afterwards) - ie. main code becomes:
def stderrfile = 'stderr.out'
try {
def dir1 = sh(script:"ls -la dir1 2>${stderrfile}", returnStdout:true).trim()
} catch (Exception ex) {
def errmsg = readFile(stderrfile)
println("Unable to read dir1: ${ex} - ${errmsg}")
}
c) Go the other way, set returnStatus=true instead, dispense with the exception handler and always capture output to a file, ie:
def outfile = 'stdout.out'
def status = sh(script:"ls -la dir1 >${outfile} 2>&1", returnStatus:true)
def output = readFile(outfile).trim()
if (status == 0) {
// output is directory listing from stdout
} else {
// output is error message from stderr
}
Caveat: the above code is Unix/Linux-specific - Windows requires completely different shell commands.
this is a sample case, which will make sense I believe!
node('master'){
stage('stage1'){
def commit = sh (returnStdout: true, script: '''echo hi
echo bye | grep -o "e"
date
echo lol''').split()
echo "${commit[-1]} "
}
}
For those who need to use the output in subsequent shell commands, rather than groovy, something like this example could be done:
stage('Show Files') {
environment {
MY_FILES = sh(script: 'cd mydir && ls -l', returnStdout: true)
}
steps {
sh '''
echo "$MY_FILES"
'''
}
}
I found the examples on code maven to be quite useful.
All the above method will work. but to use the var as env variable inside your code you need to export the var first.
script{
sh " 'shell command here' > command"
command_var = readFile('command').trim()
sh "export command_var=$command_var"
}
replace the shell command with the command of your choice. Now if you are using python code you can just specify os.getenv("command_var") that will return the output of the shell command executed previously.
How to read the shell variable in groovy / how to assign shell return value to groovy variable.
Requirement : Open a text file read the lines using shell and store the value in groovy and get the parameter for each line .
Here , is delimiter
Ex: releaseModule.txt
./APP_TSBASE/app/team/i-home/deployments/ip-cc.war/cs_workflowReport.jar,configurable-wf-report,94,23crb1,artifact
./APP_TSBASE/app/team/i-home/deployments/ip.war/cs_workflowReport.jar,configurable-temppweb-report,394,rvu3crb1,artifact
========================
Here want to get module name 2nd Parameter (configurable-wf-report) , build no 3rd Parameter (94), commit id 4th (23crb1)
def module = sh(script: """awk -F',' '{ print \$2 "," \$3 "," \$4 }' releaseModules.txt | sort -u """, returnStdout: true).trim()
echo module
List lines = module.split( '\n' ).findAll { !it.startsWith( ',' ) }
def buildid
def Modname
lines.each {
List det1 = it.split(',')
buildid=det1[1].trim()
Modname = det1[0].trim()
tag= det1[2].trim()
echo Modname
echo buildid
echo tag
}
If you don't have a single sh command but a block of sh commands, returnstdout wont work then.
I had a similar issue where I applied something which is not a clean way of doing this but eventually it worked and served the purpose.
Solution -
In the shell block , echo the value and add it into some file.
Outside the shell block and inside the script block , read this file ,trim it and assign it to any local/params/environment variable.
example -
steps {
script {
sh '''
echo $PATH>path.txt
// I am using '>' because I want to create a new file every time to get the newest value of PATH
'''
path = readFile(file: 'path.txt')
path = path.trim() //local groovy variable assignment
//One can assign these values to env and params as below -
env.PATH = path //if you want to assign it to env var
params.PATH = path //if you want to assign it to params var
}
}
Easiest way is use this way
my_var=`echo 2`
echo $my_var
output
: 2
note that is not simple single quote is back quote ( ` ).
Below is the output after running the build(with success):
$ sam build
2019-06-02 15:36:37 Building resource 'SomeFunction'
2019-06-02 15:36:37 Running PythonPipBuilder:ResolveDependencies
2019-06-02 15:36:39 Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>
[command] && echo "Yes" approach did not help me.
I tried to use this in Jenkins pipeline
def samAppBuildStatus = sh(script: '[cd sam-app-folder; sam build | grep 'Succeeded' ] && echo true', returnStatus: true) as Boolean
as one-liner script command, but does not work
How to grab the success build status using bash script? for Jenkins pipeline
Use this to grab the exit status of the command:
def samAppBuildStatus = sh returnStatus: true, script: 'cd sam-app-folder; sam build | grep "Succeeded"'
or this if you don't want to see any stderr in the output:
def samAppBuildStatus = sh returnStatus: true, script: 'cd sam-app-folder; sam build 2>&1 | grep "Succeeded"'
then later in your Jenkinsfile you can do something like this:
if (!samAppBuildStatus){
echo "build success [$samAppBuildStatus]"
} else {
echo "build failed [$samAppBuildStatus]"
}
The reason for the ! is because the definitions of true and false between shell and groovy differ (0 is true for shell).
I have a Jenkins scripted pipeline set up where I execute a number of Maven builds. I want to treat one of them as non-fatal if the root cause is a known one.
I have tried to achieve that by inspecting the Exception's message, e.g.
try {
sh "mvn -U clean verify sonar:sonar ${sonarcloudParams}"
} catch ( Exception e ) {
if ( e.getMessage().contains("not authorized to run analysis")) {
echo "Marking build unstable due to missing SonarCloud onboarding. See https://cwiki.apache.org/confluence/display/SLING/SonarCloud+analysis for steps to fix."
currentBuild.result = 'UNSTABLE'
}
}
The problem is that the exception's message is not the one from Maven, but instead "script returned exit code 1".
There is no further information in e.getCause().
How can I access the cause of the Maven build failure inside my scripted pipeline?
You can get the command output, then parse it containers specific message.
def output = sh(
script: "mvn -U clean verify sonar:sonar ${sonarcloudParams}",
returnStdout: true
).trim()
echo "mvn cmd output: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
// parse jenkins job build log
def logUrl = env.BUILD_URL + 'consoleText'
def cmd = "curl -u \${JENKINS_AUTH} -k ${logUrl} | tail -n 50"
def output = sh(returnStdout: true, script: cmd).trim()
echo "job build log: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
One option is to inspect the last log lines using
def sonarCloudNotEnabled = currentBuild.rawBuild.getLog(50).find {
line -> line.contains("not authorized to run analysis")
}
However, this does not work by default. On the Jenkins instance I'm using it errors out with
Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild. Administrators can decide whether to approve or reject this signature.
I'm using groovy+gmaven+maven to automate/construct build procedure.
When I using ant sshexec task from within groovy script, which, in turn, executed from the gmaven plugin section, I've got some log info came from sshexec. I wonder if it is possible to write NO output while using ant.sshexec() from within groovy script? I have the following script:
def ant = new AntBuilder()
ant.sshexec(host: host,
port: port,
trust: true,
username: user,
password: pass,
command: "if test -d ${installDir}; then echo true; else echo false; fi",
outputproperty: 'doesInstallDirExist')
And the output is:
[sshexec] Connecting to 192.168.56.101:22
[sshexec] cmd : if test -d /mango/tomcat7/webapps; then echo true; else echo false; fi
Can I suppress/hide it?
Or, have you tried:
ant.project.buildListeners.firstElement().messageOutputLevel = 0
There is not a simple way to do so, the sshexec Ant task is not exposing any option to be quieter.
But since you are in Groovy, you can access the logger and disable the logging just while executing the sshexec task.
Here is piece of code to temporary augment the logging level:
def ant = new AntBuilder();
def logger = ant.project.buildListeners.firstElement();
logger.messageOutputLevel = org.apache.tools.ant.Project.MSG_WARN;
ant.sshexec(...)
logger.messageOutputLevel = org.apache.tools.ant.Project.MSG_INFO;