How to integrate JetBrains TeamCity with Atlassian Stash - teamcity

Stash 2.1 comes with a new REST API that allows you to tell Stash about builds related to specific changesets. How do I let Stash know about my builds in TeamCity?

You can use this TeamCity plugin which posts to the REST API with build statuses.
Note: I am the developer
Edit: Jetbrains also have a plugin that does the same thing, see here:
http://confluence.jetbrains.com/display/TW/Commit+Status+Publisher

JetBrains now has an official plugin called "Commit Status Publisher" that can send build status to Atlassian Stash or the Gerrit Code Review tool.
Source code is on GitHub.
Note: After installing the plugin, add a build feature, called "Commit status publisher" to your TeamCity build.

In your build configurations, insert this Powershell script as the first build step:
curl -v -H "Content-Type: application/json" -X POST -d '{ \"state\": \"INPROGRESS\", \"key\": \"%teamcity.build.id%\", \"name\": \"#%build.number%: %system.teamcity.buildConfName%; %system.teamcity.projectName%\", \"url\": \"http://TEAMCITY-HOSTNAME/viewLog.html?buildId=%teamcity.build.id%\", \"description\": \"Revision: %build.vcs.number%; Triggered by: %build.triggeredBy%\" }' http://USERNAME:PASSWORD#STASH-HOSTNAME/rest/build-status/1.0/commits/%build.vcs.number%
This will let Stash know that a build for a certain changeset has started.
As your last build step, insert this Powershell script and select the option to execute it even though your build fails:
$xml = [xml](curl --request GET http://USERNAME:PASSWORD#TEAMCITY-HOSTNAME/httpAuth/app/rest/builds/%teamcity.build.id%)
Microsoft.PowerShell.Utility\Select-Xml $xml -XPath "/build" | %% { $status = $_.Node.status }
switch ($status) {
"SUCCESS" { $stashStatus = "SUCCESSFUL"; }
default { $stashStatus = "FAILED"; }
}
$do = #'
curl -v -H "Content-Type: application/json" -X POST -d '{ \"state\": \"$stashStatus\", \"key\": \"%teamcity.build.id%\", \"name\": \"#%build.number%: %system.teamcity.buildConfName%; %system.teamcity.projectName%\", \"url\": \"http://TEAMCITY-HOSTNAME/viewLog.html?buildId=%teamcity.build.id%\", \"description\": \"Revision: %build.vcs.number%; Triggered by: %build.triggeredBy%\" }' http://USERNAME:PASSWORD#STASH-HOSTNAME/rest/build-status/1.0/commits/%build.vcs.number%
'#
$do = $do -Replace '\$stashStatus', "$stashStatus"
Invoke-Expression $do
This will let Stash know that a build for a certain changeset has either succeeded or failed.

Related

Jenkins pipeline stage build is green even though an error is present

I have a Jenkins Pipline with three stages: "Build" "Test" and "Deploy".
Here is the problem I have with "Build":
The build step ensures that structure of the Control-M Automation API json files are valid.
To do this, I utilize the $endpoint/build service provided by Automation API in the build step:
stage('Build') {
environment {
CONTROLM_CREDS = credentials('xxx')
ENDPOINT = 'xxx'
}
steps {
sh '''
username=$CONTROLM_CREDS_USR
password=$CONTROLM_CREDS_PSW
# Login
login=$(curl -k -s -H "Content-Type: application/json" -X POST -d \\{\\"username\\":\\"$username\\",\\"password\\":\\"$password\\"\\} "$ENDPOINT/session/login" )
token=$(echo ${login##*token\\" : \\"} | cut -d '"' -f 1)
# Build
curl -k -s -H "Authorization: Bearer $token" -X POST -F "definitionsFile=#ctmjobs/TestCICD.json" "$ENDPOINT/build"
curl -k -s -H "Authorization: Bearer $token" -X POST "$ENDPOINT/session/logout"
'''
}
}
<snip>
Everything works as expected, but if I intentionally put an error in the json file, Jenkins detects it and prints the error in the terminal, but "Build" still goes green. Can anyone identify the error? My expectation is that the stage "Build" goes to red as soon as there is an error in the JSON file.
Here is a Jenkins output from the terminal:
+ password=****
++ curl -k -s -H 'Content-Type: application/json' -X POST -d '{"username":"xxx","password":"****"}' /automation-api/session/login
+ login='{
"username" : "xxx",
"token" : "xxx",
"version" : "9.19.200"
}'
++ echo 'xxx",
' '"version"' : '"9.19.200"
' '}'
++ cut -d '"' -f 1
+ token=xxx
+ curl -k -s -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build
{
"errors" : [ {
"message" : "unknown type: Job:Dummmy",
"file" : "Test.json",
"line" : 40,
"col" : 29
}, {
"message" : "unknown type: Job:Dummmy",
"file" : "Test.json",
"line" : 63,
"col" : 29
} ]
}+ curl -k -s -H 'Authorization: Bearer xxx' -X POST /automation-api/session/logout
{
"message" : "Successfully logged out from session xxx"
} ``
Jenkins in order to consider a stage as failed, it will check the exit code of a command executed, in your case
curl -k -s -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build
The issue is that the curl, as a command, is executed successfully.
But the body of the curl indicates that the api call failed.
You could add --fail flag to your curl. This will force curl to return an erroneous exit code when the response status is > 400
(HTTP) Fail silently (no output at all) on server errors. This is
mostly done to enable scripts etc to better deal with failed attempts.
In normal cases when an HTTP server fails to deliver a document, it
returns an HTML document stating so (which often also describes why
and more). This flag will prevent curl from outputting that and return
error 22.
curl --show-error --fail -k -H 'Authorization: Bearer xxx' -X POST -F definitionsFile=#ctmjobs/Test.json /automation-api/build

Curl returns Invalid JSON error in a Jenkins Pipeline script but returns the expected response on a bash shell run or in a Jenkins Freestyle job

I am writing a Jenkins Pipeline job for setting up AWS infrastructure using API calls to our in-house AWS CLI wrapper library. Running the raw bash scripts on a CentOS box or as a Jenkins Freestyle job runs fine. However, it fails in the context of a Pipeline job. I think that the quotes may need to be different for the Pipeline job but I am not sure how.
After further investigation, I found that the curl command returns the wrong response from the service when running the scripts within a Jenkins Pipeline job.
pipeline {
agent any
stages {
stage('Checkout code from Git'){
steps {
echo "Checkout code from a GitHub repository"
// Checkout code from a GitHub repository
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: false, recursiveSubmodules: true, reference: '', trackingSubmodules: false]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'xxxx', url: 'git#github.com:bbc/repo.git']]])
}
}
stage('Call our internal AWS CLI Wrapper System API to perform an ACTION on a specified ENVIRONMENT') {
steps {
script {
if("${params.ENVIRONMENT}" == 'int' && "${params.ACTION}" == 'create'){
echo "ENVIRONMENT=${params.ENVIRONMENT}, ACTION=${params.ACTION}"
echo ""
sh '''#!/bin/bash
# Create Neptune Cluster for the Int environment
cd blah-db
echo "Current working directory is $PWD"
CLOUD_FORMATION_FILE=$PWD/infrastructure/templates/neptune-cluster.json
echo "The CloudFormation file to operate on is $CLOUD_FORMATION_FILE"
echo "Running jq to transform the source CloudFormation file"
template=$(jq -M '.Parameters.Env.Default="int"' $CLOUD_FORMATION_FILE)
echo "Echoing the transformed CloudFormation file: \n$template"
echo "Running curl to make the http request to our internal AWS CLI Wrapper System"
curl -d "{\"aws_account\": \"1111111111\", \"region\": \"us-east-1\", \"name_suffix\": \"cluster\", \"template\": $template}" \
-H 'Content-Type: application/json' -H 'Accept: application/json' https://base.api.url/v1/services/blah-neptune/int/stacks \
--cert /path/to/client/certificate/client.crt --key /path/to/client/private-key/client.key
cd ..
pwd
# Set a timer to run for 300 seconds or 5 minutes to create a delay to allow for the Neptune Cluster to be fully provisioned first before adding instances to it.
'''
}
}
}
}
}
}
The actual result that I get from making the API call:
{"error": "Invalid JSON. Expecting property name: line 1 column 1 (char 1)"}
try change the curl as following:
curl -d '{"aws_account": "1111111111", "region": "us-east-1", "name_suffix": "cluster", "template": $template}'
Or assign the whole cmd to a variable and print it out to see it's as your wanted or not.
cmd = '''#!/bin/bash
cd blah-db
...
'''
echo cmd // compare the output string to the cmd of freestyle job.
sh cmd

Get the cause of a Maven build failure inside a Jenkins pipeline

I have a Jenkins scripted pipeline set up where I execute a number of Maven builds. I want to treat one of them as non-fatal if the root cause is a known one.
I have tried to achieve that by inspecting the Exception's message, e.g.
try {
sh "mvn -U clean verify sonar:sonar ${sonarcloudParams}"
} catch ( Exception e ) {
if ( e.getMessage().contains("not authorized to run analysis")) {
echo "Marking build unstable due to missing SonarCloud onboarding. See https://cwiki.apache.org/confluence/display/SLING/SonarCloud+analysis for steps to fix."
currentBuild.result = 'UNSTABLE'
}
}
The problem is that the exception's message is not the one from Maven, but instead "script returned exit code 1".
There is no further information in e.getCause().
How can I access the cause of the Maven build failure inside my scripted pipeline?
You can get the command output, then parse it containers specific message.
def output = sh(
script: "mvn -U clean verify sonar:sonar ${sonarcloudParams}",
returnStdout: true
).trim()
echo "mvn cmd output: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
// parse jenkins job build log
def logUrl = env.BUILD_URL + 'consoleText'
def cmd = "curl -u \${JENKINS_AUTH} -k ${logUrl} | tail -n 50"
def output = sh(returnStdout: true, script: cmd).trim()
echo "job build log: ${output}"
if(output.contains('not authorized to run analysis')) {
currentBuild.result = 'UNSTABLE'
}
One option is to inspect the last log lines using
def sonarCloudNotEnabled = currentBuild.rawBuild.getLog(50).find {
line -> line.contains("not authorized to run analysis")
}
However, this does not work by default. On the Jenkins instance I'm using it errors out with
Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild. Administrators can decide whether to approve or reject this signature.

How to retrieve build_id of latest successful build in Jenkins?

Generally, to get the artifact of the latest successful build, I do a wget on the below URL:
http://jenkins.com/job/job_name/lastSuccessfulBuild/artifact/artifact1/jenkins.txt
Is there a way, I can do a wget on lastSuccessfulBuild and get a build_id like below?
build_id=`wget http://jenkins.p2pcredit.local/job/job_name/lastSuccessfulBuild`
Yes, there is a way and it is pretty straightforward:
$ build_id=`wget -qO- jenkins_url/job/job_name/lastSuccessfulBuild/buildNumber`
$ echo $build_id
131 # that's my build number
I think the best solution is using groovy with zero dependencies.
node {
script{
def lastSuccessfulBuildID = 0
def build = currentBuild.previousBuild
while (build != null) {
if (build.result == "SUCCESS")
{
lastSuccessfulBuildID = build.id as Integer
break
}
build = build.previousBuild
}
println lastSuccessfulBuildID
}
}
You do not need specify jenkins_url or job_name etc to get last successful build id.
Then you could use it easily in all Jenkinsfile in repositories without useless configurations.
Tested on Jenkins v2.164.2
I find very useful querying permalinks file inside Jenkins workspace.
This allows you, to not only get the last successful build, but also other builds Jenkins considers relevant.
You can see it's content adding this line in Build section, in Execute Shell panel:
cat ../../jobs/$JOB_NAME/builds/permalinks
For example, in my case:
+ cat ../../jobs/$JOB_NAME/builds/permalinks
lastCompletedBuild 56
lastFailedBuild 56
lastStableBuild 51
lastSuccessfulBuild 51
lastUnstableBuild -1
lastUnsuccessfulBuild 56
From there, you would want to parse the number of the last successful build, or any other provided by permalinks, you can do this running:
lastSuccesfulBuildId=$(cat ../../jobs/$JOB_NAME/builds/permalinks | grep lastSuccessfulBuild | sed 's/lastSuccessfulBuild //')
If you want the DisplayName of the last successful job and not build number:
curl --user <username>:<tokenOrPassword> https://<url>/job/<job-name>/lastSuccessfulBuild/api/json | jq -r '.displayName'
Or in groovy
def buildName = Jenkins.instance.getItem('jobName').lastSuccessfulBuild.displayName
Pipeline script solution :
import groovy.json.JsonSlurper
def jResponse = httpRequest "https:/<yourjenkinsjoburlpath>/lastSuccessfulBuild/buildNumber"
def json = new JsonSlurper().parseText(jResponse.content)
echo "Status: ${json}"
jenkins console output:
HttpMethod: GET
URL: https://***/lastSuccessfulBuild/buildNumber
Sending request to url: https://***/lastSuccessfulBuild/buildNumber
Response Code: HTTP/1.1 200 OK
Success code from [100‥399]
[Pipeline] echo
Status: 20
To get the last successful build number using curl:
curl --user userName:password https://url/job/jobName/api/xml?xpath=/*/lastStableBuild/number
to get the job build number simply do:
def build_Number = Jenkins.instance.getItem('JobName').lastSuccessfulBuild.number

Posting to Twitter account using Windows .bat file code

I'm trying to setup an automatic tweet script that runs after a git commit. I'm using Windows 7 have curl available in the command line.
I'm not sure how to set variables with the language windows scripts run and also am not positive about the oauth process.
I have the api key and secret, and also the consumer key and secret but I'm just not sure how to wrap it all together.
Here is a paper clipped mashup of code I'm trying to use as a foundation:
#!/bin/sh
# PATH modification needed for http_post and oauth_sign
export PATH=$PATH:/usr/local/bin
toplevel_path=`git rev-parse --show-toplevel`
toplevel_dir=`basename "$toplevel_path"`
branch=`git rev-parse --abbrev-ref HEAD`
subject=`git log --pretty=format:%s -n1`
hashtags="#code #$toplevel_dir"
tweet=$hashtags' ['$branch']: "'$subject'"'
# truncate tweets that are longer than 140 characters
if [ ${#tweet} -gt 140 ]
then
tweet_trunc=$(echo $tweet | cut -c1-137)
tweet=${tweet_trunc}...
fi
//set vars
consumer_key="mPijnvYpD0sHAY8r*******"
consumer_secret="OWuvuyQeYrT3ToJgyvNdR6baNuDldmTDF5IIJCI************"
access_token="2476143012-ld78CrgnNY3kUmD0QRdvIchXeDC13nO3********"
access_secret="3HTdOlf8jCVzPi5I9usV7rIbGFtM5f****************"
//build oauth
//post data
//example curl code found during research
curl --request 'POST' 'https://api.twitter.com/1.1/statuses/update.json' --header 'Authorization: OAuth oauth_consumer_key="mPijnvYpD0sHAY8r6fkox0KBj", oauth_nonce="OWuvuyQeYrT3ToJgyvNdR6baNuDldmTDF5IIJCIablQbyHA2PS", oauth_signature="Ba6IB8uH2SjtrK8a%2FgZnqCgvIKs%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1346207448", oauth_token="14814762-vvYtBOLX8hBAQ0i0f1k4wxrioG1jOk49MJrqn3myE", oauth_version="1.0"' --verbose -F "media[]=#mack.jpg" -F "status=Test from cURL" --header "Expect: "
Any help at all is appreciated.
Bro, on Windows you should use PowerShell now. .bat is lame!
$toplevel_path = git rev-parse --show-toplevel
$toplevel_dir = Split-Path $toplevel_path -Leaf
$branch = git rev-parse --abbrev-ref HEAD
$subject = git log --pretty=format:%s -n1
$hashtags = "#code #$toplevel_dir"
$tweet = '{0} [{1}]: "{2}"' -f $hashtags, $branch, $subject
if ($tweet.length -gt 140) {
$tweet = $tweet.substring(0,137)
}
$oauths =
'oauth_consumer_key="mPijnvYpD0sHAY8r6fkox0KBj"',
'oauth_nonce="OWuvuyQeYrT3ToJgyvNdR6baNuDldmTDF5IIJCIablQbyHA2PS"',
'oauth_signature="Ba6IB8uH2SjtrK8a%2FgZnqCgvIKs%3D"',
'oauth_signature_method="HMAC-SHA1"',
'oauth_timestamp="1346207448"',
'oauth_token="14814762-vvYtBOLX8hBAQ0i0f1k4wxrioG1jOk49MJrqn3myE"',
'oauth_version="1.0"'
$header = 'Authorization: OAuth {0}' -f ($oauths -join ',')
curl --verbose --request POST -F 'media[]=#mack.jpg' `
-F 'status=Test from cURL' --header 'Expect: ' `
--header $header https://api.twitter.com/1.1/statuses/update.json
command substitution is actually better than with Bash
PowerShell "printf" is cool
arrays are cool
the continuation character ` is too small though, hard to see

Resources