How to configure jenkins pipeline with logstash plugin? - elasticsearch

Usecase: I want to send jenkins job console log to elasticsearch, from there to kibana so that i can visualise the data.
I am using logstash plugin to achieve this. For freestyle job logstash plugin configuration is working fine but for jenkins pipeline jobs I am getting all required data like build number, job name, build duration and all but it is not showing the build result i.e., success or failure it is not showing.
I tried in two ways:
1.
stage('send to ES') {
logstashSend failBuild: true, maxLines: -1
}
2.
timestamps {
logstash {
node() {
sh'''
echo 'Hello, World!'
'''
try {
stage('GitSCM')
{
git url: 'github repo.git'
}
stage('Initialize')
{
jdk = tool name: 'jdk'
env.JAVA_HOME = "${jdk}"
echo "jdk installation path is: ${jdk}"
sh "${jdk}/bin/java -version"
sh '$JAVA_HOME/bin/java -version'
def mvnHome = tool 'mvn'
}
stage('Build Stage')
{
def mvnHome = tool 'mvn'
sh "${mvnHome}/bin/mvn -B verify"
}
currentBuild.result = 'SUCCESS'
} catch (Exception err) {
currentBuild.result = 'FAILURE'
}
}
}
}
But in both ways I am not getting build result i.e., success or failure in my elasticsearch or kibana.
Can someone help.

I didn't find a clear way to do that, my solution was add those lines at the end of the Jenkinsfile:
echo "Current result: ${currentBuild.currentResult}"
logstashSend failBuild: true, maxLines: 3
In my case, I dont need it to send all console logs, only one log with the result per job.

Related

Jenkins function of shared library executed on wrong node

I have a simple shared library and want to call a function which creates a file.
My pipeline script looks like this:
#Library('jenkins-shared-library')_
pipeline {
agent {
node {
label 'node1'
}
}
stages {
stage('test') {
steps {
script {
def host = sh(script: 'hostname', returnStdout: true).trim()
echo "Hostname is: ${host}"
def testlib = new TestLibrary(script:this)
testlib.createFile("/tmp/file1")
}
}
}
}
}
This pipeline job is triggered by another job which runs on the built-in master node.
The stage 'test' is correctly executed on the 'node1'.
Problem: The created file "/tmp/file1" is created on the jenkins master, instead of "node1"
I also tried it without the shared library and load a groovy script
directly in a step:
pipeline {
agent {
node {
label 'node1'
}
}
stages {
stage('test') {
steps {
script {
def script = load "/path/to/script.groovy"
script.createFile("/tmp/file1")
}
}
}
}
}
This also creates the file on the master node, instead on "node1".
Is there no way of loading external libs or classes and execute them on the node where the stage is running on ? I dont want to place all the code directly into the step.
Ok I found it myself, groovy scripts run by definition always on the master node. No way to run scripts or shared-library code on a node other than master.

parameterized-remote-trigger is throwing 405 exception

I am trying to trigger a job A(this is configured as trigger remote) remotely from another job B, and job B needs to hold until results come back to show success or failure, I initially tried using rest API using curl command, it perfectly works.here's the curl code:
curl -v -X POST 'https://xxx.xxx/xxx-xxx/job/xxx/job/master/buildWithParameters?config_files=./jenkins/unit-tests.json' --user xxxx:110f4dfa33ba8f8ef5d8d299beb6aa1543
I choose parameterized plugin code which installed on Jenkins server because it handles the polling mechanism internally and also has handler friendly methods. please see below code for remoteJob, but it fails with 405 error, that means method not allowed in HTTP language, looks like plugin is using GET method instead of post. I added an option for logging , but it does not seems to be showing more log.
def handle = triggerRemoteJob(
remoteJenkinsName: 'remote-master',
job: 'https://xxx.xxx.com/xxx-xxx/job/xxx/job/master/buildWithParameters',
remoteJenkinsUrl: 'https://xxx.xxx.xxx/xxx-xxx/job/xxx/job/master/buildWithParameters',
auth: TokenAuth(apiToken: hudson.util.Secret.fromString('110f4dfa33ba8f8ef5d8d299beb6aa1543'), userName: 'xxxx'),
parameters: 'config_files=./jenkins/unit-tests')
I am getting following error -
[Pipeline] triggerRemoteJob
##########################################################################
Parameterized Remote Trigger Configuration:
- job: https://xxx.xxx.xxx/xxx-xxx/job/xxx/job/master/buildWithParameters
- remoteJenkinsUrl: https://xxx.xxx.xxx/xxx-xxx/job/ius/job/master/buildWithParameters
- auth: 'Token Authentication' as user 'sseri'
- parameters: [config_files=./jenkins/unit-tests]
- blockBuildUntilComplete: true
- connectionRetryLimit: 5
- trustAllCertificates: false
##########################################################################
Connection to remote server failed [405], waiting to retry - 10 seconds until next attempt. URL: https://xxx.xxx.xxx/xxx-xxx/job/xxx/job/master/buildWithParameters/api/json, parameters:
Retry attempt #1 out of 5
Please help me in this regard!
I am not sure about the plugins you are using, but it's quite simple to implement this scenario "call a downstream job from upstream and fail upstream if the downstream fail" without any plugins.
Take a look at my example below.
let say if you have 2 jobs called jobA and jobB and your goal is to call jobB from jobA and fail the jobA if jobB fail.
**Scripted Pipeline for jobA **
node() {
try {
def jobB = build(job: jobName,parameters: [string(name:"parameterName",value: "parameterValue")])
def jobBStatus = jobB.getResult()
if(jobBStatus == "failed") {
throw new RuntimeException("Downstream job-b failed with reason ...");
}
...
}catch(Exception e) {
throw e
}
}
Declarative Pipeline for jobA
pipeline {
agent any;
stages {
stage('call jobB') {
steps {
script {
def jobB = build(job: jobName,parameters: [
string(name:"parameterName",value: "parameterValue")
])
def jobBStatus = jobB.getResult()
if(jobBStatus == "failed") {
error("Downstream job-b failed with reason ...")
}
}
}
}
}
}
Try using this Parameterized-Remote-Trigger-Plugin. It should give you what you want. I'm having some problems configuring it using authentication tokens and users using Jenkinsfile but if you are using the GUI im sure you will get the job done.

How to trigger a multiple run in a single pipeline job of jenkins?

I have a pipeline job which run with below pipeline groovy script,
pipeline {
parameters{
string(name: 'Unique_Number', defaultValue: '', description: 'Enter Unique Number')
}
stages {
stage('Build') {
agent { node { label 'Build' } }
steps {
script {
sh build.sh
}
}
stage('Deploy') {
agent { node { label 'Deploy' } }
steps {
script {
sh deploy.sh
}
}
stage('Test') {
agent { node { label 'Test' } }
steps {
script {
sh test.sh
}
}
}
}
I just trigger this job multiple times with different unique ID number as input parameter. So as a result i will have multiple run/build for this job at different stages.
With this, i need to trigger a multiple run/build to be promote to next stage (i.e., from build to deploy or from deploy to test) in this pipeline job as a one single build instead of triggering each and every single run/build to next stage. Is there any possibility?
I was also trying to do the same thing and found no relevant answers. May this help to someone.
This will read a file that contains the Jenkins Job name and run them iteratively from one single job.
Please change below code accordingly in your Jenkins.
pipeline {
agent any
stages {
stage('Hello') {
steps {
script{
git branch: 'Your Branch name', credentialsId: 'Your crendiatails', url: ' Your BitBucket Repo URL '
##To read file from workspace which will contain the Jenkins Job Name ###
def filePath = readFile "${WORKSPACE}/ Your File Location"
##To read file line by line ###
def lines = filePath.readLines()
##To iterate and run Jenkins Jobs one by one ####
for (line in lines) {
build(job: "$line/branchName",
parameters:
[string(name: 'vertical', value: "${params.vertical}"),
string(name: 'environment', value: "${params.environment}"),
string(name: 'branch', value: "${params.aerdevops_branch}"),
string(name: 'project', value: "${params.host_project}")
]
)
}
}
}
}
}
}
You can start multiple jobs from one pipeline if you run something as:
build job:"One", wait: false
build job:"Two", wait: false
Your main job starts children pipelines and children pipelines will run in parallel.
You can read PipeLine Build Step documentation for more information.
Also, you can read about the parallel run in declarative pipeline
Here you can find a lot of examples for parallel running

How to ask user input only when branch matches master in Jenkinsfile pipeline?

I tried to use the following Jenkinsfile to ask input only on master branch and can't get the groovy grammer pass validation:
pipeline {
stage('Deploy') {
when {
branch 'master'
}
steps {
input(message: "Please input", parameters: [string(name: "VERSION", defaultValue="", description="")]
}
}
}
The error is:
java.lang.IllegalArgumentException: Expected named arguments but got [{name=VERSION, description=""}, null]
I searched a lot but didn't find a single one example about using input step in the Jenkinsfile with parameters.
Can anyone shed some light on this? Thanks in advance!
Finally find a way to do this, we should wrap the input with the script keyword.
stage('Deploy to k8s prod') {
when {
branch 'release-prod'
}
steps {
script {
env.VERSION = input message: '请输入参数', ok: '确定', parameters: [
string(name:'VERSION', description: "版本号")]
}
sh "echo ${env.VERSION}"
}
}
}

How to define jenkins build trigger in jenkinsfile to start build after other job

I would like to define a build trigger in my Jenkinsfile. I know how to do it for the BuildDiscarderProperty:
properties([[$class: 'jenkins.model.BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '50', artifactNumToKeepStr: '20']]])
How can I set the Build Trigger that starts the job, when another project has been built. I cannot find a suitable entry in the Java API docs.
Edit:
My solution is to use the following code:
stage('Build Agent'){
if (env.BRANCH_NAME == 'develop') {
try {
// try to start subsequent job, but don't wait for it to finish
build job: '../Agent/develop', wait: false
} catch(Exception ex) {
echo "An error occurred while building the agent."
}
}
if (env.BRANCH_NAME == 'master') {
// start subsequent job and wait for it to finish
build '../Agent/master', wait: true
}
}
I just looked for the same thing and found this Jenkinsfilein jenkins-infra/jenkins.io
In short:
properties([
pipelineTriggers([cron('H/30 * * * *')])
])
This is an example:
#Project test1
pipeline {
agent {
any
}
stages {
stage('hello') {
steps {
container('dind') {
sh """
echo "Hello world!"
"""
}
}
}
}
post {
success{
build propagate: false, job: 'test2'
}
}
}
post {} will be execute when project test1 is built and the code inside
success {} will only be executed when project test1 is successful.
build propagate: false, job: 'test2' will call project test2.
propogate: false ensures that project test1 does not wait for project test2's
completion and simply invokes it.

Resources