I'm currently trying to get behind an error that only occurs in CI, thrown by a script, called by Gradle Exec with Gradle 7.5.
The issue is, I can't see any error messages in the log as it seems they aren't picked up by Gradle.
For that reason, I've created a small Gradle Plugin located in buildSrc and an .sh script located under /scripts
import org.apache.tools.ant.taskdefs.condition.Os
import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.tasks.Exec
class ExecTestPlugin implements Plugin<Project> {
private boolean ENABLE_CUSTOM_OUTPUT = true;
#Override
void apply(Project target) {
target.tasks.register("testExec", Exec) {
group = "other"
workingDir(target.getProjectDir().getAbsolutePath() + File.separator + "scripts")
if (ENABLE_CUSTOM_OUTPUT) {
standardOutput = new ByteArrayOutputStream()
errorOutput = new ByteArrayOutputStream()
doLast {
def result = standardOutput.toString()
println "the result value is: $result"
def error = standardOutput.toString()
println "the error value is: $error"
println "exit value: " + execResult.exitValue
}
}
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
commandLine 'cmd', '/c', "test-failing-script.sh"
} else {
commandLine "sh", "-c", "test-failing-script.sh"
}
if (ENABLE_CUSTOM_OUTPUT) {
ext.output = {
return standardOutput.toString()
}
}
}
}
}
test-failing-script.sh
set -e
set -o pipefail
echo -e This message goes who knows where
echo This message goes to stderr >&2
echo This message goes to stdout >&1
#curl -sf -o /dev/null http://google.com
exit 1;
If the flag is disabled, I'm getting no output at all
If the flag is enabled, I'm getting
the result value is:
the error value is:
exit value: 0
I'm expecting exit value 1 and for both stdout and stderr some messages
Why can't I get the right response message from the called script?
It's some time ago, that I faced this issue. But I think the problem could be with sh -c not forwarding stdout and exit value. But I can't reproduce it right now. Might be it's fixed in a more recent version of Gradle. Try using Gradle 7.5
Related
I am getting runtime value in build stage stage which I stored in an environment variable . I saved that to env.cfg file under WORKSPACE .
Now I am trying to get that value in post pipeline step to be used in email communication. I tried load method but it did not work
Any help ?
post {
always {
echo $SNAPSHOT / /this always comes null
}
}
This is the way you can access an environment variable across the pipeline
pipeline {
agent any;
environment {
MESSAGE="Hello World"
}
stages {
stage('one') {
steps {
echo "${env.MESSAGE}"
sh "echo $MESSAGE"
script {
print env.MESSAGE
}
}
}
}
post {
success {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
failure {
echo "${env.MESSAGE}"
script {
print env.MESSAGE
}
}
}
}
but as per your scenario let say I have a file called .env with the content below in the current Jenkins job WORKSPACE and I want to read and make this env variable in the pipeline.
.env
export SNAPSHOT=1.0.0
export MESSAGE='Hello World'
export MESSAGE_FROM_ENV_FILE='Hello From .env file'
your pipeline should look like
scripted pipeline
node {
stage('one') {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
declarative pipeline
pipeline {
agent any;
stages {
stage('build') {
steps {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
post {
success {
sh """
source $WORKSPACE/.env
echo \$SNAPSHOT
echo \$MESSAGE
echo \$MESSAGE_FROM_ENV_FILE
"""
}
}
}
You need a global variable:
SNAPSHOT = ""
println "SNAPSHOT is ${SNAPSHOT}"
pipeline {
agent any
stages {
stage('Build') {
steps {
script {
println "SNAPSHOT is ${SNAPSHOT}"
SNAPSHOT = "Modified"
println "SNAPSHOT is now ${SNAPSHOT}"
}
}
}
}
post {
always {
echo "${SNAPSHOT}"
}
}
}
Usecase: I want to send jenkins job console log to elasticsearch, from there to kibana so that i can visualise the data.
I am using logstash plugin to achieve this. For freestyle job logstash plugin configuration is working fine but for jenkins pipeline jobs I am getting all required data like build number, job name, build duration and all but it is not showing the build result i.e., success or failure it is not showing.
I tried in two ways:
1.
stage('send to ES') {
logstashSend failBuild: true, maxLines: -1
}
2.
timestamps {
logstash {
node() {
sh'''
echo 'Hello, World!'
'''
try {
stage('GitSCM')
{
git url: 'github repo.git'
}
stage('Initialize')
{
jdk = tool name: 'jdk'
env.JAVA_HOME = "${jdk}"
echo "jdk installation path is: ${jdk}"
sh "${jdk}/bin/java -version"
sh '$JAVA_HOME/bin/java -version'
def mvnHome = tool 'mvn'
}
stage('Build Stage')
{
def mvnHome = tool 'mvn'
sh "${mvnHome}/bin/mvn -B verify"
}
currentBuild.result = 'SUCCESS'
} catch (Exception err) {
currentBuild.result = 'FAILURE'
}
}
}
}
But in both ways I am not getting build result i.e., success or failure in my elasticsearch or kibana.
Can someone help.
I didn't find a clear way to do that, my solution was add those lines at the end of the Jenkinsfile:
echo "Current result: ${currentBuild.currentResult}"
logstashSend failBuild: true, maxLines: 3
In my case, I dont need it to send all console logs, only one log with the result per job.
What is the syntax of 'post' in scripted pipeline comparing to declarative pipeline?
https://jenkins.io/doc/book/pipeline/syntax/#post
For scripted pipeline, everything must be written programmatically and most of the work is done in the finally block:
Jenkinsfile (Scripted Pipeline):
node {
try {
stage('Test') {
sh 'echo "Fail!"; exit 1'
}
echo 'This will run only if successful'
} catch (e) {
echo 'This will run only if failed'
// Since we're catching the exception in order to report on it,
// we need to re-throw it, to ensure that the build is marked as failed
throw e
} finally {
def currentResult = currentBuild.result ?: 'SUCCESS'
if (currentResult == 'UNSTABLE') {
echo 'This will run only if the run was marked as unstable'
}
def previousResult = currentBuild.getPreviousBuild()?.result
if (previousResult != null && previousResult != currentResult) {
echo 'This will run only if the state of the Pipeline has changed'
echo 'For example, if the Pipeline was previously failing but is now successful'
}
echo 'This will always run'
}
}
https://jenkins.io/doc/pipeline/tour/running-multiple-steps/#finishing-up
You can modify #jf2010 solution so that it looks a little neater (in my opinion)
try {
pipeline()
} catch (e) {
postFailure(e)
} finally {
postAlways()
}
def pipeline(){
stage('Test') {
sh 'echo "Fail!"; exit 1'
}
println 'This will run only if successful'
}
def postFailure(e) {
println "Failed because of $e"
println 'This will run only if failed'
}
def postAlways() {
println 'This will always run'
}
I have a build.gradle as follows
task setDockerHost {
group 'Docker'
description 'Gets the docker host ip from your OS'
def stdout = new ByteArrayOutputStream()
exec {
commandLine './src/main/resources/scripts/get-docker-host.sh', '60'
standardOutput = stdout
}
project.ext.set('DOCKERHOST', "$stdout")
}
tasks.withType(Test) {
doFirst { println "DockerHost is $project.DOCKERHOST" }
environment 'DOCKERHOST', "$project.DOCKERHOST"
outputs.upToDateWhen { false }
testLogging {
events 'passed', 'skipped', 'failed', 'standardOut'
}
reports.html.destination = file("${reporting.baseDir}/${name}")
}
I define a DOCKERHOST env variable as above and want to use in my groovy test class:
class MyClass extends Specification {
RESTClient client = new RESTClient("http://"+System.getenv('DOCKERHOST')+":9090/")
...
}
In the execution my println works: DockerHost is 192.168.99.100
But when I run the this test it throws:
I already tried replacing \n, \r and spaces by "". I also try removing the protocol from the URL (aka 192.168.99.10:9090) and it tells me that the same error occurs at index 0
How can I solve this?
I didn't figure it out in which char was the problem but I was able to solve it but replacing strings like crazy:
String url = ("http://" + System.getenv('DOCKERHOST') + ":9090/").replace("\n", "").replace("\r", "").replace(" ", "")
RESTClient client = new RESTClient(url)
I've spent like a day trying to figure this out... hopefully for someone else will be quicker.
I want to batch Unsign some jars with in gradle but I don't want to use the ant jar method as it is too slow.
Using the 7zip command line is much faster:
7z.exe d activemq-pool-5.7.0.jar META-INF/SIGFILE.*
Where SIGFILE is the name of the previous signature.
I am trying to do it in gradle like this
println "Unsigning jars"
file(unsignedFolder + "/jars").listFiles().each { File file ->
exec {
workingDir '../tools'
commandLine '7z.exe', 'd', file.absolutePath, 'META-INF/SIGFILE.*'
}
}
However, I get the error:
Starting process 'command '7z.exe''. Working directory: D:\code\project\tools Command: 7z.exe d D:\code\project\build\unsigned\jars\activemq-pool-5.7.0.jar META-INF/SIGFILE.*
:signWebstart FAILED
:signWebstart (Thread[Daemon,5,main]) completed. Took 0.109 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':unsignJars'.
> A problem occurred starting process 'command '7z.exe''
Thanks to this post I realised what it should be.
I am using the command line version of 7zip now - 7za. There is also a unix version at http://p7zip.sourceforge.net/ so I am packaging them both with my script and using something along the lines of the following:
import org.apache.tools.ant.taskdefs.condition.Os
task unSignJars() {
if(Os.isFamily(Os.FAMILY_WINDOWS)) {
println "*** WINDOWS "
exec {
executable "7za.exe"
args "d", "temp.jar", "META-INF/SIGN.RSA"
}
} else if(Os.isFamily(Os.FAMILY_UNIX)) {
println "*** UNIX "
exec {
executable "7za"
args "d", "temp.jar", "META-INF/SIGN.RSA"
}
} else {
println "*** NOT SUPPORTED "
}
}
This method is twice as fast as using Java nio http://thinktibits.blogspot.ca/2013/02/Delete-Files-From-ZIP-Archive-Java-Example.html which in itself is twice as fast as the ant method mention in the OP.
import java.util.*;
import java.net.URI;
import java.nio.file.Path;
import java.nio.file.*;
import java.nio.file.StandardCopyOption;
public class ZPFSDelete {
public static void main(String [] args) throws Exception {
/* Define ZIP File System Properies in HashMap */
Map<String, String> zip_properties = new HashMap<>();
/* We want to read an existing ZIP File, so we set this to False */
zip_properties.put("create", "false");
/* Specify the path to the ZIP File that you want to read as a File System */
URI zip_disk = URI.create("jar:file:/my_zip_file.zip");
/* Create ZIP file System */
try (FileSystem zipfs = FileSystems.newFileSystem(zip_disk, zip_properties)) {
/* Get the Path inside ZIP File to delete the ZIP Entry */
Path pathInZipfile = zipfs.getPath("source.sql");
System.out.println("About to delete an entry from ZIP File" + pathInZipfile.toUri() );
/* Execute Delete */
Files.delete(pathInZipfile);
System.out.println("File successfully deleted");
}
}
}
However, unix zip -d is twice as fast again but it isn't portable.