I want to batch Unsign some jars with in gradle but I don't want to use the ant jar method as it is too slow.
Using the 7zip command line is much faster:
7z.exe d activemq-pool-5.7.0.jar META-INF/SIGFILE.*
Where SIGFILE is the name of the previous signature.
I am trying to do it in gradle like this
println "Unsigning jars"
file(unsignedFolder + "/jars").listFiles().each { File file ->
exec {
workingDir '../tools'
commandLine '7z.exe', 'd', file.absolutePath, 'META-INF/SIGFILE.*'
}
}
However, I get the error:
Starting process 'command '7z.exe''. Working directory: D:\code\project\tools Command: 7z.exe d D:\code\project\build\unsigned\jars\activemq-pool-5.7.0.jar META-INF/SIGFILE.*
:signWebstart FAILED
:signWebstart (Thread[Daemon,5,main]) completed. Took 0.109 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':unsignJars'.
> A problem occurred starting process 'command '7z.exe''
Thanks to this post I realised what it should be.
I am using the command line version of 7zip now - 7za. There is also a unix version at http://p7zip.sourceforge.net/ so I am packaging them both with my script and using something along the lines of the following:
import org.apache.tools.ant.taskdefs.condition.Os
task unSignJars() {
if(Os.isFamily(Os.FAMILY_WINDOWS)) {
println "*** WINDOWS "
exec {
executable "7za.exe"
args "d", "temp.jar", "META-INF/SIGN.RSA"
}
} else if(Os.isFamily(Os.FAMILY_UNIX)) {
println "*** UNIX "
exec {
executable "7za"
args "d", "temp.jar", "META-INF/SIGN.RSA"
}
} else {
println "*** NOT SUPPORTED "
}
}
This method is twice as fast as using Java nio http://thinktibits.blogspot.ca/2013/02/Delete-Files-From-ZIP-Archive-Java-Example.html which in itself is twice as fast as the ant method mention in the OP.
import java.util.*;
import java.net.URI;
import java.nio.file.Path;
import java.nio.file.*;
import java.nio.file.StandardCopyOption;
public class ZPFSDelete {
public static void main(String [] args) throws Exception {
/* Define ZIP File System Properies in HashMap */
Map<String, String> zip_properties = new HashMap<>();
/* We want to read an existing ZIP File, so we set this to False */
zip_properties.put("create", "false");
/* Specify the path to the ZIP File that you want to read as a File System */
URI zip_disk = URI.create("jar:file:/my_zip_file.zip");
/* Create ZIP file System */
try (FileSystem zipfs = FileSystems.newFileSystem(zip_disk, zip_properties)) {
/* Get the Path inside ZIP File to delete the ZIP Entry */
Path pathInZipfile = zipfs.getPath("source.sql");
System.out.println("About to delete an entry from ZIP File" + pathInZipfile.toUri() );
/* Execute Delete */
Files.delete(pathInZipfile);
System.out.println("File successfully deleted");
}
}
}
However, unix zip -d is twice as fast again but it isn't portable.
Related
I'm currently trying to get behind an error that only occurs in CI, thrown by a script, called by Gradle Exec with Gradle 7.5.
The issue is, I can't see any error messages in the log as it seems they aren't picked up by Gradle.
For that reason, I've created a small Gradle Plugin located in buildSrc and an .sh script located under /scripts
import org.apache.tools.ant.taskdefs.condition.Os
import org.gradle.api.Plugin
import org.gradle.api.Project
import org.gradle.api.tasks.Exec
class ExecTestPlugin implements Plugin<Project> {
private boolean ENABLE_CUSTOM_OUTPUT = true;
#Override
void apply(Project target) {
target.tasks.register("testExec", Exec) {
group = "other"
workingDir(target.getProjectDir().getAbsolutePath() + File.separator + "scripts")
if (ENABLE_CUSTOM_OUTPUT) {
standardOutput = new ByteArrayOutputStream()
errorOutput = new ByteArrayOutputStream()
doLast {
def result = standardOutput.toString()
println "the result value is: $result"
def error = standardOutput.toString()
println "the error value is: $error"
println "exit value: " + execResult.exitValue
}
}
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
commandLine 'cmd', '/c', "test-failing-script.sh"
} else {
commandLine "sh", "-c", "test-failing-script.sh"
}
if (ENABLE_CUSTOM_OUTPUT) {
ext.output = {
return standardOutput.toString()
}
}
}
}
}
test-failing-script.sh
set -e
set -o pipefail
echo -e This message goes who knows where
echo This message goes to stderr >&2
echo This message goes to stdout >&1
#curl -sf -o /dev/null http://google.com
exit 1;
If the flag is disabled, I'm getting no output at all
If the flag is enabled, I'm getting
the result value is:
the error value is:
exit value: 0
I'm expecting exit value 1 and for both stdout and stderr some messages
Why can't I get the right response message from the called script?
It's some time ago, that I faced this issue. But I think the problem could be with sh -c not forwarding stdout and exit value. But I can't reproduce it right now. Might be it's fixed in a more recent version of Gradle. Try using Gradle 7.5
I'm trying to run 4 test scripts in parallel. I want to store the output.xml file for each test script in the following manner-
Test_1 -> output1.xml
Test_2 -> output2.xml
.
.
Test_4 -> output3.xml
and combine them together and generate one single report.html file
I'm getting some errors related to the output.xml file location.
There is my script in Jenkins to run the robot-
pipeline {
agent any
triggers {
cron('H */4 * * 1-5')
}
stages {
stage('Testing') {
parallel{
stage('Testing- Operator') {
steps {
sh '''robot //var//jenkins_home//workspace//Testing_Operator//Operator.robot
robot archiveDirName:\'robot-plugin\',logFileName: \'**/log*.html\',outputFileName:\'**/output*.xml\',outputPath:\'/var/jenkins_home/workspace/Testing_Pipeline\',overwriteXAxisLabel: \'\',reportFileName:\'**/report*.html\''''
}
}
stage('Testing -Manager') {
steps {
sh '''robot //var//jenkins_home//workspace//Testing_Manger//Manager.robot
robot archiveDirName:\'robot-plugin\',logFileName: \'**/log*.html\',outputFileName:\'**/output*.xml\',outputPath:\'/var/jenkins_home/workspace/Testing_Pipeline\',overwriteXAxisLabel: \'\',reportFileName:\'**/report*.html\''''
}
}
stage('Testing -Admin') {
steps {
sh '''robot //var//jenkins_home//workspace//Testing_Admin//Admin.robot
robot archiveDirName:\'robot-plugin\',logFileName: \'**/log*.html\',outputFileName:\'**/output*.xml\',outputPath:\'/var/jenkins_home/workspace/Testing_Pipeline\',overwriteXAxisLabel: \'\',reportFileName:\'**/report*.html\''''
}
}
stage('Testing -Owner') {
steps {
sh '''robot //var//jenkins_home//workspace//Testing_Owner//Owner.robot
robot archiveDirName:\'robot-plugin\',logFileName: \'**/log*.html\',outputFileName:\'**/output*.xml\',outputPath:\'/var/jenkins_home/workspace/Testing_Pipeline\',overwriteXAxisLabel: \'\',reportFileName:\'**/report*.html\''''
}
}
}
}
}
}
Here is the error-
Output: /var/jenkins_home/workspace/Testing_Pipeline/output.xml
[ ERROR ] Reading XML source '/var/jenkins_home/workspace/Testing_Pipeline/output.xml'
failed: ParseError: not well-formed (invalid token): line xx, column xx
What could be the possible cause of the error? Is the syntax correct?
Probably the output.xml is broken due to running robot in parallel.
What you can do is for all the robot commands to use the flag for a different output path (or filename), and in a final stage call rebot to produce the final report and logs.
In very brief, I want to find all files that ends with *.sql and copy them if they exist.
There might be 0 or more files in etc directory.
File sqlfiles = file('etc/' + '*.sql')
logger.info("Looking for SQL files: " + sqlfiles);
if (sqlfiles.exists())
{
logger.info("Found log SQL file: " + sqlfiles);
copy
{
from sqlfiles
into "$rpmStoredir"
}
}
else
{
logger.warn("No SQL file found - skipping");
}
With my code, the wildcard is not working here.
So adding "include" to the copy as in the below is working but I just want to figure how to add a logger if the file does not exist
copy
{
from "etc/"
include "*.sql"
into "$rpmStoredir"
}
file(...) is the wrong method to use as this returns a single java.io.File
You could do something like
FileTree myTree = fileTree('etc') {
include '*.sql'
}
if (myTree.empty) {
...
} else {
copy {
from myTree
...
}
}
See Project.fileTree(Object, Closure) and FileTree
I am using Gradle-2.11 and I am unable to find a way to create log files that logs debug level information. I don't want to do it through command line by redirecting the logs to the log file. I want Gradle code just like Apache Ant's 'record' task so that I can put that code in my build.gradle file wherever I want to create logs.
For ex: If I want to convert this ant task to gradle, then what would be the code:
<record name="${BuildLogPath}/${BuildLogFile}" append="no" loglevel="verbose" action="start"/>
Gradle integrates really nicely with Ant (https://docs.gradle.org/2.11/userguide/ant.html)
It doesn't automatically record each step. I didn't realize that is what you were asking. The updated below will produce the output and you can manually log.
ant.record(name: "${BuildLogPath}/${BuildLogFile}", append:false, loglevel: "verbose", action: "start")
ant.echo("start logging")
//... do stuff here
ant.echo(message: "end logging")
ant.record(name: "${BuildLogPath}/${BuildLogFile}", append:false, loglevel: "verbose", action: "stop")
This may do more of what you are asking. Note: This is something I adapted slightly from this excellent example:
http://themrsion.blogspot.com/2013/10/gradle-logging-writing-to-log-to-file.html
import org.gradle.logging.internal.*
String currentDate = new Date().format('yyyy-MMM-dd_HH-mm-ss-S')
String loggingDirectory = "${rootDir}/build/logs"
mkdir("${loggingDirectory}")
File gradleBuildLog = new File("${loggingDirectory}/${currentDate}_gradleBuild.log")
gradle.services.get(LoggingOutputInternal).addStandardOutputListener (new StandardOutputListener () {
void onOutput(CharSequence output) {
gradleBuildLog << output
}
})
gradle.services.get(LoggingOutputInternal).addStandardErrorListener (new StandardOutputListener () {
void onOutput(CharSequence output) {
gradleBuildLog << output
}
})
As part of my project, I need to read files from a directory and do some operations all these in build script. For each file, the operation is the same(reading some SQL queries and execute it). I think its a repetitive task and better to write inside a method. Since I'm new to Gradle, I don't know how it should be. Please help.
One approach given below:
ext.myMethod = { param1, param2 ->
// Method body here
}
Note that this gets created for the project scope, ie. globally available for the project, which can be invoked as follows anywhere in the build script using myMethod(p1, p2) which is equivalent to project.myMethod(p1, p2)
The method can be defined under different scopes as well, such as within tasks:
task myTask {
ext.myMethod = { param1, param2 ->
// Method body here
}
doLast {
myMethod(p1, p2) // This will resolve 'myMethod' defined in task
}
}
If you have defined any methods in any other file *.gradle - ext.method() makes it accessible project wide. For example here is a
versioning.gradle
// ext makes method callable project wide
ext.getVersionName = { ->
try {
def branchout = new ByteArrayOutputStream()
exec {
commandLine 'git', 'rev-parse', '--abbrev-ref', 'HEAD'
standardOutput = branchout
}
def branch = branchout.toString().trim()
if (branch.equals("master")) {
def stdout = new ByteArrayOutputStream()
exec {
commandLine 'git', 'describe', '--tags'
standardOutput = stdout
}
return stdout.toString().trim()
} else {
return branch;
}
}
catch (ignored) {
return null;
}
}
build.gradle
task showVersion << {
// Use inherited method
println 'VersionName: ' + getVersionName()
}
Without ext.method() format , the method will only be available within the *.gradle file it is declared. This is the same with properties.
You can define methods in the following way:
// Define an extra property
ext.srcDirName = 'src/java'
// Define a method
def getSrcDir(project) {
return project.file(srcDirName)
}
You can find more details in gradle documentation Chapter 62. Organizing Build Logic
An example with a root object containing methods.
hg.gradle file:
ext.hg = [
cloneOrPull: { source, dest, branch ->
if (!dest.isDirectory())
hg.clone(source, dest, branch)
else
hg.pull(dest)
hg.update(dest, branch)
},
clone: { source, dest, branch ->
dest.mkdirs()
exec {
commandLine 'hg', 'clone', '--noupdate', source, dest.absolutePath
}
},
pull: { dest ->
exec {
workingDir dest.absolutePath
commandLine 'hg', 'pull'
}
},
]
build.gradle file
apply from: 'hg.gradle'
hg.clone('path/to/repo')
Somehow, maybe because it's five years since the OP, but none of the
ext.someMethod = { foo ->
methodBody
}
approaches are working for me. Instead, a simple function definition seems to be getting the job done in my gradle file:
def retrieveEnvvar(String envvar_name) {
if ( System.getenv(envvar_name) == "" ) {
throw new InvalidUserDataException("\n\n\nPlease specify environment variable ${envvar_name}\n")
} else {
return System.getenv(envvar_name)
}
}
And I call it elsewhere in my script with no prefix, ie retrieveEnvvar("APP_PASSWORD")
This is 2020 so I'm using Gradle 6.1.1.
#ether_joe the top-voted answer by #InvisibleArrow above does work however you must define the method you call before you call it - i.e. earlier in the build.gradle file.
You can see an example here. I have used this approach with Gradle 6.5 and it works.
With Kotlin DSL (build.gradle.kts) you can define regular functions and use them.
It doesn't matter whether you define your function before the call site or after it.
println(generateString())
fun generateString(): String {
return "Black Forest"
}
tasks.create("MyTask") {
println(generateString())
}
If you want to import and use a function from another script, see this answer and this answer.
In my react-native in build.gradle
def func_abc(y){return "abc"+y;}
then
def x = func_abc("y");
If you want to check:
throw new GradleException("x="+x);
or
println "x="+x;