Gradle: remove some files from an existing war | for each war-file do: unpack, remove/filter, assemble war - gradle

I'm relatively new to gradle, so this might be a typical newbee-question.
In our gradle build, we have a set of war files (dependencies) that all include a file that we need to remove from those war files, before building our ear file.
How can I achieve the following:
- for all war files in a folder,
- extract war content to a location (using a Copy task & zipTree)
- re-pack to a new war applying a filter (using War & excludes)
I assume I'll create a new task and add some 'dependsOn' declarations.
task excludeUnwantedFiles(){
file(directoryWithOriginalWars).eachFile { file ->
???? unpack war, filter, assemble new war file ????
}
}
ear.dependsOn(excludeUnwantedFiles)
excludeUnwantedFiles.dependsOn(downloadAllOriginalWarsIntoDirectory)
How do I create that task, that executes for each war-file? What is the best way to do this?
Is there a way I can to this in one task? E.g. with the Copy task and using zipTree(fooBarWithFile.war) as 'from' and 'war(fooBarWithoutFile.war)' and applying a filter in between?
Or is this the way to go, just with a loop? Delete/Remove file from war with Gradle
Any help is highly appreciated!
Cheers,
d.
---------UPDATE-------------------
Thanx Lance Java for your solution.
As I mentioned in my comment, I faced the problem that the war files were downloaded/extracted during execution time and hence not accessable to define the new tasks at configuration time.
My workaround for this is to use tarTree (with a filter) to access the list of war-files that are not yet extracted. See my code example below:
def warFileSourceTarGz = '...tar.gz'
def nfsLibDir="$buildDir/dependencies/"
def nfsLibDownloadDir="$buildDir/downloadedDependencies/"
// task that downloads & extracts the tar.gz
task fetchNfsDependencies(type: Copy) {
from tarTree(warFileSourceTarGz)
into nfsLibDownloadDir
}
// loop through all war files inside the tar.gz and
// create a task to remove unwanted libraries for each war
task excludeUnwantedJarsFromWars(dependsOn: fetchNfsDependencies){
// access the "remote" tar.gz file to get the list of war-files to loop over
def warFileSource = tarTree(warFileSourceTarGz).matching{
include '*.war'
}
// for every war-file, create an exclude-path
warFileSource.visit { nextWarFile ->
if(nextWarFile.name.endsWith('.war')) {
String taskName = "excludeUnwantedJarsFrom_${nextWarFile.name.replace('.war', '')}"
String nextWarFilePath = nfsLibDownloadDir+"/"+nextWarFile.name
Zip tweakWarTask = tasks.create(name: taskName, type: Zip, dependsOn: fetchNfsDependencies) {
from zipTree(nextWarFilePath)
destinationDir = file(nfsLibDir)
archiveName = nextWarFile.name
// exclude these jars, as they cause classloading problems in our ear deployment.
exclude 'WEB-INF/lib/jcan-optrace*'
}
// hook into build-process
ear.dependsOn(tweakWarTask)
}
}
}
ear.dependsOn(excludeUnwantedJarsFromWars)

I'd create a Zip task per war file and wire all the tasks into the DAG by having assemble depend on them all
FileTree warFiles = fileTree(dir: 'path/to/wars', includes: ['**/*.war'])
warFiles.files.each { File warFile ->
String taskName = "tweakWar${warFile.name.replace('.war', '')}"
Zip tweakWarTask = tasks.create(name: taskName, type: Zip) {
from zipTree(warFile) {
exclude 'path/to/some/file.xml'
}
destinationDir = "$buildDir/tweakedWars"
archiveName = warFile.name
}
// wire the task into the dag
assemble.dependsOn tweakWarTask
}

Related

Extracting single jar inside zip using gradle task

I’m trying to extract a jar from a zip file which is available inside the zip as below structure.
test.zip /web-inf /lib /result1.jar
Here is the task:-
task unzip(type: Copy){
group "testgroup"
def zipFile = file("$buildDir/test.zip")
def tree = zipTree(zipFile)
from(tree.matching { include "**/result1.jar"})
into "$builddir/dir"
includeEmptyDirs = false
}
But I am getting jar file with folder structure like /web-inf/lib/result1.jar. I want jar file alone, not the folders (/web-inf/lib/).
Please correct what is wrong in the gradle task.
Gradle Copy DSL provides a method eachFile (Closure closure) that you can use to perform action on each file which will be copied/extracted, see Copy DSL
In your case, you can use this method to rename (remove parent dirs from path) your file:
task unzip(type: Copy) {
group "testgroup"
from( zipTree(file("test.zip")).matching { include "**/result1.jar" } )
into "$buildDir/dir"
def parentDir = "web-inf/lib"
eachFile { fcp ->
fcp.path = fcp.path.replaceFirst("^$parentDir", '')
}
includeEmptyDirs = false
}

How to include all files from jar into war using Gradle

I was trying plugins:
io.freefair.war-overlay - but it just overlay one war to another,
waroverlay - it has appropriate option "includeWarJars true" but it
does't works for me.
Currently I'm trying write script:
dependencies {
/* jar, his files I would like to include in war */
compile project(':my_jar_1')
}
war {
/* Step 1. Copy all from my_jar_1 into WEB-INF/classes */
into('WEB-INF/classes') {
from configurations.compile
.filter { it.name.startsWith("my_jar_1") }
.collect { zipTree(it).matching { exclude 'META-INF/**/*.*' }}
}
/* Step 2. Deleting jar from war WEB-INF/lib. I got stuck here. It println, but doesn't delete jar */
doLast {
zipTree('build/libs/war_1-0.0.0.war').files.each {
if (it.path.endsWith('.jar')) {
delete it
println 'DELETED ' + it.path
}
}
}
}
Could somebody tell me how to make it work?
Or maybe smb know more elegant solution?
Also I was trying to declare my own configuration
configurations { overlay }
dependencies {
overlay project(':my_jar_1')
}
war {
into('WEB-INF/classes') {
from configurations.overlay
...
But it shows error
FAILURE: Build failed with an exception.
What went wrong: Failed to capture snapshot of input files for task 'war' property 'rootSpec$1$1' during up-to-date check.
Failed to create MD5 hash for file '/home/user/projects/OveralJarToWar/my_jar_1/build/libs/my_jar_1-1.0-SNAPSHOT.jar'.
The content of WEB-INF/lib and WEB-INF/classes is configured by single property classpath of war task. According to documentation:
Any JAR or ZIP files in this classpath are included in the WEB-INF/lib directory. Any directories in this classpath are included in the WEB-INF/classes directory
So, in your case, the classpath should be modified as follow
war {
def myJar = project(':my_jar_1').jar.outputs
def myClassesAndResources = project(':my_jar_1').sourceSets.main.output
classpath = classpath - myJar + myClassesAndResources
}

gradle - create jar iteratively under a certain directory

I am trying to create multiple jars. I have a certain directory which contains multiple directory. Each directory has its own files(xml and sql). So I am trying to create a jar with a subdirectory name and all the files in it. Those jars will be used for junit test, so I want to create it during configuration phase in advance.
The target directory "../dist" exists outside this project.
ext.createTemplateJar = { sourceDirectory, jarFileName ->
jar {
archiveName = jarFileName
from sourceDirectory
includeEmptyDirs = false
manifest
{
attributes 'Implementation-Title' : VENDOR_NAME
}
}
println "creating a jar for ${jarFileName}"
}
tasks.withType(Jar) {
destinationDir = file("../dist")
}
task generateTemplates {
new File("${projectDir}/templates").eachFile() { file ->
if (file.isFile()) {
return
}
println "template dir is ${file.path}"
createTemplateJar(file.path, "${file.name}.jar")
}
}
When I execute this gradle generateTemplates, it runs fine, but I don't see any jar files created in the destination directory.
It seems something wrong, but I can't tell.
You should create multiple Jar tasks, one for each subdirectory inside ${projectDir}/templates, and have your generateTemplates task do nothing but depend on all of those tasks. For example (I renamed generateTemplates to allJars):
task allJars {
}
file("${projectDir}/templates").eachFile { f ->
def taskName = "jar${f.name.capitalize()}"
tasks.create(name: taskName, type: Jar) {
archiveName = "${f.name}.jar"
destinationDir = file('../dist')
// Configure each JAR however you want
}
allJars.dependsOn taskName
}
Example build:
$ ls templates/
bar baz foo test
$ ./gradlew clean allJars
:clean
:jarBar
:jarBaz
:jarFoo
:jarTest
:allJars
BUILD SUCCESSFUL
Total time: 0.615 secs
$ ls ../dist/
bar.jar baz.jar foo.jar test.jar

how to download external files in gradle?

I have a gradle project which requires some data files available somewhere on the internet using http. The goal is that this immutable remote file is pulled once upon first build. Subsequent build should not download again.
How can I instruct gradle to fetch the given file to a local directory?
I've tried
task fetch(type:Copy) {
from 'http://<myurl>'
into 'data'
}
but it seems that copy task type cannot deal with http.
Bonus question: is there a way to resume a previously aborted/interrupted download just like wget -c does?
How about just:
def f = new File('the file path')
if (!f.exists()) {
new URL('the url').withInputStream{ i -> f.withOutputStream{ it << i }}
}
You could probably use the Ant task Get for this. I believe this Ant task does not support resuming a download.
In order to do so, you can create a custom task with name MyDownload. That can be any class name basically. This custom task defines inputs and outputs that determine whether the task need to be executed. For example if the file was already downloaded to the specified directory then the task is marked UP-TO-DATE. Internally, this custom task uses the Ant task Get via the built-in AntBuilder.
With this custom task in place, you can create a new enhanced task of type MyDownload (your custom task class). This task set the input and output properties. If you want this task to be executed, hook it up to the task you usually run via task dependencies (dependsOn method). The following code snippet should give you the idea:
task downloadSomething(type: MyDownload) {
sourceUrl = 'http://www.someurl.com/my.zip'
target = new File('data')
}
someOtherTask.dependsOn downloadSomething
class MyDownload extends DefaultTask {
#Input
String sourceUrl
#OutputFile
File target
#TaskAction
void download() {
ant.get(src: sourceUrl, dest: target)
}
}
Try like that:
plugins {
id "de.undercouch.download" version "1.2"
}
apply plugin: 'java'
apply plugin: 'de.undercouch.download'
import de.undercouch.gradle.tasks.download.Download
task downloadFile(type: Download) {
src 'http://localhost:8081/example/test-jar-test_1.jar'
dest 'localDir'
}
You can check more here: https://github.com/michel-kraemer/gradle-download-task
For me works fine..
The suggestion in Ben Manes's comment has the advantage that it can take advantage of maven coordinates and maven dependency resolution. For example, for downloading a Derby jar:
Define a new configuration:
configurations {
derby
}
In the dependencies section, add a line for the custom configuration
dependencies {
derby "org.apache.derby:derby:10.12.1.1"
}
Then you can add a task which will pull down the right files when needed (while taking advantage of the maven cache):
task deployDependencies() << {
String derbyDir = "${some.dir}/derby"
new File(derbyDir).mkdirs();
configurations.derby.resolve().each { file ->
//Copy the file to the desired location
copy {
from file
into derbyDir
// Strip off version numbers
rename '(.+)-[\\.0-9]+\\.(.+)', '$1.$2'
}
}
}
(I learned this from https://jiraaya.wordpress.com/2014/06/05/download-non-jar-dependency-in-gradle/).
Using following plugin:
plugins {
id "de.undercouch.download" version "3.4.3"
}
For a task which has the purpose of only downloading
task downloadFile(type: Download) {
src DownloadURL
dest destDir
}
For including download option into your task:
download {
src DownloadURL
dest destDir
}
For including download option with multiple downloads into your task:
task downloadFromURLs(){
download {
src ([
DownloadURL1,
DownloadURL2,
DownloadURL3
])
dest destDir
}
}
Hope it helped :)
just now ran into post on upcoming download task on gradle forum.
Looks like the perfect solution to me.. Not (yet) available in an official gradle release though
Kotlin Version of #Benjamin's Answer
buildscript {
repositories {
jcenter()
google()
}
dependencies {
classpath("com.android.tools.build:gradle:4.0.1")
}
}
tasks.register("downloadPdf"){
val path = "myfile.pdf"
val sourceUrl = "https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf"
download(sourceUrl,path)
}
fun download(url : String, path : String){
val destFile = File(path)
ant.invokeMethod("get", mapOf("src" to url, "dest" to destFile))
}

Extract specific JARs from dependencies

I am new to gradle but learning quickly. I need to get some specific JARs from logback into a new directory in my release task. The dependencies are resolving OK, but I can't figure out how, in the release task, to extract just logback-core-1.0.6.jar and logback-access-1.0.6.jar into a directory called 'lib/ext'. Here are the relevant excerpts from my build.gradle.
dependencies {
...
compile 'org.slf4j:slf4j-api:1.6.4'
compile 'ch.qos.logback:logback-core:1.0.6'
compile 'ch.qos.logback:logback-classic:1.0.6'
runtime 'ch.qos.logback:logback-access:1.0.6'
...
}
...
task release(type: Tar, dependsOn: war) {
extension = "tar.gz"
classifier = project.classifier
compression = Compression.GZIP
into('lib') {
from configurations.release.files
from configurations.providedCompile.files
}
into('lib/ext') {
// TODO: Right here I want to extract just logback-core-1.0.6.jar and logback-access-1.0.6.jar
}
...
}
How do I iterated over the dependencies to locate those specific files and drop them in the lib/ext directory created by into('lib/ext')?
Configurations are just (lazy) collections. You can iterate over them, filter them, etc. Note that you typically only want to do this in the execution phase of the build, not in the configuration phase. The code below achieves this by using the lazy FileCollection.filter() method. Another approach would have been to pass a closure to the Tar.from() method.
task release(type: Tar, dependsOn: war) {
...
into('lib/ext') {
from findJar('logback-core')
from findJar('logback-access')
}
}
def findJar(prefix) {
configurations.runtime.filter { it.name.startsWith(prefix) }
}
It is worth nothing that the accepted answer filters the Configuration as a FileCollection so within the collection you can only access the attributes of a file. If you want to filter on the dependency itself (on group, name, or version) rather than its filename in the cache then you can use something like:
task copyToLib(type: Copy) {
from findJarsByGroup(configurations.compile, 'org.apache.avro')
into "$buildSrc/lib"
}
def findJarsByGroup(Configuration config, groupName) {
configurations.compile.files { it.group.equals(groupName) }
}
files takes a dependencySpecClosure which is just a filter function on a Dependency, see: https://gradle.org/docs/current/javadoc/org/gradle/api/artifacts/Dependency.html

Resources