I've got this project structure
journey\
applications\
desktop\
core\
I've got a task that downloads .zip archive.
task download_redis(type: Download) {
src 'https://github.com/MSOpenTech/redis/releases/download/win-3.2.100/Redis-x64-3.2.100.zip'
dest new File("applications/redis/install", 'Redis-x64-3.2.100.zip')
onlyIfNewer true
}
It saves zip file into applications\redis\install.
Then I've got another task that supposed to extract the content of zip file into applications/redis.
task install_redis(dependsOn: download_redis, type: Copy) {
from zipTree(download_redis.dest)
into 'applications/redis'
}
But I get this error
Cannot expand ZIP 'L:\journey\desktop\applications\redis\install\Redis-x64-3.2.100.zip' as it does not exist.
It seems trying to resolve the path to zip from desktop project, despite it being level higher - a root.
How to set for zipTree to use path from root and not from subproject? (Though both these tasks are inside desktop subproject)
You can access various paths like so:
println project(":applications").projectDir
println project(":desktop").projectDir
// root
println project(":").projectDir
Here is an example :desktop:build.gradle file that uses a spoof zip download. Note the applicationsDir variable. (I'm assuming that you are using the same plugin as the one specified.):
plugins {
id "de.undercouch.download" version "1.2"
}
apply plugin: 'de.undercouch.download'
import de.undercouch.gradle.tasks.download.Download
def applicationsDir = project(":applications").projectDir
task download_redis(type: Download) {
src 'https://github.com/codetojoy/Git_Sandbox/raw/master/tmp/example.zip'
dest new File("${applicationsDir}/redis/install", 'Redis-x64-3.2.100.zip')
onlyIfNewer true
}
task install_redis(dependsOn: download_redis, type: Copy) {
from zipTree(download_redis.dest)
into "${applicationsDir}/redis"
}
Related
In build.gradle I have the following code:
task copyModifiedSources(type:Copy) {
from ('ApiClient.java.modified') {
rename 'ApiClient.java.modified', 'ApiClient.java'
}
setDuplicatesStrategy(DuplicatesStrategy.INCLUDE)
into 'build/generated/src/main/java/com/client/invoker'
}
I am trying to copy ApiClient.java.modified (renamed to ApiClient.java before copy) file into build/generated/src/main/java/com/client/invoker folder which already has ApiClient.java file.
I am using a duplicate strategy to override APIClient.java from /invoker folder with ApiClient.java.modified.
If I rename the copying file to the file name not in /invoker folder, then it works but does not work with the duplicate file name. The file is not copied successfully.
Could you please help me to figure out what is going wrong?
Gradle -> 6.5.1
Java -> 11
the rename is same level as from not under from.
below example final file in build/gen content is 'override'
task setupFiles() {
doLast {
mkdir 'build/override'
mkdir 'build/gen'
new File('build/gen/ApiClient.java').text = "orig"
new File('build/override/ApiClient.java.modified').text = "override"
}
}
task copyModifiedSources(type: Copy, dependsOn: setupFiles) {
from 'build/override'
include '*.modified'
rename '(.*).modified', '$1'
setDuplicatesStrategy(DuplicatesStrategy.INCLUDE)
into 'build/gen'
}
run gradle clean copyModifiedSources
I’m trying to extract a jar from a zip file which is available inside the zip as below structure.
test.zip /web-inf /lib /result1.jar
Here is the task:-
task unzip(type: Copy){
group "testgroup"
def zipFile = file("$buildDir/test.zip")
def tree = zipTree(zipFile)
from(tree.matching { include "**/result1.jar"})
into "$builddir/dir"
includeEmptyDirs = false
}
But I am getting jar file with folder structure like /web-inf/lib/result1.jar. I want jar file alone, not the folders (/web-inf/lib/).
Please correct what is wrong in the gradle task.
Gradle Copy DSL provides a method eachFile (Closure closure) that you can use to perform action on each file which will be copied/extracted, see Copy DSL
In your case, you can use this method to rename (remove parent dirs from path) your file:
task unzip(type: Copy) {
group "testgroup"
from( zipTree(file("test.zip")).matching { include "**/result1.jar" } )
into "$buildDir/dir"
def parentDir = "web-inf/lib"
eachFile { fcp ->
fcp.path = fcp.path.replaceFirst("^$parentDir", '')
}
includeEmptyDirs = false
}
I'm relatively new to gradle, so this might be a typical newbee-question.
In our gradle build, we have a set of war files (dependencies) that all include a file that we need to remove from those war files, before building our ear file.
How can I achieve the following:
- for all war files in a folder,
- extract war content to a location (using a Copy task & zipTree)
- re-pack to a new war applying a filter (using War & excludes)
I assume I'll create a new task and add some 'dependsOn' declarations.
task excludeUnwantedFiles(){
file(directoryWithOriginalWars).eachFile { file ->
???? unpack war, filter, assemble new war file ????
}
}
ear.dependsOn(excludeUnwantedFiles)
excludeUnwantedFiles.dependsOn(downloadAllOriginalWarsIntoDirectory)
How do I create that task, that executes for each war-file? What is the best way to do this?
Is there a way I can to this in one task? E.g. with the Copy task and using zipTree(fooBarWithFile.war) as 'from' and 'war(fooBarWithoutFile.war)' and applying a filter in between?
Or is this the way to go, just with a loop? Delete/Remove file from war with Gradle
Any help is highly appreciated!
Cheers,
d.
---------UPDATE-------------------
Thanx Lance Java for your solution.
As I mentioned in my comment, I faced the problem that the war files were downloaded/extracted during execution time and hence not accessable to define the new tasks at configuration time.
My workaround for this is to use tarTree (with a filter) to access the list of war-files that are not yet extracted. See my code example below:
def warFileSourceTarGz = '...tar.gz'
def nfsLibDir="$buildDir/dependencies/"
def nfsLibDownloadDir="$buildDir/downloadedDependencies/"
// task that downloads & extracts the tar.gz
task fetchNfsDependencies(type: Copy) {
from tarTree(warFileSourceTarGz)
into nfsLibDownloadDir
}
// loop through all war files inside the tar.gz and
// create a task to remove unwanted libraries for each war
task excludeUnwantedJarsFromWars(dependsOn: fetchNfsDependencies){
// access the "remote" tar.gz file to get the list of war-files to loop over
def warFileSource = tarTree(warFileSourceTarGz).matching{
include '*.war'
}
// for every war-file, create an exclude-path
warFileSource.visit { nextWarFile ->
if(nextWarFile.name.endsWith('.war')) {
String taskName = "excludeUnwantedJarsFrom_${nextWarFile.name.replace('.war', '')}"
String nextWarFilePath = nfsLibDownloadDir+"/"+nextWarFile.name
Zip tweakWarTask = tasks.create(name: taskName, type: Zip, dependsOn: fetchNfsDependencies) {
from zipTree(nextWarFilePath)
destinationDir = file(nfsLibDir)
archiveName = nextWarFile.name
// exclude these jars, as they cause classloading problems in our ear deployment.
exclude 'WEB-INF/lib/jcan-optrace*'
}
// hook into build-process
ear.dependsOn(tweakWarTask)
}
}
}
ear.dependsOn(excludeUnwantedJarsFromWars)
I'd create a Zip task per war file and wire all the tasks into the DAG by having assemble depend on them all
FileTree warFiles = fileTree(dir: 'path/to/wars', includes: ['**/*.war'])
warFiles.files.each { File warFile ->
String taskName = "tweakWar${warFile.name.replace('.war', '')}"
Zip tweakWarTask = tasks.create(name: taskName, type: Zip) {
from zipTree(warFile) {
exclude 'path/to/some/file.xml'
}
destinationDir = "$buildDir/tweakedWars"
archiveName = warFile.name
}
// wire the task into the dag
assemble.dependsOn tweakWarTask
}
I'm using this Gradle SSH plugin. It has a method put that will move files from my local machine to the machine the session is connected to.
My app is fully built and exists in build/app and I'm trying to move it to /opt/nginx/latest/html/ such that the file build/app/index.html would exist at /opt/nginx/latest/html/index.html and that any subfolders of build/app are also copied over.
My build.gradle:
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'org.hidetake:gradle-ssh-plugin:1.1.4'
}
}
apply plugin: 'org.hidetake.ssh'
remotes {
target {
host = '<my target vm>'
user = 'user'
password = 'pass'
}
}
...
task deploy() << {
ssh.run {
session(remotes.target) {
put from: 'build/app/', into: '/opt/nginx/latest/html/'
}
}
}
As I have it above, it's putting all the files into /opt/nginx/latest/html/app. If I change the from to use fileTree(dir: 'build/app') then all the files get copied over but I lose the file structure, i.e. build/app/scripts/main.js gets copied to /opt/nginx/latest/html/main.js instead of the expected /opt/nginx/latest/html/scripts/main.js.
How can I copy the CONTENTS of one directory (not the directory itself) into the target directory while retaining folder structure?
Looking through the plugin's code, it says:
static usage = '''put() accepts following signatures:
put(from: String or File, into: String) // put a file or directory
put(from: Iterable<File>, into: String) // put files or directories
put(from: InputStream, into: String) // put a stream into the remote file
put(text: String, into: String) // put a string into the remote file
put(bytes: byte[], into: String) // put a byte array into the remote file'''
You're using option #1 where you are providing a File (which can also be a directory), while you should be using #2, which would be an iterable list of build/app's children. So I would try:
put (from: new File('build/app').listFiles(), into: '/opt/nginx/latest/html/')
Edit: Alternatively,
new File('build/app').listFiles().each{put (from:it, into:'/opt/nginx/latest/html/')}
You could create a FileTree object for your build/app directory and then ssh your entire tree structure to your remote instance:
FileTree myFileTree = fileTree(dir: 'build/app')
task deploy() << {
ssh.run {
session(remotes.target) {
put from: myFileTree.getDir(), into: '/opt/nginx/latest/html/'
}
}
It should copy your structure and files like:
// 'build/app' -> '/opt/nginx/latest/html/
// 'build/app/scripts' -> '/opt/nginx/latest/html/scripts/'
// 'build/app/*.*' -> 'opt/nginx/latest/html/*.*'
// ...
You could add a wildcard to copy all files in the folder:
put from: 'build/app/*', into: '/opt/nginx/latest/html/'
I have a gradle project which requires some data files available somewhere on the internet using http. The goal is that this immutable remote file is pulled once upon first build. Subsequent build should not download again.
How can I instruct gradle to fetch the given file to a local directory?
I've tried
task fetch(type:Copy) {
from 'http://<myurl>'
into 'data'
}
but it seems that copy task type cannot deal with http.
Bonus question: is there a way to resume a previously aborted/interrupted download just like wget -c does?
How about just:
def f = new File('the file path')
if (!f.exists()) {
new URL('the url').withInputStream{ i -> f.withOutputStream{ it << i }}
}
You could probably use the Ant task Get for this. I believe this Ant task does not support resuming a download.
In order to do so, you can create a custom task with name MyDownload. That can be any class name basically. This custom task defines inputs and outputs that determine whether the task need to be executed. For example if the file was already downloaded to the specified directory then the task is marked UP-TO-DATE. Internally, this custom task uses the Ant task Get via the built-in AntBuilder.
With this custom task in place, you can create a new enhanced task of type MyDownload (your custom task class). This task set the input and output properties. If you want this task to be executed, hook it up to the task you usually run via task dependencies (dependsOn method). The following code snippet should give you the idea:
task downloadSomething(type: MyDownload) {
sourceUrl = 'http://www.someurl.com/my.zip'
target = new File('data')
}
someOtherTask.dependsOn downloadSomething
class MyDownload extends DefaultTask {
#Input
String sourceUrl
#OutputFile
File target
#TaskAction
void download() {
ant.get(src: sourceUrl, dest: target)
}
}
Try like that:
plugins {
id "de.undercouch.download" version "1.2"
}
apply plugin: 'java'
apply plugin: 'de.undercouch.download'
import de.undercouch.gradle.tasks.download.Download
task downloadFile(type: Download) {
src 'http://localhost:8081/example/test-jar-test_1.jar'
dest 'localDir'
}
You can check more here: https://github.com/michel-kraemer/gradle-download-task
For me works fine..
The suggestion in Ben Manes's comment has the advantage that it can take advantage of maven coordinates and maven dependency resolution. For example, for downloading a Derby jar:
Define a new configuration:
configurations {
derby
}
In the dependencies section, add a line for the custom configuration
dependencies {
derby "org.apache.derby:derby:10.12.1.1"
}
Then you can add a task which will pull down the right files when needed (while taking advantage of the maven cache):
task deployDependencies() << {
String derbyDir = "${some.dir}/derby"
new File(derbyDir).mkdirs();
configurations.derby.resolve().each { file ->
//Copy the file to the desired location
copy {
from file
into derbyDir
// Strip off version numbers
rename '(.+)-[\\.0-9]+\\.(.+)', '$1.$2'
}
}
}
(I learned this from https://jiraaya.wordpress.com/2014/06/05/download-non-jar-dependency-in-gradle/).
Using following plugin:
plugins {
id "de.undercouch.download" version "3.4.3"
}
For a task which has the purpose of only downloading
task downloadFile(type: Download) {
src DownloadURL
dest destDir
}
For including download option into your task:
download {
src DownloadURL
dest destDir
}
For including download option with multiple downloads into your task:
task downloadFromURLs(){
download {
src ([
DownloadURL1,
DownloadURL2,
DownloadURL3
])
dest destDir
}
}
Hope it helped :)
just now ran into post on upcoming download task on gradle forum.
Looks like the perfect solution to me.. Not (yet) available in an official gradle release though
Kotlin Version of #Benjamin's Answer
buildscript {
repositories {
jcenter()
google()
}
dependencies {
classpath("com.android.tools.build:gradle:4.0.1")
}
}
tasks.register("downloadPdf"){
val path = "myfile.pdf"
val sourceUrl = "https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf"
download(sourceUrl,path)
}
fun download(url : String, path : String){
val destFile = File(path)
ant.invokeMethod("get", mapOf("src" to url, "dest" to destFile))
}