I need to create a gradle task to create a zip where I need to include all the json files that contain text "type":"customer" how can i include the check for containing text ?
is there a way I can do it:
task createZip(type: Zip) {
archiveName = "customer.zip"
destinationDir = file(testDir)
from(files(customer))
}
You could use a FileTree together with Gradle/Groovy filtering capabilities:
Let's say you have your source Customer JSON files under src/customers: you can define a Task like that:
task createZip(type: Zip) {
archiveName = "customer.zip"
destinationDir = buildDir
from fileTree('src/customers').filter { f -> f.text.contains('"type": "customer"')}
}
Note that this uses Groovy's File.getText() method to read the whole file content into memory to match the "type": "customer" expression. Be careful with performances if you have plenty or huge files to filter.
Related
I'm trying to configure a Zip task based on one of the property inside sub-projects, but the property is not yet accessible at the time of configuring the task. For instance, I want to exclude all my projects that has toexclude = true from my zip file. So, the build.gradle of the sub-projects that I want to exclude starts with this:
ext.toexclude = true;
...
And my main build.gradle has this task:
task zipContent (type: Zip){
def excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
println excludedProjects
destinationDir = "/some/path"
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
The problem is that excludedProjects is always empty. Indeed, when I am executing the task, I can see []. I believe this is due to the fact that the property that I set in the subproject's build.gradle is not available at the moment the task is configured. As a proof, if I replace the first line of the task by this:
def excludedProjects = allprojects.collect{it.name}
The task prints out all of my project's name, and the zip contains nothing (which means the problem is in the p.toexclude == true).
Also, if I try this:
task zipContent (type: Zip){
def excludedProjects = []
doFirst{
excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
println "IN DOFIRST"
println excludedProjects
}
println "IN TASK CONFIG"
println excludedProjects
destinationDir = "/some/path"
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
The task prints out IN TASK CONFIG followed by an empty array, then IN DOFIRST with the array containing only the subprojects that I set ext.toexclude == true.
So, is there a way to get the properties of the sub-projects at configuration time?
Well, the crucial question is: At which point of the build is all necessary information available?
Since we want to know each project in the build, where the extra property toexclude is set to true and it is possible (and by design) that the property is set via the build script, we need each build script to be evaluated.
Now, we have two options:
By default, subprojects are evaluated after the parent (root) project. To ensure the evaluation of each project, we need to wait for the point of the build, where all projects are evaluated. Gradle provides a listener for that point:
gradle.addListener(new BuildAdapter() {
#Override
void projectsEvaluated(Gradle gradle) {
tasks.getByPath('zipContent').with {
exclude allprojects.findAll { it.toexclude }.collect{ it.name }
}
}
})
Gradle provides the method evaluationDependsOnChildren(), to turn the evaluation order around. It may be possible to use your original approach by calling this method before querying the excluded projects. Since this method only applies on child projects, you may try to call evaluationDependsOn(String) for each project in the build to also apply for 'sibling' projects. Since this solution breaks Gradle default behavior, it may have undesired side effects.
Just define excludedProjects outside the task
def excludedProjects = allprojects.findAll{Project p -> p.toexclude == true}.collect{it.name}
task zipContent (type: Zip){
destinationDir = file("/some/path")
baseName = "myFile.zip"
exclude excludedProjects
from "/some/other/path"
}
You can call evaluationDependsOnChildren() in the root project so that child projects are evaluated before the root
Eg
evaluationDependsOnChildren()
task zipContent (type: Zip) { ... }
Another option is to use an afterEvaluate { ... } closure to delay evaluation
Eg:
afterEvaluate {
task zipContent (type: Zip) { ... }
}
I have a custom Gradle plugin that will generate Java files from a template file. I have several such template files in different locations, and I need to "compile" all of them to generate the Java files I need. Once I have the files, I want to package them into a .jar.
One way I thought I could do this was to call the "compile template" task multiple times from within the same build file. I'd call it once in a task that compiles template files in location A, again from a task that compiles template files from location B... etc., until I have all the Java files I need.
Something like this:
task compileFromLocationA <<{
compileTemplate.execute(A)...
}
task compileFromLocationB
compileTemplate.execute(B)...
...
packageJar(depends: compileFromLocationA, compileFromLocationB, ...)
...
However, you can't programmatically call a task from within another task. I suppose I could break each compileFromLocation_ task into it's own build.gradle file, but that seems like overkill. What's the "best practice" in a case like this?
This code seems to work in build.gradle by using tasks.register() - e.g. to perform multiple source code generating steps - in my case I needed to load different pairs of files (XML schema and generation options) in two different steps:
plugins {
id 'java'
id "com.plugin" version "1.0"
}
sourceSets.main.java.srcDirs += file("${buildDir}/genSrc")
sourceSets.test.java.srcDirs += file("${buildDir}/testGenSrc")
tasks.compileJava {
dependsOn tasks.named("genMessage")
}
genMessage {
codesFile = "${projectDir}/src/main/resources/file.xml"
}
def testGenModel1 = tasks.register("testGenModel1", com.plugin.TestGenModelTask.class) {
schema = "${projectDir}/src/test/resources/file.xsd"
options = "${projectDir}/src/test/resources/file.xml"
}
def testGenModel2 = tasks.register("testGenModel2", com.plugin.TestGenModelTask.class) {
schema = "${projectDir}/src/test/resources/file2.xsd"
options = "${projectDir}/src/test/resources/file2.xml"
}
tasks.compileTestJava {
dependsOn tasks.named("testGenModel1"), tasks.named("testGenModel2")
}
I want to make my gradle build inteligent when building my model.
To acquire this I was planning to read schema files, acquire what is included and then build firstly included models (if they are not present).
I'm pretty new to Groovy and Gradle, so please that into account.
What I have:
build.gradle file on root directory, including n subdirectories (subprojects added to settings.gradle). I have only one gradle build file, because I defined tasks like:
subprojects {
task init
task includeDependencies(type: checkDependencies)
task build
task dist
(...)
}
I will return to checkDependencies shortly.
Schema files located externally, which I can see.
Each of them have from 0 to 3 lines of code, that say about dependencies and looks like that:
#include "ModelDir/ModelName.idl"
In my build.gradle I created task that should open, and read those dependencies, preferably return them:
class parsingIDL extends DefaultTask{
String idlFileName="*def file name*"
def regex = ~/#include .*\/(\w*).idl/
#Task Action
def checkDependencies(){
File idlFile= new File(idlFileName)
if(!idlFile.exists()){
logger.error("File not found)
} else {
idlFile.eachLine{ line ->
def dep = []
def matcher = regex.matcher(line)
(...)*
}
}
}
}
What should I have in (...)* to find all dependencies and how should I define, that for example
subprojectA::build.dependsOn([subprojectB::dist, subprojectC::dist])?
All I could find on internet created dep, that outputted given:
[]
[]
[modelName]
[]
[]
(...)
Can gradle alter the structure of the tree while copying?
original
mod/a/src
mod/b/src
desired
dest/mod-a/source
dest/mod-b/source
dest/mod-c/source
I'm not sure where I should create a closure and override the copy tree logic
I'd like to do the gradle equivalent of ant's globmapper functionality
<property name="from.dir" location=".."/>
<property name="to.dir" location="dbutil"/>
<copy>
<fileset dir="${from.dir}" ... />
<globmapper from="${from.dir}/*/db" to="${to.dir}"/>
</copy>
Thanks
Peter
When changing file name, rename seems a good approach. When changing path you can override eachFile and modify the destination path.
This works pretty well.
copy {
from("${sourceDir}") {
include 'modules/**/**'
}
into(destDir)
eachFile {details ->
// Top Level Modules
def targetPath = rawPathToModulesPath(details.path)
details.path = targetPath
}
}
....
def rawPathToModulesPath(def path) {
// Standard case modules/name/src -> module-name/src
def modified=path.replaceAll('modules/([^/]+)/.*src/(java/)?(.*)', {"module-${it[1]}/src/${it[3]}"})
return modified
}
Please see sample below. Gradle 4.3 does not have rename/move methods, so we can do renaming on the fly.
What was happened:
Load file tree into the memory. I used zip file from dependencies in my example
Filter items, which are in the target folder
All result items will have the same prefix: if we filter files from directory "A/B/C/", then all files will be like "A/B/C/file.txt" or "A/B/C/D/file.txt". E.g. all of them will start with the same words
In the last statement eachFile we will change final name by cutting the directory prefix (e.g. we will cut "A/B/C").
Important: use type of task "Copy", which has optimizations for incremental compilation. Gradle will not do file copy if all of items below are true:
Input is the same (for my case - all dependencies of scope "nativeDependenciesScope") with previous build
Your function returned the same items with the previous build
Destination folder has the same file hashes, with the previous build
task copyNativeDependencies(type: Copy) {
includeEmptyDirs = false
def subfolderToUse = "win32Subfolder"
def nativePack = configurations.nativeDependenciesScope.singleFile // result - single dependency file
def nativeFiles = zipTree(nativePack).matching { include subfolderToUse + "/*" } // result - filtered file tree
from nativeFiles
into 'build/native_libs'
eachFile {
print(it.path)
// we filtered this folder above, e.g. all files will start from the same folder name
it.path = it.path.replaceFirst("$subfolderToUse/", "")
}
}
// and don't forget to link this task for something mandatory
test.dependsOn(copyNativeDependencies)
run.dependsOn(copyNativeDependencies)
The following works, but is there a more gradle-ish way to do this?
ant.copy(todir: destDir) {
fileset( dir: "${srcDir}/module", includes: '**/src/**')
regexpmapper(from: '^(.*)/src/(.*)$', to: /module-\1\/src\/\2/)
}
I am writing a Gradle build for a non-java project for assembling existing directories and tar archives into a .tar.gz The tar task skips if I use the definition like so:
task archive(dependsOn: 'initArchive',type: Tar) << {
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
here's the console output
:jenkins-maven-sonar:archive
Skipping task ':jenkins-maven-sonar:archive' as it has no source files.
:jenkins-maven-sonar:archive UP-TO-DATE
BUILD SUCCESSFUL
Total time: 9.056 secs
When I try to use tar task as a method it fails complaining cannot find method
task archive(dependsOn: 'initArchive') << {
tar{
baseName = project.Name
destinationDir = new File(project.buildDir.path+'/installer')
compression = Compression.GZIP
from (archiveDir)
doLast{
checksum(archivePath)
}
}
}
FAILURE: Build failed with an exception.
* Where:
Build file '/home/anadi/Code/da-ci-installers/build.gradle' line: 29
* What went wrong:
Execution failed for task ':jenkins-maven-sonar:archive'.
> Could not find method tar() for arguments [build_6a2bckppv2tk8qodr6lkg5tqft$_run_closure3_closure5_closure7#4a5f634c] on task ':jenkins-maven-sonar:archive'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
Total time: 8.749 secs
Can we run the tar task in same way as Gradle allows running copy? In the same build I have a block like follows and I want to know if tar can used in the same way
copy {
project.logger.info("Copying bundle :: "+bundle[x])
from(rootProject.projectDir.path+"/3rd-party-tools/"+bundle[x]) {
include '**/*.*'
}
into(archiveDir)
}
if not how to make sure my build does not "skip tar" task if using the first form described above.
You have fallen for the classical mistake of configuring a task in the execution phase rather than the configuration phase. The solution is to remove the << in the first code snippet.
If you find the << (and the difference it makes) confusing, a good solution is to never use << but always the more explicit doLast {}.
There is no tar method, but it's usually better to make these things a separate task anyway. (Methods like copy should only be preferred over the corresponding task if there is a strong reason.)
I had a funny situation where I got hit by this when using doLast{} on a tar task.
It was because of a multi-project build:
build.gradle
--> sub-project
--> build.gradle
In this case if you try to have a tar or a copy task in the main build file that references something from that project(":sub-project") uses it will tempt the developer to wrap it in doLast.
For example, main build.gradle file has:
task distTar(type: Tar, dependsOn: "buildDist") {
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
So they got an error that project(":sub-project").war doesn't exist. So to get around it someone put doLast {} the task and errors went away. BAD!!
task distTar(type: Tar, dependsOn: "buildDist") {
doLast { // <-- BAD!!
description "Package ProjName into a Tar file"
group "Build"
baseName = 'outbasename'
archiveName = baseName + '.tar.gz'
compression = Compression.GZIP
destinationDir = file(project(":packaging").buildDir.path)
extension = 'tar.gz'
into('outdir') {
from project(":sub-project").war
}
}
}
Then i was left to fix it. So the correct was to do it was to add
evaluationDependsOn ":sub-project"
In the main build.gradle file. Now it knows to evaluate it. I remove the incorrect doLast{} block and now the task is no longer ignored.