How to speed up slow fatJAR task? - performance

The following part is a part of my build.gradle file:
project(":App") {
dependencies {
compile(project(':Shared'))
compile(project(':Subproject1'))
compile(project(':Subproject2'))
compile(project(':Subproject3'))
compile(project(':Subproject4'))
compile(project(':Subproject5'))
}
jar {
manifest {
attributes(
'Main-Class': 'com.my.App.Main',
)
}
}
// To run, use: $ gradle :App:fatJar
task fatJar(type: Jar) {
println "fatJarBuild(): 1"
manifest.from jar.manifest
println "fatJarBuild(): 2"
classifier = 'all'
println "fatJarBuild(): 3"
from {
configurations.runtime.collect {
println "fatJarBuild(): collect" + it.absolutePath
it.isDirectory() ? it : zipTree(it)
}
} {
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
println "fatJarBuild(): 4"
with jar
}
}
Now, whenever I slightly modify a file in my codebase, it takes substantial time to finish fatJar task:
[... End of "./gradlew :App:fatJar --info" follows ... ]
:App:compileJava (Thread[Daemon worker Thread 13,5,main]) completed. Took 0.164 secs.
:App:processResources (Thread[Daemon worker Thread 13,5,main]) started.
:App:processResources
Skipping task ':App:processResources' as it is up-to-date (took 0.002 secs).
:App:processResources UP-TO-DATE
:App:processResources (Thread[Daemon worker Thread 13,5,main]) completed. Took 0.002 secs.
:App:classes (Thread[Daemon worker Thread 13,5,main]) started.
:App:classes
Skipping task ':App:classes' as it has no actions.
:App:classes (Thread[Daemon worker Thread 13,5,main]) completed. Took 0.0 secs.
:App:fatJar (Thread[Daemon worker Thread 13,5,main]) started.
:App:fatJar
Executing task ':App:fatJar' (up-to-date check took 0.003 secs) due to:
Input file /home/work/App/BrainThread.class has changed.
:App:fatJar (Thread[Daemon worker Thread 13,5,main]) completed. Took 5.993 secs.
BUILD SUCCESSFUL
Total time: 7.051 secs
Stopped 0 compiler daemon(s).
Received result Success[value=null] from daemon DaemonInfo{pid=30179, address=[c5e7f6f0-985b-48cc-88b0-ebc8aed7e75b port:33465, addresses:[/0:0:0:0:0:0:0:1%lo, /127.0.0.1]], idle=true, context=DefaultDaemonContext[uid=cc8b9da5-88b5-476a-9cf5-430af98f7f5a,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/user/.gradle/daemon,pid=30179,idleTimeout=10800000,daemonOpts=-XX:MaxPermSize=256m,-XX:+HeapDumpOnOutOfMemoryError,-Xmx1024m,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]} (build should be done).
You can see that BrainThread.class was modified and as an effect some heavy machinery started.
Can I make my build.gradle more time efficient?

Maybe someone else will come along and tell me I'm wrong, but IMO 5 seconds is not unreasonable time to build a fatjar. The task as defined
Unpacks the jar file of every single dependency,
Consolidates the unpacked files, along with your project class files and then
Repackages everything into a new jar.
Depending on how many dependencies you have, 5 seconds seems fairly ok to me. Unless the underlying Zip task is somehow optimized, I do not see this timing change.
A couple of things you can thing about though:
You could move the fatjar task out of your standard build cycle, which runs for every change. You could only run the fatjar task when you're doing a release. In most cases you should be able to test changes locally without having to build a fatjar as you have all the dependencies available anyway.
For a single file change, you're doing the unpack-repack dance with a whole lot of dependency jar. Maybe you can write an optimized fatjar task that unpacks the old fatjar, replaces the changed file and repacks it, reducing the number of file operations required.
EDIT: I just looked at the gradle shadow jar plugin, and realized that it does exactly what I described #2 above, makes incremental changes in place to the fatjar, which should give you palpable performance improvements.

Related

Failed to create APK file using Gluon

I try to generate apk from Netbeans 8 but I 'am getting an exeception.
Here is the content of my gradle file:
buildscript {
repositories {
maven {
url 'https://oss.sonatype.org/content/repositories/snapshots'
}
jcenter()
}
dependencies {
classpath 'org.javafxports:jfxmobile-plugin:1.3.0-SNAPSHOT'
}
}
apply plugin: 'org.javafxports.jfxmobile'
repositories {
jcenter()
maven {
url 'http://nexus.gluonhq.com/nexus/content/repositories/releases'
}
}
mainClassName = 'com.testgluon.TestGluonMob'
dependencies {
compile 'com.gluonhq:charm:4.2.0'
}
jfxmobile {
android {
manifest = 'src/android/AndroidManifest.xml'
compileSdkVersion = '25'
}
}
I have already installed :
multidex-1.0.1.aar
I saw this answer but no luck. it does n't help me.
Edit: the following is a part of stacktrace (using --info option from Netbeans):
:applyRetrolambda (Thread[Daemon worker Thread 3,5,main]) completed. Took 21.933 secs.
:mergeClassesIntoJar (Thread[Daemon worker Thread 3,5,main]) started.
:mergeClassesIntoJar
Putting task artifact state for task ':mergeClassesIntoJar' into context took 0.001 secs.
Task :mergeClassesIntoJar class loader hash: 83f3637f6805a7b149525a93c5faad58
Task :mergeClassesIntoJar actions class loader hash: fde60ab3b9776111ebd9bf87f24df716
Executing task ':mergeClassesIntoJar' (up-to-date check took 0.172 secs) due to:
Output property 'archivePath' file P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\allclasses.jar has changed.
:mergeClassesIntoJar (Thread[Daemon worker Thread 3,5,main]) completed. Took 3.91 secs.
:shrinkMultiDexComponents (Thread[Daemon worker Thread 3,5,main]) started.
:shrinkMultiDexComponents
Putting task artifact state for task ':shrinkMultiDexComponents' into context took 0.033 secs.
Task :shrinkMultiDexComponents class loader hash: c54ece983c65e20d963e3e8bb57cfcb0
Task :shrinkMultiDexComponents actions class loader hash: b6f3644e00f5f8abdc2639cc044bf52b
Executing task ':shrinkMultiDexComponents' (up-to-date check took 0.061 secs) due to:
Output property 'outJarFileCollection' file P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\componentClasses.jar has changed.
ProGuard, version 5.1
Reading program jar [P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\allclasses.jar]
Reading library jar [C:\Program Files\Android\android-sdk\build-tools\25.0.1\lib\shrinkedAndroid.jar]
Preparing output jar [P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\componentClasses.jar]
Copying resources from program jar [P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\allclasses.jar]
:shrinkMultiDexComponents (Thread[Daemon worker Thread 3,5,main]) completed. Took 7.079 secs.
:createMainDexList (Thread[Daemon worker Thread 3,5,main]) started.
:createMainDexList
Putting task artifact state for task ':createMainDexList' into context took 0.03 secs.
Task :createMainDexList class loader hash: c54ece983c65e20d963e3e8bb57cfcb0
Task :createMainDexList actions class loader hash: b6f3644e00f5f8abdc2639cc044bf52b
Executing task ':createMainDexList' (up-to-date check took 0.005 secs) due to:
Output property 'outputFile' file P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex\maindexlist.txt has changed.
:createMainDexList (Thread[Daemon worker Thread 3,5,main]) completed. Took 3.276 secs.
:writeInputListFile (Thread[Daemon worker Thread 3,5,main]) started.
:writeInputListFile
Putting task artifact state for task ':writeInputListFile' into context took 0.029 secs.
Task :writeInputListFile class loader hash: c54ece983c65e20d963e3e8bb57cfcb0
Task :writeInputListFile actions class loader hash: b6f3644e00f5f8abdc2639cc044bf52b
Executing task ':writeInputListFile' (up-to-date check took 0.034 secs) due to:
Output property 'inputListFile' file P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\dex\inputList.txt has changed.
:writeInputListFile (Thread[Daemon worker Thread 3,5,main]) completed. Took 0.108 secs.
:dex (Thread[Daemon worker Thread 3,5,main]) started.
:dex
Putting task artifact state for task ':dex' into context took 0.034 secs.
Task :dex class loader hash: c54ece983c65e20d963e3e8bb57cfcb0
Task :dex actions class loader hash: b6f3644e00f5f8abdc2639cc044bf52b
Executing task ':dex' (up-to-date check took 0.002 secs) due to:
No history is available.
[ant:java] Java Result: 1
:dex FAILED
:dex (Thread[Daemon worker Thread 3,5,main]) completed. Took 0.246 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':dex'.
> org.gradle.api.GradleException (no error message)
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
Note:
I don't require 25. I can use older version.
When I delete compileSdkVersion = '25', I get the this exception:
Executing: gradle :android
Arguments: [--info]
The client will now receive all logging from the daemon (pid: 5056). The daemon log file: C:\Users\user\.gradle\daemon\3.2.1\daemon-5056.out.log
Starting 14th build in daemon [uptime: 1 hrs 0 mins 21.185 secs, performance: 100%, no major garbage collections]
Executing build with daemon context: DefaultDaemonContext[uid=42b5a162-e19a-4659-8796-8ca0d6e5684c,javaHome=C:\Program Files\Java\jdk1.8.0_60,daemonRegistryDir=C:\Users\user\.gradle\daemon,pid=5056,idleTimeout=10800000,daemonOpts=-XX:MaxPermSize=256m,-XX:+HeapDumpOnOutOfMemoryError,-Xmx1024m,-Dfile.encoding=windows-1252,-Duser.country=FR,-Duser.language=fr,-Duser.variant]
Starting Build
Settings evaluated using settings file 'P:\master\settings.gradle'.
Projects loaded. Root project using build file 'P:\javafxGluon\TestGluonMob\build.gradle'.
Included projects: [root project 'TestGluonMob']
Evaluating root project 'TestGluonMob' using build file 'P:\javafxGluon\TestGluonMob\build.gradle'.
Android install directory: P:\javafxGluon\TestGluonMob\build\javafxports\android
Android temporary output directory: P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android
Resources directory: P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\resources
Multi-dex output directory: P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\multi-dex
Dex output directory: P:\javafxGluon\TestGluonMob\build\javafxports\tmp\android\dex
iOS install directory: P:\javafxGluon\TestGluonMob\build\javafxports\ios
iOS temporary output directory: P:\javafxGluon\TestGluonMob\build\javafxports\tmp\ios
Adding C:\Program Files\Android\android-sdk\extras\android\m2repository to project repositories.
Adding C:\Program Files\Android\android-sdk\extras\google\m2repository to project repositories.
All projects evaluated.
Selected primary task ':android' from project :
Tasks to be executed: [task ':validateManifest', task ':collectMultiDexComponents', task ':compileJava', task ':processResources', task ':classes', task ':compileAndroidJava', task ':copyClassesForRetrolambda', task ':applyRetrolambda', task ':mergeClassesIntoJar', task ':shrinkMultiDexComponents', task ':createMainDexList', task ':writeInputListFile', task ':dex', task ':mergeAndroidAssets', task ':mergeAndroidResources', task ':processAndroidResources', task ':processAndroidResourcesDebug', task ':validateSigningDebug', task ':apkDebug', task ':zipalignDebug', task ':android']
Using javafxports version 8.60.8
Configuring Build for Android
Using javafxports dalvik sdk from location C:\Users\user\.gradle\caches\modules-2\files-2.1\org.javafxports\dalvik-sdk\8.60.8\6630ec66e4703c910ac3fd6151a8494c8b59186b\unpacked\dalvik-sdk
Using androidSdk from location: C:\Program Files\Android\android-sdk
There was no buildToolsVersion specified, looking for most recent installed version automatically
Using the following automatically detected buildToolsVersion: 25.0.1
FAILURE: Build failed with an exception.
* What went wrong:
Configured compileSdkVersion is invalid: 21 (C:\Program Files\Android\android-sdk/platforms/android-21
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
runnig gradle -v cammand give me :
------------------------------------------------------------
Gradle 3.2.1
------------------------------------------------------------
Build time: 2016-11-22 15:19:54 UTC
Revision: 83b485b914fd4f335ad0e66af9d14aad458d2cc5
Groovy: 2.4.7
Ant: Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM: 1.8.0_60 (Oracle Corporation 25.60-b23)
OS: Windows Vista 6.0 x86
The link below show the Android SDK tools already installed:
Android SDK tools installed
jfxmobile {
downConfig {
version = '3.0.0'
plugins 'display', 'lifecycle', 'statusbar', 'storage'
}
android {
manifest = 'src/android/AndroidManifest.xml'
compileSdkVersion=25
buildToolsVersion = '25.0.0'
androidSdk='/Users/USERNAME/Library/Android/sdk'
}
ios {
infoPList = file('src/ios/Default-Info.plist')
forceLinkClasses = [
'com.gluonapplication.**.*',
'com.gluonhq.**.*',
'javax.annotations.**.*',
'javax.inject.**.*',
'javax.json.**.*',
'org.glassfish.json.**.*'
]
}
Please make sure to set your android sdk path as shown in the code above. if you do not have the android SDK, you can download it from here
https://developer.android.com/studio/index.html
in this case you will download android studio (which is fine). Download it and then install the desired version of the SDK you want.

gradle init script task not invoked : Gradle 2.14.1

Please find below init.gradle file that has a common task for all my projects :-
rootProject { apply plugin: "eclipse" }
gradle.allprojects{
ext.commonTaskForEveryBuild = {
println "Common task for every build starts here..."
println "Common task for every build ends here..."
}
}
Also find below sample build.gradle for one of my project :-
apply plugin : 'java'
clean.doFirst{
println "Before invoking commonTaskForEveryBuild"
//This is invocation of commonTaskForEveryBuild
commonTaskForEveryBuild
println "After invoking commonTaskForEveryBuild"
}
Below are the logs for the execution of "gradle clean" task for the sample build.gradle file :-
:clean
Before invoking commonTaskForEveryBuild
After invoking
commonTaskForEveryBuild
:clean UP-TO-DATE
BUILD SUCCESSFUL
Total time: 3.722 secs
Looks like clean.doFirst is called, but it's not invoking definition of commonTaskForEveryBuild. Although, there is no compile time or runtime error.
Please suggest.
It's all right, that your task is not invoked, since your script doesn't attempt to do it and it's not even a task. You just getting your commonTaskForEveryBuild instance, but do nothing with it.
You may need to read the official user guide to find out, how to call one task from another. In short - this is usually done via task dependencies. You have to make your clean task depending on commonTaskForEveryBuild task to run it before the clean task execution. This could be done like so:
clean.dependsOn commonTaskForEveryBuild
Furthermore, it's all about tasks, but in your case ext.commonTaskForEveryBuild = {...} is not even a task, but just a closure and it must be executed as commonTaskForEveryBuild()

How do I make Gradle rerun a task when its dependencies are run?

Let's say that I have a task "main" that depends on another task "dependency." I would like "main" to be rerun any time its dependency (or its dependency's dependencies) is rebuilt because "main" relies on the artifacts produced by "dependency" (or the dependencies of "dependency").
A build.gradle file containing an example of what I'm dealing with is the following:
defaultTasks 'main'
task baseDependency {
outputs.file 'deps.out'
outputs.upToDateWhen { false }
doLast {
exec {
commandLine 'bash', '-c', 'echo hello world > deps.out'
}
}
}
task dependency(dependsOn: baseDependency)
task main(dependsOn: dependency) {
outputs.file 'main.out'
doLast {
exec {
commandLine 'bash', '-c', 'echo hello world > main.out'
}
}
}
Executing gradle the first time:
:baseDependency
:dependency
:main
BUILD SUCCESSFUL
Total time: 0.623 secs
Executing it a second time:
:baseDependency
:dependency
:main UP-TO-DATE
BUILD SUCCESSFUL
Total time: 0.709 secs
I would really love if "main" were not marked "UP-TO-DATE" if its dependencies had to be rebuilt. That seems essential. How do you make sure that's the case?
The standard way to specify dependency between tasks is via task's inputs and outputs. This can be any file, fileset or directory.
In your case you should modify main task and add inputs.file 'deps.out' to its definition.
Note that gradle has an optimization that may lead to unexpected behavior in a simplistic example that you provided.
Before a task is executed for the first time, Gradle takes a snapshot
of the inputs. This snapshot contains the set of input files and a
hash of the contents of each file. Gradle then executes the task. If
the task completes successfully, Gradle takes a snapshot of the
outputs. This snapshot contains the set of output files and a hash of
the contents of each file. Gradle persists both snapshots for the next
time the task is executed.
Each time after that, before the task is executed, Gradle takes a new
snapshot of the inputs and outputs. If the new snapshots are the same
as the previous snapshots, Gradle assumes that the outputs are up to
date and skips the task. If they are not the same, Gradle executes the
task. Gradle persists both snapshots for the next time the task is
executed.
So even if you specify correct inputs in a simple example where the same file is generated the dependent task will be marked as up to date on the second and subsequent runs.
If you do not want or cannot hardcode dependency on the file you can override upToDateWhen for dependent task and calculate the condition if the task is up to date based on dependencies of this task and their state like this:
outputs.upToDateWhen { task ->
task.taskDependencies.inject(true) { r, dep ->
r && dep.values.inject(true) { res, v ->
res && (!(v instanceof Task) || v?.state.getSkipped())
}
}
}
upToDateWhen should return true if this task should not be run at all (because its output are already up-to-date). And this is the case when no dependent task was run (the gradle documentation is a bit vague about this I must admit but getSkipped seems work as expected).

Why gradle clean task starts all other non-default tasks?

I have gradle set up and running. My build.gradle has 2 tasks defined inside:
task setVersion() {
println('setVersion')
//...
}
task setIntegrationEnv() {
println('setIntegrationEnv')
//...
}
When I run
./gradlew clean
gradle runs both tasks setVersion and setIntegrationEnv and then it runs clean for all my modules (app, cloud_module) in that project, output:
Relying on packaging to define the extension of the main artifact has been deprecated and is scheduled to be removed in Gradle 2.0
setVersion
setIntegrationEnv
:cloud_module:clean
:app:clean
BUILD SUCCESSFUL
Total time: 14.18 secs
Why this happens, where this behavior is defined?
Could You please provide full build.gradle script? I'd be much easier to help You. You've probably mistaken gradle build phase with configuration phase - it's a common topic here.
General rule is that code You'd like to be run at build phase should be added as an action:
task someTask << {
println 'runtime'
}
while code You'd like to run at configuration phase should be added in task body:
task someTask {
println 'configuration
}
or all together:
task someTask {
println 'configuration'
doLast {
println 'runtime'
}
}
Additional info can be found here, here and here.

Why does my large multi-project gradle build start up slowly?

I am on a team that has a large multi-project build (~350 modules) and we have noticed that doing an individual build (-a) still takes a large amount of time. When building the entire set of things the overhead isn't so bad, but when we do something like:
cd my/individual/project
gradle -a --configure-on-demand --daemon build
that it still takes 30-40 seconds just in the configuration phase (before it starts building anything, which we measure using the --dry-run option)
We have approximately 10 custom tasks and we are setting the inputs and outputs for those tasks but we still see this inordinately long configuration time. We are using gradle 1.10
It's hard to say from a distance. A common mistake is to do work in the configuration phase that should be done in the execution phase. Here is an example:
task myTask {
copy {
from ...
to ...
}
}
Here, the project.copy method will be called (and therefore copying will take place) every single time the task is configured, which is for every single build invocation - no matter which tasks are run! To fix this, you'd move the work into a task action:
task myTask {
doLast {
copy {
from ...
into ...
}
}
}
Or even better, use a predefined task type:
task myTask(type: Copy) {
from ...
into ...
}
Some steps you can take to find the cause(s) for performance problems:
Review the build
Run with --info, --debug, or --profile
Profile the build with a Java profiler
PS: Specifying inputs and outputs doesn't shorten configuration time (but likely execution time).

Resources