We were seeing mysterious failures in our CI environment when running tests of a Kotlin code base.
gradle test compiled the code and the tests just fine. The tests ran and all appeared to pass. But then Gradle exited with code 137 (indicating it was killed with SIGKILL) and the CI system thus thought the build had failed.
Limiting the size of the JVM for Gradle didn't help. Nor did the --no-daemon and --max-workers options.
This is using Kotlin 1.2.40 and Gradle 4.3 on the Java 8 JVM.
The culprit in this case turned out to be the Kotlin compiler.
By default, the Kotlin compiler spawns a daemon in the background so that subsequent compilation jobs are faster. For a Kotlin code base of nontrivial size, this process can end up eating significant memory.
Its presence was causing Gradle to hit the memory limit on the CI container, and the Linux OOM killer was killing the Gradle process.
The solution: Tell the Kotlin compiler not to spawn the background process. This turns out to be a simple matter of setting an environment variable:
GRADLE_OPTS=-Dkotlin.compiler.execution.strategy=in-process
With that variable in place, Kotlin compilation runs inline in the Gradle JVM, and its data can thus be garbage-collected when Gradle moves on to running the tests.
It would also work to pass the option to the gradle command rather than setting it in the environment.
Related
I would like to set the org.gradle.native JVM system property to false to disable gradle native integration on our CI system where the OS version does not support it. I have no control over the OS. We use the gradle wrapper which invokes the gradle daemon to do the build.
The relevant gradle code that reads the property is here. Note that it's a static initialiser - it happens early - this may or may not be relevant. I have tried, without success:
./gradlew -Dorg.gradle.native=false
./gradlew -Dorg.gradle.jvmargs='-Dorg.gradle.native=false'
and also adding to $GRADLE_USER_HOME/gradle.properties:
systemProp.org.gradle.native=false
System info
Software info
OS:
Java: OpenJDK 12.0.2
Gradle: 5.6.2
The issue
Building Gradle multi-project with parallel builds enabled consumes almost all the CPU time. PC is not interactable during the build process
Steps to reproduce
1. git clone --recursive https://github.com/vividus-framework/vividus.git
2. cd vividus
3. ./gradlew build
In your gradle.properties file (or GRADLE_OPTS environment variable), try setting org.gradle.priority=low. On my machine it has a noticeable effect with parallel enabled, but I've also heard from some of my co-workers with older machines that this setting didn't help them too much.
You can also experiment with setting org.gradle.workers.max. It defaults to the number of CPU processors. Maybe set it to the number of logical processors minus one.
If it still stops you from interacting with your computer during the build, you should probably just disable parallel execution and let Gradle work on a single processor.
Every build takes an extra 3-4 seconds, pausing immediately after the log output prints the following.
[LIFECYCLE]
[org.jetbrains.kotlin.gradle.plugin.KotlinGradleBuildServices] Forcing
System.gc()
Why is it "forcing" this? How do I avoid this and speed up my build?
I've looked into this, and this is a consequence of having Gradle's debug-level logging enabled (eg. gradle --debug assemble).
Run Gradle without debug logging enabled (eg. gradle --info assemble) and this should not occur anymore.
References: libraries/tools/kotlin-gradle-plugin/src/main/kotlin/org/jetbrains/kotlin/gradle/plugin/KotlinGradleBuildServices.kt
Kotlin Gradle plugins calls System.gc before and after a build only when debug logging is enabled (Gradle is run with -d or --debug command line argument).
Users do not normally run Gradle with debug logging enabled because it is extremely noisy and slow, so forcing a GC is a relatively minor issue.
Historically this behaviour was added to test against memory leaks when Gradle daemon is enabled. The idea was to log a difference of used memory before and after a build, run a few builds consequently in a test, and assert that the difference is not exceeding the threshold.
I think calling System.gc should be avoided unless the test KotlinGradleIT#testKotlinOnlyDaemonMemory is running, so I've created an issue at Kotlin bugtracker https://youtrack.jetbrains.com/issue/KT-17960
How do I determine why my Gradle Daemon died? The only message I get isL
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
This occurs in an active build. Several steps will finish and a step will appear to be active and then the build fails.
This began after moving our memory args (Xmx Xms PermGen) from a shell script that called gradlew to gradle.properties and calling gradlew directly.
build.sh
export GRADLE_OPTS="\"-Xmx1024m\" \"-Xms256m\" \"-XX:MaxPermSize=256m\""
export JAVA_HOME="/usr/local/java/jdk1.6"
exec ./gradlew "$#"
Addition to gradle.properties
org.gradle.java.home=/usr/local/java/jdk1.6/
org.gradle.jvmargs=-Xmx1024m -Xms256m -XX:MaxPermSize=256m
After this change Gradle warns:
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: http://gradle.org/docs/2.2.1/userguide/gradle_daemon.html
And even though we don't ask it to, the build is running in a daemon, which ultimately fails.
Gradle build daemon disappeared unexpectedly most frequently occurs when something else kills the long-running Gradle Daemon process and the client process (the Daemon uses local TCP connections to communicate) tries to send a message and gets no response.
For example, running gradle --stop or killall java while a build is occurring will reproduce this problem.
I'm trying to improve the startup time of Gradle. The expererimental --daemon switch doesn't seem to really speed it up. So I'm thinking to use some server process independent of gradle, and make gradle connect to it. The options I found so far are
nailgun to invoke java
GroovyServ to invoke a groovy script
Since gradle is started by a shell script, it takes some tweaking. My question is: has anyone used the above options to start gradle? Or if you have successfully used another option, what's that?
My guess is that your build is doing something at configuration time that it should be doing at execution time. With m5, gradle build --profile will give you an HMTL report showing where the time goes. Another way to see what's going on is gradle build --info or gradle build --debug.