Running parallel gradle jobs - busy Daemons could not be reused - gradle

Question

I'm trying to run multiple 'gradle installdebug' commands in parallel using a java program using multiple threads. Basically, this is to compile multiple pre-written android apps together in parallel as those apps are not dependent on each other. But all other gradle commands waits with the message 'busy Daemons could not be reused' until the first command completes its execution. I need help in running all the gradle command independently at the same time. more details below.
I have 10 android app folders in windows 10 machine and my java program goes into each of this app folder and executes 'gradle installdebug' command. This happens in parallel using multiple threads. This command compiles android app into APK and install it to the running android emulator. All of the 10 commands get executed successfully and the APKs are installed as expected. The problem I'm facing is that the 9 gradle jobs are queued until the first one is finished, and then remaining 9 gradle jobs runs in parallel. I tried to run the same gradle commands with .Net c# code too, but same issue is observed. I believe this is not a multithreading issue from java code or c# code, instead it is a gradle configuration issue. Looks like I'm missing something while forming the gradle command or configuring the gradle either globally or in the android app gradle settings files.
I'm using the gradle 7.5.1 offline version that I extracted into C: drive for the gradle binary/command as the android app have no dependencies to download while compiling. At end of the question, I provided the android gradle settings files too.
This is the command that I execute from the code.
cmd.exe /c C:\gradle-7.5.1-bin\bin\gradle installdebug
Starting a Gradle Daemon (subsequent builds will be faster)
Task :preBuild UP-TO-DATE
Task :javaPreCompileDebug
Task :mergeDebugResources
and goes on....
Initially for few tasks it runs in parallel with above output then for rest of the commands the gradle outputs the following messages and waits. In this example, from the 7th command it starts queuing. But If I run the program again, only one gradle command gets the daemon, rest all gets queued. On multiple runs the busy daemons count keeps increasing even beyond 100s.
Starting a Gradle Daemon, 6 busy Daemons could not be reused, use --status for details
[ My Note: here it waits for the first gradle command to finish, then this continues. same for the rest of the 8 commands ]
Task :preBuild UP-TO-DATE
Task :javaPreCompileDebug
Task :mergeDebugResources
and goes on....
I tried the command with --no-daemon as shown below.
cmd.exe /c C:\gradle-7.5.1-bin\bin\gradle --offline --no-daemon --build-cache installdebug
With this command I don't see the 'busy Daemons could not be reused' message, but it is slow as for each command a single Daemon started and stopped as shown in below output. Also, this is still having the same problem of waiting for the first command to be complete before running other commands for other android apps.
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.5.1/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build
[ My Note: here it waits for the first gradle command to finish, then this continues. same for the rest of the 8 commands ]
> Task :preBuild UP-TO-DATE
> Task :javaPreCompileDebug
and goes on....
I have no idea why the other 9 gradle commands are waiting until the first one finished as there is no dependency between them because each of these commands are executed in separate android app folders.
To provide more info, the following is the "build.gradle" file the each of those 10 android apps has.
plugins {
id 'com.android.application'
}
android {
namespace "com.example.simple.myapp1"
compileSdk 31
defaultConfig {
applicationId "com.example.simple.myapp1"
minSdk 30
targetSdk 31
versionCode 1
versionName "1.0"
}
signingConfigs {
aerndappsigning {
keyAlias 'androiddebugkey'
keyPassword 'android'
storeFile file('platform.jks')
storePassword 'android'
}
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
debug {
signingConfig signingConfigs.aerndappsigning
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
}
dependencies {
}
the following is the "gradle.properties" file in the android app.
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
android.useAndroidX=true
android.enableJetifier=true
android.nonTransitiveRClass=true
org.gradle.daemon=true
org.gradle.parallel=true
the following is the "local.properties" file in the android app.
sdk.dir=D\:\\AndroidSdk
finally, the following is the "settings.gradle" file in the android app.
pluginManagement {
repositories {
gradlePluginPortal()
google()
mavenCentral()
}
plugins {
id 'com.android.application' version '7.1.0-alpha11'
id 'com.android.library' version '7.1.0-alpha11'
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
}
}
rootProject.name = "myapp1"
include ':app'
Before posting this question, I searched solutions online, landed up on many posts having similar output, but nothing helped as none of those use cases is similar to what I explained above like running the command in parallel in multiple threads.
The other suggested solutions I tried and their outcomes is as follows:
gradle --stop -> this stops the running gradle daemons but does not completely remove them, ending up with same 'could not reuse' problem.
C:\Users\<myuserdir>\.gradle\daemon\7.5.1 -> removing this directory removes all the stopped daemons completely, but this again ended up being slow as daemons are started again. On the next run we again started seeing the 'busy daemons' error.
org.gradle.daemon=true; org.gradle.parallel=true -> added these 2 lines to 'gradle.properties' files. no difference in output.
Please note that this 'gradle installdebug' is executed within different android app folders that are meant for different android apps. I'm not referring to the gradle configurations of sub-projects inside single android app. I'm also not referring to the parallel gradle tasks within a single android app.
I would highly appreciate if you can provide any input to resolve this issue as I have very limited knowledge in the gradle configuration. In a nutshell, I'm looking for a gradle configuration or parameters where all of the 'gradle installdebug' commands can run independently when executed from multiple threads. Is it possible to start 'n' number (say 10) of gradle daemons and keep it running and use it to serve 'n' commands in parallel and never stop until explicitly stopped?