My project uses Gradle's multi-project system. Most of my projects include the "lemur-common" library like this
dependencies {
compile project(":lemur-common")
}
"lemur-common" also has a unit test directory, which has a somewhat complicated dependency statement.
dependencies {
testCompile group: 'org.springframework.boot', name: 'spring-boot-starter-test', version: '2.3.9.RELEASE'
testCompile group: 'org.mockito', name: 'mockito-inline', version: '3.5.6'
constraints {
implementation('org.mockito:mockito-core:3.5.6') {
because 'Fixes illegal reflective access warning'
}
}
}
Now, all of my other projects need to have this same line, since they're all using spring-boot-starter-test. I've done quite a bit of fiddling, trying to express something like "project-a's testCompile should have the same dependencies as lemur-common" but I haven't gotten it to work.
Is it possible to express this in Gradle? Does anyone know how?
You'll want to create a custom Gradle plugin.
As a sibling of all your project dirs, create a buildSrc directory, which itself will have a Gradle file (it needs to be built like everything else). Make gradle file under src, which will be your plugin. Put all of the shared gradle code (the dependencies block you posted) in that file. For example:
- buildSrc
|── build.gradle
|── settings.gradle
|── src
|── main
|── groovy
|── unit-test-dependencies.gradle
Then, in the build.gradle of lemur-common, projectA, and etc. add this line
plugins {
id 'unit-test-dependencies'
}
That should do it. Gradle's documentation for this feature is here: https://docs.gradle.org/current/samples/sample_convention_plugins.html
I am new to Gradle and I am reading the documentation but I don't understand some parts of it. One of these parts is connected with buildscript block. What is its purpose?
If your build script needs to use external libraries, you can add them to the script's classpath in the build script itself. You do this using the buildscript() method, passing in a closure which declares the build script classpath.
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath group: 'commons-codec', name: 'commons-codec', version: '1.2'
}
}
Ok but what is the difference with:
repositories {
mavenCentral()
}
dependencies {
compile group: 'commons-codec', name: 'commons-codec', version: '1.2'
}
For example, why it is necessary to use buildscript?
The buildScript block determines which plugins, task classes, and other classes are available for use in the rest of the build script. Without a buildScript block, you can use everything that ships with Gradle out-of-the-box. If you additionally want to use third-party plugins, task classes, or other classes (in the build script!), you have to specify the corresponding dependencies in the buildScript block.
The global level dependencies and repositories sections list dependencies that required for building your source and running your source etc.
The buildscript is for the build.gradle file itself. So, this would contain dependencies for say creating RPMs, Dockerfile, and any other dependencies for running the tasks in all the dependent build.gradle
I appreciate Peter's answer... but it was not immediately obvious to me what the rest of the build script meant as emphasized in the answer and in the documentation.
Usually bringing in dependent functionality is for use in the Java program or whatever other program you might be writing. Bringing in Spring say, is not to be used in the build script, but in the Java program. Putting it in the buildscript closure ensures that the dependencies are available for use within the gradle build itself. Not the output program.
A bit more explanation by demonstrating Android top-level gradle file.
buildscript {
// this is where we are going to find the libraries defined in "dependencies block" at below
repositories {
google()
jcenter()
maven { url 'https://dl.bintray.com/kotlin/kotlin-eap' }
}
// everything listed in the dependencies is actually a plugin, which we'll do "apply plugin" in our module level gradle file.
dependencies {
classpath 'com.android.tools.build:gradle:3.4.2' // this is android gradle plugin
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" // kotlin gradle plugin
}
}
module level gradle file
apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
What is "plugin"? They are just java classes, which implement Plugin interface. Under the interface, it has a method "apply" to add multiple task objects with different names. Task is a class where we can implement the workflow. For instance, the build task consists of the flow of building the app.
So, what does buildscript do? It defines where to find the plugins. What does plugin do? It encompasses multiple tasks. What does task do? It provides us with the build, install, lint, etc.
My understanding might be wrong. Please don't hesitate to correct me if you find anything is misleading.
The "buildscript" configuration section is for gradle itself (i.e. changes to how gradle is able to perform the build). So this section will usually include the Android Gradle plugin.
It's a bit high level but hope helps.
For me, clear distinction started to shape once I start to understand what is a building block, method, and task. How the syntax looks like, how you can configure them etc. So I suggest you go through all these. After that, you can begin to make sense out of this syntax.
Then it's very important to know what's the type of the object build.gradle (an instance of Project class) so to know what can you have inside a build.gradle file. That would answer where that 'buildScript' and others come from. And to extend your capabilities/features (let's say android), see how plugins can help.
Last but not least, there's a very good tutorial here that talks about closures, delegates which are the concepts essential to understand the script.
buildscript block is used for the build script, NOT for the gradle build output (for example, an Android app apk). In the following code example, the encoding code is used in build script, not in the gradle build output program; so the dependecies should be added to buildscript block.
https://docs.gradle.org/current/userguide/tutorial_using_tasks.html#sec:build_script_external_dependencies
External dependencies for the build script
Instead of manipulating the
script classpath directly, it is recommended to apply plugins that
come with their own classpath. For custom build logic, the
recommendation is to use a custom plugin. If your build script needs
to use external libraries, you can add them to the script’s classpath
in the build script itself. You do this using the buildscript()
method, passing in a block which declares the build script classpath.
The block passed to the buildscript() method configures a ScriptHandler instance.
You declare the build script classpath by adding dependencies to the
classpath configuration. This is the same way you declare, for
example, the Java compilation classpath. You can use any of the
dependency types except project dependencies.
Having declared the build script classpath, you can use the classes in
your build script as you would any other classes on the classpath. The
following example adds to the previous example, and uses classes from
the build script classpath.
import org.apache.commons.codec.binary.Base64
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath group: 'commons-codec', name: 'commons-codec', version: '1.2'
}
}
tasks.register('encode') {
doLast {
def byte[] encodedString = new Base64().encode('hello world\n'.getBytes())
println new String(encodedString)
}
}
You can imagine the buildScript block as contents from the Gradle core, like plugins{} block that already goes into Gradle internally.
So all plugins from buildScript of parent build.gradle will be available in all nested build.gradle modules.
I believe that everything in the buildscript {} will be available for the current build script itself and then the all its subprojects.
And for the properties declared in the file itself outside of the buildscript {} it will not become immediately available to for buildscript of the given project itself, but to all its subprojects.
So if you want to declare something and use it for the buildscript itself right away (current buildscript and not just subproject's buildscript), declare them in the buildscript {} for the current project and it also has the side effect to let subproject use it later on.
If you just want to declare something globally (for sub-projects's buildscripts) you can declare them directly as ext {} in parent project. The parent project won't be able to use them for itself's buildscript but it will be available all the subproject to use, in or out of the buildscript clause.
For example in parent project:
ext {
kotlin_version_XX = '1.7.10'
}
buildscript {
ext {
kotlin_version = '1.7.10'
}
// kotlin_version will be available here since declared in buildscript{}
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
// will NOT be available here -- error due to kotlin_version_XX declared in project
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version_XX"
}
And if you have a subproject:
dependencies {
// both kotlin_version and kotlin_version_XX can be used here, since it was declared in parent project
implementation "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
implementation "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version_XX"
}
buildscript {
// both kotlin_version and kotlin_version_XX can even be used here for subproject's script's use, since it was already declared in parent project
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version_XX"
}
I have the following in my build.gradle:
configurations {
runtime.exclude group: 'org.apache.spark'
runtime.exclude group: 'org.apache.hadoop'
}
and for some reason this also excludes all Hadoop/Spark code from the test classpath. If I comment out this configuration - the tests are passing fine, otherwise I get all sorts of java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/MiniDFSCluster$Builder issues.
I tried to use this:
test {
classpath += configurations.compile
}
No luck.
What am I missing here?
In gradle scoping, test inherits from runtime. Your test code is excluding the minicluster dependency because runtime excludes it.
See this diagram for the scope inheritance tree for the java plugin:
Instead of adding a global exclude to the runtime configuration, you might want to make the spark dependencies into compileOnly scoped dependencies which is available since gradle 2.12.
configurations {
compileOnly 'org.apache.spark:spark:2.11'
test 'org.apache.hadoop:hadoop-minicluster:2.7.2'
}
More information about gradle scoping is available in the gradle manual:
Alternatively, you could add another configuration that inherits from runtime, and add exclusions to that, then use that as the basis of your shadowJar. This could be helpful if you want to optionally build a jar with spark dependencies bundled in or not. Your tests will use the configuration without exclusions, but the jar you package won't include the spark dependencies.
configurations {
sparkConfiguration {
extendsFrom runtime
exclude group: 'org.apache.hadoop'
exclude group: 'org.apache.spark'
}
}
task sparkExcludedJar(type: ShadowJar) {
group = "Shadow"
configurations = [project.configurations.sparkConfiguration]
classifier = 'sparkExcluded'
}
To study Gradle I am using the book Gradle in action.
There was an example of dependency definition.
dependencies {
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.1'
}
But when I do in console gradle build I've got an error
What is the problem? My whole .gradle file looks like this
apply plugin: 'java'
dependencies {
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.1'
}
You did not tell Gradle where to find commons-lang3 library. Easy fix is to add the following to your build script:
repositories {
mavenCentral()
}
Of course you can find this piece of information in documentation - http://www.gradle.org/docs/current/userguide/artifact_dependencies_tutorial.html#N10608
I was facing this same issue. I fixed it by using local gradle distribution instead of the default gradle wrapper. This is how it goes, make sure that you have Gradle installed and setup(PATH variable).
Open IntelliJ. Goto File->setting->Build, Exec, Deployment->Build tools->Gradle and use local gradle distribution and add your Gradle Home. Apply changes and try to run it now.
I have a simple project where I'd like to unjar a subset of my dependencies and pack them into the output jar.
I have the two configurations:
configurations {
embed
all
}
dependencies {
embed group: 'commons-collections', name: 'commons-collections', version: '3.2'
...
all embed
all group: 'something-not-embeddable', name: 'dontembed'
compile all
}
According to http://www.gradle.org/docs/current/userguide/dependency_management.html 50.5 Working with Dependencies section's example it should work.
In a later section of my build, I want to unjar the embed jars and use them as source for jar.
My problem is that the gradle output says:
> Could not find method all() for arguments [configuration ':embed'] on root project 'myproject'.
Can you tell me why my approach is not working and how could I fix it?
Lol, looks like I chose a bad configuration name, works with alldeps instead of all