Configuring a custom Gradle sourceSet using a closure - gradle

I'm trying to develop a Gradle plugin for a language I use (SystemVerilog). I'm still experimenting and figuring things out. Before I write the entire thing as a plugin, I thought it would be best to try out the different parts I need inside a build script, to get a feel of how things should work.
I'm trying to define a container of source sets, similar to how the Java plugin does it. I'd like to be able to use a closure when configuring a source set. Concretely, I'd like to be able to do the following:
sourceSets {
main {
sv {
include '*.sv'
}
}
}
I defined my own sourceSet class:
class SourceSet implements Named {
final String name
final ObjectFactory objectFactory
#Inject
SourceSet(String name, ObjectFactory objectFactory) {
this.name = name
this.objectFactory = objectFactory
}
SourceDirectorySet getSv() {
SourceDirectorySet sv = objectFactory.sourceDirectorySet('sv',
'SystemVerilog source')
sv.srcDir("src/${name}/sv")
return sv
}
SourceDirectorySet sv(#Nullable Closure configureClosure) {
configure(configureClosure, getSv());
return this;
}
}
I'm using org.gradle.api.file.SourceDirectorySet because that already implements PatternFilterable, so it should give me access to include, exclude, etc.
If I understand the concept correctly, the sv(#Nullable Closure configureClosure) method is the one that gives me the ability to write sv { ... } to configure via a closure.
To add the sourceSets property to the project, I did the following:
project.extensions.add("sourceSets",
project.objects.domainObjectContainer(SourceSet.class))
As per the Gradle docs, this should give me the possibility to configure sourceSets using a closure. This site, which details using custom types, states that by using NamedDomainObjectContainer, Gradle will provide a DSL that build scripts can use to define and configure elements. This would be the sourceSets { ... } part. This should also be the sourceSets { main { ... } } part.
If I create a sourceSet for main and use it in a task, then everything works fine:
project.sourceSets.create('main')
task compile(type: Task) {
println 'Compiling source files'
println project.sourceSets.main.sv.files
}
If I try to configure the main source set to only include files with the .sv extension, then I get an error:
sourceSets {
main {
sv {
include '*.sv'
}
}
}
I get the following error:
No signature of method: build_47mnuak4y5k86udjcp7v5dkwm.sourceSets() is applicable for argument types: (build_47mnuak4y5k86udjcp7v5dkwm$_run_closure1) values: [build_47mnuak4y5k86udjcp7v5dkwm$_run_closure1#effb286]
I don't know what I'm doing wrong. I'm sure it's just a simple thing that I'm forgetting. Does anyone have an idea of what that might be?

I figured out what was going wrong. It was a combination of poor copy/paste skills and the fact that Groovy is a dynamic language.
First, let's look at the definition of the sv(Closure) function again:
SourceDirectorySet sv(#Nullable Closure configureClosure) {
configure(configureClosure, getSv());
return this;
}
Once I moved this code to an own Groovy file and used the IDE to show me what is getting called, I noticed that it wasn't calling the function I expected. I was expecting a call to org.gradle.util.ConfigureUtil.configure. Since this is part of the public API, I expected it to be imported by default in the build script. As this page states, this is not the case.
To solve the issue, it's enough to add the following import:
import static org.gradle.util.ConfigureUtil.configure
This will get rid of the cryptic closure related error. It is replaced by the following error, though:
Cannot cast object 'SourceSet_Decorated#a6abab9' with class 'SourceSet_Decorated' to class 'org.gradle.api.file.SourceDirectorySet'
This is caused by the copy/paste error I mentioned. When I wrote the SourceSet class, I drew heavily from org.gradle.api.tasks.SourceSet (and org.gradle.api.internal.tasks.DefaultSourceSet). If we look at the java(Closure) method there, we'll see it has the following signature:
SourceSet java(#Nullable Closure configureClosure);
Notice that it returns SourceSet and not SourceDirectorySet like in my code. Using the proper return type fixes the issue:
SourceSet sv(#Nullable Closure configureClosure)
With this new return type, let's look again at the configuration code for the source set:
sourceSets {
main {
sv {
include '*.sv'
}
}
}
Initially, I thought it was supposed to work as follows: pass main { ... } as a Closure to sourceSets, pass sv { ... } as a Closure to main, and handle the include ... part inside sourceDirectorySet. I banged my head against the wall for a while, because I couldn't find any code in that class hierarchy that takes closures like this.
Now, I think the flow is slightly different: pass main { ... } as a Closure to sourceSets (as initially thought), but call the sv(Closure) function on main (of type sourceSet), passing it { include ... } as the argument.
Bonus: There was one more issue that wasn't related to the "compile" errors I was having.
Even after getting the code to run without errors, it still wasn't behaving as expected. I had some files with the *.svh extension that were still getting picked up. This is because, when calling getSv(), it was creating a new SourceDirectorySet each time. Any configuration that was done previously was getting thrown away each time that this function was called.
Making the sourceDirectorySet a class member and moving its creation to the constructor fixed the issue:
private SourceDirectorySet sv
SourceSet(String name, ObjectFactory objectFactory) {
// ...
sv = objectFactory.sourceDirectorySet('sv',
'SystemVerilog source')
sv.srcDir("src/${name}/sv")
}
SourceDirectorySet getSv() {
return sv
}

Related

On passing a closure to a gradle extension, why the owner of closure is not the main Projects object?

I was looking at the closure scopes and found the output counter intuitive, this code was in build.gradle file
plugins {
id 'java'
}
sourceSets {
println 'source sets closure scopes this,owner, delegate'
println this.toString() // output: root project 'multigradle'
println owner.toString() // output: DynamicObject for SourceSet container
println delegate.toString() // output: SourceSet container
}
Why the owner is not equal to this, does gradle clone the closure?
PS : For anyone who will read it in future 'multigradle' is my gradle project name.
TL;DR;
Essentially within the gradle source there is a method something like this:
public Object sourceSets(Closure closure) {
// do stuff like configuring the closure
closure.call()
}
so when you call:
sourceSets {
some code
}
(which incidentally is the same as calling sourceSets({ some code }), just with the parens removed which is ok in groovy)
"some code" is not executed immediately when the sourceSets method is called. Gradle can choose to execute it whenever they decide it's time. Specifically, they can (and do) configure things like owner and delegate before actually executing the closure.
Longer version
Turns out the sourceSets method in your build.gradle file is actually added by plugins such as the java/kotlin/groovy plugins.
As an example we can look at the java plugin and the DefaultJavaPluginConvention class which has the following code:
private final SourceSetContainer sourceSets;
#Override
public Object sourceSets(Closure closure) {
return sourceSets.configure(closure);
}
this is the method that gets called when you type sourceSets { ... } in your build.gradle file. It gets handed the closure and proceeds to hand it off to the configure method of the source set container. Note that we have not executed the closure yet, we are just passing it around as a non-executed block of code.
If we dig a little, we find the configure method in the AbstractNamedDomainObjectContainer class:
public AbstractNamedDomainObjectContainer<T> configure(Closure configureClosure) {
ConfigureDelegate delegate = createConfigureDelegate(configureClosure);
ConfigureUtil.configureSelf(configureClosure, this, delegate);
return this;
}
(SourceSetContainer is an interface and the implementing class inherits from AbstractNamedDomainObjectContainer...this is the right configure method)
Where the ConfigureUtil has the following code:
public static <T> T configureSelf(#Nullable Closure configureClosure, T target, ConfigureDelegate closureDelegate) {
if (configureClosure == null) {
return target;
}
configureTarget(configureClosure, target, closureDelegate);
return target;
}
private static <T> void configureTarget(Closure configureClosure, T target, ConfigureDelegate closureDelegate) {
if (!(configureClosure instanceof GeneratedClosure)) {
new ClosureBackedAction<T>(configureClosure, Closure.DELEGATE_FIRST, false).execute(target);
return;
}
// Hackery to make closure execution faster, by short-circuiting the expensive property and method lookup on Closure
Closure withNewOwner = configureClosure.rehydrate(target, closureDelegate, configureClosure.getThisObject());
new ClosureBackedAction<T>(withNewOwner, Closure.OWNER_ONLY, false).execute(target);
}
where the relevant part is the call to the groovy Closure rehydrate method which according to docs does the following:
Returns a copy of this closure for which the delegate, owner and thisObject are replaced with the supplied parameters. Use this when you want to rehydrate a closure which has been made serializable thanks to the dehydrate() method.
Only on the last line of the configureTarget method does gradle call execute on the action created to represent the closure. So the execution of the closure is done after the owner, the delegate and the this pointer have been configured according to gradle needs.

In gradle kotlin dsl, how to call a dynamic test extension?

I'm trying to add a new configuration option when using the gradle ScalaTest plugin:
https://github.com/maiflai/gradle-scalatest
In its source code, the config was injected into the Test class as a dynamic extension:
static void configure(Test test) {
...
Map<String, ?> config = [:]
test.extensions.add(ScalaTestAction.CONFIG, config)
test.extensions.add("config", { String name, value -> config.put(name, value) })
test.extensions.add("configMap", { Map<String, ?> c -> config.putAll(c) })
...
}
If using groovy as the dsl, calling this property is easy:
test {
configMap([
'db.name': 'testdb'
'server': '192.168.1.188'
])
}
unfortunately the kotlin dsl can't use this method due to static typing, when being invoked as a test plugin, it is clearly visible within the test scope, e.g. when using extensions.getByName:
tasks {
test {
val map = extensions.getByName("configMap")
println(map)
}
}
It yields the following output:
...
> Configure project :
com.github.maiflai.ScalaTestPlugin$_configure_closure6#45c21cac
But there is no way to retrieve or assert its type in compile time, and it ends up being useless (unless reflection is used, which is against the design philosophy of kotlin dsl). Is there a easy way for kotlin dsl to achieve the same?
I saw in the Scala test gradle plugin that the dynamic extension is defined like this:
test.extensions.add("configMap", { Map<String, ?> c -> config.putAll(c) })
The com.github.maiflai.ScalaTestPlugin$_configure_closure6#45c21cac you saw should be a closure of type (Map<String, Any>) -> Unit, which means you can do that. We'll have to change the map values so let's assume that it's also mutable.
extensions.getByName("configMap").closureOf<MutableMap<String, Any?>> {
this["db.name"] = "testdb"
this["server"] = "192.168.1.188"
}
This builds fine but I don't have Scala installed and never used Scala test. I have no idea if it actually works, so please tell me.

Understand Gradle-Groovy semantics

I am having a very hard time understanding the semantics of gradle scripts w.r.t how they are seen in groovy.
1) What does the following snippet mean?
task copy(type: Copy) {
into "target"
with baseSpec
}
As I understand it, it seems to me that task is instantiated with a named parameter "type" and it's value "Copy". I have no idea what is "into", "with". Are they parameters of the task class? BTW, Is task a class or interface?
2) What is a "script block"? Is it a closure?
3) What is an "Action"? Are they also closures or objects of interface instantiated with anonymous class?
Basically, I am lost how to put all of this together as a plain groovy ?
Groovy is a powerful language for building DSL (Domain Specific Language). Gradle use this as many others libraries.
It's based on several properties of Groovy
Parenthesis are optionals
fun("myparameter")
fun "myparameter"
You can have named parameters on a method
fun prop:'value', otherprop:'othervalue'
fun([prop:'value', otherprop:'othervalue'])
If the last parameters of a method is a closure, it can be written outside the method call
fun(prop:'value') {
//..closure call
}
fun([prop:'value'], { /*closure*/ })
You can get/set any property or invoke any method on a groovy object : you can add behavior dynamically, through missingMethod, missingProperty, getProperty or setProperty, ..
object.somefun "42"
object.missingMethod("somefun", ["42"])
In a closure, you have a special object, called delegate. it can be setted at runtime, and any non-local property or method invocation can be delegated to this delegate
def fun = { copy "this_file" }
def fun = { delegate.copy("this_file") }
See this documentation or the Builder pattern
with this properties, your script can be written (it's not really true because of AST transformation..) :
task(copy([type: Copy], { it ->
delegate.into("target")
delegate.with(baseSpec)
}))
delegate is an object which implement missingMethod, and generate objects based on the method call and the context.
a more complexe script :
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath group: 'commons-codec', name: 'commons-codec', version: '1.2'
}
}
is equivalent to :
buildscript({ it ->
delegate.repositories({delegate.mavenCentral()})
delegate.dependencies({delegate.classpath([group:'commons-codec', name:'commons-codec', version:'1.2'])})
})

Gradle extension as NamedDomainObjectContainer of Closure

I'm trying to build a Gradle plugin that would allow the following:
myPluginConfig {
something1 {
// this is a closure
}
somethingElse {
// this is another closure
}
// more closures here
}
To achieve this I'm fairly certain I need to use a NamedDomainObjectContainer to wrap a Closure collection, so I've set up the following plugin:
class SwitchDependenciesPlugin implements Plugin<Project> {
void apply(Project project) {
// add the extension
project.getExtensions().myPluginConfig = project.container(Closure)
// read the current configuration
NamedDomainObjectContainer<Closure> config = project.myPluginConfig
// test it out, always returns []
System.out.println(config)
}
}
What am I doing wrong, do I need to use project.extensions.create instead? If so, how?
EDIT: my use case consists in adding dependencies according to some variables defined in the project hierarchy. For example, the following configuration would add the red project if the variable red is defined on project.ext, or gson otherwise:
myPluginConfig {
redTrue {
compile project(':red')
}
redFalse {
compile 'com.google.code.gson:gson:2.4'
}
greenTrue {
compile project(':green')
}
}
For this use case I need to have dynamic names for myPluginConfig, and therefore either a Map or a NamedDomainObjectContainer.
Can you elaborate what you try to model here? I think you have two options. One is to use NamedDomainObjectContainer. Here you need a class that represents "something". Have a look at the Gradle userguide chapter about maintaining multiple domain objects (see https://docs.gradle.org/current/userguide/custom_plugins.html#N175CF) in the sample of the userguide, the "thing" is 'Book'. The build-in configuration syntax like you described above comes for free.
If you want to have a syntax like above without the need for maintaining multiple domain objects, you can simply add a method that takes a Closure as a parameter to your Extension class:
void somethingToConfigure(Closure) {
}
You cannot have Closure as a type for NamedDomainObjectContainer simply because the type you use must have a property called name and a public constructor with a single String parameter.
To overcome this, you may create a wrapper around Closure with a name field added.

Multiple ways of using custom Gradle plugins in build.gradle

With reference to the Gradle docs section 59.2 I have created a simple plugin to illustrate various (seemingly working) ways to use a custom Gradle plugin's DSL exposed via plugin extensions. For example the following plugin definition and class
class GreetingPlugin implements Plugin<Project> {
void apply(Project project) {
// Add the 'greeting' extension object
project.extensions.create("greeting", GreetingPluginExtension)
// Add a task that uses the configuration
project.task('hello') << {
println project.greeting.message
}
}
}
class GreetingPluginExtension {
def String message = 'Hello from GreetingPlugin'
}
can be called in four ways
greeting.message 'Hi from Gradle with dot'
greeting.message = 'Hi from Gradle with dot and assigment'
greeting { message 'Hi from Gradle with block' }
greeting { message = 'Hi from Gradle with block with assignment' }
what is the correct and recommended way? Are there implications of using one way over another? In that simple example they all appear to work
As Opal said, all 4 ways are good, depending on your requirements in readability. If you have more things to configure than just the message, then the configuration block using a closure may be more convenient.
There's also a difference between using the assignment operator and omitting it: when using the assignment operator, you are explicitly setting a property, while omitting it means calling a method with that name. In that case, I prefer using the assignment operator.
You can have a look here: http://groovy-lang.org/style-guide.html#_getters_and_setters

Resources