The typical Kotlin configuration in a Gradle project is very boilerplate, and I'm looking for a way of abstracting it out into an external build script so that it can be reused.
I have a working solution (below), but it feels like a bit of a hack as the kotlin-gradle-plugin doesn't work out of the box this way.
It's messy to apply any non-standard plugin from an external script as you can't apply the plugin by id, i.e.
apply plugin: 'kotlin' will result in Plugin with id 'kotlin' not found.
The simple (well, usually) workaround is to apply by the fully qualified classname of the plugin, i.e.
apply plugin: org.jetbrains.kotlin.gradle.plugin.KotlinPluginWrapper
which in this case throws a nice little exception indicating that the plugin probably wasn't meant to be called this way:
Failed to determine source cofiguration of kotlin plugin.
Can not download core. Please verify that this or any parent project
contains 'kotlin-gradle-plugin' in buildscript's classpath configuration.
So I managed to hack together a plugin (just a modified version of the real plugin) which forces it to find the plugin from the current buildscript.
kotlin.gradle
buildscript {
ext.kotlin_version = "1.0.3"
repositories {
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
}
dependencies {
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
}
apply plugin: CustomKotlinPlugin
import org.jetbrains.kotlin.gradle.plugin.CleanUpBuildListener
import org.jetbrains.kotlin.gradle.plugin.KotlinBasePluginWrapper
import org.jetbrains.kotlin.gradle.plugin.KotlinPlugin
import org.jetbrains.kotlin.gradle.tasks.KotlinTasksProvider
/**
* Wrapper around the Kotlin plugin wrapper (this code is largely a refactoring of KotlinBasePluginWrapper).
* This is required because the default behaviour expects the kotlin plugin to be applied from the project,
* not from an external buildscript.
*/
class CustomKotlinPlugin extends KotlinBasePluginWrapper {
#Override
void apply(Project project) {
// use String literal as KOTLIN_COMPILER_ENVIRONMENT_KEEPALIVE_PROPERTY constant isn't available
System.setProperty("kotlin.environment.keepalive", "true")
// just use the kotlin version defined in this script
project.extensions.extraProperties?.set("kotlin.gradle.plugin.version", project.property('kotlin_version'))
// get the plugin using the current buildscript
def plugin = getPlugin(this.class.classLoader, project.buildscript)
plugin.apply(project)
def cleanUpBuildListener = new CleanUpBuildListener(this.class.classLoader, project)
cleanUpBuildListener.buildStarted()
project.gradle.addBuildListener(cleanUpBuildListener)
}
#Override
Plugin<Project> getPlugin(ClassLoader pluginClassLoader, ScriptHandler scriptHandler){
return new KotlinPlugin(scriptHandler, new KotlinTasksProvider(pluginClassLoader));
}
}
This can then be applied in any project (i.e. apply from: "kotlin.gradle") and you're up and running for Kotlin development.
It works, and I haven't had any issues yet, but I'm wondering if there is a better way? I'm not really keen on merging in changes to the plugin every time there's a new version of Kotlin.
Check out the nebula-kotlin-plugin. It seems very close to what you're trying to achieve there.
The problem here is that there is a known gradle bug about the inability to apply plugins by id from init scripts. That's why you need to use fully qualified class name as a workaround.
E.g. I have the following in the init script and it works:
apply plugin: org.jetbrains.kotlin.gradle.plugin.KotlinPlatformJvmPlugin
By the way, I created a gradle plugin for preparing custom gradle distributions with common setup defined in init script - custom-gradle-dist. It works perfectly for my projects, e.g. a build.gradle for a library project looks like this (this is a complete file, all repository, apply plugin, dependencies etc setup is defined in the init script):
dependencies {
compile 'org.springframework.kafka:spring-kafka'
}
Related
I'm trying to write a custom Gradle task class and use it in another subproject. I'm having problems tying the build together.
In this example, I have named the subprojects "a" (for application) and "p" (for plugin, although I'm not using a plugin object but just providing a task class to the build script).
settings.gradle
include 'p'
include 'a'
p/build.gradle
apply plugin: 'java'
apply plugin: 'groovy'
apply plugin: 'maven-publish'
group 'test'
version '1.0'
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
}
}
}
dependencies {
implementation gradleApi()
implementation localGroovy()
}
p/src/main/groovy/p/MyTask.groovy
package p
import org.gradle.api.DefaultTask;
import org.gradle.api.tasks.TaskAction;
class MyTask extends DefaultTask {
#TaskAction
void run() {
System.out.println('yay!');
}
}
a/build.gradle
buildscript {
repositories {
mavenLocal()
}
dependencies {
classpath 'test:p:1.0'
}
}
group = 'test'
version = '1.0'
apply plugin: 'java'
task myTask(type: p.MyTask) {
}
The "plugin" is built by running, inside the p folder:
../gradlew clean build publishToMavenLocal
Inside the a folder:
../gradlew myTask
prints "yay!"
However, while developing, bugs happen. When I simulate a bug in MyTask:
MyTask() {
throw new RuntimeException("an error");
}
and build the plugin (in folder p):
../gradlew clean build publishToMavenLocal
it fails as expected.
Now I "fix" the bug by removing the broken constructor again, and rebuild in folder p:
../gradlew clean build publishToMavenLocal
but this command fails with the same error.
As far as I understand, the reason is that:
the broken plugin is in my local maven repo
trying to build the plugin detects the settings.gradle in the parent folder
gradle tries to configure all projects referenced from settings.gradle
this loads the broken plugin
the build fails
To verify, I comment out the include line for a in settings.gradle, and it works again. Reverting settings.gradle, and it still works, because now the "fixed" plugin is in my maven repo, and rebuilding the plugin will just overwrite it again with the working version.
The bottom line is that bugs in my custom task class (or custom plugin, or any other buildscript code) have the potential to make themself un-compilable, with the workaround being to edit or temporarily rename settings.gradle. The more complex the project gets, the more cumbersome this becomes: Renaming does not work if the plugin code itself contains multiple subprojects, and even commenting out becomes "commenting out the right lines".
What is the intended way to fix this problem?
Complex logic for a single (multi-) project is best organized in buildSrc. You can for the most part regard it as a normal sub-project but just for the build classpath instead. Plugins and tasks that you create here are automatically available for all projects in the multi-project.
If, for some reason, you rather continue to work with a local Maven repository, you could think about publishing stable releases of the plugin with a version number so it is easier to roll back.
I have created a custom Kotlin compiler plugin for Gradle. It was inspired by kotlin-allopen (2) and sample-kotlin-compiler-plugin, and is supposed to make all Kotlin classes non-final.
The problem is, I'm unable to use it in my projects, I only get the following:
Caused by: org.gradle.api.plugins.UnknownPluginException: Plugin with id 'no.synth.kotlin.plugins.kotlin-really-allopen' not found.
at org.gradle.api.internal.plugins.DefaultPluginManager.apply(DefaultPluginManager.java:131)
I have tried both the "new" plugin syntax:
plugins {
id "no.synth.kotlin.plugins.kotlin-really-allopen" version "0.1"
}
.. and the old one:
buildscript {
repositories {
mavenLocal()
}
dependencies {
classpath "no.synth.kotlin.plugins:kotlin-really-allopen:0.1"
}
}
apply plugin: "kotlin-really-allopen" // I've tried "no.synth.kotlin.plugins.kotlin-really-allopen" as well
So what am I doing wrong? Here's the plugin: https://github.com/henrik242/kotlin-really-allopen
EDIT: I have updated the repository with an example app and a README.md to easily reproduce the problem.
Your Gradle plugin doesn't seem to contain any entry under META-INF/gradle-plugins.
Gradle requires that every plugin ID is mapped to the implementation class, and this mapping is stored in META-INF/gradle-plugins resources.
To map the plugin ID kotlin-really-allopen, you would need a resource file
src/main/resources/META-INF/gradle-plugins/kotlin-really-allopen.properties.
See: Wiring for a custom plugin
You can also use the Gradle Plugin Development Plugin, which automatically generates these entries from the build script DSL.
Also, your repository doesn't seem to contain an actual Gradle plugin implementation, there's only the part that the compiler needs to load. For an example that contains the Gradle part too, take a look at kevinmost/debuglog.
Move apply plugin: "kotlin-really-allopen" in your build.gradle module app on top
I'm currently trying to improve the way our projects share their configuration. We have lots of different multi-module gradle projects for all of our libraries and microservices (i.e. many git repos).
My main goals are:
To not have my Nexus repository config duplicated in every project (also, I can safely assume that the URL won't change)
To make my custom Gradle plugins (published to Nexus) available to every project with minimal boilerplate / duplication (they should be available to every project, and the only thing the project cares about is the version it's using)
No magic - it should be obvious to developers how everything is configured
My current solution is a custom gradle distribution with an init script that:
adds mavenLocal() and our Nexus repository to the project repos (very similar to the Gradle init script documentation example, except it adds repos as well as validating them)
configures an extension that allows our gradle plugins to be added to the buildscript classpath (using this workaround). It also adds our Nexus repo as a buildscript repo as that's where the plugins are hosted. We have quite a few plugins (built upon Netflix's excellent nebula plugins) for various boilerplate: standard project setup (kotlin setup, test setup, etc), releasing, publishing, documentation, etc and it means our project build.gradle files are pretty much just for dependencies.
Here is the init script (sanitised):
/**
* Gradle extension applied to all projects to allow automatic configuration of Corporate plugins.
*/
class CorporatePlugins {
public static final String NEXUS_URL = "https://example.com/repository/maven-public"
public static final String CORPORATE_PLUGINS = "com.example:corporate-gradle-plugins"
def buildscript
CorporatePlugins(buildscript) {
this.buildscript = buildscript
}
void version(String corporatePluginsVersion) {
buildscript.repositories {
maven {
url NEXUS_URL
}
}
buildscript.dependencies {
classpath "$CORPORATE_PLUGINS:$corporatePluginsVersion"
}
}
}
allprojects {
extensions.create('corporatePlugins', CorporatePlugins, buildscript)
}
apply plugin: CorporateInitPlugin
class CorporateInitPlugin implements Plugin<Gradle> {
void apply(Gradle gradle) {
gradle.allprojects { project ->
project.repositories {
all { ArtifactRepository repo ->
if (!(repo instanceof MavenArtifactRepository)) {
project.logger.warn "Non-maven repository ${repo.name} detected in project ${project.name}. What are you doing???"
} else if(repo.url.toString() == CorporatePlugins.NEXUS_URL || repo.name == "MavenLocal") {
// Nexus and local maven are good!
} else if (repo.name.startsWith("MavenLocal") && repo.url.toString().startsWith("file:")){
// Duplicate local maven - remove it!
project.logger.warn("Duplicate mavenLocal() repo detected in project ${project.name} - the corporate gradle distribution has already configured it, so you should remove this!")
remove repo
} else {
project.logger.warn "External repository ${repo.url} detected in project ${project.name}. You should only be using Nexus!"
}
}
mavenLocal()
// define Nexus repo for downloads
maven {
name "CorporateNexus"
url CorporatePlugins.NEXUS_URL
}
}
}
}
}
Then I configure each new project by adding the following to the root build.gradle file:
buildscript {
// makes our plugins (and any others in Nexus) available to all build scripts in the project
allprojects {
corporatePlugins.version "1.2.3"
}
}
allprojects {
// apply plugins relevant to all projects (other plugins are applied where required)
apply plugin: 'corporate.project'
group = 'com.example'
// allows quickly updating the wrapper for our custom distribution
task wrapper(type: Wrapper) {
distributionUrl = 'https://com.example/repository/maven-public/com/example/corporate-gradle/3.5/corporate-gradle-3.5.zip'
}
}
While this approach works, allows reproducible builds (unlike our previous setup which applied a build script from a URL - which at the time wasn't cacheable), and allows working offline, it does make it a little magical and I was wondering if I could do things better.
This was all triggered by reading a comment on Github by Gradle dev Stefan Oehme stating that a build should work without relying on an init script, i.e. init scripts should just be decorative and do things like the documented example - preventing unauthorised repos, etc.
My idea was to write some extension functions that would allow me to add our Nexus repo and plugins to a build in a way that looked like they were built into gradle (similar to the extension functions gradleScriptKotlin() and kotlin-dsl() provided by the Gradle Kotlin DSL.
So I created my extension functions in a kotlin gradle project:
package com.example
import org.gradle.api.artifacts.dsl.DependencyHandler
import org.gradle.api.artifacts.dsl.RepositoryHandler
import org.gradle.api.artifacts.repositories.MavenArtifactRepository
fun RepositoryHandler.corporateNexus(): MavenArtifactRepository {
return maven {
with(it) {
name = "Nexus"
setUrl("https://example.com/repository/maven-public")
}
}
}
fun DependencyHandler.corporatePlugins(version: String) : Any {
return "com.example:corporate-gradle-plugins:$version"
}
With the plan to use them in my project's build.gradle.kts as follows:
import com.example.corporateNexus
import com.example.corporatePlugins
buildscript {
repositories {
corporateNexus()
}
dependencies {
classpath(corporatePlugins(version = "1.2.3"))
}
}
However, Gradle was unable to see my functions when used in the buildscript block (unable to compile script). Using them in the normal project repos/dependencies worked fine though (they are visible and work as expected).
If this worked, I was hoping to bundle the jar into my custom distribution , meaning my init script could just do simple validation instead of hiding away the magical plugin and repo configuration. The extension functions wouldn't need to change, so it wouldn't require releasing a new Gradle distribution when plugins change.
What I tried:
adding my jar to the test project's buildscript classpath (i.e. buildscript.dependencies) - doesn't work (maybe this doesn't work by design as it doesn't seem right to be adding a dependency to buildscript that's referred to in the same block)
putting the functions in buildSrc (which works for normal project deps/repos but not buildscript, but is not a real solution as it just moves the boilerplate)
dropping the jar in the lib folder of the distribution
So my question really boils down to:
Is what I'm trying to achieve possible (is it possible to make custom classes/functions visible to the buildScript block)?
Is there a better approach to configuring a corporate Nexus repo and making custom plugins (published to Nexus) available across lots of separate projects (i.e. totally different codebases) with minimal boilerplate configuration?
If you want to benefit from all the Gradle Kotlin DSL goodness you should strive to apply all plugins using the plugins {} block. See https://github.com/gradle/kotlin-dsl/blob/master/doc/getting-started/Configuring-Plugins.md
You can manage plugin repositories and resolution strategies (e.g. their version) in your settings files. Starting with Gradle 4.4 this file can be written using the Kotlin DSL, aka settings.gradle.kts. See https://docs.gradle.org/4.4-rc-1/release-notes.html.
With this in mind you could then have a centralized Settings script plugin that sets things up and apply it in your builds settings.gradle.kts files:
// corporate-settings.gradle.kts
pluginManagement {
repositories {
maven {
name = "Corporate Nexus"
url = uri("https://example.com/repository/maven-public")
}
gradlePluginPortal()
}
}
and:
// settings.gradle.kts
apply(from = "https://url.to/corporate-settings.gradle.kts")
Then in your project build scripts you can simply request plugins from your corporate repository:
// build.gradle.kts
plugins {
id("my-corporate-plugin") version "1.2.3"
}
If you want your project build scripts in a multi-project build to not repeat the plugin version you can do so with Gradle 4.3 by declaring versions in your root project. Note that you also could set the versions in settings.gradle.kts using pluginManagement.resolutionStrategy if having all builds use the same plugins version is what you need.
Also note that for all this to work, your plugins must be published with their plugin marker artifact. This is easily done by using the java-gradle-plugin plugin.
I promised #eskatos that I would come back and give feedback on his answer - so here it is!
My final solution consists of:
Gradle 4.7 wrapper per project (pointed at a mirror of http://services.gradle.org/distributions setup in Nexus as a raw proxy repository, i.e. it's vanilla Gradle but downloaded via Nexus)
Custom Gradle plugins published to our Nexus repo along with plugin markers (generated by the Java Gradle Plugin Development Plugin)
Mirroring the Gradle Plugin Portal in our Nexus repo (i.e. a proxy repo pointing at https://plugins.gradle.org/m2)
A settings.gradle.kts file per project that configures our maven repo and gradle plugin portal mirror (both in Nexus) as plugin management repositories.
The settings.gradle.kts file contains the following:
pluginManagement {
repositories {
// local maven to facilitate easy testing of our plugins
mavenLocal()
// our plugins and their markers are now available via Nexus
maven {
name = "CorporateNexus"
url = uri("https://nexus.example.com/repository/maven-public")
}
// all external gradle plugins are now mirrored via Nexus
maven {
name = "Gradle Plugin Portal"
url = uri("https://nexus.example.com/repository/gradle-plugin-portal")
}
}
}
This means that all plugins and their dependencies are now proxied via Nexus, and Gradle will find our plugins by id as the plugin markers are published to Nexus as well. Having mavenLocal in there as well facilitates easy testing of our plugin changes locally.
Each project's root build.gradle.kts file then applies the plugins as follows:
plugins {
// plugin markers for our custom plugins allow us to apply our
// plugins by id as if they were hosted in gradle plugin portal
val corporatePluginsVersion = "1.2.3"
id("corporate-project") version corporatePluginsVersion
// 'apply false` means this plugin can be applied in a subproject
// without having to specify the version again
id("corporate-publishing") version corporatePluginsVersion apply false
// and so on...
}
And configures the gradle wrapper to use our mirrored distribution, which when combined with the above means that everything (gradle, plugins, dependencies) all come via Nexus):
tasks {
"wrapper"(Wrapper::class) {
distributionUrl = "https://nexus.example.com/repository/gradle-distributions/gradle-4.7-bin.zip"
}
}
I was hoping to avoid the boilerplate in the settings files using #eskatos's suggestion of applying a script from a remote URL in settings.gradle.kts. i.e.
apply { from("https://nexus.example.com/repository/maven-public/com/example/gradle/corporate-settings/1.2.3/corporate-settings-1.2.3.kts" }
I even managed to generate a templated script (published alongside our plugins) that:
configured the plugin repos (as in the above settings script)
used a resolution strategy to apply the version of the plugins associated with the script if the requested plugin id was one of our plugins and the version wasn't supplied (so you can just apply them by id)
However, even though it removed the boilerplate, it meant our builds were reliant on having a connection to our Nexus repo, as it seems that even though scripts applied from a URL are cached, Gradle does a HEAD request anyway to check for changes. It also made it annoying to test plugin changes locally, as I had to point it manually at the script in my local maven directory. With my current config, I can simply publish the plugins to maven local and update the version in my project.
I'm quite happy with the current setup - I think it's far more obvious to developers now how the plugins are applied. And it's made it far easier to upgrade Gradle and our plugins independently now that there's no dependency between the two (and no custom gradle distribution required).
I've been doing something like this in my build
buildscript {
project.apply {
from("${rootProject.projectDir}/sharedValues.gradle.kts")
}
val configureRepository: (Any) -> Unit by extra
configureRepository.invoke(repositories)
}
In my sharedValues.gradle.kts file I have code like this:
/**
* This method configures the repository handler to add all of the maven repos that your company relies upon.
* When trying to pull this method out of the [ExtraPropertiesExtension] use the following code:
*
* For Kotlin:
* ```kotlin
* val configureRepository : (Any) -> Unit by extra
* configureRepository.invoke(repositories)
* ```
* Any other casting will cause a compiler error.
*
* For Groovy:
* ```groovy
* def configureRepository = project.configureRepository
* configureRepository.invoke(repositories)
* ```
*
* #param repoHandler The RepositoryHandler to be configured with the company repositories.
*/
fun repositoryConfigurer(repoHandler : RepositoryHandler) {
repoHandler.apply {
// Do stuff here
}
}
var configureRepository : (RepositoryHandler) -> Unit by extra
configureRepository = this::repositoryConfigurer
I follow a similar patter for configuring the resolution strategy for plugins.
The nice thing about this pattern is that anything you configure in sharedValues.gradle.kts can also be used from your buildSrc project meaning that you can reuse repository declarations.
Updated:
You can apply another script from a URL, for example doing this:
apply {
// This was actually a plugin that I used at one point.
from("http://dl.bintray.com/shemnon/javafx-gradle/8.1.1/javafx.plugin")
}
Simply host your script that you want all your builds to share on some http server (would highly recommend using HTTPS so your build can't be targeted by a man in the middle attack).
The downside of this is that I don't think that scripts applied from urls aren't cached so they will be re-downloaded every time you run your build.
This may have been fixed by now, I'm not certain.
A solution offered to me by Stefan Oehme when I was having a similar problem was to vendor my own custom distribution of Gradle. According to him this is a common thing to do at large companies.
Simply create a custom fork of the gradle repo, add your companies special sauce to every project using this custom version of gradle.
I encountered a similar problem when common config is replicated in each and every project. Solved it by a custom gradle distribution with the common settings defined in init script.
Created a gradle plugin for preparing such custom distributions - custom-gradle-dist. It works perfectly for my projects, e.g. a build.gradle for a library project looks like this (this is a complete file):
dependencies {
compile 'org.springframework.kafka:spring-kafka'
}
I have two projects (in a single git repository) that should have the same
repository {
}
section in their build.gradle.kts, but otherwise are completely unrelated.
Can I factor this common part out and include it in each respective build.gradle.kts? How?
Update In the 0.11.0 release, applyFrom(uri) was removed.
You should now instead use:
apply {
from("dir/myfile.gradle")
}
Old answer
With Groovy build scripts you can do something like apply from: 'dir/myfile.gradle' where dir/myfile.gradle is a file containing your shared repositories block.
In a similar fashion with Gradle Script Kotlin (at least with 0.4.1), you can use the applyFrom(script: Any) method.
build.gradle.kts
applyFrom("dir/myfile.gradle")
If you need to apply it from a subproject you could do something like:
applyFrom("${rootProject.rootDir}/dir/myfile.gradle")
No idea if it works with kotlin however you can try equivalent from plain gradle:
lol.gradle
apply plugin: 'java'
repositories {
mavenCentral()
}
build.gradle
apply from: 'lol.gradle'
Above works fine. Mind that lol.gradle has java plugin applied - it adds context where repositories is present hence can be applied.
We use an init script bundled in a custom gradle distribution to apply our corporate Nexus repository to every gradle project. It's worth considering if you have a lot of projects.
I encountered a similar problem when common config is replicated in each and every project. Solved it by a custom gradle distribution with the common settings defined in init script.
Created a gradle plugin for preparing such custom distributions - custom-gradle-dist. It works perfectly for my projects, e.g. a build.gradle for a library project looks like this (this is a complete file, all repository, plugin, common dependencies etc are defined in the custom init script):
dependencies {
compile 'org.springframework.kafka:spring-kafka'
}
I'm creating gradle custom plugin and one of my tasks needs to be sure that another plugin applied to same project. Because it will operate on top of it.
I want for users of my plugin to avoid setting up an explicit dependency to another plugin - I want to do it inside my plugin.
So, I want to have this plugin (https://plugins.gradle.org/plugin/org.hidetake.ssh) applied. It's my dependency.
The way how I create plugin - I just create a class code on groovy, put it in buildSrc\src\main\groovy and apply groovy plugin in project. So my custom plugin is visible to gradle on build phase. It works, I have few other plugins done this way for same project, so it's fine for now.
I've looked through other topics and google for same question, but I can not make this work for me. This how I apply the code:
void apply(Project project) {
project.buildscript {
repositories {
maven {
url "https://plugins.gradle.org/m2/"
}
}
dependencies {
classpath "org.hidetake:gradle-ssh-plugin:1.1.3"
}
}
project.apply plugin: "org.hidetake.ssh"
...
The error message I got: Plugin with id 'org.hidetake.ssh' not found.
I tried to do it via gradle api also using project.repositories.mavenCentral() and project.dependencies.add and project.apply(plugin:'org.hidetake.ssh') then - doesn't work also - same error message. Tried to use long notation in project.dependencies.add("myConfig",[group:'org.hidetake', name:'gradle-ssh-plugin', version:'1.1.3']) - no result.
Appreciate if someone can guide to the correct syntax\way to make it work.
Ok, finally I got it. To solve the issue you need to do the following:
Place build.gradle in your buildSrc directory.
Declare dependency for the plugin as runtime. Like this:
repositories {
jcenter()
}
dependencies {
runtime 'org.hidetake:gradle-ssh-plugin:2.6.0'
}
Apply plugin explicitly in your own plugin definition. Like this:
void apply(Project project) {
project.pluginManager.apply('org.hidetake.ssh')
...