So, I have been trying to write JUnit tests for my APIs which interact with Elasticsearch. This is like Integration test where I need to setup Elasticsearch before I can run my code.
For all the tests, I need to create a test task which will do following:
Download the zip from
compile group: 'org.elasticsearch.distribution.zip', name: 'elasticsearch', version: '6.1.1', ext: 'pom'
Run elasticsearch executable present in /bin of unzipped file. When running this executable, take as argument elasticsearch.yml file in the command.
Once the all the tests are run, stop the elasticsearch executable, and clean the zipped folder.
Things that I can improve is that if zip file is already present in my gradle cache, then don't download it again and again.
Thanks
Add following dependency to your build.gradle file
configurations {
elasticDist
}
...
dependencies {
elasticDist group: 'org.elasticsearch.distribution.zip', name: 'elasticsearch', version: '6.1.2', ext: 'zip'
}
Add tasks to unzip and cleanup, setup dependencies
task unzip(type: Copy) {
// to download distribution
dependsOn configurations.elasticDist
from { // use of closure defers evaluation until execution time
configurations.elasticDist.collect { zipTree(it) }
}
into file("${buildDir}/your/destination")
}
task cleanElastic(type:Delete) {
delete file("${buildDir}/your/destination")
}
test.dependsOn('unzip')
test.finalizedBy('cleanElastic')
Using your test framework of choice, configure setUp and tearDown to start and stop elastic respectively.
Related
Another strange behaviour of gradle...
So I've found this post:
Gradle exclude module for Copy task
Totally fine and works like a charm to exclude things from copying.
But here is where it gets interesting. This is how my Copy Task looks:
task copyDependencies(type: Copy) {
into "$buildDir/libs/dependencies"
from configurations.runtime {
exclude module: 'groovy'
exclude module: 'aws-java-sdk-s3'
exclude module: 'commons-io'
}
}
If I try to run the Application through Gradles 'application run' task. It fails with "Main Class xxx couldn't be found or loaded". Digging deeper into the problem I noticed that Groovy couldn't be resolved.
I don't even run this Task, or depend on it.
But if I comment out line 4 like this:
task copyDependencies(type: Copy) {
into "$buildDir/libs/dependencies"
from configurations.runtime {
//exclude module: 'groovy'
exclude module: 'aws-java-sdk-s3'
exclude module: 'commons-io'
}
}
The Application starts like normal, until it reaches a point where it needs Commons-IO. I still want to use this copyDependencies Task at other times, without changing the code there though.
Can somebody explain me this behaviour ?
I imagine manipulating the configuration.runtime anywhere in the gradle file, changes it for every other task ?
In your from configuration block, you are referencing the runtime configuration, but in the same time you are altering this configuration by adding some exclusion rules. This will alter the original (and unique) runtime configuration which will be used by all other tasks in your build project, as you have guessed. This explains the "Main Class xxx couldn't be found or loaded" error you get when trying to execute the run task, since the runtime configuration (classpath) does not contain the needed library.
If you want to write exclusions rules by group and/or module in your copyDependencies task, one possible way would be to work on a copy of the original runtime configuration; you could define a new Configuration for this purpose:
configurations{
runtimeDeps.extendsFrom runtime
}
task copyDependencies(type: Copy) {
into "$buildDir/libs/dependencies"
from configurations.runtimeDeps {
exclude module: 'groovy'
exclude module: 'aws-java-sdk-s3'
exclude module: 'commons-io'
}
}
I'm new to Gradle and I have no past experience for Gradle builds. What I'm trying to achieve are compile Java classes, copy some files to build dir with some resource files and make a zip file of the whole directory.
Zip file contents
bin
----data
----java
----properties
----resources
----run.sh
----classes <---- Java compiles classes here!
lib
----sqlite-jdbc-3.23.1.jar
properties
----MANIFEST.MF
I tried to Gradle build for Java but it makes too much tasks for this and I think it's overkill for my purposes. I want to write simple compile and copy task for this. Or if I need to use all java build tasks, how can I edit those task operations?
I think you should use both Java and Distribution plugins, that will help you to create a simple build script. Java plugin will bring you some tasks that you might not need in you case, but it still worth using it instead of creating your own task for source compilation.
An example of script you could use as a starter:
plugins {
id 'java'
id 'distribution'
}
// Java plugin configuration
// ---------------------------------------------------
sourceCompatibility = 1.8
targetCompatibility = 1.8
repositories {
jcenter()
}
dependencies {
compile group: 'org.xerial', name: 'sqlite-jdbc', version: '3.23.1'
}
// Distribution plugin configuration
// ------------------------------------------------------
distributions {
main {
baseName = 'myDist'
contents {
// 1. by default, content under /src/main/dist will be automatically included in the target dist zip
// 2. include compiled classes output dir into /bin/classes
from(project.sourceSets.main.output.classesDir) {
into('bin/classes/')
}
// 3. include all dependent jars into /libs
from(configurations.compile) {
into('libs/')
}
// 4. other resources to include in the target zip
// ...
}
}
}
// Make distZip task depend on compileJava task
distZip {
dependsOn compileJava
// EDIT : change target file path
eachFile { file ->
String path = file.relativePath
String newPath = path.substring(path.indexOf("/")+1,path.length())
println "Changed path to file from $path to $newPath"
file.setPath(newPath)
}
}
This script assumes that you follow standard directory layout in your project, for your source code and resources ( src/main/java/ , src/main/dist/ , ..), else you will have to adapt the script.
I need to download a file from Nexus through Gradle but the file 'ext' is null. Here is my build.gradle file:
configurations {
iewin64driver
chromelinuxdriver}
dependencies {
iewin64driver "com.seleniumdrivers:ie-driver:latest.release:win64#exe"
chromelinuxdriver "com.seleniumdrivers:chrome-driver:latest.release:linux"
}
//Download selenium drivers from Nexus to libs directory and rename
task downloadDrivers(type: Copy) {
//IEDriver Win64
from configurations.iewin64driver
into file("/libs")
rename '(.*)win64(.*)', "iedriver.exe"
//ChromeDriver Linux
from configurations.chromelinuxdriver
into file("/libs")
rename 'chrome-driver(.*)linux', "chromedriver-linux"
println "completed download drivers"
}
While the ieWindriver64 which refers to filename "ie-driver-3.9.0-win32.exe" downloads without any issues, the second one chromelinuxdriver does not download because the filename to be downloaded is "chrome-driver-2.37-linux"
Note this file does not have any extension.
By default gradle looks for .jar if extension is not provided and returns error message:
"Could not find chrome-driver-linux.jar"
I also tried different options like but doesn't work:
dependencies {
chromelinuxdriver group: "com.seleniumdrivers", name: "chrome-driver",
version: "2.37", classifier: "linux", ext: null }
It may not be possible. This plugin, which helps incorporate WebDriver into a Gradle build, downloads the drivers outside of Gradle's dependency mechanism.
You might want to check out the code to see how they do it, or simply use the plugin.
I want to include the Java Bindings for V8 ("J2V8") in a Java project. The reasons are that (i) the V8 JavaScript engine is much faster then the JavaScript engine shipped with the JRE and (ii) the library I am using is available in JavaScript only and a port to Java is much effort.
The issue is that J2V8 is compiled for different platforms: linux 64bit, macos 64bit, windows 64 bit, windows 32 bit.
I now want to generate different JARs, containing all the dependencies (fat jars):
jabref-linux_x86_64.jar
jabref-macosx_x86_64.jar
jabref-windows_x86_32.jar
jabref-windows_x86_64.jar
jabref-all.jar - the platform indipendent JAR without v8 engine
I am currently creating fat jars using the shadow plugin.
Note that the project is not an Android project. There, with the Android plugin, it seems to be straight-forward to do that.
The first idea is to introduce configurations and configuration-specific dependencies:
configurations {
linux_x86_64Compile.extendsFrom compile
macosx_x86_64Compile.extendsFrom compile
windows_x86_32Compile.extendsFrom compile
windows_x86_64Compile.extendsFrom compile
}
dependencies {
compile configuration: 'linux_x86_64', group: 'com.eclipsesource.j2v8', name: 'j2v8_linux_x86_x64', version: '4.6.0'
compile configuration: 'macosx_x86_64', group: 'com.eclipsesource.j2v8', name: 'j2v8_macosx_x86_x64', version: '4.6.0'
compile configuration: 'windows_x86_32', group: 'com.eclipsesource.j2v8', name: 'j2v8_win32_x86', version: '4.6.0'
compile configuration: 'windows_x86_64', group: 'com.eclipsesource.j2v8', name: 'j2v8_win32_x86_x64', version: '4.6.0'
...
}
But now I'm stuck. In pseudocode, I'd like to do:
task releaseSingleJar(dependsOn: "shadowJar", name) {
doLast {
copy {
from("$buildDir/libs/JabRef-${project.version}-fat.jar")
into("$buildDir/releases/")
rename { String fileName ->
fileName.replace('-fat', '-$name')
}
}
}
}
task releaseJars() {
forEach name in "linux_x86_64", "macosx_x86_64", "windows_x86_32", "windows_x86_64", "all":
if (name != "all") activate configuration $name
releaseSingleJar($name)
shadowJar is from the shadow plugin.
Background information
Video showing the difference of speed in our setting: https://github.com/JabRef/jabref/pull/2250#issuecomment-264824598
Current state of the integration in JabRef: https://github.com/JabRef/jabref/pull/3180
Related questions
The question Using Gradle to manage Java web app with flavors like Android has a similar title, but asks for source directories, whereas I ask for dependencies. Further, I want to generate a fat JAR and there a plain JAR seems to be enough. However, it might be that the solution is similar. A hint was to use the gradle-java-flavours plugin with the main source being JavaFlavoursExtension.groovy.
Following questions are similar to this one. However, the setting is related to Android apps and not to plain Java apps.
How to define different dependencies for different product flavors
Gradle: add dependency for a specific flavour of the library
Change dependency through a task in gradle
Use different resources for different application flavors using gradle
You might be interested in my gradle-java-flavours plugin which creates source sets, configurations and compile, jar and test tasks for each flavour in a java only project.
eg:
import com.github.jengelman.gradle.plugins.shadow.tasks.*
plugins {
id 'com.lazan.javaflavours' version '1.2'
id 'com.github.johnrengelman.shadow' version '1.2.4'
}
def flavours = ['linux_x86_64', 'macosx_x86_64', ...]
javaFlavours {
flavours.each {
flavour it
}
}
dependencies {
linux_x86_64Compile 'aaa:aaa:1.0'
linux_x86_64Runtime 'bbb:bbb:1.0'
macosx_x86_64TestCompile 'ccc:ccc:3.0'
}
flavours.each { String flavour ->
SourceSet flavourSourceSet = sourceSets.getByName(flavour)
Configuration flavourRuntime = configurations.getByName("${flavour}Runtime")
JavaCompile flavourCompileTask = tasks.getByName("compile${flavour.capitalize()}Java")
Task shadowJarTask = tasks.create(name: "${flavour}ShadowJar", type: ShadowJar) {
classifier = "${flavour}-all"
dependsOn flavourCompileTask
// todo configure (likely based on Configuration and SourceSet)
}
artifacts {
archives shadowJarTask
}
// wire the task into the DAG
assemble.dependsOn shadowJarTask
}
I am struggling with the Gradle build lifecycle; specifically with the split between the configuration and execution phases. I have read a number of sections in the Gradle manual and have seen a number of ideas online, but have not found a solution to the following problem:
I want to run a specific task to produce an artifact at the end of my java-library-distribution build that is a flattened version of the runtime configuration jars. That is, I only want to produce the artifact when I run the specific task to create the artifact.
I have created the following task:
task packageSamplerTask(type: Tar, dependsOn: distTar) {
description "Packages the build jars including dependencies as a flattened tar file. Artifact: ${distsDir}/${archivesBaseName}-${version}.tar"
from tarTree("${distsDir}/${archivesBaseName}-${version}.tar").files
classifier = 'dist'
into "${distsDir}/${archivesBaseName}-dist-${version}.tar"
}
Although this task does produce the required artifact, the task runs during gradle's configuration phase. This behavior has the following consequences:
Irrespective of which task I run from the command line, this packageSamplerTask task is always run, often unnecessarily; and
If I clean the project, then the build fails on the next run because $distsDir doesn't exist during the configuration phase (obviously).
It appears that if I extend the Copy task in this manner I'm always going to get this kind of premature behavior.
Is there a way to use the << closure / doLast declarations to get what I want? Or is there something else I'm missing / should be doing?
Update
After further work I have clarified my requirements, and resolved my question as follows (specifically):
"I want to package my code and my code's dependencies as a flat archive of jars that can be deployed as a jMeter plugin. The package can then be installed by unpacking into the jMeter lib/ext directory, as is. The package, therefore, must not include the jMeter jars (and their dependencies) which are used for building and testing"
Because Gradle doesn't appear to support the Maven-like provided dependency management, I created a new configuration for my package which excludes the jMeter jars.
configurations {
jmpackage {
extendsFrom runtime
exclude group: 'org.apache.jmeter', name: 'ApacheJMeter_core', version: '2.11'
exclude group: 'org.apache.jmeter', name: 'ApacheJMeter_java', version: '2.11'
}
}
And then created the following task (using the closure recommendation from Peter Niederwieser):
task packageSamplerTask(type: Tar, dependsOn: assemble) {
from { libsDir }
from { configurations.jmpackage.getAsFileTree() }
classifier = 'dist'
}
This solution appears to work, and it allows me to use just theGradle java plugin, too.
The task declaration is fine, but the flattening needs to be deferred too:
...
from { tarTree("${distsDir}/${archivesBaseName}-${version}.tar").files }
Also, the Tar file should be referred to in a more abstract way. For example:
from { tarTree(distTar.archivePath).files }
First your task isn't executed in the configuration phase but like EVERY task it is configured in that phase. And your closure is just a configuration of your task (a Configuration closure, not an Action closure). That is why your code is "executed" in the configuration phase".
If you want your code to be executed in the execution phase have to write it in a doLastclosure or doFirst. But in your case it is better to keep it in a configuration closure, because you are configuring your task.
To make sure your build doesn't fail because of the missing folder, you can create it with distsDir.mkdirs().