Manage multiple database with the flyway migrations gradle plugin - gradle

We have two databases for which we'd like to manage their migrations using flyway's gradle plugin.
I'd like to have a single task that can migrate both databases. However, I can't seem to get the flywayMigrate task to be called twice from a single task.
Here's what I have...
task migrateFoo() {
doFirst {
flyway {
url = 'jdbc:mysql://localhost/foo'
user = 'root'
password = 'password'
locations = ['filesystem:dev/src/db/foo']
sqlMigrationPrefix = ""
initOnMigrate = true
outOfOrder = true
}
}
doLast {
tasks.flywayMigrate.execute()
}
}
task migrateBar() {
doFirst {
flyway {
url = 'jdbc:mysql://localhost/bar'
user = 'root'
password = 'password'
locations = ['filesystem:dev/src/db/bar']
sqlMigrationPrefix = ""
initOnMigrate = true
outOfOrder = true
}
}
doLast {
tasks.flywayMigrate.execute()
}
}
task migrate(dependsOn: ['migrateFoo','migrateBar']) {}
Explicitly calling either migrateFoo or migrateBar from the command line works fine, however, if I try to call the migrate task only database foo is updated.
Both the doFirst and doLast tasks of the migrateBar task are called, however, the tasks.flywayMigrate.execute() task isn't called the second time from migrateBar.
How can I get flyway to migrate both foo and bar from a single task?

First, you should never call execute() on a task (bad things will happen). Also, a task will be executed at most once per Gradle invocation.
To answer your question, apparently the flyway plugin doesn't support having multiple tasks of the same type. Looking at its implementation, I think you'll have to roll your own task. Something like:
import com.googlecode.flyway.core.Flyway
import org.katta.gradle.plugin.flyway.task.AbstractFlywayTask
class MigrateOtherDb extends AbstractFlywayTask {
#Override
void executeTask(Flyway flyway) {
// set any flyway properties here that differ from
// those common with other flyway tasks
println "Executing flyway migrate"
flyway.migrate()
}
task migrateOtherDb(type: MigrateOtherDb)
I recommend to file a feature request to support multiple tasks of the same type, with a convenient way to configure them.

I also had the same problem. I wanted to run flyway migrations for different databases and even for the same database with different configurations in ONE gradle build.
for each database i need to migrate normal data tables and static data tables, so i use two flyway version tables and also two locations for the scripts. E.g.
ENV: dev MIGRATION1: data (locations: db/scripts/data table: _flyway_version_data)
MIGRATION2: static (locations: db/scripts/static table: _flyway_version_static)
ENV: test MIGRATION1 ....
MIGRATION2 ....
As Peter states above, the flyway tasks are only executed ONCE no matter how often you call them.
The workaround i found does not seem to be nicest, but it works:
in build.gradle
task migrateFlywayDevData(type: GradleBuild) {
buildFile = 'build.gradle'
tasks = ['flywayMigrate']
startParameter.projectProperties = [env: "dev", type="data"]
}
task migrateFlywayDevStatic(type: GradleBuild) {
buildFile = 'build.gradle'
tasks = ['flywayMigrate']
startParameter.projectProperties = [env: "test", type="static"]
}
....(task defs for test env)
Basically i create a new gradle build for each of the configurations.
"buildFile = 'build.gradle'"
refers to itself, so all code is contained in one build.gradle file.
The gradle call is then:
gradle migrateFlywayDevData migrateFlywayDevStatic ...
This is the first version. so code might be easily improved.
However this solution lets you execute flyway tasks multiple times with one gradle call.
Feel free to comment (flyway plugin configuration is not shown here)

Related

Set the properties in the configuration phase of a task in another task

I have spent the last few hours trying to find a solution for my requirement, without luck:
I have a task that has to run some logic in a certain path:
task run(type: MyPlugin) {
pathForPlugin = myPath //Defined as a property in another gradle file
}
I want to set the "pathForPlugin" property dynamically in another task because it has to be read from some configuration file.
task initPaths(type: PathFinder) {
configurationFile = 'C:\\myConfig.conf'
}
The myConfig.conf would look like this:
pathForPlugin = 'C:\\Correct\\Path'
The problem is that "initPaths" has to run before the configuration phase of "run".
I have tried several approaches for this (GradleBuild task, dependsOn, Using Properties in the Plugin for "Lazy Configuration") but every approach only takes effect in the Execution phase leading to the "pathForPlugin" always staying at its default value.
Is there some way i can realize this or should i look for another solution outside of the gradle build?
I found a solution for the problem:
Instead of defining a task "initPaths" i directly used the java class "Pathfinder" in the build script:
import mypackage.PathFinder;
new PathFinder(project).run()
You only have to make sure that this part is above the definition of the task where the properties are used.
I admit this is a bit of a "hacky" solution but it works fine for my requirement.
you can do like this:
ext {
myPath //use it as a global variable that you can set and get from different gradle tasks and files
}
task firstTask {
doLast {
ext.myPath = "your path"
}
}
task run(type: MyPlugin) {
doFirst { //executed on runtime not on task definition
pathForPlugin = ext.myPath //Defined as a property in another gradle file
}
}
//example 2 - create run task dynamic
task initPath {
doLast {
tasks.create(name: "run", type: MyPlugin) {
pathForPlugin = ext.myPath
}
}
}

Conditional logic in gradle build to only execute certain code when a specific task is running

So, I am using a plugin in my gradle build (the plugin is org.flywaydb.flyway but that is not really relevant). I want to validate the caller has passed in a runtime parameter when tasks from this plugin are executing but not when other tasks are executing.
I pass options to the flyway plugin based on a supplied parameter. I want an error to be returned when a flywayTask is being executed and no parameter is supplied. When a non-flyway task is being run, I do not want to validate if the parameter is supplied.
gradle -PmyParam=myValue flywayMigration
=> should run code and there should be no error
gradle flywayMigration
=> should run code and should produce error (as no parameter supplied)
gradle jar
=> should not run code and no error should be produced
I have been reading about gradle configuration and execution which is fine but I still can't find a way to only run the code when the flyway plugin is bveing executed OR specific flyway tasks are being executed.
This is my current code:
if(gradle.taskGraph.hasTask("flywayMigrate")) {
flyway {
def dbCode, dbUser, dbPassword, dbUrl
if (!project.hasProperty("db_env")) {
throw new GradleException("Expected db_env property to be supplied for migration task. Can be passed" +
" at command line e.g. [gradle -Pdb_env=ex1 flywayMigrate]")
} else {
// do stuff
}
user = balh
password = blah
url = blah
driver = 'oracle.jdbc.OracleDriver'
cleanDisabled = true
baselineOnMigrate = true
baselineVersion = '1.0.0'
}
}
To be clear, I only want this code:
if (!project.hasProperty("db_env")
to run for flyway tasks.
The code above throws this error:
Task information is not available, as this task execution graph has not been populated.
I've tried a few things here, any advice would be appreciated.
It's not really clear to me, what exactly do you want to do in case if this property is provided, but I think, you can do it without accesing task graph, just try to use doFirst Closure of the flywayMigrate task. Just something like this:
flywayMigrate.doFirst {
if(!project.hasProperty("db_env")) {
throw ...
} else {
//Do something
}
}
And leave your plugin configuration free of any additional logic.
As for exception, have you tried to wait until graph is ready? It's usualy done as follows:
gradle.taskGraph.whenReady {taskGraph ->
if(gradle.taskGraph.hasTask("flywayMigrate")) {
...
}
}
Update: to answer the question from the comments
if I can attach doFirst to multiple tasks?
Yes, you can use somthing like:
//declare task names
def names = ["taskA", "taskB", "taskC"]
tasks.findAll {it ->
//filter tasks with names
if (it.name in names)
return it
}.each { it ->
//add some extra logic to it's doFirst closure
it.doFirst {
println 'hello'
}
}
Just check, that all the tasks are exists before this configuration.

How to make Gradle run tasks in certain order?

Let's say that I have created separate tasks for running integration and acceptance tests in my gradle build script. When I run the build task I want to run testing tasks before in this order: unit tests(test task), integration tests (intergationTest task) and acceptance tests (acceptanceTest task). Is this possible and how?
You are looking for "should run after" described in Gradle documentation - http://www.gradle.org/docs/current/userguide/more_about_tasks.html
Here's how you can do it without creating artificial dependencies:
https://caffeineinduced.wordpress.com/2015/01/25/run-a-list-of-gradle-tasks-in-specific-order/
TLDR; version:
//--- build aliases : define a synonym here if you want a shortcut to run multiple targets
def buildAliases = [
'all' : ['clean', 'assemble', 'runProvisioner', 'stopTomcat', 'installTomcat', 'deployToTomcat', 'startTomcat'],
'rebuild' : ['clean', 'assemble']
]
def expandedTaskList = []
gradle.startParameter.taskNames.each {
expandedTaskList << (buildAliases[it] ? buildAliases[it] : it)
}
gradle.startParameter.taskNames = expandedTaskList.flatten()
println "\n\n\texpanded task list: ${gradle.startParameter.taskNames }\n\n"
This is what I did on my projects.
check.dependsOn integTest
integTest.mustRunAfter test
tasks.withType(Pmd) {
mustRunAfter integTest // Pointing to a task
}
tasks.withType(FindBugs) {
mustRunAfter tasks.withType(Pmd) // Pointing to a group of tasks under Pmd
}
tasks.withType(Checkstyle) {
mustRunAfter tasks.withType(FindBugs)
}
It helped me to order tasks by group.
I created this helper method based on a solution that I found on Gradle forum.
Task.metaClass.runFirst = { Task... others ->
delegate.dependsOn(others)
delegate.mustRunAfter(others[0])
for (def i=0; i < others.size() - 1; i++) {
def before = others[i]
def after = others[i+1]
after.mustRunAfter(before)
}
}
Then you can create tasks X, A, B and C and use like this:
X.runFirst A, B, C
The first answer in the list turned out to be great for me. I used
X.shouldRunAfter Y
UPDATE: While using this "solution" for a short while i found out, that it does not work 100% as intended. I don't know why though. Any help to make it work would be appreciated. Maybe it does not work properly when there is more than one task in the Set?! While testing i did add some dummy-Tasks which only printed a text inbetween each of the other tasks and all seemed to be ok.
After some attempts with other solutions i came up with this solution:
It uses the mustRunAfter command to chain Sets of Tasks into the required order.
I'm working with Sets instead of individual Tasks, because i got circular dependency issues otherwise, since some tasks already depended on each other.
Also important to note: the isNotEmpty() check was essential, as it would otherwise break the enforced ordering if an ampty set was passed to the method.
tasks.register("cleanAndGenerate") {
var lastTasks: Set<Task> = setOf()
fun findTasks(taskName: String): Set<Task> {
return if (taskName.startsWith(":")) { // task in specific sub-project
setOf(tasks.findByPath(taskName)!!)
} else { // tasks in all (sub-)projects
getTasksByName(taskName, true)
}
}
fun dependsOnSequential(taskName: String) {
val tasks = findTasks(taskName)
tasks.forEach { task -> task.mustRunAfter(lastTasks) }
dependsOn(tasks)
if (tasks.isNotEmpty()) {
lastTasks = tasks
}
}
dependsOnSequential(":project1:clean") // task in specific sub-project
dependsOnSequential(":project2:clean")
dependsOnSequential("task1") // tasks in all (sub-)projects
dependsOnSequential("task2")
// add more as needed
}

How to reuse existing task in Gradle

I have question how to reuse existing task in Gradle and what is recommended way?
I would like to change some properties of reused task.
I've found that it is possible to do in dependsOn or call execute, but is it recommended way? I've read that execute is not recommended. It looks like dependsOn is better way.
Example:
task task1() {
description = "task1"
doLast {
println description
}
}
task callModifiedTask1ByDependsOn() {
dependsOn {
task1 {
description = "modified task 1 by dependsOn"
}
}
}
task callModifiedTask1ByExecute() << {
project.task1 {
description = "modified task 1 by execute"
}.execute()
}
Hmm... Calling execute() explicitly seems not a good idea. Mind the fact that execute() method is part of internal API of Gradle so it can potentially change.
According to reusable task You can do it exactly the same as it's done in Gradle: e.g. copy task. Just implement the task and configure the part that is changed for execution (e.g. from). Your example with dependsOn follows this scenario more or less.
A task cannot be "reused", and cannot be called from another task (only depended upon). Instead, declare multiple tasks.

Moving built-in gradle tasks work to doLast/built-in tasks shourtcuts

I want to create a simple sync task that slightly change it behaviour depending on build type (e.g. debug/release) and I use boolean variable 'dummy' decrared in gradle.taskGraph.whenReady:
gradle.taskGraph.whenReady {taskGraph ->
dummy = false
if (taskGraph.hasTask(':dummybuild')) {
dummy = true
}
}
The problem is that task configured by the following way has configuration scope, i.e. before whenReady so it doesn't have access to the 'dummy' variable:
task copySkins(type: Sync) {
from skinsFrom
into skinsInto
rename skinsRename
exclude symbianExclude
if (!dummy) exclude dummyExclude
}
Right now I'm using this workaround
task copySkins {
inputs.dir skinsFrom
outputs.dir skinsInto
doLast {
task skins(type: Sync) {
from skinsFrom
into skinsInto
rename skinsRename
exclude symbianExclude
if (!dummy) exclude dummyExclude
}
skins.execute()
}
}
Is it possible to
detect/setup some build properties in some other place except whenReady
move sync task work to doLast
or at least have some shortcut for sync task (.execute() looks quite ugly)
1) whenReady event allows user to access fully-initialized task graph: all initialization is finished and tasks are ready to run. The only situation, when you need to detect/setup build properties here, is when you need to introspect current build setup.
If you do not need this information, you can place your initialization anywhere in your build script. At the very end, it is nothing but groovy script.
apply plugin: 'java'
def now = new Date()
compileJava.doFirst {
println "It is ${now}. We are starting to compile"
}
2) You can not move sync task work to doLast. But you can always add your actions to doFirst ;) I think, this should work:
task copySkins(type: Sync) {
from skinsFrom
into skinsInto
rename skinsRename
exclude symbianExclude
doFirst {
if (!dummy) exclude dummyExclude
}
}
3) With all said before, missing sync task shortcut should not be that painfull

Resources