I have an in-house lib with all the sprocs to create some audit table and triggers for the tables that I would be adding in future as migration scripts and I want these sprocs to be deployed to my schema before any of the migrations runs. I have a specific DDL that does it. The reason I want it that way is whenever I have a new table that I need to add, I can call this sproc to create those triggers and audit tables. That way my migration scripts would be cleaner and simpler.Also, this would help in keeping the H SQL and oracle scripts as much in sync as possible as I would not care about having audit or triggers for H SQL while doing dev work.
I have extracted from the library into a folder the DLL running the sprocs and it is in a separate folder as I want to have them separate to my migration scripts.
I don't have a Flyway bean created because spring boot did it for me. I don't see any properties configuration to create callbacks.
I am using spring boot 1.3.3-RELEASE and prefer to use flyway 3.2.1 which comes with spring boot parent dependencies.
Could someone suggest the best way to do this?
If there is a higher version of flyway supporting this, would you recommend override we specific version of flyway provided by spring boot with the new one?
Option 1:
Use callbacks. Flyway version 3.x has support for these which was improved upon in version 4. This includes hooks for beforeMigrate, beforeEachMigrate and beforeBaseline which is what you are after.
There are a few ways to do this:
Add the named SQL files (eg beforeMigrate.sql, beforeEachMigrate.sql) to the migrations directory.
Configure your callback Java classes with the callbacks property.
spring-boot offers FlywayMigrationStrategy to hook into the lifecycle and get access to the Flyway object. See the spring-boot Flyway documentation.
Option 2.
Add those stored procedures to V__1 so they are callable from each subsequent migration. Means you will have to forgo your desire of having them separated from the migrations of course - but much simpler.
#Component
public class FlywayFactory implements FlywayMigrationStrategy {
#Override
public void migrate(Flyway flyway) {
flyway.setCallbacks(new FlywayCallbackService());
flyway.migrate();
}
}
This did the work. thanks for the answer #markdsievers
Related
I have a spring boot project which initially uses hibernate.ddl-auto: update command to create schema. Now i want to migrate to liquibase, so for this steps followed are:
ran diffChangeLog on cmd to generate existing schema -> got generated in db/changelog folder.
ran changelogSync on cmd, I was able to create liquibase related table(changeLog and lock) and all existing schema related insert statement were also inserted.
So far so good. now i want to move this change to production, When my spring boot project starts it should automatically run changelogSync command before hand and then boot strap the project, so that i don't need to run these command manually. how can i achieve this?
As far as diffChangeLog is concerned it is used to point out the differences in general and generates changes to resolve most of them. Where the changeLogSync commands ONLY marks all undeployed changes in your changelog as executed in your database, it does not deploy any changes. From what understand this is not the best approach to migrate the database.
There are 2 ways which you can use. Change your pipeline(in jenkins or bamboo or any other tool) so you can execute changelogSync before even you start your application(advantage of this is that liquibase command is totally decoupled from your app). The second approach is to implement CommandLineRunner like this:
#Component
public class CommandLineAppStartupRunner implements CommandLineRunner {
#Override
public void run(String...args) throws Exception {
Runtime rt = Runtime.getRuntime();
Process pr = rt.exec("liquibase changelogSync");//or any other command
}
}
You can even pass your command as argument to command line runner if you want.
I have a SpringBoot application with in memory H2 database and Spring Data JPA.
I need to configure a #Scheduled job that drops and recreates the schema and loads it with fresh data from a file.
How can I programmatically recreate the schema in my application?
You can use database version control tool like eg Liquibase to create and maintain database schema definition as well as initial data. Than, you will be able to easily invoke database migration including drop of whole schema during applicaiton runtime. IT has some integration with Spring Boot already.
Keep in mind, that you will have to lock database access in order to execute migration - DDL is not transactional, so database will be of no use anyway during the migration process and you app can yeld many errors during that time.
If locking is not an option - you should be able to create another instance or at least separate schema in running instance, run migration against it and if everything is done, "switch" peristence context to use brand new schema (and probably remove the old one)
In my spring-boot project I'm using out-of-the-box integration for Flyway (org.flywaydb.flyway-core) and have some migration scripts which are executed on startup and managed via default flyway_schema_history-table.
The project also uses a module bringing its own flyway migration scripts, which are migrated programmatically and keeping track of migrations in an other moduleX_schema_history-table.
As the migrations of the main-project needs to work on some of the tables created via the modules migration, the module-migration needs to happen before the flyway-plugin migrates the main-projects scripts.
How can I achieve executing modules migration programmatically and before the main-apllications fly-integration kicks in?
How and when is the flyway-plugin migration triggered?
//EDIT:
I tried to execute code before FlywayAutoConfiguration via a #Configuration class annotated with
#AutoConfigureBefore({FlywayAutoConfiguration.class})
#AutoConfigureAfter({DataSourceAutoConfiguration.class})
but unfortunatly the class is still instanciated after FlywayAutoConfiguration.
//EDIT:
I asked the related (more general) question how to order AutoConfiguration from modules, too How to use #AutoConfigureOrder in a spring boot configuration class from a module - Stack Overflow
Flyway migration will kick-in during startup if you configured it correctly in your application yaml/properies file.
In order to start another flyway migration before the main app migration you can use flyway command line or create an separate custom_pom.xml project file for another module and trigger it via mvn plugin from command line: mvn flyway:migrate before executing the main app.
I'm trying to populate my database with around 150 different values (one for each row).
So far, I've found two different ways to implement the inserts, but none of them seems to be the best way to do it.
Flyway + Postgres: One of them is to create a migration file and make use of the COPY command from postgres but to do so, I need to give superuser permissions to the user and that doesn't seem to be a good choice.
Spring boot: place a data.sql file in the classpath with a lot of inserts. If I'm not wrong I would have to write 150 insert into... statements.
In previous projects, I have used liquibase and it has a loadData command which is very convenient to do what is says it does. You just give the file, table name and that's it. You end up with your csv file values in your table rows.
Is there an alike way to do that in flyway? What is the best way to populate the database?
Actually there is a way, you can find more info on the official documentation's page
You need to add some spring boot properties too:
spring.flyway.enabled=true
spring.flyway.locations=classpath:/db/migration
spring.flyway.schemas=public
Properties details here
In my case, a use Repetables scripts by my needs but take care with the prefixes
Flyway is a direct competitor of liquidbase, so if you need to track the status of migrations, manage distributed migration (many instances of the same service start simultaneously, and only one instance should actually execute a migration), check upon startup which migration should be applied and execute only relevant migrations, and all other benefits that you have previously expected from "migration management system", then you should use Flyway rather than managing SQLs directly.
Spring boot has integrations with both Flyway and Liquidbase, so you can place your migrations in the "resources" folder, define a couple of properties and spring boot will run Flyway automatically.
For example, here you can find a tutorial of Flyway integration with spring boot.
Since flyway's migrations are SQL files- you can place there whatever you want (even plSQL I believe), it will even manage transaction per migration guaranteeing that the migration "atomicity" (all or nothing, no partial migration).
So the straightforward approach would be creating a SQL file file with 150 inserts and running it via flyway in spring or even via maven depending on your actual setup.
If you want more fine-grained control and the SQL is not flexible enough, its possible to implement Migration in Java Code. See Official Flyway Documentation
How can I insert data to a table on Spring Boot application start? My application is generated by JHipster. What I need to check is that if that particular data already exist in that table or not. If it doesn't I should add it.
If your application was generated by JHipster, then it should already include and be properly configured for Liquibase.
You have a few options:
You can use Liquibase's insert change to insert data.
You can put your data in a CSV file and use the loadData change to load it. Look for the loadData tag in 00000000000000_initial_schema.xml for examples.
You can use the sql change to run native SQL directly.
All three options can be combined with preconditions to make sure the data doesn't exist before trying to insert it. If you need the changeset to run every time you boot your application, you can use the runAlways="true" attribute (docs for that are on this page).
You can create a function in your service class which checks if the data already exists or not and if not then save that data.
You can implement an ApplicationRunner and use a repository to do whatever you need to do.
If the ApplicationRunner is a spring bean it is run on application startup.
For more sophisticated requirements I would try to rely on a tool like flyway that has good integration with spring boot