I am trying to enable initialization of a database (execution of .sql file containing INSERT statements) in application that uses Spring, Hibernate and Flyway.
The issue is that I want this initialization to be performed only on 'local' profile and to be entirely skipped on 'production' (so How to deploy initial data on database tables through flyway doesn't solve it).
If the app weren't using Flyway, I could achieve this conditional initialization by simply using spring.datasource.data=initialization_script.sql in 'local' profile and putting all the inserts inside such file. I don't know how to do such things in Flyway.
Flyway has an option to use profile-specific configurations. Some migrations can be performed conditionally only when a particular profile is active.
To do this one can use (as example):
spring.flyway.locations=classpath:/db/migration,classpath:/local/db/migration in properties file. With this setup migrations from 'local/db/migration' are run only if 'local' profile is active
Related
As I understand from Spring reference docs, we can use data.sql and schema.sql for initializing the database.
When there are multiple instances of the application running against a single database, how to make the data.sql, schema.sql run only once and instead of executing in all the containers.
You need to disable the default spring data initialization. You can do this through your properties file by
spring.datasource.initialization-mode=never
While Spring do not recommend using multiple data source initialization technologies, if you want script-based DataSource initialization to be able to build upon the schema creation performed by Hibernate, set
spring.jpa.defer-datasource-initialization
To true, this will defer data source initialization until after any EntityManagerFactory beans have been created and initialized.
And for managing your database across multiple application instance you can use Flyway and Liquibase, sing the basic schema.sql and data.sql scripts alongside Flyway or Liquibase is not recommended and support will be removed in a future release.
We have a (internal to company, external to project) library (jar) that includes some Liquibase scripts to add tables to a schema to support the functionality of that library.
Using SpringBoot and Maven to run integration tests with H2, we have been using sql files, listed in property files, to initialise the DB.
We want to be able to add data to the tables created by the external (to the project) library for the ITs but finding the tables haven't been created by Liquibase when SpringBoot/SpringData attempts to run the insert statements in our sql files.
Given the errors we're seeing (tables not existing when spring attempts to run the insert.sql files) it looks like spring is executing those files before Liquibase has done its thing.
How can I ensure Liquibase config run by the library to create tables has completed before Spring does it's thing with running sql files specified by the spring.datasource.data property?
We don't really want to include the test data in the library (which was working, but introduced other issued we are trying to work around with liquibase inserting test data into production DB).
what about using different context for your tests?
so you will have application.properties in your test folder and there you will define another changelog that will include all changelogs that are needed (even from library) and you will also include the .sql file that you are running probably with jpa? Try to look here if that helps.
I have a SpringBoot application with in memory H2 database and Spring Data JPA.
I need to configure a #Scheduled job that drops and recreates the schema and loads it with fresh data from a file.
How can I programmatically recreate the schema in my application?
You can use database version control tool like eg Liquibase to create and maintain database schema definition as well as initial data. Than, you will be able to easily invoke database migration including drop of whole schema during applicaiton runtime. IT has some integration with Spring Boot already.
Keep in mind, that you will have to lock database access in order to execute migration - DDL is not transactional, so database will be of no use anyway during the migration process and you app can yeld many errors during that time.
If locking is not an option - you should be able to create another instance or at least separate schema in running instance, run migration against it and if everything is done, "switch" peristence context to use brand new schema (and probably remove the old one)
I'm trying to populate my database with around 150 different values (one for each row).
So far, I've found two different ways to implement the inserts, but none of them seems to be the best way to do it.
Flyway + Postgres: One of them is to create a migration file and make use of the COPY command from postgres but to do so, I need to give superuser permissions to the user and that doesn't seem to be a good choice.
Spring boot: place a data.sql file in the classpath with a lot of inserts. If I'm not wrong I would have to write 150 insert into... statements.
In previous projects, I have used liquibase and it has a loadData command which is very convenient to do what is says it does. You just give the file, table name and that's it. You end up with your csv file values in your table rows.
Is there an alike way to do that in flyway? What is the best way to populate the database?
Actually there is a way, you can find more info on the official documentation's page
You need to add some spring boot properties too:
spring.flyway.enabled=true
spring.flyway.locations=classpath:/db/migration
spring.flyway.schemas=public
Properties details here
In my case, a use Repetables scripts by my needs but take care with the prefixes
Flyway is a direct competitor of liquidbase, so if you need to track the status of migrations, manage distributed migration (many instances of the same service start simultaneously, and only one instance should actually execute a migration), check upon startup which migration should be applied and execute only relevant migrations, and all other benefits that you have previously expected from "migration management system", then you should use Flyway rather than managing SQLs directly.
Spring boot has integrations with both Flyway and Liquidbase, so you can place your migrations in the "resources" folder, define a couple of properties and spring boot will run Flyway automatically.
For example, here you can find a tutorial of Flyway integration with spring boot.
Since flyway's migrations are SQL files- you can place there whatever you want (even plSQL I believe), it will even manage transaction per migration guaranteeing that the migration "atomicity" (all or nothing, no partial migration).
So the straightforward approach would be creating a SQL file file with 150 inserts and running it via flyway in spring or even via maven depending on your actual setup.
If you want more fine-grained control and the SQL is not flexible enough, its possible to implement Migration in Java Code. See Official Flyway Documentation
I'm facing very weird issue while integrating flyway DB migration with spring boot application.
When I run the application from executable WAR using command line, it creates new DB at the start-up of application.
Now, If I switch the application run mode to IDE (i.e. run from STS), it again fires all the script from my db/migration folder. I can see the installed_on column time changes every-time I switch between these 2 run modes. I have tried enabling baselineOnMigrate property, but didn't get any effect of it.
Do you think its something related to spring boot embedded tomcat ? because at both run it creates individual tomcat which is embedded.
Please find my spring boot application.properties below:
mssql.dbname=issueDB
mssql.password=password
mssql.dbserver=localhost
mssql.port=1501
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.url=jdbc:sqlserver://${mssql.dbserver}:${mssql.port};databaseName=${mssql.dbname}
spring.datasource.username=user
spring.datasource.password=${mssql.password}
spring.flyway.baselineOnMigrate=true
spring.flyway.locations=classpath:db/migration/testissue
spring.flyway.out-of-order=true
spring.flyway.baseline-version=1.3
spring.flyway.placeholder-prefix=$
spring.flyway.placeholder-suffix=$
spring.flyway.mixed=true
spring.flyway.cleanOnValidationError=true
I suppose, it could be caused by this property spring.flyway.cleanOnValidationError=true. According to the docs:
Whether to automatically call clean or not when a validation error occurs.
This is exclusively intended as a convenience for development. Even tough we strongly recommend not to change migration scripts once they have been checked into SCM and run, this provides a way of dealing with this case in a smooth manner. The database will be wiped clean automatically, ensuring that the next migration will bring you back to the state checked into SCM.
May be that you got some validation problems if you are running your application in different ways on the same database and flyway just clean your database and overwrite it with the current scripts state.