I am developing a custom spring-boot-starter that can be included in any project. For it work it need to create a couple of tables in the database in the main application. For this I use liquibase migrations which are described in the spring-boot-starter module. Also in the starter in the application.properties file I describe the properties
spring.datasource.url=jdbc:postgresql://localhost:5432/message
spring.liquibase.change-log=classpath:liquibase/changelog.xml
When starting the main application, which contains my starter in dependencies, I get an exception:
Description:
Liquibase failed to start because no changelog could be found at 'classpath:/db/changelog/db.changelog-master.yaml'.
Action:
Make sure a Liquibase changelog is present at the configured path.
The application looks for a path to migrations and migrations inside the project, but does not look for them inside the starter scope.
Please tell me what needs to be done so that when the application starts, the migrations described in the starter are also pulled up?
How to set up liquibase? Or maybe there is some other way to run sql scripts if the required tables are missing?
Related
We have a (internal to company, external to project) library (jar) that includes some Liquibase scripts to add tables to a schema to support the functionality of that library.
Using SpringBoot and Maven to run integration tests with H2, we have been using sql files, listed in property files, to initialise the DB.
We want to be able to add data to the tables created by the external (to the project) library for the ITs but finding the tables haven't been created by Liquibase when SpringBoot/SpringData attempts to run the insert statements in our sql files.
Given the errors we're seeing (tables not existing when spring attempts to run the insert.sql files) it looks like spring is executing those files before Liquibase has done its thing.
How can I ensure Liquibase config run by the library to create tables has completed before Spring does it's thing with running sql files specified by the spring.datasource.data property?
We don't really want to include the test data in the library (which was working, but introduced other issued we are trying to work around with liquibase inserting test data into production DB).
what about using different context for your tests?
so you will have application.properties in your test folder and there you will define another changelog that will include all changelogs that are needed (even from library) and you will also include the .sql file that you are running probably with jpa? Try to look here if that helps.
In my spring-boot project I'm using out-of-the-box integration for Flyway (org.flywaydb.flyway-core) and have some migration scripts which are executed on startup and managed via default flyway_schema_history-table.
The project also uses a module bringing its own flyway migration scripts, which are migrated programmatically and keeping track of migrations in an other moduleX_schema_history-table.
As the migrations of the main-project needs to work on some of the tables created via the modules migration, the module-migration needs to happen before the flyway-plugin migrates the main-projects scripts.
How can I achieve executing modules migration programmatically and before the main-apllications fly-integration kicks in?
How and when is the flyway-plugin migration triggered?
//EDIT:
I tried to execute code before FlywayAutoConfiguration via a #Configuration class annotated with
#AutoConfigureBefore({FlywayAutoConfiguration.class})
#AutoConfigureAfter({DataSourceAutoConfiguration.class})
but unfortunatly the class is still instanciated after FlywayAutoConfiguration.
//EDIT:
I asked the related (more general) question how to order AutoConfiguration from modules, too How to use #AutoConfigureOrder in a spring boot configuration class from a module - Stack Overflow
Flyway migration will kick-in during startup if you configured it correctly in your application yaml/properies file.
In order to start another flyway migration before the main app migration you can use flyway command line or create an separate custom_pom.xml project file for another module and trigger it via mvn plugin from command line: mvn flyway:migrate before executing the main app.
I have a spring-boot application with PostgreSQL. Some of the tables are created using models and other tables have to be created prior to the start of the application or while starting the application. That can be done by running an SQL file while startup.
But DB tend to change over time, we may have to alter some of the tables, add some new tables without disturbing the existing data in the tables, etc.
Is there a way to add new SQL files, and run only the SQL files which was not run in spring-boot application each time when we rebuild and rerun? And don't run any of the SQL files while start-up if everything were already executed?
For your scenario liquibase is the best solution. You can merge liquibase on your spring boot application.
Ex: https://javadeveloperzone.com/spring-boot/spring-boot-liquibase-example/
You can use flyway. It allows you to have versioned sql scripts:
flywaydb.org
flywaydb spring boot plugin
Examples:
Spring Boot Database Migrations with Flyway
Incrementally changing your db with java and flyway
I'm facing very weird issue while integrating flyway DB migration with spring boot application.
When I run the application from executable WAR using command line, it creates new DB at the start-up of application.
Now, If I switch the application run mode to IDE (i.e. run from STS), it again fires all the script from my db/migration folder. I can see the installed_on column time changes every-time I switch between these 2 run modes. I have tried enabling baselineOnMigrate property, but didn't get any effect of it.
Do you think its something related to spring boot embedded tomcat ? because at both run it creates individual tomcat which is embedded.
Please find my spring boot application.properties below:
mssql.dbname=issueDB
mssql.password=password
mssql.dbserver=localhost
mssql.port=1501
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.url=jdbc:sqlserver://${mssql.dbserver}:${mssql.port};databaseName=${mssql.dbname}
spring.datasource.username=user
spring.datasource.password=${mssql.password}
spring.flyway.baselineOnMigrate=true
spring.flyway.locations=classpath:db/migration/testissue
spring.flyway.out-of-order=true
spring.flyway.baseline-version=1.3
spring.flyway.placeholder-prefix=$
spring.flyway.placeholder-suffix=$
spring.flyway.mixed=true
spring.flyway.cleanOnValidationError=true
I suppose, it could be caused by this property spring.flyway.cleanOnValidationError=true. According to the docs:
Whether to automatically call clean or not when a validation error occurs.
This is exclusively intended as a convenience for development. Even tough we strongly recommend not to change migration scripts once they have been checked into SCM and run, this provides a way of dealing with this case in a smooth manner. The database will be wiped clean automatically, ensuring that the next migration will bring you back to the state checked into SCM.
May be that you got some validation problems if you are running your application in different ways on the same database and flyway just clean your database and overwrite it with the current scripts state.
I have a Spring Boot application and want to use Liquibase to generate the changelogs for my JPA entities. However, I encounter different issues, depending on my approach.
My first approach is to use the diff goal of the maven plugin. The url is my H2 development database with the H2 driver and the reference URL is something like "hibernate:spring:myBasePackage.myEntityPackage?dialect=org.hibernate.dialect.H2Dialect" with the driver "liquibase.ext.hibernate.database.connection.HibernateDriver". In that case Liquibase seems to recognize my entities, but prints the differences to the console. Also the differences do not have the form of a changelog file.
My second approach would be to use the generateChangeLog goal of the maven plugin. In this case my url is "hibernate:spring:myBasePackage.myEntityPackage?dialect=org.hibernate.dialect.H2Dialect" with the driver "liquibase.ext.hibernate.database.connection.HibernateDriver. In this case I am getting an error "Unable to resolve persistence unit root URL: class path resource [] cannot be resolved to URL because it does not exist". This error can be found in both the Spring an the Liquibase issue trackers, but it seems like it is always said that this error is already fixed.
My third approach is basically like the second, but in this case I am using a "hibernate:classic" url with an implementation of "CustomClassicConfigurationFactory", which registers my annotated classes explicitly. This does work. However, in this case I have to do this in my application-jar. I have to add my application-jar as a dependency for the maven-plugin. Thus I have to build my application-jar (and install it to the local Maven repository), before I can generate the changelogs. This seems to be cumbersome.
My questions are:
Is there an easier way to generate the changelogs for JPA entities in a Spring boot based application?
Why are the first two approaches not working?
Is there a way to simplify the third approach?
I am using:
Spring Boot 1.5.4.RELEASE
Liquibase-Hibernate4 3.6
Liquibase 3.5.3
Many thanks in advance.
In the first approach using liquibase:diff , the change set for the entities (create table change set) will not be generated since liquibase do not assume new jpa entity as change.
In the second approach generateChangeLog, it generates the change log from the given data base.It won't look into your jpa entities.
In order to generate the ddl scripts for your jpa entities , just submit the following to your jpa properties
<property key="javax.persistence.schema-generation.scripts.action">drop-and-create</property>
<property key="javax.persistence.schema-generation.scripts.create-target">./ddl/create.sql</property>
<property key="javax.persistence.schema-generation.scripts.drop-target">./ddl/drop.sql</property>
The above will generate the scripts the ddl folder under the root folder.
You can check the other properties here https://thoughts-on-java.org/standardized-schema-generation-data-loading-jpa-2-1/