Flyway: Found non-empty schema(s) "public" without schema history table! Use baseline() - on Empty database - spring-boot

I am trying to configure flyway with kotlin Spring boot, jpa and postgreSQL. My gradle dependencies are:
dependencies {
implementation('org.springframework.boot:spring-boot-starter-data-jpa')
implementation('org.springframework.boot:spring-boot-starter-web')
implementation('com.fasterxml.jackson.module:jackson-module-kotlin')
implementation('org.flywaydb:flyway-core')
implementation('com.google.code.gson:gson')
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.jetbrains.kotlin:kotlin-reflect")
runtimeOnly('org.postgresql:postgresql')
testImplementation('org.springframework.boot:spring-boot-starter-test')
}
My application.properties file is:
spring.datasource.driverClassName=org.postgresql.Driver
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.url=jdbc:postgresql://${JDBC_DATABASE_URL}/jpaTestDatabase
spring.datasource.username=${JDBC_DATABASE_USERNAME}
spring.datasource.password=${JDBC_DATABASE_PASSWORD}
flyway.baseline-on-migrate=true
flyway.locations=classpath:src/main/kotlin/db/migration
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=validate
spring.session.store-type=none
Creating tables and entries using jpa and hibernate works as expected.
However a sample migration on an empty database results in:
org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException:
Found non-empty schema(s) "public" without schema history table! Use baseline() or set baselineOnMigrate to true to initialize the schema history table.
my directory structure is the default one generated by spring initializr and my migrations are in: demo/src/main/kotlin/db/migration
I only have a single migration which is the kotlinized version of the example migration found here which I adapted to look line this:
class V1__Sample : BaseJavaMigration() {
override fun migrate(context: Context?) {
val statement = context?.connection?.prepareStatement(
"""
CREATE TABLE article (
id bigserial primary key,
name varchar(20) NOT NULL,
desc text NOT NULL
);
"""
)
statement.use { it?.execute() }
}
}
What am I missing here? Why does Flyway keep complaining about finding non-empty schema(s) "public" without schema history table, when the database is completelly empty (clean docker image)?

Assuming that you are using spring-boot version 2.
In spring boot 2 the prefix is "spring.flyway" so try adding prefix spring like below.
spring.flyway.baseline-on-migrate = true
OR
spring.flyway.baselineOnMigrate = true

may be you can try mvn flyway:clean && mvn flyway:migrate

Please check the search path of your database, if the public schema (on which flyway is creating its log tables) is not in the first place, it may not be able to find the log table and may complain that schema history is not found...
Please note that, if you are baselining, you need to remove old scripts from the scripts folder, else it will re-attempt it.

Related

Running 'mvn clean install' on Spring Boot application using env variables in application.properties

Hello I am trying to package my Spring Boot app into a jar.
I want to deploy this app to AWS Beanstalk and so I will be injecting some variables into application.properties using Environment variables.
spring.data.mongodb.uri=${MONGODB_URI}
spring.data.mongodb.auto-index-creation=true
spring.servlet.multipart.max-file-size=-1
spring.servlet.multipart.max-request-size=-1
CLOUDINARY_URL=${CLOUDINARY_URL}
jwt-secret=${JWT_SECRET}
server.port=5000
However when I run the maven command (mvn clean install), during the package process the code is executed and it is failing stating that
BeanCreationException: Error creating bean with name 'customBeansConfig': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'CLOUDINARY_URL' in value "${CLOUDINARY_URL}"
I have a class CustomBeansConfig:
#Configuration
public class CustomBeansConfig {
#Value("${CLOUDINARY_URL}")
private String cloudinaryUrl;
#Bean
public Cloudinary cloudinary(){
System.out.println("cloudinaryUrl = " + cloudinaryUrl);
return new Cloudinary(cloudinaryUrl);
}
}
Please help me to create the jar file
If I have understood you correctly, one approach may be to use different application.properties files for different environments. For example application-dev.properties for the Dev environment and application-prod.properties for the Prod environment. Then your CLOUDINARY_URL may be assigned different literal values appropriate to each.
Then when deploying to each environment, bundle your JAR with the -Denv option, as in
mvn -Denv=dev clean install
OR
mvn -Denv=prod clean install
... and upload the resulting JAR file to the corresponding AWS environment.
Running the Spring Boot application with a such config property, got me the following error:
Caused by: java.lang.IllegalArgumentException: Circular placeholder reference 'CLOUDINARY_URL' in property definitions
Changing the name of your Spring property from CLOUDINARY_URL to, for example, cloudinary.service.url will resolve the issue.
In such case, your config file should look like this:
spring.data.mongodb.uri=${MONGODB_URI}
spring.data.mongodb.auto-index-creation=true
spring.servlet.multipart.max-file-size=-1
spring.servlet.multipart.max-request-size=-1
cloudinary.service.url=${CLOUDINARY_URL}
jwt-secret=${JWT_SECRET}
server.port=5000
And your configuration file like this:
#Configuration
public class CustomBeansConfig {
#Value("${cloudinary.service.url}")
private String cloudinaryUrl;
#Bean
public Cloudinary cloudinary(){
System.out.println("cloudinaryUrl = " + cloudinaryUrl);
return new Cloudinary(cloudinaryUrl);
}
}
Also, I would advise you to avoid creating Spring configuration properties using the underscore format, since it usually used for the environment variables, maybe be confusing and may cause such interesting issues.

Flyway spring boot + java, new local database created by hibernate but now migrate tries to apply migrations that already happened

I initially created my project using hibernate to create tables like most people do, but then following recommendations I started using flyway to do db migrations.
Problem is I erased my entire local system including db and trying to spin it u again but I get conflicts of hibernate and flyway.
I'm using the java api by the way. So when I went to rebuild the database locally I turned on
spring.jpa.hibernate.ddl-auto=${HIBERNATE_DDL:create} just for the first run, then turned it to validate
So it built all the tables, but now when I try to launch the application it will try to run the first migration which is
ALTER TABLE public.auth ADD COLUMN resent boolean
which will cause an error on boot because that new column was added by hibernate
Error Code : 0
Message : ERROR: column "resent" of relation "auth" already exists
Location : db/migration/V1__Add_Resent_To_Auth.sql (/Users/brian/code/slap/build/resources/main/db/migration/V1__Add_Resent_To_Auth.sql)
Line : 1
Statement : ALTER TABLE public.auth
ADD COLUMN resent boolean
So how do I tell flyway that the current version is V9 and only run migrations after that. Shouldn't it just go look at the flyway_schema_history and see version 9 is the last entry then run migrations after that? I must be missing something
I tried doing this in my config to set the baseline version first
#Configuration
class FlyWay() {
#Value("\${spring.datasource.url}")
lateinit var url: String
#Value("\${spring.datasource.username}")
lateinit var username: String
#Value("\${spring.datasource.password}")
lateinit var password: String
#Bean
fun migrate() {
val flyway = Flyway.configure().baselineVersion("9.0").dataSource(url, username, password).load()
flyway.migrate()
}
}
no such luck it still tries to run V1
I tried adding it to application.properties too
spring.flyway.baselineVersion=9.0
same error
Scenario as I understood it:
Tables already exist
State of the tables corresponds to version "9.0"
Flyway baseline version should be set once for the local test DB
It might be useful to set the version via command line, since it is to be applied to the test database only once and then the normal migration strategies are to be applied.
Documentation see: https://flywaydb.org/documentation/usage/commandline/baseline
On macOS the Flyway command line client can be installed with brew install flyway.
Instructions
make sure the table flyway_schema_history is deleted. Use your preferred SQL client:
drop table flyway_schema_history;
then set the baseline version using the Flyway command line client (this example uses a Postgres database):
flyway -user=stephan -password= -url=jdbc:postgresql://localhost:5432/stephan -baselineVersion="9.0" -locations="src/main/resources/db/migration" baseline
check in SQL client:
select version from flyway_schema_history ;
This should show now "9.0". After that, the Spring Boot application should behave as usual.
Test
Alternative
For those people who prefer to do this with a Maven command:
drop the table flyway_schema_history like shown above
use the command mvn flyway:baseline -Dflyway.baselineVersion="9.0" to set the baseline version
This requires a bit of configuration in the pom.xml file, e.g. if using a Postgres database:
<build>
...
<plugins>
...
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>7.1.1</version>
<configuration>
<url>jdbc:postgresql://localhost:5432/stephan</url>
<user>stephan</user>
</configuration>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.2.18</version>
</dependency>
</dependencies>
</plugin>
...
Test with Maven
A quick test shows the same result.
Why not export the SQL-script from your database (created by Hibernate) and add it as the first Flyway script into your application? It's the cleanest solution as Hibernate doesn't need to be started manually again when the application will run on other systems.
Just try once after adding the following line in your application.yml
spring.flyway.baseline-on-migrate: true

Liquibase Gradle plugin appears to have url and referenceUrl reversed

I have a very simple Spring Boot 2.0.4 project. Following the various examples for setting up the Liquibase Gradle plugin I wanted to be able to run the diffChangeLog target to update my change log XML file. The configuration looks like this:
liquibase {
activities {
main {
url 'jdbc:postgresql://localhost:5432/example_db'
username 'user'
password 'password'
driver 'org.postgresql.Driver'
referenceUrl 'hibernate:spring:com.example?dialect=org.hibernate.dialect.PostgreSQL9Dialect&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy'
referenceDriver 'liquibase.ext.hibernate.database.connection.HibernateDriver'
classpath 'src/main'
changeLogFile "src/main/resources/db/changelog/master.xml"
}
runList = 'main'
}
}
The database is empty except for the databasechangelog and databasechangeloglock tables. When I run gradle diffChangeLog the change log XML file is never updated. The output from gradle diff shows "NONE" for everything. My project does have an entity and it is annotated with #Entity.
What am I doing wrong?

#ConditionalOnProperty not working when building jar locally although it is working fine when getting included from remote repo

To be more specific in the below code if I am providing only the property of MongoDB(spring.data.mongodb) in application.yml it is throwing RunTime exception stating that
PostgresDBPurgeController bean can not be created missing spring.datasource
but as per #ConditionalOnProperty if the property is not there Bean creation would not happen.
In detail,I have the below code in a common jar and I have 2 different app(1 use postgres as db ,another mongo as db ) which use this same jar and both the apps are working fine when I am including the jar from remote repo.
But when I build the common jar locally and and run the app (which use mongo as db ) locally it is throwing above error.
#ConditionalOnProperty(prefix = "spring.data.mongodb", name = "host")
#ConditionalOnProperty(prefix = "spring.datasource", name = "driver-class-name", havingValue = "org.postgresql.Driver")
public class PostgresDBPurgeController {

Spring boot: populate h2 db from schema in test/resources

On my local machine I load an in-memory h2 database to start my spring boot application in a safe environment, here's the properties:
spring.datasource.url: jdbc:h2:mem:DB_TEST;Mode=Oracle
spring.datasource.platform: h2
spring.jpa.hibernate.ddl-auto: none
spring.datasource.continue-on-error: false
spring.jpa.database-platform: org.hibernate.dialect.Oracle10gDialect
Then, in my src/main/resources I have the file schema-h2.sql containing my local db initiations.
That's fine, but then I also have some junit tests I want to execute:
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyTest {
#Autowired
private MyController controller;
#Test
public void myTest(){
controller.doSomething();
}
This is also fine, as the schema-h2.sql is seen.
Anyway according to me it would be better to put the schema-h2.sql in src/test/resources as it has to be used only on my local environment. Doing so also allows maven to exclude it from the final build and that is also pretty fine.
Anyway if I put it there the test keeps working...but the main application breaks as the schema-h2.sql is not found!
How to modify the above properties to specify that the shema-h2.sql has to be searched inside of the test/resources folder?
Thanks
For normal mode, the properties file is put in src/main/resources,
and for the testing method, the properties file in the src/test/resources folder.
By Trying to run a test-class, eclipse runs EACH file ending with .sql (and thus containing a script to create tables or to insert data) it finds under src/main/resources and src/test/resources.
So if you put a script-file schema.sql (containing a script that creates a table: create table ..) in both folders, you'll get an "table already exits" error, if you let jut one, the test will run smoothly.
If you put a script-file (that insert data in a table) in both folder, both scripts will be run.
You can also use the #PropertySource("..") in your repository to tell spring where to find the properties-file to use.

Resources