I initially created my project using hibernate to create tables like most people do, but then following recommendations I started using flyway to do db migrations.
Problem is I erased my entire local system including db and trying to spin it u again but I get conflicts of hibernate and flyway.
I'm using the java api by the way. So when I went to rebuild the database locally I turned on
spring.jpa.hibernate.ddl-auto=${HIBERNATE_DDL:create} just for the first run, then turned it to validate
So it built all the tables, but now when I try to launch the application it will try to run the first migration which is
ALTER TABLE public.auth ADD COLUMN resent boolean
which will cause an error on boot because that new column was added by hibernate
Error Code : 0
Message : ERROR: column "resent" of relation "auth" already exists
Location : db/migration/V1__Add_Resent_To_Auth.sql (/Users/brian/code/slap/build/resources/main/db/migration/V1__Add_Resent_To_Auth.sql)
Line : 1
Statement : ALTER TABLE public.auth
ADD COLUMN resent boolean
So how do I tell flyway that the current version is V9 and only run migrations after that. Shouldn't it just go look at the flyway_schema_history and see version 9 is the last entry then run migrations after that? I must be missing something
I tried doing this in my config to set the baseline version first
#Configuration
class FlyWay() {
#Value("\${spring.datasource.url}")
lateinit var url: String
#Value("\${spring.datasource.username}")
lateinit var username: String
#Value("\${spring.datasource.password}")
lateinit var password: String
#Bean
fun migrate() {
val flyway = Flyway.configure().baselineVersion("9.0").dataSource(url, username, password).load()
flyway.migrate()
}
}
no such luck it still tries to run V1
I tried adding it to application.properties too
spring.flyway.baselineVersion=9.0
same error
Scenario as I understood it:
Tables already exist
State of the tables corresponds to version "9.0"
Flyway baseline version should be set once for the local test DB
It might be useful to set the version via command line, since it is to be applied to the test database only once and then the normal migration strategies are to be applied.
Documentation see: https://flywaydb.org/documentation/usage/commandline/baseline
On macOS the Flyway command line client can be installed with brew install flyway.
Instructions
make sure the table flyway_schema_history is deleted. Use your preferred SQL client:
drop table flyway_schema_history;
then set the baseline version using the Flyway command line client (this example uses a Postgres database):
flyway -user=stephan -password= -url=jdbc:postgresql://localhost:5432/stephan -baselineVersion="9.0" -locations="src/main/resources/db/migration" baseline
check in SQL client:
select version from flyway_schema_history ;
This should show now "9.0". After that, the Spring Boot application should behave as usual.
Test
Alternative
For those people who prefer to do this with a Maven command:
drop the table flyway_schema_history like shown above
use the command mvn flyway:baseline -Dflyway.baselineVersion="9.0" to set the baseline version
This requires a bit of configuration in the pom.xml file, e.g. if using a Postgres database:
<build>
...
<plugins>
...
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>7.1.1</version>
<configuration>
<url>jdbc:postgresql://localhost:5432/stephan</url>
<user>stephan</user>
</configuration>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.2.18</version>
</dependency>
</dependencies>
</plugin>
...
Test with Maven
A quick test shows the same result.
Why not export the SQL-script from your database (created by Hibernate) and add it as the first Flyway script into your application? It's the cleanest solution as Hibernate doesn't need to be started manually again when the application will run on other systems.
Just try once after adding the following line in your application.yml
spring.flyway.baseline-on-migrate: true
Related
I got latest Spring Boot app and springdoc.swagger-ui on board.
<dependency>
<groupId>io.swagger</groupId>
<artifactId>swagger-annotations</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-ui</artifactId>
<version>1.2.32</version>
</dependency>
My application.properties contains springdoc.swagger-ui.path=/swagger-ui-openapi.html
When I run application via Intellij IDEA http://localhost:8080/swagger-ui-openapi.html brings me to http://localhost:8080/swagger-ui/index.html?configUrl=/v3/api-docs/swagger-config
and Swagger UI page loads successfully.
But if I start the app via command line: "java -jar my-app.jar", I got 404 in browser and Error in logs 'Circular view path [error]' when trying to reach http://localhost:8080/swagger-ui-openapi.html
and it redirrects me to http://localhost:8080/swagger-ui/index.html?configUrl=/v3/api-docs/swagger-config
javax.servlet.ServletException: Circular view path [error]: would dispatch back to the current handler URL [/error] again. Check your ViewResolver setup! (Hint: This may be the result of an unspecified view, due to default view name generation.)
However http://localhost:8080/v3/api-docs is reachable and schema is available at this address.
How can I fix this?
What worked in my case when your application is running behind a proxy, a load-balancer or in the cloud.
In your Spring Boot application make sure your application handles this header: X-Forwarded-For.
There are two ways to achieve this:
In your properties file add:
server.use-forward-headers=true
If this is not enough, Spring Framework provides a ForwardedHeaderFilter. You can register it as a Servlet Filter in your application by setting server.forward-headers-strategy is set to FRAMEWORK.
Since Spring Boot 2.2, this is the new property to handle reverse proxy headers:
In your properties file add
server.forward-headers-strategy=framework
And you can add the following bean to your application:
#Bean
ForwardedHeaderFilter forwardedHeaderFilter() {
return new ForwardedHeaderFilter();
}
If you already have static content on your root, and you don’t want it to be overridden by springdoc-openapi-ui configuration, you can just define a custom configuration of the swagger-ui, in order not to override the configuration of your files from in your context-root:
For example use in your properties file:
springdoc.swagger-ui.path= /swagger-ui/api-docs.html
ref:
https://springdoc.org/
For this problem, my conclusion is:
(1) Starting it in IDEA is fine
(2) Repackaging the jar with spring-boot-maven-plugin and starting it with 'java -jar' is fine as well.
(3) if I tried to starting with such as 'java -classpath ".:.conf" DemoApplication', it does not work.
So, for packaging, i use the spring-boot-maven-plugin.
You don't need swagger-annotations v1.6.1 dependency for springdoc-openapi;
By default, with springdoc you need no additonal settings of any ViewResolver.
You can have a look at some sample code:
https://github.com/springdoc/springdoc-openapi-demos
I'm trying to execute some DB initialization for a Spring Boot application against a MySQL Database that is running in a container.
During the authentication process, I receive an error "Table not found". I've checked the DB and no tables have been created indeed.
Is there something missing in DB properties?
spring.datasource.url = jdbc:mysql://172.17.0.2:3306/schema
spring.datasource.username = user
spring.datasource.password = password
spring.datasource.testWhileIdle = true
spring.datasource.validationQuery = SELECT 1
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto = create-drop
spring.jpa.hibernate.naming-strategy = org.hibernate.cfg.ImprovedNamingStrategy
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.data.rest.basePath=/
spring.datasource.data = classpath:/data.sql
spring.datasource.schema = classpath:/schema.sql
All in all, the JDBC Settings work fine provided that I create the DDL from the mysql command line. So it's just not executing the data.sql and schema.sql at startup.
Do I need some extra properties for mysql ?
You can update the application.properties with following properties.
application.properties:
spring.jpa.hibernate.ddl-auto=none
spring.jpa.generate-ddl=false
spring.datasource.initialization-mode=always
By default value of spring.jpa.generate-ddl is false. If you set spring.jpa.generate-ddl=true or spring.jpa.hibernate.ddl-auto to any of value validate, update, create, create-drop. Spring boot generates schema scripts and creates the tables based on the entities available in your application.
Now you should create the schema.sql and data.sql files under resources folder and set the property spring.datasource.initialization-mode=always in application.properties.
You can also run the scripts based on the platform. Let's suppose you want to run the scripts for hsqldb database, then set spring.datasource.platform=hsqldb in application.properties file and created scripts file schema-hsqldb.sql and data-hsqldb.sql under resources folder.
You can find more details about how to load schema.sql and data.sql on startup with spring boot.
Everything works as expected until a spring-boot update the pom file from
<version>1.5.8.RELEASE</version>
to
<version>2.3.3.RELEASE</version>
A switch back and update all test are passed without any exceptions. In Version 2.3.3 a test fault caused by missing data in database.
I've spend hours to solve the ddl initialization for Spring Version 2.3.3.RELEASE without find regular solution.
As you have
spring.jpa.hibernate.ddl-auto = create-drop
Spring doesn't run the schema.sql and data.sql.
Try it with
spring.jpa.hibernate.ddl-auto = none
Check the docs
In a JPA-based app, you can choose to let Hibernate create the schema or use schema.sql, but you cannot do both. Make sure to disable spring.jpa.hibernate.ddl-auto if you use schema.sql.
For spring boot version 2.7, ensure that you have configured spring.sql.init.mode property in the application.properties file
Example:
spring.sql.init.mode=always
spring.jpa.hibernate.ddl-auto=update
spring.jpa.generate-ddl=true
I am trying to configure flyway with kotlin Spring boot, jpa and postgreSQL. My gradle dependencies are:
dependencies {
implementation('org.springframework.boot:spring-boot-starter-data-jpa')
implementation('org.springframework.boot:spring-boot-starter-web')
implementation('com.fasterxml.jackson.module:jackson-module-kotlin')
implementation('org.flywaydb:flyway-core')
implementation('com.google.code.gson:gson')
implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
implementation("org.jetbrains.kotlin:kotlin-reflect")
runtimeOnly('org.postgresql:postgresql')
testImplementation('org.springframework.boot:spring-boot-starter-test')
}
My application.properties file is:
spring.datasource.driverClassName=org.postgresql.Driver
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.url=jdbc:postgresql://${JDBC_DATABASE_URL}/jpaTestDatabase
spring.datasource.username=${JDBC_DATABASE_USERNAME}
spring.datasource.password=${JDBC_DATABASE_PASSWORD}
flyway.baseline-on-migrate=true
flyway.locations=classpath:src/main/kotlin/db/migration
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=validate
spring.session.store-type=none
Creating tables and entries using jpa and hibernate works as expected.
However a sample migration on an empty database results in:
org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
Invocation of init method failed; nested exception is org.flywaydb.core.api.FlywayException:
Found non-empty schema(s) "public" without schema history table! Use baseline() or set baselineOnMigrate to true to initialize the schema history table.
my directory structure is the default one generated by spring initializr and my migrations are in: demo/src/main/kotlin/db/migration
I only have a single migration which is the kotlinized version of the example migration found here which I adapted to look line this:
class V1__Sample : BaseJavaMigration() {
override fun migrate(context: Context?) {
val statement = context?.connection?.prepareStatement(
"""
CREATE TABLE article (
id bigserial primary key,
name varchar(20) NOT NULL,
desc text NOT NULL
);
"""
)
statement.use { it?.execute() }
}
}
What am I missing here? Why does Flyway keep complaining about finding non-empty schema(s) "public" without schema history table, when the database is completelly empty (clean docker image)?
Assuming that you are using spring-boot version 2.
In spring boot 2 the prefix is "spring.flyway" so try adding prefix spring like below.
spring.flyway.baseline-on-migrate = true
OR
spring.flyway.baselineOnMigrate = true
may be you can try mvn flyway:clean && mvn flyway:migrate
Please check the search path of your database, if the public schema (on which flyway is creating its log tables) is not in the first place, it may not be able to find the log table and may complain that schema history is not found...
Please note that, if you are baselining, you need to remove old scripts from the scripts folder, else it will re-attempt it.
On my local machine I load an in-memory h2 database to start my spring boot application in a safe environment, here's the properties:
spring.datasource.url: jdbc:h2:mem:DB_TEST;Mode=Oracle
spring.datasource.platform: h2
spring.jpa.hibernate.ddl-auto: none
spring.datasource.continue-on-error: false
spring.jpa.database-platform: org.hibernate.dialect.Oracle10gDialect
Then, in my src/main/resources I have the file schema-h2.sql containing my local db initiations.
That's fine, but then I also have some junit tests I want to execute:
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyTest {
#Autowired
private MyController controller;
#Test
public void myTest(){
controller.doSomething();
}
This is also fine, as the schema-h2.sql is seen.
Anyway according to me it would be better to put the schema-h2.sql in src/test/resources as it has to be used only on my local environment. Doing so also allows maven to exclude it from the final build and that is also pretty fine.
Anyway if I put it there the test keeps working...but the main application breaks as the schema-h2.sql is not found!
How to modify the above properties to specify that the shema-h2.sql has to be searched inside of the test/resources folder?
Thanks
For normal mode, the properties file is put in src/main/resources,
and for the testing method, the properties file in the src/test/resources folder.
By Trying to run a test-class, eclipse runs EACH file ending with .sql (and thus containing a script to create tables or to insert data) it finds under src/main/resources and src/test/resources.
So if you put a script-file schema.sql (containing a script that creates a table: create table ..) in both folders, you'll get an "table already exits" error, if you let jut one, the test will run smoothly.
If you put a script-file (that insert data in a table) in both folder, both scripts will be run.
You can also use the #PropertySource("..") in your repository to tell spring where to find the properties-file to use.
I want to pass variable from maven command line to spring boot main class,but i have no idea,here is my maven pom file:
<properties>
<maven.tomcat.home>Here should add argument from command line</maven.tomcat.home>
</properties>
Here is my spring boot main class:
#PropertySource(value ={"file:#maven.tomcat.home#/em/easymobile-application.properties"})
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
So when i run maven by command line as below ,the application will deploy as war file to tomcat
mvn clean install -Dmaven.tomcat.home=/usr/local/apache-tomcat-8.0.33/webapps
Then i start up tomcat,the error says easymobile-application.properties is not found,so do you know how to load the easymobile-application.properties from maven command line using #PropertySource or something else.Thanks!!
I don't think that's the right approach, assuming at some point you get that to work, what happens if you need / have to deploy the webapp to /usr/local/apache-tomcat-8.0.xx? Would you need to compile the artifact again? That's not good. A better approach is to only produce 1 binary / artifact that could be deployed / installed in every environment: staging / prod / etc..
Like mentioned before, if the app needs files path configuration, I would suggest to use a config properties file instead of a Maven property.
Thanks! I have solved the problem using a simple way ,i just add the path of the folder which contains application.properties in tomcat\bin\catalina.sh
CLASSPATH=:/usr/local/apache-tomcat-8.0.33
the #PropertySource points to "classpath:em/easymobile-application.properties"