Completely auto DB upgradable Spring boot application - spring-boot

I am trying to use flyway for DB migrations and Spring boot's flyway support for auto-upgrading DB upon application start-up and subsequently this database will be used by my JPA layer
However this requires that schema be present in the DB so that primary datasource initialization is successful. What are the options available to run a SQL script that will create the required schema before flyway migrations happen.
Note that If I use flyway gradle plugin (and give the URL as jdbc:mysql://localhost/mysql. It does create the schema for me. Am wondering if I could make this happen from Java code on application startup.

Flyway does not support full installation when schema is empty, just migration-by-migration execution.
You could though add schema/user creation scripts in the first migration, though then your migration scripts need to be executed with sysdba/root/admin user and you need to set current schema at the beginning of each migration.
If using Flyway, the least problematic way is to install schema for the first time manually and do a baseline Flyway task (also manually). Then you are ready for next migrations to be done automatically.
Although Flyway is a great tool for database migrations it does not cover this particular use case well (installing schema for the first time).

"Am wondering if I could make this happen from Java code on application startup."
The simple answer is yes as Flyway supports programmatic configuration from with java applications. The starting point in the flyway documentation can be found here
https://flywaydb.org/documentation/api/
flyway works with a standard JDBC DataSource and so you can code the database creation process in Java and then have flyway handle the schema management. In many environment you are likely to require 2 steps anyway as the database/schema creation will need admin rights to the database, while the ongoing schema management will need an account with reduced access rights.

what you need is to implement the interface FlywayCallback
in order to kick start the migration manually from you code you can use the migrate() method on the flyway class
tracking the migration process can be done through the MigrationInfoService() method of the flyway class

Unfortunately if your app has a single datasource that expects the schema to exist, Flyway will not be able to use that datasource to create the scheme. You must create another datasource that is not bound to the schema and use the unbounded datasource by way of a FlywayMigrationStrategy.
In your properties file:
spring:
datasource:
url: jdbc:mysql://localhost:3306/myschema
bootstrapDatasource:
url: jdbc:mysql://localhost:3306
In your config file:
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties primaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSource primaryDataSource() {
return primaryDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#ConfigurationProperties("spring.bootstrapDatasource")
public DataSource bootstrapDataSource() {
return DataSourceBuilder.create().build();
}
And in your FlywayMigrationStrategy file:
#Inject
#Qualifier("bootstrapDataSource")
public void setBootstrapDataSource(DataSource bootstrapDataSource) {
this.bootstrapDataSource = bootstrapDataSource;
}
#Override
public void migrate(Flyway flyway) {
flyway.setDataSource(bootstrapDataSource);
...
flyway.migrate()
}

Related

Is it possible to have two embedded databases running in Spring Boot that are populated using spring.jpa.generate-ddl?

I have two databases that I connect to in my application.
I want to set up a dev-only profile that mocks these databases using an embedded H2 database, and I would like to have their schemas auto-created by using spring.jpa.generate-ddl=true. The entity classes for each database are in different java packages, which I hope might help me here.
Is such a thing possible using spring's autoconf mechanisms?
It is possible to use multiple databases in spring boot.
But spring boot can automatically configure only one database.
You need to configure the second database yourself.
#Bean
#ConfigurationProperties(prefix="second.datasource")
public DataSource secondDataSource(){
return DataSourceBuilder
.create()
.driverClassName("org.h2.Driver")
.build();
}
If you just need a jdbc connection, this would be already sufficient. As you want to use JPA you need also a second JPA configuration, that uses the second data source.
#Bean(name="secondEntityManager")
public LocalContainerEntityManagerFactoryBean mySqlEntityManagerFactory(EntityManagerFactoryBuilder builder,DataSource secondDataSource){
return builder.dataSource(secondDataSource)
.packages("com.second.entity")
.build();
}
You can find the code above and more in this post

Neo4J ogm testing create temporary database

i'm using spring and at the moment running my tests create new objects in my "real" embedded database. I want to create a new one or a temporary db just for testing. I'm new with spring and neo4j so could anyone please help?
thanks a lot
If you are using the embedded driver with SDN/OGM you just need to configure it without providing a path. Then it will create embedded database in /tmp/.. which gets deleted on jvm exit.
E.g. if you are using java configuration
#Bean
public Configuration getConfiguration() {
Configuration config = new Configuration();
config
.driverConfiguration()
.setDriverClassName("org.neo4j.ogm.drivers.embedded.driver.EmbeddedDriver");
return config;
}
See docs for full documentation
http://docs.spring.io/spring-data/data-neo4j/docs/current/reference/html/#_configuring_the_embedded_driver

How to disable H2's DATABASE_TO_UPPER in Spring Boot, without explicit connection URL

I'm aware that H2 has a boolean property/setting called DATABASE_TO_UPPER, which you can set at least in the connection URL, as in: ;DATABASE_TO_UPPER=false
I’d like to set this to false, but in my Spring Boot app, I don’t explicitly have a H2 connection URL anywhere. Implicitly there sure is a connection URL though, as I can see in the logs:
o.s.j.d.e.EmbeddedDatabaseFactory: Shutting down embedded database:
url='jdbc:h2:mem:2fb4805b-f927-49b3-a786-2a2cac440f44;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
So the question is, what's the easiest way to tell H2 to disable DATABASE_TO_UPPER in this scenario? Can I do it in code when creating the H2 datasource with EmbeddedDatabaseBuilder (see below)? Or in application properties maybe?
This is how the H2 database is explicitly initialised in code:
#Configuration
#EnableTransactionManagement
public class DataSourceConfig {
#Bean
public DataSource devDataSource() {
return new EmbeddedDatabaseBuilder()
.generateUniqueName(true)
.setType(EmbeddedDatabaseType.H2)
.setScriptEncoding("UTF-8")
.ignoreFailedDrops(true)
.addScripts("db/init.sql", "db/schema.sql", "db/test_data.sql")
.build();
}
}
Also, I'm telling JPA/Hibernate not to auto-generate embedded database (without this there was an issue that two in-memory databases were launched):
spring.jpa.generate-ddl=false
spring.jpa.hibernate.ddl-auto=none
You can't w\ the generateUniqueName, but if you call setName("testdb;DATABASE_TO_UPPER=false") you can add parameters. I doubt this is officially supported, but it worked for me.
The spring code that generates the connection url is like this:
String.format("jdbc:h2:mem:%s;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false", databaseName)
You may want abandon using explicit creation via EmbeddedDatabaseBuilder. Spring Boot creates H2 instance automatically based on configuration. So I would try this in application.properties:
spring.datasource.url=jdbc:h2:file:~/testdb;DATABASE_TO_UPPER=false

Spring Boot / Spring Data import.sql doesn't run Spring-Boot-1.0.0.RC1

I've been following the development of Spring Boot, and sometime between the initial version 0.0.5-BUILD-SNAPSHOT and the current version I am using 1.0.0.RC1 I am no longer running my import.sql script.
Here is my configuration for LocalContainerEntityManager and JpaVendorAdapter
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
DataSource dataSource, JpaVendorAdapter jpaVendorAdapter) {
LocalContainerEntityManagerFactoryBean lef = new LocalContainerEntityManagerFactoryBean();
lef.setDataSource(dataSource);
lef.setJpaVendorAdapter(jpaVendorAdapter);
lef.setPackagesToScan("foo.*");
return lef;
}
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
hibernateJpaVendorAdapter.setShowSql(true);
hibernateJpaVendorAdapter.setGenerateDdl(true);
hibernateJpaVendorAdapter.setDatabase(Database.POSTGRESQL);
return hibernateJpaVendorAdapter;
}
Interesting the hibernate.hbm2ddl.auto still seems to run, which I think is part of the definition of my SpringBootServletInitializer
#Configuration
#ComponentScan
#EnableAutoConfiguration
public class Application extends SpringBootServletInitializer {
However, I also noticed that the tables generated no longer have underscores and changed their shape when generated?
However, that could be the result of updating my org.postgresql version like so:
Previously:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.2-1004-jdbc41</version>
</dependency>
Now:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>9.3-1100-jdbc41</version>
</dependency>
I also had to change pggetserialsequence to pg_get_serial_sequence to get the script to run at all from pgadmin?
I guess I'm confusing what's going on, but most importantly I want to get back to having my import.sql run.
I have been following the sample project: https://github.com/spring-projects/spring-boot/tree/master/spring-boot-samples/spring-boot-sample-data-jpa
And their import.sql isn't running either on 1.0.0-BUILD-SNAPSHOT
The import.sql script is a Hibernate feature I think (not Spring or Spring Boot). It must be running in the sample otherwise the tests would fail, but in any case it only runs if ddl-auto is set to create the tables. With Spring Boot you should ensure that spring.jpa.hibernate.ddl-auto is set to "create" or "create-drop" (the latter is the default in Boot for an embedded database, but not for others, e.g. postgres).
If you want to unconditionally run a SQL script, By default Spring Boot will run one independent of Hibernate settings if you put it in classpath:schema.sql (or classpath:schema-<platform>.sql where <platform> is "postgres" in your case).
I think you can probably delete the JpaVendorAdapter and also the LocalContainerEntityManagerFactoryBean (unless you are using persistence.xml) and let Boot take control. The packages to scan can be set using an #EntityScan annotation (new in Spring Boot).
The default table naming scheme was changed in Boot 1.0.0.RC1 (so nothing to do with your postgres dependency). I'm not sure that will still be the case in RC2, but anyway you can go back to the old Hibernate defaults by setting spring.jpa.hibernate.naming-strategy=org.hibernate.cfg.ImprovedNamingStrategy.
Hey I came across similar issue. My sql script was not getting invoked initially. Then I tried renaming the file from "import.sql" to "schema.sql", it worked. May be give this a shot. My code can be found here - https://github.com/sidnan/spring-batch-example
In addition to what was already said, it's worth noting you can use the data.sql file to import/intialize data into your tables. Just put your data.sql into the root of the classpath (eg: if you're running a Spring Boot app, you put it in the src/main/resources path).
Like was said before, use it together with the property ddl-auto=create-drop, so that it won't crash trying to insert the existing data.
You can also set up which specific file to execute using the spring.datasource.data property. Check out more info here: http://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html
Note: the schema.sql mentioned before would contain the whole DB definition. If you want to use this, ensure that Hibernate doesn't try to construct the DB for you based on the Java Entities from your project. This is what de doc says:
If you want to use the schema.sql initialization in a JPA app (with
Hibernate) then ddl-auto=create-drop will lead to errors if Hibernate
tries to create the same tables. To avoid those errors set ddl-auto
explicitly to "" (preferable) or "none"

Auto insert default record when deploying Spring MVC App with Spring Security

I am looking for a way to auto insert a default admin account, using JPA, when my spring mvc application is deployed.
My database is generated based on the Entities.
I want to kick off something that will insert a default admin user, assign roles, every time the application is deployed.
It depends on which implementation of JPA you use.
If you use Hibernate you can add import.sql file (that contains records to load) to the class path. More info here.
As a workaround you can also use dbunit tool.
I would recommend having a migration utility that will keep your database in synch with your codebase - these are typically DDL's, but again the queries to insert default admin user, assign roles etc can also be part of this migration utility. There are very good one's available - Flyway is one that I have used, Liquibase is another one.
There is a very good comparison of the different migration utilities on the Flyway homepage also that you can look at.
i use CommandLineRunner interface.
#Component
public class CommandLineAppStartupRunner implements CommandLineRunner {
#Autowired
UserRepository userRepository;
#Override
public void run(String...args) throws Exception {
User admin = new user(firstName);
userRepository.save(admin);
}
}
before the app starts this class will be executed.
you can find other ways here : Guide To Running Logic on Startup in Spring

Resources