How to drop in-memory h2db between Spring Integration tests? - spring

Question

I am using Liquibase in my Spring web application. I have a bunch of entities with hundreds of tests for REST APIs in the integration tests for each entity like User, Account, Invoice, License etc. All of my integration tests pass when run by class but a lot of them fail when ran together using gradle test. It is very likely there is data collision between the tests and I am not interested in spending time to fix clean up data as of now. I prefer dropping the DB and context after every class. I figured I could use #DirtiesContext at class and so I annotated by test with it.
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Application.class, SecurityConfiguration.class},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#DirtiesContext
public class InvoiceResourceIntTest {
I see that after adding the annotation, web application context starts for every class but when the Liquibase initialization happens, the queries are not run because checksum matches. Since this is an in-memory DB, I expected the DB to be destroyed along with spring context but it is not happening.
I also set jpa hibernate ddl-auto to create-drop but that did not help. The next option I am considering is instead of mem, write h2db to a file and delete that file in #BeforeClass of my integration test class files. I prefer dropping db automatically in memory instead of managing it in test, but want to try here as a last option. Thanks for the help.
Update:
I updated test as below.
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Application.class, SecurityConfiguration.class},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
properties = "spring.datasource.name=AccountResource")
#DirtiesContext
public class AccountResourceIntTest {
I have set unique names for each integration test. I still don't see the database being new because I can see only Liquibase checksum in the logs.
Here is my app config from application.yml
spring:
datasource:
driver-class-name: org.h2.Driver
url: jdbc:h2:mem:myApp;DB_CLOSE_DELAY=-1
name:
username:
password:
jpa:
database-platform: com.neustar.registry.le.domain.util.FixedH2Dialect
database: H2
open-in-view: false
show_sql: true
hibernate:
ddl-auto: create-drop
naming-strategy: org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
properties:
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: true
hibernate.hbm2ddl.auto: validate
My project is generated from JHipster 2.x version if it matters. Please see my database configuration class below. AppProperties are application specific properties (Different from Spring).
#Configuration
public class DatabaseConfiguration {
private static final int LIQUIBASE_POOL_INIT_SIZE = 1;
private static final int LIQUIBASE_POOL_MAX_ACTIVE = 1;
private static final int LIQUIBASE_POOL_MAX_IDLE = 0;
private static final int LIQUIBASE_POOL_MIN_IDLE = 0;
private static final Logger LOG = LoggerFactory.getLogger(DatabaseConfiguration.class);
/**
* Creates data source.
*
* #param dataSourceProperties Data source properties configured.
* #param appProperties the app properties
* #return Data source.
*/
#Bean(destroyMethod = "close")
#ConditionalOnClass(org.apache.tomcat.jdbc.pool.DataSource.class)
#Primary
public DataSource dataSource(final DataSourceProperties dataSourceProperties,
final AppProperties appProperties) {
LOG.info("Configuring Datasource with url: {}, user: {}",
dataSourceProperties.getUrl(), dataSourceProperties.getUsername());
if (dataSourceProperties.getUrl() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database URL!");
throw new ApplicationContextException("Data source is not configured correctly, please specify URL");
}
if (dataSourceProperties.getUsername() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database user!");
throw new ApplicationContextException(
"Data source is not configured correctly, please specify database user");
}
if (dataSourceProperties.getPassword() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database password!");
throw new ApplicationContextException(
"Data source is not configured correctly, "
+ "please specify database password");
}
PoolProperties config = new PoolProperties();
config.setDriverClassName(dataSourceProperties.getDriverClassName());
config.setUrl(dataSourceProperties.getUrl());
config.setUsername(dataSourceProperties.getUsername());
config.setPassword(dataSourceProperties.getPassword());
config.setInitialSize(appProperties.getDatasource().getInitialSize());
config.setMaxActive(appProperties.getDatasource().getMaxActive());
config.setTestOnBorrow(appProperties.getDatasource().isTestOnBorrow());
config.setValidationQuery(appProperties.getDatasource().getValidationQuery());
org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource(config);
LOG.info("Data source is created: {}", dataSource);
return dataSource;
}
/**
* Create data source for Liquibase using dba user and password provided for "liquibase"
* in application.yml.
*
* #param dataSourceProperties Data source properties
* #param liquibaseProperties Liquibase properties.
* #param appProperties the app properties
* #return Data source for liquibase.
*/
#Bean(destroyMethod = "close")
#ConditionalOnClass(org.apache.tomcat.jdbc.pool.DataSource.class)
public DataSource liquibaseDataSource(final DataSourceProperties dataSourceProperties,
final LiquibaseProperties liquibaseProperties, final AppProperties appProperties) {
LOG.info("Configuring Liquibase Datasource with url: {}, user: {}",
dataSourceProperties.getUrl(), liquibaseProperties.getUser());
/*
* This is needed for integration testing. When we run integration tests using SpringJUnit4ClassRunner, Spring
* uses
* H2DB if it is in the class path. In that case, we have to create pool for H2DB.
* Need to find a better solution for this.
*/
if (dataSourceProperties.getDriverClassName() != null
&& dataSourceProperties.getDriverClassName().startsWith("org.h2.")) {
return dataSource(dataSourceProperties, appProperties);
}
if (dataSourceProperties.getUrl() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database URL!");
throw new ApplicationContextException("Liquibase is not configured correctly, please specify URL");
}
if (liquibaseProperties.getUser() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database user!");
throw new ApplicationContextException(
"Liquibase is not configured correctly, please specify database user");
}
if (liquibaseProperties.getPassword() == null) {
LOG.error("Your Liquibase configuration is incorrect, please specify database password!");
throw new ApplicationContextException(
"Liquibase is not configured correctly, please specify database password");
}
PoolProperties config = new PoolProperties();
config.setDriverClassName(dataSourceProperties.getDriverClassName());
config.setUrl(dataSourceProperties.getUrl());
config.setUsername(liquibaseProperties.getUser());
config.setPassword(liquibaseProperties.getPassword());
// for liquibase pool, we dont need more than 1 connection
config.setInitialSize(LIQUIBASE_POOL_INIT_SIZE);
config.setMaxActive(LIQUIBASE_POOL_MAX_ACTIVE);
// for liquibase pool, we dont want any connections to linger around
config.setMaxIdle(LIQUIBASE_POOL_MAX_IDLE);
config.setMinIdle(LIQUIBASE_POOL_MIN_IDLE);
org.apache.tomcat.jdbc.pool.DataSource dataSource = new org.apache.tomcat.jdbc.pool.DataSource(config);
LOG.info("Liquibase data source is created: {}", dataSource);
return dataSource;
}
/**
* Creates a liquibase instance.
*
* #param dataSource Data source to use for liquibase.
* #param dataSourceProperties Datasource properties.
* #param liquibaseProperties Liquibase properties.
* #return Liquibase instance to be used in spring.
*/
#Bean
public SpringLiquibase liquibase(#Qualifier("liquibaseDataSource") final DataSource dataSource,
final DataSourceProperties dataSourceProperties, final LiquibaseProperties liquibaseProperties) {
// Use liquibase.integration.spring.SpringLiquibase if you don't want Liquibase to start asynchronously
SpringLiquibase liquibase = new AsyncSpringLiquibase();
liquibase.setDataSource(dataSource);
liquibase.setChangeLog("classpath:config/liquibase/master.xml");
liquibase.setContexts(liquibaseProperties.getContexts());
liquibase.setDefaultSchema(liquibaseProperties.getDefaultSchema());
liquibase.setDropFirst(liquibaseProperties.isDropFirst());
liquibase.setShouldRun(liquibaseProperties.isEnabled());
return liquibase;
}
}

Answer 1

This is because each test shares the same database and that the lifecycle of H2 is not in our control. If you start a process (the VM) and require a database named foo, close the application context, start a new one and require foo again you'll get the same instance.
In the upcoming 1.4.2 release we've added a property to generate a unique name for the database on startup (see spring.datasource.generate-unique-name) and that value will be set to true by default on 1.5.
In the meantime, you can annotate each test with #SpringBootTest(properties="spring.datasource.name=xyz") where xyz is different for a test that requires a separate DB.

Answer 2

If I understand everything correctly liquibase takes care of database status. For every file, also for the test data, liquibase creates a checksum in a table to check whether something has changed or not. The h2 instance still alive after a #DirtiesContext so the checksums still exists in the database. Liquibase thinks that everything is correct but the test data may have changed.
To force liquibase to drop the database and recreate a completely new database you must set the properties in application.yml (that one for tests):
liquibase:
contexts: test
drop-first: true
or as an alternative you can hardcode it:
liquibase.setDropFirst(true);
You can either annotate your test with #DirtiesContext, which slows down the test because the whole application context gets rebuild.
Or you can create a custom TestExecutionListener which is much faster. I've created a custom TestExecutionListener, which recreates the database and keeps the context.
public class CleanUpDatabaseTestExecutionListener
extends AbstractTestExecutionListener {
#Inject
SpringLiquibase liquibase;
#Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE;
}
#Override
public void afterTestClass(TestContext testContext) throws Exception {
//This is a bit dirty but it works well
testContext.getApplicationContext()
.getAutowireCapableBeanFactory()
.autowireBean(this);
liquibase.afterPropertiesSet();
}
if you are using the TestExecutionListener you must add this Listener to your test with:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = Application.class)
#WebAppConfiguration
#IntegrationTest
#TestExecutionListeners(listeners = {
DependencyInjectionTestExecutionListener.class,
TransactionalTestExecutionListener.class,
CleanUpDatabaseTestExecutionListener.class,
})
public class Test {
//your tests
}
NOTE: DO NOT USE #DirtiesContext and the TestExecutionListener together, this will lead to an error.

Answer 3

Solved by removing username, url and password parameters.
spring:
autoconfigure:
exclude: org.springframework.boot.autoconfigure.security.SecurityAutoConfiguration
jackson:
serialization:
indent_output: true
datasource:
driver-class-name: org.hsqldb.jdbcDriver
generate-unique-name: true
jpa:
hibernate:
dialect: org.hibernate.dialect.HSQLDialect
ddl-auto: validate
show-sql: true
h2:
console:
enabled: false
liquibase:
change-log: classpath:/liquibase/db.changelog-master.xml
drop-first: true
contexts: QA