Spring #Transactional and rollback not working, spring integration tests - spring

I'm doing an integration test in spring and in this example I testing a service layer.
I have a problem where during the addition test in the service, rollback not working, and always add an item to the base ,but not delete it.
I put annotation # Transactional and # TestPropertySource on test class,
also have application-test.properties and the test is successful but the rollback is not executed and a new item is always added to the test database.
My test class and test method for add address(last one):
#RunWith(SpringRunner.class)
#SpringBootTest(classes = TicketServiceApplication.class)
#Transactional
#TestPropertySource("classpath:application-test.properties")
public class AddressServiceIntegrationTest {
#Autowired
private AddressService addressService;
#Autowired
private AddressRepository addressRepository;
#Test
public void findAllSuccessTest(){
List<Address> result = addressService.finfAllAddress();
assertNotNull(result);
assertFalse(result.isEmpty());
assertEquals(2, result.size());
}
#Test
public void findOneAddressExistTest_thenReturnAddress(){
Long id = AddressConst.VALID_ID_ADDRESS;
Address a = addressService.findOneAddress(id);
assertEquals(id, a.getId());
}
#Test(expected = EntityNotFoundException.class)
public void findOneAddressNotExistTest_thenThrowException(){
Long id = AddressConst.NOT_VALID_ID_ADDRESS;
Address a = addressService.findOneAddress(id);
}
#Test
public void addAddressSuccessTest(){
int sizeBeforeAdd = addressRepository.findAll().size();
Address address = AddressConst.newAddressToAdd();
Address result = addressService.addAddress(address);
int sizeAfterAdd = addressRepository.findAll().size();
assertNotNull(result);
assertEquals(sizeBeforeAdd+1, sizeAfterAdd);
assertEquals(address.getCity(), result.getCity());
assertEquals(address.getState(), result.getState());
assertEquals(address.getNumber(), result.getNumber());
assertEquals(address.getLatitude(), result.getLatitude());
assertEquals(address.getLongitude(), result.getLongitude());
assertEquals(address.getStreet(), result.getStreet());
}
}
My application-test.properties :
spring.datasource.url= jdbc:mysql://localhost:3306/kts_test&useSSL=false&
useUnicode=true&characterEncoding=utf8
spring.datasource.username = root
spring.datasource.password = root
spring.jpa.show-sql = true
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
And when execute add address test, every time in my kts_test data base ( db used for testing) is added new item and not rollback.
This is a log from the console where you can see that the rollback was called but did not execute, because when I refresh the database after the test the new item was left, it was not deleted.
INFO 10216 --- [ main] o.s.t.c.transaction.TransactionContext : Rolled back transaction for test: [DefaultTestContext#3e58a80e testClass = AddressServiceIntegrationTest, testInstance = com.ftn.services.address.AddressServiceIntegrationTest#4678ec43, testMethod = addAddressSuccessTest#AddressServiceIntegrationTest, testException = [null],...
Lastly, to write that I tried #Transactional above methods, I also tried # Rollback above methods or # Rollback (true), tried changing application-test.properties and I no longer know what the error might be.
If anyone can help I would be grateful. Thank you.

Hibernate is by default creating tables with MyISAM storage engine - this engine does not support transactions. You need to have InnoDB tables:
Either manage your DB migrations manually with a tool like Flyway (imho preferred option so you can have a complete control) and turn off recreating database in tests.
Or set the correct engine in configuration:
spring.jpa.properties.hibernate.dialect.storage_engine=innodb
OR (deprecated)
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect

Related

Populate a database with TestContainers in a SpringBoot integration test

I am testing TestContainers and I would like to know how to populate a database executing a .sql file to create the structure and add some rows.
How to do it?
#Rule
public PostgreSQLContainer postgres = new PostgreSQLContainer();
The easiest way is to use JdbcDatabaseContainer::withInitScript
Advantage of this solution is that script is run before Spring Application Context loads (at least when it is in a static block) and the code is quite simple.
Example:
static {
postgreSQLContainer = new PostgreSQLContainer("postgres:9.6.8")
.withDatabaseName("integration-tests-db")
.withUsername("sa")
.withPassword("sa");
postgreSQLContainer
.withInitScript("some/location/on/classpath/someScript.sql");
postgreSQLContainer.start();
}
JdbcDatabaseContainer is superclass of PostgreSQLContainer so this solution should work not only for postgres, but also for other containers.
If you want to run multiple scripts you can do it in a similar manner
Example:
static {
postgreSQLContainer = new PostgreSQLContainer("postgres:9.6.8")
.withDatabaseName("integration-tests-db")
.withUsername("sa")
.withPassword("sa");
postgreSQLContainer.start();
var containerDelegate = new JdbcDatabaseDelegate(postgreSQLContainer, "");
ScriptUtils.runInitScript(containerDelegate, "some/location/on/classpath/someScriptFirst.sql");
ScriptUtils.runInitScript(containerDelegate, "some/location/on/classpath/someScriptSecond.sql");
ScriptUtils.runInitScript(containerDelegate, "ssome/location/on/classpath/someScriptThird.sql");
}
There are also other options
Spring Test #Sql annotation
#SpringBootTest
#Sql(scripts = ["some/location/on/classpath/someScriptFirst.sql", "some/location/on/classpath/someScriptSecond.sql"])
public class SomeTest {
//...
}
ResourceDatabasePopulator from jdbc.datasource.init or r2dbc.connection.init when using JDBC or R2DBC consecutively
class DbInitializer {
private static boolean initialized = false;
#Autowired
void initializeDb(ConnectionFactory connectionFactory) {
if (!initialized) {
ResourceLoader resourceLoader = new DefaultResourceLoader();
Resource[] scripts = new Resource[] {
resourceLoader.getResource("classpath:some/location/on/classpath/someScriptFirst.sql"),
resourceLoader.getResource("classpath:some/location/on/classpath/someScriptSecond.sql"),
resourceLoader.getResource("classpath:some/location/on/classpath/someScriptThird.sql")
};
new ResourceDatabasePopulator(scripts).populate(connectionFactory).block();
initialized = true;
}
}
}
#SpringBootTest
#Import(DbInitializer.class)
public class SomeTest {
//...
}
Init script in database URI when using JDBC
It is mentioned in offical Testcontainers documentation:
https://www.testcontainers.org/modules/databases/jdbc/
Classpath file:
jdbc:tc:postgresql:9.6.8:///databasename?TC_INITSCRIPT=somepath/init_mysql.sql
File that is not on classpath, but its path is relative to the working directory, which will usually be the project root:
jdbc:tc:postgresql:9.6.8:///databasename?TC_INITSCRIPT=file:src/main/resources/init_mysql.sql
Using an init function:
jdbc:tc:postgresql:9.6.8:///databasename?TC_INITFUNCTION=org.testcontainers.jdbc.JDBCDriverTest::sampleInitFunction
package org.testcontainers.jdbc;
public class JDBCDriverTest {
public static void sampleInitFunction(Connection connection) throws SQLException {
// e.g. run schema setup or Flyway/liquibase/etc DB migrations here...
}
...
}
When using Spring Boot, I find it easiest to use the JDBC URL support of TestContainers.
You can create a application-integration-test.properties file (typically in src/test/resources with something like this:
spring.datasource.url=jdbc:tc:postgresql://localhost/myappdb
spring.datasource.driverClassName=org.testcontainers.jdbc.ContainerDatabaseDriver
spring.datasource.username=user
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.jpa.hibernate.ddl-auto=none
# This line is only needed if you are using flyway for database migrations
# and not using the default location of `db/migration`
spring.flyway.locations=classpath:db/migration/postgresql
Note the :tc part in the JDBC url.
You can now write a unit test like this:
#RunWith(SpringRunner.class)
#DataJpaTest
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) #ActiveProfiles("integration-test")
public class UserRepositoryIntegrationTest {
#Autowired
private MyObjectRepository repository;
#PersistenceContext
private EntityManager entityManager;
#Autowired
private JdbcTemplate template;
#Test
public void test() {
// use your Spring Data repository, or the EntityManager or the JdbcTemplate to run your SQL and populate your database.
}
Note: This is explained in Practical Guide to Building an API Back End with Spring Boot, chapter 7 in more detail (Disclaimer: I am the author of the book)
Spring framework provides the ability to execute SQL scripts for test suites or for a test unit. For example:
#Test
#Sql({"/test-schema.sql", "/test-user-data.sql"})
public void userTest {
// execute code that relies on the test schema and test data
}
Here's the documentation.
You can also take a look at Spring Test DBUnit which provides annotations to populate your database for a test unit. It uses XML dataset files.
#Test
#DatabaseSetup(value = "insert.xml")
#DatabaseTearDown(value = "insert.xml")
public void testInsert() throws Exception {
// Inserts "insert.xml" before test execution
// Remove "insert.xml" after test execution
}
Also, you can take a look at DbSetup, which provides a java fluent DSL to populate your database.
There is one more option, if you are defining Postgres container manually without fancy testcontainers JDBC url stuff, not related to Spring directly. Postgres image allows to link directory containing sql scripts to container volume and auto-executes them.
GenericContainer pgDb = new PostgreSQLContainer("postgres:9.4-alpine")
.withFileSystemBind("migrations/sqls", "/docker-entrypoint-initdb.d",
BindMode.READ_ONLY)
Also if you need something in runtime, you can always do
pgDb.execInContainer("psql ....").
You can use DatabaseRider, which uses DBUnit behind the scenes, for populating test database and TestContainers as the test datasource. Following is a sample test, full source code is available on github here.
#RunWith(SpringRunner.class)
#SpringBootTest
#DataJpaTest
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE) #ActiveProfiles("integration-test")
#DBRider //enables database rider in spring tests
#DBUnit(caseInsensitiveStrategy = Orthography.LOWERCASE) //https://stackoverflow.com/questions/43111996/why-postgresql-does-not-like-uppercase-table-names
public class SpringBootDBUnitIt {
private static final PostgreSQLContainer postgres = new PostgreSQLContainer(); //creates the database for all tests on this file
#PersistenceContext
private EntityManager entityManager;
#Autowired
private UserRepository userRepository;
#BeforeClass
public static void setupContainer() {
postgres.start();
}
#AfterClass
public static void shutdown() {
postgres.stop();
}
#Test
#DataSet("users.yml")
public void shouldListUsers() throws Exception {
assertThat(userRepository).isNotNull();
assertThat(userRepository.count()).isEqualTo(3);
assertThat(userRepository.findByEmail("springboot#gmail.com")).isEqualTo(new User(3));
}
#Test
#DataSet("users.yml") //users table will be cleaned before the test because default seeding strategy
#ExpectedDataSet("expected_users.yml")
public void shouldDeleteUser() throws Exception {
assertThat(userRepository).isNotNull();
assertThat(userRepository.count()).isEqualTo(3);
userRepository.delete(userRepository.findOne(2L));
entityManager.flush();//can't SpringBoot autoconfigure flushmode as commit/always
//assertThat(userRepository.count()).isEqualTo(2); //assertion is made by #ExpectedDataset
}
#Test
#DataSet(cleanBefore = true)//as we didn't declared a dataset DBUnit wont clear the table
#ExpectedDataSet("user.yml")
public void shouldInsertUser() throws Exception {
assertThat(userRepository).isNotNull();
assertThat(userRepository.count()).isEqualTo(0);
userRepository.save(new User("newUser#gmail.com", "new user"));
entityManager.flush();//can't SpringBoot autoconfigure flushmode as commit/always
//assertThat(userRepository.count()).isEqualTo(1); //assertion is made by #ExpectedDataset
}
}
src/test/resources/application-integration-test.properties
spring.datasource.url=jdbc:tc:postgresql://localhost/test
spring.datasource.driverClassName=org.testcontainers.jdbc.ContainerDatabaseDriver
spring.datasource.username=test
spring.datasource.password=test
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQL9Dialect
spring.jpa.hibernate.ddl-auto=create
spring.jpa.show-sql=true
#spring.jpa.properties.org.hibernate.flushMode=ALWAYS #doesn't take effect
spring.jpa.hibernate.naming-strategy=org.hibernate.cfg.ImprovedNamingStrategy
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
And finally the datasets:
src/test/resources/datasets/users.yml
users:
- ID: 1
EMAIL: "dbunit#gmail.com"
NAME: "dbunit"
- ID: 2
EMAIL: "rmpestano#gmail.com"
NAME: "rmpestano"
- ID: 3
EMAIL: "springboot#gmail.com"
NAME: "springboot"
src/test/resources/datasets/expected_users.yml
users:
- ID: 1
EMAIL: "dbunit#gmail.com"
NAME: "dbunit"
- ID: 3
EMAIL: "springboot#gmail.com"
NAME: "springboot"
src/test/resources/datasets/user.yml
users:
- ID: "regex:\\d+"
EMAIL: "newUser#gmail.com"
NAME: "new user"
After some reviews, I think that it is interesting to review the examples from Spring Data JDBC which use Test Containers:
Note: Use Java 8
git clone https://github.com/spring-projects/spring-data-jdbc.git
mvn clean install -Pall-dbs
I will create a simple project adding some ideas about previous project referenced.
Juan Antonio

Spring H2 Test DB does not reset before each test

EDIT: As C. Weber suggested in the comments, the solution is to add #Transactional to the test class.
I have some tests that use an H2 in-memory DB. I need to reset the DB before each test. Although my SQL scripts are run each a test is executed, the DB is not properly reset, resulting in a missing needed entry after a delete test.
Test class:
#RunWith(SpringRunner.class)
#SpringBootTest
#AutoConfigureTestDatabase(replace=Replace.ANY, connection=EmbeddedDatabaseConnection.H2)
public class RepositoryTests {
#Autowired
private Repository repository;
#Autowired
private DataSource dataSource;
#Before
public void populateDb() {
Resource initSchema = new ClassPathResource("database/schema.sql");
Resource initData = new ClassPathResource("database/data.sql");
DatabasePopulator dbPopulator = new ResourceDatabasePopulator(initSchema, initData);
DatabasePopulatorUtils.execute(dbPopulator, dataSource);
}
#Test
public void testMethod1() {
// ...
repository.delete("testdata");
}
#Test
public void testMethod2() {
// ...
Object test = repository.get("testdata");
// is null but should be an instance
}
}
schema.sql drops all tables before recreating them. data.sql inserts all needed test data into the DB.
Running the testMethod2 alone succeeds. However, running all tests makes the test fail with a NullPointerException.
I have successfully tried to use #DirtiesContext, however this is not an option because I can't afford to have a 20 second startup for each 0.1 second test.
Is there another solution?
The Spring Test Framework provides a mechanism for the behaviour you want for your tests. Simply annotate your Test class with #Transactional to get the default rollback behaviour for each test method.
There are ways to configure the transactional behaviour of tests and also some pitfalls (like using RestTemplate inside test method), which you can read more about in the corresponding chapter of the Spring manual.
Spring Test Framework

Spring boot #Transactional doesn't rollback

I am using Spring boot application, on that i am trying to achieve Transactional management. But Spring doesn't rollback the data which saved in same method.
Code base: https://github.com/vinothr/spring-boot-transactional-example
Can any one help me?
This is my repository class for 'Test' entity.
#Repository
public interface TestRepository extends CrudRepository<com.example.demo.Test, Long> {
}
I have created one end-point which used to save the data to 'Test' entity. After save happen, I thrown RunTimeException, but it is not rollbacking the saved value
#GetMapping("/test")
#Transactional
public void create() {
System.out.println("test");
final Test p = createTest();
testRepository.save(p);
final Test p1 = createTest();
testRepository.save(p1);
throw new RuntimeException();
}
It works fine after I changed into 'InnoDB' engine because I was using 'MyISAM' engine which doesn't support transaction.
ALTER TABLE my_table ENGINE = InnoDB;
Try indicate #Transactional(rollbackFor = RuntimeException.class)

Spring Jdbc embedded database lifecycyle

I am using spring embedded jdbc database configuration in spring context file for Junit testing. I am using in memory database for testing. I am trying to unit test only DAO layer and for unit testing I am using spring container's in memory database.
When I am running Junit test case I am not seeing first test case values in second test case (testAddressCreate in testAddressUpdate test case). I am not using #Before or #After in my Junit for now. I am not sure how spring is creating in memory database and starting it. According to behavior it seems that it is creating or resetting before each test case. Does anyone know about it? Also how can we connect to spring created in memory database. Please suggest.
Spring Configuration is :
<jdbc:embedded-database id="dataSource" type="HSQL">
<jdbc:script location="classpath:createdb.sql" />
Junit test case is :
#ContextConfiguration(locations="classpath:test-context.xml")
#Transactional
public class GenericDaoImplTest extends AbstractTransactionalJUnit4SpringContextTests {
#Autowired
private GenericDao<Address, Long> addressDao;
#Test
public void testAddressCreate() {
Address address = new Address();
address.setAddress1("first address");
address.setCity("First City");
address.setCountry("First one");
address.setPostalCode("22222");
boolean result = addressDao.create(address);
Assert.assertEquals(true, result);
List<Address> listOfAddress = addressDao.findAll();
Assert.assertNotNull(listOfAddress);
for(Address addressTemp : listOfAddress){
System.out.println(addressTemp.getAddress1());
System.out.println(addressTemp.getAddressId());
}
}
#Test
public void testAddressUpdate(){
Address address = new Address();
address.setAddress1("second address");
address.setCity("Second City");
address.setCountry("Second one");
address.setPostalCode("11111");
boolean result = addressDao.create(address);
Assert.assertEquals(true, result);
address.setAddress1("Updated Second Address");
Assert.assertNotNull(addressDao.update(address));
List<Address> listOfAddress = addressDao.findAll();
Assert.assertNotNull(listOfAddress);
for(Address addressTemp : listOfAddress){
System.out.println(addressTemp.getAddress1());
System.out.println(addressTemp.getAddressId());
}
}
}
By default, a #Transactional test method will rollback any changes made to the database during the test method. If you want to change this behavior you need to annotate your method with #Rollback(false). I don't think the documentation is specific about the default behavior, but the Javadocs mentions this here:
Retrieves the TransactionConfigurationAttributes for the specified class which may optionally declare or inherit #TransactionConfiguration. If TransactionConfiguration is not present for the supplied class, the default values for attributes defined in TransactionConfiguration will be used instead.
And the default value defined in TransactionConfiguration for rollback is "true".
These being said, you need something like this to be able to keep the values from the first #Test method in the second one:
#Test
#Rollback(false)
public void testAddressCreate() {
Address address = new Address();
...
}
For connecting at the in-memory HSQLDB, take a look at this post on Stackoverflow.

junit 4 TransactionalTestExecutionListener insert test data only once for all tests in class?

I have a junit 4 test class testing a DAO.
unit test:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(locations = {
"classpath:/WEB-INF/applicationContext-db.xml",
"classpath:/WEB-INF/applicationContext-hibernate.xml",
"classpath:/WEB-INF/applicationContext.xml" })
#TestExecutionListeners({DependencyInjectionTestExecutionListener.class, TransactionalTestExecutionListener.class})
#DataSetLocation("test/java/com/yada/yada/dao/dbunit-general.xml")
#TransactionConfiguration(transactionManager="transactionManager", defaultRollback = true)
#Transactional
public class RealmDAOJU4Test {
#Autowired
private DbUnitInitializer dbUnitInitializer;
#Autowired
private RealmDAO realmDAO;
#BeforeTransaction
public void setupDatabase() {
// use dbUnitInitializer to insert test data
}
#Test
public void testGetById() {
Integer id = 2204;
Realm realm = realmDAO.get(id);
assertEquals(realm.getName().compareToIgnoreCase(
"South Technical Realm"), 0);
assertEquals(8, realm.getRealmRelationships().size());
}
// more test methods annotated here
}
The #BeforeTransacation method runs before EVERY test method. What I would like to do is: use my DbUnitInitializer to load data into my database - ONCE when the class is created. Then have each test in the class do what it needs to do with the database, then roll back (not commit) it's changes. It seems over kill to re-insert all the same data from my test files before EVERY test. Is there a way to accomplish this?
or
Is the correct way to write these tests to completely load the database before EVERY test? If so, what function does the defaultRollback=true have in this situation?
thanks for helping me along in my thinking...
You need to use a TestExecutionListener and set up your database in the beforeTestClass method. See the Annotations section of the Testing chapter in the Spring user guide.

Resources