SpringBoot, test containers to test queries sent to external DB - spring-boot

My Spring app has its own DB for persistence.
The same app needs to send ad-hoc queries to external databases.
Queries are provided by users.
App takes SQL query provided by user
App takes external database type (postgres / oracle / whatever jdbc)
App submits adhoc query in runtime to external DB
App returns result as json to user
Is there any way to utilize spring test containers in order to test this functionaly?
My goal is:
Write tests for every supported DB
each test starts test container with supported DB (some subset of these I suppose: https://www.testcontainers.org/modules/databases/)
each test uploads sample data to container DB
each test runs set of "must work" queries against it.
I see many examples where App itself is tested against test containers, but can I just start container w/o plugging it as App persistence DB?

can I just start container w/o plugging it as App persistence DB?
Yes, this is perfectly possible.
Testcontainers in itself has nothing to do with Spring or Spring Boot.
What you would do is:
pick the container you want to use (different container for different databases
instantiate the container
start it up
construct a DataSource from it
Use that DataSource for your tests.
Spring Data JDBC does exactly that to run tests against various databases.
I add the class doing that for MySQL in the end.
It is a Spring application context configuration, but you could put that in a JUnit before method, a JUnit 4 rule or a JUnit 5 extension or just a normal method that you call at the start of your test.
#Configuration
#Profile("mysql")
class MySqlDataSourceConfiguration extends DataSourceConfiguration {
private static final MySQLContainer MYSQL_CONTAINER = new MySQLContainer().withConfigurationOverride("");
static {
MYSQL_CONTAINER.start();
}
/*
* (non-Javadoc)
* #see org.springframework.data.jdbc.testing.DataSourceConfiguration#createDataSource()
*/
#Override
protected DataSource createDataSource() {
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setUrl(MYSQL_CONTAINER.getJdbcUrl());
dataSource.setUser(MYSQL_CONTAINER.getUsername());
dataSource.setPassword(MYSQL_CONTAINER.getPassword());
dataSource.setDatabaseName(MYSQL_CONTAINER.getDatabaseName());
return dataSource;
}
#PostConstruct
public void initDatabase() throws SQLException, ScriptException {
ScriptUtils.executeSqlScript(createDataSource().getConnection(), null, "DROP DATABASE test;CREATE DATABASE test;");
}
}

Related

Cassandra unit spring - Launch an embedded Cassandra only once per test class

I am setting up integration tests for my spring-boot application using cassandra-unit-spring maven dependency. I am able to run my tests which invoke the spring-boot application which in turn accesses an in-memory embedded Cassandra database.
Below is the code for my test class
#RunWith(SpringJUnit4ClassRunner.class)
#TestExecutionListeners({CassandraUnitDependencyInjectionTestExecutionListener.class, DependencyInjectionTestExecutionListener.class})
#CassandraDataSet(value = "cassandra/dbcreate.cql", keyspace = "test")
#EmbeddedCassandra
#SpringBootTest({"spring.data.cassandra.port=9142"})
public class IntegrationTest {
#Autowired
private TestRepository testRepository;
#Test
public void testFindById() {
Token token = generateRandomToken();
testRepository.insert(token);
Optional<Token> tokenStored = testRepository.findById(token.getKey());
compareReplayToken(token, tokenStored.get()); //This method does the assertions
}
}
This single test invokes the embedded Cassandra and creates the keyspace and tables from the commands in the cassandra/dbcreate.cql file. After the test runs, the keyspace and tables are dropped.
Till now, it is fine. But, if I try to add multiple tests in this class, this approach creates the keyspace and tables at the beginning of each test and then drops them once the test runs.
And the dbcreate.cql file has a lot of commands to create multiple tables and when these commands run for each test, this makes my tests really slow.
Also, this problem multiplies when I try to have multiple such test classes.
Possible solution that I could think of is:
Have a separate cql file for each test class that has limited cql commands concerned with that class only - Again, this doesn't solve the problem of the database reset for each test in a single class
I want to run all my integration tests for a single launch of this embedded Cassandra and the tables and keyspace should be created and dropped only once for a fast execution
What should be the ideal solution for such a problem?
Any help is much appreciated.
Thanks!

H2 database with Multiple Test Classes in SpringBoot

In my SpringBoot application, I have one test classes inside /src/test/java.
For Testing (Unit Tests). I want to use the In memory H2 database. I have the following Database Url
jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;INIT=runscript from 'classpath:/schema.sql'\\;runscript from 'classpath:/data.sql'
So when I run the test. the database is created and the scripts (schema.sql and data.sql) runs correctly as expected. it creates some tables and puts some test data over there.
Now the problem is I added another Test class and written some tests there. so whats happening now is, first test class runs succesfully, but when the second class loads, it tries to run the scripts (schema.sql and data.sql) again on the in memory H2 database. and that obviously fails. because those tables are already there in the DB.
Can anyone please suggest how Can i achieve the behaviour I want. such that my scripts should run only once and then all the test classes should use that same database.
My Test class example is below
#RunWith(SpringRunner.class)
#SpringBootTest()
public class CreateServiceTest {
#Autowired
private CreateRepo repo;
#Test
public void testCreation(){
// test code here
}
With spring boot the h2 database can be defined uniquely for each
test. Just override the data source URL for each test
#SpringBootTest(properties =
{"spring.config.name=myapp-test-h2","myapp.trx.datasource.url=jdbc:h2:mem:trxServiceStatus"})
The tests can run in parallel.
refer: https://stackoverflow.com/a/49644877

Share database connection between JUnit 5 test classes

I have 6 JUnit classes into different test packages. I want to use application.properties file which is used by Spring framework to setup database connection.
The problem is how to create database connection when first JUnit test is run and reuse it for all Classes. At the end to close the connection properly.
You can use Testcontainers for this. If you want to reuse an existing database container (e.g. PostgreSQL or MySQL), make sure to use a manual container lifecycle approach and start the container once (e.g. inside an abstract test class). Testcontainers calls this Singleton containers:
abstract class AbstractContainerBaseTest {
static final MySQLContainer MY_SQL_CONTAINER;
static {
MY_SQL_CONTAINER = new MySQLContainer();
MY_SQL_CONTAINER.start();
}
}
class FirstTest extends AbstractContainerBaseTest {
#Test
void someTestMethod() {
String url = MY_SQL_CONTAINER.getJdbcUrl();
// create a connection and run test as normal
}
}
Once your JVM terminates, your container will be deleted.
Another approach would be the reuse feature of Testcontainers:
static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer()
.withDatabaseName("test")
.withUsername("duke")
.withPassword("s3cret")
.withReuse(true);
If your database setup is similar for all your tests and you opt-in for this feature, Testcontainers will reuse an already started container. Keep in mind that with this approach you have to clean up the container for yourself as they stay alive after all tests finished.

Access to h2 web console while running junit test in a Spring application

I'm building a Spring application and I need to inspect my H2 in-memory database while I'm running my JUnit tests from a web browser.
In my Spring configuration I have a bean which is responsible of creating my database schema and populating it with some data which will be used within my JUnit tests. I've also added a bean in my test context which creates a web server where I eventually will look for my data.
<bean id="org.h2.tools.Server-WebServer" class="org.h2.tools.Server"
factory-method="createWebServer" init-method="start" lazy-init="false">
<constructor-arg value="-web,-webAllowOthers,-webPort,11111" />
</bean>
Everything seems ok because the database is populated properly since I can access to its data from my JUnit tests and H2 Server only runs while I'm in my test-phase (I can know that, because if I try to access to my_ip:111111 before debugging my tests I cannot connnect but I can connect afterwards once I've started my tests).
Anyway If I open my H2 console from a web browser no schema is shown in it. Any ideas??
Many thanks!!
As this is probably going to be a test-debugging feature, you can add it at runtime with your #Before:
import org.h2.tools.Server;
/* Initialization logic here */
#BeforeAll
public void initTest() throws SQLException {
Server.createWebServer("-web", "-webAllowOthers", "-webPort", "8082")
.start();
}
And then connect to http://localhost:8082/
Note: unless you need this to run as part of your CI build, you'll need to remove this code when you're finished debugging
For future reference here's another way to do it:
Start database and web servers (version can differ):
$ cd .../maven_repository/com/h2database/h2/1.4.194
$ java -cp h2-1.4.194.jar org.h2.tools.Server -tcp -web -browser
TCP server running at tcp://169.254.104.55:9092 (only local connections)
Web Console server running at http://169.254.104.55:8082 (only local connections)
Set database url for tests in code to jdbc:h2:tcp://localhost:9092/mem:mytest.
Run or debug tests.
Click Connect in browser window which opened in step 1.
Jar file for H2 can be downloaded at https://mvnrepository.com/artifact/com.h2database/h2.
Server can be started via #Before in test file like in snovelli's answer, but only in case connection to database in established afterwards, which might be a problem.
I guess the problem is that you are connecting to h2db directly from your application. Not through the server you are launching with bean. Because of this your app and h2db-web-interface can't share one in-memory database.
You should change jdbcUrl in tests to something like jdbc:h2:tcp://localhost/mem:my_DB;DB_CLOSE_DELAY=-1;MODE=Oracle and in browser you should connect to the same url.
With jdbc urls like jdbc:h2:tcp://localhost/... all connections will go through the h2db-server and you can view database state in browser.
If you have defined the jdbc url to something like jdbc:h2:mem:db in your properties, when the database is created it actually gets a bit longer name.
Add a #Autowired DataSource dataSource to your test class, set a debug point somewhere, and inspect that datasource with dataSource.getConnection() and look at the url property. In the case I'm running right this moment, it is
jdbc:h2:mem:43ed83d6-97a1-4515-a925-a8ba53cd322c
Plugging that into the web cosole shows everything I'm expecting.
It isn't the most straightforward way, but it does work.
#snovelli answer above is good.
To debug a particular test case in your IDE, add a infinite loop at the end of the test case and go to browser and launch the console and you can query the data.
Something like below
import org.h2.tools.Server;
/* Initialization logic here */
#BeforeAll
public void initTest() throws SQLException {
Server.createWebServer("-web", "-webAllowOthers", "-webPort", "8082")
.start();
}
#Test
void testMyDBOperation() {
//some db operations like save and get
while(true) {
}
}
now you can go to browser and launch the console at http://localhost:8082/
Of course delete above two changes after debugging
It is not the answer, but a debugging tip.
When you finally access h2-conole http://127.0.0.1:8082/ you may notice that database changes are not shown.
This is because the test cases are not transactional and the data is not committed. Although this behavior is good, as each test case, must run in predefined environment. It is not good if you want to debug and see database changes.
To achieve this, add #Commit annotation above test case and put a dummy line in a #AfterAll annotated method, to stop test and let you see the h2 console ( The h2 server will stop as the test finish).
#AfterAll
public static void finalizeTest() throws Exception {
System.out.print("Just put a break point here");
}
#Test
#Commit
void should_store_an_article() {
// Your test here
}

Auto insert default record when deploying Spring MVC App with Spring Security

I am looking for a way to auto insert a default admin account, using JPA, when my spring mvc application is deployed.
My database is generated based on the Entities.
I want to kick off something that will insert a default admin user, assign roles, every time the application is deployed.
It depends on which implementation of JPA you use.
If you use Hibernate you can add import.sql file (that contains records to load) to the class path. More info here.
As a workaround you can also use dbunit tool.
I would recommend having a migration utility that will keep your database in synch with your codebase - these are typically DDL's, but again the queries to insert default admin user, assign roles etc can also be part of this migration utility. There are very good one's available - Flyway is one that I have used, Liquibase is another one.
There is a very good comparison of the different migration utilities on the Flyway homepage also that you can look at.
i use CommandLineRunner interface.
#Component
public class CommandLineAppStartupRunner implements CommandLineRunner {
#Autowired
UserRepository userRepository;
#Override
public void run(String...args) throws Exception {
User admin = new user(firstName);
userRepository.save(admin);
}
}
before the app starts this class will be executed.
you can find other ways here : Guide To Running Logic on Startup in Spring

Resources