I am setting up integration tests for my spring-boot application using cassandra-unit-spring maven dependency. I am able to run my tests which invoke the spring-boot application which in turn accesses an in-memory embedded Cassandra database.
Below is the code for my test class
#RunWith(SpringJUnit4ClassRunner.class)
#TestExecutionListeners({CassandraUnitDependencyInjectionTestExecutionListener.class, DependencyInjectionTestExecutionListener.class})
#CassandraDataSet(value = "cassandra/dbcreate.cql", keyspace = "test")
#EmbeddedCassandra
#SpringBootTest({"spring.data.cassandra.port=9142"})
public class IntegrationTest {
#Autowired
private TestRepository testRepository;
#Test
public void testFindById() {
Token token = generateRandomToken();
testRepository.insert(token);
Optional<Token> tokenStored = testRepository.findById(token.getKey());
compareReplayToken(token, tokenStored.get()); //This method does the assertions
}
}
This single test invokes the embedded Cassandra and creates the keyspace and tables from the commands in the cassandra/dbcreate.cql file. After the test runs, the keyspace and tables are dropped.
Till now, it is fine. But, if I try to add multiple tests in this class, this approach creates the keyspace and tables at the beginning of each test and then drops them once the test runs.
And the dbcreate.cql file has a lot of commands to create multiple tables and when these commands run for each test, this makes my tests really slow.
Also, this problem multiplies when I try to have multiple such test classes.
Possible solution that I could think of is:
Have a separate cql file for each test class that has limited cql commands concerned with that class only - Again, this doesn't solve the problem of the database reset for each test in a single class
I want to run all my integration tests for a single launch of this embedded Cassandra and the tables and keyspace should be created and dropped only once for a fast execution
What should be the ideal solution for such a problem?
Any help is much appreciated.
Thanks!
Related
In my SpringBoot application, I have one test classes inside /src/test/java.
For Testing (Unit Tests). I want to use the In memory H2 database. I have the following Database Url
jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;INIT=runscript from 'classpath:/schema.sql'\\;runscript from 'classpath:/data.sql'
So when I run the test. the database is created and the scripts (schema.sql and data.sql) runs correctly as expected. it creates some tables and puts some test data over there.
Now the problem is I added another Test class and written some tests there. so whats happening now is, first test class runs succesfully, but when the second class loads, it tries to run the scripts (schema.sql and data.sql) again on the in memory H2 database. and that obviously fails. because those tables are already there in the DB.
Can anyone please suggest how Can i achieve the behaviour I want. such that my scripts should run only once and then all the test classes should use that same database.
My Test class example is below
#RunWith(SpringRunner.class)
#SpringBootTest()
public class CreateServiceTest {
#Autowired
private CreateRepo repo;
#Test
public void testCreation(){
// test code here
}
With spring boot the h2 database can be defined uniquely for each
test. Just override the data source URL for each test
#SpringBootTest(properties =
{"spring.config.name=myapp-test-h2","myapp.trx.datasource.url=jdbc:h2:mem:trxServiceStatus"})
The tests can run in parallel.
refer: https://stackoverflow.com/a/49644877
I have 6 JUnit classes into different test packages. I want to use application.properties file which is used by Spring framework to setup database connection.
The problem is how to create database connection when first JUnit test is run and reuse it for all Classes. At the end to close the connection properly.
You can use Testcontainers for this. If you want to reuse an existing database container (e.g. PostgreSQL or MySQL), make sure to use a manual container lifecycle approach and start the container once (e.g. inside an abstract test class). Testcontainers calls this Singleton containers:
abstract class AbstractContainerBaseTest {
static final MySQLContainer MY_SQL_CONTAINER;
static {
MY_SQL_CONTAINER = new MySQLContainer();
MY_SQL_CONTAINER.start();
}
}
class FirstTest extends AbstractContainerBaseTest {
#Test
void someTestMethod() {
String url = MY_SQL_CONTAINER.getJdbcUrl();
// create a connection and run test as normal
}
}
Once your JVM terminates, your container will be deleted.
Another approach would be the reuse feature of Testcontainers:
static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer()
.withDatabaseName("test")
.withUsername("duke")
.withPassword("s3cret")
.withReuse(true);
If your database setup is similar for all your tests and you opt-in for this feature, Testcontainers will reuse an already started container. Keep in mind that with this approach you have to clean up the container for yourself as they stay alive after all tests finished.
My Spring app has its own DB for persistence.
The same app needs to send ad-hoc queries to external databases.
Queries are provided by users.
App takes SQL query provided by user
App takes external database type (postgres / oracle / whatever jdbc)
App submits adhoc query in runtime to external DB
App returns result as json to user
Is there any way to utilize spring test containers in order to test this functionaly?
My goal is:
Write tests for every supported DB
each test starts test container with supported DB (some subset of these I suppose: https://www.testcontainers.org/modules/databases/)
each test uploads sample data to container DB
each test runs set of "must work" queries against it.
I see many examples where App itself is tested against test containers, but can I just start container w/o plugging it as App persistence DB?
can I just start container w/o plugging it as App persistence DB?
Yes, this is perfectly possible.
Testcontainers in itself has nothing to do with Spring or Spring Boot.
What you would do is:
pick the container you want to use (different container for different databases
instantiate the container
start it up
construct a DataSource from it
Use that DataSource for your tests.
Spring Data JDBC does exactly that to run tests against various databases.
I add the class doing that for MySQL in the end.
It is a Spring application context configuration, but you could put that in a JUnit before method, a JUnit 4 rule or a JUnit 5 extension or just a normal method that you call at the start of your test.
#Configuration
#Profile("mysql")
class MySqlDataSourceConfiguration extends DataSourceConfiguration {
private static final MySQLContainer MYSQL_CONTAINER = new MySQLContainer().withConfigurationOverride("");
static {
MYSQL_CONTAINER.start();
}
/*
* (non-Javadoc)
* #see org.springframework.data.jdbc.testing.DataSourceConfiguration#createDataSource()
*/
#Override
protected DataSource createDataSource() {
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setUrl(MYSQL_CONTAINER.getJdbcUrl());
dataSource.setUser(MYSQL_CONTAINER.getUsername());
dataSource.setPassword(MYSQL_CONTAINER.getPassword());
dataSource.setDatabaseName(MYSQL_CONTAINER.getDatabaseName());
return dataSource;
}
#PostConstruct
public void initDatabase() throws SQLException, ScriptException {
ScriptUtils.executeSqlScript(createDataSource().getConnection(), null, "DROP DATABASE test;CREATE DATABASE test;");
}
}
My Spring Boot application also starts a gRPC service along with its REST (HTTP) service. I've written specific tests for gRPC and REST. When I run a gradle test these tests are run sequentially, however; there is no reason they can't be run in parallel.
What I'm shooting for here is a single instance of my Spring Boot application running while the tests are executed in parallel.
I've tried setting the test section in my gradle file so it has 'forkCount', I also tried setting options such that parallel="classes", but this produces an error about the 'parallel' being an unknown property (maybe a junit 5 thing?)
test {
options {
parallel = "classes"
// forkCount = 2
}
}
The forkCount option is not what I'm looking for since it will start multiple instances of the spring application.
I've also tried removing the #RunWith from the test classes and making a separate test class (which has the #RuWith annotation) that has the following method in it
#Test
void testRunner() {
JUnitCore.runClasses(ParallelComputer.classes(), {GrpcTests.class, RestTests.class});
}
But the tests still appear to run sequentially.
I've tried several other things as well, sorry I don't have all of them handy.
Goal
Ideally what I'm hoping for is a single instance of my Spring Boot app running while the test classes run in parallel (bonus kudos if I can get the methods to run in parallel too)
Java Version: "1.8.0_171"
Spring Boot Version: 2.0.4.RELEASE
Per the recommendation I tried adding the
#Test
public void contextLoads() throws Exception {
}
And adding the 'maxParallelForks' entry in the gradle file, I had already been using the #SpringBootTest annotation but this behaved the same as when I used 'forkCount` in that at least 2 instances where started as can be seen by the test shutdown log
2019-04-25 10:24:17.245 LogLevel=INFO 53838 --- shutting down gRPC server since JVM is shutting down
...
2019-04-25 10:24:30.125 LogLevel=INFO 53839 --- shutting down gRPC server since JVM is shutting down
You can see I get two shutdown messages and the PIDs are shown (53838 & 53839).
You need to combine #SpringBootTest with maxParallelForks.
Annotate your unit tests with #SpringBootTest. #SpringBootTest will boot up a Spring Boot context that will be cached across all your tests.
"A nice feature of the Spring Test support is that the application context is cached in between tests, so if you have multiple methods in a test case, or multiple test cases with the same configuration, they only incur the cost of starting the application once"
See:
https://spring.io/guides/gs/testing-web/
Add the following to your build.gradle. To run multiple test at the same time.
tasks.withType(Test) {
maxParallelForks = 4 //your choice here
}
See https://guides.gradle.org/performance/#parallel_test_execution
In my application I have a set of of DAOs which I inject into my application layer. For an acceptance test I'm writing, I want to preload the dev_server datastore with data, so I use the same Spring config in my JUnit test (using the #ContextConfiguration annotation) to inject an instance of the relevant DAO into my test. When I actually go to store some data eg:
dao.add(entity)
I get the dreaded "No API environment is registered for this thread."
Caused by: java.lang.NullPointerException: No API environment is registered for this thread.
at com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppId(DatastoreApiHelper.java:108)
at com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppIdNamespace(DatastoreApiHelper.java:118)
....
This is probably because my test case hasn't read in the GAE application-web.xml with the app details (although I'm guessing here I could really be wrong); so it doesn't know to write to the same datastore that the app running on the dev_server is reading/writing to.
How can I get my test to "point" to the same datastore as the app? Is there some "datasource" mechanism that I can inject both into the app and the test? Is there a way to get my test to force the datastore api to read the needed config?
Here is a page that talks about how to do unit tests that connect to a dev datastore. Is this the kind of thing you're looking for? Basically it talks about two classes, LocalServiceTestHelper and LocalDatastoreServiceTestConfig that you can use to set up an environment for testing. While the example given is for unit tests, I believe it will also work for your situation.
You can then configure things like whether the dev datastore is written to disk or just kept in memory (for faster tests). If you want this data to go to the same place as your dev server, you will probably want to adjust this, as I think the default is the "in memory" option. If you look at the javadoc there is a "setBackingStoreLocation" method where you can point to whatever file you want.
I've found the solution!!!!
For some reason the Namespace, AppID and the AuthDomain fields of the test datastore have to match that of the dev_server, then the dev_server can see the entities inserted by the test.
You can see the values for the environment (dev_server or test code) with the following statements
System.out.println(NamespaceManager.get());
System.out.println(ApiProxy.getCurrentEnvironment().getAppId());
System.out.println(ApiProxy.getCurrentEnvironment().getAuthDomain());
In your instance of LocalServiceTestHelper (eg: gaeHelper), you can set the values for the test environment
// the NamespaceManager is thread local.
NamespaceManager.set(NamespaceManager.getGoogleAppsNamespace());
gaeHelper.setEnvAppId(<the name of your app in appengine-web.xml>);
gaeHelper.setEnvAuthDomain("gmail.com");
Then the dev_server will see your entities. However because of synchronisation issues, if the test writes to the datastore after the dev_server has been started the dev_server wont see it unless it can be forced to reread the file (which I haven't figured out yet). Else the server has to be restarted.
I've found a workaround, although it's not very nice because each test method doesn't clean up the Datastore, as explained in the article Local Unit Testing for Java, however, the Datastore starts clean each time the Test class is run, so it's not so bad, provided that you're careful about that.
The problem is, that when using SpringJUnit4ClassRunner, the spring environment is created before the #Before annotation can be run, the solution is use #BeforeClass and use a static variable for LocalServiceTestHelper, to have them created before the Spring Environment is set up.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration("classpath:META-INF/spring/context-test.xml")
#Transactional
public class MyTest {
#Inject
private MyService myService;
private static final LocalServiceTestHelper helper =
new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig());
#BeforeClass
public static void beforeClass() {
helper.setUp();
}
#AfterClass
public static void afterClass() {
helper.tearDown();
}
If anyone has a better solution, I'll be glad to hear!