How do I use Spring embedded database initialization scripts during the integration-test phase? - spring

I can use Spring to create and initialize embedded databases either programmatically:
#Before
public void setUp() {
database = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.addScript("schema.sql")
.addScript("init.sql")
.build();
....
}
or via Spring configuration:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:schema.sql"/>
<jdbc:script location="classpath:init.sql"/>
</jdbc:initialize-database>
Here my scripts schema.sql and init.sql are stored in the directory src/test/resources. So if tests using the embedded databse are run with:
mvn test
then the files, being in src/test/resources are available, and everything is fine.
But now suppose I want to instead use the embedded database together with an embedded web server (Jetty or embedded Tomcat) in an integration test, run via
mvn integration-test
Now in this late phase, I want to do an almost end-to-end test that hitting certain urls for the web service return the expected HTTP responses, assuming certain data in the embedded database. But at this point, the war deployed in the embedded webserver does not have files from src/test/resources so my initialization scripts are not present, and I get the error
Caused by: java.io.FileNotFoundException: class path resource [schema.sql] cannot be opened because it does not exist
Now everything will work if I put the scripts in src/main/resources but these scripts are only for filling up an embedded database for testing and they do not belong in the war file at all. Does anyone know how to set up an integration test so that an emedded database can be used, without polluting the actual war file that will be deployed?
I was hoping that Spring's embedded database initialization could use something other than a file. I thought about JNDI but that would seem to require polluting the web.xml file with a resource definition for test data which would (again) appear in the deployable war. Or perhaps there are options using cargo? Some programmatic tricks with Spring that I don't know?

You should be able to do this:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="file:///${project.basedir}/src/test/resources/schema.sql"/>
<jdbc:script location="file:///${project.basedir}/src/test/resources/init.sql"/>
</jdbc:initialize-database>

Related

Create Externalized Configuration in spring-boot along with profiles

I have a spring-boot application with annotations instead of context.xml.
In my src/main/resources folder I have: application-dev.properties and application-test.properties.
which work perfectly for different profiles (while running with VM option like -Dspring.profiles.active=dev)
Now I need to externalize this properties with file in /opt/software/Tomcat8/conf/app.properties
Some props override each other, some don't.
in Tomcat config context.xml I say:
<Environment name="app.properties"
value="file:///opt/software/Tomcat8/conf/app.properties"
type="java.lang.String" override="false"/>
How to use it via JNDI in my application configuring app with no XML but annotations in Spring-bot application class?
I need it to have priority to inner jar properties according to
Link to Spring-boot.doc
One solution I found was to have the vm argument -Dloader.path with the external path when executing the application. Please keep in mind if you're using a fat jar you may need to create the package in Zip model, otherwise it will not work.

Multiple Embedded HSQLDB databases in jUnit errors during build

I'm working on a new Spring Batch (3.0.3.RELEASE) application where there will be multiple databases accessed during the jobs. For testing we are using HSQLDB (2.3.2) as the embedded database.
In my Application context I have the following.
<jdbc:embedded-database id="dataSource">
</jdbc:embedded-database>
<jdbc:embedded-database id="proDataSource">
<jdbc:script location="classpath:script-tables.sql" />
<jdbc:script location="classpath:script-constraints.sql" />
</jdbc:embedded-database>
<jdbc:embedded-database id="altDataSource">
<jdbc:script location="classpath:script-alt-tables.sql" />
</jdbc:embedded-database>
When I run a single test in Eclipse, things are fine. When I build from the command line, after the first test, I get errors
Failed to execute SQL script statement at line 3 of resource class path resource [script-promrkt-promo.sql]
object name already exists: PROMRKT
It appears to me that the population process in EmbeddedDatabaseFactory is receiving an already populated database. From what I can tell is that after each test there is not a SHUTDOWN being executed and HSQLDB is leaving the already populated database in memory.
I have re-reviewed the documentation and in a Spring Doc this does show a explicit shutdown command. But if spring starts up the embedded database when my test starts why doesn't it shut it down when the test completes ?
Is it expected the embedded databases will remain after each unit test for the same application context?
What is the order that spring starts up an embedded database and when is the transactional context initialized?
Do I need to use a database cleaner ?
Can the populate be updated to only populate when the database is first started, and rollback to the original script configuration when my test is complete ( kinda like how the AbstractTransactionalSpringContextTests worked )
Do I need some transactional markers? Spring Batch's JobRepo is properly being populated and destroyed between each test. Why are my custom dataSources not ?
The script the log message is complaining about isn't in your configuration. I presume it's being executed somewhere else? If that's the case, you'll probably need to add #DirtiesContext to your tests so that Spring doesn't cache the context (I'm assuming you're using the SpringJunit4Runner with #ContextConfiguration but can't be sure since your actual test isn't in the question).
If my assumption is correct, Spring caches the context in an effort to improve performance over the running of a unit test suite. If your test modifies the context in a way that can impact other tests (like running scripts in one test that need to be run again in others), you mark the tests with #DirtiesContext and Spring won't cache the context. You can use the annotation at either the method or class level. You can read more about the annotation here: http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/test/annotation/DirtiesContext.html
I spent a lot of time looking at this and reading the Spring Framework documentation (gasp!) and tracing through code. There are some interesting changes in 4.1 spring core, especially the testing.
I found out that ApplicationContext(s) are cached at the JVM level now. If a second test asks for a context the TestContext looks first in it's cache to see if some other test has already asked for the identical configuration.
I have some profiles for some of my tests. A test with a different profile but the same #ContextConfiguration causes the that context to be re-loaded with the profile applied. When the "Bean Loader" arrives at creating the embedded databases, the EmbeddedDatabaseFactory does not take into consideration that the embedded database (in memory HSQLDB) may have already been created or cached from previous tests and does not need to be re-initialized.
Therefore I added some logic to the EmbeddedDatabaseFactory.initDatabase() checking if the database already exists before re-initializing & running the DatabasePopulator.
List existingDataBases = org.hsqldb.DatabaseManager.getDatabaseURIs();
boolean isExisting = false;
String localDBName = StringUtils.lowerCase(this.databaseName);
for (Object object : existingDataBases) {
if (object.toString().contains(localDBName)) {
isExisting = true;
break;
}
}
// Now populate the database
if (!isExisting && this.databasePopulator != null) {
( of course this isn't quite kosher for what spring would need but it gets the point across )
In my opinion it looks like an issue partially with the EmbeddedDatabaseFactory and the TestContext caching mechanism. My "jdbc:embedded-database" definitions do not have any profiles associated with them. Why does the cache need to re-create them and not load them out of the existing cached beans?
You can try to force creation of new embedded database by setting unique name with generateUniqueName(true) each time new object is created.
Here is an example:
embeddedDatabase = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.generateUniqueName(true)
.addScripts("db/sql/create-db.sql", "db/sql/insert-data.sql")
.build();

Normalizing Spring Resource for use with JUnit & war?

I'm probably thinking about this incorrectly, but here's what I'm up against:
I'm injecting Spring Resource objects into my app that give me the location of security certificates, for example:<property name="certificateResource" value="SomeCert.p12" /> where certificateResource is of type org.springframework.core.io.Resource
Running under JUnit the Resource is a classpath resource and everything is fine. When deployed as a war under Tomcat the the Resource is in a Servlet Context and requires WEB-INF/classes/ prepended to the certificate.
I've tried a number of Resource prefix and wildcard combinations but can't come-up with a single string that satisfies both contexts. So far the "solution" is to override the bean definition in src/test/resources/test-applicationContext.xml but that means maintaining the strings in two locations.
Any pointers to better solutions would be appreciated, thanks.
I have tried to make a small but descriptive application using spring-test and spring-webmvc and using resources referenced in xml configs while these xml's are reused by production and testing spring configurations. Here is what I recieved: github.com/isopov/spring-resource-test
The central is the referencing of the prodcution xml config from the testing config:
<import resource="file:src/main/webapp/WEB-INF/applicationContext.xml" />
it is also possible to not import one xml from the other, but give each test several configs:
#ContextConfiguration(locations = { "classpath:test-applicationContext.xml",
"file:src/main/webapp/WEB-INF/applicationContext.xml" })
the resource itself resides in src/main/resources (I assumed you are using Maven or something derived from the "Standard Directory Layout") so is always accessible with "classpath:hello.txt" from any spring config.
I tested this with maven build, as web-application and running UTs inside Eclipse.

Spring Persistence archive and entitymanager is null when accessing from web-application

I've created a jar file through spring roo (maven project - persistence archive) unit tests are running fine, the concerned files are on the following location
jarFile/META-INF/persistence.xml
jarFile/META-INF/applicationContext.xml
jarFile/META-INF/applicationContext-jpa.xml
jarFile/META-INF/database.properties
Unit tests are running fine.
Because its a maven project I added it to local repository by executing the command "mvn install" and after that I added it as a dependency to another maven based web-application.
I am running the web application using mvn jetty:run command. the concerned files in web application are.
webApp/WEB-INF/web.xml
webApp/WEB-INF/applicationContext.xml
The Problem
* Its loading the webapp/WEB-INF/applicationContext.xml but how can I verify its loading the child jarFile/META-INF/applicationContext.xml or not? actually when i try to access the service class methods from persistence archive the entityManager is NULL.
* If i try to put contextConfigLocation directive (tried various options) within web.xml, Its not even loading the webapp/WEB-INF/applicationContext.xml.
What I want
Use the service methods (which uses entitymanager) from persistence archive from within my web application.
Thanks in advance.
Found the answer by digging around a bit.
Actually I was confused with various ways/syntax to include the context file, was trying with all sort of classpath*:xxx syntax but actually the WEB-INF is not on the class path so following have to be added to web.xml to load the main webApp/WEB-INF/applicationContext.xml
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>/WEB-INF/applicationContext.xml</param-value>
</context-param>
Then had to add the following to the webApp/WEB-INF/applicationContext.xml
<import resource="classpath*:META-INF/spring/applicationContext*.xml" />
Now the webapplication is loading the context file from jar file as well. And things are working.

Where to store database passwords in a spring application or use JNDI?

Seems like a simple task. I have a webapp which requires a database connection. I'd like to be able to drop an updated .war file on this app server and load a new version without having to re-edit an applicationConfig.xml file to specify the database connection parameters for production.
Is using the container to setup the data source and then referencing it from JNDI the preferred way to go? I think it is cleaner having it all defined in the spring .xml file, but I can't come up with a clean way to allow the production password to be set only once as we roll out new versions.
So, how do you specify your database connection information in a spring application so that you can upgrade it without having to re-edit the files?
If you use JNDI, how do you handle setting up of your tests since the JNDI is not going to be available outside of the container?
Thanks!
The typical way to externalize database connection properties is to store them in a .properties file and load using <context:property-placeholder .../> . Then you can have different .properties files for testing and production.
If you choose JNDI, you can use a Spring's mock JNDI support for testing.
One approach is for your Spring configuration file to be composed of fragments related to specific layers in your application.
One such fragment could contain your DataSource defintion. For production, this fragment would use a jee:jndi-lookup. And then for test, have a different fragment would use a DriverManagerDataSource ?
Update:
If you want to change the datasource after deployment, then you can use this technique, along with changing the which datasource is injected into your other beans using a PropertyPlaceholderConfigurer as explained in an old post I wrote
eg:
<bean class="foo.bar.SomeClassNeedingDataSource"">
<property name="dataSource" ref="${the.datasource.to.inject}" />
</bean>
<jee:jndi-lookup id="jndiDataSource" ... />
<bean id="driverManagerDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
...
</bean>
# the properties file
the.datasource.to.inject = jndiDataSource
#the.datasource.to.inject = driverManagerDataSource

Resources