I'm using Spring Test to test my Spring Application, which is basically a REST webservice that serves JSON and some other uploaded media.
The unit tests are working fine in all developer's machines, and some of them tests for strings returning inside the JSON responses, and a little subset of them tests special characters.
The application is completely configured to use UTF-8 with everything, however the JSON response is returning with the wrong charset, so the test fails.
I tried to configure on /etc/default/jenkins the following line:
JAVA_ARGS="-Djava.awt.headless=true -Dfile.encoding=UTF8"
But I still get the same result.
I also had the chance to see what Charset.defaultCharset() was, when running a particular test... And it appeared something like US-ASCII...
Can someone please tell me what am I doing wrong?
I found out what was wrong.
In my Spring Test configuration, I was reading a DML.sql file to create initial test data, and the line that initialized the database was like this:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:sql/DML.sql" />
</jdbc:initialize-database>
That left my configuration dependent from the server charset, and in my case I had that weird "US-ASCII" set in my server.
I made it independent by setting the encoding attribute to UTF-8:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:sql/DML.sql" encoding="UTF-8" />
</jdbc:initialize-database>
So the values that came from the H2 database could be tested against values with special characters.
Related
I can't entirely configure property-placeholder from system property because I can't give comma-separated list of resources.
I'm trying to do like:
<context:property-placeholder location="${config-location}" />
I use system property to configure this. It works if I give one location only, like "classpath:main.properties", but it does not if I'm trying this: "classpath:main1.properties,classpath:main2.properties".
If I use this latter exact value directly in xml configuration it works fine. I guess it resolves comma-separation earlier than placeholders. It should be the other way around.
P.S : version 4.3.4
Another possible worth tryout would be,
<context:property-placeholder location="#{systemProperties['config-location']}" />
how to read System environment variable in Spring applicationContext
I'm working on a new Spring Batch (3.0.3.RELEASE) application where there will be multiple databases accessed during the jobs. For testing we are using HSQLDB (2.3.2) as the embedded database.
In my Application context I have the following.
<jdbc:embedded-database id="dataSource">
</jdbc:embedded-database>
<jdbc:embedded-database id="proDataSource">
<jdbc:script location="classpath:script-tables.sql" />
<jdbc:script location="classpath:script-constraints.sql" />
</jdbc:embedded-database>
<jdbc:embedded-database id="altDataSource">
<jdbc:script location="classpath:script-alt-tables.sql" />
</jdbc:embedded-database>
When I run a single test in Eclipse, things are fine. When I build from the command line, after the first test, I get errors
Failed to execute SQL script statement at line 3 of resource class path resource [script-promrkt-promo.sql]
object name already exists: PROMRKT
It appears to me that the population process in EmbeddedDatabaseFactory is receiving an already populated database. From what I can tell is that after each test there is not a SHUTDOWN being executed and HSQLDB is leaving the already populated database in memory.
I have re-reviewed the documentation and in a Spring Doc this does show a explicit shutdown command. But if spring starts up the embedded database when my test starts why doesn't it shut it down when the test completes ?
Is it expected the embedded databases will remain after each unit test for the same application context?
What is the order that spring starts up an embedded database and when is the transactional context initialized?
Do I need to use a database cleaner ?
Can the populate be updated to only populate when the database is first started, and rollback to the original script configuration when my test is complete ( kinda like how the AbstractTransactionalSpringContextTests worked )
Do I need some transactional markers? Spring Batch's JobRepo is properly being populated and destroyed between each test. Why are my custom dataSources not ?
The script the log message is complaining about isn't in your configuration. I presume it's being executed somewhere else? If that's the case, you'll probably need to add #DirtiesContext to your tests so that Spring doesn't cache the context (I'm assuming you're using the SpringJunit4Runner with #ContextConfiguration but can't be sure since your actual test isn't in the question).
If my assumption is correct, Spring caches the context in an effort to improve performance over the running of a unit test suite. If your test modifies the context in a way that can impact other tests (like running scripts in one test that need to be run again in others), you mark the tests with #DirtiesContext and Spring won't cache the context. You can use the annotation at either the method or class level. You can read more about the annotation here: http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/test/annotation/DirtiesContext.html
I spent a lot of time looking at this and reading the Spring Framework documentation (gasp!) and tracing through code. There are some interesting changes in 4.1 spring core, especially the testing.
I found out that ApplicationContext(s) are cached at the JVM level now. If a second test asks for a context the TestContext looks first in it's cache to see if some other test has already asked for the identical configuration.
I have some profiles for some of my tests. A test with a different profile but the same #ContextConfiguration causes the that context to be re-loaded with the profile applied. When the "Bean Loader" arrives at creating the embedded databases, the EmbeddedDatabaseFactory does not take into consideration that the embedded database (in memory HSQLDB) may have already been created or cached from previous tests and does not need to be re-initialized.
Therefore I added some logic to the EmbeddedDatabaseFactory.initDatabase() checking if the database already exists before re-initializing & running the DatabasePopulator.
List existingDataBases = org.hsqldb.DatabaseManager.getDatabaseURIs();
boolean isExisting = false;
String localDBName = StringUtils.lowerCase(this.databaseName);
for (Object object : existingDataBases) {
if (object.toString().contains(localDBName)) {
isExisting = true;
break;
}
}
// Now populate the database
if (!isExisting && this.databasePopulator != null) {
( of course this isn't quite kosher for what spring would need but it gets the point across )
In my opinion it looks like an issue partially with the EmbeddedDatabaseFactory and the TestContext caching mechanism. My "jdbc:embedded-database" definitions do not have any profiles associated with them. Why does the cache need to re-create them and not load them out of the existing cached beans?
You can try to force creation of new embedded database by setting unique name with generateUniqueName(true) each time new object is created.
Here is an example:
embeddedDatabase = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.generateUniqueName(true)
.addScripts("db/sql/create-db.sql", "db/sql/insert-data.sql")
.build();
It seems like there's been a few iterations of property support in spring it's hard to tell what's best practice and the manuals are written from the point of view of someone who is familiar with every other iteration. I feel like this should be a simple and common requirement but given how hard it's been please correct me if there's a more idiomatic way.
What I want is to pass an additional properties file to my spring web app based on a context property which the client is setting using a tomcat descriptor like so
<Context path="/foo" reloadable="true">
<Parameter name="foo.config" value="file:${catalina.base}/conf/foo.properties"/>
</Context>
In spring for the live profile I have this
<beans profile="live">
<context:property-placeholder location="classpath:timetabling.live.properties,${timetabling.config}"
ignore-resource-not-found="true" />
</beans>
So I'd assumed this doesn;t work because I'm trying to configure placeholder suppport with a placeholder. If I use a system property however then this works fine. I know that spring 3.1 has baked in support for system and environment properties so I guess my question is how can I augment this support with something context aware before the placeholder is resolved?
--Update--
looking at http://blog.springsource.org/2011/02/15/spring-3-1-m1-unified-property-management/ particularly at footnote 1, I would expect to have a DefaultWebEnvironment which should already have aceess to context init params. Now I am more confused, can someone provide me with a concrete example of context property retrieval? At this point I feel like I've read every javadoc available and they are just not helpful.
<context:property-placeholder /> sets up a PropertyPlaceholderConfigurer which reads from .properties, system properties and environment variables. A Tomcat context.xml however sets up a servlet context init parameter. So what you need is a ServletContextPropertyPlaceholderConfigurer.
I can use Spring to create and initialize embedded databases either programmatically:
#Before
public void setUp() {
database = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.addScript("schema.sql")
.addScript("init.sql")
.build();
....
}
or via Spring configuration:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="classpath:schema.sql"/>
<jdbc:script location="classpath:init.sql"/>
</jdbc:initialize-database>
Here my scripts schema.sql and init.sql are stored in the directory src/test/resources. So if tests using the embedded databse are run with:
mvn test
then the files, being in src/test/resources are available, and everything is fine.
But now suppose I want to instead use the embedded database together with an embedded web server (Jetty or embedded Tomcat) in an integration test, run via
mvn integration-test
Now in this late phase, I want to do an almost end-to-end test that hitting certain urls for the web service return the expected HTTP responses, assuming certain data in the embedded database. But at this point, the war deployed in the embedded webserver does not have files from src/test/resources so my initialization scripts are not present, and I get the error
Caused by: java.io.FileNotFoundException: class path resource [schema.sql] cannot be opened because it does not exist
Now everything will work if I put the scripts in src/main/resources but these scripts are only for filling up an embedded database for testing and they do not belong in the war file at all. Does anyone know how to set up an integration test so that an emedded database can be used, without polluting the actual war file that will be deployed?
I was hoping that Spring's embedded database initialization could use something other than a file. I thought about JNDI but that would seem to require polluting the web.xml file with a resource definition for test data which would (again) appear in the deployable war. Or perhaps there are options using cargo? Some programmatic tricks with Spring that I don't know?
You should be able to do this:
<jdbc:initialize-database data-source="dataSource">
<jdbc:script location="file:///${project.basedir}/src/test/resources/schema.sql"/>
<jdbc:script location="file:///${project.basedir}/src/test/resources/init.sql"/>
</jdbc:initialize-database>
I am building an application using spring mvc and jpa using jboss7 and mysql in eclipse ide. I am having a strange problem. All my jsp pages are encoded with charset: utf8, which I think is working correctly. But whenever I try to post a data from the jsp to the controller, my data gets encoded with a different encoding style. I tried to look for the header using firebug and was astonished to see that the post request has a header with content-type : "text/plain;charset=ISO-8859-1". I have already configured the SetCharacterEncodingFilter for UTF-8 in my web.xml (it is the first filter). But still the problem exists.
I also set "org.apache.catalina.connector.URI_ENCODING" to value="UTF-8".But in vain .
Also I have added bean messageSource with property defaultEncoding set to "UTF-8".
<bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource" >
The problem still exists. Please help
Thanks in advance.
The request header is set by the browser, so your application can't control it. Usually, in your HTML form you could put an accept-charset=utf-8 attribute to specify the encoding, but that doesn't necessarily work. See this question Setting the character encoding in form submit for Internet Explorer.
you should need to set the encoding of the JVM like follow :
-Dfile.encoding=UTF-8 -Dfile.io.encoding=UTF-8 -DjavaEncoding=UTF-8
thus there wont be any doubt at all.