Integration tests using jetty, hsql, jndi, and spring - spring

I am attempting to create some integration tests for my Spring web app using Jetty accessing a local HSQL database. The goal: run the tests using Selenium (or similar), mock/stub out all external systems, and setup a HSQL database to hit instead of our shared Oracle database. The tests are started during a maven build (the integration-test phase).
The database is initialized by Spring's "jdbc:initialize-database", and is registered as a JNDI datasource in Jetty.
After days of trying different configuration, I have finally gotten to the point where the database is created, initialized, and I think registered as a Jetty resource, but when the test cases run, it just hangs; I think because it is waiting for the database to become available.
Maven configuration
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.3.3</version>
<executions>
<execution>
<id>start</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<container>
<containerId>jetty7x</containerId>
<dependencies>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</dependency>
<dependency>
<groupId>c3p0</groupId>
<artifactId>c3p0</artifactId>
</dependency>
</dependencies>
</container>
<configuration>
<home>${project.build.directory}/cargo/configurations/jetty7x</home>
<properties>
<cargo.jetty.createContextXml>false</cargo.jetty.createContextXml>
<cargo.datasource.datasource>
cargo.datasource.url=jdbc:hsqldb:file:../../../myDB|
cargo.datasource.driver=org.hsqldb.jdbcDriver|
cargo.datasource.username=sa|
cargo.datasource.password=|
cargo.datasource.type=javax.sql.DataSource|
cargo.datasource.jndi=jdbc/myDataSource
</cargo.datasource.datasource>
</properties>
</configuration>
<deployables>
<deployable>
<location>target/myApp</location>
<properties>
<context>myApp</context>
</properties>
</deployable>
</deployables>
</configuration>
</plugin>
Spring configuration
<bean id="localDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="org.hsqldb.jdbcDriver"/>
<property name="jdbcUrl" value="jdbc:hsqldb:file:target/myDB"/>
<property name="user" value="sa"/>
<property name="password" value=""/>
</bean>
<jdbc:initialize-database data-source="mydataSource" ignore-failures="DROPS">
<jdbc:script location="classpath:/sql-scripts/schema/create-schema.sql"/>
<jdbc:script location="classpath:/sql-scripts/schema/create-tables.sql"/>
<jdbc:script location="classpath:/sql-scripts/testdata/data-load.sql"/>
</jdbc:initialize-database>
I am probably missing something, I tried to piece together the configuration through advice from many other posts. Any help would be appreciated.

The recommended method of using HSQLDB for tests, especially complex test setups, is running a Server.
Initially, you start an HSQLDB server using the shell, independently of your test setup. Use the Server property server.silent=false to see immediately the connections and statements on the console.
After some progress has been made, you can customize the server settings. See the Guide:
http://www.hsqldb.org/doc/2.0/guide/listeners-chapt.html
And a summary of different options for testing:
http://www.hsqldb.org/doc/2.0/guide/deployment-chapt.html#dec_app_dev_testing
You may need to use the MVCC transaction model. This reduces the locks and sometimes avoids the connections hanging as a result of on one waiting for the other to commit.

Related

How to use Liquibase contexts with Jooq code generation

I have a Springboot project that uses Liquibase for database migration and Jooq for database access and the associated code generation. This works fine when Jooq introspects a database that has all the changes applied, but now I wanted to transition to an in-memory H2 database for the code generation so that Jooq is not dependent on my actual (Postgres) database.
But when generating sources with Jooq I get an error now, for a duplicate key exception on a column where I have a unique constraint. I noticed that is because I am using Liquibase contexts in order to insert different data in test, dev and production environments. Jooq seems to ignore these contexts though and applies all changes to same database, and when I insert the same data in test and dev the generation fails. So how can I ensure that Jooq and Liquibase use the correct context (and maven profile) already at the generate sources stage?
Some excerpts from my setup:
pom.xml
<profile>
<id>local</id>
<properties>
<activatedProperties>local</activatedProperties>
</properties>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
...
<plugin>
<groupId>org.jooq</groupId>
<artifactId>jooq-codegen-maven</artifactId>
<version>${jooq.version}</version>
<!-- The plugin should hook into the generate goal -->
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
<!-- Specify the plugin configuration.
The configuration format is the same as for the standalone code generator -->
<configuration>
<generator>
<database>
<name>org.jooq.meta.extensions.liquibase.LiquibaseDatabase</name>
<properties>
<property>
<key>sort</key>
<value>liquibase</value>
</property>
<property>
<key>scripts</key>
<value>src/main/resources/liquibase/changelog-master.xml</value>
</property>
<property>
<key>unqualifiedSchema</key>
<value>none</value>
</property>
<property>
<key>defaultNameCase</key>
<value>lower</value>
</property>
</properties>
</database>
<target>
<packageName>com.graphite.horses</packageName>
<directory>target/generated-sources/jooq</directory>
</target>
<generate>
<javaTimeTypes>true</javaTimeTypes>
</generate>
</generator>
</configuration>
</plugin
Liquibase change file:
<changeSet id="addInitialCredentialsValuesLocal" author="daniel" context="local">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
<changeSet id="addInitialCredentialsValuesTest" author="daniel" context="test">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
And this is where it fails since "my-token" is inserted again in the Jooq's in-memory database even though the test context should not be active.
Starting from jOOQ 3.14.0 and 3.13.2 (see #9872), the "contexts" parameter can be passed along to the Liquibase Database instance like this:
<!-- The property "changeLogParameters.contexts" will be passed on to the
liquibase.database.Database.update() call (jOOQ 3.13.2+).
See https://www.liquibase.org/documentation/contexts.html -->
<property>
<key>changeLogParameters.contexts</key>
<value>!test</value>
</property>
See also the example configuration here:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-liquibase/

Connecting to HSQL DB from Spring

I start HSQL DB with a Maven plugin (I use the DB in integration tests):
<plugin>
<groupId>com.btmatthews.maven.plugins.inmemdb</groupId>
<artifactId>inmemdb-maven-plugin</artifactId>
<version>1.4.3</version>
<configuration>
<monitorKey>inmemdb</monitorKey>
<monitorPort>11527</monitorPort>
</configuration>
<executions>
<execution>
<id>run</id>
<goals>
<goal>run</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<daemon>true</daemon>
<type>hsqldb</type>
<database>MY_DB</database>
<username>user1</username>
<password>111</password>
</configuration>
</execution>
<execution>
<id>stop</id>
<goals>
<goal>stop</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
When the DB is up and running I am able to create a JDBC connection:
Class.forName("org.hsqldb.jdbcDriver");
Connection connection = DriverManager.getConnection("jdbc:hsqldb:hsql://localhost:9001/MY_DB", "sa", "");
System.out.print(connection);
This code prints: org.hsqldb.jdbc.JDBCConnection#2d22efc3.
However, when I try to use the same connection settings to configure a data source in Spring, I get an exception (java.net.ConnectException: Connection refused):
<bean id="dataSource" class="org.apache.tomcat.dbcp.dbcp.BasicDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:hsql://localhost:9001/MY_DB" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
I also tried username:user1 and password:111, it produces another exception: org.hsqldb.HsqlException: invalid authorization specification - not found: user1.
How can I connect to this DB from Spring?
On the advice of counsel, I am adding as answer that your hsql db was not running when you tried to connect from Spring.

How to use Liquibase to update database in embedded Glassfish instance

I'm in the process of converting our db management to Liquibase. This is running along nicely.
As a subsequent step I want to assure that all future modifications gets tested before deploy to common environments, continuous integration style. I try to do this using the following setup:
Build our ear containing EJB webservices
Using maven-embedded-glassfish-plugin to start an embedded instance of Glassfish 3 during pre-integration-test maven ohase
Create my datasource as part of start goal
Deploy ear during deploy goal
Still in pre-integration-test, I run liquibase:update on the same database URL. In this case a H2 file database
I then want to run our SoapUI tests on the deployed application
But when i get this far the application can't find any data in the database. So the question is if I've missed something in my setup or if there's a better way to organize my intended goal?
pom.xml, embedded Glassfish
<plugin>
<groupId>org.glassfish.embedded</groupId>
<artifactId>maven-embedded-glassfish-plugin</artifactId>
<version>4.0</version>
<configuration>
<ports>
<http-listener>9090</http-listener>
<https-listener>9191</https-listener>
</ports>
<goalPrefix>embedded-glassfish</goalPrefix>
<app>${project.build.directory}/school-application-${project.version}.ear</app>
<name>school-application</name>
<commands>
<command>create-jdbc-connection-pool --datasourceclassname=org.h2.jdbcx.JdbcDataSource --restype=javax.sql.DataSource --property URL=jdbc\:h2\:~/tmpLB\;AUTO_SERVER\=TRUE schoolDSPool</command>
<command>create-jdbc-resource --connectionpoolid schoolDSPool jdbc/schoolDS</command>
</commands>
</configuration>
<dependencies>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.176</version>
</dependency>
</dependencies>
<executions>
<execution>
<goals>
<goal>start</goal>
<goal>admin</goal>
<goal>deploy</goal>
<goal>undeploy</goal>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
pom.xml, Liquibase
<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>3.1.1</version>
<dependencies>
<dependency>
<groupId>company.school</groupId>
<artifactId>school-db</artifactId>
<version>${project.version}</version>
<systemPath>../school-db/target</systemPath>
<type>jar</type>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.176</version>
</dependency>
</dependencies>
<executions>
<execution>
<phase>integration-test</phase>
<configuration>
<promptOnNonLocalDatabase>false</promptOnNonLocalDatabase>
<changeLogFile>db.changelog-master.xml</changeLogFile>
<driver>org.h2.Driver</driver>
<url>jdbc:h2:~/tmpLB;AUTO_SERVER=TRUE</url>
<logging>info</logging>
</configuration>
<goals>
<goal>update</goal>
</goals>
</execution>
</executions>
</plugin>
I have one changeset in the changelog inserting data in targeted tables.
Do I have the right users set up?
Is there a way to run Liquibase in the same process as Glassfish and use a mem: database instead?
Thx and regards,
Christian
Ok, so there was an "easy" solution to the problem.
There was no data in the database since that changeset in liquibase changelog couldn't complete. I had the insert statements in a separate sql file that I called using the <sqlFile> liquibase tag. But the insert was violating some foreign key constraints and didn't get executed.
So what put me off was the fact that Liquibase seems to hide any errors from included sql files. Will try try reproduce that and Jira it if I succeed.
/Christian

How to disable Spring's JpaExceptionTranslatorAspect

I'm migrating from Spring 2.5.6 to 3.2.5. The jar spring-aspects-3.2.5 contains the new aspect JpaExceptionTranslatorAspect which translates standard JPA exceptions into Spring exceptions. It seems to be a Roo-specific aspect. This aspect gets automatically weaved into repositories (annotated with #Repository). Consequently, standard JPA exceptions are not caught anymore and the application is broken.
How can I exclude JpaExceptionTranslatorAspect from being weaved? If it can't be done, is there any other workaround? Or am I missing some piece of configuration?
I'm using AspectJ 1.7.4 and AspectJ Maven Plugin 1.4.
What I have already gathered:
Spring rejected the issue because it's a build issue
AspectJ Maven Plugin rejected the issue because the AspectJ compiler doesn't support excluding specific aspects from a library
However, I wonder if those pieces of information are up to date.
First, upgrade aspectj-maven-plugin to 1.5 and add the complianceLevel tag in the configuration of the plugin (otherwise it will try to compile with java 1.4 compliance by default).
Then you can specify the exclusion through the xmlConfigured property of the aspectj-maven-plugin. This property references a file from your local directory (i.e. where your pom.xml is)
pom.xml exemple :
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.5</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
<complianceLevel>${maven.compiler.target}</complianceLevel>
<xmlConfigured>myCtAspects.xml</xmlConfigured>
<aspectLibraries>
<aspectLibrary>
<groupId>org.springframework</groupId>
<artifactId>spring-aspects</artifactId>
</aspectLibrary>
</aspectLibraries>
<showWeaveInfo>true</showWeaveInfo>
<weaveMainSourceFolder>true</weaveMainSourceFolder>
<proceedOnError>${maven.aspectj.failOnError}</proceedOnError>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
<phase>process-resources</phase>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjrt</artifactId>
<version>${aspectj.version}</version>
</dependency>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjtools</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
Then in myCtAspects.xml file, you just have to specify all the wanted aspects explicitly, including Spring Aspects. In your case:
<?xml version="1.0"?>
<aspectj>
<aspects>
<!-- Spring Aspects -->
<aspect name="org.springframework.beans.factory.aspectj.AbstractInterfaceDrivenDependencyInjectionAspect"/>
<aspect name="org.springframework.beans.factory.aspectj.AnnotationBeanConfigurerAspect"/>
<aspect name="org.springframework.transaction.aspectj.AnnotationTransactionAspect"/>
<!-- Your Application Aspects -->
</aspects>
</aspectj>
Please try to use aop-autoproxy's include proprety with some invert regexp (something like ^((?! JpaExceptionTranslatorAspect).)*$).

Starting jetty with spring xml as a background process/thread

My goal is to set up a jetty test server and inject a custom servlet to test some REST classes in my project. I was able to launch the server with spring xml and run tests against that server. The issue I'm having is sometimes after the server started, the process stopped at the point before running the tests. It seems jetty didn't go to background. It works every time on my computer. But when I deployed to my CI server, it doesn't work. It also doesn't work when I'm on VPN. (Strange.)
The server should be completed initialized as when the tests stuck, I was able to access the server using a browser.
Here is my spring context xml:
....
<bean id="servletHolder" class="org.eclipse.jetty.servlet.ServletHolder">
<constructor-arg ref="courseApiServlet"/>
</bean>
<bean id="servletHandler" class="org.eclipse.jetty.servlet.ServletContextHandler"/>
<!-- Adding the servlet holders to the handlers -->
<bean id="servletHandlerSetter" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="servletHandler"/>
<property name="targetMethod" value="addServlet"/>
<property name="arguments">
<list>
<ref bean="servletHolder"/>
<value>/*</value>
</list>
</property>
</bean>
<bean id="httpTestServer" class="org.eclipse.jetty.server.Server" init-method="start" destroy-method="stop" depends-on="servletHandlerSetter">
<property name="connectors">
<list>
<bean class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<property name="port" value="#{settings['webservice.server.port']}" />
</bean>
</list>
</property>
<property name="handler">
<ref bean="servletHandler" />
</property>
</bean>
Running latest Jetty 8.1.8 server and Spring 3.1.3. Any idea?
I figured it out. My bad. The IP address for the testing web server (jetty) that my REST client connect to is set to a internal IP address (not localhost) that only available to my local host. So that why the tests couldn't start when I'm on VPN or on CI server.
The xml is actually working and I think it is better than starting jetty in separate ant task. Because spring manage the jetty lifecycle. When the tests finish, spring will shutdown jetty automatically.
If you are using Maven, you can let the jetty-maven-plugin start and stop Jetty as part of your build process. You create your Spring project as usual, add the plugin to your .pom file. mvn jetty:run allows you to run the web app unassembled, and mvn jetty:run-war allows you to run a .war file. I guess that what you really want is to have Jetty started in the pre-integration test phase and stopped in the post integration test phase (ref):
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<configuration>
<scanIntervalSeconds>10</scanIntervalSeconds>
<stopKey>foo</stopKey>
<stopPort>9999</stopPort>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>

Resources