Starting jetty with spring xml as a background process/thread - spring

My goal is to set up a jetty test server and inject a custom servlet to test some REST classes in my project. I was able to launch the server with spring xml and run tests against that server. The issue I'm having is sometimes after the server started, the process stopped at the point before running the tests. It seems jetty didn't go to background. It works every time on my computer. But when I deployed to my CI server, it doesn't work. It also doesn't work when I'm on VPN. (Strange.)
The server should be completed initialized as when the tests stuck, I was able to access the server using a browser.
Here is my spring context xml:
....
<bean id="servletHolder" class="org.eclipse.jetty.servlet.ServletHolder">
<constructor-arg ref="courseApiServlet"/>
</bean>
<bean id="servletHandler" class="org.eclipse.jetty.servlet.ServletContextHandler"/>
<!-- Adding the servlet holders to the handlers -->
<bean id="servletHandlerSetter" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="servletHandler"/>
<property name="targetMethod" value="addServlet"/>
<property name="arguments">
<list>
<ref bean="servletHolder"/>
<value>/*</value>
</list>
</property>
</bean>
<bean id="httpTestServer" class="org.eclipse.jetty.server.Server" init-method="start" destroy-method="stop" depends-on="servletHandlerSetter">
<property name="connectors">
<list>
<bean class="org.eclipse.jetty.server.nio.SelectChannelConnector">
<property name="port" value="#{settings['webservice.server.port']}" />
</bean>
</list>
</property>
<property name="handler">
<ref bean="servletHandler" />
</property>
</bean>
Running latest Jetty 8.1.8 server and Spring 3.1.3. Any idea?

I figured it out. My bad. The IP address for the testing web server (jetty) that my REST client connect to is set to a internal IP address (not localhost) that only available to my local host. So that why the tests couldn't start when I'm on VPN or on CI server.
The xml is actually working and I think it is better than starting jetty in separate ant task. Because spring manage the jetty lifecycle. When the tests finish, spring will shutdown jetty automatically.

If you are using Maven, you can let the jetty-maven-plugin start and stop Jetty as part of your build process. You create your Spring project as usual, add the plugin to your .pom file. mvn jetty:run allows you to run the web app unassembled, and mvn jetty:run-war allows you to run a .war file. I guess that what you really want is to have Jetty started in the pre-integration test phase and stopped in the post integration test phase (ref):
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<configuration>
<scanIntervalSeconds>10</scanIntervalSeconds>
<stopKey>foo</stopKey>
<stopPort>9999</stopPort>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>

Related

How to use Liquibase contexts with Jooq code generation

I have a Springboot project that uses Liquibase for database migration and Jooq for database access and the associated code generation. This works fine when Jooq introspects a database that has all the changes applied, but now I wanted to transition to an in-memory H2 database for the code generation so that Jooq is not dependent on my actual (Postgres) database.
But when generating sources with Jooq I get an error now, for a duplicate key exception on a column where I have a unique constraint. I noticed that is because I am using Liquibase contexts in order to insert different data in test, dev and production environments. Jooq seems to ignore these contexts though and applies all changes to same database, and when I insert the same data in test and dev the generation fails. So how can I ensure that Jooq and Liquibase use the correct context (and maven profile) already at the generate sources stage?
Some excerpts from my setup:
pom.xml
<profile>
<id>local</id>
<properties>
<activatedProperties>local</activatedProperties>
</properties>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
...
<plugin>
<groupId>org.jooq</groupId>
<artifactId>jooq-codegen-maven</artifactId>
<version>${jooq.version}</version>
<!-- The plugin should hook into the generate goal -->
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
<!-- Specify the plugin configuration.
The configuration format is the same as for the standalone code generator -->
<configuration>
<generator>
<database>
<name>org.jooq.meta.extensions.liquibase.LiquibaseDatabase</name>
<properties>
<property>
<key>sort</key>
<value>liquibase</value>
</property>
<property>
<key>scripts</key>
<value>src/main/resources/liquibase/changelog-master.xml</value>
</property>
<property>
<key>unqualifiedSchema</key>
<value>none</value>
</property>
<property>
<key>defaultNameCase</key>
<value>lower</value>
</property>
</properties>
</database>
<target>
<packageName>com.graphite.horses</packageName>
<directory>target/generated-sources/jooq</directory>
</target>
<generate>
<javaTimeTypes>true</javaTimeTypes>
</generate>
</generator>
</configuration>
</plugin
Liquibase change file:
<changeSet id="addInitialCredentialsValuesLocal" author="daniel" context="local">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
<changeSet id="addInitialCredentialsValuesTest" author="daniel" context="test">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
And this is where it fails since "my-token" is inserted again in the Jooq's in-memory database even though the test context should not be active.
Starting from jOOQ 3.14.0 and 3.13.2 (see #9872), the "contexts" parameter can be passed along to the Liquibase Database instance like this:
<!-- The property "changeLogParameters.contexts" will be passed on to the
liquibase.database.Database.update() call (jOOQ 3.13.2+).
See https://www.liquibase.org/documentation/contexts.html -->
<property>
<key>changeLogParameters.contexts</key>
<value>!test</value>
</property>
See also the example configuration here:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-liquibase/

Connecting to HSQL DB from Spring

I start HSQL DB with a Maven plugin (I use the DB in integration tests):
<plugin>
<groupId>com.btmatthews.maven.plugins.inmemdb</groupId>
<artifactId>inmemdb-maven-plugin</artifactId>
<version>1.4.3</version>
<configuration>
<monitorKey>inmemdb</monitorKey>
<monitorPort>11527</monitorPort>
</configuration>
<executions>
<execution>
<id>run</id>
<goals>
<goal>run</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<daemon>true</daemon>
<type>hsqldb</type>
<database>MY_DB</database>
<username>user1</username>
<password>111</password>
</configuration>
</execution>
<execution>
<id>stop</id>
<goals>
<goal>stop</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
When the DB is up and running I am able to create a JDBC connection:
Class.forName("org.hsqldb.jdbcDriver");
Connection connection = DriverManager.getConnection("jdbc:hsqldb:hsql://localhost:9001/MY_DB", "sa", "");
System.out.print(connection);
This code prints: org.hsqldb.jdbc.JDBCConnection#2d22efc3.
However, when I try to use the same connection settings to configure a data source in Spring, I get an exception (java.net.ConnectException: Connection refused):
<bean id="dataSource" class="org.apache.tomcat.dbcp.dbcp.BasicDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:hsql://localhost:9001/MY_DB" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
I also tried username:user1 and password:111, it produces another exception: org.hsqldb.HsqlException: invalid authorization specification - not found: user1.
How can I connect to this DB from Spring?
On the advice of counsel, I am adding as answer that your hsql db was not running when you tried to connect from Spring.

Integration tests using jetty, hsql, jndi, and spring

I am attempting to create some integration tests for my Spring web app using Jetty accessing a local HSQL database. The goal: run the tests using Selenium (or similar), mock/stub out all external systems, and setup a HSQL database to hit instead of our shared Oracle database. The tests are started during a maven build (the integration-test phase).
The database is initialized by Spring's "jdbc:initialize-database", and is registered as a JNDI datasource in Jetty.
After days of trying different configuration, I have finally gotten to the point where the database is created, initialized, and I think registered as a Jetty resource, but when the test cases run, it just hangs; I think because it is waiting for the database to become available.
Maven configuration
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.3.3</version>
<executions>
<execution>
<id>start</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<container>
<containerId>jetty7x</containerId>
<dependencies>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</dependency>
<dependency>
<groupId>c3p0</groupId>
<artifactId>c3p0</artifactId>
</dependency>
</dependencies>
</container>
<configuration>
<home>${project.build.directory}/cargo/configurations/jetty7x</home>
<properties>
<cargo.jetty.createContextXml>false</cargo.jetty.createContextXml>
<cargo.datasource.datasource>
cargo.datasource.url=jdbc:hsqldb:file:../../../myDB|
cargo.datasource.driver=org.hsqldb.jdbcDriver|
cargo.datasource.username=sa|
cargo.datasource.password=|
cargo.datasource.type=javax.sql.DataSource|
cargo.datasource.jndi=jdbc/myDataSource
</cargo.datasource.datasource>
</properties>
</configuration>
<deployables>
<deployable>
<location>target/myApp</location>
<properties>
<context>myApp</context>
</properties>
</deployable>
</deployables>
</configuration>
</plugin>
Spring configuration
<bean id="localDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="org.hsqldb.jdbcDriver"/>
<property name="jdbcUrl" value="jdbc:hsqldb:file:target/myDB"/>
<property name="user" value="sa"/>
<property name="password" value=""/>
</bean>
<jdbc:initialize-database data-source="mydataSource" ignore-failures="DROPS">
<jdbc:script location="classpath:/sql-scripts/schema/create-schema.sql"/>
<jdbc:script location="classpath:/sql-scripts/schema/create-tables.sql"/>
<jdbc:script location="classpath:/sql-scripts/testdata/data-load.sql"/>
</jdbc:initialize-database>
I am probably missing something, I tried to piece together the configuration through advice from many other posts. Any help would be appreciated.
The recommended method of using HSQLDB for tests, especially complex test setups, is running a Server.
Initially, you start an HSQLDB server using the shell, independently of your test setup. Use the Server property server.silent=false to see immediately the connections and statements on the console.
After some progress has been made, you can customize the server settings. See the Guide:
http://www.hsqldb.org/doc/2.0/guide/listeners-chapt.html
And a summary of different options for testing:
http://www.hsqldb.org/doc/2.0/guide/deployment-chapt.html#dec_app_dev_testing
You may need to use the MVCC transaction model. This reduces the locks and sometimes avoids the connections hanging as a result of on one waiting for the other to commit.

Maven Exposing properties files appassembler

To load environment specific values, in my src/main/resources folder, I have some properties files in different subfolders i.e.
com/app/ws/webservices-dev.properties
com/app/ws/webservices-test.properties
com/app/jms/jms-dev.properties
com/app/jms/jms-test.properties
I am loading these properties through spring
<bean id="placeholderProperties" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:/com/app/jms/jms-${ENVIRONMENT:dev}.properties</value>
<value>classpath:/com/app/ws/webservices-${ENVIRONMENT:dev}.properties</value>
</list>
</property>
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
<property name="ignoreResourceNotFound" value="true" />
<property name="ignoreUnresolvablePlaceholders" value="true" />
<property name="order" value="0" />
</bean>
ENVIRONMENT is environment variable.
I am using appassembler-maven-plugin to generate the executable .sh file.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<binFileExtensions>
<unix>.sh</unix>
</binFileExtensions>
<programs>
<program>
<mainClass>com.app.MainApp</mainClass>
<name>MainApp</name>
</program>
</programs>
</configuration>
</plugin>
As a result of this all my properties files become part of my generated jar file. I want to expose some of the properties to set their values at deployment time. I have tried following configuration
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<configurationSourceDirectory>src/main/resources/com/app/bootstrap</configurationSourceDirectory>
<configurationDirectory>conf</configurationDirectory>
<copyConfigurationDirectory>true</copyConfigurationDirectory>
<includeConfigurationDirectoryInClasspath>true</includeConfigurationDirectoryInClasspath>
<binFileExtensions>
<unix>.sh</unix>
</binFileExtensions>
<programs>
<program>
<mainClass>com.app.MainApp</mainClass>
<name>MainApp</name>
</program>
</programs>
</configuration>
</plugin>
But Spring does not load the properties; maybe due to the given path in spring config (see above).
What maven configuration should I use to move my properties files in appassembler\conf folder during package time and have spring load them from the classpath. I am after the configuration that works for both development(in eclipse, unit tests as maven builds) and in deployment.

Deploying multiple war files into jetty8

How do I deploy multiple webapps WAR files into Jetty 8 with maven-jetty-plugin?
<contextHandlers>
<contextHandler implementation="org.mortbay.jetty.webapp.WebAppContext">
<war>${basedir}/dir/mywar.war</war>
<contextPath>/path</contextPath>
</contextHandler>
Seems to work only on older plugin versions.
Use the following snippet from a pom.xml. This is adapted from the Jetty server instructions, and although it's for Jetty7 it can easily be adapted for later versions.
pom.xml
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<!-- Jetty 7.3+ requires Maven 3+ -->
<!-- Keep with Jetty 7.6.0 to avoid startup delays from Servlet API 3.0 -->
<version>7.6.0.RC1</version>
<configuration>
<stopKey>STOP</stopKey>
<stopPort>8009</stopPort>
<scanIntervalSeconds>10</scanIntervalSeconds>
<!-- Provide some JNDI resources (optional) -->
<jettyEnvXml>src/test/resources/jetty-jndi-config.xml</jettyEnvXml>
<!-- Register this application as a context -->
<webAppConfig>
<contextPath>/example</contextPath>
</webAppConfig>
<!-- Allow resources on the test classpath to be available -->
<useTestClasspath>true</useTestClasspath>
<!-- Add in any supporting application contexts (use dependencies section) -->
<contextHandlers>
<!-- Supporting WAR (note the use of a property entry for version, and see the dependency later - also Jetty 7 uses org.eclipse...) -->
<contextHandler implementation="org.eclipse.jetty.webapp.WebAppContext">
<war>
${settings.localRepository}/org/example/supporting-war/${supporting-war.version}/supporting-war-${supporting-war.version}.war
</war>
<contextPath>/supporting-war</contextPath>
</contextHandler>
</contextHandlers>
<connectors>
<!-- Later versions of Jetty don't require the Connector to be specified -->
<connector implementation="org.mortbay.jetty.nio.SelectChannelConnector">
<port>8080</port>
<maxIdleTime>60000</maxIdleTime>
</connector>
<!-- SSL for localhost support -->
<connector implementation="org.mortbay.jetty.security.SslSocketConnector">
<port>8443</port>
<maxIdleTime>60000</maxIdleTime>
<!-- Provide a local key store for serving up SSL certificates -->
<keystore>src/test/resources/jetty-ssl.keystore</keystore>
<!-- Pick any password you like -->
<password>jetty6</password>
<keyPassword>jetty6</keyPassword>
</connector>
</connectors>
</configuration>
<dependencies>
<!-- This ensures that WAR files are downloaded from the repo -->
<!-- Example supporting WAR -->
<dependency>
<groupId>org.example</groupId>
<artifactId>supporting-war</artifactId>
<version>${supporting-war.version}</version>
<scope>compile</scope>
<type>war</type>
</dependency>
</dependencies>
</plugin>
I've left the SSL and JNDI configuration in there just in case anyone needs to see how they are configured. Obviously, they will need the supporting files. The SSL assumes that you've already created a suitable key store containing an SSL certificate for, say, localhost. The JNDI configuration file is as follows:
jetty-jndi-config.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
<Configure class="org.mortbay.jetty.webapp.WebAppContext">
<New id="ExampleDB" class="org.mortbay.jetty.plus.naming.Resource">
<Arg>java:jdbc/ExampleDB</Arg>
<Arg>
<New class="com.mchange.v2.c3p0.ComboPooledDataSource">
<Set name="driverClass">oracle.jdbc.driver.OracleDriver</Set>
<Set name="jdbcUrl">jdbc:oracle:thin:#//host:port/schema</Set>
<Set name="user">user</Set>
<Set name="password">password</Set>
<!-- Configure a simple connection test with timeout for subsequent queries -->
<Set name="preferredTestQuery">select 1 from dual</Set>
<Set name="checkoutTimeout">5000</Set>
</New>
</Arg>
</New>
</Configure>
This will allow a JNDI resource lookup using, for example, a Spring bean factory like this:
<bean id="exampleDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jdbc/ExampleDB"/>
<property name="resourceRef" value="true"/>
</bean>
Note that the C3P0 and Oracle references will introduce dependencies that are ostensibly local to your Jetty server, so should be placed in the <plugin><dependencies> section along with the WARs. They don't have to be in the main dependencies.
So now your Maven build will contain an embedded Jetty web server, configured to work with multiple WARs, all tied into the pom.xml version, providing both HTTP and HTTPS and backed with a pooled database connection. That's pretty much everything you need right out of the box for an integrated development environment.

Resources