How to use Liquibase contexts with Jooq code generation - spring-boot

I have a Springboot project that uses Liquibase for database migration and Jooq for database access and the associated code generation. This works fine when Jooq introspects a database that has all the changes applied, but now I wanted to transition to an in-memory H2 database for the code generation so that Jooq is not dependent on my actual (Postgres) database.
But when generating sources with Jooq I get an error now, for a duplicate key exception on a column where I have a unique constraint. I noticed that is because I am using Liquibase contexts in order to insert different data in test, dev and production environments. Jooq seems to ignore these contexts though and applies all changes to same database, and when I insert the same data in test and dev the generation fails. So how can I ensure that Jooq and Liquibase use the correct context (and maven profile) already at the generate sources stage?
Some excerpts from my setup:
pom.xml
<profile>
<id>local</id>
<properties>
<activatedProperties>local</activatedProperties>
</properties>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
</profile>
...
<plugin>
<groupId>org.jooq</groupId>
<artifactId>jooq-codegen-maven</artifactId>
<version>${jooq.version}</version>
<!-- The plugin should hook into the generate goal -->
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
</execution>
</executions>
<!-- Specify the plugin configuration.
The configuration format is the same as for the standalone code generator -->
<configuration>
<generator>
<database>
<name>org.jooq.meta.extensions.liquibase.LiquibaseDatabase</name>
<properties>
<property>
<key>sort</key>
<value>liquibase</value>
</property>
<property>
<key>scripts</key>
<value>src/main/resources/liquibase/changelog-master.xml</value>
</property>
<property>
<key>unqualifiedSchema</key>
<value>none</value>
</property>
<property>
<key>defaultNameCase</key>
<value>lower</value>
</property>
</properties>
</database>
<target>
<packageName>com.graphite.horses</packageName>
<directory>target/generated-sources/jooq</directory>
</target>
<generate>
<javaTimeTypes>true</javaTimeTypes>
</generate>
</generator>
</configuration>
</plugin
Liquibase change file:
<changeSet id="addInitialCredentialsValuesLocal" author="daniel" context="local">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
<changeSet id="addInitialCredentialsValuesTest" author="daniel" context="test">
<insert tableName="credentials">
<column name="key" value="my-token"/>
<column name="platform" value="web"/>
</insert>
</changeSet>
And this is where it fails since "my-token" is inserted again in the Jooq's in-memory database even though the test context should not be active.

Starting from jOOQ 3.14.0 and 3.13.2 (see #9872), the "contexts" parameter can be passed along to the Liquibase Database instance like this:
<!-- The property "changeLogParameters.contexts" will be passed on to the
liquibase.database.Database.update() call (jOOQ 3.13.2+).
See https://www.liquibase.org/documentation/contexts.html -->
<property>
<key>changeLogParameters.contexts</key>
<value>!test</value>
</property>
See also the example configuration here:
https://www.jooq.org/doc/latest/manual/code-generation/codegen-liquibase/

Related

TestNG Report is not displaying more than 21 tests

I have a maven project that is running testNG tests via a Testng xml. The XML has the following listeners
<listener class-name="org.uncommons.reportng.HTMLReporter" />
<listener class-name="org.uncommons.reportng.JUnitXMLReporter" />
I have 24 tests in the xml with all of them running packages e.g
<test name="First Test"
preserve-order="true">
<packages>
<package
name="testProject.first.*" />
</packages>
</test>
For some reason all 24 tests run but in the testng report I only ever see 21 tests. The 21 tests that show are not always consistent so that rules out the possibillity of a problem with the setup of some tests. I am wondering if there is possibly a max number of tests that run and if I need to combine tests? or is there a maximum number of tests parameter somewhere that can be set?
The following is my maven-surefire-plugin setup
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<properties>
<property>
<name>usedefaultlisteners</name>
<value>false</value>
</property>
<property>
<name>listener</name>
<value>org.uncommons.reportng.HTMLReporter,org.uncommons.reportng.JUnitXMLReporter</value>
</property>
<systemPropertyVariables>
<environment>${env.browser}</environment>
</systemPropertyVariables>
<systemPropertyVariables>
<environment>${env.environment}</environment>
</systemPropertyVariables>
<systemPropertyVariables>
<environment>${env.testlinkRelease}</environment>
</systemPropertyVariables>
</properties>
<suiteXmlFiles>
<suiteXmlFile>src/testCleanup/${cleanup.suite}</suiteXmlFile>
<suiteXmlFile>src/limelightTests/${test.suite}</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
Thanks for any help.
I found the issue. I had a screenshot listener that was running after #AfterMethod had run. #AfterMethod quit the driver so the screenshot listener ended up throwing an error which was not caught and caused the test to be left out of the report.

override property value on pom.xml

A part of code in my pom.xml looks like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${suite.xml}</suiteXmlFile>
</suiteXmlFiles>
<properties>
<property>
<name>usedefaultlisteners</name>
<value>false</value>
</property>
<property>
<name>listener</name>
<value>org.uncommons.reportng.HTMLReporter,org.uncommons.reportng.JUnitXMLReporter
</value>
</property>
</properties>
<testFailureIgnore>true</testFailureIgnore>
<systemProperties>
<property>
<name>org.uncommons.reportng.title</name>
<value>${title}</value>
</property>
</systemProperties>
<argLine>-Xms128m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=256m
-XX:+UseConcMarkSweepGC
</argLine>
</configuration>
</plugin>
I am sending -Dtitle from my jenkins job and it's working fine. Like wise I want to change the value in tag <testFailureIgnore>true</testFailureIgnore>. But the problem here is: I don't want to change the default value. I just want to parameterize this only in case if user provide "false" value from jenkins. If user do not provide anything then it should be "true".
Create a separate profile in pom where you can provide different values.
Plugin definition in pom will contain default value.
So if you run simple maven command : mvn install it will use default value
and if you run using profile that is mvn -Pprofile install then it will use value specified in profile.
Add one property <maven.test.failure.ignore>true</maven.test.failure.ignore> under <properties> tag of your pom.
Replace line testFailureIgnore line by <testFailureIgnore>${maven.test.failure.ignore}</testFailureIgnore>.
In case of "false" you need to pass one extra parameter -Dmaven.test.failure.ignore=false
It easy. First you have to add a shell comand to your job. In this command you could make a sed comand an replace this line on the other.
Example:
sed -i 's/ <maven.test.failure.ignore>true</maven.test.failure.ignore> /<maven.test.failure.ignore>false</maven.test.failure.ignore>/g' pom.xml

Integration tests using jetty, hsql, jndi, and spring

I am attempting to create some integration tests for my Spring web app using Jetty accessing a local HSQL database. The goal: run the tests using Selenium (or similar), mock/stub out all external systems, and setup a HSQL database to hit instead of our shared Oracle database. The tests are started during a maven build (the integration-test phase).
The database is initialized by Spring's "jdbc:initialize-database", and is registered as a JNDI datasource in Jetty.
After days of trying different configuration, I have finally gotten to the point where the database is created, initialized, and I think registered as a Jetty resource, but when the test cases run, it just hangs; I think because it is waiting for the database to become available.
Maven configuration
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.3.3</version>
<executions>
<execution>
<id>start</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<container>
<containerId>jetty7x</containerId>
<dependencies>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
</dependency>
<dependency>
<groupId>c3p0</groupId>
<artifactId>c3p0</artifactId>
</dependency>
</dependencies>
</container>
<configuration>
<home>${project.build.directory}/cargo/configurations/jetty7x</home>
<properties>
<cargo.jetty.createContextXml>false</cargo.jetty.createContextXml>
<cargo.datasource.datasource>
cargo.datasource.url=jdbc:hsqldb:file:../../../myDB|
cargo.datasource.driver=org.hsqldb.jdbcDriver|
cargo.datasource.username=sa|
cargo.datasource.password=|
cargo.datasource.type=javax.sql.DataSource|
cargo.datasource.jndi=jdbc/myDataSource
</cargo.datasource.datasource>
</properties>
</configuration>
<deployables>
<deployable>
<location>target/myApp</location>
<properties>
<context>myApp</context>
</properties>
</deployable>
</deployables>
</configuration>
</plugin>
Spring configuration
<bean id="localDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="org.hsqldb.jdbcDriver"/>
<property name="jdbcUrl" value="jdbc:hsqldb:file:target/myDB"/>
<property name="user" value="sa"/>
<property name="password" value=""/>
</bean>
<jdbc:initialize-database data-source="mydataSource" ignore-failures="DROPS">
<jdbc:script location="classpath:/sql-scripts/schema/create-schema.sql"/>
<jdbc:script location="classpath:/sql-scripts/schema/create-tables.sql"/>
<jdbc:script location="classpath:/sql-scripts/testdata/data-load.sql"/>
</jdbc:initialize-database>
I am probably missing something, I tried to piece together the configuration through advice from many other posts. Any help would be appreciated.
The recommended method of using HSQLDB for tests, especially complex test setups, is running a Server.
Initially, you start an HSQLDB server using the shell, independently of your test setup. Use the Server property server.silent=false to see immediately the connections and statements on the console.
After some progress has been made, you can customize the server settings. See the Guide:
http://www.hsqldb.org/doc/2.0/guide/listeners-chapt.html
And a summary of different options for testing:
http://www.hsqldb.org/doc/2.0/guide/deployment-chapt.html#dec_app_dev_testing
You may need to use the MVCC transaction model. This reduces the locks and sometimes avoids the connections hanging as a result of on one waiting for the other to commit.

How can I use maven profile Id value in spring bean files?

mvn -P dev
If I build my project using profile dev, then I want to use dev.properties in my spring bean like below. Is it possible ? If so , how could I get profile name ?
<bean id="xyz" class="abc.xyz">
<property name="propertyFile" value="${maven_profile_id}.properties" />
</bean>
Thanks in advance.
You can use Maven profiles to add a 'profile' property to the build:
<profiles>
<profile>
<id>dev</id>
<properties>
<profile>dev</profile>
</properties>
</profile>
</profiles>
Then pass the value into your application using a system property, here's an example with surefire:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<profile>${profile}</profile>
</systemPropertyVariables>
</configuration>
</plugin>
Finally this can be referenced in you application:
<bean id="xyz" class="abc.xyz">
<property name="propertyFile" value="${profile}.properties" />
</bean>
Alternatively, if you are using Spring 3.1 or later you might find the XML profile feature meets your needs (although it may be overkill).
Create a properties file that will be populated using Maven's resource filtering that specifies the profile you are using at build time.
build.properties
activatedProfile=${profileId}
pom.xml (You don't need to filter the complete directory, customise as required)
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
<resources>
</build>
Add a profileId (or whatever you want to call it) property under each different profile:
<profile>
<id>dev</id>
<properties>
<profileId>dev</profileId>
</properties>
</profile>
<profile>
<id>qa</id>
<properties>
<profileId>qa</profileId>
</properties>
</profile>
You can then use ${activatedProfile}.properties as value for a bean
<bean id="xyz" class="abc.xyz">
<property name="propertyFile" value="${activatedProfile}.properties" />
</bean>

Using custom reporters with the maven surefire plugin

I'm trying to use a custom reporter for TestNG with the maven surefire plugin. I already have a custom listener and that seems to get picked up. However, it doesn't look like the custom reporter is being used at all:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<properties>
<property>
<name>listener</name>
<value>com.mystuff.test.TestNGListener</value>
</property>
<property>
<name>reporter</name>
<value>com.mystuff.test.MyEmailableReporter</value>
</property>
</properties>
</configuration>
</plugin>
Any idea why this is happening?
I figured this out. It looked like the following doesn't work at all:
<property>
<name>reporter</name>
<value>com.mystuff.test.MyEmailableReporter</value>
</property>
in spite of documentation to the contrary. In the TestNG class it appears that there is a method called TestNG#addListener(IReporter listener), which contrary to its name, accepts a report which implements IReporter. Maven Surefire (v2.12.1) calls this method to add listeners and reports. However, it doesn't look for reports under a property with name reporter. Instead, you have to add your custom reporter to the listener property:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<properties>
<property>
<name>listener</name>
<value>com.mystuff.test.TestNGListener,com.mystuff.test.MyEmailableReporter</value>
</property>
</properties>
</configuration>
</plugin>
Not very intuitive.

Resources