How to run junit tests when jetty is configured to start/end at pre/post-integration-test? - maven

I want to run junit tests, while jetty is running , how to correctly configure this?
Now it seems that the tests run before jetty is started so they can't get a connection and the project build fails.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.7.1</version>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>8.1.10.v20130312</version>
<configuration>
<connectors>
<connector implementation="org.eclipse.jetty.server.bio.SocketConnector">
<port>9090</port>
<maxIdleTime>60000</maxIdleTime>
</connector>
</connectors>
<scanIntervalSeconds>10</scanIntervalSeconds>
<stopKey>foo</stopKey>
<stopPort>9999</stopPort>
<jvmArgs>-Xms1024m -Xmx2048m</jvmArgs>
</configuration>
<executions>
<execution>
<id>start-jetty</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
<configuration>
<scanIntervalSeconds>0</scanIntervalSeconds>
<daemon>true</daemon>
</configuration>
</execution>
<execution>
<id>stop-jetty</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit4</artifactId>
<version>2.16</version>
</dependency>
</dependencies>
</plugin>
Please help me with this! How to ensure that the junit tests (during the maven build) run in the integration-phase?

The Maven test phase is for running 'Unit' tests, that is, small tests that check that the smallest possible functionality or 'Unit' is functioning as expected.
These are run by the Maven Surefire plugin. You can find more info on this here:
http://maven.apache.org/surefire/maven-surefire-plugin/
What you have are integration tests. That is, they are broader tests that check that different parts of your code integrates together. This is run by the Maven Failsafe plugin which you can find more details on here:
http://maven.apache.org/surefire/maven-failsafe-plugin/
What you need to do is identify your integration tests as integration tests to the Failsafe plugin. By default, it picks up classes named **/IT*.java, **/*IT.java, and **/*ITCase.java. The naming conventions between the plugins is mutually exclusive by default so your tests should only run in one or the other.
If you only have integration tests then you can configure Surefire to run during the integration-test phase instead. Instructions on doing that here:
http://docs.codehaus.org/pages/viewpage.action?pageId=62120

Related

SonarQube coverage is always 0

I have a project that uses SonarQube to analyse my code. As the title suggests, the coverage metric is always appears to be 0 in the SQ report (on my server) despite me having multiple Junit tests. I currently run the following command
clean package deploy sonar:sonar -Dsonar.projectKey=SomeName -Dmaven.test.skip=true
when building my project in Jenkins (Jenkins Maven project).
I had similar issue some time back with different project. I manage to resolved it via this article. However, it didn't help this time. After searching for a bit, I found article 1 and article 2 (as well as some more with similar ideas). Both had some good suggestions but unfortunately nothing worked.
I noticed that I also get the following warning every time (don't know what causes it)
Cobertura report not found at /var/lib/jenkins/jobs/SomeProjectName/workspace/SomeFolderName/target/site/cobertura/coverage.xml
At first, I tried to add cobertura plugin (as suggested here) but that just removed the warning (coverage remained 0 in the report). My current understanding is that it overwrites Jacoco but I failed to find the reason why or solution to it.
My plugins:
<build>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<verbose>true</verbose>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<finalName>uber-${project.artifactId}-${project.version}</finalName>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.9.2</version>
<executions>
<execution>
<configuration>
<outputDirectory>${project.basedir}/target/generated-sources</outputDirectory>
<sourceDirectory>${project.basedir}/src/main/resources/</sourceDirectory>
</configuration>
<goals>
<goal>schema</goal>
<goal>protocol</goal>
<goal>idl-protocol</goal>
</goals>
<id>schemas</id>
<phase>generate-sources</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
My properties:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>11</java.version>
</properties>
There are a few things that have to be set correctly to result in your test coverage being shown in SonarQube. Some of these settings have defaults that may work, but I try to avoid magic defaults, in favor of ensuring that my settings agree with each other.
Surefire and Jacoco have to have settings that agree with each other, and SonarQube has to know where to find the coverage data file.
I'm going to suggest some property values for you to use.
sonar.coverage.jacoco.xmlReportPaths: ${basedir}/target/jacoco_report/jacoco.xml
jacoco.path: ${basedir}/target/jacoco_report
Then, here is an excerpt for your jacoco plugin:
<!-- Prepares the property pointing to the JaCoCo runtime agent which
is passed as VM argument when Maven the Surefire plugin is executed. -->
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!-- Ensures that the code coverage report for unit tests is created
after unit tests have been run. -->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${jacoco.path}</outputDirectory>
</configuration>
</execution>
Finally, this goes in the configuration block of the Surefire plugin:
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
However, in your case you have a very high-level issue that would result in zero coverage even if you had all of these settings done correctly. Thanks to #Ian W for pointing this out.
When I saw you write "despite me having multiple Junit tests", I took your word for it that you're actually executing the unit tests. You have "-Dmaven.test.skip=true" on your command line, which causes it to not execute any unit tests. Remove that.
I am using java 11 with maven version 3.8.3 configured with jacoco 0.8.3 plugin.
the Code coverage is operational. SonadeQuebe dont't dispaly 0%
command:
mvn sonar:sonar -Dsonar.projectKey=diag -Dsonar.host.url=http://sonar.XXXXX/sonar -Dsonar.login=xxxx -Dsonar.password=xxxxx
In pom.xml
<properties>
<java.version>11</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<jacoco-maven-plugin.version>0.8.3</jacoco-maven-plugin.version>
<sonar-maven-plugin.version>3.4.0.905</sonar-maven-plugin.version>
<sonar.java.codeCoveragePlugin>jacoco</sonar.java.codeCoveragePlugin>
<sonar.tests>${project.basedir}/src/test/</sonar.tests>
...
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco-maven-plugin.version}</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>

Integration tests create files for the packaged jar

I'm generating spring-restdocs snippets with rest assured in my integration tests.
With maven and the failsafe plugin that defaults to the integration-test and verify phase. Problem is, that asciidoctor-maven-plugin (to generate the final HTML file out of the snippets) and maven-resources-plugin (to copy the final HTML file into the correct place) run before the integration tests in prepare-package.
With that asciidoctor naturally fails, because the snippets are not generated yet.
If I configure asciidoctor to run at post-integration-test, it succeeds, but then the finished HTML page is not in my jar, cause the jar was already created in the package phase.
So I feel like the only option is to run my integration tests already in the tests phase (probably with surefire instead of failsafe).
I could also split out the Documentation related tests from the rest of the integration tests, but I actually like to have them in the seemingly correct place.
I wonder if there is a best practice I'm ignoring?
Should integration tests never produce something to put inside the jar?
It looks to me, that integration tests in maven are meant for cross-jar tests (hence running them after package). And not like I'm using them, just for bigger tests that involve multiple parts (especially DB) all within one jar.
What I would like:
run all tests
compile documentation
package everything into the jar
Excerpt from my pom.xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<skip>${skipSurefire}</skip>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/*IT.java</include>
<include>**/*E2ET.java</include>
</includes>
</configuration>
</plugin>
<!-- Compile API documentation -->
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>1.5.8</version>
<executions>
<execution>
<id>generate-docs</id>
<phase>prepare-package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<backend>html</backend>
<doctype>book</doctype>
</configuration>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.springframework.restdocs</groupId>
<artifactId>spring-restdocs-asciidoctor</artifactId>
<version>${spring-restdocs.version}</version>
</dependency>
</dependencies>
</plugin>
<!-- Package API documentation -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.outputDirectory}/public/docs
</outputDirectory>
<resources>
<resource>
<directory>
${project.build.directory}/generated-docs
</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
The phases integration-test and verify are just the default bindings of the integration-test and the verify goal. You can bind your integration tests to the test phase, as well:
...
<artifactId>maven-failsafe-plugin</artifactId>
...
<executions>
<execution>
<id>integration-tests</id>
<phase>test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
and such achieve the order you like/need.

Running spock integrations tests in its own folder in maven

I am trying to only run integration tests in maven with mvn. I have my tests in src/it/java for java integration tests and src/it/groovy for Spock integration tests. When I run mvn failsafe:integration-test, only the java integration tests in src/it/java are run.
This is my pom:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-integration-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/it/java</source>
<source>src/it/groovy</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<testSourceDirectory>src/it/groovy</testSourceDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.6</version>
<configuration>
<testSources>
<testSource>
<directory>src/it/groovy</directory>
<includes>
<include>**/*IT.groovy</include>
</includes>
</testSource>
</testSources>
</configuration>
<executions>
<execution>
<id>groovy-compile</id>
<goals>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
Any idea what I am missing here?
It looks like the issue is that your running only failsafe:integration-test. I am able to run groovy and java tests using the command:
mvn verify
In the failsafe docs it suggests to use the verify phase here - http://maven.apache.org/surefire/maven-failsafe-plugin/index.html
If mvn verify still doesn't run your groovy tests, check your_project/target/test-classes/path_to_your_test for expected class files and make sure that lines up with the includes option in the pom.

maven-failsafe-plugin is not executing my integration tests

Bellow a part of my actual pom.
Testng tests for integration tests have been assigned a "integration" group in the #Test annotations.
To do little test I did not exclude the "integration" group during the test phase.
When building using for example mvn verify or mvn install the integration tests get executed in the test phase, but not the verify or integration-test phase.
The number of tests run remains 0. Somehow they are not picked up. Anyone have an idea on what might be wrong?
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.1</version>
<configuration>
<skip>false</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.1</version>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</execution>
</executions>
<configuration>
<skip>false</skip>
<excludedGroups>unit</excludedGroups>
</configuration>
</plugin>
Update: Adding TestNG as a dependency to the failsafe plugin does not
help
maven-failsafe-plugin by default includes only files matching following patterns:
<includes>
<include>**/IT*.java</include>
<include>**/*IT.java</include>
<include>**/*ITCase.java</include>
</includes>
How did you name your test class(es)?

JMS integration tests with Maven and Glassfish

I'm developing an app requiring MDB, running over Glassfish 3.1. I've managed already to run unit/integration tests of simple EJBs using embedded container with no problem. Now I'm trying to create integration tests for my MDBs.
1) I tried launching the Glassfish embedded server programatically, but it does not support creation of JMS queues.
2) I run a Glassfish server from the Maven plugin.
Now I can create queues, and deploy my MDBs, no problem at all. Now, I just can't figure out a way of running JUnit.
- When I create an InitialContext, it times-out when accessing the local server. I have no ways of accessing my beans.
I found a workaround, but it's not serving my needs perfectly:
In my test sources, I created a simple Singleton #Startup bean. In the #PostConstruct method, I call the unit test classes I want to achieve. In order for this bean to be deployed, I have a special special maven build rule that packages some of my tests files in the EJB jar. Deploying this special jar results in my tests being launch. To make it clear, here's an extract of my Maven file:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<phase>pre-integration-test</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/test-ejb</outputDirectory>
<resources>
<resource>
<directory>${project.build.directory}/classes</directory>
</resource>
<resource>
<directory>${project.build.directory}/test-classes</directory>
<includes>
<include>**/*TestTrigger.class</include>
<include>**/*IntegrationTest.class</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-ejb-plugin</artifactId>
<version>2.3</version>
<configuration>
<ejbVersion>3.1</ejbVersion>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>ejb</goal>
</goals>
<configuration>
<classifier>TEST</classifier>
<outputDirectory>${project.build.directory}/test-ejb</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.glassfish</groupId>
<artifactId>maven-embedded-glassfish-plugin</artifactId>
<version>${glassfish.version}</version>
<configuration>
<goalPrefix>glassfish</goalPrefix>
<app>target/${project.build.finalName}-TEST.jar</app>
<port>8080</port>
<name>MyApp</name>
<serverID>embedded</serverID>
</configuration>
<executions>
<execution>
<id>start</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>admin</id>
<phase>pre-integration-test</phase>
<goals>
<goal>admin</goal>
</goals>
<configuration>
<commands>
<param>create-jms-resource --restype javax.jms.QueueConnectionFactory jms/TestQueueConnectionFactory</param>
<param>create-jms-resource --restype javax.jms.Queue --property imqDestinationName=ceQueue jms/ceQueue</param>
</commands>
</configuration>
</execution>
<execution>
<id>deploy</id>
<phase>pre-integration-test</phase>
<goals>
<goal>deploy</goal>
</goals>
</execution>
<execution>
<id>stop</id>
<phase>post-integration-test</phase>
<goals>
<goal>undeploy</goal>
<goal>stop</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
</dependency>
</dependencies>
</plugin>
</plugins>
Now, is there any way my IntegrationTest can be launched using surfire, in order to produce a proper report and fail build if test don't pass? Not to mention Cobertura.
Thank you for your help.
I didn't solve my problem so to say. What I had to do is upgrade to GlassFish 3.1.1. This version supports JMS in embedded mode. Therefore I can run a server programatically and deploy the queues I want using the admin command runner.
I just lost a bit of time trying to name my connection factory jms/connectionFactory where the jms/ prefix was unnecessary and was failing all lookups.
I also had to include a thread sleep in all my unit tests or else the server would close before tests are over.

Resources