IT Code Coverage for multi-module Maven projects that have differing parents - maven

I've spent the day yesterday up and down StackOverflow and Google and have come close to some solutions, but nothing is working. I hope someone could just tell me whether this is even possible.
We have a multi-module Maven project. I just learned that the structure is like this:
Web Service XYX
+- pom.xml
+- Web Module
+- pom.xml (parent pom is WebService 123)
+- API Module
+- pom.xml (parent pom is API ABC)
+- API Implementation
+- pom.xml (parent pom is API ABC)
What I have done is setup UT and IT Code Coverage from Jacoco to Sonar via Jenkins. The UT Coverage gets sent to Sonar correctly for the UT combined.
The IT Coverage is always 0%. My IT is run against the Web Module after starting the WAR up using the Maven Tomcat plugin. The code that it should be exercising lives inside the module API Implementation. Even though a jacoco-it.exec file was generated and analyzed by Sonar, it always shows me that no lines of code were hit. Is there a way to actually do this?
BTW - I know it can be done because this article indicates exactly what I need:
http://www.sonarqube.org/measure-code-coverage-by-integration-tests-with-sonar/
However, the article fails to mention if the modules are referred to the same parent in the project. I'm making an assumption that it does.

After searching up and down the interwebs, the best solution seem to have been add another module for ITs that actually referred to the main pom as the parent. Used Cargo to grab and run the WAR from the IT module using the Tomcat Maven Module. The Code Coverage now has all the classes, not just the ones from the Web Module.

I doubt it's due to using different parent poms.
When analyzing the coverage sonar checks the code of each module against the coverage file that is specified in the sonar.jacoco.itReportPath property. The default is target/jacoco-it.exec. So when analyzing WebModule it checks for coverage info in WebModule/target/jacoco-it.exec.
So what I do for my projects, is to use a central file in the root module for the the IT coverage data instead.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.4.201502262128</version>
<executions>
<execution>
<id>prepare-it-agent</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
<append>true</append>
<propertyName>failsafeArgLine</propertyName>
</configuration>
</execution>
</executions>
</plugin>
The ${session.executionRootDirectory} property is the root of execution, in your case the WebServiceXYZ if you run the build of that. This also works if you have multi-module with more than one level of nesting.
Now you need to point sonar to use that file when analyzing IT coverage. So you have to set the sonar.jacoco.itReportPath to that file. Unfortunately, this does not work with the session.executionRootDirectory property and you have to set the absolute path to the file manually. I do not recommend to specify the absolute path in the pom.xml as this path is specific to your build environment. So either set the path in Sonar or as System property of your build environment. I set it directly in the Sonar Project Settings (Java > Jacoco), for example /opt/buildroot/myProject/target/jacoco-it.exec.
Now sonar will check that file for the IT coverage analysis of each module.

Related

Serenity BDD Multi-Module JUnit Screenplay Report Aggregation Problems

I am setting up a multi-module maven project to create UI tests for a large browser based application using junit and the screenplay bdd pattern.
Each module works and reports as expected individually, however, if I try to run the tests at the top level of the project then - the tests run, but the report doesn't include ALL of the child modules.
I get a report with details from only 1 of the modules.
I have created a simple test example as follows;
The "simple" Maven project was created in Eclipse and the module test projects were created with
mvn archetype:generate -Dfilter=screenplay
and modified the tests to be different on each one.
The pom was updated to use serenity version 2.0.81
I have the following in the poms at all levels in the project
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>${serenity.version}</version>
<executions>
<execution>
<id>serenity-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
Project folder structure is;
UITesting
pom.xml
serenity.properties
demo_one
src
test
java
resources
uk / co / test /
features
feature_folder_one
package-info.java
StoryClassOne.java
tasks
ui
pom.xml
serenity.properties
demo_two
src
test
java
resources
uk / co / test /
features
feature_folder_two
package-info.java
StoryClassTwo.java
tasks
ui
pom.xml
serenity.properties
and same for demo 3
and I set the output directory in the each level serenity.properties file
serenity.outputDirectory=c:/temp/uireport
I want the Requirements, Features and Stories tabs to consolidate all of the child module report data. The actual tests and tags are consolidated, but nothing else is.
some feedback on this from serenity core github issues states;
This won't happen automatically - multi-module tests aren't supported in this way.

maven surefire reports in default lifecyle

How can i generate surefire reports in the maven's default lifecycle. We have jobs setup in teamcity with goals as
mvn clean install -Pprod
This job runs all the junits, i want to create HTML report of all the tests running in the project. I came across the sure-fire-reports plugin but it generates the report only in site phase it does not generates the report during the default clean install phase.
Can one please help how can i generate report default lifecycle
I tried including the surefire-reports plugin, in test phase as below but doesnot not works
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>report-only</goal>
</goals>
</execution>
</executions>
</plugin>
If you need the very report generated by maven-surefire-report-plugin, I see no other way than execute mvn site, because a report is executed only within the site phase of the build lifecycle.
Instead, if you just need an HTML-readable report, I'd suggest you this walk-around:
Take advantage of the XML files generated by the maven-surefire-plugin in the target/surefire-reports directory.
Code your own transformation sheet (XSL) to transform them to the desired HTML format.
In the pom, set a transformation in the next phase (for example, prepare-package) through the xml-maven-plugin.
If you put the XSL in the parent project and set this transformation in the parent pom, all the submodule projects should inherit it and produce the HTML reports during the corresponding build.
And last: How to browse the child HTML reports from the parent project? Hum... I'd say to code an Ant script to browse all the submodules and list the HTML files and produce an HTML index with them. This script should be executed only from the parent project.

autoweaving AspectJ fails during unit tests in IntelliJ

We have a series of unit tests and they were passing fine prior to me trying to add some aspects for dependency injection and logging duration of methods being called in our rest end points.
In the unit tests prior to the tests failing, we get two odd errors:
[AppClassLoader#14dad5dc] error aspect 'com.lutherconsulting.aphirm.log.DurationLoggingAspect' woven into 'com.lutherconsulting.aphirm.rest.ClientRest' must be defined to the weaver (placed on the aspectpath, or defined in an aop.xml file if using LTW).
and
[AppClassLoader#14dad5dc] error aspect 'com.lutherconsulting.aphirm.log.DurationLoggingAspect' woven into 'com.lutherconsulting.aphirm.log.DurationLoggingAspect' must be defined to the weaver (placed on the aspectpath, or defined in an aop.xml file if using LTW).
We are using the aspectj maven plugin to just let it autoweave the aspects into the web application. The configuration for that from our pom.xml for Maven is below.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<version>1.8</version>
<configuration>
<showWeaveInfo>true</showWeaveInfo>
<complianceLevel>1.8</complianceLevel>
<source>1.8</source>
<target>1.8</target>
<Xlint>ignore</Xlint>
<encoding>UTF-8</encoding>
<verbose>true</verbose>
</configuration>
<executions>
<execution>
<goals>
<goal>test-compile</goal>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
The strange thing is that this all works perfectly fine if I build the war file and deploy it to a Tomcat instance, or if I run all of our cucumber feature tests. When I do either of those, The aspect weaves fine and I get data on the duration of the rest methods I annotated logged to a database correctly. If I run a specific test package from intelliJ or try to run all junit tests in intellij, it fails with those two errors
Is this something I'm just missing in Intellij as a run/debug configuration in the way it executes unit tests? I didn't think our structure of our app was different than any normal web app
- src
| - main
| - java
| - packages
| - resources
| - test
| - java
| - packages
| - resources
- pom.xml
I appreciate any ideas on
In the end, what turned out to fix this was to go into the project structure in IntelliJ, on the AspectJ settings there is a check box for Post-Compile Weave Mode. Checking this made sure weaving occurred in Intellij prior to the tests executing.
As far as I understand you are running your tests from IntelliJ using its own runner and not maven.
Therefore you have configured the weaver to be run using maven through the aspectj-maven-plugin. The problem is that your IntelliJ runner is not running maven, hence its weaver plugin is not being run either.
I can come up with an idea and you could run your maven test goal within IntelliJ to run all your tests with the maven configuration, so it will detect aspectj-maven-plugin and run the weaver too. Here you can check how to run maven goals:
https://www.jetbrains.com/idea/help/executing-maven-goal.html
On the other hand, according to this link you have to enable Load Time Weaving in IntelliJ
http://www.aspectprogrammer.org/blogs/adrian/2006/02/a_practical_gui_2.html
Quoting the link it says:
Open the "Run/Debug Configurations" dialog using the drop-down in the
toolbar. Click the "+" icon to create a new configuration and name it
e.g. "tests".
For this project, I've selected "All in package" and search for tests
"In whole project".
Now all you have to do is add the VM startup parameter that brings in
the AspectJ LTW agent:
-javaagent:lib/aspectjweaver.jar
The part after the ":" should be the path to your copy of aspectjweaver.jar. In this case, I've copied the
aspectjweaver.jar from the Spring distribution into the lib directory
of my project (it doesnt' need to be on the project's classpath). You
can use the jar from AspectJ 5 final release too if you want to.
Also, you can check to configure AspectJ facet, check this link to read about it
https://www.jetbrains.com/idea/help/aspectj.html

Sonar configuration in multimodule maven project using tycho for unit tests and jacoco for coverage

We're using maven to run a sonar analysis and it works well except for the code coverage results with jacoco. We have an eclipse project that uses tycho-surefire-plugin for testing. I've not overriden the argLines properties so solutions involving that line may not be appropiate.
Facts :
Maven structure structure:
parent
master
module 1
module ...
module n
Testing structure:
client.admin (eclipse-plugin packaging)
client.admin.test.fragment (eclipse-test-plugin packaging)
Properties that are correctly set and identified
sonar.junit.reportsPath
sonar.jacoco.reportPath,
sonar.jacoco.itReportPath
sonar.core.codeCoveragePlugin
sonar.language
The main problem is with the following properties
sonar.test
sonar.sources
sonar.java.binaries
As seen in the Testing structure in the client.admin.test.fragment tests are contained in the /src folder and the sources are located in the project client.admin in the /src folder too.
When we run the analysis we get the following error :
[WARN] Coverage information was not collected. Perhaps you forget to include
debug information into compiled classes?
I believe this has to do with the properties sonar.java.binaries that goes looking for the sources in target/classes of the fragment project (client.admin.project) that are in fact located in the host project (client.admin). In the fragment project we've configured sonar.tests and sonar.sources properties so that they call the /src folder of the corresponding projects.
In the sonar Analysis Parameters page there says that only sonar.sources is a maven valid property, sonar.tests and sonar.java.binaries cannot apparently be configured in maven. How then could I attach the binaries to the project. I've tried copying the folder target/classes from the host project but I got the same message. Is there any workaround in maven ?
Edit 1
There is one jacoco.exec file that is generated for the whole project that can be found at the parent folder. This was done configuring the jacoco.destFile and sonar.jacoco.reportPath properties
Jacoco plugin in main pom :
<!-- Jacoco Plugin -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<destFile>${sonar.jacoco.reportPath}</destFile>
<append>true</append>
</configuration>
</execution>
<execution>
<id>default-report</id>
<goals>
<goal>report</goal>
</goals>
<configuration>
<dataFile>${sonar.jacoco.reportPath}</dataFile>
<outputDirectory>${jacoco.reports.outputDirectory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
Maven Plugin Versions:
sonar: 2.4
jacoco: 0.7.1.201405082137
Properties
<sonar.language>java</sonar.language>
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
<sonar.junit.reportsPath>${project.build.directory}/surefire-reports/</sonar.junit.reportsPath>
<sonar.jacoco.reportPath>${basedir}/../../../main/**.master/target/jacoco.exec</sonar.jacoco.reportPath>
<jacoco.reports.outputDirectory>${basedir}/../../../main/**.master/target/site/jacoco</jacoco.reports.outputDirectory>
<sonar.sources>src</sonar.sources>
In the test projects (eclipse-test-plugin) we changed added the property sonar.sources to go find the sources from the src folder of the project that we're testing for example in client.admin.test.fragment we go search the src from the client.admin
The following properties were commented in code because they're not supported in maven according to documentation and to the debug output.
<!--<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>-->
<!--<sonar.tests></sonar.tests>-->
<!--<sonar.java.binaries></sonar.java.binaries>-->
First, you must tell the JaCoCo agent to report all coverage data into one common file. Second, you tell the Sonar JaCoCo plugin to read the coverage data from the aggregated file.
To do so, set the properties "jacoco.destFile" and "sonar.jacoco.reportPath" in your parent pom.xml to the same absolute path, e.g.:
<properties>
<jacoco.destFile>/home/jenkins/jobs/my.project/workspace/parent/target/jacoco.exec</jacoco.destFile>
<sonar.jacoco.reportPath>/home/jenkins/jobs/my.project/workspace/parent/target/jacoco.exec</sonar.jacoco.reportPath>
</properties>
Note that these properties will be inherited to all child poms, so you can't use Maven expressions like ${project.build.directory} because this would evaluate to a different directory for each pom.
You could create a small helper Mojo which automatically resolves an absolute path on the current build machine and then injects the properties into the Maven model.

How to use maven to publish multiple artifacts of an ivy project(with multiple modules) to a maven repository(nexus)

I'm working on a complex multi-module open source ivy project, which has ant's build.xml at the top level to kick off each ivy module's build. But the goal here is not to modify the original build scripts(both ivy.xml and build.xml), and using maven as an outer layer to kick off ant build, and then fetch the built results and publish them to nexus server.
The difficulty here is that, the built artifacts here are multiple jars, and we need to publish all these jars to nexus server with maven. Since one pom.xml only maps one maven artifafct, and in this case multiple artifacts are build not through maven but ivy. So I wonder if there's a feasible way to achieve my goal.
Currently, in the top level pom.xml, I'm using maven-antrun-plugin to invoke build.xml on top level, and using build-helper-maven-plugin to attache artifacts, but it doesont' work.
Currently I'm working on a similar task to yours. We have a huge, full of legacy system with whole build written in ant. That is how we handle this task:
No matter what, you will have to accept it, maven = jar per artifact (well, you can use attachments with qualifiers, but it's a real abuse and highly NOT recommended). It has it's philosophy after it: in the end of the day your system consists of (as you said yourself) modules, so each module has to have it's version, sources and (most important) the dependencies to other modules.
To reuse the existing ant code you can look on the antrun plugin. What we did, is "simply" separated all the common build code (i.e generators execution, attachments creation, assemblies and so on) to parent poms that are of type "pom". Then we execute the relevant targets simply by activating properties in children poms. Here is an example
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>EXECUTION_NAME</id>
<phase>generate-sources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target if="EXECUTION_TRIGGER_PROPERTY">
<taskdef resource="net/sf/antcontrib/antlib.xml"/>
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
And in the child pom we simply define
<properties>
<EXECUTION_TRIGGER_PROPERTY>true</EXECUTION_TRIGGER_PROPERTY>
</properties>
remember to look at maven lifecycle guide to choose the proper phase for your execution.
Don't forget that you can use maven plugins to make things easier. E.g instead of running <javac> task in ant, breaking to artifacts with jar type do all the compile for you. You can also find plugins that generate javadoc, jaxws and so on.
As you can see it's not that simple to make your system work with maven. It will require you to rethink how your build works. On the other hand the ability to see and understand your dependencies, the ease of working in modern IDE's, binary repositories and so on are worth it in most of the cases.

Resources