Not able to run a category in Junit4 tests - maven

Have a test method annotated with category:
public interface BVT {} //is placed in package net.test.categories, file name BVT.java
public class TestClass{
#Test
#Category(BVT.class)
public void someTest(){
System.out.println("smoke");
} }
I use Junit 4.12 and surefire 3.0.0-M3
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>${threadCount}</forkCount>
<reuseForks>false</reuseForks>
<skip>false</skip>
<groups>${testGroup}</groups>
</configuration>
</plugin>
If I try to run test from BVT category
mvn clean test -DtestGroup="net.test.categories.BVT"
I get
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
Have no ideas why the test was not run/skipped...
Using of created in pom profile didn't help - it just runs all the tests.
Adding this runner dependency also doesn't help:
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>3.0.0-M3</version>
</dependency>
</dependencies>
Creating a Suite runs the needed test but I don't want to list all required classes in this suite class, just want to run a certain group from command line

From the documentation:
You can use JUnit categories by using the groups parameter.
Using mvn -D... sets a system property, not a plugin parameter.
So you need something like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
...
<groups>${myGroup}</groups>
</configuration>
</plugin>
And run it with mvn ... -DmyGroup="net.test.categories.BVT"

Didn't find reason, but adding include section in surefire configuration solved the issue.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>${threadCount}</forkCount>
<reuseForks>false</reuseForks>
<skip>false</skip>
<groups>${testGroup}</groups>
<includes>
<include>*</include>
</includes>
</configuration>
</plugin>

Related

Always getting "Tests run: 0, Failures: 0, Errors: 0, "even though testng test got executed using "mvn clean test -Dsurefire.suiteXmlFiles=testng.xml"

I am expecting count for Total tests, Passed, Failed should show on console.
I am using latest Testng version 7.4.0. and 2.22.0 surefire version.
I have tried every possible solution but didn't work for me.
Please find my code for maven surefire plugin.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${surefire.suiteXmlFile}</suiteXmlFile>
</suiteXmlFiles>
<includes>
<include>*.*</include>
</includes>
<testFailureIgnore>true</testFailureIgnore>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<reportsDirectory>target/test-report</reportsDirectory>
</configuration>
<dependencies>
</dependencies>
</plugin>
Please let me know if anyone knows any solution.

Running Selenium(TestNG) tests from Azure Devops

I'm working on running my automation test scripts (which is a Maven project
based on java and using test ng ) in AzureDevops CI/CD pipeline. I have pushed all my code to Github and created a Maven task to run the tests.
But when I try to run the Maven task , it is not considering testng.xml, where I have all the tests configurations like which tests to be run(order of the tests) and other configuration stuff like listener to create some custom reports. It is just running all the test classes present in my complete Maven project.
Please help me which config / task I should in add in my build inorder for the test to run based on my testng.xml
Below is the Pom.xml
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
TestAutomation
Test
0.0.1-SNAPSHOT
jar
Test
http://maven.apache.org
org.seleniumhq.selenium
selenium-java
3.14.0
org.testng
testng
6.11
compile
org.apache.poi
poi
3.16-beta2
com.relevantcodes
extentreports
2.41.2
commons-io
commons-io
2.6
log4j
log4j
1.2.17
com.googlecode.json-simple
json-simple
1.1.1
<!-- for gson https://mvnrepository.com/artifact/com.google.code.gson/gson -->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>
<properties>
<property>
<name>listener</name>
<value>com.qa.ExtentReportListner.ExtentRerportListener</value>
</property>
</properties>
</configuration>
</plugin>
</plugins>
</build>

NoClassDefFoundError when attempting to run unit tests when building Jenkins plugin

I have a custom Jenkins plugin that I've built. I have some test cases that leverage some code out of the jenkins-test-harness project (namely the Junit JenkinsRule.)
Anyway, the unit tests run and pass when I run them in IntelliJ. No exceptions, no errors.
When I try to run them from the Maven commandline, however, all of the tests that rely on the JenkinsRule fail:
[ERROR] testGetLastDate(com.mycompany.myplugin.portlet.utils.UtilsHudsonTest) Time elapsed: 0 s <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.hudson.test.TestPluginManager
at org.jvnet.hudson.test.JenkinsRule.<init>(JenkinsRule.java:325)
at com.mycompany.myplugin.portlet.utils.UtilsHudsonTest.<init>(UtilsHudsonTest.java:19)
That TestPluginManager is in the jenkins-test-harness jar. The JenkinsRule that attempts to call it is right next to it in the same package in the same jar, so clearly the jar itself is successfully on the classpath.
I can't quite figure out why I can't run this job from the commandline using mvn test.
My POM is very simple -- I declare a Parent on the Jenkins plugin-3.2 pom:
<parent>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>plugin</artifactId>
<version>3.2</version>
<relativePath/>
</parent>
And I have a dependency on the test harness:
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>org.jenkins-ci.plugins</groupId>
<artifactId>dashboard-view</artifactId>
<version>${dashboard-view.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.jenkins-ci.main</groupId>
<artifactId>jenkins-test-harness</artifactId>
<scope>test</scope>
<version>2.34</version>
</dependency>
</dependencies>
(That's the entire dependencies section, no omissions.)
And in my build section, I only declare the compiler, surefire, and hpi plugins:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
</plugin>
<plugin>
<groupId>org.jenkins-ci.tools</groupId>
<artifactId>maven-hpi-plugin</artifactId>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>hpi</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
</plugins>
</build>
(The surefire config is inherited from the plugin-3.2 parent.)
Here's the bit of the UtilHudsonTest that causes the error -- line 19 is the #Rule annotation:
public class UtilsHudsonTest {
#Rule // This is line 19
public JenkinsRule j = new JenkinsRule();
#Test
public void testGetLastDate() throws Exception {
FreeStyleProject prj = j.createFreeStyleProject("prj1");
prj.scheduleBuild2(0).get();
FreeStyleProject prj2 = j.createFreeStyleProject("prj2");
prj2.scheduleBuild2(0).get();
List<Job> jobs = new ArrayList<>();
jobs.add(prj);
jobs.add(prj2);
LocalDate lastDate = Utils.getLastDate(jobs);
assertNotNull(lastDate);
}
// other tests
}
I'm kind of stumped why this would work in IntelliJ and not in Maven. A mvn dependency:tree shows that the correct and expected dependencies. Any ideas in how to get this plugin to successfully run its test from commandline?

Maven not outputting junit CHECKSTYLE report

After default config, maven is only outputting the Checkstyle report for source classes, not Junit test classes. Note I'm not looking for the surefire test report output.
from pom.xml:
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>2.17</version>
<reportSets>
<reportSet>
<reports>
<report>checkstyle</report>
</reports>
</reportSet>
</reportSets>
<configuration>
<configLocation>src/main/resources/apcheckstyle.xml</configLocation>
<module name="LineLength">
<property name="max" value="120"/>
</module>
</configuration>
</plugin>
</plugins>
I changed dependency scope for junit from test to compile:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>compile</scope>
</dependency>
Maven is configured to generate the checkstyle reports using the mvn site plugin.
I tried adding a user property includeTestResources as true, but this did not help. The documentation states that the default value is true, so I took it out.
Here is the maven output during checkstyle generation.
from javadocs:
Generating C:\Users\Administrator\Projects\lht657aws\target\site\testapidocs\help-doc.html...
checkstyle plugin runs:
[INFO] Generating "Checkstyle" report --- maven-checkstyle-plugin:2.17
[INFO] There are 210 errors reported by Checkstyle 6.11.2 with src/main/resources/apcheckstyle.xml ruleset.
is following the problem-? all the java under src is getting processed, just java under test is being skipped:
[WARNING] Unable to locate Source XRef to link to - DISABLED
after checkstyle, maven project info report starts
[INFO] Generating "Dependencies" report --- maven-project-info-reports-plugin:2.9
Add the Maven Surefire Report plugin to your pom XML
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<outputDirectory>path/to/the/report</outputDirectory>
</configuration>
</plugin>
And add the goal surefire-report:report to your maven call
See the documentation here: Maven Surefire Report Plugin
EDIT
I've misunderstood the question, try to add this section with desired configurations to your pom.
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>2.17</version>
<reportSets>
<reportSet>
<id>main_checks</id>
<reports>
<report>checkstyle</report>
</reports>
<configuration>
<sourceDirectory>${project.build.sourceDirectory}</sourceDirectory>
<includeTestSourceDirectory>false</includeTestSourceDirectory>
<configLocation>config/maven_checks.xml</configLocation>
<outputDirectory>${project.reporting.outputDirectory}/main-checks/</outputDirectory>
</configuration>
</reportSet>
<reportSet>
<id>test_checks</id>
<reports>
<report>checkstyle</report>
</reports>
<configuration>
<sourceDirectory></sourceDirectory>
<testSourceDirectory>${project.build.testSourceDirectory}</testSourceDirectory>
<includeTestSourceDirectory>true</includeTestSourceDirectory>
<configLocation>config/sun_checks.xml</configLocation>
<outputDirectory>${project.reporting.outputDirectory}/test-checks/</outputDirectory>
</configuration>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
Maven sure fire plugin is used for generating the Test reports. See the additional documentation here:
http://maven.apache.org/surefire/maven-surefire-report-plugin/
You need to include this plugin in your POM.xml and provide path to create reports (generally a directory in project). While making the build make sure you are including goal for this.

Maven failsafe plugin will not run test classes annotated with JUnit Category

I have an interface like this:
public interface IntegrationTest {
}
I configure the failsafe plugin like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14</version>
<configuration>
<groups>acme.test.IntegrationTest</groups>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
</executions>
</plugin>
If I then create an integration test like this
#Category(IntegrationTest.class)
public class ExampleClassIntegrationTest {
#Test
public void slow_and_painful_test() {
This test will not run.
If I however name the class according to the Inclusions and Exclusions of Tests
**/IT*.java
**/*IT.java
**/*ITCase.java
Like so:
#Category(IntegrationTest.class)
public class ExampleClassIT {
#Test
public void slow_and_painful_test() {
The test runs fine. Why do I have to name the test AND have an annotation when i use the groups-tag? Am I missing something? The documentation on using JUnit states that you can use the Category annotation at the class level.
Thats because these are the default java classes which fail safe plugin includes when executed.
You can however over ride this in your pom with tag :
E.g
<includes>
<include>**/*.java</include>
</includes>
To include all the java files.
You should either add JUnit as a dependency (>4.8) which is already done or in particular add the following to your failsafe-plugin configuration:
<plugins>
[...]
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.14.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.14.1</version>
</dependency>
</dependencies>
</plugin>
[...]
</plugins>
But i assume it will not change the situation.

Resources