So I have a test suite written in JUnit, like below, basic mantra is that all the test cases which are listed in #Suite.SuiteClasses are going to be included in for running of the tests.
package bddDemo.TestSuite;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
//serenityBDDDemo.features.calculator.MathsUsingAnAngularCalculator.class,
//serenityBDDDemo.features.API.automateRESTAPI.class
})
public class TestSuite {
}
This runs perfectly well in maven, I can do mvn clean verify and what not, but when I tried to convert this project to a Gradle project, suite is not run, instead all the tests are run, weather are not marked for inclusion for running in the suite(which is kinda embarrassing).
Can you suggest a way to mark few tests cases for running and make a test suite in a gradle?
Related
Lets say I created a new and unique Java test framework, or I am retro-fitting a legacy project where the tests are just main(String[] args)-entrypoint classes and I would like to run those with gradle test and have gradle exit with 0 if none of the tests fail/nothing throws.
"without built-in frameworks" - meaning, without useJUnit(), useJUnitPlatform() et al.
Using Java 11, Gradle 6.5.1
In the build.gradle there is an option:
maxParallelForks = 8
I have two classes, first has 2 tests and second has 30 tests.
When I run tests using Gradle it runs only in 2 threads:
./gradlew clean test
One thread - tests for the first class, second thread - tests for the second class.
But, how to make Gradle to execute all tests in parallel?
So tests from the same class could be run in parallel in 8 threads.
If you're using JUnit 5, you can do this using JUnit's parallel execution instead of Gradle's. Note that this will be within the same JVM (I believe Gradle's maxParallelForks uses separate JVMs).
E.g. in your Gradle script:
test {
...... (your existing stuff here)
systemProperties = [
'junit.jupiter.execution.parallel.enabled': 'true'
'junit.jupiter.execution.parallel.mode.default': 'concurrent'
]
}
Different test frameworks may have equivalents.
Other than that, a workaround might be to split your tests into a separate class per test case. You can use things like test class inheritance to avoid duplication with this.
If you have more than the 2 test classes you've mentioned here, you'd only need to split up the few longest running classes to get most of the benefit.
I have a gradle project (converted from an original ANT project). The directory structure is as advised by gradle so my source is in src\main\java and test is in src\test\java.
However, because the package structure of src\test is almost exactly similar to src\main, during the test run phase in gradle, I get a bunch of
java.lang.NoClassDefFoundError
even though I can clearly see the code under ,say, src\main\A.B.C\Y\X.java. I am suspecting that while running the tests, the class to import is being searched under src\test\A.B.C\Y where, of course, the class X is not present
We are looking for a way to fail the build if there are any unit tests with out any sort of verifications. IntelliJ seems to "inspect" these tests by specifying the list of valid assertions.
org.junit.Assert assert.*|fail.*
junit.framework.Assert assert.*|fail.*
org.junit.jupiter.api.Assertions assert.*|fail.*
org.assertj.core.api.Assertions assertThat
com.google.common.truth.Truth assert.*
com.google.common.truth.Truth8 assert.*
org.mockito.Mockito verify.*
org.mockito.InOrder verify
org.junit.rules.ExpectedException expect.*
org.hamcrest.MatcherAssert assertThat
Is there any "Maven" equivalent plugin which we can hook up to the build? Thanks in advance!
I have maven's surefire plugin installed for junit. I have some test suites set up with the following code(stripped to protect the innocent):
package com.company.qa.guiauto.test.suites;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass1;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass2;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass3;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass4;
import org.junit.runner.RunWith;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
TestClass1.class, //20 Junit #Tests
TestClass2.class, //33 Junit #Tests
TestClass3.class, //4 Junit #Tests
TestClass4.class //13 Junit #Tests
})
public class SomeSuite {
}
However, when jenkins runs these tests the results are quite variable. Sometimes it runs 70 tests. Sometimes 68, sometimes 30. Can anyone suggest a reason why this would be variably quitting the test run before its actually done? As you can see here, we have a high of 72 tests, a low of 44 tests, and a last run of 67 tests. The number of tests has not changed.
Looking at your gist, I can see different test classes being executed in the 2 sessions:
Succesful Output File from Jenkins
##Succesful run here.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.company.qa.guiauto.test.regression.contentcenter.TestClass1
Unsuccessful output file
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.company.qa.guiauto.test.regression.test.TestClass1
I was able to find the issue.
It was an issue that was fixed in the latest version of Surefire(2.17) in ticket http://jira.codehaus.org/browse/SUREFIRE-1055.
Upgrading our version of Surefire fixed it.