I have maven's surefire plugin installed for junit. I have some test suites set up with the following code(stripped to protect the innocent):
package com.company.qa.guiauto.test.suites;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass1;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass2;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass3;
import com.company.qa.guiauto.test.regression.contentcenter.TestClass4;
import org.junit.runner.RunWith;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
TestClass1.class, //20 Junit #Tests
TestClass2.class, //33 Junit #Tests
TestClass3.class, //4 Junit #Tests
TestClass4.class //13 Junit #Tests
})
public class SomeSuite {
}
However, when jenkins runs these tests the results are quite variable. Sometimes it runs 70 tests. Sometimes 68, sometimes 30. Can anyone suggest a reason why this would be variably quitting the test run before its actually done? As you can see here, we have a high of 72 tests, a low of 44 tests, and a last run of 67 tests. The number of tests has not changed.
Looking at your gist, I can see different test classes being executed in the 2 sessions:
Succesful Output File from Jenkins
##Succesful run here.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.company.qa.guiauto.test.regression.contentcenter.TestClass1
Unsuccessful output file
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.company.qa.guiauto.test.regression.test.TestClass1
I was able to find the issue.
It was an issue that was fixed in the latest version of Surefire(2.17) in ticket http://jira.codehaus.org/browse/SUREFIRE-1055.
Upgrading our version of Surefire fixed it.
Related
I have cucumber tests with rest-Assured. I tried to use #Skip tags to exclude some tests. But Jenkins integration test stage in the pipeline shows as unstable(Amber).
Code that enables skip
#RunWith(CucumberWithSerenity.class)
#CucumberOptions(
plugin = {"pretty", "html:target/reports/cucumber-html-report",
"html:target/cucumber-reports/cucumber-pretty",
"json:target/cucumber.json"},
tags = {"not #skip"},
glue = {"myproject.stepdefinitions"},
features = "src/it/resources/features/")
public class MyCucumberTests {
}
Serenity reports correctly shows that particular test as skipped
How to make the test Jenkins pipeline stage green as no tests failed.
Using Java 11, Gradle 6.5.1
In the build.gradle there is an option:
maxParallelForks = 8
I have two classes, first has 2 tests and second has 30 tests.
When I run tests using Gradle it runs only in 2 threads:
./gradlew clean test
One thread - tests for the first class, second thread - tests for the second class.
But, how to make Gradle to execute all tests in parallel?
So tests from the same class could be run in parallel in 8 threads.
If you're using JUnit 5, you can do this using JUnit's parallel execution instead of Gradle's. Note that this will be within the same JVM (I believe Gradle's maxParallelForks uses separate JVMs).
E.g. in your Gradle script:
test {
...... (your existing stuff here)
systemProperties = [
'junit.jupiter.execution.parallel.enabled': 'true'
'junit.jupiter.execution.parallel.mode.default': 'concurrent'
]
}
Different test frameworks may have equivalents.
Other than that, a workaround might be to split your tests into a separate class per test case. You can use things like test class inheritance to avoid duplication with this.
If you have more than the 2 test classes you've mentioned here, you'd only need to split up the few longest running classes to get most of the benefit.
So I have a test suite written in JUnit, like below, basic mantra is that all the test cases which are listed in #Suite.SuiteClasses are going to be included in for running of the tests.
package bddDemo.TestSuite;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
//serenityBDDDemo.features.calculator.MathsUsingAnAngularCalculator.class,
//serenityBDDDemo.features.API.automateRESTAPI.class
})
public class TestSuite {
}
This runs perfectly well in maven, I can do mvn clean verify and what not, but when I tried to convert this project to a Gradle project, suite is not run, instead all the tests are run, weather are not marked for inclusion for running in the suite(which is kinda embarrassing).
Can you suggest a way to mark few tests cases for running and make a test suite in a gradle?
I have a great problem and I have tried to solve this problem, but all time is the same.
I have this scenario with cucumber
feature file
And this is the steps' file
steps' file
Finally I run with
package Steps;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#CucumberOptions(
features = {"./src/test/java/features"},
glue = {"./src/test/java/Steps"},
plugin = {"pretty", "html:reports/cucumber-html-report","json:cucumber.json"}
)
public class Orquestador {
}
And the consolo of intellij appear this:
ans
these are all my files
Files
I am using Mac OS
Thanks :)
Cucumber is not finding your step definitions. Provide the correct "glue" to the location of your step definitions when running the feature file.
How to know which version of JUnit do I have installed on my PC?
Second question- How do I upgrade it?
This will print the version of jUnit.
java -cp <path-to-junit-folder>/junit.jar junit.runner.Version
The version that you are running depends on the classpath of the running application. It's possible to have multiple versions on the same machine, but if this is in question you can just write a test for it:
import junit.runner.Version;
class SomeTests {
void testJUnitVersion() {
assertEquals("4.8.2", Version.id());
}
}
To upgrade to a newer version of JUnit, you just have to download the new version and replace the old version with the new jar. It's as simple as that.
There isn't really a nice way to tell which version of JUnit you're currently using, unless you're using maven or something like that. In general, you can tell between
JUnit 3: if you're using the junit.framework.* classes
JUnit 4: if you're using the org.junit.* classes (the tests have #Test annotations on them)
In general, JUnit is backward compatible (JUnit 3 tests can be run under JUnit 4), but JUnit 4 is recommended.
Eclipse shows its runner version on JUnit view when you run your tests
In netbeans in the projects window, expand the test libraries folder of a project that has a test in it. It will say the version in the name of the jar file.
With JUnit 5 you can do something like this:
import java.util.ServiceLoader;
import org.junit.platform.engine.TestEngine;
import java.util.Iterator;
final ServiceLoader<TestEngine> loader = ServiceLoader.load(TestEngine.class);
final Iterator<TestEngine> iterator = loader.iterator();
final TestEngine testEngine = iterator.next();
final String version = testEngine.getVersion().get();
Upgrade process depends on your project structure. If it’s a Maven project, you can set the version in your pom.xml. If it’s still unclear I recommend creating another question for your specific build environment.