I have a bunch of feature files(around 15) where each feature file has a thread that needs to sleep for atleast 3 mins (for some complex background app to execute some functions). I need a way to execute all of them in parallel.
I have a simple CucumberRunnerTestCLass.
#RunWith(Cucumber.class)
#CucumberOptions(features = "src/test/java/com/sample/demo/features",glue{""}, plugin = {"pretty","html:target/cucumber"}, tags = {"#integrationTesting"})
public class JUnitRunnerTest {
}
Any help is very appreciated. Thanks
Provide threads like:
#RunWith(Cucumber.class)
#CucumberOptions(threads=5,features = "src/test/java/com/sample/demo/features",glue{""}, plugin = {"pretty","html:target/cucumber"}, tags = {"#integrationTesting"})
public class JUnitRunnerTest {
}
Related
I am trying to write a groovy script which will contain functions that are common to the SoapUI test Suite. Specifically I want to write a script that will contain all the logs that are output from the test suite.
GroovyScript1 will call a function in the GroovyScripts.groovy file. All is present in a SoapUI test suite.
I have not found any helpful advice on how to perform this task.
To specify again, I want to call a function contained in another Groovy Script.
Yes You can do this by following steps,
In your "GroovyScripts.groovy" file add below code,
class GLF
{
def log
def context
def testRunner
def GLF(logIn, contextIn, testRunnerIn)
{
this.log = logIn
this.context = contextIn
this.testRunner = testRunnerIn
}
//Till abobe line you must keep same code except class name
public String returnVal()
{
return 'Himanshu'
}
}
context.setProperty("Rt", new GLF(log, context, testRunner))
============================ END GroovyScripts.groovy ==========
Now in your "GroovyScript1" file you should use below code,
lib = testRunner.testCase.testSuite.project.testSuites["GroovyLibraryFunction"].testCases["TestCase 1"].testSteps["EndpointVerification"]
lib.run(testRunner, context)
def RT = context.Rt
def PT = RT.returnVal()
log.info PT
This way you can achive your target.
Let's say that I have created separate tasks for running integration and acceptance tests in my gradle build script. When I run the build task I want to run testing tasks before in this order: unit tests(test task), integration tests (intergationTest task) and acceptance tests (acceptanceTest task). Is this possible and how?
You are looking for "should run after" described in Gradle documentation - http://www.gradle.org/docs/current/userguide/more_about_tasks.html
Here's how you can do it without creating artificial dependencies:
https://caffeineinduced.wordpress.com/2015/01/25/run-a-list-of-gradle-tasks-in-specific-order/
TLDR; version:
//--- build aliases : define a synonym here if you want a shortcut to run multiple targets
def buildAliases = [
'all' : ['clean', 'assemble', 'runProvisioner', 'stopTomcat', 'installTomcat', 'deployToTomcat', 'startTomcat'],
'rebuild' : ['clean', 'assemble']
]
def expandedTaskList = []
gradle.startParameter.taskNames.each {
expandedTaskList << (buildAliases[it] ? buildAliases[it] : it)
}
gradle.startParameter.taskNames = expandedTaskList.flatten()
println "\n\n\texpanded task list: ${gradle.startParameter.taskNames }\n\n"
This is what I did on my projects.
check.dependsOn integTest
integTest.mustRunAfter test
tasks.withType(Pmd) {
mustRunAfter integTest // Pointing to a task
}
tasks.withType(FindBugs) {
mustRunAfter tasks.withType(Pmd) // Pointing to a group of tasks under Pmd
}
tasks.withType(Checkstyle) {
mustRunAfter tasks.withType(FindBugs)
}
It helped me to order tasks by group.
I created this helper method based on a solution that I found on Gradle forum.
Task.metaClass.runFirst = { Task... others ->
delegate.dependsOn(others)
delegate.mustRunAfter(others[0])
for (def i=0; i < others.size() - 1; i++) {
def before = others[i]
def after = others[i+1]
after.mustRunAfter(before)
}
}
Then you can create tasks X, A, B and C and use like this:
X.runFirst A, B, C
The first answer in the list turned out to be great for me. I used
X.shouldRunAfter Y
UPDATE: While using this "solution" for a short while i found out, that it does not work 100% as intended. I don't know why though. Any help to make it work would be appreciated. Maybe it does not work properly when there is more than one task in the Set?! While testing i did add some dummy-Tasks which only printed a text inbetween each of the other tasks and all seemed to be ok.
After some attempts with other solutions i came up with this solution:
It uses the mustRunAfter command to chain Sets of Tasks into the required order.
I'm working with Sets instead of individual Tasks, because i got circular dependency issues otherwise, since some tasks already depended on each other.
Also important to note: the isNotEmpty() check was essential, as it would otherwise break the enforced ordering if an ampty set was passed to the method.
tasks.register("cleanAndGenerate") {
var lastTasks: Set<Task> = setOf()
fun findTasks(taskName: String): Set<Task> {
return if (taskName.startsWith(":")) { // task in specific sub-project
setOf(tasks.findByPath(taskName)!!)
} else { // tasks in all (sub-)projects
getTasksByName(taskName, true)
}
}
fun dependsOnSequential(taskName: String) {
val tasks = findTasks(taskName)
tasks.forEach { task -> task.mustRunAfter(lastTasks) }
dependsOn(tasks)
if (tasks.isNotEmpty()) {
lastTasks = tasks
}
}
dependsOnSequential(":project1:clean") // task in specific sub-project
dependsOnSequential(":project2:clean")
dependsOnSequential("task1") // tasks in all (sub-)projects
dependsOnSequential("task2")
// add more as needed
}
I have several folders trying to seperate my tests up into unit, integration, 3rd party, db. This way I can have my tests seperated into chunks purposes, to make TDD easier/faster. Here is the task that I am trying to use.
task integrationTest(type: Test) {
testClassesDir = sourceSets.integration.output.classesDir
classpath = sourceSets.integration.runtimeClasspath
maxParallelForks 8
maxHeapSize = "4048m"
}
I know there is testReportDir, but it's deprecated. I want to be able to use the new method.
I have tried the following closures:
reports {
html = file("$buildDir/reports/intTests")
}
reports {
setDestination = file("$buildDir/reports/intTests")
}
reports {
destinationDir = file("$buildDir/reports/intTests")
}
destinationDir = file("$buildDir/reports/intTests")
I think you want
integrationTest.reports.html.destination = file("$buildDir/reports/intTests")
You may want to consult the api docs for TestTaskReports which shows that the html report is a DirectoryReport extending ConfigurableReport and that that provides the destination accessor referred to in the one liner above.
Maybe you can implement your own task of type TestReport similar to here. Note the feature is incubating though.
Right now we've got a project that builds in two jobs. 1) Is the standard build with unit tests. 2) is the integration tests. They work like this:
build the whole project, run unit tests, start integration test job
build the whole project, deploy it to the integration server, run client side integration tests against integration server
The problem is step 2) now takes over an hour to run and I'd like to parallelize the integration tests so that they take less time. But I'm not exactly sure how I can/should do this. My first thought is that I could have two step 2)s like this:
build the whole project, run unit tests, start integration test job
build the whole project, deploy it to the integration server1, run client side integration tests against integration server1
build the whole project, deploy it to the integration server2, run client side integration tests against integration server2
But then, how do I run half the integration tests on integration server1, and the other half on integration server2? I am using maven, so I could probably figure out something with failsafe and a complex includes/excludes pattern. But that sounds like something that would take a lot of effort to maintain. EG: when someone adds a new integration test class, how do I ensure that it gets run on one of the two servers? Does the developer have to modify the maven patterns?
I found this great article on how to do this, but it gives a way to do it in Groovy code. I pretty much followed these steps, but I haven't written the code to distribute the tests evenly by duration. But this is still a useful tool so I'll share it.
import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;
import org.junit.Ignore;
import org.junit.extensions.cpsuite.ClassesFinder;
import org.junit.extensions.cpsuite.ClasspathFinderFactory;
import org.junit.extensions.cpsuite.SuiteType;
import org.junit.runner.RunWith;
import org.junit.runners.AllTests;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
#RunWith(AllTests.class)
public class DistributedIntegrationTestRunner {
private static Logger log = LoggerFactory.getLogger(DistributedIntegrationTestRunner.class);
public static TestSuite suite() {
TestSuite suite = new TestSuite();
ClassesFinder classesFinder = new ClasspathFinderFactory().create(true,
new String[]{".*IntegrationTest.*"},
new SuiteType[]{SuiteType.TEST_CLASSES},
new Class[]{Object.class},
new Class[]{},
"java.class.path");
int nodeNumber = systemPropertyInteger("node.number", "0");
int totalNodes = systemPropertyInteger("total.nodes", "1");
List<Class<?>> allTestsSorted = getAllTestsSorted(classesFinder);
allTestsSorted = filterIgnoredTests(allTestsSorted);
List<Class<?>> myTests = getMyTests(allTestsSorted, nodeNumber, totalNodes);
log.info("There are " + allTestsSorted.size() + " tests to choose from and I'm going to run " + myTests.size() + " of them.");
for (Class<?> myTest : myTests) {
log.info("I will run " + myTest.getName());
suite.addTest(new JUnit4TestAdapter(myTest));
}
return suite;
}
private static int systemPropertyInteger(String propertyKey, String defaultValue) {
String slaveNumberString = System.getProperty(propertyKey, defaultValue);
return Integer.parseInt(slaveNumberString);
}
private static List<Class<?>> filterIgnoredTests(List<Class<?>> allTestsSorted) {
ArrayList<Class<?>> filteredTests = new ArrayList<Class<?>>();
for (Class<?> aTest : allTestsSorted) {
if (aTest.getAnnotation(Ignore.class) == null) {
filteredTests.add(aTest);
}
}
return filteredTests;
}
/*
TODO: make this algorithm less naive. Sort each test by run duration as described here: http://blog.tradeshift.com/just-add-servers/
*/
private static List<Class<?>> getAllTestsSorted(ClassesFinder classesFinder) {
List<Class<?>> allTests = classesFinder.find();
Collections.sort(allTests, new Comparator<Class<?>>() {
#Override
public int compare(Class<?> o1, Class<?> o2) {
return o1.getSimpleName().compareTo(o2.getSimpleName());
}
});
return allTests;
}
private static List<Class<?>> getMyTests(List<Class<?>> allTests, int nodeNumber, int totalNodes) {
List<Class<?>> myTests = new ArrayList<Class<?>>();
for (int i = 0; i < allTests.size(); i++) {
Class<?> thisTest = allTests.get(i);
if (i % totalNodes == nodeNumber) {
myTests.add(thisTest);
}
}
return myTests;
}
}
The ClasspathFinderFactory is used to find all test classes that match the .*IntegrationTest pattern.
I make N jobs and they all run this Runner but they all use different values for the node.number system property, so each job runs a different set of tests. This is how the failsafe plugin looks:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.12.4</version>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<includes>
<include>**/DistributedIntegrationTestRunner.java</include>
</includes>
<skipITs>${skipITs}</skipITs>
</configuration>
</plugin>
The ClasspathFinderFactory comes from
<dependency>
<groupId>cpsuite</groupId>
<artifactId>cpsuite</artifactId>
<version>1.2.5</version>
<scope>test</scope>
</dependency>
I think there should be some Jenkins plugin for this, but I haven't been able to find one. Something that's close is the Parallel Test Executor, but I don't think this does the same thing I need. It looks like it runs all the tests on a single job/server instead of multiple servers. It doesn't provide an obvious way to say, "run these tests here, and those tests there".
I believe you already found a solution by now, but I'll leave a path for the others who'll open this page asking the same question:
Parallel test executor plugin:
"This plugin adds a new builder that lets you easily execute tests defined in a separate job in parallel. This is achieved by having Jenkins look at the test execution time of the last run, split tests into multiple units of roughly equal size, then execute them in parallel."
https://wiki.jenkins-ci.org/display/JENKINS/Parallel+Test+Executor+Plugin
Yes, Parallel Test Executor is a cool plugin if you've got 2 slave or one slave with 8 executor because the this plugin based on "tests splitting" so e.g: you split your junit tests into 4 different array, these arrays will run on 4 different executor on that slave what you specified. I hope you got it :D, it depends on the number of executors on that slave where you want to run parallel testing or you should decrease split tests count to 2 from 4.
I am running our automated tests using TestNG. The reason we picked TestNG is because we can send variables inputs into the test methods example public void testXX( String userId ) and the userId can change for each test.
The code below shows three different userIds I can use to execute my tests. So my exact same test will run three times for each of the three different users. This feature is awesome and really enables me to have multiple tests under different scenarios because each of our users carry different profiles.
// All valid Pricing Leads
#DataProvider(name = "userIds")
public Object[][] createPricingLeadUsersParameters() {
return new Object[][] {
{ "TestUser001" },
{ "TestUser002" },
{ "TestUser003" }
};
}
#Test( dataProvider = "userIds" )
public void createGroup( String userIds) {
............
}
The problem I am having right now is during certain conditions I can only have one userId used or else all of my tests will fail. I would like to keep my exact same test but only pass in on userId not the three shown above. It there a way to configure TestNG to make this variable on the command line so at times I would use the three defined, but under another condition it would only be one of the three or a new userId?
Sure, there are plenty of ways to do this. How about passing a system property when you run TestNG?
java -Dfoo=bar org.testng.TestNG...
and then your data provider can test the value of foo with System.getProperty() and adjust what it returns accordingly.