Testng split dataProvider between parallel methods - parallel-processing

Question

I have a dataProvider which is reading the data from a text file.
#DataProvider (name = "dynamicDP", parallel = true)
public Iterator<Object> matchIDs() throws IOException {
final List<Object> list = new ArrayList<>();
for (final String line: Files.readAllLines(Paths.get("C:\\mypath"),
StandardCharsets.UTF_8))
list.add(new Object[]{ line});
return list.iterator();
}
My text file is really simple it only contains these data below (each pair of letters on a separate line):
AA BB CC DD EE FF GG HH II KK
Here is my test class:
public class dataProviderParallelTest {
#Test (dataProvider="dynamicDP")
public void verifyDPdata(String comingFromDP){
System.out.printf("%nDP#1..: "+comingFromDP);
}
#Test (dataProvider="dynamicDP")
public void verifyDPdata2(String comingFromDP){
System.out.printf("%nDP#2..: "+comingFromDP);
}
}
Here is the output:
[TestNG] Running:
C:\projects\test\currentTest.xml
DP#1..: AA
DP#2..: BB
DP#1..: BB
DP#2..: AA
DP#1..: CC
DP#2..: CC
DP#1..: DD
DP#1..: EE
DP#2..: EE
DP#2..: DD
DP#1..: FF
DP#2..: FF
DP#1..: GG
DP#1..: HH
DP#2..: HH
DP#2..: GG
DP#1..: II
DP#2..: II
DP#1..: KK
DP#2..: KK
===============================================
Regression
Total tests run: 20, Failures: 0, Skips: 0
===============================================
And here is my XML file that I'm using to start my test:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Regression" parallel="methods" thread-count="2" data-provider-thread-count="2">
<test name="smokeTest11">
<classes>
<class name="regression.bo.dataProviderParallelTest"/>
</classes>
</test>
</suite>
What I have tried:
I've read this article: cedricBlog
and this stackoverflow post:stackOverFlow
What am I trying to achieve:
I am trying to share the data between the two threads. At the moment I just achieved both of the threads to execute the same data provided by the DP. My aim is to split the data between the two methods and have an output like this (DP data shared between the 2 methods):
DP#1..: AA
DP#2..: BB
DP#1..: DD
DP#2..: EE
DP#1..: CC
DP#2..: GG
DP#1..: KK
DP#1..: HH
DP#2..: II
DP#2..: FF
Is this even possible or am I missing something? Thanks in advance for your help!

Answer 1

Take the method as parameter to your dataprovider. Divide your data in two lists, whichever way you like. Based on method name, return one list to each.
eg.
For example, the following code prints the name of the test method inside its #DataProvider:
#DataProvider(name = "dynamicDP", parallel = true)
public Object[][] dynamicDP(Method m) {
System.out.println(m.getName());
//Divide list in two lists
if (m.getName().equals("Met1")
return list1
else
return list2
}
#Test(dataProvider="dynamicDP")
public void test1(String s) {
}
#Test(dataProvider="dynamicDP")
public void test2(String s) {
}
HTH

Answer 2

Okay, posting and my solution for reference. In short it is this. Count the lines in the file, and start reading 1 line at a time (from 10 different threads) until you reach the EOF. Here it is:
public volatile int currentLine=0;
public static Object writeLock = new Object();
public static Object readLock = new Object();
public long currentThread = Thread.currentThread().getId();
#Test(invocationCount = 50)
public void readOneLineGetID() throws IOException{
countLines();
if(currentLine==noOfLines){throw new SkipException("%nSkipping this test method as we got all the records we need.");}
long threadID = Thread.currentThread().getId();
synchronized(readLock){
if(currentLine<noOfLines){
System.out.printf("%nCurrent thread is..: "+ threadID);
readASpecificLineFromATextFile(currentLine);
System.out.printf("%n----------------------------------------------------------");
}
}
synchronized(writeLock){
currentLine++;
}
}
So I have 10 of these methods and I am pushing my tests concurrently onto a grid hub which then uses 10 different data providers to feed the nodes.
Only, minor point is the invocationCount (ideally i should divide by 10 and then set the invocation count on every single method accordingly; but as I don't have time to re-invent the wheel and this runs very very fast (only dealing with 200 line files normally, I decided to skip the rest of methods after the line number reaches EOF ;) )
Here is the output (10 methods, with 50 invocation count finding 10 mockup lines from my file and skipping the rest):
After some tinkering with the sync, beatiful!!! :)