Unable to execute maven tests at some times using citrus test frame work - maven

Question

I am running citrus test cases using maven command which looks like this
mvn integration-test -Dfilename.pattern=**/*Rest.xml -Dhtmlreport.name=report4.html
I am executing above command using java procesess executor here is the code
public class TestExecutor {
String[] command = new String[3];
command[0] = "cmd";
command[1] = "/c";
command[2] = "mvn integration-test -Dfilename.pattern=**/*"+xmlName+" "+"-Dhtmlreport.name="+report_name;
}
Process p = Runtime.getRuntime().exec(command);
I am exposing above code as rest service. user sends system properties through url. I am passing that information to maven fail safe plugin using maven command here is the maven fail safe plugin properties.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.20</version>
<configuration>
<systemProperties>
<citrus.xml.file.name.pattern>${filename.pattern}</citrus.xml.file.name.pattern>
<citrus.html.report.file>${htmlreport.name}</citrus.html.report.file>
</systemProperties>
</configuration>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
after my rest service receiving this information it execute maven command which was mentioned above. What the problem I am facing is when the rest service used as single instance the command was executing fine. But when I tested this code in for multi users ie is in multi threaded environment. Some times every user hit(Tests) was executing fine but most of the time this code was not executing all the users test cases(ie is some users test cases were excuting fine ) but for remaing users the console out put was just like this. There are no tests executed. I am using testNg to execute my test cases.
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------< com.walmart.citrus:multi-user-handling >---------------
[INFO] Building Citrus Integration Test 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # multi-user-handling ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:compile (default-compile) # multi-user-handling ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to D:\springnotes\multi-user-handling\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # multi-user-handling ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) # multi-user-handling ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.20:test (default-test) # multi-user-handling ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) # multi-user-handling ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.20:integration-test (integration-tests) # multi-user-handling ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 48.568 s
[INFO] Finished at: 2018-08-23T23:41:20+05:30
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "test" could not be activated because it does not exist.
for remaining users in successes case of executing the test case out put was like this
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------< com.citrus:multi-user-handling >---------------
[INFO] Building Citrus Integration Test 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # multi-user-handling ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:compile (default-compile) # multi-user-handling ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # multi-user-handling ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) # multi-user-handling ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.20:test (default-test) # multi-user-handling ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) # multi-user-handling ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.20:integration-test (integration-tests) # multi-user-handling ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use forkCount and reuseForks instead.
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.walmart.tests.SampleXmlIT
23:41:11,094 INFO citrus.Citrus| Unable to locate Citrus application properties
23:41:11,137 INFO BeanDefinitionReader| Loading XML bean definitions from URL [file:/D:/springnotes/multi-user-handling/target/test-classes/citrus-context.xml]
23:41:11,709 DEBUG BeanDefinitionReader| Loaded 0 bean definitions from location pattern [classpath*:citrus-context.xml]
23:41:13,545 INFO port.LoggingReporter|
23:41:13,545 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:13,545 INFO port.LoggingReporter| .__ __
23:41:13,545 INFO port.LoggingReporter| ____ |__|/ |________ __ __ ______
23:41:13,545 INFO port.LoggingReporter| _/ ___\| \ __\_ __ \ | \/ ___/
23:41:13,545 INFO port.LoggingReporter| \ \___| || | | | \/ | /\___ \
23:41:13,545 INFO port.LoggingReporter| \___ >__||__| |__| |____//____ >
23:41:13,545 INFO port.LoggingReporter| \/ \/
23:41:13,545 INFO port.LoggingReporter|
23:41:13,545 INFO port.LoggingReporter| C I T R U S T E S T S 2.7.3
23:41:13,546 INFO port.LoggingReporter|
23:41:13,546 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:13,546 DEBUG port.LoggingReporter| BEFORE TEST SUITE
23:41:13,546 INFO port.LoggingReporter|
23:41:13,546 INFO port.LoggingReporter|
23:41:13,546 INFO port.LoggingReporter| BEFORE TEST SUITE: SUCCESS
23:41:13,546 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:13,546 INFO port.LoggingReporter|
23:41:13,582 DEBUG t.TestContextFactory| Created new test context - using global variables: '{}'
23:41:14,618 INFO port.LoggingReporter|
23:41:14,618 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:14,618 DEBUG port.LoggingReporter| STARTING TEST 20180326024725Rest <com\walmart\resources>
23:41:14,618 INFO port.LoggingReporter|
23:41:14,618 DEBUG citrus.TestCase| Initializing test case
23:41:14,619 DEBUG context.TestContext| Setting variable: citrus.test.name with value: '20180326024725Rest'
23:41:14,619 DEBUG context.TestContext| Setting variable: citrus.test.package with value: 'com\walmart\resources'
23:41:14,619 DEBUG citrus.TestCase| Test variables:
23:41:14,619 DEBUG citrus.TestCase| citrus.test.name = 20180326024725Rest
23:41:14,619 DEBUG citrus.TestCase| citrus.test.package = com\test\resources
23:41:14,619 INFO port.LoggingReporter|
23:41:14,619 DEBUG port.LoggingReporter| TEST STEP 1/1: echo
23:41:14,620 INFO actions.EchoAction|
hmmm
23:41:14,620 INFO port.LoggingReporter|
23:41:14,620 DEBUG port.LoggingReporter| TEST STEP 1/1 SUCCESS
23:41:14,621 INFO port.LoggingReporter|
23:41:14,621 INFO port.LoggingReporter| TEST SUCCESS 20180326024725Rest (com\walmart\resources)
23:41:14,621 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:14,621 INFO port.LoggingReporter|
[TestHTMLReporter] Access is denied
23:41:15,207 INFO port.LoggingReporter|
23:41:15,207 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:15,208 DEBUG port.LoggingReporter| AFTER TEST SUITE
23:41:15,208 INFO port.LoggingReporter|
23:41:15,208 INFO port.LoggingReporter|
23:41:15,208 INFO port.LoggingReporter| AFTER TEST SUITE: SUCCESS
23:41:15,208 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:15,209 INFO port.LoggingReporter|
23:41:15,209 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:15,209 INFO port.LoggingReporter|
23:41:15,209 INFO port.LoggingReporter| CITRUS TEST RESULTS
23:41:15,209 INFO port.LoggingReporter|
23:41:15,212 INFO port.LoggingReporter| 20180326024725Rest ............................................. SUCCESS
23:41:15,212 INFO port.LoggingReporter|
23:41:15,212 INFO port.LoggingReporter| TOTAL: 1
23:41:15,212 DEBUG port.LoggingReporter| SKIPPED: 0 (0.0%)
23:41:15,212 INFO port.LoggingReporter| FAILED: 0 (0.0%)
23:41:15,212 INFO port.LoggingReporter| SUCCESS: 1 (100.0%)
23:41:15,212 INFO port.LoggingReporter|
23:41:15,212 INFO port.LoggingReporter| ------------------------------------------------------------------------
23:41:15,212 DEBUG report.HtmlReporter| Generating HTML test report
23:41:15,213 DEBUG util.FileUtils| Reading file resource: 'test-detail.html' (encoding is 'windows-1252')
23:41:15,584 DEBUG util.FileUtils| Reading file resource: 'test-report.html' (encoding is 'windows-1252')
23:41:15,591 INFO report.HtmlReporter| Generated HTML test report: target\citrus-reports\report8.html
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.61 s - in com.walmart.tests.SampleXmlIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[WARNING] Could not delete temp directory D:\springnotes\multi-user-handling\target\surefire because Directory D:\springnotes\multi-user-handling\target\surefire unable to be deleted.
[WARNING] File encoding has not been set, using platform encoding Cp1252, i.e. build is platform dependent! The file encoding for reports output files should be provided by the POM property ${project.reporting.outputEncoding}.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51.719 s
[INFO] Finished at: 2018-08-23T23:41:19+05:30
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "test" could not be activated because it does not exist.
Here is my rest service code
#RestController
public class RestRunner {
static String namePattern="";
#RequestMapping(value="runurtest", method=RequestMethod.GET)
#ResponseBody
public String rest(#RequestParam String xmlName,#RequestParam String report_name) throws Exception
{
//TestExecutor class executes maven command in java procesess
new TestExecutor().exec(xmlName,report_name);
//report manger class sends html report user
return new ReportManager().readHTMLFile(report_name);
}
}

Answer 1

By default, the maven-failsafe-plugin will only execute test classes with the following wildcard patterns:
**/IT*.java **/*IT.java **/*ITCase.java
In case your test classes do not follow any of these naming conventions, then failsafe will not execute your tests.
You can configure the maven-failsafe-plugin to use different naming patterns. Please see how to do this in https://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html