I have the following test cases:
Login - Logout
Login - Search - Logout
Login - Search - Click on result - Logout
All these 3 testcases are considered to be independent and are within a single testNg class.
How do I validate the First test case (i.e. how to check if login is successful) ?
I have tried using assert, but when assert fails the remaining test cases (which are actually independent on the first one) will all be skipped.
How do I make sure that I validate all the test cases and irrespective of whether they pass or fail i should continue to execute all the test cases within that testNG class?
It looks like you are looking for ErrorCollector. Or you can use custom SoftAssert. This is my ErrorCollector realization for JUnit:
#Rule //collects errors without aborting test
public ErrorCollector collector = new ErrorCollector();
#Test
public void SampleTest() throws Exception {
...
try {
// Verify label
Assert.assertEquals("label"
,driver.findElement(By.xpath("some_xpath")).getText());
} catch (AssertionError e) {
System.out.println("Report Error: " + e);
collector.addError(e);
}
...
}
Related
I am testing a service which heavily relies on project reactor.
For many tests I am mocking the return value of the component responsible for API calls.
The tests are split over multiple files.
When I run the tests of one file, they are green, but when I execute all of the test files at once, some tests fail, with the error message indicating that the mocking did not succeed (Either the injected component returned null, or the implementation of the actual component is invoked).
In the logs, there is no information about the mocking failing.
A code example:
interface API {
Flux<Bird> getBirds();
}
#Component
class BirdWatcher {
API api;
BirdWatcher(API api) {
this.api = api;
}
Flux<Bird> getUncommonBirds() {
return api.getBirds() // Although this is mocked in the test, in some runs it returns `null` or calls the implementation of the actual component
.filter(Bird::isUncommon);
}
}
#SpringBootTest
class BirdWatcherTests {
#Autowired
BirdWatcher birdWatcher;
#MockBean
API api;
#Test
void findsUncommonBirds() {
// Assemble
Bird birdCommon = new Bird("Sparrow", "common");
Bird birdUncommon = new Bird("Parrot", "uncommon");
Mockito.when(api.getBirds()).thenReturn(Flux.just(birdCommon, birdUncommon));
// Act
Flux<Bird> uncommonBirds = birdWatcher.getUncommonBirds();
// Assert
assertThat(uncommonBirds.collectList().block().size(), equalTo(1));
}
}
For me the issue seems like a race condition, but I don't know where and how this might happen, and how I can check and fix this.
I am using spring-boot-test:2.7.8, pulling in org.mockito:mockito-core:4.5.1 org.mockito:mockito-junit-jupiter:4.5.1, and org.junit.jupiter:junit-jupiter:5.8.2, with gradle 7.8.
For reactor, spring-boot-starter-webflux:2.7.8, depending on reactor:2.7.8.
Actually question related to testng-failed.xml has already been asked many times but my problem is little different. I want to run all the failed test cases together so what i did is in my pom I passed testng-failed.xml.
But the problem I am facing is first my testng.xml runs then testng-failed.xml and then it testng-failed.xml gets overridden. Due to this , suppose if i give a second time fresh run to my testcases, testng.xml runs, then my testng-failed.xml has previously failed test cases so it runs the previously failed cases and then updates testng-failed.xml with this time failed cases.
I dont knoe which listener to add to handle this issue that whenever i run first testng.xml should run , then it should override testng-failed.xml and then testng-failed.xml should run.
I am using Maven, selenium, testng.
I just eneterd testng-failed.xml in my pom as shown below. Please let me know which listner to use
<suiteXmlFiles>
<suiteXmlFile>src/resources/testng/testng.xml</suiteXmlFile>
<suiteXmlFile>test-output/testng-failed.xml</suiteXmlFile>
</suiteXmlFiles>
Create class 'RetryListener' by implementing 'IAnnotationTransformer'.
public class RetryListener implements IAnnotationTransformer {
#Override
public void transform(ITestAnnotation testannotation, Class testClass,
Constructor testConstructor, Method testMethod) {
IRetryAnalyzer retry = testannotation.getRetryAnalyzer();
if (retry == null) {
testannotation.setRetryAnalyzer(Retry.class);
}
}
}
Now Create another class.
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 1;
// Below method returns 'true' if the test method has to be retried
else 'false'
//and it takes the 'Result' as parameter of the test method that just
ran
public boolean retry(ITestResult result) {
if (retryCount < maxRetryCount) {
System.out.println("Retrying test " + result.getName() + " with status "
+ getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
retryCount++;
return true;
}
return false;
}
public String getResultStatusName(int status) {
String resultName = null;
if(status==1)
resultName = "SUCCESS";
if(status==2)
resultName = "FAILURE";
if(status==3)
resultName = "SKIP";
return resultName;
}
}
And Now Add below lines in your testNG xml file
<listeners>
<listener class-name="com.pack.test.RetryListener"/>
</listeners>
and Do not pass Xml file in pom.xml
Hope it will works
Thanks
Why are you running the testng xml and failed test xml in the same testng task. You should have to separate build task, first that runs testng xml and generates the failed tests xml and then another task running the failed test xml. It will work.
I implemented run one time and rerun three times only the newly failed tests.
mvn $par1=$pSuiteXmlFile test > $test1log
mvn $par1=$failedRelPath test > $failed1log
mvn $par1=$failedRelPath test > $failed2log
mvn $par1=$failedRelPath test > $failed3log
It works, but with small test-cases-count. I have a suite with 300 tests in it and somehow the testng-failed.xml is not created by surefire/testng after the main (first) run. When the suite is smaller, the testng-failed.xml is created as required.
I've created a significant number of methods to help me perform unit test assertions in Visual Studio 2010. I've also enabled "Double-click a Failed or Inconclusive unit test result displays the point of failure in the test" option in the Test Execution options. My issue is that with these helper methods, I'd like the point of failure to be the stack frame calling my helper method, not the exception being thrown within the helper method.
Obviously, I can do "ShowDetails" and click higher in the stack, but that will pretty much cancel out any time saved by using the helper method.
I've tried using the various [Debugger*] attributes on my method without success.
Here's some example code to illustrate my issue.
public void MyTest()
{
// ACT
var res = DoSomething();
// ASSERT
AssertDateRange(res, TimeSpan.FromDays(7));
}
public static void AssertDateRange(DateTime value, TimeSpan range)
{
var difference = DateTime.Now.Subtract(value);
if (Math.Abs(range.TotalMilliseconds) - Math.Abs(difference.TotalMilliseconds) < 0)
{
throw new AssertFailedException("DateTime was not within the expected range from now.");
}
}
I'd like for the double-click to place me on the method call within the MyTest method, while it now places me on the throw within the AssertDateRange method.
I am running our automated tests using TestNG. The reason we picked TestNG is because we can send variables inputs into the test methods example public void testXX( String userId ) and the userId can change for each test.
The code below shows three different userIds I can use to execute my tests. So my exact same test will run three times for each of the three different users. This feature is awesome and really enables me to have multiple tests under different scenarios because each of our users carry different profiles.
// All valid Pricing Leads
#DataProvider(name = "userIds")
public Object[][] createPricingLeadUsersParameters() {
return new Object[][] {
{ "TestUser001" },
{ "TestUser002" },
{ "TestUser003" }
};
}
#Test( dataProvider = "userIds" )
public void createGroup( String userIds) {
............
}
The problem I am having right now is during certain conditions I can only have one userId used or else all of my tests will fail. I would like to keep my exact same test but only pass in on userId not the three shown above. It there a way to configure TestNG to make this variable on the command line so at times I would use the three defined, but under another condition it would only be one of the three or a new userId?
Sure, there are plenty of ways to do this. How about passing a system property when you run TestNG?
java -Dfoo=bar org.testng.TestNG...
and then your data provider can test the value of foo with System.getProperty() and adjust what it returns accordingly.
I am in reference to Spring Roo In Action (book from Manning). Somewhere in the book it says "Roo marks the test class as #Transactional so that the unit tests automatically roll back any change.
Here is the illustrating method:
#Test
#Transactional
public void addAndFetchCourseViaRepo() {
Course c = new Course();
c.setCourseType(CourseTypeEnum.CONTINUING_EDUCATION);
c.setName("Stand-up Comedy");
c.setDescription(
"You'll laugh, you'll cry, it will become a part of you.");
c.setMaxiumumCapacity(10);
c.persist();
c.flush();
c.clear();
Assert.assertNotNull(c.getId());
Course c2 = Course.findCourse(c.getId());
Assert.assertNotNull(c2);
Assert.assertEquals(c.getName(), c2.getName());
Assert.assertEquals(c2.getDescription(), c.getDescription());
Assert.assertEquals(
c.getMaxiumumCapacity(), c2.getMaxiumumCapacity());
Assert.assertEquals(c.getCourseType(), c2.getCourseType());
}
However, I don't understand why changes in this method would be automatically rolled back if no RuntimeException occurs...
Quote from documentation:
By default, the framework will create and roll back a transaction for each test. You simply write code that can assume the existence of a transaction. [...] In addition, if test methods delete the contents of selected tables while running within a transaction, the transaction will roll back by default, and the database will return to its state prior to execution of the test. Transactional support is provided to your test class via a PlatformTransactionManager bean defined in the test's application context.
So, in other words, SpringJUnit4ClassRunner who runs your tests always do transaction rollback after test execution.
I'm trying to find a method that allows me to do a rollback when one of the elements of a list fails for a reason within the business rules established (ie: when throw my customize exception)
Example, (the idea is not recording anything if one element in list fails)
public class ControlSaveElement {
public void saveRecords(List<MyRecord> listRecords) {
Boolean status = true;
foreach(MyRecord element: listRecords) {
// Here is business rules
if(element.getStatus() == false) {
// something
status = false;
}
element.persist();
}
if(status == false) {
// I need to do roll back from all elements persisted before
}
}
...
}
Any idea? I'm working with Roo 1.2.2..