I am using MStest (.net core) for my test project, Is there a way to continue the test execution on failure? - mstest

I want my test to proceed next line after the assert fails. I tried "try catch block" but tests are not failing. Do we have anything that can fix this issue in fluent assertion [https://fluentassertions.com/] or anyways to handle this in code?

Sounds you are searching for AssertionScope of FluentAssertions.
using (new AssertionScope())
{
5.Should().Be(10); // will fail but not throw
DoSomeOtherStuff();
} // assertion will raise exception
All failed asserts will be collected until AssertionScope is disposed.

Related

How to check whether an UI element exists without failing Xamarin Test Cloud?

I am using Visual Studio C# with Xamarin to develop an iOS app. I have an UI automation which runs in Xamarin Test Cloud.
I want to check if a pop up window exists, and if it exists I want to dismiss it, othervise test should just continue. The code I am using in the test method is like below:
try
{
app.WaitForElement(x => x.Text("Update available"));
app.Tap(x => x.Marked("Ignore"));
}
catch (TimeoutException)
{
// TODO: This catch doesn't work. Test in cloud still fails.
}
The problem is that the catch doesn't work. When the test runs in Xamarin Test Cloud, the test still fails with time out error finding the element.
Does anyone know how to do it without failing the test?
You can use the app.Query method to check if the element exist if you want to avoid using the WaitForElement
The app.Query returns array of elements for the query

Jenkins reports a test build as a success, even though there are failures caused by exceptions?

I have SeleniumWebdriver/TestNG/Maven/Java continuous integration tests that are being run every time after a deploy. Sometimes an element is missing from the user interface and the tests throw an exception (which is later caught in the code, because in the catch statement I turn off the browser), so the build is marked as a success.
The strange thing is, I had failures in tests caused by exceptions before as well, and the build was still considered a successfull one.
How can I configure my maven pom.xml file or the jenkins build in order for it to mark every test that has thrown an exception, a FAILURE?
EDIT: After getting robjohncox's responce, I now have another thing I need to do:
How exactly do I throw the error again?
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
quit(driver);
sendMail();
}
Is it done this way?
throw e;
I think the problem relates to the fact that you are catching the exception in the code. Because you are handling the exception, it doesn't propagate up to your test runner, and therefore the test runner isn't aware that an exception was thrown.
After catching the exception and turning off your browser, you should re-throw the exception and then the test failures should be reported by your testing framework. The code would look something like this:
public void myTestCase() {
try {
// Do the testing
}
catch(Exception ex) {
// Turn off the browser
throw ex
}
}

Surefire Report not generating on Script Error

I have a couple simple tests that looks like this.
public class My1Test extends AutoBaseFunctions{
#Test
public void test1(){
startDriver(STARTPAGE,DRIVER2USE);
schoolLogin("XXX", "XXX");
toolbarNav("toolsSingle","Manage Users");
getElmObject("input[type='checkbox'][name='includeUsersHiddenFromDirectory']",loctype.CSS).click();
getElmObject("Searchxx",loctype.LINKTEXT).click();
driver.quit();
//driver.close();
}
}
The problem I have is this: I am purposely causing a NoSuchElement Exception to happen when doing a findElement on the search button. When I run the mvn surefire-report:report command the output stops here
until I MANUALLY close the window that was opened - only then does it finish generating the report.
I understand that when the script errors out it dies and never gets to the driver.quit line. But, if I put it in a try/catch block, the report shows the test as passed and doesn't report the error details. I also tried putting this as part of a #Suite test and using #After/#AfterClass but that doesn't close the driver window either.
What am I doing wrong? How do I get the report to generate after an error is encountered without having to "be present" to manually close the open windows?
You need to put it in try/catch and in catch block do
try{
}catch(Exception e){
Assert.fail(e.getMessage());
}finally{
// CLOSE ANY OPEN RESOURCES HERE !!!!
}
If you have any open resources, don't forget to close them in finally block.

How to debug a plugin in on-line version?

I've created a plugin and registered it using hte registration tool. I've also added a step that is supposed to handle a message of creation of an instance. Sadly, the intended behavior doesn't occur.
My guess is that something inside the plugin crashes but I have no idea on how to debug it. Setting up breakpoints is not going to work agains on-line version, I understand, so I'm not even trying.
For legal and technical reasons, I won't be able to lift over the solution to an on-premise installation, neither. Is guessing my only option?
For server-side (plugins) I'm using ITracingService. For client-side I log everything to console. The downside with the first is that you actually need to crash the execution to get to see anything. The downside with the latter is that plugins sometimes get executed without GUI being invoked at all.
When it comes to heavier projects, I simply set up a WCF web service that I call from the plugin and write to that. That way, on one screen, I'm executing the plugin while on the other, I'm getting a nice log file (or just put the sent information to on the screen).
You could, for instance, start with a very basic update of a field on the instance of your entity that's being created. When you have that working, you can always fall back to the last working version. If you don't even get that to work, it mean, probably, that you're setting up the plugin registration incorrectly.
A very efficient way would be to lift over the solution to an on-premise version where you have full control but I see in your question that it's not en option.
In case you could lift the solution to an on-premise version, here's a link on how to debug plugins.
Don't forget that you also have access to the ITracingService.
You can get a reference to it in your Execute method and then write to it every so often in your code to log variables or courses of action that you are attempting or have succeeded with. You can also use it to surface more valuable information when an exception occurs.
It's basically like writing to a console. Then, if anything causes the plug-in to crash at runtime then you can see everything that you've traced when you click Download Log File on the error shown to the user.
Beware though - unless your plug-in actually throws an exception (deliberate or otherwise) then you have no access to whatever was traced.
Example:
public void Execute(IServiceProvider serviceProvider)
{
// Obtain the execution context from the service provider.
IPluginExecutionContext context =
(IPluginExecutionContext)serviceProvider.GetService(
typeof(IPluginExecutionContext));
// Get a reference to the tracing service.
ITracingService tracingService =
(ITracingService)serviceProvider.GetService(typeof(ITracingService));
try
{
tracingService.Trace("Getting entity from InputParameters...");
// may fail for some messages, since "Target" is not present
var myEntity = (Entity)context.InputParameters["Target"];
tracingService.Trace("Got entity OK");
// some other logic here...
}
catch (FaultException<OrganizationServiceFault> ex)
{
_trace.Trace(ex.ToString());
while (ex.InnerException != null)
{
ex = (FaultException<OrganizationServiceFault>)ex.InnerException;
_trace.Trace(ex.ToString());
}
throw new InvalidPluginExecutionException(
string.Format("An error occurred in your plugin: {0}", ex));
}
catch (Exception ex)
{
_trace.Trace(ex.ToString());
while (ex.InnerException != null)
{
ex = ex.InnerException;
_trace.Trace(ex.ToString());
}
throw;
}
}

Access denied error ( Visual Studio and WatiN )

I'm using the WatiN testing tool with Visual Studio 2005. When I try to select a value from my list box I am getting an "access denied" error.
I have seen this a lot with select lists recently when using the WatiN 2.0 beta. Instead of using the aSelectList.Select(strText) option, it seems to work better when you do this:
ie.SelectList(Find.ById("MySelect")).Option(Find.ByText("Option 1")).Select();
This can also happen when changing an ASP.NET control that cause an auto-postback. The first change will register, but the next element you try to access will throw an "Access Denied" error because it is still trying to access the old page. In this case you can try using ie.WaitForComplete(), but sometimes this is required:
ie.SelectList(Find.ById("AutoPostBackSelect")).Option(Find.ByText("Option")).Select();
System.Threading.Thread.Sleep(200); //Sleep to make sure post back registers
ie.WaitForComplete();
ie.SelectList(Find.ById("MySelect")).Refresh()
ie.SelectList(Find.ById("MySelect")).Option(Find.ByText("Option 1")).Select();
This is a bug in the select list where if the list is not ready to accept input, and it can throw one several exception types. We solve it like this:
try
{
_domContainer.SelectList(_control.WatinAttribute).Focus();
_domContainer.SelectList(_control.WatinAttribute).Select(value);
}
catch (Exception e)
{
Console.WriteLine("Select list eception caught: " + e.Message + e.StackTrace);
// we have tried once already and failed, so let's wait for half a second
System.Threading.Thread.Sleep(500);
_domContainer.SelectList(_control.WatinAttribute).Select(value);
}
And yes I know that swallowing all exceptions like this is normally bad, but if the exception occurs again, it is thrown to the test code and the test fails.
I noticed this happens if you try and select a value that is already selected.
You can work around this with a pre-check:
if(_sel_ddlPeriodFromDay.GetValue("value")!="1")
_sel_ddlPeriodFromDay.SelectByValue("1");
or maybe use a try catch?
try{_sel_ddlPeriodFromDay.SelectByValue("1");}
catch{}

Resources