I have the following Test:
[Test]
public void GrantResourceOwnerCredentials_NullClientID_ThrowsArgumentNullException()
{
Assert.Throws<ArgumentNullException>(() =>{ new ApplicationOAuthProvider(null,null); });
}
The test passed, but when i run code coverage, it highlights
new ApplicationOAuthProvider(null,null);
as "Coverage Partially Touched Area".
How do i fix that?
By the way, I am new to Unit Testing/Code Coverage so go easy on me.
Thanks
When you see a result of "Partially touched", what that means is that there are multiple code paths that can be traversed through a method (such as through an if block, a null check, etc.), and only one path was executed by your tests.
For example, if you had the following method:
public string IsThisEvenOrOdd(int number)
{
return (number % 2 == 0) ? "Even" : "Odd";
}
And you write the following unit test:
public string IsThisEvenOrOdd_PositiveEvenNumber_ReturnsEven()
{
var number = 2;
var expected = "Even";
var actual = IsThisEvenOrOdd(number);
Assert.AreEqual(expected, actual);
}
If you run that test, it should succeed. However, it only tests one path - the "Even" path. You'll need to write at least one other test to test the "Odd" result before that method will show up as "fully covered" by MSTest.
Related
I've encountered a wierd behaviour of a groovy script when running the whole suite. I have a script which orders my testcases alphabetically before they run and it seems to be running forever even when the whole test suite finishes.
After I click on it to see details and immediately go back to test suite it shows that it's finished and no longer running.
Is there something wrong with my script please? I don't see any infinite loops or anything like that. Is it just a bug in the ReadyAPI? Thanks for advice.
My sorting script:
ArrayList<String> testCaseList = new ArrayList<String>();
for (testCase in testRunner.testCase.testSuite.getTestCaseList()) {
testCaseList.add(testCase.getName());
}
testCaseList.sort();
int i = 0;
for (tCase in testCaseList) {
def curCase = testRunner.testCase.testSuite.getTestCaseByName(tCase);
curIndex = testRunner.testCase.testSuite.getIndexOfTestCase(curCase);
testRunner.testCase.testSuite.moveTestCase(curIndex,i-curIndex);
i++;
}
Currently, looks you have separate test case for sorting. But actually, that is not a valid test case of yours.
So, the first change to be made is that the script should be moved from test case to Setup Script of test suite.
Here is the Test Suite's Setup Script which does it alphabetical order. Should be paid special attention in case if the test case names have numbers in it, have to do natural order. Otherwise, it should be ok.
Please follow the in-line comments.
//Get the sorted order of the test case which is expected order
def newList = testSuite.testCaseList.name.sort()
log.info "Expected order of test cases: ${newList}"
//Get the current index of the test case
def getTestCaseIndex = { name -> testSuite.getIndexOfTestCase(testSuite.getTestCaseByName(name))}
//Closure definition and this is being called recursively to make the desired order
def rearrange
rearrange = {
def testCaseNames = testSuite.testCaseList.name
if (testCaseNames != newList) {
log.info testCaseNames
newList.eachWithIndex { tc, index ->
def existingIndex = getTestCaseIndex(tc)
if (index != existingIndex) {
testSuite.moveTestCase(index, existingIndex-index)
rearrange()
}
}
} else {
log.info 'All cases sorted'
}
}
//Call the closure
rearrange()
With that Setup Script, when test suite is executed, automatically the test cases are moved alphabetically. Hence, no separate test case required for just ordering.
Now, the suite gets executed with desired test cases and the current issue mentioned in the question should not be there at all.
I have a method where I tried to parallelise the calc using GPARS and calculate an aggregate boolean 'And' result across the calls. This method is wrapped as a #ActiveObject which will deliver the result as a dataflow - the code below has the original approach where I tried to store the aggregate using AtomicBoolean to protect it.
This didn't work (sometimes my tests would pass others they would fail) on the calculated 'end truth'. To fix this I changed from AtomicBoolean to a Agent(boolean) approach and I think it's 'fixed' it - at least my spock tests are continuously succeeding.
Where was my logic flawed trying to use AtomicBoolean to build the final result? It felt like it should work - but doesn't and I don't understand why.
Method below - I've put the original version, and the corrected version below
#ActiveMethod
def evaluateAllAsync () {
AtomicBoolean result = new AtomicBoolean(true)
GParsPool.withPool {
// do as parallel
conditions.eachParallel { condition ->
println "evalAllAsync-parallel intermediate result start value is ${result.get()} and condition with expression ${condition.expression} evaluated to ${condition.evaluate()}"
result.getAndSet(result.get() && condition.evaluate())
println "recalc bool value is now ${result.get()}"
}
}
println "evalAllAsync-parallel final result value is ${result.get()}"
result.get()
}
Fixed issue by using Agent form like this
#ActiveMethod
def evaluateAllAsync () {
def result = new Agent (true)
GParsPool.withPool {
// do as parallel
conditions.eachParallel { condition ->
println "evalAllAsync-parallel intermediate result start value is ${result.val} and condition with expression ${condition.expression} evaluated to ${condition.evaluate()}"
result << { def res = it && condition.evaluate(); println "start> $it and finish> $res"; updateValue(res)}
println "recalc bool value is now ${result.val}"
}
}
println "evalAllAsync-parallel final result value is ${result.val}"
result.val
}
I put debug println in here just so I could see what code was doing.
The version with the Agent to protect the bool aggregate value appears to be working.
Why doesn't the Atomic Boolean work?
Well, this is an issue in how you use AtomicBoolean. You always overwrite by force (getAndSet()) the value stored in it and ignore the possibility that other threads could have changed it while the current thread is busy 'evaluating'.
You perhaps wanted to use the compareAndSet() method instead:
def valueToUse = result.get()
result.compareAndSet(valueToUse, valueToUse && condition.evaluate())
I have following Extension Method which is just a negation of Linq.Any()
These two UnitTests do test it completely
[TestMethod]
public void EnumerableExtensions_None_WithMatch()
{
Assert.IsTrue(_animals.None(t => t.Name == "Pony"));
}
[TestMethod]
public void EnumerableExtensions_None()
{
var emtpyList = new List<Animal>(); { };
Assert.IsTrue(emtpyList.None());
}
As you can see in the picture, when I run a Code Coverage Analysis, the delegate body is not covered (white selection), because of the deferred execution.
This question comes close to the problem:
Code Coverage on Lambda Expressions
But does not quite solve it: Since the List must stay empty, it's impossible to actually step into that piece of code.
I am tempted to mark the segment with [ExcludeFromCodeCoverage] ...
How would you write the UnitTest?
You need to test that None() returns false when given a non-empty list of Animal. As it is, you never execute your default lambda expression.
You might even find a bug...
This is the correct way to write the Test. Even found a bug!
public void EnumerableExtensions_None()
{
// _animals HAS entries
Assert.IsFalse(_animals.None());
}
Code Coverage 100%!
One colleague in my team says that some methods should have both preconditions & postconditions. But the point is about code coverage, those conditions were not being called (not tested) until an invalid implementation implemented (just used in unit test). Lets take below example.
public interface ICalculator
{
int Calculate(int x, int y);
}
public int GetSummary(int x, int y)
{
// preconditions
var result = calculator.Calculate(x, y);
// postconditions
if (result < 0)
{
**throw new Exception("...");**
}
return result;
}
Two options for us:
1/ Remove test implementations + postconditions
2/ Keep both test implementations + postconditions
Can you give some advice please?
Keep pre- and post-conditions.
You'll need at least four tests here: combinations of (pre, post) x (pass, fail). Your failing post-condition test will pass if the expected exception is thrown.
This is easy to do in JUnit with its #Test(expected = Exception.class) annotation.
Be careful with colleagues that make blanket statements like "X must always be true." Dogma in all its forms should be avoided. Understand the reasons for doing things and do them when they make sense.
These conditions should be seen from design view. They ensure the calculator should be working fine, returning result in a range of expected values.
You should see the MS code contracts project to take a document there.
I'm trying to test my application using the XCTest framework.
I want my single test case to fail if some logical condition holds (using an assertion).
I don't want the rest of the code in the test case to run, because this might lead to problems (access to null pointers, for example)
I also want the rest of the test case to run normally, and just the failed test to be marked as failed.
I've noticed XCTestCase has a property called continueAfterFailure.
However, setting it to YES caused the failed test to continue executing lines after the assertion, and setting it to NO caused the rest of the tests not to run at all.
Is there a solution to this issue?
Pascal's answer gave me the idea to achieve this properly. XCTool now behaves like OCUnit when an assertion fails: the execution of the test case is aborted immediately, tearDown invoked and the next test case is run.
Simply override the method invokeTest in your base class (the one that inherits from the XCTestCase class):
- (void)invokeTest
{
self.continueAfterFailure = NO;
#try
{
[super invokeTest];
}
#finally
{
self.continueAfterFailure = YES;
}
}
That's it!
The easiest way is to add:
continueAfterFailure = false
into setUp() method. So it will look like this:
Swift
override func setUp() {
super.setUp()
continueAfterFailure = false
}
Objective-C
- (void)setUp {
[super setUp];
[self setContinueAfterFailure:NO];
}
One option would be to check the condition normally, then fail and return from the test if it is false.
Something like this:
if (!condition) {
XCFail(#"o noes");
return;
}
You could wrap this up in a helper macro to preserve readability.
BDD test libraries like Kiwi are more elegant for this sort of thing, as they make it easier to share setup between many tests which leads to fewer assertions per test.
I am able to use continueAfterFailure and let the other tests run by using this pattern:
self.continueAfterFailure = NO;
#try
{
// Perform test code here
}
#finally
{
self.continueAfterFailure = YES;
}
In Swift projects, I use a helper function (defined in a shared superclass of all my tests which itself extends XCTestCase):
/// Like `XCTFail(...)` but aborts the test.
func XCTAbortTest(_ message: String,
file: StaticString = #file, line: UInt = #line
) -> Never {
self.continueAfterFailure = false
XCTFail(message, file: file, line: line)
fatalError("never reached")
}
As the comment suggests, the call to fatalError is never actually executed; XCTFail aborts the test in an orderly fashion (tearDown is called, next test runs, etc.). The call is only there to trick the compiler into accepting Never as return type since XCTFail returns Void (it does return if continueAfterFailure == true).
Note that self.continueAfterFailure is reset to the default true for every test method. You can also make that explicit in setUp().