XCTAssertTrue doesn't stop routine - xcode

Question

Xcode doesn't terminate a test routine at failed assertions. Is this correct?
I fail to understand the reason behind this and I'd like it to behave like assert and have it terminate the program.
With the following test, it will print "still running".
Is this intended?
- (void)testTest
{
XCTAssertTrue(false, #"boo");
NSLog(#"still running");
}
I don't see how this would be useful, because often subsequent code would crash when pre-conditions aren't met:
- (void)testTwoVectors
{
XCTAssertTrue(vec1.size() == vec2.size(), #"vector size mismatch");
for (int i=0; i<vec1.size(); i++) {
XCTAssertTrue(vec1[i] == vec2[i]);
}
}

Answer 1

you can change this behavior of XCTAssert<XX>.
In setup method change value self.continueAfterFailure to NO.
IMO stopping the test after test assertion failure is better behavior (prevents crashes what lead to not running other important tests). If test needs continuation after a failure this means that test case is simply to long an should be split.

Answer 2

Yes, it is intended. That's how unit tests work. Failing a test doesn't terminate the testing; it simply fails the test (and reports it as such). That's valuable because you don't want to lose the knowledge of whether your other tests pass or fail merely because one test fails.
If (as you say in your addition) the test method then proceeds to throw an exception, well then it throws an exception - and you know why. But the exception is caught, so what's the problem? The other test methods still run, so your results are still just what you would expect: one method fails its test and then stops, the other tests do whatever they do. You will then see something like this in the log:
Executed 7 tests, with 2 failures (1 unexpected) in 0.045 (0.045) seconds
The first failure is the XCTAssert. The second is the exception.

Answer 3

Just to clarify, you are correct that if a test generates a "failure", that this individual test will still continue to execute. If you want to stop that particular test, you should simply return from it.
The fact that the test resumes can be very useful, whereby you can identify not only the first issue that resulted in the test failure, but all issues resulting in the test failure.
You say:
I don't see how this would be useful, because often subsequent code would crash when pre-conditions aren't met:
- (void)testTwoVectors
{
XCTAssertTrue(vec1.size() == vec2.size(), #"vector size mismatch");
for (int i=0; i<vec1.size(); i++) {
XCTAssertTrue(vec1[i] == vec2[i]);
}
}
Your example is ironic, because, yes, I can understand why you'd want to return if the two vectors were different sizes, but if they happened to be the same size, the second half of your example test is a perfect example of why you might not want it to stop after generating a failure.
Let's assume that the vectors were the same size and five of the items were not the same. It might be nice to have the test report all five of those failures in this test, not just the first one.
When review test results, it's just sometimes nice to know all of the sources of failure, not just the first one.