When a before hook fails; the tests within that suite are skipped; how could i reliably get the list of those skipped tests ?
I tried to listen to the EVENT_TEST_FAIL on the Mocha runner and extract the information from there:
const Mocha = require('mocha');
const {
EVENT_TEST_FAIL,
} = Mocha.Runner.constants;
class MyReporter {
constructor(runner) {
runner
.on(EVENT_TEST_FAIL, (test, err) => {
console.log(err, test);
console.log("failed tests", test.parent.tests.map(t => t.fullTitle()));
});
}
}
module.exports = MyReporter;
But only issue is that, this code cannot distinguish between a failure in the before hook and a failure in a particular test case.
As a result, this code reports all tests in the suite as failed, even when only one test is failed.
Related
Morning all. I have a slightly unusual design to my tests. A typical example might be
describe('1', () => {
describe('2', () => {
before()
describe('3', () => {
it('1')
// ...
it('n')
});
});
});
If there is a failure in one of the individual tests (it 1..n), I want to re-run ALL of those tests, and run the "before" code first too - ie from "describe 2". If I use a before hook then re-runs don't trigger that again. If I change to a beforeEach, then it gets called before each and every "it" block, which I don't want.
Effectively, each it block is a test check, describe 3 is a test step, describe 2 a test spec, and describe 1 a test "group"
Can anyone suggest a way I can re-run a test spec (describe 2) when one test check fails, including re-running the before code for that spec?
(I know this is probably anti-pattern etc, but....)
You can externalise the before() callback function, and use the test:after:run event to trigger it on a retry.
I haven't tested this extensively, but the gist is
const beforeCallback = () => {...}
before(beforeCallback)
Cypress.on('test:after:run', (result) => {
if (result.currentRetry < result.retries && result.state === 'failed') {
beforeCallback()
}
})
it('fails', {retries:3}, () => expect(false).to.eq(true)) // failing test to check it out
I am using addContext() that mocha provides to append failure screenshots in the Mochawesome html report. I have written this under support/index.js
Cypress.on('test:after:run', (test, runnable) => {
if (test.state === 'failed') {
const screenshot = `FailureScreenshots/${Cypress.spec.name
}/${runnable.parent.title} -- ${test.title} (failed).png`;
addContext({ test }, screenshot);
}
});
This works perfectly when there is a failure(and there are no Test Retries) it just appends the failure screenshot in the html report bases on test.state. However, in case of Test Retries, where on the first run the test failed but on the second run the test passed, it still attaches the failure screenshot. How can I prevent that? It should only append the screenshot when the test finally fails after the number of retries have been exhausted.
I am trying to write a function in wdio.conf.js that executes when a test passes.
At the end of the test, it shows all of the tests passing, however it never hits the console.log("testpassed") code shown here:
afterTest: function (test) {
if (test.passed === true) {
console.log("testpassed")
}
},
If I console.log - 'test' - it prints [object Object].
However, if I console log test.passed it prints undefined.
At the end of the test it shows all tests as passed.
What am I doing wrong?
Further investigation: These are the only keys returned in the 'test' array:
type,title,fn,body,async,sync,_timeout,_slow,_retries,timedOut,_currentRetry,pending,file,parent,ctx,_events,_eventsCount,callback,timer
So there doesn't seem to be a key for test.passed
afterTest: function (test, context, { passed }) {
if(passed){
console.log("THE TEST PASSED");
}
},
Automated the e2e tests in CI enabled framework with protractor and jasmine in VSTS. Used the jasmine custom reporter to load the e2e test results under sauce labs tab in VSTS build definition. But, it is not loading all the tests. It is displaying only the last e2e test ran in the build. Console log is getting printed for all the e2e tests. Please see the code below.
let sauceLabsReporter: jasmine.CustomReporter = {
specDone: (result: jasmine.CustomReporterResult): void => {
Util.log('*** sauceLabsReporter: result.fullName:', result.fullName);
Util.log('*** sauceLabsReporter: result.status:', result.status);
Util.log('*** sauceLabsReporter: result.testCaseId:', result.testCaseId);
if (result.testCaseId) {
result.fullName = `(Testcase ID: ${result.testCaseId}): ${result.fullName}`;
Util.log('*** sauceLabsReporter: UPDATED result.fullName:', result.fullName);
}
Util.updateSauceLabsJobTitle(result.fullName);
if (result.status) {
Util.updateSauceLabsTestState(result.status);
}
}
};
export function updateSauceLabsJobTitle(title: string): promise.Promise<void> {
let fullTitle: string = `${title} | (${getHostname()})`;
return browser.executeScript(`sauce:job-name=${fullTitle}`)
.then(() => browser.getSession())
.then((session: Session) => {
if (isThisVSTSBuildAgent()) {
//The VSTS Sauce Labs add-on gets information by parsing the console log.
//tslint:disable-next-line:no-console
console.log(`SauceOnDemandSessionID=${session.getId()} job-name=${fullTitle}`);
}
});
}
export function updateSauceLabsTestState(state: string): promise.Promise<{}> {
return browser.executeScript(`sauce:job-result=${state}`);
}
I work on the same project as Padma. It had to do with restartBrowserBetweenTests being set to false which conflates all tests into a single job. After setting it to true, each test became its own job.
Is there a way to get the filename of current test in mocha reporter?
I couldn't find anything in the base and examples.
Actually, file name is passed to Suite in file field in mocha starting from this pull request. It's just nowadays mocha most commonly is ran as a karma plugin (namely, karma-mocha plugin), and, talking of December'14, this plugin just does not pass file name information further.
To make this answer self-consistent, here's how Suite is formed in mocha (it's tdd implementation, but it it is similar for bdd):
context.suite = function(title, fn){
var suite = Suite.create(suites[0], title);
suite.file = file;
suites.unshift(suite);
fn.call(suite);
suites.shift();
return suite;
};
And here's how suits are formed in karma-mocha/lib/adapter.js:
runner.on('test end', function(test) {
var skipped = test.pending === true;
var result = {
id: '',
description: test.title,
suite: [],
success: test.state === 'passed',
skipped: skipped,
time: skipped ? 0 : test.duration,
log: test.$errors || []
};
var pointer = test.parent;
while (!pointer.root) {
result.suite.unshift(pointer.title);
pointer = pointer.parent;
}
tc.result(result);
});
But you know what, I guess this is a nice thing to issue as a feature request in karma-mocha project.