I'm using Nightwatch JS to run my e2e tests with the Mocha runner.
I want to integrate an HTML reporter that with the suite.
I'm trying to use the nightwatch-html-reporter package. But as far as I understand there is a problem with the CLI commands (it's written in the Nightwatch docs that --reporter will not work when using mocha).
I also copied the code sample from nightwatch-html-reporter to my globals.js but it doesn't seem to work either.
The tests run but there is no output anywhere.
Here is my folder structure:
project
src
spec
e2e
globals
globals.js
tests
smoke
testFile.js
nightwatch.conf.js
Here is my conf file:
const seleniumServer = require('selenium-server-standalone-jar');
const chromeDriver = require('chromedriver');
module.exports = {
src_folders: ['src/spec/e2e/tests'],
output_folder: 'report',
page_objects_path: [
'src/spec/e2e/pageObjects'
],
globals_path: 'src/spec/e2e/globals/globals.js',
custom_commands_path: 'src/spec/e2e/customCommands',
selenium: {
start_process: true,
server_path: seleniumServer.path,
host: '127.0.0.1',
port: 4444,
cli_args: {
'webdriver.chrome.driver': chromeDriver.path
}
},
test_runner: {
type: 'mocha',
options: {
ui: 'bdd',
reporter: 'list'
}
},
test_settings: {
default: {
launch_url: 'http://URL',
silent: true,
desiredCapabilities: {
browserName: 'chrome',
javascriptEnabled: true,
acceptSslCerts: true,
chromeOptions: {
args: [
"--no-sandbox",
"start-fullscreen"
]
}
}
}
}
};
And here is my global.js file:
var HtmlReporter = require('nightwatch-html-reporter');
var reporter = new HtmlReporter({
openBrowser: true,
reportsDirectory: __dirname + '/reports'
});
module.exports = {
reporter: reporter.fn
};
I don't think it will work with nightwatch-html-reporter as it is probably not a mocha reporter (but correct me if I'm wrong).
You want to use built in or custom mocha reporters when using nightwatch with mocha.
You can use a custom mocha html reporter like mochawesome but you'll have to hack around a bit and I offer no guarantees as I only tested those hacks lightly.
Here are the instructions to use mochawesome
(tested with
"mocha": "^5.2.0",
"mochawesome": "^3.1.1",
"nightwatch": "^0.9.21")
npm install mochawesome
Modify mochawesome node_modules\mochawesome *.js files to require mocha-nightwatch instead of mocha. (See instructions/explanations towards the end of the answer)
Presuming you're using a nightwatch.conf.js, configure your test runner to the equivalent of
test_runner : {
type : "mocha",
options : {
ui : "bdd",
reporter : require("mochawesome") // Please observe that you can pass a custom report constructor function here, not just reporter names
}
}
Run tests and observe that you still see the default console reporter (spec) but that at the end of the run you also see an output like:
[mochawesome] Report HTML saved to C:\projects\myWebApp\mochawesome-report\mochawesome.html
Open the html report.
This solution is hackish and fragile because nightwatch comes with it's own version of mocha.
When you install nightwatch you will see in your node_modules a mocha-nightwatch folder. This is the mocha that is being used by nightwatch.
However mochawesome doesn't use mocha-nightwatch. If you look at node_modules\mochawsome\dist\mochawesome.js you will see lines of code like:
var Base = require('mocha/lib/reporters/base');
var Spec = require('mocha/lib/reporters/spec');
This means is requires mocha, not mocha-nightwatch.
Those lines should ideally be: require('mocha-nightwatch/...).
So please change them in all *.js files that need fixing.
You could also fork mochawesome and make them like that ;)
Debugging notes:
Try putting some additional console.logs in node_modules\mocha-nightwatch\lib\mocha.js in the Mocha.prototype.reporter function. That's how I figured out what's going on.
If you use Mocha you can always go with mochawsome: https://www.npmjs.com/package/mochawesome
I haven't tried it myself but it looks pretty neat.
Related
I want Cypress to execute custom scripts before and after running the specs suite.
The before:run and after:run APIs look perfect for that.
But the same snippet that works perfectly in before:run doesn’t seem to work in after:run:
module.exports = {
e2e: {
setupNodeEvents(on, config) {
on("before:run", details => {
console.log("Before run:")
exec("npm run cal", {}, handleExecOutput)
})
on("after:run", results => {
console.log("After run:")
exec("npm run cal", {}, handleExecOutput)
})
},
},
};
The cal script is defined in package.json as simply calling the native cal command, as a minimal test.
This is logging "Before run" and the calendar (not exactly together) before running the specs, and only "After run" after running the specs. However, if I change npm run cal to simply cal, the command is executed in both cases.
I’ve tried it with and without promises, with yarn and npm, always the same result. What am I missing?
I'm currently trying to get code coverage on my fastify routes using Mocha and NYC.
I've tried instrumenting the code beforehand and then running the tests on the instrumented code as well as just trying to setup NYC in various ways to get it to work right.
Here is my current configuration. All previous ones produced the same code coverage output):
nyc config
"nyc": {
"extends": "#istanbuljs/nyc-config-typescript",
"extension": [
".ts",
".tsx"
],
"exclude": [
"**/*.d.ts",
"**/*.test.ts"
],
"reporter": [
"html",
"text"
],
"sourceMap": true,
"instrument": true
}
Route file:
const routes = async (app: FastifyInstance, options) => {
app.post('/code', async (request: FastifyRequest, response: FastifyReply<ServerResponse>) => {
// route logic in here
});
};
The integration test:
import * as fastify from fastify;
import * as sinon from 'sinon';
import * as chai from 'chai';
const expect = chai.expect;
const sinonChai = require('sinon-chai');
chai.use(sinonChai);
describe('When/code POST is called', () => {
let app;
before(() => {
app = fastify();
// load routes for integration testing
app.register(require('../path/to/code.ts'));
});
after(() => {
app.close();
});
it('then a code is created and returned', async () => {
const {statusCode} = await apiTester.inject({
url: '/code',
method: 'POST',
payload:{ code: 'fake_code' }
});
expect(statusCode).to.equal(201);
});
});
My unit test call looks like the following:
nyc mocha './test/unit/**/*.test.ts' --require ts-node/register --require source-map-support/register --recursive
I literally get 5% code coverage just for the const routes =. I'm really banging my head trying to figure this one out. Any help would be greatly appreciated! None of the other solutions I have investigated on here work.
I have a detailed example for typescript + mocha + nyc. It also contains fastify tests including route tests (inject) as well as mock + stub and spy tests using sinon. All async await as well.
It's using all modern versions and also covers unused files as well as VsCode launch configs. Feel free to check it out here:
https://github.com/Flowkap/typescript-node-template
Specifically I think that
instrumentation: true
messes up the results. Heres my working .nycrc.yml
extends: "#istanbuljs/nyc-config-typescript"
reporter:
- html
- lcovonly
- clover
# those 2 are for commandline outputs
- text
- text-summary
report-dir: coverage
I have proper coverage even for mocked ans tub parts of fastify in my above mentioned example.
I'm using wdio to run tests. I reduced maxInstances to 1.But wdio logging indicates that it creates a new session before each spec file. How can i run all webdriver.io spec files in single browser session?Thank you in advance.
wdio.conf.js is:
exports.config = {
specs: ['./test/specs/**/*.js'],
maxInstances: 1,
capabilities: [{
maxInstances: 1,
browserName: 'chrome',
}],
sync: true,
logLevel: 'verbose',
coloredLogs: true,
screenshotPath: './errorShots/',
baseUrl: process.env.ROOT_URL,
waitforTimeout: 10000,
connectionRetryTimeout: 90000,
connectionRetryCount: 3,
services: ['chromedriver'],
framework: 'mocha',
reporters: ['dot', 'spec', 'allure'],
mochaOpts: {
ui: 'bdd',
timeout: 99999999
},
}
Try this workaround. It actually works for me with webdriverio v4
List all of your specs in a single file. You can take advantage of autocomplete feature of the IDE you are using, e.g.
specs.js
require('./test/specs/test1');
require('./test/specs/test2');
// etc.
require('./test/specs/testN');
In your wdio.conf.js file, list the above spec.js file as the only spec, i.e.
wdio.conf.js
exports.config = {
specs: ['./test/specs/specs.js'],
// etc.
}
WebdriverIO will run each test file in a different session. To run them all in the same session, you'd need to put all your tests in the same file.
If you're finding yourself needing to run all your tests in the same session, perhaps you should re-work your tests... Maybe use WebdriverIO's "before" hook if you need to do a common set up, like logging in to a site.
I'm trying to generate a report on nightwatch cucumber using nightwatch-html-reporter but I'm not able to make it work.
The library I am using is [Nightwatch html Reporter][1], I followed the steps described but I'm getting the error when reading the reports directory:
Reading reports directory...
events.js:160
throw er; // Unhandled 'error' event
^
TypeError: Cannot read property 'name' of undefined
The correct configuration would be:
Create file in root tests, same level of package.json.
var reporter = require('cucumber-html-reporter');
var options = {
theme: 'bootstrap',
jsonFile: 'reports/cucumber.json',
output: 'reports/index.html',
reportSuiteAsScenarios: true,
launchReport: false
};
reporter.generate(options);
And configure runner in package.json. Example:
"scripts": {
"e2e": "npm-run-all test report --continue-on-error",
"test": "nightwatch",
"report": "node create-html-report.js"
}
Make sure it is set up this way, or enter more details of the error.
I have installed the chrome jet brains extension
I have tests like this:
describe('Service tests', function () {
beforeEach(module('app'));
it('should have a Service', inject(function($injector) {
var exist = $injector.has('dataService');
etc
but no luck getting breakpoints to hit any where in the tests. I can get the debugger to break when writing debugger, but an unable to step through.
Do you have karma-coverage set up in your karma config? It uses instrumented code, so debugging is not possible. Related tickets: http://github.com/karma-runner/karma/issues/630, http://youtrack.jetbrains.com/issue/WEB-8443
If you are building with Webpack you might need to specify the devtools option in your webpack config property in karma.conf.js like this:
module.exports = (config) => {
config.set({
webpack: {
...,
devtool: 'inline-source-map'
}
})
};
This solution works for me with Webpack v3.
If by any chance you are using Angular and you have removed all the coverage related stuff from your karma.config file and are still unable to hit the breakpoints, look into the angular.json. It might be having the codeCoverage bit set to true.
"test": {
...
"options": {
...
"codeCoverage": false,
...
}
...
}