I am using Chimp.js to run E2E tests on CircleCI against my staging server which runs a Meteor app. Tests sometimes fails, and it would be great to take screenshot of UI to debug those failing tests.
Is it possible to save screenshots with Chimp and Mocha? Chimp uses webdriver.io which has ability to save screenshots by calling browser.saveScreenshot('./snapshot.png');
http://webdriver.io/api/utility/saveScreenshot.html#Example
But how to save screenshots only when tests fail?
And how to view those screenshots on CircleCI?
To save screenshot exactly after Mocha test fails, you can use code similar to this one. Screenshot is saved in afterEach() function if test in it block fails.
describe('some feature test', function () {
it('first it block', function () {
signInPage.open();
...
});
it('second it block', function () {
...
});
afterEach(function () {
if (this.currentTest.state === 'failed') {
browser.saveScreenshot('/tmp/artifacts/screenShot.png');
}
});
});
Not this should work fine on local computer.
To be able to save and view screenshot on circleCI you can use artifacts: https://circleci.com/docs/2.0/artifacts/#uploading-artifacts
Put code similar to this one to your config.yml
version: 2
jobs:
my_fancy_test:
...
steps:
...
- run: |
mkdir /tmp/artifacts
cd app && npm run my-fancy-test
- store_artifacts:
path: /tmp/artifacts
If test fails on CircleCI, screenShot.png should be copied and visible in artifacts tab on CircleCI:
Related
I want to do some clearing up after all .spec files have been run by Cypress. For this I created another .spec file that does several API calls. I need Cypress to run this .spec file only after all tests form all the other files have ran. One more thing, my .spec files are being run by Cypress in parallel mode, via 4 machines.
I found out there are the "after" hooks I could use, but as far as I read, these hooks apply per only one .spec file, not all of them.
This is not exactly the answer you are looking for, but maybe this information can be useful to you in some way:
One of the antipatterns I have heard multiple times from Cypress gurus is cleaning the application state using after or afterEach hooks.
If I remember correctly, the reasoning was that if the test fails or some step of the after hook fails, it will never get to the end of the script, thus we cannot be sure that the application state is prepared for the future tests.
That's why it is suggested to use before or beforeEach hooks, to prepare application state right before running tests.
If you decide to run some scripts before or after tests in different specs, then the support file can be useful, because it is rendered before each spec:
https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file
As a final thought, if you want something to happen before or after running Cypress, then maybe the best way to accomplish this is to prepare a separate script and run it using whatever you use to run Cypress (node, docker, etc.).
Sorry that this answer got so long :)
To conclude my thoughts:
Better prepare the state for the tests, not clean up after them
Separate scripts can be run before or after tests, and they do not have to be inside Cypress (using node, bash, docker...)
Hope this helps!
To run a single spec out of process, take a look at the Module Api.
With this you can create a node script that can be run after all parallel processes have completed.
./scripts/e2e-run-cleanup.js
const cypress = require('cypress')
cypress
.run({
spec: './cypress/e2e/cleanup.cy.js', // your cleanup spec
})
.then((results) => {
console.log(results)
})
.catch((err) => {
console.error(err)
})
Some configuration:
cypress.config.js
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
baseUrl: 'http://localhost:1234',
excludeSpecPattern: ['cleanup.cy.js'], // exclude this one from the normal run
}
})
package.json
"scripts": {
...
"test:headless": "yarn cypress:run",
...
"cleanup": "yarn ./scripts/e2e-run-cleanup.js", // script to kick off
"posttest:headless": "yarn cleanup", // also after local run
// "post" prefix automatically
// runs this after "test:headless" script
...
}
main.yml
...
jobs:
install:
...
ui-chrome-tests:
...
cleanup:
...
needs: ui-chrome-tests # ensure tests have run
steps:
- run: yarn cleanup
You can use the After Run API for this use case:
module.exports = (on, config) => {
on('after:run', (results) => {
// run some code after the run
})
}
or
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
on('after:run', (results) => {
// run some code after the run
})
}
}
})
I have configured Mochaawsome report in cypress. There are 2 .js files in my project. each file have 2 testcases. I am running both .js file through command on command terminal. Below is the command that i am using:
cypress run --reporter mochawesome \ --reporter-options reportDir=reporter-config.json,overwrite=false,henter code heretml=false,json=true
My Terminal show execution results of both js files both when i go and check html report then it is only showing results of second js file.
My HTML report is generating at Path:
cypress/reporter-config.json overwrite=false html=false json=true/mochawesome.html
My First js file
describe('First Test Suite', function() {
it('First Test Case', function() {
cy.log("-------1st Suite 1st Testcase-----");
})
})
My Second js file
describe('My Second Test Suite', function() {
it('My Second Test Case', function() {
cy.log("-------2nd Suite 2nd Testcase-----");
})
it('My Third Test Case', function() {
cy.log("-------2nd Suite 3rd Testcase-----");
})
})
HTML Report Screenshot
Cypress json file
package json file
reporter-config json file
I believe this is because thats the functionality.
There is an additional npm module that merges reports together so you get a collective rather than the last run test.
https://www.npmjs.com/package/mochawesome-merge
You could then have an npm script that merges the report.
Like this:
"merge_reports": "mochawesome-merge --reportDir mochawesome-report > mochawesome-report/output.json",
And run this after the tests with:
npm run merge_reports
I want to reset my database before each "spec" start. NOT before each test
I see that i can add code in support/index.js
before(function () {
cy.exec('npm run db:reset')
// This run only once before ALL the spec
})
beforeEach(function () {
cy.log('RUN BEFORE EACH TEST IN EACH SPEC')
})
I want to run before each spec file. I am using GUI and clicking "Run all specs";
I have multiple spec file and in each spec have multiple test.
UPDATE : My test in GUI failed because database wasn't getting reset. I tried running test in CLI and all tests pass. So does that mean in CLI it does run before each spec ? Is it issue only in GUI ?
I am using WebDriverIO and want to generate Allure Reports. I followed all steps mentioned in Allure
I did:
$ npm install wdio-allure-reporter --save-dev
package.json has:
"wdio-allure-reporter": "~0.0.2"
My wdio.conf.js:
reporters: [allure],
reporterOptions: {
allure: {
outputDir: 'allure-results'
}
},
When I do allure generate './allure-results' --clean
Report successfully generated to allure-report
But when go to /allure-report folder and open index.html, its a blank page. Also there is nothing in the .allure-results folder.
Can someone help please and direct in the right direction. What am I missing?
I had the same experience with allure when combining with wdio. No matter what combination I tried via https://docs.qameta.io/allure/latest/#_commandline, I kept getting a blank html report.
I found somewhere a mention of using serve instead of generate. I used the command ./node_modules/.bin/allure serve allure-results/. and VWOLAH! (?) It worked! It runs a local server with test results and data loaded in.
Although ... it doesn't seem to grab all test data, it seems to grab the very last test that ran and only that.
Use the following piece of code in the wdio.conf.js
reporters: ['allure'],
reporterOptions: {
allure: {
outputDir: 'allure-result',
disableWebdriverStepsReporting: true,
disableWebdriverScreenshotsReporting: false,
useCucumberStepReporter: false
}
},
Command to generate allure report
node_modules/.bin//allure generate allure-results/&& node_modules/.bin/allure open
#jazz, try updating the version of your wdio-allure-reporter.
In my package.json, I have "wdio-allure-reporter": "^0.1.2",
I'm trying to convert a project from browserify+mochify to webpack.
The webpack docs demonstrate how to use mocha-loader to run the tests with webpack-dev-server, but assumes a single entry point into the tests.
All the existing tests were designed with mochify in mind which does not require a single entry point as it recursively bundles ./test/*.js.
The setup below sort of works for me. It still uses mochify to run the tests (because it has all the phantomjs interfacing), but doesn't rely on anything from browserify. If you run webpack --watch, it reruns all tests when a file changes.
webpack.config.js:
var path = require("path");
var child_process = require('child_process');
module.exports = {
entry: {
tests: "./tests.js"
},
output: {
filename: "tests.js", // Should be a unique name
path: "/tmp"
},
plugins: [
// Automatically run all tests when webpack is done
function () {
this.plugin("done", function (stats) {
child_process.execSync('mochify /tmp/tests.js', { stdio: 'inherit'});
});
}
],
};
tests.js:
// List all test dirs here if you have multiple
var contexts = [
require.context('./dir1/test', true, /\.js$/),
require.context('./dir2/test', true, /\.js$/)
];
contexts.forEach(function (context) {
context.keys().forEach(context);
});
Another approach is described here: https://stackoverflow.com/a/32386750/675011
This is not exactly an answer, but it solved the problem for me. The reason I wanted to combine mochify and webpack was that I wanted to use the browser console to debug my mocha tests. Since my mocha tests themselves don't rely on a browser, it was enough for me to finally realize I could use a node debugger and it would bring up the Chrome console, (almost) solving my problem. node-inspector is the node debugger, but I'm using babel, so I needed babel-node-debug, but that doesn't yet work with babel6, but there's an unmerged pull request that fixes it: https://github.com/CrabDude/babel-node-debug/pull/12.