I am using Cypress for end to end testing, and I would like to be able to see all run test suites, in the browser, even after they are run. Currently, after each test suite is completed (test which are stored in separate files), the browser reloads and I cannot see previously run tests, and after the final test suite, the browser closes. Is there an option to change this behavior so that I can run all test files, have all the results visible in the browser and that the browser doesn't close at the end?
I am currently running tests using this command: ./node_modules/.bin/cypress run --headed --spec 'cypress/integration/tests/*'
where /tests is the folder where I currently have my files.
I have added --no-exit but in this case cypress doesn't move to the next test file and only the first one runs.
A workaround solution could be to generate reports with Mochawsome, for each Test Spec, and then merge and view those rendered reports. The reports will contain the results from the tests, test bodies, any errors that occurred and some other bits of information.
If you read through the page in the link it shows you how to generate individual reports then combine them together, and then render them as HTML. These can then be viewed in the browser.
This command can be used to install what's needed npm install --save-dev mochawesome mochawesome-merge mochawesome-report-generator
and then add the Reporter configuration to the cypress.json:
{
"reporter": "mochawesome",
"reporterOptions": {
"reportDir": "cypress/results",
"overwrite": false,
"html": false,
"json": true
}
}
Keep in mind that it may not give you the level of detail that is contained in the Cypress Dashboard in the browser, for example, what was yielded from a request.
Cypress has a lot of possible command with a lot of possible config too.
Read this.
And if you use npm just run like this :
npm run cypress:open
and in your package.json :
"scripts": {
"cypress:open": "cypress open"
}
Related
I need to support a custom directory structure on my project that uses Cypress for testing. As I am using cucumber preprocessor plugin to handle feature files (trying to work with ATDD approach but also I am using other plugins to manage accessibility and performance testing) I though why not try to split the directory structure this way
package.json
src/
testing/
---> acceptance/
---> accessibility/
---> performance/
---> cypress.acceptance.json
---> cypress.accessibility.json
---> cypress.performance.json
Where the cypress.X.json files are the Cypress configuration files that will act as the cypress.json for each testing folder.
To manage this I am adding some scripts to my package.json to make things easier
"scripts": {
...
"test:acceptance": "npx cypress run --config-file test/cypress.acceptance.json",
"test:performance": "npx cypress run --config-file test/cypress.performance.json",
"test:accessibility": "npx cypress run --config-file test/cypress.accessibility.json",
...
}
If we focus on the acceptance testing for a moment, the Cypress configuration (test/cypress.acceptance.json) looks something like this (yes I had to override all the defaults to make Cypress happy!)
{
"baseUrl": "http://localhost:3000/",
"video": true,
"cypress-cucumber-preprocessor": {
"nonGlobalStepDefinitions": true,
"nonGlobalStepBaseDir": "test/acceptance/integration"
},
"testFiles": "**/*.feature",
"integrationFolder": "test/acceptance/integration",
"fixturesFolder": "test/acceptance/fixture",
"screenshotsFolder": "test/acceptance/screenshots",
"videoFolder": "test/acceptance/videos",
"pluginsFile": "test/acceptance/plugins/index.js"
}
From what I can tell this seems to be doable and in the docs https://github.com/TheBrainFamily/cypress-cucumber-preprocessor#configuration they give the options above in the "cypress-cucumber-preprocessor" where they also suggest the nonGlobalStepBaseDir should point at the same path as the integration folder (which I have done... except that all of this doesn't work. The error I am getting is the step definitions are not found (will post only one of the failed test for shortness, the others are the same)
Running: features/product-catalogue/product-catalog.feature (4 of 8)
Product Catalog
1) A customer is able to see the product catalog
2) The product catalog displays the navigation bar
3) The catalogue page view defaults to grid on mobile
0 passing (2s)
0 pending
3 failing
1) Product Catalog
A customer is able to see the product catalog:
Error: Step implementation missing for: products are available on the system
at Context.resolveAndRunStepDefinition (http://localhost:3000/__cypress/tests?p=test/acceptance/integration/features/product-catalogue/product-catalog.feature:12789:11)
at Context.eval (http://localhost:3000/__cypress/tests?p=test/acceptance/integration/features/product-catalogue/product-catalog.feature:12104:35)
Note that the internal directory structure used to work before when it was inside the traditional cypress/integration/features folder. At the end of the day I have just renamed things around and pushed one level down to test/acceptance/integration/features.
From what I can tell it's the plugin not able to get the base folder for the non-global step definitions, but I can't see why.
Any help would be highly appreciated, thank you
After running Hardhat tests in the console with npx hardhat test I decided that being able to set break points would help me iterate faster.
How can I get Webstorm to run the underlying functions started by npx hardhat test so that I can use the built in Debugger?
I've since discovered that hardhat runs mocha under the hood.
To debug in WebStorm you can:
delete your existing configurations
create a new mocha configuration
set any configurations in 'Node options'. Note: since I'm forking the main net it takes a while for tests to start so I added the --timeout 10000 because mocha's default timeout is only 2000ms
select the mocha package, WebStorm doesn't select it by default
set your test file pattern
add const {ethers} = require('hardhat'); to your test file because it is no longer injected by hardhat during run time.
If the green debug icon does not appear I had success in closing and reopening WebStorm.
At this point I could successfully set break points in my test file but not in the MyContract.sol file. This is not surprising given that the contract is compiled before its run.
Create or open the package.json file for your Hardhat project.
Add a test NPM run script and save the file. Your package.json should look something like this.
{
"name": "hardhat-project",
"scripts": {
"test": "hardhat test"
},
"devDependencies": {
"#nomiclabs/hardhat-ethers": "2.0.2",
"#nomiclabs/hardhat-waffle": "2.0.1",
"chai": "4.3.4",
"ethereum-waffle": "3.4.0",
"ethers": "5.4.4",
"hardhat": "2.6.0"
}
}
In the left gutter of the editor pane, a little play icon should appear, click it and then click Debug "test".
I go through the instructions in a little more detail here, but this is the general idea. https://allendefibank.medium.com/how-to-debug-solidity-contracts-in-webstorm-hardhat-2ea0d3c4d582
If you're use typescript you need to import ts-mocha instead of mocha
I have a few different environments in which I am running Cypress tests (i.e. envA, envB, envC)
I run the tests like so:
npm run cypress:open -- --env apiEndpoint=https://app-envA.mySite.com
npm run cypress:open -- --env apiEndpoint=https://app-envB.mySite.com
npm run cypress:open -- --env apiEndpoint=https://app-envC.mySite.com
As you can see, the apiEndpoint varies based on the environment.
In one of my Cypress tests, I am testing a value that changes based on the environment being tested.
For example:
expect(resourceTiming.name).to.eq('https://cdn-envA.net/myPage.html')
As you can see the text envA appears in this assertion.
The issue I'm facing is that if I run this test in envB, it will fail like so:
Expected: expect(resourceTiming.name).to.eq('https://cdn-envB.net/myPage.html')
Actual: expect(resourceTiming.name).to.eq('https://cdn-envA.net/myPage.html')
My question is - how can I update the spec files so that the correct URL is asserted when I run in the different environments?
I am wondering if there's a way to pass a value from the command line to the spec file to tell the spec file which environment I'm in, but I'm not sure if that's possible.
You can directly use the Cypress.env('apiEndpoint') in your assertions, so that whatever you're passing via CLI, your spec files has the same value -
expect(resourceTiming.name).to.eq(Cypress.env('apiEndpoint'))
And if you want to check that when you pass https://app-envA.mySite.com and the url you expect in the spec file is https://cdn-envA.net/myPage.html, You can use:
expect(resourceTiming.name).to.eq(Cypress.env('apiEndpoint').replace('app', 'cdn').replace('mySite.com', 'net') + '/myPage.html')
Your best bet, in my opinion, is to utilize environment configs (envA.json, envB.json, etc)
Keep all of the property names in the configs identical, and then apply the values based on the environment:
// envA.json file
"env": {
"baseUrl": "yourUrlForEnvA.com"
}
// envB.json file
"env": {
"baseUrl": "yourUrlForEnvB.com"
}
That way, you can call Cypress.env('baseUrl') in your test, and no matter what, the right property should be loaded in.
You would call your environment from the command line with the following syntax:
"cypress run --config-file cypress\\config\\envA.json",
This sets up the test run to grab the right config from the start.
Calling the url for login, for example, would be something like:
cy.login(Cypress.env('baseUrl'))
Best of luck to you!
Looking at this post (Cypress - How can I run test files in order), I have several scripts specified under testFiles in cypress.json but opening Cypress with npm run cypress still shows all of the scripts in my repo and nothing happens.
Is there a way/trick to (a) automatically running the files specified in cypress.json in the UI and (b) a way to "toggle back" to all of the files in the repo (because I have some scratch files I use to isolate test features and additional tests that will eventually be added to the list)
The section from my cypress.json looks like
"testFiles:" :[
"/venueadmin/events/venueAdminCreateEvent.spec.js",
"/renter/renterInvalidLogin.spec.js",
"/renter/renterSignUp.spec.js"
]
etc.
Figured it out. There's a duplicate :
Also, if there's a slash pre-pended on the script name that needs to be removed
"testFiles" :[
"venueadmin/events/venueAdminCreateEvent.spec.js",
"renter/renterInvalidLogin.spec.js",
"renter/renterSignUp.spec.js",
I am using WebDriverIO and want to generate Allure Reports. I followed all steps mentioned in Allure
I did:
$ npm install wdio-allure-reporter --save-dev
package.json has:
"wdio-allure-reporter": "~0.0.2"
My wdio.conf.js:
reporters: [allure],
reporterOptions: {
allure: {
outputDir: 'allure-results'
}
},
When I do allure generate './allure-results' --clean
Report successfully generated to allure-report
But when go to /allure-report folder and open index.html, its a blank page. Also there is nothing in the .allure-results folder.
Can someone help please and direct in the right direction. What am I missing?
I had the same experience with allure when combining with wdio. No matter what combination I tried via https://docs.qameta.io/allure/latest/#_commandline, I kept getting a blank html report.
I found somewhere a mention of using serve instead of generate. I used the command ./node_modules/.bin/allure serve allure-results/. and VWOLAH! (?) It worked! It runs a local server with test results and data loaded in.
Although ... it doesn't seem to grab all test data, it seems to grab the very last test that ran and only that.
Use the following piece of code in the wdio.conf.js
reporters: ['allure'],
reporterOptions: {
allure: {
outputDir: 'allure-result',
disableWebdriverStepsReporting: true,
disableWebdriverScreenshotsReporting: false,
useCucumberStepReporter: false
}
},
Command to generate allure report
node_modules/.bin//allure generate allure-results/&& node_modules/.bin/allure open
#jazz, try updating the version of your wdio-allure-reporter.
In my package.json, I have "wdio-allure-reporter": "^0.1.2",