jestjs - how to parametrize test execution from cli in ci? - continuous-integration

i have 4 environments :
dev (developers area)
test (test area)
preprod (pre production environment)
production (production environment)
these environments needs different configuration to execute tests (differents urls, usernames, assets, and so on).
how to pass there configurations to jest as a parameter in continous integration?

As you can read here, jest would not permits to pass custom arguments you can use to handle custom configuration loaded at runtime.
i propose a workaround working for me.
create a configuration file, e.g. config.js
edit config.js and exports modules switching for the environment
switch (env) {
case "test":
module.exports = {
baseUrl: 'https://test.website.com'
}
break;
case "production":
module.exports = {
baseUrl: 'https://production.website.com'
}
break;
}
create a javascript files for every environment you need
test-configuration.js
production-configuration.js
edit these files writing in the environment variables
for example test-configuration.js will be
process.env.ENVIRONMENT = "test"
load configuration for your test files as it was a static file
const config = require('./config.js')
use jest setupFiles to add a setupFiles that load the environment variables.
for example, running
jest --setupFiles=./test-configuration.js
jest will load the test-configuration.js file that will set "test" on the "process.env.ENVIRONMENT" variables, so config.js file will "switch" on the "test" environment and all your test will use it.
so now you can (or CI can) loads configuration as needed.

For anyone facing the same issue – can't pass environment url to your custom setup file and tests. The solution might be dumb but it works without modifying the code much.
In package.json modify your scripts to export environment before running jest:
"scripts": {
"test": "jest",
"test:dev": "export ENVIRONMENT=https://dev.environment/ && jest",
"test:prod": "export ENVIRONMENT=https://prod.environment/ && jest"
}
Then you can access your code:
const page = await browser.newPage();
await page.goto(process.env.ENVIRONMENT);
console.log(process.env.ENVIRONMENT);

Related

Is there a way I can run a specified .spec file once all "x" .spec files have been run? Cypress

I want to do some clearing up after all .spec files have been run by Cypress. For this I created another .spec file that does several API calls. I need Cypress to run this .spec file only after all tests form all the other files have ran. One more thing, my .spec files are being run by Cypress in parallel mode, via 4 machines.
I found out there are the "after" hooks I could use, but as far as I read, these hooks apply per only one .spec file, not all of them.
This is not exactly the answer you are looking for, but maybe this information can be useful to you in some way:
One of the antipatterns I have heard multiple times from Cypress gurus is cleaning the application state using after or afterEach hooks.
If I remember correctly, the reasoning was that if the test fails or some step of the after hook fails, it will never get to the end of the script, thus we cannot be sure that the application state is prepared for the future tests.
That's why it is suggested to use before or beforeEach hooks, to prepare application state right before running tests.
If you decide to run some scripts before or after tests in different specs, then the support file can be useful, because it is rendered before each spec:
https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file
As a final thought, if you want something to happen before or after running Cypress, then maybe the best way to accomplish this is to prepare a separate script and run it using whatever you use to run Cypress (node, docker, etc.).
Sorry that this answer got so long :)
To conclude my thoughts:
Better prepare the state for the tests, not clean up after them
Separate scripts can be run before or after tests, and they do not have to be inside Cypress (using node, bash, docker...)
Hope this helps!
To run a single spec out of process, take a look at the Module Api.
With this you can create a node script that can be run after all parallel processes have completed.
./scripts/e2e-run-cleanup.js
const cypress = require('cypress')
cypress
.run({
spec: './cypress/e2e/cleanup.cy.js', // your cleanup spec
})
.then((results) => {
console.log(results)
})
.catch((err) => {
console.error(err)
})
Some configuration:
cypress.config.js
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
baseUrl: 'http://localhost:1234',
excludeSpecPattern: ['cleanup.cy.js'], // exclude this one from the normal run
}
})
package.json
"scripts": {
...
"test:headless": "yarn cypress:run",
...
"cleanup": "yarn ./scripts/e2e-run-cleanup.js", // script to kick off
"posttest:headless": "yarn cleanup", // also after local run
// "post" prefix automatically
// runs this after "test:headless" script
...
}
main.yml
...
jobs:
install:
...
ui-chrome-tests:
...
cleanup:
...
needs: ui-chrome-tests # ensure tests have run
steps:
- run: yarn cleanup
You can use the After Run API for this use case:
module.exports = (on, config) => {
on('after:run', (results) => {
// run some code after the run
})
}
or
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
on('after:run', (results) => {
// run some code after the run
})
}
}
})

Using different URLs for local and CI testing

I want to use Cypress to test locally and on CI at the same time. On CI, I would like to test a production version of my application, which has a different baseUrl than the local version, obviously. I am using the https://github.com/bjowes/cypress-ntlm-auth package to ause windows authentication for the CI, and to do so I have to call cy.ntlm line in my tests. I want to make an IF function that calls the cy.ntlm line ONLY if the baseUrl matches the production one. If the baseUrl is localhost then I would like the cy.ntlm line to be ignored. So my bottom line questions are, how do I let cypress know that I want to use 2 different URLs and how do I pack that into an IF statement? Thank you
You can check the baseUrl to conditionally call cy.ntlm,
const baseUrl = Cypress.config('baseUrl')! // use non-null assertion operator
const isLocal = baseUrl.includes('localhost')
if (!isLocal) {
cy.ntlm(...)
}
When using Typescript with Cypress you will get complaints because Typescript has no idea if you have set the baseUrl configuration or not.
You can overcome that by adding ! after getting the baseUrl value.
Ref Typescript - Non-null assertion operator
I separated the steps to make it clearer.
Assuming your cypress config file has the baseUrl. You can then update the baseUrl using the CLI during run time. For this create two different scripts with the staging and production URL's in your package.json like this:
"scripts": {
"test:local": "cypress run --config baseUrl=https://example.com/staging",
"test:ci": "cypress run --config baseUrl=https://example.com/production"
}
Then to run the scripts in CI use npm run test:ci and for local use npm run test:local.

How to pass values from command line to cypress spec file?

I have a few different environments in which I am running Cypress tests (i.e. envA, envB, envC)
I run the tests like so:
npm run cypress:open -- --env apiEndpoint=https://app-envA.mySite.com
npm run cypress:open -- --env apiEndpoint=https://app-envB.mySite.com
npm run cypress:open -- --env apiEndpoint=https://app-envC.mySite.com
As you can see, the apiEndpoint varies based on the environment.
In one of my Cypress tests, I am testing a value that changes based on the environment being tested.
For example:
expect(resourceTiming.name).to.eq('https://cdn-envA.net/myPage.html')
As you can see the text envA appears in this assertion.
The issue I'm facing is that if I run this test in envB, it will fail like so:
Expected: expect(resourceTiming.name).to.eq('https://cdn-envB.net/myPage.html')
Actual: expect(resourceTiming.name).to.eq('https://cdn-envA.net/myPage.html')
My question is - how can I update the spec files so that the correct URL is asserted when I run in the different environments?
I am wondering if there's a way to pass a value from the command line to the spec file to tell the spec file which environment I'm in, but I'm not sure if that's possible.
You can directly use the Cypress.env('apiEndpoint') in your assertions, so that whatever you're passing via CLI, your spec files has the same value -
expect(resourceTiming.name).to.eq(Cypress.env('apiEndpoint'))
And if you want to check that when you pass https://app-envA.mySite.com and the url you expect in the spec file is https://cdn-envA.net/myPage.html, You can use:
expect(resourceTiming.name).to.eq(Cypress.env('apiEndpoint').replace('app', 'cdn').replace('mySite.com', 'net') + '/myPage.html')
Your best bet, in my opinion, is to utilize environment configs (envA.json, envB.json, etc)
Keep all of the property names in the configs identical, and then apply the values based on the environment:
// envA.json file
"env": {
"baseUrl": "yourUrlForEnvA.com"
}
// envB.json file
"env": {
"baseUrl": "yourUrlForEnvB.com"
}
That way, you can call Cypress.env('baseUrl') in your test, and no matter what, the right property should be loaded in.
You would call your environment from the command line with the following syntax:
"cypress run --config-file cypress\\config\\envA.json",
This sets up the test run to grab the right config from the start.
Calling the url for login, for example, would be something like:
cy.login(Cypress.env('baseUrl'))
Best of luck to you!

Heroku failed to load ./.env

My Problem
I am having trouble loading my environment variables on Heroku production.
When pushing to Heroku I get following error message during the build script:
Failed to load ./.env.
Current Setup
I am using a .env file in the root of my app locally. I can succesfully load my environment variables using the dotenv-webpack plugin as follows:
//webpack.config
const Dotenv = require('dotenv-webpack')
module.exports = {
// other settings...
plugins: [
new Dotenv(),
]
};
Loading the environment variables:
//server.js
require('dotenv').config();
console.log(process.env.MY_VARIABLE);
This works like a charm locally, but fails on Heroku.
Note: My config vars have been set on Heroku, so that's not the problem.
What I tried
I have already tried to force load the .env file from the root of my app like this:
new Dotenv({ path: path.resolve(__dirname, './.env') });
Someone also pointed out that the Heroku environment might be system wide environment variables so I tried to load them using:
new Dotenv({ systemVars: true });
Neither of these attempts worked for me.
My guess
I have noticed that Heroku saves their .env file under ./tmp/build_someRandomBuildId/.env. My guess is that the .env file is not on the root of the directory, hence why dotenv can't find it. There is also no way to hardcode the location of this file in my Webpack configuration as the build ID is randomized with every build. Is there a way to tell Webpack to look for the file in a dynamic location?
Today i stumbled upon this problem, i tried several solutions but none worked. My App was working locally but in production mode (heroku) it was not loading process.env correctly.
then i found this https://www.npmjs.com/package/dotenv-webpack
//webpack.config.js
plugins: [
new Dotenv({ systemvars: true }),
],
Just setting systemvars to true does the trick..
For now I have tested this using different keys for the .env file and the heroku dashboard; They are not connected, and they replace themself correctly in production or dev mode.
Use the package "dotenv-webpack" instead of "dotenv".
I hope this saves some time to anyone facing the same problem
I finally found the solution, leaving this here for others who have the same problem as I did.
I used dotenv-webpack to set my environment variables locally, which worked like a charm. Heroku on the other hand sets their environment variables automatically, so there is no need to set them yourself. There is no need to look for a .env file. All I had to do was split up my webpack.config in 2 separate files.
//webpack.dev
require('dotenv').config();
plugins: [
new Dotenv()
],
Load .env file locally.
//webpack.prod
require('dotenv').config();
plugins: [
new webpack.DefinePlugin({
'process.env': {
'YOUR_VARIABLE': JSON.stringify(process.env.YOUR_VARIABLE),
}
});
]
Get your environment variables from Heroku and write them to your own process.env
If you are not using Webpacks, the idea's solution is similar to the accepted answer.
Heroku works on "production" mode by default, so if the problem in Heroku is with Dotenv (which should not be used anyways in Heroku), disable the use of Dotenv in production time like this:
if (process.env.NODE_ENV !== 'production') {
require('dotenv').config() }
}
...and then, access env variables just by doing:
var someVar = process.env.SOME_VARIABLE;
Don't forget to set the environment variables on Heroku first by using Console Commandline in your app's dashboard, or with an app.json file.

In test runner cannot see tests in newly created folder - cypress

I created new folder 'apitests' in cypress project and created a JavaScript test file in it. It does not show up in the test runner.
I have used the default configuration in cypress.json as specified in Cypress documentation
"testFiles": "**/*.*",
I expected my new folder 'apitests' and JavaScript test file to show in the test runner. Here is the end result.
Here is the file structure.
You've put your apitests folder directly in cypress/, while Cypress by default looks in cypress/integration/ folder.
You can change that by using integrationFolder config option, but I'd personally just keep the spec files in cypress/integration as is the default.
I had to add this to cypress.config.js:
module.exports = defineConfig({
e2e: {
specPattern: [
'cypress/e2e/*.js',
'cypress/e2e/**/*.js'
]
}
});
I had it initially, but removed it, to hope that Cypress would automatically find all my tests. But that resulted in new tests not being added to the test-runner.
My Cypress-version: 11.x.x (and I updated to 12.0.2 as a debugging attempt).
The solution was found in the Cypress-documentation for config.
You've put your apitests folder directly in cypress/, while Cypress by default looks in cypress/integration/ folder.
You will get an idea about it

Resources