I am writing tests for back-end which uses MongoDB and Elasticsearch. The problem is that without wrapping test with setTimeout test fails, and it looks like elasticsearch can't index mock data into db before test. Here is the code:
let elasticSearch = require('elasticsearch');
let elasticClient = new elasticSearch.Client({
host: 'localhost:9200'
});
let app = require('./dist/app'); //path to my application
let supertest = require('supertest');
before((done) => {
elasticClient.index(elasticMockData, function() {
done();
});
});
beforeEach(() => {
request = supertest(app);
});
it('should get data from elastic', () => {
setTimeout(() => { // if I comment this timeout, test will fail
request.get('/api/elastic')
.end((err, res) => {
expect(res.body.hits.hits.length).not.to.equal(0);
})
}, 1000); // if I comment this timeout, test will fail
});
I think you will agree that timeout is not an elegant and nice solution, which slows every test to 1 second or more. Maybe, am I missing something?
Found a solution, maybe it will be useful for someone.
According to Elasticsearch docs:
By default, the document will be available for get() actions immediately, but will only be available for searching after an index refresh (which can happen automatically or manually).
So, in this case, done() should be called within another callback function:
before((done) => {
elasticClient.index(elasticMockData, function() {
elasticClient.indices.refresh(function (err: any, res: any) {
if (err) {
return;
}
done();
});
});
});
Related
We are using segment in our application and i need to implement an E2E test in order to verify the number of segment calls, i must be sure that every event will be called only once.
I've been searching for a while, i've found this command that verifies the number of api calls:
Cypress.Commands.add(`verifyCallCount`, (alias, expectedNumberOfCalls) => {
const resolvedAlias = alias[0] === `#` ? alias.substring(1) : alias;
cy.get(`${resolvedAlias}.all`, { timeout: 20000 }).then((calls) => {
cy.wrap(calls.length).should(`equal`, expectedNumberOfCalls);
});
});
I use this command after waiting for the api call:
cy.wait(`#${eventAlias}`, { timeout: 20000 })
.then((interception) => {
return JSON.parse(interception.request.body);
})
.then(() => cy.verifyCallCount(eventAlias, 1););
Here is also the place where i add my alias for the api call.
beforeEach(() => {
cy.intercept('POST', 'https://api.segment.io/v1', (req) => {
const body = JSON.parse(req.body);
if (body.hasOwnProperty('type') && body.type === SampleEvent) {
req.alias = eventAlias;
}
});
});
});
Using this approach, when i run the test on local environment, it passes without any problem. but the same test fails on github's actions. and this is the error:
AssertionError: Timed out retrying after 10000ms: Expected to find element: `eventAlias.all`, but never found it.
I think that the .get() command is not being executed after .wait(), i tried to change the order of the commands, but it's not helping.
How can i fix this problem in github actions?
Is there any other way to verify the number of api calls in cypress?
I appreciate any help, thank you.
The answer you used from here Verify number of times request was made is wrong.
The line const resolvedAlias = alias[0] === '#' ? alias.substring(1) : alias removes the initial #, but it needs to be kept.
Also the timeout in cy.get('${resolvedAlias}.all', { timeout: 20000 }) has no effect, it doesn't wait 20 seconds for all calls to happen.
In your test scenario there may be 0, 1, or 2 calls. You want to fail if there is 0 calls or 2 calls, and pass if there is exactly 1 call.
This is enough to fail if there is 0 calls
cy.wait(`#${eventAlias}`, { timeout: 20000 })
To fail if there are 2 calls, you must use a hard wait, then verify the call count
cy.wait(`#${eventAlias}`, { timeout: 20_000 })
cy.wait(2_000) // wait an interval for any extra call to occur
cy.get(`#${eventAlias}.all`)
.its('length')
.should(`equal`, 1); // if two calls happened in interval, fail here
I notice you mention github actions. I had similar problems when testing an API call in CI, the test runs much slower and cause flakiness.
I suggest mocking the response to get better, more consistent performance from your test.
Ref: Controlling the response
As a bonus, there is no need for any long timeout because your mock replies immediately.
beforeEach(() => {
cy.intercept('POST', 'https://api.segment.io/v1', (req) => {
const body = JSON.parse(req.body);
if (body.hasOwnProperty('type') && body.type === SampleEvent) {
req.alias = eventAlias;
// here send mock response without any network delay
req.reply({
headers: {
Set-Cookie: 'newUserName=Peter Pan;'
},
statusCode: 201,
body: {
name: 'Peter Pan'
}
})
}
});
});
})
it('tests there is only a single POST from app', () => {
cy.wait(`#${eventAlias}`)
cy.wait(100)
cy.get(`#${eventAlias}.all`).then((calls) => {
cy.wrap(calls.length).should(`equal`, 1);
});
})
Your goal is to ensure only 1 API call.
You will need the test to wait and see if a 2nd call occurs.
it('accurately test that only one API call happens', () => {
const numOfRequests = 1
cy.intercept('**/api/*', cy.spy().as('api-spy'))
cy.visit('/');
cy.wait(1000)
cy.get('#api-spy').its('callCount').should('equal', numOfRequests)
})
I tested with a simple page that deliberately calls twice, with a delay 100ms between calls,
<script>
fetch('api/1')
setTimeout(() => fetch('api/2'), 100) // delayed 2nd fetch we want to test for
</script>
Without the hard wait the test gives me a false pass.
I also tried inverting the logic, but it still needs a hard wait to test correctly
cy.intercept('**/api/*', cy.spy().as('api-spy'))
cy.visit('/');
cy.wait(1000)
cy.get('#api-spy').its('callCount')
.should('not.equal', 0)
.and('not.equal', 2) // false pass without hard wait
})
Counting inside the routeHandler that checks body.type
2nd alias for call count
before(() => {
cy.wrap(0).as('eventCount')
})
beforeEach(() => {
cy.intercept('POST', 'https://api.segment.io/v1', (req) => {
const body = JSON.parse(req.body);
if (body.hasOwnProperty('type') && body.type === SampleEvent) {
req.alias = eventAlias;
cy.get('#eventCount').then(count => {
cy.wrap(count + 1).as('eventCount')
})
}
});
});
});
it('checks the count', () => {
cy.visit('/');
cy.wait(1000)
cy.get('#eventCount')
.should('equal', 1)
})
Incrementing a global
let eventCount = 0;
beforeEach(() => {
cy.intercept('POST', 'https://api.segment.io/v1', (req) => {
const body = JSON.parse(req.body);
if (body.hasOwnProperty('type') && body.type === SampleEvent) {
req.alias = eventAlias;
eventCount += 1
}
});
});
});
it('checks the count', () => {
cy.visit('/');
cy.wait(1000)
.then(() => {
cy.wrap(eventCount)
.should('equal', 1)
})
})
When you want to get all of the alias calls, you will need to use # to signify the alias. So the custom command will need to be updated.
Cypress.Commands.add(`verifyCallCount`, (registeredAlias, expectedNumberOfCalls) => {
if(alias[0] !== '#') {
throw new Error ('alias does not start with '#')
}
cy.get(`${registeredAlias}.all`, { timeout: 20000 }).then((calls) => {
cy.wrap(calls.length).should(`equal`, expectedNumberOfCalls);
});
});
Usage
cy.intercept('call').as('call')
// some action to trigger call
cy.wait('#call')
// some other actions
cy.verifyCallCount('#call')
Is there any other way to verify the number of api calls in cypress?
This is a concise way to count the api calls and wait for them to finish.
You can pass a cy.spy() in as a "response" and you can use that to count the number of times the intercept was hit.
Using .should() in the Cypress assertion will wait until the expected number of requests to come back.
it('test', () => {
const numOfRequests = 5;
cy.intercept('https://api.segment.io/v1', cy.spy().as('api-spy'));
// Do something to trigger 5 requests
cy.get('#api-spy').its('callCount').should('equal', numOfRequests);
});
If there are a sequence of different endpoints you are waiting for such as /v1/login followed by a /v1/getData etc, the URL in the cy.intercept may need to use a wildcard.
For example:
cy.intercept('https://api.segment.io/v1/**')
i am new to cypress and i am trying to check if the element exists on a page once the api call is finished.
i do a http post to url 'things/thing1' and once this api finishes i want to check if span element is present on page.
i have tried something like below.
const setUp = () => {
cy.apiPatchSomethings(something1)
.then(() => {
cy.reload();
});
}
describe('Suite name', () => {
before(() => {
setUp();
});
it('test case', () => {
cy.contains('span');
}
});
the above code doesnt work. even before span element is seen on page it checks for span element.
if i use cy.wait(10000) like below it works
it('test case', () => {
cy.wait(10000);
cy.contains('span');
});
but i dont want to use cy.wait. is there some other way to solve this. could someone help me with this. thanks.
Cypress command cy.contains() when called with a single argument is looking for content,
Syntax
cy.contains(content)
cy.contains(content, options)
cy.contains(selector, content)
cy.contains(selector, content, options)
but I'm guessing you are looking for a span element, so use
cy.get('span')
or
cy.contains('span', 'my-content-in-span')
Assuming that's not the problem, just some arbitrary sample code...
Your can modify the setup function to return a promise, in order to wait for the reload.
const setUp = () => {
return cy.apiPatchSomethings(something1) // need a return here
.then(() => {
return new Cypress.Promise(resolve => { // inner return also
cy.reload()
resolve(true) // resolve will signal reload is finished
})
});
}
Because setup() is invoked inside before() Cypress will wait for the promise to resolve before proceeding.
Please don't add extra waits or timeouts, which is too often suggested. This will only lead to flaky tests.
Note if you don't mind ditching the setup() function, it becomes a lot simpler
describe('Suite name', () => {
before(() => {
cy.apiPatchSomethings(something1)
.then(() => cy.reload() ); // all commands here will be completed
// before the tests start
});
it('test case', () => {
cy.contains('span', 'my-content-in-span');
}
});
1.You can wait for span to be visible. The default timeout that cypress provides is 4 seconds.
cy.contains('span').should('be.visible')
2.If you want to give a custom timeout(eg. 10 sec) specific to this command, you can use:
cy.contains('span', { timeout: 10000 }).should('be.visible')
3.If you want to increase the timeout globally you mention this in your cypress.json file:
"defaultCommandTimeout": 10000
and, then just use:
cy.contains('span').should('be.visible')
Now, all your commands will have a default timeout for 10 seconds.
I'm building a web scraper that uses puppeteer. I'd obviously like to ensure that I don't break things as I work the kinks out and so I'm writing some implementation tests.
How would I go about testing out the code below? The issue is that newPage() is nested and I can't figure out how to create a spy for it.
Any ideas? Should I structure the code differently to make it easier to test (from what I've read this a big no-no). Happy to hear your suggestions.
//myFile
myFn(){
let browser = puppeteer.launch()
let page = browser.newPage();
}
describe('searchAddress', () => {
beforeEach(() => {
browserSpy = spyOn(puppeteer,'launch')
pageSpy = spyOn(puppeteer,'newPage') // <--- ????
})
it('should ensure the calls were made', async () => {
await myFn()
expect(sleepSpy).toHaveBeenCalled();
expect(pageSpy).toHaveBeenCalled();
});
});
In this case the spyOn(puppeteer,'launch') should return an object that contains a spy object for newPage call. I mean the followings:
describe('searchAddress', () => {
let newPageSpy;
let browserSpy;
beforeEach(() => {
// ARRANGE
newPageSpy = jasmine.createSpy();
let browserMock = { newPage: newPageSpy };
browserSpy = spyOn(puppeteer, 'launch').and.returnValue(browserMock);
});
it('should ensure the calls were made', async () => {
// ACT
await myFn();
// ASSERT
expect(newPageSpy).toHaveBeenCalled();
});
});
I have read that jest tests in the same tests file execute sequentially. I have also read that when writing tests that involve callbacks a done parameter should be used.
But when using promises using the async/await syntax that I am using in my code below, can I count on the tests to but run and resolve in sequential order?
import Client from '../Client';
import { Project } from '../Client/types/client-response';
let client: Client;
beforeAll(async () => {
jest.setTimeout(10000);
client = new Client({ host: 'ws://127.0.0.1', port: 8080 , logger: () => {}});
await client.connect();
})
describe('Create, save and open project', () => {
let project: Project;
let filename: string;
beforeAll(async () => {
// Close project
let project = await client.getActiveProject();
if (project) {
let success = await client.projectClose(project.id, true);
expect(success).toBe(true);
}
})
test('createProject', async () => {
project = await client.createProject();
expect(project.id).toBeTruthy();
});
test('projectSave', async () => {
filename = await client.projectSave(project.id, 'jesttest.otii', true);
expect(filename.endsWith('jesttest.otii')).toBe(true);
});
test('projectClose', async () => {
let success = await client.projectClose(project.id);
expect(success).toBe(true);
});
test('projectOpen', async () => {
project = await client.openProject(filename);
expect(filename.endsWith('jesttest.otii')).toBe(true);
});
})
afterAll(async () => {
await client.disconnect();
})
From the docs:
...by default Jest runs all the tests serially in the order they were encountered in the collection phase, waiting for each to finish and be tidied up before moving on.
So while Jest may run test files in parallel, by default it runs the tests within a file serially.
That behavior can be verified by the following test:
describe('test order', () => {
let count;
beforeAll(() => {
count = 0;
})
test('1', async () => {
await new Promise((resolve) => {
setTimeout(() => {
count++;
expect(count).toBe(1); // SUCCESS
resolve();
}, 1000);
});
});
test('2', async () => {
await new Promise((resolve) => {
setTimeout(() => {
count++;
expect(count).toBe(2); // SUCCESS
resolve();
}, 500);
});
});
test('3', () => {
count++;
expect(count).toBe(3); // SUCCESS
});
});
For sure it depends on test runnner configured. Say for Jasmine2 it seems impossible to run tests concurrently:
Because of the single-threadedness of javascript, it isn't really possible to run your tests in parallel in a single browser window
But looking into docs' config section:
--maxConcurrency=
Prevents Jest from executing more than the specified amount of tests at the same time. Only affects tests that use test.concurrent.
--maxWorkers=|
Alias: -w. Specifies the maximum number of workers the worker-pool will spawn for running tests. This defaults to the number of the cores available on your machine. It may be useful to adjust this in resource limited environments like CIs but the default should be adequate for most use-cases.
For environments with variable CPUs available, you can use percentage based configuration: --maxWorkers=50%
Also looking at description for jest-runner-concurrent:
Jest's default runner uses a new child_process (also known as a worker) for each test file. Although the max number of workers is configurable, running a lot of them is slow and consumes tons of memory and CPU.
So it looks like you can configure amount of test files running in parallel(maxWorkers) as well as concurrent test cases in scope of single worker(maxConcurrency). If you use jest as test runner. And this affects only test.concurrent() tests.
For some reason I was unable to find anything on test.concurrent() at their main docs site.
Anyway you can check against your environment by yourselves:
describe('checking concurrent execution', () => {
let a = 5;
it('deferred change', (done) => {
setTimeout(() => {
a = 11;
expect(a).toEqual(11);
done();
}, 1000);
});
it('fails if running in concurrency', () => {
expect(a).toEqual(11);
});
})
Sure, above I used Jasmine's syntax(describe, it) so you may need to replace that with other calls.
How can I test Observables with Jest?
I have an Observable that fires ~every second, and I want to test that the 1st event is correctly fired, before jest times out.
const myObservable = timer(0, 1000); // Example here
it('should fire', () => {
const event = myObservable.subscribe(data => {
expect(data).toBe(0);
});
});
This test passes, but it also passes if I replace with toBe('anything'), so I guess I am doing something wrong.
I tried using expect.assertions(1), but it seems to be only working with Promises.
There are some good examples in the Jest documentation about passing in an argument for the test. This argument can be called to signal a passing test or you can call fail on it to fail the test, or it can timeout and fail.
https://jestjs.io/docs/en/asynchronous.html
https://alligator.io/testing/asynchronous-testing-jest/
Examples
Notice I set the timeout to 1500ms
const myObservable = timer(0, 1000); // Example here
it('should fire', done => {
myObservable.subscribe(data => {
done();
});
}, 1500); // Give 1500ms until it fails
Another way to see if it fails using setTimeout
const myObservable = timer(0, 1000); // Example here
it('should fire', done => {
myObservable.subscribe(data => {
done();
});
// Fail after 1500ms
setTimeout(() => { done.fail(); }, 1500);
}, timeToFail);
My preferred way to test observables, without fake timers and timeouts, is to async, await and use resolves or rejects on an expected converted promise.
it('should do the job', async () => {
await expect(myObservable
.pipe(first())
.toPromise())
.resolves.toEqual(yourExpectation);
});
Update:
In Rxjs 7 and onwards, you can use lastValueFrom or firstValueFrom for the promise convertion
it('should do the job', async () => {
await expect(lastValueFrom(myObservable))
.resolves.toEqual(yourExpectation);
});
test('Test name', (done) => {
service.getAsyncData().subscribe((asyncData)=>{
expect(asyncData).toBeDefined();
done();
})
});
})
the correct way to test any RXJS observable (Jest or no) is to the use the TestScheduler in rxjs/testing:
e.g.:
import { TestScheduler } from 'rxjs/testing';
import { throttleTime } from 'rxjs/operators';
const testScheduler = new TestScheduler((actual, expected) => {
// asserting the two objects are equal - required
// for TestScheduler assertions to work via your test framework
// e.g. using chai.
expect(actual).deep.equal(expected);
});
// This test runs synchronously.
it('generates the stream correctly', () => {
testScheduler.run((helpers) => {
const { cold, time, expectObservable, expectSubscriptions } = helpers;
const e1 = cold(' -a--b--c---|');
const e1subs = ' ^----------!';
const t = time(' ---| '); // t = 3
const expected = '-a-----c---|';
expectObservable(e1.pipe(throttleTime(t))).toBe(expected);
expectSubscriptions(e1.subscriptions).toBe(e1subs);
});
});
From the RXJS marble testing testing docs.
Trying to convert observables, etc. into promises works fine if you have a simple observable. As soon as things become more complicated you are going to struggle without using marble diagrams and the correct testing library.
There are 2 approaches mentioned above
Taking argument done in our test and call it when we have tested.
Convert our observable to promise using firstValueFrom(myObs) or lastValueFrom(myObs). and use async await with them...
If we have multiple observables to test then we have to nest the observables in our test as we can call done() only once. In that case async await approach can come handy.
In this example when we call filter Customer all three observables emits values so we have to test all of them.
it('Filter Customers based on Producers- Valid Case Promise way ',async()=>{
service.filterCustomers('Producer-1');
await expect(firstValueFrom(service.customers$)).resolves.toEqual(['Customer-1']);
await firstValueFrom(service.customers$).then((customers:string[])=>{
expect(customers).toEqual(['Customer-1']);
expect(customers.length).toBe(1);
})
await expect(firstValueFrom(service.products$)).resolves.toEqual([]);
await expect(firstValueFrom(service.types$)).resolves.toEqual([]);
}).
Here's an Angular approach using fakeAsync
Suppose we have a FooService with an Observable closed$ that emit every time we call the dismiss() method of the service.
#Injectable()
export class FooService {
private closeSubject$ = new Subject<void>();
public close$ = this.closeSubject$.asObservable();
public dismiss() {
this.closeSubject$.next();
}
}
Then we can test the close$ emission like this
describe('FooService', () => {
let fooService: FooService;
beforeEach(() => {
TestBed.configureTestingModule({
providers: [FooService]
});
fooService= TestBed.inject(FooService);
});
it('should emit a close event upon calling dismiss()', fakeAsync(() => {
const callbackSpy = jest.fn();
fooService.close$.subscribe(() => {
callbackSpy();
});
fooService.dismiss();
tick();
expect(callbackSpy).toHaveBeenCalledTimes(1);
}));
});