For now I have the following code...
it("takes a long time", function(done) {});
Problem is when it times out I get the following message...
Error: Timeout - Async callback was not invoked within timeout
specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
Can I change this message to something more specific?
That is an error regarding your test, not an error of the code you're testing. If your test is properly written, you shouldn't encounter that error. The test itself is timing out. If you're writing a test to see how/whether some code times out, you would write the test to handle that, rather than the test just throwing an error because you didn't call done() in time.
// this function times out if passed "true" as second argument
function testThis(done, shouldTimeout) {
setTimeout(function() {
shouldTimeout || done();
}, 500);
}
it("timed out", function(done) {
var timedOut = false;
setTimeout(function() {
testThis(function() {
// this shouldn't be called because we're telling `testThis` to timeout
timedOut = true;
}. true);
});
setTimeout(function() {
expect(timedOut).toBeFalsy();
done();
}, 505);
});
If you needed to wait a long time to see if something times out, you can change how long Jasmine will wait before a test throws a timeout error by changing jasmine.DEFAULT_TIMEOUT_INTERVAL = 10000;
Jasmine does have a mock for setInterval and setTimeout that let you run that code synchronously, and saves all that wasted time waiting.
Related
After reading a lot of docs and try to find a solution to my problem I didn't find anything, so here we go.
I have the following problem testing my end to end flow, the flow I'm testing does launch requests continuously but in one case I expect these requests to stop. In other words, I want to throw an error if the request is made and continue with errors when hits the timeout without any request.
cy.wait('#my-request', { timeout: 20000 })
I expect this to timeout if the app works fine so I tried to do this.
cy.wait('#my-request', { timeout: 20000 })
.its('status').should('not.eq', 404)
.its('status').should('not.eq', 200);
I expected to execute the chained tasks but this only happens when the request is made, as well as tried to use a .then but I have the same problem.
Adding a global on fail event can help us but also limits to not execute additional code when this test is failing and we force it to be marked as done.
In the test definition we can add the done callback like in the example.
it('Description', (done) => {
// other test stuff
cy.on('fail', (err) => {
if (err.name === 'CypressError' && err.message.includes('routeAlias') && err.message.includes('Timed out')) {
done();
return true;
}
throw err;
});
cy.wait('#routeAlias', { timeout: 20000 })
.then(() => {
throw new Error('Error request found.');
});
});
// Any remaining code won't be executed if you need to reset something you need to create a new step, like in my case I did a new step to click a cancel button and prepare the app for the next test.
Now our test passes when this specific error is caught but any other error will lead to a test error.
This kind of workarounds are not recommended by cypress but unless cypress adds a catch to manage some errors it's the only way to fix my problem.
If I have the following module:
module.exports = kontinue => {
Promise.resolve({error:null})
.then(o => {
console.log('promise resolved');
// say something goes wrong here
if(true)
kontinue({error:'promise resolved but something else went wrong'});
else kontinue(o);
})
.catch(error => {
console.log('caught error');
kontinue({error:'promise rejected, or resolved but then continuation threw exception'})
});
};
And the following test:
const assert = require('assert').strict;
const target = require('./the_above_code.js');
it('should not timeout', (done) => {
target((sut) => {
console.log('continuation called');
assert.ok(false); // the test for sut.error === what I expected was failing
done();
});
});
It outputs:
promise resolved
continuation called
caught error
...
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
I realise this is because the .catch() is returning a new promise which is not resolving, but that's not what I really want during testing.
How do I test the object a promise resolves to, fail the test if necessary, have Mocha report that failure?
Perhaps there is somewhere else other than in the continuation (which never returns in the code that uses this module) that I can put the tests?
I'm sure monads can reduce the amount of boilerplate code here, but using them surely would violate Kernighan's maxim.
I have a redux-observable epic that polls an endpoint, getting progress updates until the progress is 100%. The polling interval is acheived using debounceTime like so:
function myEpic(action$, store, dependencies) {
return action$.ofType('PROCESSING')
.do(action => console.log(`RECEIVED ACTION: ${JSON.stringify(action)}`))
.debounceTime(1000, dependencies.scheduler)
.mergeMap(action => (
dependencies.ajax({ url: action.checkUrl })
.map((resp) => {
if (parseInt(resp.progress, 10) === 100) {
return { type: 'SUCCESS' };
}
return { checkUrl: resp.check_url, progress: resp.progress, type: 'PROCESSING' };
})));
}
This works fine but I'd like to write an integration test that tests the state of the store when progress is at 25%, then at 50%, then at 100%.
In my integration tests I can set dependencies.scheduler to be new VirtualTimeScheduler().
This is how I'm trying to do it at the moment (using jest):
describe('my integration test', () => {
const scheduler = new VirtualTimeScheduler();
beforeEach(() => {
// Fake ajax responses
const ajax = (request) => {
console.log(`FAKING REQUEST FOR URL: ${request.url}`);
if (request.url === '/check_url_1') {
return Observable.of({ progress: 25, check_url: '/check_url_2' });
} else if (request.url === '/check_url_2') {
return Observable.of({ progress: 50, check_url: '/check_url_3' });
} else if (request.url === '/check_url_3') {
return Observable.of({ progress: 100 });
}
return null;
};
store = configureStore(defaultState, { ajax, scheduler });
});
it('should update the store properly after each call', () => {
store.dispatch({ checkUrl: '/check_url_1', progress: 0, type: 'PROCESSING' });
scheduler.flush();
console.log('CHECK CORRECT STATE FOR PROGRESS 25');
scheduler.flush();
console.log('CHECK CORRECT STATE FOR PROGRESS 50');
scheduler.flush();
console.log('CHECK CORRECT STATE FOR PROGRESS 100');
});
});
My expected output would be:
RECEIVED ACTION: {"checkUrl":"/check_url_1","progress":0,"type":"PROCESSING"}
FAKING REQUEST FOR URL: /check_url_1
CHECK CORRECT STATE FOR PROGRESS 25
RECEIVED ACTION: {"checkUrl":"/check_url_2","progress":25,"type":"PROCESSING"}
FAKING REQUEST FOR URL: /check_url_2
CHECK CORRECT STATE FOR PROGRESS 50
RECEIVED ACTION: {"checkUrl":"/check_url_3","progress":50,"type":"PROCESSING"}
# CHECK CORRECT STATE FOR PROGRESS 100
But instead the output I get is
RECEIVED ACTION: {"checkUrl":"/check_url_1","progress":0,"type":"PROCESSING","errors":null}
FAKING REQUEST FOR URL: /check_url_1
RECEIVED ACTION: {"checkUrl":"/check_url_2","progress":25,"type":"PROCESSING","errors":null}
CHECK CORRECT STATE FOR PROGRESS 25%
CHECK CORRECT STATE FOR PROGRESS 50%
CHECK CORRECT STATE FOR PROGRESS 100%
At which time the test finishes. I'm configuring the store so that I can mock ajax requests and the scheduler used for debounceTime like as recommended here
So my question is how can I test the state of my store after each of the three ajax requests?
Interestingly enough, I played around with your code and am fairly confident you just found a bug in the debounceTime operator, which causes the apparent swallowing the scheduled debounce. The bad news is that even if that bug is fixed, you're code still wouldn't do what you're looking for order wise.
Bear with me as shit is about to get real:
Epic receives action PROCESSING and schedules debounce, yielding execution to your test
Your test calls scheduler.flush() and the VirtualScheduler executes the scheduled debounce work, which will pass along the original PROCESSING action to the mergeMap
Fake ajax is made, which synchronously emits a response
Response is mapped to the second PROCESSING action
Your epic emits that second action synchronously
The second action is recursively received by your epic and given to the debounce
The debounceTime operator now schedules that second action on the VirtualScheduler but the debounceTime operator is in the middle of executing the previously scheduled work still from the first action.
The call stack unwinds a bunch up until it reaches inside the previously scheduled debounce work from the first action that had just next()'d the first action. The rxjs code for debounceTime then sets this.lastValue = null and this.hasValue = false This is the rxjs bug, it needs to be done before nexting into the destination
The stack unwinds some more to the running flush() method of the VirtualScheduler, which now dequeues the second scheduled debounced action because it was added the scheduled work array synchronously, before this the flushing finished. Remember, we've only called scheduler.flush() ONCE so far, which is the function we're in back in at this point.
The second scheduled debounce work is run, but this.hasValue === false because the first scheduled one set it, so the debounceTime operator does not emit anything.
Stack unwinds to our first scheduler.flush()
We console.log('CHECK CORRECT STATE FOR PROGRESS 25')
All the other scheduler.flush() calls do nothing as there's nothing scheduled.
This is technically a bug, but it's not surprising that no one has run into it since running debounce synchronously without any delay defeats the point of it, except when you're testing, of course. This ticket is basically the same thing and OJ says RxJS doesn't make reentrancy guarantees, but I that might be up for debate in this case. I've filed a PR with the fix to discuss
Remember, this bug wouldn't have solved your underlying question about the ordering, but would have prevented the actions from being swallowed.
Off the top of my head I'm not sure how you would do what you'd like to do specifically if you want to maintain 100% synchronous behavior (VirtualScheduler). You'd need some way of yielding to your test in between debounces. For me when and if I write integration tests I mock out very little, if anything. e.g. let the debounces actually debounce either naturally or by mocking out setTimeout to advance them quicker but still keeping them async which will yield back to your test allowing you to check the state, but making your test also async.
For anyone wanting to reproduce, here's the StackBlitz code I used
The answer was to re-write the test asynchronously. Also note-worthy is that I had to mock the ajax requests by returning an Observable.fromPromise rather than just a regular Observable.of, otherwise they would still get swallowed up by the debounce. Something along these lines (using jest):
describe('my integration test', () => {
const scheduler = new VirtualTimeScheduler();
beforeEach(() => {
// Fake ajax responses
const ajax = request => (
Observable.fromPromise(new Promise((resolve) => {
if (request.url === '/check_url_1') {
resolve({ response: { progress: 25, check_url: '/check_url_2' } });
} else if (request.url === '/check_url_2') {
resolve({ response: { progress: 50, check_url: '/check_url_3' } });
} else {
resolve({ response: { progress: 100 } });
}
}))
);
store = configureStore(defaultState, { ajax, timerInterval: 1 });
});
it('should update the store properly after each call', (done) => {
let i = 0;
store.subscribe(() => {
switch (i) {
case 0:
console.log('CHECK CORRECT STATE FOR PROGRESS 0');
break;
case 1:
console.log('CHECK CORRECT STATE FOR PROGRESS 25');
break;
case 2:
console.log('CHECK CORRECT STATE FOR PROGRESS 50');
break;
case 3:
console.log('CHECK CORRECT STATE FOR PROGRESS 100');
done();
break;
default:
}
i += 1;
});
store.dispatch({ checkUrl: '/check_url_1', progress: 0, type: 'PROCESSING' });
});
});
I also set the timer interval to 1 by passing it as a dependency. In my epic I set it like this: .debounceTime(dependencies.timerInterval || 1000)
Here's the current test:
describe('/POST Register Page', function() {
it('it should register new user', function(/*done*/) {
chai.request(server)
.post('/auth/register')
.send(new_user_data)
.end(function(res) {
expect(res).to.have.status(2017);
// done();
})
})
})
The last I checked, there's no http code as 2017, however, it still passes:
Registration
Get register page
GET /auth/register 200 6.989 ms - 27
✓ it should get register page
/POST Register Page
✓ it should register new user
2 passing (147ms)
I want to simply post something, then get a response back, and play with the response.
If I include the done(), I get the timeout error:
1) Registration /POST Register Page it should register new user:
Error: Timeout of 3000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
I can't get anything to work, as for whether wrong or right, the tests pass.
Although this get request is passing as expected:
describe('Get register page', function() {
it('it should get register page', function(done) {
chai.request(server)
.get('/auth/register')
.end(function(err, res) {
expect(err).to.be.null;
expect(res).to.have.status(200);
done();
})
})
})
I'm new at this mocha-cum-chai-chai-http thing, and the experience so far is weird.
Thanks.
Your POST request is probably taking longer than 3 seconds to complete, therefore mocha throws the timeout error.
You can try setting the timeout to a larger value like:
describe('/POST Register Page', function() {
// timeout in milliseconds
this.timeout(15000);
// test case
it('it should register new user', function(done) {
chai.request(server)
.post('/auth/register')
.send(new_user_data)
.end(function(res) {
expect(res).to.have.status(200);
done();
})
})
})
With some trial, you can figure out an optimum value of timeout to set in your tests.
When you don't use the done() callback, mocha simply skips the assertions without waiting for the actual response to arrive. Since the assertions in .end() block never get executed, mocha passes the test as it faces no assertions. I had faced something similar when I first started out with TDD, which I learned about the hard way.
Reference:
Because the end function is passed a callback, assertions are run
asynchronously. Therefore, a mechanism must be used to notify the
testing framework that the callback has completed. Otherwise, the test
will pass before the assertions are checked.
it('should for something', function check(done) {
browser.sleep(2000);
$('.csTx').isPresent().then(function(result) {
if(result) {
done();
} else {
xPage.clickBack();
check(done);
}
})
}, 30000);
Can someone explain how done() works and what this is for. I googled it but cannot find any information that would be easy enough for me to understand. I am automating with protractor and jasmine. please consider the above code.
You need to use done if your test creates a parallel TaskQueue in your test's Control Flow (read more about promises and control flow).
For example:
describe('Control Flow', function() {
function logFromPromise(text) {
var deferred = protractor.promise.defer();
deferred.then(function() {
console.log(text);
});
deferred.fulfill();
return deferred;
}
it('multiple control flows', function() {
setTimeout(function() {
logFromPromise('1');
});
logFromPromise('0');
});
}
Calling setTime creates a parallel Task Queue in the control:
ControlFlow
| TaskQueue
| | Task<Run fit("multiple control flows") in control flow>
| | | TaskQueue
| | | | Task <logFromPromise('0');>
| TaskQueue
| | Task <setTimeout>
Protractor thinks the test is "done" after 0 is printed. In this example, 1 will probably be printed after the test is completed. To make protractor wait for Task <setTimeout>, you need to call the done function:
it('multiple control flows', function(done) {
setTimeout(function() {
logFromPromise('1').then(function() {
done();
});
});
logFromPromise('0');
});
If you can, let protractor handle when the test is "done". Having parallel TaskQueues can lead to unexpected race conditions in your test.
Here is a sample describe that you can run and see what happens. I have to mention that I don't use Protractor so there might exist some additional considerations to be made concerning its specific capabilities.
describe('Done functionality', function(){
var echoInOneSecond = function(value){
console.log('creating promise for ', value);
return new Promise(function(resolve, reject){
console.log('resolving with ', value);
resolve(value);
});
};
it('#1 this will untruly PASS', function(){
var p = echoInOneSecond('value #1');
p.then(function(value){
console.log('#1 expecting...and value is ', value);
expect(value).toBe('value #1');
});
});
it('#2 this will NOT FAIL', function(){
var p = echoInOneSecond('value #2');
p.then(function(value){
console.log('#2 expecting... and value is ', value);
expect(value).not.toBe('value #2');
});
});
it('3 = will truly FAIl', function(done){
var p = echoInOneSecond('value #3');
p.then(function(value){
console.log('#3 expecting... and value is ', value);
expect(value).not.toBe('value #3');
done();
});
});
it('4 = this will truly PASS', function(done){
var p = echoInOneSecond('value #4');
p.then(function(value){
console.log('#4 expecting... and value is ', value);
expect(value).toBe('value #4');
done();
});
});
});
when running the test you will note the sequence: first promises #1, #2, #3 will be created and resolved one by one. Please note that expectation for #1 and #2 will not be run yet because promises are resolved asynchronously.
Then, since #3 test uses done, after #3 promise is created, functions for thens of all previous promises are evaluated: you will see '#1 expecting...' and '#2 expecting...', but jasmine won't care about that because tests #1 and #2 are already finished and everything concerning them done. Only after those #3 expectation is made and it will truly fail because jasmine does take care of everything that happens before done() is made.
And then you can watch #4 test normal flow -- creating promise, resolving, expectation, everything considered by jasmine so expectation will truly pass.
I haven't used Protractor. For Jasmine, my understanding is that done makes Jasmine wait but not in the traditional sense of timeout. It is not like a timer which is always run. I think done acts as a checkpoint in Jasmine. When Jasmine sees that a spec uses done, it knows that it cannot proceed to the next step (say run next spec or mark this spec as finished i.e. declare verdict of the current spec) unless the code leg containing done has been run.
For example, jasmine passes this spec even though it should fail as it doesn't wait for setTimeout to be called.
fit('lets check done',()=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//the spec should fail as i is 0 but Jasmine passes it!
},1000);
//jasmine reaches this point and see there is no expectation so it passes the spec. It doesn't wait for the async setTimeout code to run
});
But if my intention is that Jasmine waits for the the async code in setTimeout, then I use done in the async code
fit('lets check done',(done)=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//with done, the spec now correctly fails with reason Expected 0 to be truthy.
done();//this should make jasmine wait for this code leg to be called before declaring the verdict of this spec
},1000);
});
Note that done should be called where I want to check the assertions.
fit('lets check done',(done)=>{
let i=0;
setTimeout(function(){
console.log("in timeout");
expect(i).toBeTruthy();//done not used at the right place, so spec will incorrectly ypass again!.
//done should have been called here as I am asserting in this code leg.
},1000);
done();//using done here is not right as this code leg will be hit inn normal execution of it.
});
In summary, think of done as telling Jasmine - "I am done now" or "I'll be done when this code hits"