We have a medium sized angular app with currently about 700 unit tests.
A few weeks ago, perfectly fine tests started to break. Even stranger: running the tests twice can yield to different results, i.e. different tests may break.
In the console, we always find the error :
Uncaught TypeError: You provided 'undefined' where a stream was expected.
But the stack trace gives no hint to where the root of the error is actually located (see end of this post). The stack trace shows a connection to the mergeMap operator, but it turns out that we use this operator no where in our app and nowhere in our tests.
I stepped through all spec files and let them run on their own (with fdescribe). Every single spec file passes without errors. Running them all together leads to the described breakage.
Of course my guess was that we were facing an async problem so I took the effort to go through all the tests and wrap each one of them in an async environment. I also checked that every subscription gets unsubscribed at some point - this was the case for our app but not always for our tests.
However, the error still persists.
It's a big issue for our project. Any advice is very welcome.
Maybe somebody knows a way to locate the part of our tests that is causing the problem?
We now use jasmine 3.3.0, karma v3.1.4 and Angular 7.1.3.
We did the update of jasmine and angular a week ago because we hoped to get rid of the problem. Only one thing changed: before the update, tests didn't break at random but at a fixed number of tests (in our case, 639 Tests would cause a test to break, 638, 640, 641... etc would pass; 648 would break again). I assume it has something to do with the random seed that jasmine is now using.
Here is the full stack trace:
<!-- language: lang-none -->
Uncaught TypeError: You provided 'undefined' where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
at subscribeTo (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeTo.js:41)
at subscribeToResult (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeToResult.js:11)
at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._innerSub (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:74)
at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._tryNext (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:68)
at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._next (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:51)
at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/Subscriber.js.Subscriber.next (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Subscriber.js:54)
at Observable._subscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeToArray.js:5)
at Observable.push../node_modules/rxjs/_esm5/internal/Observable.js.Observable._trySubscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Observable.js:43)
at Observable.push../node_modules/rxjs/_esm5/internal/Observable.js.Observable.subscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Observable.js:29)
at MergeMapOperator.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapOperator.call (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:29)
at ____________________Elapsed_3_ms__At__Thu_Dec_27_2018_10_03_35_GMT_0100__Mitteleurop_ische_Normalzeit_ ()
at Object.onScheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone-testing.js:108)
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:401)
at Object.onScheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:297)
at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:401)
at Zone.push../node_modules/zone.js/dist/zone.js.Zone.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:232)
at Zone.push../node_modules/zone.js/dist/zone.js.Zone.scheduleMacroTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:255)
at scheduleMacroTaskWithCurrentZone (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:1114)
at :9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:2090
Oooof, sounds like things have turned flaky. We had a run in with random breaking of unit tests recently. Have you been updating your Angular and Karma versions consistently?
What we ran into is that the way unit tests are setup by default (by the Angular CLI) has changed, and that older tests were not running the proper async ways.
The error you are seeing does differ from what we saw, but I'm certain this is an avenue worth exploring to remove any flakiness introduced by the unit tests setup.
As taken from https://angular.io/guide/testing#calling-compilecomponents
describe('BannerComponent', () => {
let component: BannerComponent
let fixture: ComponentFixture<BannerComponent>
beforeEach(async(() => {
TestBed.configureTestingModule({
declarations: [ BannerComponent ],
}).compileComponents(); // compile template and css
}));
beforeEach(() => {
fixture = TestBed.createComponent(BannerComponent)
component = fixture.componentInstance
fixture.detectChanges()
})
it('should create', () => {
expect(component).toBeTruthy()
})
Extra attention for the first beforeEach() which has an async() => {} in there, and a required .compileComponent().
The second beforeEach() is to define and populate the component variable within the shared context of the describe().
I hope this helps you figure out what is causing the flakiness. As the iterator issue stemming from RxJS seems to be pointing towards a test that is relying on state being set by a previous test, where it receives an input in the form of an Observable. If this Observable is set or defined later than the tests execution, you may be running into issues like the one you're describing.
It may be caused by async execution order of Jasmine test cases. In older versions of Jasmine, async execution order was set to false by default. But in recent versions of Jasmine, async execution order was set to true by default.
Primary Reasons: Variable has been overridden in other test case which has executed before this test case.
Solutions:
We need to find out why our variable is getting undefined. Put a console above statement where undefined is thrown. Reinitialize that variable using beforeEach.
Set random to false in your karma.config.js ex. https://github.com/karma-runner/karma-jasmine
Related
I am testing a large number of really long forms (ca 50 forms, most of them are 150+ fields), and one set of tests is checking whether all fields exist in each form. Using the latest Cypress 10 version.
Need ideas how to make the tests most bullet proof in two dimensions:
Test stability. As the forms are long, they consist of several sections on separate pages, but not all pages are loading with the same speed. I don't like adding cy.wait() everywhere, previously I also used checking if the page loader is rolling but here we don't have the loader. Are there any more nice ways to make sure the page has loaded before it tests the assertions?
Does skipping failed assertions make any sense or again there's something better to use if I want to get the list of all missing fields as a result of the test? Or is it not a good idea at all?
Thank you in advance and sorry for the long and theoretical question.
I guess you are performing cy.get() on each selector from a list of expected field selectors?
If you are concerned about page loading taking too long, just increase the timeout for each selector (there's no need to cy.wait() for the page).
Skipping failures makes sense, and you could do it with soft assertions but a more straight-forward way is to use cypress-if
An example
const fails = []
cy.fixture('field-selectors.json').then(selectors => {
cy.wrap(selectors).each(selector => {
cy.get(selector, {timeout:10_000})
.if()
.else() // fails go to the following command in chain
.then(() => fails.push(selector)
})
.then(() => {
if (fails.length) {
throw new Error('Some selectors failed: ' + fails.join(', '))
}
})
})
New cypress user here, I am aware that cypress does not handle variables like how testcafe and others do due to the asyn nature of it. Using the example given and what I could find I have this as an example:
cy.get('selector').invoke('text').as('text_needed')
cy.get('#text_needed')
const txtneeded = this.text_needed
cy.log(txtneeded)
This looks at a given selector, takes what it finds and uses it as text and set it as a variable usable at any time in the test and outputs it to the log. The plan is to use that text in a search filter in another page to find the item it references.
The problem is that it fails with Cannot read properties of undefined (reading 'text_needed')
Is this because the content of the selector is not assigned to text properly, the outer html is <a data-v-78d50a00="" data-v-3d3629a7="" href="#">PO90944</a> The PO90944 is what I want to capture.
Your help would be appreciated!
You cannot save an alias and access it via this.* in the same execution context (callback) because it's a synchronous operation and your alias is not yet resolved at this time.
This is a correct way to go:
cy.get('selector').invoke('text').as('text_needed')
cy.get('#text_needed').then(txtneeded => {
cy.log(txtneeded)
})
First, make sure to define it as traditional function, not as an arrow function as this context doesn't work as you'd expect there, more info here.
Next, typically in a single test you should use .then() callback to perform additional actions on elements, and use aliases when you need to share context between hooks or different tests, so please consider the following:
// using aliases together with this within the single test won't work
cy.get(<selector>).invoke('text').as('text_needed')
cy.get('#text_needed').should('contain', 'PO90944') // works fine
cy.log(this.text_needed) // undefined
// this will work as expected
cy.get(<selector>).invoke('text').then(($el) => {
cy.wrap($el).should('contain', 'PO90944'); // works fine
cy.log($el) // works fine
});
Setting alias in beforeEach hook for example, would let you access this.text_needed in your tests without problems.
Everything nicely explained here.
Edit based on comments:
it('Some test', function() {
cy.visit('www.example.com');
cy.get('h1').invoke('text').as('someVar');
});
it('Some other test', function() {
cy.visit('www.example.com');
cy.log('I expect "Example Domain" here: ' + this.someVar);
});
And here's the output from cypress runner:
I am trying mocha test case to check rest api, but the issue is if I have 3 test cases and first one fails then rest are not executing. It stops at first only.
Here is following code:
describe('suite 1',function(){
it('tc1',function(done){
// some test case with failure
should([1]).equal([]);
done();
})
it('tc2',function(done){
// some test case with success
should([]).equal([]);
done();
})
})
In above code I am not able to get report like
2 test cases.
1 Passed.
1 Fail.
It fails in middle, in here it fails on first test case only.
Both tests are failing because they both should fail.
The both tests fail because you are using equal which checks for object identity. That is, it uses === to check for the equality. Now, open an interactive Node session and try this:
[] === []
You'll get false. That's because each new empty array is a new JavaScript object and === will be true only if the two arrays are the same object.
Note that you get the result you expect in your first test but not for the reason you (probably) think. The test fails for the same reason I've just explained. The fact that one array contains an element but the other is empty is not taken into account by should.
You should use eql to test whether the arrays have the same members.
I'm writing some page-object driven tests using Protractor and Astrolabe.
Jasmine is being used to implement describe/it style specs.
Adding custom matchers won't work using this.addMatchers (TypeError: Object #<Object> has no method 'toContainLowered'), so I used this guide to implement them.
It seems to be working, until I look closely at the output of my test run:
$> grunt test:func
Running "test:func" (test) task
Running "shell:protractor" (shell) task
Using the selenium server at http://localhost:4444/wd/hub
..
Finished in 6.727 seconds
2 tests, 1 assertion, 0 failures
Here is my code:
var loginPage = require('./../pages/loginPage');
describe('Login page', function () {
var ptor = loginPage.driver;
beforeEach(function () {
jasmine.Matchers.prototype.toContainLowered = function (expected) {
return this.actual.toLowerCase().indexOf(expected) > -1;
};
loginPage.go();
ptor.waitForAngular();
});
it('should display login page', function () {
expect(loginPage.currentUrl).toEqual(ptor.baseUrl);
});
it('should display an error when the username or password is incorrect', function() {
loginPage.login('bad', 'credentials');
ptor.waitForAngular();
expect(loginPage.lblError.getText()).toContainLowered('invalid username and/or password');
// expect(loginPage.lblError.getText()).toContain('Invalid Username and/or Password');
});
});
If I uncomment the last line and remove the toContainLowered matcher, I get the proper output:
2 tests, 2 assertions, 0 failures
I'm having a really difficult time debugging this promise-based code, and any efforts to put a console.log(this.actual.toLowerCase().indexOf(expected) > -1); will print false, which is confusing.
I even tried replacing the entire function definition with just return false;. Which still does not do anything. Finally, I tried passing no argument to the matcher, which should have thrown an Invalid Argument Error or something.
How do I define my own matchers in Jasmine when using Protractor/Astrolabe tests?
I've had similar problems with matchers, in particular with the .not matchers, which all seem to not work. I hypothesise that protractor is extending the Jasmine matchers to deal with the promises, and that that extension hasn't been applied to the .not, or to the custom matchers.
In my case, I wanted a .not.toMatch, and so I just wrote a convoluted regex that gave me what I wanted, with the not embedded in the regex.
I note that your matcher is called "toContainLowered", so perhaps you're looking for lowercase, and therefore you could instead do this with a regex by using .toMatch?
The issue I raised on this on the protractor github is here: https://github.com/angular/protractor/issues/266
I also see, in this code file: https://github.com/angular/protractor/blob/master/jasminewd/spec/adapterSpec.js, that the last commit is marked as "patched matcher should understand not". That might either fix the custom matchers for you, or provide an indication of what needs to be done to fix that custom matcher.
EDIT: now looking further into that issue thread, I see you've already been there. Which makes my answer somewhat superfluous. :-)
I'm more than a little confused about Modernizr and it's relation to Yepnope.js. As I understand it, Modernizr comes with Yepnope.js (assuming you select the Modernizr.load() option). According to the Yepnope documentation, there are optional prefix plugins which can used. For example, you can test for versions of IE (assuming you also load the yepnope.ie-prefix.js script). However, when I attempt to run the following, I get 'undefined' alert:
Modernizr.load({
load: 'ie!my-ie-specific.js',
complete : function (url, result, key){
alert(url, result, key);
}
});
What am I doing wrong? Does Modernizr include Yepnope completely or only bits and pieces?
I got stuck on this too. Struggled for an hour.
The complete callback doesn't support tests. Change it to callback which fires when the external script is loaded.
complete runs when all scripts are loaded, OR if nothing is loaded.
I wish complete took the result variable though. I could use that.