TravisCI passes despite tests failing, Gulp setup with tests on Jasmine + Frisby - jasmine

I'm using the gulp-jasmine-node plugin to run my Jasmine/Frisby.js tests like so:
gulp.task('runJasmine', function() {
return gulp.src(['test/*spec.js'])
.pipe(jasmineNode({
timeout: 10000
}));
});
gulp.task('test', ['runJasmine']);
Running gulp test locally I get the following (snippet only):
2) Frisby Test: Get API root
[ GET http://localhost:8080/api ]
Message:
Error: Header 'content-type' not present in HTTP response
Stacktrace:
Error: Header 'content-type' not present in HTTP response
at null.<anonymous>
...
Finished in 0.042 seconds
1 test, 2 assertions, 2 failures, 0 skipped
My .travis.yml:
language: node_js
node_js:
- "0.12"
- "4.2.4"
- "5.3.0"
before_script:
- npm install -g gulp
script: gulp test
And the corresponding raw CI output on Travis (end snippet only):
Finished in 0.054 seconds
[31m1 test, 2 assertions, 2 failures, 0 skipped
[0m travis_time:end:00bb9734:start=1455472901504935117,finish=1455472903105906383,duration=1600971266
[0K [32;1mThe command "gulp test" exited with 0.[0m
Done. Your build exited with 0.
I'm not sure why the build is passing despite failures - is there a certain configuration in Travis when using gulp, depending on test tool used? Is it due to the process exiting with code 0, and if so, do I need to modify my gulp command to exit the process on failure (not ideal as would interrupt other tasks in the runner).
Can't seem to find this problem or identical setup pre-addressed. Please help! Also, this is my first Stack Overflow Q, so hopefully I've provided all info needed. Thanks :)

This answer is not related to jasmin/frisby. But still might help other people looking on the gulp exiting with 0 on failing tests and having travis ci pass.
Here's my gulp test
gulp.task('test', function () {
mochify('./src/js/__tests__/**/*.js', {
extension: ['.jsx', '.js'],
node: true,
path: ['./src/js/'],
watch: false,
})
.transform(babelify)
.transform(envify({
_: 'purge',
NODE_ENV: 'production',
}))
.on('error', function(){
// Exits gulp with error
process.exit(1);
}).bundle()
});
The part which I figured out was that you can manually exit gulp with an exit code non 0 via process.exit(1); and then travis will fail.

Related

Jest not running testing files where npm is

I'm running into an error in my testing environment for Typescript where jest <file-path> isn't working, but npm tests is. I have a sample file main.test.ts with the following test:
import * as main from './main';
test('is 1 + 1 = 2?', () => {
expect(1 + 1).toBe(2)
})
Running npm test I have no issues running my tests , and all are said to pass. However, running jest main.test.ts (and similar issues for jest ./src/main.test.ts) I am getting the following error:
Test suite failed to run
Jest encountered an unexpected token
To the best of my knowledge, npm is running jest so I don't see why this might be happening. Thanks

Cypress 10 vs Cucumber vs Esbuild compatibility

I used Cypress 10 for a while with Cucumber preprocessor. Everything worked without issues. Today I updated Cypress to the latest version 10.7.0 and also #badeball/cypress-cucumber-preprocessor to 12.2.0. I have there also #bahmutov/cypress-esbuild-preprocessor 2.1.3
In my cypress.config.js file I have:
e2e: {
baseUrl: 'http://localhost:4200',
specPattern: 'cypress/e2e/features',
setupNodeEvents(on, config) {
const createEsbuildPlugin =
require('#badeball/cypress-cucumber-preprocessor/esbuild').createEsbuildPlugin
const createBundler = require('#bahmutov/cypress-esbuild-preprocessor')
require('#badeball/cypress-cucumber-preprocessor').addCucumberPreprocessorPlugin
on('file:preprocessor', createBundler({
plugins: [createEsbuildPlugin(config)],
}));
}
},
But now, when I start Cypress test runner, all the test are still there (all feature files), but after I click on ANY test case, I have this error:
Error: Build failed with 1 error:
node_modules/common-ancestor-path/index.js:17:37: ERROR: [plugin: feature] Reduce of empty array with no initial value
at failureErrorWithLog (C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:1605:15)
at C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:1251:28
at runOnEndCallbacks (C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:1034:63)
at buildResponseToResult (C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:1249:7)
at C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:1358:14
at C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:666:9
at handleIncomingPacket (C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:763:9)
at Socket.readFromStdout (C:\Users\JS\Desktop\test\node_modules\esbuild\lib\main.js:632:7)
at Socket.emit (node:events:527:28)
at addChunk (node:internal/streams/readable:324:12)
at readableAddChunk (node:internal/streams/readable:297:9)
at Readable.push (node:internal/streams/readable:234:10)
at Pipe.onStreamRead (node:internal/stream_base_commons:190:23)
And also Cypress says: No tests found. Cypress could not detect tests in this file.
Seems the issue is with esbuild, but not sure. I tried to downgrade the cucumber preprocessor, then it is working again.
Seems it is a compatibility problem between cucumber and esbuild.
Each library is managed by someone else, they do not cooperate, so I am really thinking to stop using Cucumber, but only pure Cypress.

Storing Artifacts From a Failed Build

I am running some screen diffing tests in one of my Cloud Build steps. The tests produce png files that I would like to view after the build, but it appears to upload artifacts on successful builds.
If my test fail, the process exits with a non-zero code, which results in this error:
ERROR: build step 0 "gcr.io/k8s-skaffold/skaffold" failed: step exited with non-zero status: 1
Which further results in another error
ERROR: (gcloud.builds.submit) build a22d1ab5-c996-49fe-a782-a74481ad5c2a completed with status "FAILURE"
And no artifacts get uploaded.
I added || true after my tests, so it exits successfully, and the artifacts get uploaded.
I want to:
A) Confirm that this behavior is expected
B) Know if there is a way to upload artifacts even if a step fails
Edit:
Here is my cloudbuild.yaml
options:
machineType: 'N1_HIGHCPU_32'
timeout: 3000s
steps:
- name: 'gcr.io/k8s-skaffold/skaffold'
env:
- 'CLOUD_BUILD=1'
entrypoint: bash
args:
- -x # print commands as they are being executed
- -c # run the following command...
- build/test/smoke/smoke-test.sh
artifacts:
objects:
location: 'gs://cloudbuild-artifacts/$BUILD_ID'
paths: [
'/workspace/build/test/cypress/screenshots/*.png'
]
Google Cloud Build doesn't allow us to upload artifacts (or run some steps ) if a build step fails. This is the expected behavior.
There is an already feature request created in Public Issue Tracker to allow us to run some steps even though the build has finished or failed. Please feel free to star it to get all the related updates on this issue.
A workaround per now is as you mentioned using || true after the tests or use || exit 0 as mentioned in this Github issue.

Karma Start, Karma Init, npm test not exiting?

Is it normal behavior for karma start, karma init, npm test to not exit after running? I want to run additional commands but I have to open another window to do it.
I believe you should try below config parameter for Karma..
singleRun
Type: Boolean
Default: false
CLI: --single-run, --no-single-run
Description: Continuous Integration mode.
If true, Karma will start and capture all configured browsers, run tests and then exit with an exit code of 0 or 1 depending on whether all tests passed or any tests failed.
Source - http://karma-runner.github.io/1.0/config/configuration-file.html

Protractor + Jasmine 2

I have config and specs files at shared folder of remote server. I run tests from local machine via console:
C:\Users\User>protractor \remoteServer\AutoSPA\config\protractor.conf.js
With jasmine framework all works good, but if I set framework: 'jasmine2' in protractor.conf.js tests don`t run:
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://10.17.175.168:50910/wd/hub
Started
No specs found
Finished in 0.001 seconds
Uncaught exception: Error while waiting for Protractor to sync with the page: "angular could not be found on the window"
[launcher] Process exited with error code 1
EXAMPLE:
conf.js:
exports.config = {
specs: ['./test.js'],
capabilities : {'browserName' : 'chrome'},
framework: 'jasmine2',
jasmineNodeOpts: {showColors: true}
};
test.js in the same folder:
describe("test", function(){
it("navigate", function(){
browser.get("https://angularjs.org/")
})
});
if it's local folder it works correctly, but if it's remote folder it doesn't find spec test.js.
I investigated and debug a little bit more and found out that Jasmine 1.3 used FS and Path for specs search, but Jasmine 2 used GLOB. Glob doesn't work with remote paths.

Resources