I'm using cucumber (ruby) + watir for running behavior driven tests, and I'm trying to do a couple of things for integrating it with Hudson CI:
Currently, the browser window pops up on the build slave each time the job runs. Is there a way to hide the browser window while the tests run?
I am using the jUnit formatter and having Hudson parse the test results. However, I'd like to print a test summary with the feature/scenario names and the test result (pass/fail) on the console window so that the email notification sent out lists the test summary. How do I do that?
Currently, cucumber prints a lot of debug output including deprecated function warnings. The -q flag doesn't seem to do anything for me. Is there a way I can stop cucumber from printing anything on the screen?
Edit: I'm running the tests on a Windows 7 desktop.
Related
I used to use Cypress 9 on previous projects.
By default, when running cypress open or cypress open --browser chrome used to run all tests for all React components.
However I installed Cypress 10 for the first time on a project that didn't have e2e tests yet. I added test specs, but I don't see any option to run all tests altogether.
It seems I have to run the tests one by one, clicking on each of them.
Can anyone please suggest how do I run all the tests automatically?
It's been removed in Cypress v10, here are the change notes related
During cypress open, the ability to "Run all specs" and "Run filtered specs" has been removed. Please leave feedback around the removal of this feature here. Your feedback will help us make product decisions around the future of this feature.
The feedback page to register your displeasure is here
You can create a "barrel" spec to run multiple imported specs.
I can't vouch for it working the same as v9 "Run all tests", but can't see any reason why not.
// all.spec.cy.js
import './test1.spec.cy.js' // relative paths
import './test2.spec.cy.js'
...
As #Constance says, reinstated in v11.20.
But still a very handy technique if you want to run a pre-defined subset of your tests.
In Cypress version 11.2.0 the Run All button has been reinstated.
You need to set experimentalRunAllSpecs to true in cypress.config.js.
Please see Configuration - End-to-End Testing
If Cypress Test Runner is not a must, I suggest to utilize the CLI/Node Cmd approach
You can trigger all the test(s) by npx cypress run(Still the video recording & screenshot on failed steps would be saved in the respective folders) to run all or with any other cypress flags to filter out specific spec files, or browser etc.
As per the feedback discussion there is a workaround the same as #Fody's answer that will achieve the same result as v9. Also worth noting though is the section on Continuous Integration and the Update 1 that includes a fix for preventing this workaround creating issues with the cypress run command.
Are there any current workarounds?
Yes. If you are impacted by the omission of this feature, it is possible to achieve the same level of parity as 9.x with a workaround Gleb Bahmutov explains here: https://glebbahmutov.com/blog/run-all-specs-cypress-v10/
This will still inherit the same problems as the previous implementation (which is why it was removed) but it will work in certain cases where the previous implementation was not problematic for your use case
https://github.com/cypress-io/cypress/discussions/21628#discussion-4098510
It was removed because people used it wrong.
The Test Runner is for debugging single tests. But by running all tests, then performance will quickly become a problem and crash the entire suite.
Running all tests should only be performed from the CLI.
Sources
https://github.com/cypress-io/cypress/issues/681
https://github.com/cypress-io/cypress/discussions/21628
I'm using default XCTest test framework to write and test application & I could able to do that. But to generate Test-Case report I'm not seeing any option except Xcode Server. Is there any other way to generate Test Case Report!! Any Suggestion !
when u run your test, a report is generated in
/Users/username/Library/Developer/Xcode/DerivedData/projectname/Logs/Test/TestSummary.plist
there you will get your all test information. But if you want a html format report you have to use
xcpretty.
let me know if you want to use xcpretty for html report or u can do your job using the TestSummary.plist
If you run your tests using fastlane/scan, an HTML report will be generated at the end of each test run as standard. It will open automatically in your browser when the tests are finished unless you tell it not to.
Scan is an easy way to simplify your test runs and make them configurable. The report is generated by xcpretty.
Is it possible to find out why the tests on the jenkins server failed? Meaning like an error message? I am new to testing and although I did some front end testing with Chutzpah, this is a totally new concept. Could someone please throw a light on this?Also, is there anyway to debug my tests in jenkins?
I am running my tests on visual studio and all the tests are passing but when I run them on jenkins, I see that some of them are failing and some of them are passing
There should be a "Console log" link on each test; click on that.
As Oli Charlesworth sad use the "console log" (+1) there you get (nearly) same errors like in VS. Mostly I scroll down to the end of the log and search for the error bottom up, because the error is usually one of the last actions.
I do not know anyway how you can debug tests in Jenkins.
I am doing Test Driven Development using the Play Framework and want to keep run the current failing test quickly. I am finding clicking Start! to be to slow since I have a mostly keyboard driven workflow. So ideally I would like a way to just reload the page and have it rerun the currently failing test.
I ended up leaving play test running and reloading the following page - note the auto=yes parameter
http://localhost:9000/#tests?select=Application.test.html&auto=yes
My Google-fu is weak and so I turn to the hive mind...
I have a Selenium script I originally developed in the IDE, which I am now trying to adapt to run on IE8 via Selenium-RC. I'm trying to debug an error where an element is not being found.
The problem is that as soon as the error occurs, the script exits and RC closes the browser. This makes it a bit hard to poke around with Firebug, etc, to see what is going on under the covers.
In the IDE, I could simply set a breakpoint. Is there a way to freeze the state of RC and the browser when an error occurs?
Any other ideas?
Thanks!
Depending on how you're running your RC tests, you could set a breakpoint in your IDE and run the test in debug mode. For example I use Eclipse to run tests using Java. Another option is to put a long pause in your test, just make sure you remove it again afterwards..!
I am running selenium rc tests as part of my test suite, so I just fire up --pdb (in python) and script pauses when exception occurs - before tearDown that closes the browser is invoked.
I have also had a plugin where I took a screenshot when something went wrong.
If you are using JUnit and SeleniumTestCase, you can add an empty tear down method to your test code which should prevent the browser from shutting down by overriding the base SeleniumTestCase tear down method. I use this on failing tests which I need to examine the resulting browser state after a failure.
In these situations I comment out the selenium.close() and selenium.stop() commands in my code which stops the browser from closing.
IIRC screenshot is only supported for firefox, so that won't help in your case.