I am building a CI pipeline with Bamboo and I run automated tests using Appium. I have successfully run the Appium server and test scripts but I would like to have results in for example xml file so I could pass it to Bamboo. Do you know any way to achieve this?
it might be a bit late to respond on your question, but I'm using Appium with node.js and mocha to write my tests. To create a proper xml with the results I'm doing:
mocha --recursive -R xunit test/ > test-reports.xml
(test/ is the folder containing all my test files). This is generating a proper xml. I haven't tested it with Bamboo but I assume it's capable of reading XUnit test reports.
Related
I'm trying to get our automation environment set up to run headless through Jenkins on a linux server. The automation is set up through Ruby and uses Cucumber. I've run into an issue trying to get the feature files to run.
I have several plug ins for Jenkins, Xfvb, Cucumber reports, Cucumber-plugin.
I have setup the code in Ruby set up correctly (at least I believe so) for one of the feature files to run as headless.
But I can't quite get to test that specific file. When I try to run the build after enable the Cucumber-plugin it tries to run a different feature file. It seems to want to run the entire project. I've tried to add a new profile in my cucumber.yml (name jenkins) file and specify a tag like below:
jenkins: --tags #DisposalFeeService --no-source --format pretty --format html --out results.html --format json --out reports.json
I only have 1 feature file that is tagged with that tag and my build runs a completely different feature file. The feature is failing for good reasons, nothing to do with the Jenkins set-up, it would fail anyways. I also haven't set up all the features to run as headless, nor do I want to at this time. I'm just trying to show that I can get a feature file to run in through Jenkins.
If it is helpful, here is my cucumber-plug in in Jenkins: Cucumber Jenkins plug in
We have a set of Jasmine tests that run successfully in the local web server. http-server.
We would like to run these tests from the command line during the TeamCity build process without having to start a webserver.
Opening the html file with chrome using --disable-web-security flag results in
Failed to load module script: The server responded with a
non-JavaScript MIME type of "". Strict MIME type checking is enforced
for module scripts per HTML spec.
Probably because script references with type="module" in the SpecRunner.html file
<script src="../src/js/App/app.js" type="module"></script>
Jasmine tests are coupled to DOM/jQuery so that they need to be run in a browser. The SpecRunner html page includes script references with type="module".
How can we run these tests during a TeamCity build and fail the build if there are any test failures?
Thanks.
Here's my solution. We are already using jest for react tests and jest docs say
If you are using Jasmine, or a Jasmine like API (for example Mocha),
Jest should be mostly compatible, which makes it less complicated to
migrate to.
npm install jest puppeteer jest-puppeteer http-server
jest: To run the tests using Jasmine API
puppeteer: To run tests in headless chrome browser
http-server: Local webserver to navigate to the jasmine spec runner page
jest-puppeteer: To configure starting the local server before running the tests and stopping it afterwards
Add jest-puppeteer-config.json to start the server like this
Jest Puppeteer integrates a functionality to start a server when
running your test suite. It automatically closes the server when tests
are done.
Add a jest-puppeteer test to navigate to Jasmine SpecRunner page and assert that there are no failures.
Update your jest configuration docs
Create npm script to a run jest test that in turn runs Jasmine tests in the browser.
At this point, one should be able to run browser Jasmine tests from the command line locally or on the build server.
Here is a screenshot that shows the files and test run results both locally and on TeamCity.
How can we run these tests during a TeamCity build and fail the build
if there are any test failures?
This solution enabled us to restore around hundred legacy browser-coupled Jasmine tests as part of the build with minimum effort (did not have to update the tests).
Feel free to suggest alternatives.
I'm running protractor flake on Jenkins for automation testing using the framework protractor cucumber. Each time the tests run, a json is outputted. Is there a way to delete the old json output or a bash command that can detect if protractor flake is rerun. I have protractor flake set to have 3 attempts. So if protractor flake reruns the 3rd time, I don't want the 1st and 2nd json output from protractor cucumber to be in the folder.
That way the cucumber report won't have flake tests in it.
This is the command I have right now on Jenkins when I do a build:
rm -rf e2e/reports
mkdir e2e/reports
npm install
node flake e2e/staging.protractor.conf.js
There is an onCleanup field/function in the protractor config. I recommend using that to write yourself a function that will delete the files in /e2e/reports. You can use the exitcode to delete the file if the first or second attempt fails. This will be a little tricky as you will need to find a way to keep track of which iteration you are on so that the report doesn't get deleted if the third run fails also.
See the comments in the protractor config for more details.
I've configured and am executing mocha tests in WebStorm, so I know the module is working properly. But I can't seem to make it run from a Bamboo task. The task runs with Success but there are 0 tests executed.
This is my configuration atm:
app/ is my working dir. Tried also with app/node_modules/mocha/bin/ and other possibilities. I am not sure which exactly is the Mocha executable of all the mocha named files in the module...
Or maybe the problem lies in the tests dir? I've got test files, respectively in app/test/unit/models/ and app/test/unit/services/. But in WebStorm I configured it with the general test dir - just /app/test. Configuring the Mocha task in Bamboo with the specific test folders did not yield result...
I believe the problem comes from wrong directory configurations in the task, but I've tried writing whatever paths already and I've got no idea what's missing or wrong...
I noticed from your screenshot that the "Parse test results produced by this task" box isn't checked. This is what tells Bamboo to parse the output of the tests that you run.
I've been working with Cucumber and Watir for the past few months locally to run some regression tests on our application. I have several Cucumber feature files in one repository with one step file and one environment file. Recently I have moved from running this locally to running it on Jenkins along with the cucumber-jvm reports plugin.
I set the build steps up with the following:
cucumber --format json all/features/publish.feature > result_publish.json
cucumber --format json all/features/signin.feature > result_signin.json
cucumber --format json all/features/reports.feature > result_reports.json
When there are no failures with the tests all feature files run successfully one after the other. However, if there is a failure with the first test the build will fail and the subsequent tests will not run.
Is there any way I can force all feature files to run even if one of them fails?
Put the features in the same folder and run
cucumber --format json all/features/integration -o results.json
This will create one single report with all the tests and will run all the features regarding if they fail or not