I'm trying to get our automation environment set up to run headless through Jenkins on a linux server. The automation is set up through Ruby and uses Cucumber. I've run into an issue trying to get the feature files to run.
I have several plug ins for Jenkins, Xfvb, Cucumber reports, Cucumber-plugin.
I have setup the code in Ruby set up correctly (at least I believe so) for one of the feature files to run as headless.
But I can't quite get to test that specific file. When I try to run the build after enable the Cucumber-plugin it tries to run a different feature file. It seems to want to run the entire project. I've tried to add a new profile in my cucumber.yml (name jenkins) file and specify a tag like below:
jenkins: --tags #DisposalFeeService --no-source --format pretty --format html --out results.html --format json --out reports.json
I only have 1 feature file that is tagged with that tag and my build runs a completely different feature file. The feature is failing for good reasons, nothing to do with the Jenkins set-up, it would fail anyways. I also haven't set up all the features to run as headless, nor do I want to at this time. I'm just trying to show that I can get a feature file to run in through Jenkins.
If it is helpful, here is my cucumber-plug in in Jenkins: Cucumber Jenkins plug in
Related
I am new to using JMeter with GitLab CI/CD. I have created loadTest suite and want to configure it in Gitlab CI/CD pipeline. I have run.sh file in my loadTest project which do following:
Create and set up test data
Download JMeter dependencies and plugins (if JMeter already exists, skip this step)
Run the test
Clean up test data
Questions:
Is it a good idea to download JMeter in your project folder while you run the script or use JMeter docker image and run the tests ? I am concern about keeping JMeter folder in GitLab repo and occupying unnecessary space.
How to install Jmeter on Gitlab runner machine ?
Thank you in advance.
If your run.sh script downloads JMeter you don't need to put JMeter into Github repo. Using Docker is also possible, you can either check existing images or create your own with your own test data and plugins.
Gitlab runners are normal Linux machines (if you choose the relevant executor), my expectation is that your run.sh should work just fine there as it is (or with minimal amendments, like installing Java prior to installing JMeter)
I am storing my cucumber html report under cucumber-html-reports folder and json files are storing under cucumber-json folder. Now I want delete/empty these two folders. I need a plug in to achieve this.
The correct way to do this is to add an npm script to clean up the report folder and execute it before the test run. Cypress has an example npm script on their website for this. https://docs.cypress.io/guides/tooling/reporters#Command-line-3
Usually, the CI/CD pipeline would run the clean up script before the test run kicks off. Alternatively, you could add a little bit of code to your plugins file to exec the npm script before the test run as well.
One sort of hacky option, there is a trashAssetsBeforeRuns option in the Cypress config (set to true by default). It's used to clean up the downloads, videos, and screenshots folders before a test run. If you map your reporter results to one of these directories that might work, although I can't guarantee success with it since I haven't done that.
I am building a CI pipeline with Bamboo and I run automated tests using Appium. I have successfully run the Appium server and test scripts but I would like to have results in for example xml file so I could pass it to Bamboo. Do you know any way to achieve this?
it might be a bit late to respond on your question, but I'm using Appium with node.js and mocha to write my tests. To create a proper xml with the results I'm doing:
mocha --recursive -R xunit test/ > test-reports.xml
(test/ is the folder containing all my test files). This is generating a proper xml. I haven't tested it with Bamboo but I assume it's capable of reading XUnit test reports.
I've been working with Cucumber and Watir for the past few months locally to run some regression tests on our application. I have several Cucumber feature files in one repository with one step file and one environment file. Recently I have moved from running this locally to running it on Jenkins along with the cucumber-jvm reports plugin.
I set the build steps up with the following:
cucumber --format json all/features/publish.feature > result_publish.json
cucumber --format json all/features/signin.feature > result_signin.json
cucumber --format json all/features/reports.feature > result_reports.json
When there are no failures with the tests all feature files run successfully one after the other. However, if there is a failure with the first test the build will fail and the subsequent tests will not run.
Is there any way I can force all feature files to run even if one of them fails?
Put the features in the same folder and run
cucumber --format json all/features/integration -o results.json
This will create one single report with all the tests and will run all the features regarding if they fail or not
I am new to CI and would like your thoughts and input on how to go about my problem. I would like to first start off that I have been wrestling with this for 2 days(and I don't have that much background in sys ad) so please play nice?(I am mainly a front-end web dev) :)
Basically my plan was to install jenkins then make a CI env with these steps:
poll for any changes to github
if there are, run the build script:
a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my repo?)
b. run cucumber
c. if all tests pass go to 3, else fail
run any rake setup stuff
run the server(deploy)
I have done some of the stuff by cheating:
in my local, i switch my rvm to the correct one i need(rvm use 1.8.7-p174#mygemset)
run jenkins(java -jar jenkins.war) so that it gets the RVM ruby as default
run spork in a separate terminal(because for some reason my cucumbers don't run without spork - that's another problem)
build the project manually by clicking Build
so basically, I want to automate these stuff. Maybe what I need is a set of steps to follow(general or specific, depending on your taste) so I can setup my CI up and running.
Keep in mind that my "cheats" won't do as I want to test different projects with different setups and the startup cheat just won't do. Currently, my project build was successful because all I did was to run cucumber(and all my cukes pass). I want it to be able to deploy after it passes so maybe some help there also? Thanks
Okay I will try and help you as best I can.
poll for any changes to github
This can be easily done with the Github Plugin located here
if there are, run the build script: a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my
repo?) b. run cucumber c. if all tests pass go to 3, else fail
Then all you would is run the build script you have configure in the in the build from
Select "Add Build Step" -> "Execute shell".
You can either do that which is probably what I would do because when you create build you want them to be portable so you can start up in new jenkins instances, so you dont have to setup your build machine, with build specific files.
Then you run your tests, if they fail the build should fail regardless here is some information on running ruby on rails tests. if you need to manually fail a build in a script based on a result usually exiting a script with non-zero will fail the build. If not continue and run your rake and deployment scripts.
Just a few notes on Jenkins it wont do everything for you but if you can do it manaually Jenkins can automate it. So anything you have setup running manually with a little bit of effort you can get up and running automated with Jenkins
Here is another answer you might find helpful in your general setup and ideology behind Jenkins.
Goodluck!