So i have a script test.sh which firsts build a test docker image then runs the docker image which in turn runs the test. I'ld like to manually add a command to send the coverage result to coveralls.io
Can anyone help with a clue to the right direction i'm heading?
In general, manual submission to coveralls works as follows:
Add your repo in Coveralls, so that it's assigned a token.
Create a file .coveralls.yml which at minimum should contain this one line:
repo_token: "coveralls-token-for-your-repo"
Run tests in a way that generates some file with coverage information.
In Python, for example, you'd run python3 -m coverage ... and file .coverage will be created.
Submit to Coveralls, using a script that they provide.
Again, taking Python for example, you'd do:
$ pip3 install coveralls # a Python package for submitting to Coveralls
$ coveralls # yes, just this
The output might be:
Submitting coverage to coveralls.io...
Coverage submitted!
Job #10.1
https://coveralls.io/jobs/353453468453
The coveralls command will look for .coveralls.yml file in the current directory, so be sure to put it there.
If you're not using Python, however, for steps 3. and 4. you'll need to read about generating and submitting coverage data for your language in Coveralls docs. For selected languages the support is just as good as for Python, meaning that the commands might differ slightly, but it should still be a breeze.
Related
I am storing my cucumber html report under cucumber-html-reports folder and json files are storing under cucumber-json folder. Now I want delete/empty these two folders. I need a plug in to achieve this.
The correct way to do this is to add an npm script to clean up the report folder and execute it before the test run. Cypress has an example npm script on their website for this. https://docs.cypress.io/guides/tooling/reporters#Command-line-3
Usually, the CI/CD pipeline would run the clean up script before the test run kicks off. Alternatively, you could add a little bit of code to your plugins file to exec the npm script before the test run as well.
One sort of hacky option, there is a trashAssetsBeforeRuns option in the Cypress config (set to true by default). It's used to clean up the downloads, videos, and screenshots folders before a test run. If you map your reporter results to one of these directories that might work, although I can't guarantee success with it since I haven't done that.
I'm trying to get our automation environment set up to run headless through Jenkins on a linux server. The automation is set up through Ruby and uses Cucumber. I've run into an issue trying to get the feature files to run.
I have several plug ins for Jenkins, Xfvb, Cucumber reports, Cucumber-plugin.
I have setup the code in Ruby set up correctly (at least I believe so) for one of the feature files to run as headless.
But I can't quite get to test that specific file. When I try to run the build after enable the Cucumber-plugin it tries to run a different feature file. It seems to want to run the entire project. I've tried to add a new profile in my cucumber.yml (name jenkins) file and specify a tag like below:
jenkins: --tags #DisposalFeeService --no-source --format pretty --format html --out results.html --format json --out reports.json
I only have 1 feature file that is tagged with that tag and my build runs a completely different feature file. The feature is failing for good reasons, nothing to do with the Jenkins set-up, it would fail anyways. I also haven't set up all the features to run as headless, nor do I want to at this time. I'm just trying to show that I can get a feature file to run in through Jenkins.
If it is helpful, here is my cucumber-plug in in Jenkins: Cucumber Jenkins plug in
I'm running protractor flake on Jenkins for automation testing using the framework protractor cucumber. Each time the tests run, a json is outputted. Is there a way to delete the old json output or a bash command that can detect if protractor flake is rerun. I have protractor flake set to have 3 attempts. So if protractor flake reruns the 3rd time, I don't want the 1st and 2nd json output from protractor cucumber to be in the folder.
That way the cucumber report won't have flake tests in it.
This is the command I have right now on Jenkins when I do a build:
rm -rf e2e/reports
mkdir e2e/reports
npm install
node flake e2e/staging.protractor.conf.js
There is an onCleanup field/function in the protractor config. I recommend using that to write yourself a function that will delete the files in /e2e/reports. You can use the exitcode to delete the file if the first or second attempt fails. This will be a little tricky as you will need to find a way to keep track of which iteration you are on so that the report doesn't get deleted if the third run fails also.
See the comments in the protractor config for more details.
I have a number of unit tests written for my project, executed with CTest. I would like to integrate the results into my TeamCity build. I've downloaded and set up the plugin for my testing framework (Boost Test).
The problem that I have run into is that the tests run with CTest output to Testing/Temporary/LastTest.log, whereas TeamCity is trying to read the results from standard out. To get around this, my testing step is.
make test
cat Testing/Temporary/LastTest.log
which works, but feels like a hack.
Is there any way to get TeamCity to read from this file in addition to standard out? Alternatively, is there any way to tell ctest to output to standard out in addition to this LastTest.log file?
This question is similar, but I would like it to work for all output rather than just on failure: CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)
Teamcity has additional build features which allow to process CTest reports. I am not sure if it'll work or not but you could try adding an additional build feature in your build step to read CTest report.
I am new to CI and would like your thoughts and input on how to go about my problem. I would like to first start off that I have been wrestling with this for 2 days(and I don't have that much background in sys ad) so please play nice?(I am mainly a front-end web dev) :)
Basically my plan was to install jenkins then make a CI env with these steps:
poll for any changes to github
if there are, run the build script:
a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my repo?)
b. run cucumber
c. if all tests pass go to 3, else fail
run any rake setup stuff
run the server(deploy)
I have done some of the stuff by cheating:
in my local, i switch my rvm to the correct one i need(rvm use 1.8.7-p174#mygemset)
run jenkins(java -jar jenkins.war) so that it gets the RVM ruby as default
run spork in a separate terminal(because for some reason my cucumbers don't run without spork - that's another problem)
build the project manually by clicking Build
so basically, I want to automate these stuff. Maybe what I need is a set of steps to follow(general or specific, depending on your taste) so I can setup my CI up and running.
Keep in mind that my "cheats" won't do as I want to test different projects with different setups and the startup cheat just won't do. Currently, my project build was successful because all I did was to run cucumber(and all my cukes pass). I want it to be able to deploy after it passes so maybe some help there also? Thanks
Okay I will try and help you as best I can.
poll for any changes to github
This can be easily done with the Github Plugin located here
if there are, run the build script: a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my
repo?) b. run cucumber c. if all tests pass go to 3, else fail
Then all you would is run the build script you have configure in the in the build from
Select "Add Build Step" -> "Execute shell".
You can either do that which is probably what I would do because when you create build you want them to be portable so you can start up in new jenkins instances, so you dont have to setup your build machine, with build specific files.
Then you run your tests, if they fail the build should fail regardless here is some information on running ruby on rails tests. if you need to manually fail a build in a script based on a result usually exiting a script with non-zero will fail the build. If not continue and run your rake and deployment scripts.
Just a few notes on Jenkins it wont do everything for you but if you can do it manaually Jenkins can automate it. So anything you have setup running manually with a little bit of effort you can get up and running automated with Jenkins
Here is another answer you might find helpful in your general setup and ideology behind Jenkins.
Goodluck!