I am trying to generate tests report into teamcity, everytime built is made. I've researched the several methods and ended up finding Istanbul's teamcity reporter: teamcity
Now I am trying to integrate it into teamcity, but I've not found any instructions about it.
All you need to do is add a script target in your package.json that uses the reporter:
"script": {
...
"coverage:teamcity": "nyc --reporter=teamcity mocha ...etc..."
...
}
Then, in your build step, simply invoke it via command line npm run coverage:teamcity
Teamcity will automatically parse the output and generate the coverage summary for you.
Related
I have found if I only add karate core dependency and run my tests, they run fine and report is generated.
So what is importance of making a test runner class? I can run my karate tests without it as well. Kindly explain!
With Karate runner class , you can use #KarateOptions to include or exclude feature files that you want to run eg #KarateOptions(features = "classpath:FeatureFiles/test.feature" , tags = "~#Smoke") will run all feature files other than the one having #Smoke tag .
How to pass parameter to run Karate tests from cmd/terminal as maven project
If we want to run only 'Smoke' tests then code can be written as :
Open cmd/terminal
cd 'karate project path'
mvn test -Dkarate.options="--tags #Smoke classpath:FeatureFiles"
I have a test project that is built with Maven (Java). I can either execute the test from IntelliJ manually or from the command line by writing mvn test.
I put the project on CircleCI and it generated a yml file. And it was able to execute the tests on pipelines as well without any issue at first
Then I made something stupid. Initially tests were in this root: src/main/java/api/test. But I decided to move the test class to this root: src/test/java/api.
I did this change because src/test/java was the actual test folder created automatically when you create a Maven project. (The other test folder was created by me manually so I thought this was not the best practice and therefore decided to move the test class to src/test/java. Basically what I did is, I created a package named api under src/test/java and moved the test class to this package. After that I deleted src/main/java/api/test as it is empty now.
After this change, I didn't observe any issue. I could still run the tests from IntelliJ or from the command line by mvn test command. But after I commit my changes, I just checked the pipelines and saw this:
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.793
My project is still built and tested on CircleCI but it obviously does not execute the test class. I dont know why it is happening. I checked the yml file and there is nothing related with paths and it was working before. Here is my yml file:
# Use the latest 2.1 version of CircleCI pipeline process engine.
# See: https://circleci.com/docs/2.0/configuration-reference
version: 2.1
jobs:
# Below is the definition of your job to build and test your app, you can rename and customize it as you want.
build-and-test:
# These next lines define a Docker executor: https://circleci.com/docs/2.0/executor-types/
# You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
# Be sure to update the Docker image tag below to openjdk version of your application.
# A list of available CircleCI Docker Convenience Images are available here: https://circleci.com/developer/images/image/cimg/openjdk
docker:
- image: cimg/openjdk:11.0
steps:
# Checkout the code as the first step.
- checkout
# Use mvn clean and package as the standard maven build phase
- run:
name: Build
command: mvn -B -DskipTests clean package
# Then run your tests!
- run:
name: Test
command: mvn test
workflows:
# Below is the definition of your workflow.
# Inside the workflow, you provide the jobs you want to run, e.g this workflow runs the build-and-test job above.
# CircleCI will run this workflow on every commit.
# For more details on extending your workflow, see the configuration docs: https://circleci.com/docs/2.0/configuration-reference/#workflows
sample:
jobs:
- build-and-test
As I said the same yml file was working fine before I moved the test class. Now it probably can not locate my test file. What could be the problem here and how can I solve this? Any help is appreciated.
I have a NodeJs application. Currently I am using team city for build and deployment of this application.
Now I want to run unit test cases before deployment. I have used Mocha framework with Chai to write test cases.
I don't see any runner type for Mocha or Node Js in team city.
I know some plugin is needed to be installed on teamcity server.
does any one know what is the plugin and what steps I need to follow?
You don't have to install any specific TeamCity plugin, you have to use test reporter capable of writing TeamCity service messages, e.g. mocha-teamcity-reporter, which is just another npm package.
You'll get you tests consumed by TeamCity after you run mocha --reporter mocha-teamcity-reporter test in your build step, so Command-Line Runner may be used for this purpose.
It is a good practice to extract this command to a separate script in your package.json, e.g:
"test:ci": "mocha --reporter mocha-teamcity-reporter test"
and use npm run test:ci in your build step.
I've been working with Cucumber and Watir for the past few months locally to run some regression tests on our application. I have several Cucumber feature files in one repository with one step file and one environment file. Recently I have moved from running this locally to running it on Jenkins along with the cucumber-jvm reports plugin.
I set the build steps up with the following:
cucumber --format json all/features/publish.feature > result_publish.json
cucumber --format json all/features/signin.feature > result_signin.json
cucumber --format json all/features/reports.feature > result_reports.json
When there are no failures with the tests all feature files run successfully one after the other. However, if there is a failure with the first test the build will fail and the subsequent tests will not run.
Is there any way I can force all feature files to run even if one of them fails?
Put the features in the same folder and run
cucumber --format json all/features/integration -o results.json
This will create one single report with all the tests and will run all the features regarding if they fail or not
How do I run MSTest as part of my build process in TeamCity? What are the pitfalls?
This answer is specifically for TeamCity 7.1 on Windows, but may apply to other environments.
In your TeamCity build configuration, on the General Settings page
Artifact paths: Artifacts\MSTest => MSTest
Create a new Command Linebuild step
Custom script: if not exist Artifacts\MSTest mkdir Artifacts\MSTest
Create a new MSTestbuild step
List assembly files: **\bin\**\*.Tests.dll
Results file: Artifacts\MSTest\testResults.trx
Pitfalls
Using wildcards when specifying which test assemblies to run
You can use wildcards when specifying which test assemblies to run in the MSTest build step, although it is unclear exactly how they work. A bug report has been filed.
The build process doesn't stop when tests fail
Be aware that if some of your tests fail and the build is marked as failed, the MSTest build step itself does not fail. This causes problems if you have build steps after the MSTest build step which you don't want to run if you have test failures (e.g. it may not make sense to produce an installer or documentation of a build you know has bugs). The problem will hopefully be fixed in later versions of TeamCity.
If you want your build process to stop when you have test failures, you can create a new build step that uses the TeamCity REST API to detect if the current build has been marked as failed (remember that when tests fail, the build step is not marked as failed, but the build is), and then fail the current build step explicitly. Example:
Create a new Powershell build step
Script: Source code
Source code: See script below
Make sure your newly created build step comes immediately after your MSTest build step
Make sure every build step after this one has Execute step set to Only if all previous steps were successful
Script:
$xml = [xml](curl --request GET http://USERNAME:PASSWORD#HOSTNAME/httpAuth/app/rest/builds/%teamcity.build.id%)
Microsoft.PowerShell.Utility\Select-Xml $xml -XPath "/build" | % { $status = $_.Node.status }
if ($status -eq "FAILURE") {
throw "Failing build step on purpose"
}