In xunit reporter of mocha, it generates the report for attributes tests, failures, skipped, error. but i want for mocha's pending, dropped and blocked reports also. is these reports are generated using xunit? can we customize the xunit reporter to generate the report which has
pending
dropped
blocked
test cases report. please help me to find solution for this.
Looks like this is a known issue and pending tests have been added to the xunit reporter.
First are you using the latest version of mocha?
https://github.com/visionmedia/mocha/pull/1051/files that pull request has been merged into the latest version of mocha and should including pending requests in xunit?
You can always fork mocha and edit lib/reporters/xunit.js and add
runner.on('dropped', function(test){
tests.push(test);
});
runner.on('blocked', function(test){
tests.push(test);
});
Or instead of forking mocha. Copy Xunit and make a custom mocha reporter. You can use https://github.com/startswithaj/mocha-spec-cov as a template.
Related
I'm wondering if anyone has a way of rerunning failed tests automatically on Team City? I'm using C#, XUNIT and Selenium for a suite of automated UI tests. They can be flakey and a simple rerun will pass the test on the second try. I can't seem to find a solution. I've used other frameworks before that allow you to pass a console param for reruns.
This is something you have to do at the level of your test runner, i.e. xUnit. You can consider xUnitRetry plugin (the link has installation documentation):
[Retry(5)] //will try 5 times
public void TryAFewTimes()
{
tried++;
Assert.True(tried >= 5);
}
I'm not able to see the Integration Tests in new sonarqube 6.7 version, here is what I'm doing,
Run Sonarqube task with Gradle to generate the report along with code coverage.
Here, Sonarqube is by default invoking test task and using the test.exec file from build/jacoco generated by Jacoco plugin.
However, I've a task depends on the test task to place an integrationTest.exec report in the build/jacoco as the integrations tests are run outside build(for some reason)
Though I place the integrationTest.exec file explicitly while Sonarqube execution it is still not considering the report and due to which I do not see any change in the coverage computed also I do not see Integration Tests section under coverage like I see Unit Tests.
Firstly, I would like to understand that whether is it feasible to export the integration tests result like I did above or not?
Second, is the latest Sonarqube having the capability to show integration tests report in the coverage section?
Your comments would help me.
Thanks in advance!
We are running selenium tests using TestNG in TeamCity.
Is there a way to know the progress of the current run beyond the progress bar and build log?
Information of interest is:
Which tests were run and passed
Which tests were run and failed
...
Thx
As I know testng plugin for teamcity does not have such option. You can try real-time reporter e.g. extentreports. It could be easily connected via testng listener and pass results to report server.
Here is a link to another question which I answered recently which addresses something similar to what you're looking for with ExtentReports.
ITestListener - ExtentReport
TeamCity can report the tests in a build and you should really get that in order to use TeamCity at full power. It can list the test failures in a build with details, display test history, you can assign investigations of a test to your team members or mute the test failures...
If you use TeamCity's Ant or Maven build steps, TestNG tasks should be recognized automatically and build status should turn into something like "Tests failed: 1 (1 new), passed: 301".
Otherwise try to generate XML report in one of supported format and use XML Report build feature in TeamCity.
Yet another alternative is to report the tests in a completely custom way through TeamCity service messages.
All these approaches should update TeamCity build status and details as soon as the test is reported by the build tool. For service messages this is as soon as a test finish is reported, for Ant this is after each test run, for Maven this is after finishing tests of a module and for the XML test report build feature, this is while the XML is being saved on disk, even in incomplete form.
If you are rather looking for a specific test progress, that can be seen in the build log or you can notify TeamCity about stages in the build from within the test/script.
I'm running a set of tests with mocha and i need the tests to run in the browser, therefore ; i'm doing this :
mocha.setup('bdd')
the tests do run fine in the browser, but i need this to be executed in jenkins. I must run the tests in the browser and i'm trying to get the reporter to output xunit reports .
i've tried the xunit-file package and another one called "mocha-multi" , it looks like browser tests only support the html reporter. Is there a way to output both the xunit and browser reports? or at least to execute the tests in the browser and report xunit results?
so there is no way to have two reporters if you're running the tests in the browsers and not in node js. After digging into mocha,i realized that there is a tests results variable as part of the mocha objects that you can instantiate in the browser. You need to use this to actually access the resutls and write the report yourself.
I am using the Gallio MSBuildTasks library to run MbUnit tests inside JetBrains TeamCity. When tests pass they report as having passed once, however when they fail they appear as having failed twice. Once with the Namespace and once without. This looks something like:
http://imageshack.us/photo/my-images/191/capturegv.png/
There are two tests here (Test and Test2) and they both fail. However TeamCity reckons they have each failed twice, displaying each one with a namespace (UnitTests) and then without. This is not just the report in the TeamCity website interface, I have pre-test commit enabled and they show up as failing twice in there too.
Has anyone else experienced this and if so how have you solved it?