How can we combine different build reports in team city?
Consider all builds are chained and final job will show report of all jobs in chain.
You can configure artifact dependencies from the last build configuration in the build chain to all needed build configurations. So all the reports will be downloaded by the last build configuration.
If the reports are in HTML format you can publish it as TeamCity artifact and configure Report tab to make the reports available as an extra tabs on the build or project level.
Another approach is to use XML Report processing build feature that allows using report files produced by an external tool in TeamCity. TeamCity will parse the specified files on the disk and report the results as the build results.
For more details see how to present data in TeamCity in the documentation.
Related
I have a TeamCity project that runs Jmeter via Maven in a build called "PerfRunner" under the project "Audit-Service-Performance". When it runs Jmeter produces a target dir with a report html file, index.html.
I want TeamCity to create a report tab with the latest report from the latest successful.
Folder structure relative to artifact:
target
-jmeter
--bin
--lib
--logs
--reports
---jmeter-deng-audit-standalone_time_stamp
----index.html
--results
--testFiles
The problem is that the time_stamp in meter-deng-audit-standalone_time_stamp is auto generated and therefore changes with each build.
When I try to create a report tab I do a project or build report tab I tried:
target/jmeter/**/index.html. But it doesn't generate the report tab.
The only way I can get it to work is if I address a static file for a particular build which of course wont be the latest once a new build is run.
The MVN command runs clean. How can I make it dynamic?
You need to publish this index.html page as the build artifact. If you want to share the page with the others you can go for one of the following patterns of accessing build artifacts:
/repository/download/BUILD_TYPE_EXT_ID/.lastFinished/index.html
/repository/download/BUILD_TYPE_EXT_ID/.lastSuccessful/index.html
/repository/download/BUILD_TYPE_EXT_ID/.lastPinned/index.html
/repository/download/BUILD_TYPE_EXT_ID/BUILD_ID:id/index.html
/repository/download/BUILD_TYPE_EXT_ID/BUILD_NUMBER/index.html
You can also consider using JMeter Teamcity Plugin - it adds Performance Statistics tab where you can observe performance trend for last builds
Check out How to Run JMeter Tests with TeamCity for Continuous Integration article for more information on how to configure JMeter to work with Teamcity
We are running selenium tests using TestNG in TeamCity.
Is there a way to know the progress of the current run beyond the progress bar and build log?
Information of interest is:
Which tests were run and passed
Which tests were run and failed
...
Thx
As I know testng plugin for teamcity does not have such option. You can try real-time reporter e.g. extentreports. It could be easily connected via testng listener and pass results to report server.
Here is a link to another question which I answered recently which addresses something similar to what you're looking for with ExtentReports.
ITestListener - ExtentReport
TeamCity can report the tests in a build and you should really get that in order to use TeamCity at full power. It can list the test failures in a build with details, display test history, you can assign investigations of a test to your team members or mute the test failures...
If you use TeamCity's Ant or Maven build steps, TestNG tasks should be recognized automatically and build status should turn into something like "Tests failed: 1 (1 new), passed: 301".
Otherwise try to generate XML report in one of supported format and use XML Report build feature in TeamCity.
Yet another alternative is to report the tests in a completely custom way through TeamCity service messages.
All these approaches should update TeamCity build status and details as soon as the test is reported by the build tool. For service messages this is as soon as a test finish is reported, for Ant this is after each test run, for Maven this is after finishing tests of a module and for the XML test report build feature, this is while the XML is being saved on disk, even in incomplete form.
If you are rather looking for a specific test progress, that can be seen in the build log or you can notify TeamCity about stages in the build from within the test/script.
I've used Jenkins CI for a few years and I want learn TeamCity.
In Jenkins I tracked FxCop issues by enabling code analysis on my .net projects and then telling the Violations plugin where to find the code analysis XML files i.e. MSBUILD would put a file named [project name].CodeAnalysisLog.xml in the build output directory and you could use something like **/*/*CodeAnalysisLog.xml to find those files with the Violations plugin.
TeamCity has its own FxCop runner but I don't want to run FxCop again because MSBuild has already done that for me.
I just want to be able to tell TeamCity where to find the XML files and have it produce the trend graph in the same way Jenkins does with the Violations plugin.
I have a similar issue with StyleCop, jsLint and cssLint. MSBuild build tasks or batch commands that run as part of the build produce XML output. I would like to use this output to create trend graphs.
To my knowledge TeamCity is different from Jenkins for the reports in a sense that it takes them in HTML format. That's what we have had to do for our code quality analysis product - convert our XML report to HTML for TeamCity. Due to that we can't do violation trend report when integrated with TeamCity unlike the Jenkins plug-in we have. We haven't looked whether there were TeamCity changes in the last couple of years that would enable the XML report input.
We use the build in coverage application in TeamCity 6 (about to upgrade to 7.1)
If we wish to see the code coverage (or other metrics) of a particular build it is fine as we can navigate to that build, but it would be great if we could pluck out a few interesting metrics from all/some of the current projects/build configurations and display them all together.
For convenience I would expect the new display to be accessible from within TeamCity itself, however if there are solutions that require a separate solution we could look at them.
If you want to compare a set of common metrics (e.g. code coverage) across different projects and over time then SonarQube is probably what you want.
You can integrate it with TeamCity by adding a sonar-project.properties file to each project and calling sonar-runner from a command line build step.
I have configured one project in sonar and integrated sonar with maven for build time analysis of the project.
After analysis, report is generated and uploaded to Sonar for browsing. But once another user compiles the same project their report overwrites mine.
Basically I want that one user's report on one project is not overwritten by report from other user. A user must be able to see their current violations independently. Is it possible in Sonar?
Sonar stores it's analysis on a daily basis, which explains why it's kind of pointless to run analysis several times in a day. Each analysis run will overwrite that day's existing results, which in turn spoils ongoing statistical analysis.
I would suggest running Sonar, from a dedicated build server like Jenkins (which has a Sonar plug-in). This daily analysis will populate the Sonar database and keep the project dashboard current. This architecture also enables you to keep the database credentials confidential.
Obviously developers would like to see the results of their bug fixing. For that I'd recommend running the Sonar Eclipse plug-in. The latest version will run the same Sonar analysis locally. Recent versions of Sonar also enable you to assign violations to developers for resolution.
This is not possible, last performed analysis will always be the one you browse in the interface. However, I guess what you need is the Issue Report Plugin which will enable analysis to store results locally, with the dry-run option.
This way your developers will be able to run an analysis on their code and see the violation delta without pushing the results.
You can do it by explicitly setting the below properties in pom.xml
<properties>
<sonar.projectKey>Test</sonar.projectKey>
<sonar.projectName>Test</sonar.projectName>
</properties>
Every user should set different projectKey and projectName, if you want that one user's report on one project is not overwritten by report from other user.