We use the build in coverage application in TeamCity 6 (about to upgrade to 7.1)
If we wish to see the code coverage (or other metrics) of a particular build it is fine as we can navigate to that build, but it would be great if we could pluck out a few interesting metrics from all/some of the current projects/build configurations and display them all together.
For convenience I would expect the new display to be accessible from within TeamCity itself, however if there are solutions that require a separate solution we could look at them.
If you want to compare a set of common metrics (e.g. code coverage) across different projects and over time then SonarQube is probably what you want.
You can integrate it with TeamCity by adding a sonar-project.properties file to each project and calling sonar-runner from a command line build step.
Related
I have a legacy project, which is having 40M+ lines of code. I just want to configure the sonar during build but when I run sonar with default settings, it fails with OutOfMemoryException or with TimeOutException.
I got to know that this is because of the large codebase that I am having, so I increased the memory arguments to use 5 GB of memory. I tried with more memory but the teamcity server doesn't have much free memory. Still failed.
Finally what I did was, built two pipelines, included a particular file pattern in one using sonar.inclusions rest of the files in other and separated my code into two different pipelines. Now when I run the sonar, it works fine and generates two different reports with different project keys.
But my requirement is to generate a single report because I can't attach two reports in bitbucket. My goal is to show the report in bitbucket. If it is possible to fetch two reports in single repository, that will also do the thing.
Can anyone please help me here to generate a single report even though I run sonar in multiple pipelines in teamcity?
I don't think you can. A project (you call it 'report') is the only unit you can scan in one execution of the sonar scanner - you can't scan part of a project.
If you're using Enterprise edition, you can create a portfolio of multiple projects, which will automatically generate and maintain aggregated metrics.
It has been cleared here that from Sonarqube version 6.2 that coverage reports are merged and there won't be separate unit and integration coverage report anymore.
We still interested to have these two coverage reports separately. So, We have three sonar projects: unit-tests, integration-tests, whole-project(which is responsible to create overall coverage report)
Problem: All source files are analysed in all three projects. Since the number of files are too many, it takes several minutes to perform the analysis.
Question: Is it possible to turn off sonar issue analyser somehow in a project? It is desired to report only test coverage in the the first two projects(unit-tests & integration-tests) without analysing all files, and then run the issue analyzer only on the last project(whole-project). It could help us to analyse all files once instead of three times.
Additional info: We use sonar gradle plugin version 2.6.2 and sonarqube version 7.4
SonarQube/SonarCloud main responsibility is informing users about issues. Displaying code coverage is just an additional feature. It means there is no flag/parameter which allows you to do it.
Luckily, there is a workaround. You can create empty quality profiles, and use them to scan those two projects (unit-tests & integration-tests). You will get 0 issues because there are zero rules enabled.
The following feature request should be interesting for you: Making test coverage measures mode useful. Feel free to vote on it.
SonarQube Quality Profiles are not being used during the sonarqube scan:
We have sonar tasks installed and enabled for build definition what we are seeing is that the quality profiles are being stopped for one build run and it is again started using the quality profiles for the next run automatically. We are consistently seeing the same behavior for the alternate build runs.
Image where we can see the profiles are stopped and started:
What you're seeing is the result of a bad configuration somewhere. You indicate your comments that along with the toggling of profiles, you also see large swings in issue counts
as most of the file types get excluded from analysis resulting in very few issues reported.
So let's break this down:
The profile events you're seeing simply record/reflect the changes in profile use from one analysis to another. If I have a project with Java and JavaScript, the first analysis will use the default profiles for that language. Then, let's say I use the deprecated property sonar.language to restrict analysis to just Java files and analyze again. Since JavaScript is no longer found in my project, the default JavaScript profile will not be used, and a profile event will be recorded.
That seems to be what you're seeing in your activity log.
So now to the detective work: why is this happening? First, this swings back and forth. That indicates configurations set not at the project level (in SonarQube itself) but properties that are only sometimes passed during the analysis, or some other analysis-side circumstance. There are a few possible causes which you'll need to investigate independently:
sonar.language - if this deprecated property is used during analysis, it will limit the by-default multi-language analysis to a single language. It could be in your properties files or passed on the analysis command line -Dsonar.language=foo
exclusions - exclusions are difficult to set properly from the analysis side, but this can happen
improper/incomplete checkout - is it possible that only part of your project is checked out?
In investigating this, you should be aware that analysis-side properties can be set at two levels; at the individual project/analysis or in the global scanner configuration.
I'm guessing that your CI system has multiple slaves and languages are dropped - or not - from your project depending on which slave the job lands on that night.
I was experiencing a similar problem and I believe I have tracked down the root of the issue for us. Capturing the source code and properties used during the build, I was unable to find any discrepancies in sonar properties or source code collected, as suggested in the previous answer.
Our solution we were trying to analyze with SonarQube has a few C# projects that are part of an external core solution, which is shared between several components. Compiling any one of these other components requires that core be compiled first. I believe the issue lies in that when compiling my component to be analyzed, MsBuild would sometimes rebuild some of the projects included in core. Depending on if they were rebuilt or not, the number of projects and files (and hence issues) would swing wildly in one direction or the other.
By enforcing MsBuild to clean my component solution before compiling,
MsBuild.exe MySolution.sln /t:Clean,Build
I ensure a consistent set of projects are built and analyzed by sonarqube. I am 30 builds in with this new approach and I have no more flopping back and forth between using and not using a quality profile.
I was wondering if someone would be able to explain what Build runners do and also what i would need to use for just a static HTML / CSS / JS site, or even an already compiled .NET site.
I will be hooking up each project to its equiv SVN and grabbing updates from there, but not 100% sure what the build runners do or which i should use as technically i dont need to build the site.
Sorry it may be too much to answer but i am just struggling to get my head round Team City
Thank you
Build runners are just a process for a specific task, for example the MSBuild runner is set up by putting information into specific fields which it then uses to call MSBuild on the target build agent. You could just as easily use the Command Line runner and build up the MSBuild run command manually.
Build runner is a part of TeamCity that allows integration with a
specific build tool (Ant, MSBuild, Command line, etc.). In a build
configuration, the build runner defines how to run a build and report
its results. Each build runner has two parts:
server-side settings that are configured through the web UI
agent-side part that executes a build on agent
You need to choose your runner depending on the task that you want to do and the technology that you have wrapped your project in. If there is no runner for your specific task then you can use the lowest common denominator which would be the Command Line runner.
The way I approach this would be to see how I can achieve what I want to from my own environment be that calling a rake, MSBuild or batch file. I then see how I can then apply that to a tool. Do not create a process around a tool but choose a tool that fits to your process.
I am tasked to improve quality and implement TeamCity for continuous integration. My experience with TeamCity is very limited - I use mostly TFS myself and have some experience with CC.NET.
A lot should happen within a build process... actually the build is already pushed into three different configurations that will run one after the next.
My main problem is that in each of those I actually would need to start multiple runners. For example, the first build step shall consist of:
The generation of new AssemblyInfo.cs files for consistent in assembly numbering
The actual compilation
A partial unit test run (all tests that run fast and check core functionality)
An FxCop run
A StyleCop run
The current version of TeamCity only allows to configure one runner ... which leaves me stuck with a lot of things.
How you would approach this? My current idea is going towards using the MsBuild runner for everything and basically start my own MsBuild based script which then does all the things, pretty much the way that TFS handles it (and the same way i did things back in the cc.net way with my own Nant build script).
On a further problem the question is how to present statistical information, for example from unit tests running in different stages (build configurations). We have some further down that take some time to run and want that to run in a 2nd or 3rd step (the latest for example testing database generation code which, including loading base data, takes about 15+ minutes to run). OTOH we would really like test results to be somehow consolidated.
Anyone any ideas?
Thanks.
TeamCity 6.0 allows multiple build steps for a single build configuration. Isn't it what you're looking for?
You'll need to script this out, at least parts of it. TeamCity provides some nice UI based config for some of your needs, but not all. Here's my suggestion:
Create an msbuild script to handle your first two bullet points, AssemblyInfo generation and compilation. Configure the msbuild runner to run your script, and to run your tests. Collect your assemblies as artifacts.
Create a second build configuration for FxCop. Trigger it from the first build. Give it an 'artifact dependency' on the first build, which is how it gets a hold of your dlls.
For StyleCop, TC doesn't support it out of the box like it does FxCop. Add it to your msbuild script manually, and have it produce an html report (which TeamCity can then display).
You need to take a look at the Dependencies functionality in the TeamCity. This feature allows you to create a sequence of build configurations. In other words, you need to create a build configuration for each step and then link all them as dependencies.
For consolidating test results please take a loot at the Artifact Dependencies. It might help.