Is there a way to show different reports inside TeamCity build page for different types of tests?
We have the only one Gradle build for all checks (and we don't want to have separate builds for every check type). But we want to show separate test reports for every type of tests (UI, unit, etc).
Related
When I run tests in Xcode, as long as I run all the tests collectively they execute normally. However, if I run only one individual test class or one individual test function within a test class, it doesn't execute any tests at all and prints:
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Upon debugging I've found that it does execute individual tests if I choose a particular other scheme I have, but my other two schemes don't execute the individual tests. I also noticed that if I edit one of the two problematic schemes to use the same Build Configuration as the working scheme, the individual tests do work (each of my schemes uses a different build configuration).
How do I make the problematic build configurations work to let me execute individual tests?
I was able to figure it out!
If I click on my project in the Project navigator, then click on my project in the left sidebar of the page that appears, and then go to the Info tab, I can see at the bottom of the Configurations section it says:
"Use ConfigurationName for command-line builds"
If I click the dropdown and change it to a different build configuration, it makes the scheme that uses that build configuration able to execute individual tests! And, as one might expect, it also makes the scheme previously able to execute individual tests no longer able to.
It would be nice if I could dynamically change the build configuration for command-line builds based on which scheme is active in Xcode, but as of yet I have not figured out how to do so.
So for now, just know that you can manually change this build configuration value whenever you want to run individual tests for a scheme.
I have a legacy project, which is having 40M+ lines of code. I just want to configure the sonar during build but when I run sonar with default settings, it fails with OutOfMemoryException or with TimeOutException.
I got to know that this is because of the large codebase that I am having, so I increased the memory arguments to use 5 GB of memory. I tried with more memory but the teamcity server doesn't have much free memory. Still failed.
Finally what I did was, built two pipelines, included a particular file pattern in one using sonar.inclusions rest of the files in other and separated my code into two different pipelines. Now when I run the sonar, it works fine and generates two different reports with different project keys.
But my requirement is to generate a single report because I can't attach two reports in bitbucket. My goal is to show the report in bitbucket. If it is possible to fetch two reports in single repository, that will also do the thing.
Can anyone please help me here to generate a single report even though I run sonar in multiple pipelines in teamcity?
I don't think you can. A project (you call it 'report') is the only unit you can scan in one execution of the sonar scanner - you can't scan part of a project.
If you're using Enterprise edition, you can create a portfolio of multiple projects, which will automatically generate and maintain aggregated metrics.
We use the build in coverage application in TeamCity 6 (about to upgrade to 7.1)
If we wish to see the code coverage (or other metrics) of a particular build it is fine as we can navigate to that build, but it would be great if we could pluck out a few interesting metrics from all/some of the current projects/build configurations and display them all together.
For convenience I would expect the new display to be accessible from within TeamCity itself, however if there are solutions that require a separate solution we could look at them.
If you want to compare a set of common metrics (e.g. code coverage) across different projects and over time then SonarQube is probably what you want.
You can integrate it with TeamCity by adding a sonar-project.properties file to each project and calling sonar-runner from a command line build step.
I'm new to load testing in Visual Studio/MSTest, and I created a new Load Test recently to validate some high-traffic scenarios for a WCF service. I want to add this to the tests project for the service, but I don't want the test to be executed whenever I "Run All Tests in Solution" nor as part of our Continuous Integration build-verification process because a) it takes 5 minutes to run, and b) the service call that it is testing generates many thousands of email messages. Basically, I'd like to do the equivalent of adding the [Ignore] attribute to a unit test so that the load test is only executed when I explicitly choose to run it.
This MSDN Article ("How to: Disable and Enable Tests") suggests that the only to disable the test is to use Test Lists (.vsmdi files), but I don't have much experience with them, they seem like a hassle to manage, I don't want to have to modify our CI Build Definition, and this blog post says that Test Lists are deprecated in VS2012. Any other ideas?
Edit: I accepted Mauricio's answer, which was to put the load tests into a separate project and maintain separate solutions, one with the load tests and one without. This enables you to run the (faster-running) unit tests during development and also include the (slower-running) load tests during build verification without using test lists.
This should not be an issue for your CI Build Definition. Why?
To run unit tests as part of your build process you need to configure the build definition to point to a test container (usually a .dll file containint your test classes and methods). Load tests do not work this way, they are defined within .loadtest files (which are just xml files) that are consumed by the MSTest engine.
If you do not make any further changes to your CI Build definition the load test will be ignored.
If you want to run the test as part of a build, then you need to configure the build definition to use the .loadtest file.
Stay away from testlists. Like you said, they are being deprecated in VS11.
Edit: The simplest way to avoid running the load test as part of Visual Studio "Run All" tests is to create a different solution for your load tests.
Why don't you want to use Test Lists. I think is the best way to do that. Create different Test Lists for each test type (unit test, load test...) and then in your MSTest command run the Test List(s) you want:
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests (only UnitTests)
MSTest \testmetadata:testlists.vsmdi \testlist:LoadTests (only LoadTests)
MSTest \testmetadata:testlists.vsmdi \testlist:UnitTests \testlist:LoadTests (UnitTests & LoadTests)
I would like to use some build parameters from Project 1 in Project 2. I know that I can make Project 1 a dependency of Project 2 and then access its build parameters as described in Dependencies Properties, but I do not want Project 1 to be built in response to a build of Project 2. For example, suppose I want Project 2 to be built nightly, while I only want Project 1 built monthly.
Is there any way Project 2 can access Project 1's build parameters under these conditions?
I would use a build configuration template that is shared between the two projects.
This means you can share properties between the projects, but also override certain ones in each individual project.
We use this for hourly builds that are not tagged and nightly ones that are tagged.
Then use a different build trigger to set one off nightly and the other monthly.
EDIT
I'll just expand slightly as a result of your comment.
In TeamCity we have 2 build configuration for the same project. One that builds on every check-in to give developers quick feedback on their contribution (build within 15 minutes). It does the following:
Builds the project in Debug
Runs all unit tests
Checks results of build into Subversion
The other configuration runs every night at midnight; it build everything and as a result takes a long time (around 45 minutes). It does the following:
Build the project in Debug and Release
Runs all unit tests
Builds Sandcastle documentaion
Checks results of build into Subversion
Grabs the Sandcastle output at an artefact so developers can easily download it.
As you pointed out this isn't as straightforward as one would like; however you can use the following to achieve it:
We use the Autoincrementer to share build numbers between the two configurations (they both increment the same build number when built).
We have a property on template that defines what artefacts to collect and is referenced from the artefacts field. The property is overridden on the second build config to define the sandcastle output to grab.
Sharing VCS Roots is mentioned on the documentation. Both our builds get the source from the same place, and tag the results to the same place. One VCS is most definitely all we need.
Bit of a long edit but I think it goes exactly on the lines of what you're trying to achieve. I appreciate I should have included this in the original answer.
HTH
Dependency is different from Build Triggering in TeamCity. If you make one project dependent on another ( artifact dependency ), it does not mean that the the latter will trigger the former.
Even when one project has been defined as dependent on another ( and also, even if not ) you have to specify explicitly the build trigger ( in this case a Finish build trigger ) for the dependent project to be triggered.