Is it possible to run tests in parallel with mbunit and AppDomain/Process level isolation?
Related
There is no distinction in SonarQube between unit tests and integration tests. That is a problem.
We have large tests (or integration tests) which stretch across external systems. When these tests fail, only by human evaluation it can be decided whether it is a failure of our application or of the external system. If it is a external system failure than we can ignore it.
If SonarQube would have the distinction between unit tests and integration tests, then I would configure that the quality gate fails if more than 0 unit tests fail and allow integration tests to fail.
Because there is no such distinction anymore in SonarQube we have to solve it in the Jenkins pipeline, expecting all non-large tests (unit tests) to succeed and allowing large tests (integration tests) to fail.
I would prefer not to display any test results in the Jenkins build but have it only in SonarQube. But SonarQube is not capable of that logic and it does not separate the test results (large and non-large).
Maybe SonarQube will reconsider that decision to combine all different kinds of test results? It should be even more flexible than unit/integration tests and allow groups of test to be customized, like small/medium/large tests, with quality gates for each of them.
Or is there a better solution then we currently use?
I'd like to spread the execution of junit tests across different systems, when maven surefire is running the tests. Is this possible?
There are ~ 20,000 junit tests, and when running 5 in parallel (forks), it takes approximately 40 minutes. Running more in parallel starts to hit resource problems. If I could run half of the tests classes on a different VM (or something like kubernetes, etc), then that could cut the execution time in half.
Is this possible?
I have some long running JUnit tests which I want to run first to avoid them being queued and executed last and thus delaying the whole test execution process.
I currently use maven surefire to run my tests using:
forkCount: 5
reuseForks: false
Is there a way to specify which tests should be run first? Or a way to optimize the order in which the tests are run?
The runOrder=balanced parameter does not seem to work in combination with forkCount > 0 and reuseForks false.
Unfortunately, you can’t determine the test execution order. Here is a clumsy workaround for this problem. For integration tests, I use a JUnit extension to run tests in specific order.
I need to create a build in such a way that, unit test, integration test and performance test can be executed separately using Maven and Gradle.
Acceptance Criteria :
Seperate Build files (Maven and Gradle) for unit test, Integration test and Performance test.
The test execution flexible enough to be executed on Demand like
Unit test Alone (Default profile)
Unit Test & Integration Test (Profile With Integration Test)
Unit test and Performance test.(Profile With PerformanceTest)
Start reading the documentation:
https://maven.apache.org/guides/
http://books.sonatype.com/mvnref-book/reference/
https://docs.gradle.org/current/userguide/userguide.html
I'm running functional tests using Spock where the tests manipulate the database, and the Specs can conflict with each other if run simultaneously. From my web debugger, it appears that when I run gradle test that the calls to the web service from different Specs are being called simultaneously. The Specs themselves are being called with the #Stepwise annotation, and the tests are run in the correct order. How can I have Specs run separately in some order? They do not need to run in any specific order.