Have anyone tried running test cases in parallel using NUnit 3.2? I have few thousand test cases written using NUnit(2.6.3) and i want to run them in parallel. Since NUnit 2.6.3 don't have the feature to run test cases in parallel, I thought of switching to NUnit 3.2. When i read the documentation, it says it support running test fixtures in parallel not test cases. Some web site says NUnit 3.2 support running test cases in parallel. I'm confused. Any help much appreciated.
NUnit 3 intends to allow running all kinds of tests (suites, fixtures, simple test methods, test cases) in parallel.
As far as NUnit 3.2, we only run tests in parallell down to the fixture level. The test methods/cases under the fixture run one at a time. So long as you have a relatively large number of fixtures, this gives an equivalent performance increase to running the methods in parallel. However, in the extreme, for example with a single fixture and 1000 methods, you will see no improvement.
No promises, but I imagine we'll be running test cases in parallel for 3.4.
Related
lets take one spec file in cypress in that i have 5 test cases i have to run sequentially
I am tried with npx cypress run command it will run test cases one by one is it write or wrong? i am confused.
Yes it is totally normal. Here is a link to the documentation of Cypress, which explains what Parallelization is and how to set it up.
You need to activate parallelization first, in order to use it. However it is not recommended to be done on a singular machine. It needs a significant amount of resources.
Cypress Parallelization
If your project has a large number of tests, it can take a long time for tests to complete running serially on one machine. Running tests in parallel across many virtual machines can save your team time and money when running tests in Continuous Integration (CI).
Cypress can run recorded tests in parallel across multiple machines since version 3.1.0. While parallel tests can also technically run on a single machine, we do not recommend it since this machine would require significant resources to run your tests efficiently.
This guide assumes you already have your project running and recording within Continuous Integration. If you have not set up your project yet, check out our Continuous Integration guide. If you are running or planning to run tests across multiple browsers (Firefox, Chrome, or Edge), we also recommend checking out our Cross Browser Testing guide for helpful CI strategies when using parallelization.
Turning on parallelization
Refer to your CI provider's documentation on how to set up multiple machines to run in your CI environment.
Once multiple machines are available within your CI environment, you can pass the --parallel key to cypress run to have your recorded tests parallelized.
cypress run --record --key=abc123 --parallel
Running tests in parallel requires the --record flag be passed. This ensures Cypress can properly collect the data needed to parallelize future runs. This also gives you the full benefit of seeing the results of your parallelized tests in Cypress Cloud. If you have not set up your project to record, check out our setup guide.
Source: Cypress Documentation
I am building a Go provider for Pruner (a CLI that runs only the tests that ran through the lines you changed, saving you time).
For that, I need to be able to see per-test coverage. Not just a full coverage report after all tests have run, but I need a way to know which tests ran through what line.
Is that possible in Go?
I tried using -func, but it just gives me the method names of the original code, not the test code. In other words, I can't know what code each individual test runs through.
I need a way to know which tests ran through what line.
Is that possible in Go?
It's not supported by the tools. But you can do it. It's just very inefficient.
The way to do this is to run:
go test -cover -run=TheName/OfSome/SpecificTest
Then run this for each test in your suite.
Naturally, this will make your tests much more cumersome to manage, and incredibly slow.
So I would consider whether this is truly a requrement for your use case.
Go is optimized, from the ground-up, to compile quickly. If you have a Go project so long, that running all the tests is too slow, you may want to consider other alternatives. Some suggestions:
Run more tests in parallel, so the total runtime is reduced.
Take advantage of Short mode, and only run short tests by default, saving long-running tests for special cases.
If you really need to run only a subset of tests, do it on a per-package basis, not on a per-test basis.
I have a test suite consisting of around 100 UI scenario tests that I am running with Mocha. When I run the suite in parallel (using the --parallel flag), the number of tests that are run is inconsistent and fluctuates. For instance, it might be 87 tests in one run and 94 in the next. Limiting the number of parallel jobs to for instance 10 (using the --jobs option) seems to mitigate the problem. However, this feels like a workaround rather than addressing the root cause. Thus, I am wondering what the reason is that Mocha runs a fluctuating number of tests in parallel mode (often not the entire suite), and how this issue can be addressed properly?
I am using version 8.1.1 of Mocha, and mocha-trx-reporter at 3.3.1 to report the test results. I do not use any options other than --recursive (to run the test suite) and --parallel.
I need to have (I have to, it's not my decision and I cannot alter it) test-sets in both jasmine and cucumber. So I've made two folders with specs and two conf.js files, one for each framework.
When I need to run both jasmine and cucumber tests, I need to run protractor with one config and then run it again with another config.
So the question is: is there any way to make protractor run jasmine and cucumber tests consequently (e.g. first run all jasmine tests, then run all cucumber tests) in "one click"?
If you need any details about environment: I'm currently running tests in IDEA, and later there will be a Jenkins (or Hudson) job for it.
P.S.: I think, the question won't be so important when we'll use Jenkins, because we can make two jobs running one after another. But even though we might not need it, I'm still curious if it's possible.
I have all the tests for my web application (written with the Visual Studio test framework -- Microsoft.Quality DLLs) divided into several (currently two) ordered tests. Is there an easy way to find all the tests that are not in any list?
(The reason I need to use ordered tests is because the initial tests test that installation/setup/configuration of my application worked, and subsequent tests would fail without that.)
There's on easy way to do this. The best thing to do is switch to a framework that doesn't require every test to be on a list -- I recommend MbUnit. It has a great DependsOn attribute to easily configure dependencies between tests.