I have several tests in a class. However, one of those tests sends a POST request to an API and receives an ID. I need to use that ID for the rest of the tests.
I guess it would need to satisfy two things:
Make sure that the test that gets the ID is always executed first.
Set the value of the variable after the first test and use it for the remaining tests.
How can I perform some tests (it() calls) about an object under test, then call a method() on it, and then perform some more tests on the object's state modified by method()?
The problem is that a call to it() schedules the actual assertion to be performed much later, and as a result of this ALL the tests will actually run after the call to method(), including those that I want to run before that call. How to overcome this?
We have several options in Maven Surefire Plugin to describe forking. One of the options, forkCount, is explained like this:
Option to specify the number of VMs to fork in parallel in order to
execute the tests. When terminated with "C", the number part is
multiplied with the number of CPU cores. Floating point value are only
accepted together with "C". If set to "0", no VM is forked and all
tests are executed within the main process.
Basing on this I may guess that if a forked regime and used, and forks are reused (reuseForks=true) then the same JVM will be used for several tests. This means, if I have loaded some class into memory, the static members of that class can be reused in some other test and fail it unexpectedly.
Is my understanding correct?
You're right. Tests with static elements are not thread-safe and should be excluded from parallel execution:
If the Suite or Parameterized is annotated with #NotThreadSafe, the suite classes are executed in single thread. You can also annotate individual test class referenced by Suite, and the other unannotated test classes in the Suite can be subject to run in parallel.
Note: As designed by JUnit runners, the static methods annotated with #BeforeClass and #AfterClass are called in parent thread. Assign classes to the #NotThreadSafe Suite to prevent from this trouble.
I've written many unit tests in a file.
The problem is they don't run in order.
I first make an entry to the database in one method and delete the same entry in another method.
Insert() appears before Remove() in my test file.
But still Remove() runs first and hence I am not able to execute the test cases effectively since it won't find the entry. Reason could be Remove() takes less execution time than Insert()
Can we set the sequence to the test cases?
You can prefix the test names with character alphabetically
like
aTestSomething
bTestAnotherThing
:)
better way
How to order methods of execution using Visual Studio to do integration testing?
I have unit tests defined for my Visual Studio 2008 solution. These tests are defined in multiple methods and in multiple classes across several files.
I've read in a blog article that when using MSTest, it is a mistake to think that you can depend on the order of execution of your tests:
Execution Interleaving: Since each instance of the test class is instantiated separately on a different thread, there are no guarantees
regarding the order of execution of unit tests in a single class, or
across classes. The execution of tests may be interleaved across
classes, and potentially even assemblies, depending on how you chose
to execute your tests. The key thing here is – all tests could be
executed in any order, it is totally undefined.
That said, I have to have a pre-execution step before any of these tests gets to run. That is, I actually want to define an order of execution somehow. For example, 1) first create the database; 2) test that it's created; then 3) run the remaining 50 tests in arbitrary order.
Any ideas on how I can do that?
I wouldn't test that the database is successfully created; I will assume that all subsequent tests will fail if it is not, and it feels in a way that you would be testing the test code.
Regarding a pre-test step to set up the database, you can do that by creating a method and decorating it with the ClassInitialize attribute. That will make the test framework execute that method prior to any other method within the test class:
[ClassInitialize()]
public static void InitializeClass(TestContext testContext)
{
// your init code here
}
Unit tests should all work standalone, and should not have dependencies on each other, otherwise you can't run a single test in isolation.
Every test that needs the database should then just create it on demand (if it's not already been created - you can use a singleton/static class to ensure that if multiple tests are executed in a batch, the database is only actually created once).
Then it won't matter which test executes first; it'll just be created the first time a test needs a database to use.
In theory it is correct that tests should be independent of each other and be able to run standalone. But in practice, there is a difference between theory and practice, and VS2010 gives me a hard time with its fixed order of execution (random order that is always the same).
Here are some examples:
I have a unit test that cross checks the dates between some tables and verifies that everything is in agreement. Obviously it is of no use to run this test on an empty database, so I want to to run SOME TIME AFTER the unit test that inserts data. Sorry VS2010 doesn't let you do this.
OK, cool, then I will add it to the insert unit test as an epilogue. But then I want to cross check other 10 things and instead of having a unit test ("Make sure that entities with various parameters can be inserted without crashes") I end up having a mega-test.
Then another case.
My unit test inserts entities, just insert, to make sure that this part of the logic works ok. Then I have a multi-threaded version of the test, to make sure that there are no deadlocks and stuff. Clearly I need the multi-threaded test to run SOME TIME AFTER the single threaded test, and ONLY if the single threaded test succeeds. Sorry, VS2010 can't do this.
Another case. I have a unit test that deletes ALL entities of a given kind in the database. This should result in a bunch of empty tables and lots of zeros in other tables. Clearly it is useless to run it on an empty database, so the test inserts 10.000 entities if it finds the DB empty. However, if it runs AFTER the multithreaded test, it will find 250.000 entities, and to delete ALL of them takes TIME. Sorry, VS2010 won't let me do anything about it.
The funny thing is that because of this situation my unit tests started slowly turning into mega-tests, that took more than 30 mins to complete (each) and then VS2010 would time them out, cause the default test timeout is 30 mins. OMG please help! :-)