I'm using MochaJS (10.1) as my testing framework with WDIO and we use this.retries(3) which tell the test to retry if failing the first time (sometimes we have flaky tests, race conditions, etc). But we'd like to capture which retry attempt actually passed.
I see there is a protected currentRetry in the mocha runner class. Not sure why they would protect it, but I'm trying to figure out how I can maybe make a subclass for that to expose it to my test.
I think I can do that with something like:
class myRunnable extends Mocha.Runnable {
public get myCurrentRetry() {return super.currentRetry}
}
but not sure how to get my tests to use myRunnable rather than normal mocha
Related
Im implementing a HTTP test in Laravel, but I observed that whenever my test fails, the tearDown method doesn't get called. How can I achieve to call it even the test fails, programmatically? I just want to clean up things in each test.
UPDATE
The method that I've mentioned is a Laravel's built-in. I'll explain two lifecycles of a test in Laravel, first is the setUp, which is a method that you may override to include some initializations you might need in each test. The second one is the tearDown, which is used to clean up any "residue" from the test. So my Base Test class looks like this...
public function setUp() {
...some initializations
}
public function tearDown() {
$this->someModelInstance->delete();
}
All are working fine but whenever a test fails, only the setUp has been called. All the "residue" remains in the database. What I want is to "clean up" the "residue" even a test fails.
I am having a hard time on how to start testing the integration with an external API ( Vimeo API ) for things like ( deleting a video - uploading a video - etc..).
Should I hit the real Vimeo's server while testing?
Is it a bad idea to do this like:
use Tests\TestCase;
use Vimeo\Laravel\VimeoManager;
class VimeoApiTest extends TestCase
{
protected function setUp() : void
{
parent::setUp();
$this->vimeo = new VimeoManager();
}
/** #test */
public function a_video_can_be_deleted()
{
$video = $this->vimeo->upload($fakeVideo);
// make http request to delete the video
$result = $this->vimeo->delete($video['id']);
$this->assertEquals('success', $result['status']);
}
}
In my eyes testing a package should not be your responsibility. Testing your implementation of the package is. Your example tests the package's VimeoManager directly. Which is in my opinion not what you should do.
Making a request to your API route that uploads the video is what you should test. During this test you do not want to upload it to the real Vimeo API, but you want to mock it.
Laravel includes a package that can do this, it is called Mockery. You can mock methods of classes to return a value without the initial logic being executed. In this case you would mock the delete method of the VimeoManager.
Laravel also provides so called Facades which can easily be mocked. I can see this package makes use of such facade. In this case you can do the following to test implemention of lets say your delete request.
/** #test */
use Vimeo\Laravel\Facades\Vimeo;
public function your_test()
{
Vimeo::shouldReceive('delete')
->once()
->with(...) // The parameters is should receive.
->andReturn(...); // The value it should return.
// Make the delete request to your API.
}
Well, you can test however you see fit. I've find it useful to have two types of tests.
A test which interacts with "something" else (service, system, etc...) is known as an integration test. These are nice and give some piece of mind, but are linked to the system that you are interacting with being online, this isn't always the case.
The second type of test can have a couple of different names, but that isn't really the point. The point of this second type of test is that you can "mock" out external/internal dependencies, ensuring that the "thing" your code depends on is online/behaves how you want. Mocking is when you manipulate a "thing" to respond a certain way. Usually, this is done via some framework or language feature. These types of test put a way larger burden on your code, and count more in my opinion.
I'm generally against mocking server to which my tests should be connected.
The main disadvantage of mocking servers and use simulators is fact that implementation of API may change (and for example throw exception/new status code or timeouts may be shorter) or there may be not compatibility between APIs between 2 versions of server.
Should I hit the real Vimeo's server while testing?
If You have possibility then create your local Vimeo's server using docker :)
Your tests will find out if something changes in API in future
I'm busy learning to unit test in Laravel and I have a question.
I want one test that has to do with authentication to run first before running other all other tests. Can this be achieved?
I have 4 test classes with a few methods so far.
In PHPUnit, every single test is run with a fresh environment, all the setting up and tearing down is done for every test, so the order does not matter. If you're trying to authenticate as part of your other tests, say for example you're testing your API and you need your tests to authenticate as a user, Laravel's testing facility has methods and helpers to let you do just that.
You'll need to do all of your setup in the setUp method of your test cases. That method is executed before each and every test is run. Similarly, the tearDown method is run after every one of your tests have completed:
public class YourTest extends TestCase
{
protected function setUp()
{
// Do your setup here
}
protected function tearDown()
{
// Your cleanup here, if you're using Mockery, this
// is a good place to call Mockery::close();
}
public function testSomething()
{
// Your test code, you can assume that setUp has
// already run and you're ready to go.
}
}
Alternatively, you could create a method that does all of your setup and call it at the beginning of each of your tests.
I am trying to implement my Spring website using TDD technique.
There are some TDD rules:
Test behaviour instead of state.
Tests shouldn't depends on
implementation.
I created UsersService empty class which depends on crud UsersRepository.
Now, I am trying to write test for signing up new users, but I don't know how to do this properly.
#Test
public void signUp_shouldCheckIfUserExistsBeforeSign() throws ServiceException {
// given
User user = new User();
user.setEmail(EMAIL);
when(usersRepository.save(user)).thenReturn(user);
when(usersRepository.exists(anyString())).thenReturn(Boolean.FALSE);
// when
usersService.signUp(user);
// then
thrown.expect(UserAlreadyExistsServiceException.class);
usersService.signUp(user);
}
This code tests behaviour but also enforce me to implement my service using exists() method instead of findByEmail() for example.
How should this test looks like?
Your test seems to reflect some confusion about behavior and implementation. It seems like you expect a state change when you call signUp() the first time, but because you're using mocks I don't think that will happen, so don't call signUp() twice if you're using mocks (and the expect() should be before signUp(), I believe). If you weren't using mocks, it would be a valid test to call signUp() twice, with no implementation dependency, but you're (wisely, IMHO) using mocks to avoid slow, database-dependent tests for easily mockable dependencies, so call signUp() just one time and let the mocks simulate the state. It makes sense to mock your storage interface when testing your service behavior.
As for your 2 testing rules, you can't use mocks without some notion of implementation (I prefer to think of it as "interactions" - especially if you mock interfaces rather than concrete classes). You seem to have a modular design, so I wouldn't worry about simulating an obvious interaction. If you change your mind later about the interaction (whether you should retrieve a user object instead of the boolean existence check), you change your test - no big deal, IMHO. Having unit tests should make you LESS afraid to change your code. It's true that mocks can make tests more brittle if the interactions need to be changed. The flip side of that is you think more about those interactions before coding them, which is good, but don't get stuck.
Your dilemma about whether to retrieve the user by email or to check its existence with a boolean exists() call sounds like a case of YAGNI to me. If you don't know what you're going to do with the User object you retrieve besides checking whether it's null, go with the boolean. If you change your mind later, you may have a few broken tests to (easily) fix, but you'll have a clearer idea of how things should work.
So your test might look like this if you decide to stick with exists():
#Test
public void signUp_shouldCheckIfUserExistsBeforeSign() throws ServiceException {
// given
User user = new User();
user.setEmail(EMAIL);
when(usersRepository.exists(anyString())).thenReturn(Boolean.FALSE);
thrown.expect(UserAlreadyExistsServiceException.class);
// when
usersService.signUp(user);
// then - no validation because of expected exception
}
BTW, (this is a side issue, and there are lots of different approaches to expecting exceptions in tests which are covered elsewhere on StackOverflow) it would be nice to be able to put the expect() call in the "then" section, but it has to be before signUp(). You could alternately (if you don't want to call expect() in the "given" section) use the expected parameter of #Test instead of calling expect(). Apparently JUnit 5 will allow wrapping the throwing call in an expectation call that returns the exception thrown or fails if none is thrown.
Testing behavior is good, but the production code will need to exhibit that behavior, which will affect the implementation to some degree.
Focus the test on a single behavior:
#Test
public void signUpFailsIfUserEmailAlreadyExists() throws ServiceException {
// given
User user = new User();
user.setEmail(EMAIL);
when(usersRepository.emailExists(EMAIL)).thenReturn(Boolean.TRUE);
// when
usersService.signUp(user);
// then
thrown.expect(UserAlreadyExistsServiceException.class);
}
I have a class API which has full code coverage and uses DI to mock out all the logic in the main class function (Job.Run) which does all the work.
I found a bug in production where we werent doing some validation on one of the data input fields.
So, I added a stub function called ValidateFoo()... Wrote a unit test against this function to Expect a JobFailedException, ran the test - it failed obviously because that function was empty. I added the validation logic, and now the test passes.
Great, now we know the validation works. Problem is - how do I write the test to make sure that ValidateFoo() is actually called inside Job.Run()? ValidateFoo() is a private method of the Job class - so it's not an interface...
Is there anyway to do this with NMock2.0? I know TypeMock supports fakes of non interface types. But changing mock libs right now is not an option. At this point if NMock can't support it, I will simply just add the ValidateFoo() call to the Run() method and test things manually - which obviously I'd prefer not to do considering my Job.Run() method has 100% coverage right now. Any Advice? Thanks very much it is appreciated.
EDIT: the other option I have in mind is to just create an integration test for my Job.Run functionality (injecting to it true implementations of the composite objects instead of mocks). I will give it a bad input value for that field and then validate that the job failed. This works and covers my test - but it's not really a unit test but instead an integration test that tests one unit of functionality.... hmm..
EDIT2: IS there any way to do tihs? Anyone have ideas? Maybe TypeMock - or a better design?
The current version of NMock2 can mock concrete types (I don't remember exactly which version they added this, but we're using version 2.1) using the mostly familiar syntax:
Job job = mockery.NewMock<Job>(MockStyle.Transparent);
Stub.On(job).Method("ValidateFoo").Will(Return.Value(true));
MockStyle.Transparent specifies that anything you don't stub or expect should be handled by the underlying implementation - so you can stub and set expectations for methods on an instance you're testing.
However, you can only stub and set expectations on public methods (and properties), which must also be virtual or abstract. So to avoid relying on integration testing, you have two options:
Make Job.ValidateFoo() public and virtual.
Extract the validation logic into a new class and inject an instance into Job.
Since all private are all called by public methods (unless relying on reflection runtime execution), then those privates are being executed by public methods. Those private methods are causing changes to the object beyond simply executing code, such as setting class fields or calling into other objects. I'd find a way to get at those "results" of calling the private method. (Or mocking the things that shouldn't be executed in the private methods.)
I can't see the class under test. Another problem that could be pushing you to want access to the private methods is that it's a super big class with a boatload of private functionality. These classes may need to be broken down into smaller classes, and some of those privates may turn into simpler publics.