Are there any best practices in writing unit tests when 90% of the time I'm building the Oauth connecting class, I need to actually be logging into the remote service?
I am building a rubygem that logs in to Twitter/Google/MySpace, etc., and the hardest part is making sure I have the settings right for that particular provider, and I would like to write tests for that.
Is there a recommended way to do that? If I did mocks or stubs, I'd still have to spend that 90% of the time figuring out how to use the service, and would end up writing tests after the fact instead of before...
On a related note, if I created test API keys for each Oauth provider, and I just used it for the gem for testing purposes, is there any issue in leaving the api key and secret in plain view in the tests? So other people could test it out more realistically too.
nothing wrong with hitting live services in your integration tests.
you should stub out the oauth part completely in your unit tests though
If you're planning on open sourcing the code you're writing, leaving API keys in the code might not be a good idea as you might hit API usage limits or rate limiting when the gem becomes popular and people run the tests frequently which will lead to unexpected test failures.
Related
I'm using PHP with Laravel 5.5 framework.
I recently started writing unitTests for my code and I got a few questions:
What is the best way to interact with my database?
Should I use InMemoryDB like SQLite or Mock everything with Mockery.
If I have an interaction with DB than that is still unitTesting or Integration Testing?
Thank you for the answers in advence
I work in a company where we strive for 80% code coverage, in general we test mostly End-2-End, with database and mocking external calls, we use SQLite so our testsuite can run quickly in a local environment. When the case make sense, we unit tests it, as an example an Tax service i did for different countries i unit tested, because it was very input output based.
Why we prefer End-2-End:
It's quicker if you don't have to make unit, integration and end to end testing
You test the endpoint will actually will be used
I prefer to run with a real database if you are running with Continous Integration
There is drawbacks with SQLite, mainly it does not work as other RDB where there is a lot of settings and limitations, on top of my head i had problems with foreign key enforcing etc.
So to answer your question:
It's smart to use SQLite at least locally
In Unit Testing you are only testing one class and mocking everything else, you are basically testing that the code can execute. Note this is a oversimplified version on a very complex subject.
I am starting a new project to create an authentication api using IdentityServer4 following TDD. Many microservices and websites will be using this to authenticate users. But I could not figure out first 3 acceptance tests for the project. Any help will be highly appreciated.
Note: I have recently read goos
Well, in the book they suggest starting with the simplest success case possible. For an authentication service that would probably be a successful authentication.
So your first acceptance test could look something like that:
When: receiving valid user data
Then: authentication should be successful
That may seem awfully small for an acceptance test that is supposed to test a whole system, but your system is also very small and there aren't many user stories to handle. Basically only authentication success, fail and maybe a test that covers the case when a user has tried to log in too many times without success.
Your unit tests then can go more into detail about the actual authentication mechanism, but the acceptance test should always be about the user story.
I guess one could also argue that you don't need to write acceptance tests for your authentication service at all, since it is only a part of your system and you should rather write acceptance tests for your whole system, meaning when you have brought all the microservices together or for each individual website that will rely on that service. The main reason for this kind of argument is that acceptance testing is about testing from the outside in and your authentication service is already a rather deep component of your system.
I would like to do integration testing on my Web API Controllers.
When the integration test starts the whole request/response pipeline of the Web API should be processed so its a real integration test.
I have read some blogs about non-InMemory-tests or InMemory tests. I need to know what is the difference and what of those approaches matches my above criteria?
I would really be glad about some explanations from people who really dealt with integration testing on Web API for self-hosting or IIS hosting (if there is a difference in testing...)
Not sure what you mean by non-in-memory testing but with integration testing involving an in-memory hosted web API, the requests are sent directly to the HttpServer, which is basically the first component to run in ASP.NET Web API pipeline. This means, the requests do not hit the network stack. So, you don't need to worry about running on specific ports, etc and also if you write good amount of tests, the time it takes to run all your tests will not be too big, since you deal with in-memory and not network. You should get comparable running times as a typical unit test. Look at this excellent post from Kiran for more details on in-memory testing. In-memory testing will exercise all the components you setup to run in the pipeline but one thing to watch out for is formatters. If you send ObjectContent in the request, there is no need to run media-formatters, since the request is already in deserialized format and hence media formatting does not happen.
If you want to get more closer and willing to take a hit on the running time, you can write your tests using a self-host. Is that what you mean by non-in-memory testing? As an example, you can use OWIN self-hosting. You can use Katana hosting APIs and host your web API and hit it with your requests. Of course, this will use the real HttpListener and the requests do traverse the network stack although it is all happening in the same machine. The tests will be comparatively slower but you get much closer to your prod runs probably.
I personally have not seen anyone using web-hosting and doing lots of integration testing. It is technically possible to fire off your requests using HttpClient and inspect the response and assert stuff but you will not have lot of control over arranging your tests programatically.
My choice is to mix and match, that is, use in-memory tests as much as possible and use Katana-based host only for those specific cases I need to really hit the network.
I'm developing a Rails app with a lot of dependencies on external APIs, for example Delicious.
All APIs share two workflows:
On the first call they are going to load all data since the beginning of time.
All following calls will load data filtered by the last execution time (if supported).
Testing them in real means I must create a test account for each API or at least use my private one. Even with VCR, because they would be called once. And my biggest problem: I would have to mess around a lot with Date's and Time's to emulate the two different workflows mentioned above. Though Timecop makes it really easy, it feels like a pain in the ass.
Another approach is to fake the API calls and their corresponding responses completely, but this means no real tests and furthermore I would never realize changes or problems with the APIs.
Any suggestions? Maybe a good combination of both ways?
Do both.
Start by mocking/stubbing everything in your regular run-frequently test suite. Do this for all the fine-grained model/controller testing.
Then add end-to-end testing (eg in integration tests) that cover usual workflow-scenarios that hit the real (test) servers.
Alternatively use a different test suite for the end-to-end testing eg cucumber instead of Test::Unit, or selenium/Watir whatever as long as it's different to your usual test suite
I'm writing a client library against an API that communicates using UDP through a socket connection. I'm trying to write tests as I go however I'm running into a few problems.
Should I just build a testing server and start that up in my test helper? I don't want to hammer the external servers when running my tests.
Or should I just mock the hell out of the UDPSocket lib and test against that?
My problem with the first option is that I then have to keep up with changes to the API and make sure that my dummy server emulates them, which could lead to false positives and brittle tests. And my problem with 2 is that excessive mocking could also lead to brittle tests in case anything in UDPSocket changes.
However I've been spiking on this for a couple of days now and having big gaps of missing test coverage is making me a bit nervous. What would you do?
thanks
I tend to mock this sort of thing. Anytime you're communicating with an external host, you probably want to mock. It doesn't make a whole lot of sense for your unit tests to be dependent on internet connectivity.
I would also advise against going to the trouble of making a sophisticated emulated server. Mocking is pretty much the same thing, without having to write the logic. Either way, you're not going to be talking to The Real Thing.