I am writing an open source gem that interacts with an sms service. I want to test the interaction, however it needs account information and a phone number to run. It also needs feedback to determine if sms messages were being sent correctly. This causes two problems:
I can't put the account information in the test file, as the gem is open source and anyone could get to it.
I need the person running the test to give information to the script as it is running (eg checking the phone to see if a message was received).
What techniques or libraries are available that can help with this? I'm currently using rspec and making it prompt for parameters (using gets), however it is pretty cluncky at the moment. I can't be the first person using ruby to have this problem, and I feel that I'm missing a gem or something that solves this problem.
Use mocks
What are your tests testing, specifically? That a given login/password works? Probably not. Most likely you want to make sure your code reacts to the API properly. Therefore, I'd suggest mocking. Save the output of the API calls and use a mock service to return those responses. Then test. Your tests will be faster and less brittle as a happy side-effect.
More information on mocking with RSpec is here:
http://rspec.info/documentation/mocks/
Re 1) Why not just save configuration options in a YAML file and load them at the beginning of your tests?
Re 2) Are there maybe any web services for that? E.g. one where you can send a message to and query an API to see if it worked. I know this can be unreliable, but the same is true for a user's phone company network.
+1 for Mark Thomas' answer on mocking. Two more alternative mock object libraries for Ruby: FlexMock and Mocha
Related
I maintain a web API written in Ruby. It connects to many third party web services. When writing tests, I stub any function that would need to connect to the network and return bottled data instead.
It has happened to me before that I forget this stubbing step, and my integration tests end up actually connecting to a third party service.
With that in mind, I would like to prevent Ruby from being able to open network connections. When attempted, I would like it to raise an exception instead, pointing out what function I forgot to stub.
Is this possible? What central Ruby function would I need to override to achieve this with minimal other side effects?
What about WebMock? Did you try it? https://github.com/bblimke/webmock
This line should help:
WebMock.disable_net_connect!(allow_localhost: true)
Manual stubbing is, as you've just said, unreliable.
A better solution might be to wrap your code that calls external services behind a facade, and use dependency injection to pass the web handling service into the facade on creation. Your Test Suite then just needs to do the same with a stub service. You'd only need to do this once, and any test which was then testing external code would use the stubbed service.
Check out VCR.
First, take a look at its documentation and see if it's what you need, which I suppose it is. We've been using it at my company for a few years to record one HTTP test for a spec and replay the results for subsequent tests.
We've found it to be invaluable when dealing with external APIs.
I'm currently working on an application that uploads a file to a web service (using spring resttemplate). This upload function returns an id which can be used to download the uploaded file later on.
I want this scenario covered by a test (i'm not talking about unit test - maybe integration or functional test, whichever is appropriate).
What i want to do is the download test case will depends on the result of the upload test (since the id will comes from the upload function) - this will be tested against an actual web service for me to confirm if the upload and download functions works properly.
I'm not sure if this approach that i want to do is correct so if any one can suggest a good approach how to implement it, it would be greatly appreciated.
Thanks in advance!
Since this upload/download functionality is already covered on Unit level
I want this scenario covered by a test (i'm not talking about unit test - maybe integration or functional test, whichever is appropriate).
I know Test chaining is considered harmful
the download test case will depends on the result of the upload test (since the id will comes from the upload function)
and can cause lots of overlap between tests, so changes to one can cascade outwards and cause failures everywhere. Further more the tests should have Atomicity (isolation). But if the trade-off in your case suite you, my advice is to use it.
What you can look at, is a proper Test Fixture strategy. The other Fixture Setup patterns can help you with this.
Sounds like an 'acceptance test' is what is required. This would be basically an integration test of a subsystem for the desired feature.
Have a look at Cucumber as a good easy framework to get started.
Here you would define your steps
Given:
When:
Then:
and you can then test the feature as a whole.
Services external (that You have no control over) to Your application has to be mocked, even in e2e test.
This means that service where You are uploading file should be faked. Just setup dummy http server that is pretending to be real service.
With such fake service you can setup it's behaviour for every test, in example you can prepare file to be downloaded with given id.
Pseudo code:
// given
file = File(id, content);
fakeFileService.addFile(file);
// when
applicationRunner.downloadFile(file.id());
// then
assertThatFileWasDownloaded(file);
This is a test which checks if application can download given file.
File class is some domain object in Your application, not a system
File!
fakeFileService is instance that controls dummy file service.
applicationRunner is a wrapper around Your application that makes
it do what you want.
I recommend You to read "Growing Object-Oriented software guided by tests".
I am using rspec in my test and i would like to protect my API key it when i publish my gem on github.
What are the best practices to do that? should i use VCR and then remove my key from the git log?
Broadly speaking, here are three approaches I have used in the past in similar situations. Which you choose will depend on the details your particular situation.
Test user supplies API key
If your test suite requires, or at least prefers, actual API calls with an actual API key, you can have the caller of the tests supply the credentials when running the tests.
Two most common ways of doing this are:
File in project with well-known name which is not checked into version control. Include an example with fake credentials, which is checked into version control, along with instructions for users to supply their real credentials into the real file before calling test suite.
Read from environment variables. Include instructions for users to set appropriate environment variables before calling test suite.
Otherwise,
Mock out the API
This can be the VCR approach you described. This could also be patching the API call to return some fake results.
Test your domain-specific code separate from the API interaction
Assume the API and the API client behave how you expect. Then, factor out the parts of your code which construct the API input and process the API output. Test properties of your input generated. Test behavior of output processor with known or fake output.
Finally, a warning:
If you have ever committed your API key to version control, it will visible in the history. If you have ever pushed to a public hosting service, it will have been exposed to the Internet, most notably, it will have been exposed to specialized bots which scrape newly-pushed commits for sensitive credentials. If this is you, change your credentials now!
I can't find the original blog post at the moment, but there was at least one report of someone accidentally pushing their AWS credentials to GitHub. They subsequently woke up to a several thousand dollar bill.
I'm developing a Rails app with a lot of dependencies on external APIs, for example Delicious.
All APIs share two workflows:
On the first call they are going to load all data since the beginning of time.
All following calls will load data filtered by the last execution time (if supported).
Testing them in real means I must create a test account for each API or at least use my private one. Even with VCR, because they would be called once. And my biggest problem: I would have to mess around a lot with Date's and Time's to emulate the two different workflows mentioned above. Though Timecop makes it really easy, it feels like a pain in the ass.
Another approach is to fake the API calls and their corresponding responses completely, but this means no real tests and furthermore I would never realize changes or problems with the APIs.
Any suggestions? Maybe a good combination of both ways?
Do both.
Start by mocking/stubbing everything in your regular run-frequently test suite. Do this for all the fine-grained model/controller testing.
Then add end-to-end testing (eg in integration tests) that cover usual workflow-scenarios that hit the real (test) servers.
Alternatively use a different test suite for the end-to-end testing eg cucumber instead of Test::Unit, or selenium/Watir whatever as long as it's different to your usual test suite
Are there any best practices in writing unit tests when 90% of the time I'm building the Oauth connecting class, I need to actually be logging into the remote service?
I am building a rubygem that logs in to Twitter/Google/MySpace, etc., and the hardest part is making sure I have the settings right for that particular provider, and I would like to write tests for that.
Is there a recommended way to do that? If I did mocks or stubs, I'd still have to spend that 90% of the time figuring out how to use the service, and would end up writing tests after the fact instead of before...
On a related note, if I created test API keys for each Oauth provider, and I just used it for the gem for testing purposes, is there any issue in leaving the api key and secret in plain view in the tests? So other people could test it out more realistically too.
nothing wrong with hitting live services in your integration tests.
you should stub out the oauth part completely in your unit tests though
If you're planning on open sourcing the code you're writing, leaving API keys in the code might not be a good idea as you might hit API usage limits or rate limiting when the gem becomes popular and people run the tests frequently which will lead to unexpected test failures.