I maintain a web API written in Ruby. It connects to many third party web services. When writing tests, I stub any function that would need to connect to the network and return bottled data instead.
It has happened to me before that I forget this stubbing step, and my integration tests end up actually connecting to a third party service.
With that in mind, I would like to prevent Ruby from being able to open network connections. When attempted, I would like it to raise an exception instead, pointing out what function I forgot to stub.
Is this possible? What central Ruby function would I need to override to achieve this with minimal other side effects?
What about WebMock? Did you try it? https://github.com/bblimke/webmock
This line should help:
WebMock.disable_net_connect!(allow_localhost: true)
Manual stubbing is, as you've just said, unreliable.
A better solution might be to wrap your code that calls external services behind a facade, and use dependency injection to pass the web handling service into the facade on creation. Your Test Suite then just needs to do the same with a stub service. You'd only need to do this once, and any test which was then testing external code would use the stubbed service.
Check out VCR.
First, take a look at its documentation and see if it's what you need, which I suppose it is. We've been using it at my company for a few years to record one HTTP test for a spec and replay the results for subsequent tests.
We've found it to be invaluable when dealing with external APIs.
Related
I'm currently working on an application that uploads a file to a web service (using spring resttemplate). This upload function returns an id which can be used to download the uploaded file later on.
I want this scenario covered by a test (i'm not talking about unit test - maybe integration or functional test, whichever is appropriate).
What i want to do is the download test case will depends on the result of the upload test (since the id will comes from the upload function) - this will be tested against an actual web service for me to confirm if the upload and download functions works properly.
I'm not sure if this approach that i want to do is correct so if any one can suggest a good approach how to implement it, it would be greatly appreciated.
Thanks in advance!
Since this upload/download functionality is already covered on Unit level
I want this scenario covered by a test (i'm not talking about unit test - maybe integration or functional test, whichever is appropriate).
I know Test chaining is considered harmful
the download test case will depends on the result of the upload test (since the id will comes from the upload function)
and can cause lots of overlap between tests, so changes to one can cascade outwards and cause failures everywhere. Further more the tests should have Atomicity (isolation). But if the trade-off in your case suite you, my advice is to use it.
What you can look at, is a proper Test Fixture strategy. The other Fixture Setup patterns can help you with this.
Sounds like an 'acceptance test' is what is required. This would be basically an integration test of a subsystem for the desired feature.
Have a look at Cucumber as a good easy framework to get started.
Here you would define your steps
Given:
When:
Then:
and you can then test the feature as a whole.
Services external (that You have no control over) to Your application has to be mocked, even in e2e test.
This means that service where You are uploading file should be faked. Just setup dummy http server that is pretending to be real service.
With such fake service you can setup it's behaviour for every test, in example you can prepare file to be downloaded with given id.
Pseudo code:
// given
file = File(id, content);
fakeFileService.addFile(file);
// when
applicationRunner.downloadFile(file.id());
// then
assertThatFileWasDownloaded(file);
This is a test which checks if application can download given file.
File class is some domain object in Your application, not a system
File!
fakeFileService is instance that controls dummy file service.
applicationRunner is a wrapper around Your application that makes
it do what you want.
I recommend You to read "Growing Object-Oriented software guided by tests".
Using WebMock to stub an Oauth 2 Provider. The issue is that I want to use Rack Test.
Rack Test only runs against an instance of the app, and does not know about external HTTP hosts like the Oauth 2 Provider.
WebMock.stub_request(:get, "https://test.oauth-provider.com/oauth/authorize") won't work because the request is sent to the app as /oauth/authorize.
Is there a way for WebMock to respond to local requests? For example:
WebMock.stub_request(:get, "/oauth/authorize")
It seems to me, you're a little bit confused about what you're testing, and must draw the clear border, where your system is (so-called SuT, system under test) and where are the external parties.
It is very important since:
SuT is what you're going to interact with during your tests (i.e. your Rack application);
external parties are to be mocked (i.e. external web services, and... which may be surprising, other libraries, like OAuth, SQL drivers etc.)
That means that if you're trying to use WebMock for any part of your application (which it looks like what you're doing due to the question about mocking a relative URL), you're clearly doing something wrong.
Closer to your task, if I were you, I would:
Pick a good, well-tested OAuth library and drop it into the application.
When it comes to testing, just use my own simple stub objects instead of the real OAuth implementation classes. This will shift the focus on testing the behavior my service implements. Verifying OAuth library is really a double work, since it's already covered by its authors.
Hope this helps!
Some of my apps grab data from the internet. I'm getting into the habit of writing unit tests, and I suspect that if I write a test for values returned from the internet, I will run into latency and/or reliability problems.
What's a valid way to test data that "lies on the other end of a web request"?
I'm using the stock unit testing toolkit that comes with Xcode, but the question theoretically applies to any testing framework.
Unit test is focused specifically on the business logic of your class. There would no latency, reliability etc as you would use some mock object to simulate what you actually interact.
What you are describing is some form of integration testing and for the OP seems like is not what you intent.
You should "obscure" the other end by mocking and not really access the network, a remote database etc.
Among others:
introduce artificial latency in requests
use another machine on the same network or at least another VM
test connection failures (either by connecting to a non existent server or cutting physically the connection
test for incomplete data (connection could be cut half way)
test for duplicate data (app could try to submit the request more than once if it thinks it was not successful - and in some scenarios may lead to lost data)
All of these should fail gracefully (either on the server side or on the client side)
I posed this question to the venerable folks on #macdev IRC channel on freenode.net, and I got a few really good answers.
Mike Ash (of mikeash.com) suggests implementing a local web server inside my app. For complex cases, I'd probably do this. However, I'm just using some of the built in initWithContentsOfURL:(NSURL *)url method of NSData.
For simpler cases, mike says an alternate method is to pass base64 encoded dummy data directly to the NSData initializer. (Use data://dummyDataEncodedAsBase64GoesAfterTheDataProtocolThingy.)
Similarly, "alistra" suggests using local file URLs that point to files containing mock data.
I am writing an open source gem that interacts with an sms service. I want to test the interaction, however it needs account information and a phone number to run. It also needs feedback to determine if sms messages were being sent correctly. This causes two problems:
I can't put the account information in the test file, as the gem is open source and anyone could get to it.
I need the person running the test to give information to the script as it is running (eg checking the phone to see if a message was received).
What techniques or libraries are available that can help with this? I'm currently using rspec and making it prompt for parameters (using gets), however it is pretty cluncky at the moment. I can't be the first person using ruby to have this problem, and I feel that I'm missing a gem or something that solves this problem.
Use mocks
What are your tests testing, specifically? That a given login/password works? Probably not. Most likely you want to make sure your code reacts to the API properly. Therefore, I'd suggest mocking. Save the output of the API calls and use a mock service to return those responses. Then test. Your tests will be faster and less brittle as a happy side-effect.
More information on mocking with RSpec is here:
http://rspec.info/documentation/mocks/
Re 1) Why not just save configuration options in a YAML file and load them at the beginning of your tests?
Re 2) Are there maybe any web services for that? E.g. one where you can send a message to and query an API to see if it worked. I know this can be unreliable, but the same is true for a user's phone company network.
+1 for Mark Thomas' answer on mocking. Two more alternative mock object libraries for Ruby: FlexMock and Mocha
I have a C# module responsible for acquiring the list of network adapters that are "connected to the internet" on a windows Vista machine. The module uses the "Network List Manager API" (or NLM API) to iterate over all network connections and returns all those for which the IsConnectedToInternet value is true.
I received some suggestions for the implementation of this module in this SO question
To test this module I've decided to write a helper that returns the list of internet connected interfaces based on another logic, so it would be a sort of a "reality check" for the original module's logic. Note that for the test helper I am willing to use detection methods that might be considered bad practice for production code (e.g. relying on some internet resource like "Google" to be available - in case it shuts down, blocked by our internal firewall etc. it's relatively easy to fix the test as opposed to a deployed product base).
The alternative detection method I chose was to try to connect to "www.google.com:80" with a TcpClient. My problem: When I have more than one connected adapter (e.g. both wireless and LAN) the detection method fails for one of them with the error "A connect request was made on an already-connected socket".
My question is three fold:
How would you go about testing such a module in general? Do you support the idea of doing the same thing in a different way and comparing the results or is it an overkill and I should rely on the system's API? My main problem here, is that it's very hard to pre-configure the system so that I'll know what the expected results are in advance.
What alternative logic would you suggest? One thing that was suggested in the aforementioned question was looking at the routing table - what about considering each adapter that has a routing entry with a destination of 0.0.0.0 as "connected to the internet"? Other suggestions?
Do you understand why I get the "already-connected" error with the current test logic?
I can only answer your question about the unit test.
The code you're testing is, in your own words, "a C# module responsible for acquiring the list of network adapters that are 'connected to the internet' on a windows Vista machine. The module uses the 'Network List Manager API' (or NLM API) to iterate over all network connections and returns all those for which the IsConnectedToInternet value is true."
If I were writing this module, I would first use an interface for the NLM API, call it...NLMAPIService. Now, for the real code, create an Adapter that implements NLMAPIService and adapts the real NLM API.
For testing, create a class FakeNLMAPI that implements NLMAPIService and has all of its data in-memory somewhere, or in an XML file, or whatever. Your module calls methods only on the NLMAPIService, so you don't have to change any "real" code depending on whether you're testing or not.
Therefore, in your test setup method, you can instantiate FakeNLMAPI and pass it to your module, and in production, instantiate your NLM API Adapter.
I'm going to assume that you can instantiate and modify the object that represents a network connection. If not, you can follow the same pattern for faking the actual network connection object.
Dependency Injection is a very handy pattern to deal with issues like this. Instead of simply using the NLM API components directly in your code define an interface and a class that implements it and serves as a proxy to the NLM API. Pass an instance of this class to your module in the constructor and have your module use it. In your unit tests, instead of the real proxy object, use a mock object that returns known information -- it doesn't even have to reference the NLM API -- to use in testing the logic of your module. Granted, your proxy class will need some testing as well, but the logic in it is much simpler -- probably just some data marshaling. You might be able to convince yourself of its correctness or, if not, do some manual testing on it to make sure that it is working properly.
UnitTests shouldn't access to external resources. To UnitTest your method, I would stub out the Network List Manager API.
You still need an acceptance test layer. In that test environment you should replicate various configurations you expect to support in your environment, setup your own webhosts, routers, machine config. Acceptance testing should be done at the user experience level using a tool like Fitnesse.