I am writing a gem which adds dependencies to a gemfile from the command line. Given the gem name, it grabs the latest version from rubygems and adds it to the user's gemfile.
I practice test driven development using rspec. I'm wondering how do I mock the existence of a file and check it's contents?
Is there any pattern for doing this sort of thing? Tips, links or examples will help. I'm not set on a specific way of testing this. If mocking isn't the way to go, please let me know. Thanks!
If you want to mock a file object that only needs to be written and read from (no filesystem operations), try Ruby's builtin StringIO. Just require 'stringio', create a 'file' with any contents by using StringIO.new("Some contents"), and check its contents by using the string method on the StringIO object.
Unit Tests vs. Integration Tests
You didn't post any code, so I'm going to have to make some assumptions. In general, exercising behavior from Ruby's core or standard libraries is a waste of time. In addition, your tests should exercise methods that define class behavior, not necessarily every single method in a class.
When you cross the boundary to testing I/O, you're often doing integration tests. Sometimes integration tests are necessary, but they should make a very small subset when compared to unit tests.
With all that said, while integration tests that involve filesystem, database, or network I/O are often slower than unit tests, whether or not they are slow enough to warrant stubbing, mocking, or test-specific work-arounds will be specific to your code base.
Options for Testing
If you really need to do this, you have a lot of options. Some of these options include:
Using a RAM disk for your filesystem I/O.
Stubbing calls to IO, File, or FileUtils.
Avoiding the issue by using a file fixture.
Avoiding the issue by using a StringIO fixture.
Re-writing your class under test to accept String and/or StringIO in addition to File objects.
Using a gem like FakeFS to handle the stubbing/mocking for you.
This isn't meant to be an canonically exhaustive list. There are doubtless other options, but one or more of the above should handle any common use case.
Related
Suppose that I have a Java program that uses JRuby to allow the user to use Ruby scripting to control the behaviour of some funny character in a window.
Users can share their Ruby code with the community, so others can execute the snippets on their own copy of the program and see the funny character do stuff.
I have a security concern with this, though, as users may contribute malicious Ruby code to the community.
The obvious precaution is that users shouldn't run the snippets of untrusted users. However, due to the nature of the community, the point is to check out the creations of strangers.
So, it has occurred to me that maybe I can restrict the capabilities of JRuby.
There may be other things, but some of the restrictions off of my head would be:
Do not allow any sort of networking.
No access to the filesystem.
Do not permit system DOS calls.
Can't require/import ruby code/gems/etc.
Can't create new processes etc.
Is there a reasonable way to restrict JRuby functionality?
I have thought of, perhaps, redefining several constants that are required for that sort of operations. For instance,
File = nil
But I am unsure of what constants to nullify exactly, and whether this is effective at all.
since your requirements are concrete you would likely need to implement those restrictions yourself ... some pointers :
rubygems can be disable within JRuby - assuming it's fine for you to boot that way, otherwise chaing load/require is a good option
same for system and similar calls that create a new process
instead of doing File = nil early on you might end up needing to review File/IO methods one by one
undefine Java constant and java methods to disallow smart cheating with Java APIs
I am about to change my test automation language from Java to Ruby (I have new job where Ruby fits better into the tech stack).
I have a lot of experience with Java and Webdriver but can see that wrappers such as Watir and Capybara appear to be used more in Ruby than directly accessing the Webdriver API.
My concern about using a such a library is performance. I normally try integrate 3rd party grids such as Saucelabs into my test frameworks but have learnt that caching of selenium web element objects is important as continually finding elements can have a performance impact.
If I use libraries such as Capybara, do I lose the ability to control caching strategies? I have previously investigated Geb and found that the framework continually re-created webelements rather than caching, and it appeared inflexible in changing that behaviour.
Is my concern that these libraries help you to avoid writing boiler plate code but at the cost of performance valid?
TL;DR
Convention over configuration; use page-object for caching.
Here are my thoughts on this matter. And please consider this less of an answer, and more of a response for discussion. I want feedback on this answer, feel free to give it.
One of the primary patterns in Ruby (and it actually comes from Rails), is Convention over Configuration. The basic idea is that when there is a convention, either dictated by the language or the community, you roll with it whenever possible. In your case, I would recommend using the community frameworks if at all possible. This will make it easier for other developers to use your code and easier to seek help if you need it.
As far as the actual caching goes, with this I am less familiar. I do know the page-object gem stores elements instead of recreating them during every usage*. This would appear to conform with your requirements of caching elements. In any case, I highly recommend this gem as it enforces the page-object model of testing.
Note: I am not sure if the page-factory mixin supports this caching of objects, or if it recreates the class on every usage.
*You can see how the Element is stored in page-object by viewing the source code to the Element class.
def initialize(element, platform)
#element = element
I won't comment on performance in general as I haven't done extensive comparisons (and you shouldn't really believe anyone who can't point to benchmarks). However, here's a performance tip (with or without wrappers):
Internally, WebDriver uses HTTP to send commands to the browser (or an intermediate server). Since the protocol is quite chatty, you can often gain a lot by switching out the default Net::HTTP-based client with one that supports Keep-Alive (assuming the server supports it). That way the Ruby client will open one socket per browser and reuse it for the duration of the session, instead of opening one per command.
Here's code that uses the net-http-persistent gem:
require "selenium/webdriver"
require "selenium/webdriver/remote/http/persistent"
client = Selenium::WebDriver::Remote::Http::Persistent.new
driver = Selenium::WebDriver.for :remote, url: 'http://...', http_client: client
Unfortunately, I can't say anything specific about Capybara, but I can about watir-webdriver. By default, it always relocates element on the page, so you can't cache it. This behaviour can be turned off with
Watir.always_locate = false
though you can experience stale elements in this case (it, however, actually depends more on you just like with plain WebDriver). If you're planning you use watir-webdriver with page-object gem, its caching won't work as described in #screenmutt answer by default.
Also, there are two nice blog posts from Zejlko Fillipn about watir-webdriver performance when using SauceLabs and TestingBot.
Another issue of watir-webdriver is cross-browser performance. Because it uses XPath under the hood, you'll face significant performance problems when testing in IE.
I'm developing a Rails app with a lot of dependencies on external APIs, for example Delicious.
All APIs share two workflows:
On the first call they are going to load all data since the beginning of time.
All following calls will load data filtered by the last execution time (if supported).
Testing them in real means I must create a test account for each API or at least use my private one. Even with VCR, because they would be called once. And my biggest problem: I would have to mess around a lot with Date's and Time's to emulate the two different workflows mentioned above. Though Timecop makes it really easy, it feels like a pain in the ass.
Another approach is to fake the API calls and their corresponding responses completely, but this means no real tests and furthermore I would never realize changes or problems with the APIs.
Any suggestions? Maybe a good combination of both ways?
Do both.
Start by mocking/stubbing everything in your regular run-frequently test suite. Do this for all the fine-grained model/controller testing.
Then add end-to-end testing (eg in integration tests) that cover usual workflow-scenarios that hit the real (test) servers.
Alternatively use a different test suite for the end-to-end testing eg cucumber instead of Test::Unit, or selenium/Watir whatever as long as it's different to your usual test suite
I am writing an open source gem that interacts with an sms service. I want to test the interaction, however it needs account information and a phone number to run. It also needs feedback to determine if sms messages were being sent correctly. This causes two problems:
I can't put the account information in the test file, as the gem is open source and anyone could get to it.
I need the person running the test to give information to the script as it is running (eg checking the phone to see if a message was received).
What techniques or libraries are available that can help with this? I'm currently using rspec and making it prompt for parameters (using gets), however it is pretty cluncky at the moment. I can't be the first person using ruby to have this problem, and I feel that I'm missing a gem or something that solves this problem.
Use mocks
What are your tests testing, specifically? That a given login/password works? Probably not. Most likely you want to make sure your code reacts to the API properly. Therefore, I'd suggest mocking. Save the output of the API calls and use a mock service to return those responses. Then test. Your tests will be faster and less brittle as a happy side-effect.
More information on mocking with RSpec is here:
http://rspec.info/documentation/mocks/
Re 1) Why not just save configuration options in a YAML file and load them at the beginning of your tests?
Re 2) Are there maybe any web services for that? E.g. one where you can send a message to and query an API to see if it worked. I know this can be unreliable, but the same is true for a user's phone company network.
+1 for Mark Thomas' answer on mocking. Two more alternative mock object libraries for Ruby: FlexMock and Mocha
I would like to create a small Rails application that would allow users to give a few snippets of code to benchmark in multiple implementations and versions of Ruby. I am capable of creating the application, I am just afraid of users mucking around in the filesystem or doing other destructive actions. Is there any way to prevent this?
There is $SAFE:
The variable $SAFE determines Ruby's level of paranoia.
The various "safety levels" are noted at the link, there's also some examples (which still work) of using a thread, $SAFE, and load to wrap untrusted code. $SAFE on ruby might be worth a look as well.