A wee while ago I ended up on a page which hosted several ruby tools, which had 'crazy' names like 'mangler' or 'executor' or something. The tool's job was to modify you production code (at runtime) in order to prove that your tests were precise.
Unfortunately I would now like to find that tool again, but can't remember what it was called. Any ideas?
I think you're thinking about Heckle, which flips your code to make sure your tests are accurate. Here:
http://seattlerb.rubyforge.org/heckle/
Maybe you're thinking of the Flay project and related modules:
http://ruby.sadi.st/Ruby_Sadist.html
Also you can try my mutant. Its AST based and currently runs under MRI and RBX in > 2.0 mode. It only has a killer for rspec3, but others are possible also.
Related
If a project passes the tests in Ruby 2.0.0-p648 and Ruby 2.3.1, does it make sense to also test with versions such as 2.1.8 and 2.2.3?
Has there been any language features that worked in Ruby 2.0 and Ruby 2.3 but were temporarily not working or worked differently in for example Ruby 2.2?
You should test your code for environments you intend it to be used in. Ruby language version is one thing that can vary. You might also consider testing against JRuby and Rubinius if your code is supposed to support it - e.g. it is provided as a public gem.
Logically, testing on earliest and latest versions should cover most failure scenarios with respect to language features (although not necessarily all language bugs since new ones can be introduced). As far as I know there has not been a Ruby feature that was deliberately added or removed in one version and then that decision reversed in a later version. Unless: Perhaps in your production code you are detecting a feature's existence and then using the feature in full - in which case an intermediate Ruby version which has the feature but not in its latest state could fail.
There may be other caveats too, and philosophically speaking when you start testing you want to avoid too much logical "this should work because . . ." thinking. The point of testing is to demonstrate that your code doesn't fail in ways you have covered (well, there's more depth to it than that, but the answer would get far too long if it dove into test philosophies). If you want to declare "works in all versions of Ruby MRI from 2.0.0 to 2.3.1" then you will feel safer making that statement if you have actually tested it. In fact when making such a statement in a public place, I would usually just say something closer to raw fact - "tested in versions 2.0.0, 2.1.4 and 2.3.1".
Obviously there are diminishing returns. If you have no problem in 2.1.9, it is very unlikely you will have a problem in 2.1.10 - at some point it will cost you more to check every minor variation, even just to look at the test results, than the benefit of extra coverage.
The usual answer to this problem is to test as many variations as your automated test environment can handle and you can be bothered to set up and maintain. If using multiple versions of Ruby is done for you in parallel by your test service provider - e.g. you are using Travis - then it is relatively cheap to test multiple versions. So you may as well get as much coverage on environment variations that you expect to see "in the wild" as you can be bothered to look at.
I'm thinking of removing some tests from my test suite. I don't think it'll lead to code being untested, but I'm not certain. Are there any tools that would enable me to identify code that's tested by the tests I want to remove, but not by anything else?
If you are using Ruby 1.9 how about SimpleCov?
You have some code you want to remove associated with an obsolete piece of functionality from a ruby project. How do ensure that you get rid of all of the code?
Some guidelines that usually help in refactoring ruby apply, but there are added challenges because having code that isn't being called by anything won't break any unit tests.
Update: Has anyone written anything that allows you to guess based on your version control history if there are commits where you have since deleted most, but not all, of the code and can point out the remaining code?
Current thoughts:
Identify the outermost part of the stack associated with the obsolete functionality: the binary script calling it, or the unit tests calling it.
Look for methods that are only called by methods associated with the obsolete functionality. I often use git grep for this.
In theory, running mutation testing and looking for code that used to be mutation resistant when the old test suite applied, but is now mutation prone might help. It only helps if your code was well-tested in the first place! (Or you can use code coverage tools such as rcov rather than mutation testing)
Running test suites will ensure you haven't removed anything you shouldn't have!
Using autotest can save you time if you're constantly running tests.
If your code was well-structured, it should be easier to find related methods that need to be removed.
Especially in a dynamically typed language, there is no easy way to do this. If you have unittests, thank the developer that wrote them because it will help you remove the code correctly. But you're basically SOL. Remove the code, if it breaks, put it back, figure out where it broke, attempt to work around it, and repeat.
Look at your code coverage. Any code which isn't covered may be part of the code you have left to remove (if any). (Just be sure you have removed you tests. =])
Does Ruby have any tools along the lines of pylint for analyzing source code for errors and simple coding standards?
It would be nice if it could be integrated with cruisecontrolrb for continuous integration.
Or does everyone write such good tests that they don't need source code checkers!
I reviewed a bunch of Ruby tools that are available here
http://devver.wordpress.com/2008/10/03/ruby-tools-roundup/
most of the tools were mentioned by webmat, but if you want more information I go pretty in depth with examples.
I also highly recommend using Metric-Fu it gives you a on gem/plugin install of 3 of the more popular tools and is built with cruisecontrolrb integration in mind.
The creator has a great post that should help get you up and running in no time.
http://jakescruggs.blogspot.com/2008/04/dead-simple-rails-metrics-with-metricfu.html
There has been a lot of activity in Ruby tools lately which I think is a good sign of a growing and maturing language.
Check these out:
on Ruby Inside, an article presenting Towelie, Flay and Simian, all tools to find code duplication
Towelie
flay
Simian (a more general tool that supports Ruby among other languages)
reek: a code smell detector for Ruby
Roodi: checks the style of your Ruby code
flog: a code complexity analyzer
rcov: will give you a C0 (if I remember correctly) code coverage analysis. But be careful though. 100% coverage is very costly and doesn't mean perfect code.
heckle: changes your code in subtle manners and runs your test suite to see if it catches it. Brutal :-)
Since they're all command-line tools they can all be integrated simply as cc.rb tasks. Just whip out your regex skillz to pick the important part of the output.
I recommend you first try them out by hand to see if they play well with your codebase and if you like the info they give you. Once you find a few that give you value, then spend time integrating them in your cc.
One recently-updated interesting-looking tool is Ruby Object Oriented Design Inferometer - roodi for short. It's at v1.3.0, so I'm guessing it's fairly mature.
I haven't tried it myself, because my code is of course already beyond reproach (hah).
As for test coverage (oh dear, I haven't tried this one either) there's rcov
Also (look, I am definitely going to try some of these today. One at least) you might care to take a look at flog and flay for another style checker and a refactoring candidate finder.
There's also the built-in warnings you can enable with a quick:
ruby -w
Or setting the global variable $VERBOSE to true at any point.
Code metrics on ruby toolbox website.
Rubocop is a widely-used static code analyzer.
I just released Excellent which implements several checks on Ruby code. It combines roodi, reek and flog and also adds some Rails specific checks:
gem sources -a http://gems.github.com
sudo gem install simplabs-excellent
May be helpful...
What do you think about this build tool? I'm thinking of migrating from maven2 to raven (my poms are getting bigger and bigger), but I'd like to hear some opinions first.
Thanks!
#andre:
Thank's for writing but I was actually looking for real experiences using raven. Anyway, the fact that nobody wrote is an indicator by itself (it seems few people are using it)
I haven't used either Raven or Buildr, but I have heard good things about the latter. In this blog article by Assaf Arkin, there is a nice case study: a 5,443 line, 52 file Maven configuration was reduced to 485 lines of Buildr. And, even though everybody says "Ruby is slow", Buildr was 2-6x faster than Maven.
Also, unlike Raven, Buildr seems to still be maintained: it is currently in the incubator stage as an official Apache project.
pom growth is a problem that everybody faces w/ maven I guess, but maven is at least maintained (2.1. just around the corner) and the raven project looks pretty dead to me. No updates this year and the mailinglist archives are also very small. It looks to me as it's too risky to switch your build process to a tool w/o a living community. Not quite the answer you wanted I guess, but my 2 cents.
I don't know anything about raven.
You should check out plain old rake, which lets you create very powerful tasks.
I've also heard about sake, which is just like rake tasks but system-wide, instead of being only available inside one of your projects.
They may not be specialized for Java, but they sure beat the hell out of plain old bash or (heaven forbid) batch scripts.