So, whenever I attempt to learn Test Driven Development something always bothers me and I figure there must be something fundamental that I am missing about how unit tests and the process of test-driven-development are supposed to be "done."
Okay, so let's say you have a simple Class, called Calculator. Let's say you have one method that you are attempting to "drive" with your tests. The method is called "Add". It is supposed to take two numbers and give you the sum of those two numbers.
Let's say the test looks like this:
[TestMethod]
public void AddingTwoAndThreeEquals5()
{
var myCalulator = new Calculator();
var result = myCalulator.Add(2, 3);
Assert.AreEqual(5, result);
}
Makes sense as a test, right?
Let's say this is my implementation choice:
public int Add(int a, int b)
{
return 5;
}
This works. The test will pass. Hey, the implementation is even about as simple as it gets--no redundancy whatsoever. How could you refactor that? You're "Green" and good to go! :)
But that is obviously a horrible implementation choice! It will fail almost instantly in Production (just imagine the use-case where "2" and "2" are "Added" together to get "5"--the thought! gasp)! If Test-Driven-Development is just focused on getting the tests to pass as quickly as possible, what is to prevent you from "gaming" the test like this? Surely this is not the quality of code Test Driven Development is supposed to produce, but where is it expected that the code would be corrected to be more "flexible" as it would obviously need to be to function even remotely well in Production?
Do there just need to be enough tests that getting them all to pass at once while doing things like this would not be possible? Is this the sort of thing that should be caught on a code-review with a peer before check-in? Is it just up to the developer to be reasonable and know that this could not possibly pass muster in Production even if it technically makes the test pass--in other words, is the developer just to use common sense to know what is and is not in the spirit of the process even if they technically are getting the tests to go to "Green"?
This is obviously an extreme example as no one would likely check code like this in. But what about Test Driven Development is supposed to prevent "driving" code like this? Its a conceptual stumbling block that I always have when I undertake to learn Test Driven Development and I feel like there is just something fundamental that I am not getting about what Test Driven Development is and is not supposed to "do" for us.
Thank you so much for bearing with me through my question!
Cheers! :)
This works. The test will pass. Hey, the implementation is even about as simple as it gets--no redundancy whatsoever.
Ah - but that's not true, you do have duplication here, and you need to see it.
public int Add(int a, int b)
{
return 5;
}
becomes
public int Add(int a, int b)
{
return 2 + 3;
}
becomes
public int Add(int a, int b)
{
assert a == 2;
assert b == 3;
return a + b;
}
Now the duplication is removed, and you can move on to the next test.
In practice, Beck uses "refactor" somewhat differently from Fowler's definition. In Beck's examples, if a change keeps the tests passing, it counts as a refactor even though the behavior in examples-you-aren't-testing-yet might change.
So we'd expect Beck to skip that last example, and proceed directly to:
public int Add(int a, int b)
{
return a + b;
}
I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence -- Kent Beck, 2008
Now, today's picture is a bit more complicated than that. Rough timeline: Kent (re)introduces test first programming when he begins writing about Extreme Programming (XP). Test Driven Development by Example is published a few years later, and by then the ideas (a) have already been spread and (b) have a number of different interpreters, with the own interpretations, goals, agendas etc.
We've got a lot of different ideas like "Simplest Thing", and "You Aren't Gonna Need It" that are common in the same communities, and people try out applying them to the tests and refactoring of TDD. Sometimes the slogans translate well, other times... not so much?
what about Test Driven Development is supposed to prevent "driving" code like this?
So the first thing to recognize is that "Driven" here is pure marketing garbage. There was a time when people believed that the design followed from the tests - but that didn't really work out, and in retrospect there were some pretty clear tells that it wasn't going to work out. Michael Feathers referred to this period as the Look, Ma, no hands era.
There are some cases where the implementation of the tests gives hints that there is a problem in your design (too much irrelevant set up is a common example). But structure invariant tests simply can't "drive" a structure - all "refactorings" are reversible.
What those tests can do is alert you if a small code change you make to improve the design changes any of the behaviors you measuring. If so, you can evaluate and remedy the contradiction at once (usually by reverting the small change that introduced the fault) and move on.
(Contrast this with a "test first" approach, where the test fails until you have the entire design typed in. That same test may then still fail, and now you have to go digging for the fault you introduced. The shortened feedback loop is the big idea of TDD.)
Now, off the top of my head, a better test is to have two randomly generated integers and the result of Add is the result of adding the first and second integer. Suddenly the test becomes way more robust.
Test Driven Development / Test Driven Design is a design practice, not a test practice. Defending against the malfeasance of the Enterprise Developer from Hell is not a TDD goal.
Your cheat is actually something that is quite common while driving the design ("making the test pass, committing any crime necessary"). It works with 2 and 3 and even with a virtually infinite number of other combinations of numbers. The cheat is resolved by either getting rid of duplication as VoiceOfUnreason shows (5 is duplicated in your test and in your production code, replacing it by a+b removes this duplication) or by "triangulation" (writing more tests that expect different results). TDD is not about being "artificially stupid" or even malicious.
The duplication here is in the test and the production code. Notice the following two lines
Assert.AreEqual(5, result);
return 5;
To remove this duplication you need to think about making your code more generic. As #EricSchaefer rightly mentioned, to make the code more generic you need to follow the triangulation technique.
Related
I'm using VS 2012, but that's not really important.
What is important is that I'm trying to do some TDD by writing all my tests first and then creating the code.
However, the app will not compile because none of my objects or methods exist.
Now, to my mind, I should be able to create ALL my tests but still run my app so I can debug etc. The tests shouldn't prevent compilation because objects and methods are missing.
I thought the whole point of it was that as you develop your tests you can begin to see duplications etc so that you can refactor before writing a single line of code.
So the question is, is there a way to do this or am I doing this wrong?
EDIT
I am using VS2012 and C#
I see a small problem with
writing all my tests first and then creating the code.
You don't need to write ALL your tests first, you just need one, make it fail, make it pass and repeat. That means ideally at any point you should have ideally one failing test.
A compile failure counts as a failed test in that sense. So the next step is to make it pass - i.e. add stubs or return default values to make it compile. The test would then be red.. then work at getting it to green.
Test Driven Development is about very small iterations. You don't define all your tests up front. You create one test based on one fraction of one requirement. Then you implement the code to pass that test. Once it's passing, you work on another fraction of a requirement.
The idea is that trying to do all the design up front (whether it be creating detailed class diagrams or creating a bunch of tests) means that you will find it too expensive to change a weakness in your design, so you won't improve your code.
Here's an example. Let's say you decide to use inheritance to relate two objects, but when you started implementing the objects, you found that made testing them tough. You discover it would be much easier to test each object individually, and relate them via containment instead. What is happening is the tests are driving your design in a more loosely coupled direction. This is a very good outcome of TDD - you are using tests to improve the design.
If you had written all your tests in advance assuming your design decision of inheritance was a good choice, you would either throw away a lot of work, or you would say "it's too tough to make a change like that now, so I'll just live with this sub-optimal design instead."
You can certainly create business-rule-related acceptance tests in advance. Those are called behavior tests (part of Behavior Driven Development, or BDD) and they are good to test features of the software from the user's point of view. But those are NOT unit tests. Unit tests are for testing code from the developer's point of view. Creating the unit tests in advance defeats the purpose of TDD, because it will make testing harder, it will prevent you from improving your code, and will often lead to rebellion and failure of the practice. That's why it's important to do it right.
What is important is that I'm trying to do some TDD by writing all my tests first and then creating the code.
The problem is that "writing all my tests first" is most emphatically not "do[ing] some TDD". Test driven development consists of lots of small repetitions of the "red-green-refactor" cycle:
Add a unit test to the test suite, run it and watch it fail (red)
Add just enough code to the system under test to make all the tests
pass (green)
Improve the design of the system under test (typically
by removing duplication) while keeping all the tests passing
(refactor)
If you code an entire huge test suite up front, you'll spend forever trying to get to the "green" (all tests passing) state.
However, the app will not compile because none of my objects or methods exist.
That's typical of any compiled language; it's not a TDD issue per se. All it means is that, in order to watch the new test fail, you may have to write a minimal stub for whatever feature you're currently working on to make the compiler happy.
For example, I might write this test (using NUnit):
[Test]
public void DefaultGreetingIsHelloWorld()
{
WorldGreeter target = new WorldGreeter();
string expected = "Hello, world!";
string actual = target.Greet();
Assert.AreEqual(expected, actual);
}
And I'd have to then write this much code to get the app to compile and the test to fail:
public class WorldGreeter
{
public string Greet()
{
return String.Empty;
}
}
Once I've gotten the solution to build and I've seen the one failing test, I can add the code to make the first test pass:
public string Greet()
{
return "Hello, world!";
}
Once all tests pass, I can look through the system under test and see what could be done to improve the design. However, it's essential to the TDD discipline to go through both the "red" and "green" steps before playing around with refactoring.
I thought the whole point of it was that as you develop your tests you can begin to see duplications etc so that you can refactor before writing a single line of code.
Martin Fowler defines refactoring as "a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior" (emphasis added). If you haven't written a single line of code, there's nothing to refactor.
So the question is, is there a way to do this or am I doing this wrong?
If you're looking to do TDD, then, yes, I fear you are doing this wrong. You may well be able to deliver great code doing what you're doing, but it isn't TDD; whether or not that's a problem is for you to decide for yourself.
You should be able to create your empty class with stub functions, no?
class Whatever {
char *foo( const char *name ) {}
int can_wibble( Bongo *myBongo ) {}
}
Then you can compile.
No. It's about coding just enough to verify the implementation of the required use cases
You can define your tests cases early, but to code the test cases them you iteratively write a test, have it fail. Then write some code that ensures that the code passes.
Then rinse and repeat until all your test cases are covered,
Edit to address comment.
As you build out the code, that's where your programming designs and faults are identified. Extreme programming lends it self to you being able to change code with out care as the test base protects your requirements. Your intentions are good but the reality is that you'll refactor/redesign test test cases as you discover design issues and flaws through building out the code and test base.
However IMHO, in a very general case, a test that doesn't compile is effectively a meta test that's failing that needs to be corrected before moving on. It's telling you to write some code!
Use mock, from Wikipiedia: mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.
Please refer this.
I found this using dynamic for objects that don't exists yet:
https://coderwall.com/p/0uzmdw
Since I'm a TDD newbie, I'm currently developing a tiny C# console application in order to practice (because practice makes perfect, right?). I started by making a simple sketchup of how the application could be organized (class-wise) and started developing all domain classes that I could identify, one by one (test first, of course).
In the end, the classes have to be integrated together in order to make the application runnable, i.e. placing necessary code in the Main method which calls the necessary logic. However, I don't see how I can do this last integration step in a "test first" manner.
I suppose I wouldn't be having these issues had I used a "top down" approach. The question is: how would I do that? Should I have started by testing the Main() method?
If anyone could give me some pointers, it will be much appreciated.
"Top-down" is already used in computing to describe an analysis technique. I suggest using the term "outside-in" instead.
Outside-in is a term from BDD, in which we recognise that there are often multiple user interfaces to a system. Users can be other systems as well as people. A BDD approach is similar to a TDD approach; it might help you so I will describe it briefly.
In BDD we start with a scenario - usually a simple example of a user using the system. Conversations around the scenarios help us work out what the system should really do. We write a user interface, and we can automate the scenarios against that user interface if we want.
When we write the user interface, we keep it as thin as possible. The user interface will use another class - a controller, viewmodel, etc. - for which we can define an API.
At this stage, the API will be either an empty class or a (programme) interface. Now we can write examples of how the user interface might use this controller, and show how the controller delivers value.
The examples also show the scope of the controller's responsibility, and how it delegates its responsibilities to other classes like repositories, services, etc. We can express that delegation using mocks. We then write that class to make the examples (unit tests) work. We write just enough examples to make our system-level scenarios pass.
I find it's common to rework mocked examples as the interfaces of the mocks are only guessed at first, then emerge more fully as the class is written. That will help us to define the next layer of interfaces or APIs, for which we describe more examples, and so on until no more mocks are needed and the first scenario passes.
As we describe more scenarios, we create different behaviour in the classes and we can refactor to remove duplication where different scenarios and user interfaces require similar behaviour.
By doing this in an outside-in fashion we get as much information as to what the APIs should be as possible, and rework those APIs as quickly as we can. This fits with the principles of Real Options (never commit early unless you know why). We don't create anything we don't use, and the APIs themselves are designed for usability - rather than for ease of writing. The code tends to be written in a more natural style using domain language more than programming language, making it more readable. Since code is read about 10x more than it's written, this also helps make it maintainable.
For that reason, I'd use an outside-in approach, rather than the intelligent guesswork of bottom-up. My experience is that it results in simpler, more strongly decoupled, more readable and more maintainable code.
If you moved stuff out of main(), could you not test that function?
It makes sense to do that, since you might want to run with different cmd-args and you'd want to test those.
You can test your console application as well. I found it quite hard, look at this example:
[TestMethod]
public void ValidateConsoleOutput()
{
using (StringWriter sw = new StringWriter())
{
Console.SetOut(sw);
ConsoleUser cu = new ConsoleUser();
cu.DoWork();
string expected = string.Format("Ploeh{0}", Environment.NewLine);
Assert.AreEqual<string>(expected, sw.ToString());
}
}
You can look at the full post here.
"bottom up" can save you a lot of work cos you already have low-level classes (tested) and you can use them in higher level tests. in opposite case you'd have to write a lot of (probably complicated) mockups. also "top down" often doesn't work as it's supposed: you think out some nice high-level model, test and implement it, and then moving down you realize that some of your assumptions were wrong (it's quite typical situation even for experienced programmers). of course patterns help here but it's not silver bullet too.
to have complete test coverage, you'll need to test your Main in any case. I'm not sure it's worth the effort in all cases. Just move all logic from Main to separate function (as Marcus Lindblom proposed), test it and let Main to just pass command-line arguments to your function.
console output is a simplest possible testing ability, and if you use TDD it can be used just to output the final result w/o any diagnosis and debug messages. so make your top-level function to return solid result, test it and just output it in Main
I recommend "top down" (I call that high level testing). I wrote about that:
http://www.hardcoded.net/articles/high-level-testing.htm
So yes, you should test your console output directly. Sure, initially it's a pain to setup a test so that it's easy to test console output directly, but if you create appropriate helper code, your next "high level" tests will be much easier to write. By doing that, you empower yourself to unlimited re-factoring potential. With "bottom up", your initial class relationship is very rigid because changing the relationship means changing the tests (lots of work and dangerous).
There was an interesting piece in MSDN magazine of December, which describes "how the BDD cycle wraps the traditional Test-Driven Development (TDD) cycle with feature-level tests that drive unit-level implementation". The details may be too technology specific, but the ideas and process outline sound relevant to your question.
The main should be very simple in the end. I don't know how it looks like in c#, but in c++ it should look something like this :
#include "something"
int main( int argc, char *argv[])
{
return TaskClass::Run( argc, argv );
}
Pass already created objects to the constructors of classes, but in unit tests pass mock objects.
For more info about TDD, take a look at these screencasts. They explain how to do agile development, but it also talks how to do TDD, with examples in c#.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
One of the most unpleasant (and unfortuantely most frequent) situations I am confronted with in my all day life as a developer is that I have to fix bugs or add features into code that is badly designed. Now as a good craftsman I would like to leave the code in a better state than I found it. Often new features can not be implemented if I do not refactor the design. Well - they could, but that would make the code even worse.
Unfortunately this is exactly what I tend to have a hard time with. I feel that if there is one thing that is hard, it is to refactor bad code, especially when you have deadlines. Touching bad and complex code that more or less works is scary. As a result I introduce even more clutter when I hack a new feature into the code without modifiying existing code.
Now my question is How can I learn to cope with bad code? How can I learn to understand huge codebases and then refactor parts of it without breaking stuff that already worked and without exceeding the deadline? Is there any literature you can recommend? Do you have any general tips for me?
General tip:
if (it is working)
Do (not touch it);
else
{
Make (as few modifications as possible)
Or (it will stop working at all);
}
This is experience of generations.
Michael Feathers wrote a good book about this subject exactly.
Working Effectively with Legacy Code.
Another great book is by Martin Fowler, Kent Beck and others:
Refactoring: Improving the Design of Existing Code.
Refactoring needs the safety harness of a unit test suite to remove that "Have I broken it?" feeling. Covering the bad code in a blanket of tests will help you whilst you strive for good clean code.
Pex is a tool that I find useful (if you are in the .NET world) for creating tests for legacy code.
Legacy code == code without tests!
When I have to deal with adding functionality to bad code, my usual approach is:
Write automated tests for every important feature that must work (as most bad code doesn't have any tests).
Do code changes.
Make sure the tests are still working.
That give you at least some confidence that you didn't break everything. As for how to learn coping with bad code, I guess it's just about experience.
Well, if you're going to refactor large amounts of code in a project I'd recommend using some decent version control, so you can branch and fall back easily. Given, this is probably is an open door, but crucial imo.
Furthermore, before you start getting in to complex OO, try breaking methods and functions into smaller ones. Ensuring some level of atomicity in each of the functions, which makes the code alot easier to maintain, read and manage.
It's all about the small stuff, break it down into logical units of operation, I'm doing a refactor action on a 1k lines method. It does al sorts of fancy stuff.
My first goal is to get as much stuff out of it in smaller pieces, when that's done I'll start thinking about a better OO design, which is much easier because I've a much better grasp on things.
Aspirin works good too.
I am currently in this situation. My approach is to answer some questions before touching the code:
Is the code really that bad? If yes, what are the common errors? ==> maybe concentrate on those first
What is the main runtime flow in the code? Maybe you can discard quite a few constructs from it.
Try to layer/modularize the code without changing it. This leads to some reduction of interdependencies
Try to poke the code with tests. If the code base is entangled beyond hope: use something like PowerMock to simulate objects that are not (yet) in need of change
Have an integration environment available, in which you can test changes in a production near environment.
Don't shy away to rewrite parts of the code base. But try to not implement too much new stuff in it
Try to team up, discuss designs, principles, solutions
This is tough work, and nobody will thank you for it. Be proud of small improvements and enjoy a good work done :)
I think that it is always good to have a general idea of how everything works in the software you are developing/improving. That's where the design documents and other documents made after or during the development process come in. I believe that if someone before you has not done proper documentation, then at least you should write a couple of lines somewhere about what you experience throughout your development process. I usually use OneNote or other stuff to write notes about what I encounter, and usually keep listing things which I feel would require refactoring. And if there is some downtime during the project, I usually go back to that list and try to improve things bit by bit.
So basically, if someone before you hasn't done it right, it would be good if at least you could help reduce the problems for any other developer who would come across the same code.
Refactoring: Improving the Design of Existing Code
It depends on number of factors but the most important is if you have authority to modify it.
In case you do, refactor it. For example, rename classes/functions/variables. Extract and generalize functionalities. See Refactoring: Improving the Design of Existing Code (Bible for the subject).
Before you begin doing that ensure that code is in a proper version control (VC) and have a good set of test cases. VC let you roll back and test cases help catching unexpected side-effects.
I suggest Distributed Version Control like Mercurial/Bazaar and Git because it is very refactoring is not exactly structured like adding features.
If there were no tests (common), you must create them. Read Working Effectively With Legacy Code. Especially about "Seal point" (not about Siamese cat :p).
In case you don't create a wrapper API that is cleaner.
For example:
Old code ====================
const ACT_SHOW = 'show';
const ACT_HIDE = 'hide';
function int DoThing(Object $Obj, Stirng $Action, Object $Param1, Object $Param1) {
....;
}
Added code ==================
enum Actions {
show, hide;
};
class ActionDoer {
private obj;
ActionDoer($Object) {
this.obj = $Object;
}
function int act(Actions $Action, $Param1, $Param1) {
this.act($Action.toString(), $Param1, $Param1) ;
}
function int act(String $Action, $Param1, $Param1) {
DoThing(this.obj, $Action, $Param1, $Param1) ;
}
function int show() {
this.act(Actions.show, null, null);
}
function int hide(Color $ToBGColor, long $FadeTime) {
this.act(Actions.hide, $ToBGColor, $FadeTime);
}
}
This way, the old code is not touched and the extension can be done using the new code. Good example of this method is jQuery where the old (default) way of accessing DOM is painful.
Hope this helps.
Since I've started using rspec, I've had a problem with the notion of fixtures. My primary concerns are this:
I use testing to reveal surprising behavior. I'm not always clever enough to enumerate every possible edge case for the examples I'm testing. Using hard-coded fixtures seems limiting because it only tests my code with the very specific cases that I've imagined. (Admittedly, my imagination is also limiting with respect to which cases I test.)
I use testing to as a form of documentation for the code. If I have hard-coded fixture values, it's hard to reveal what a particular test is trying to demonstrate. For example:
describe Item do
describe '#most_expensive' do
it 'should return the most expensive item' do
Item.most_expensive.price.should == 100
# OR
#Item.most_expensive.price.should == Item.find(:expensive).price
# OR
#Item.most_expensive.id.should == Item.find(:expensive).id
end
end
end
Using the first method gives the reader no indication what the most expensive item is, only that its price is 100. All three methods ask the reader to take it on faith that the fixture :expensive is the most expensive one listed in fixtures/items.yml. A careless programmer could break tests by creating an Item in before(:all), or by inserting another fixture into fixtures/items.yml. If that is a large file, it could take a long time to figure out what the problem is.
One thing I've started to do is add a #generate_random method to all of my models. This method is only available when I am running my specs. For example:
class Item
def self.generate_random(params={})
Item.create(
:name => params[:name] || String.generate_random,
:price => params[:price] || rand(100)
)
end
end
(The specific details of how I do this are actually a bit cleaner. I have a class that handles the generation and cleanup of all models, but this code is clear enough for my example.) So in the above example, I might test as follows. A warning for the feint of heart: my code relies heavily on use of before(:all):
describe Item do
describe '#most_expensive' do
before(:all) do
#items = []
3.times { #items << Item.generate_random }
#items << Item.generate_random({:price => 50})
end
it 'should return the most expensive item' do
sorted = #items.sort { |a, b| b.price <=> a.price }
expensive = Item.most_expensive
expensive.should be(sorted[0])
expensive.price.should >= 50
end
end
end
This way, my tests better reveal surprising behavior. When I generate data this way, I occasionally stumble upon an edge case where my code does not behave as expected, but which I wouldn't have caught if I were only using fixtures. For example, in the case of #most_expensive, if I forgot to handle the special case where multiple items share the most expensive price, my test would occasionally fail at the first should. Seeing the non-deterministic failures in AutoSpec would clue me in that something was wrong. If I were only using fixtures, it might take much longer to discover such a bug.
My tests also do a slightly better job of demonstrating in code what the expected behavior is. My test makes it clear that sorted is an array of items sorted in descending order by price. Since I expect #most_expensive to be equal to the first element of that array, it's even more obvious what the expected behavior of most_expensive is.
So, is this a bad practice? Is my fear of fixtures an irrational one? Is writing a generate_random method for each Model too much work? Or does this work?
I'm surprised no one in this topic or in the one Jason Baker linked to mentioned
Monte Carlo Testing. That's the only time I've extensively used randomized test inputs. However, it was very important to make the test reproducible, by having a constant seed for the random number generator for each test case.
This is an answer to your second point:
(2) I use testing to as a form of documentation for the code. If I have hard-coded fixture values, it's hard to reveal what a particular test is trying to demonstrate.
I agree. Ideally spec examples should be understandable by themselves. Using fixtures is problematic, because it splits the pre-conditions of the example from its expected results.
Because of this, many RSpec users have stopped using fixtures altogether. Instead, construct the needed objects in the spec example itself.
describe Item, "#most_expensive" do
it 'should return the most expensive item' do
items = [
Item.create!(:price => 100),
Item.create!(:price => 50)
]
Item.most_expensive.price.should == 100
end
end
If your end up with lots of boilerplate code for object creation, you should take a look at some of the many test object factory libraries, such as factory_girl, Machinist, or FixtureReplacement.
We thought about this a lot on a recent project of mine. In the end, we settled on two points:
Repeatability of test cases is of paramount importance. If you must write a random test, be prepared to document it extensively, because if/when it fails, you will need to know exactly why.
Using randomness as a crutch for code coverage means you either don't have good coverage or you don't understand the domain enough to know what constitutes representative test cases. Figure out which is true and fix it accordingly.
In sum, randomness can often be more trouble than it's worth. Consider carefully whether you're going to be using it correctly before you pull the trigger. We ultimately decided that random test cases were a bad idea in general and to be used sparingly, if at all.
Lots of good information has already been posted, but see also: Fuzz Testing. Word on the street is that Microsoft uses this approach on a lot of their projects.
My experience with testing is mostly with simple programs written in C/Python/Java, so I'm not sure if this is entirely applicable, but whenever I have a program that can accept any sort of user input, I always include a test with random input data, or at least input data generated by the computer in an unpredictable way, because you can never make assumptions about what users will enter. Or, well, you can, but if you do then some hacker who doesn't make that assumption may well find a bug that you totally overlooked. Machine-generated input is the best (only?) way I know of to keep human bias completely out of the testing procedures. Of course, in order to reproduce a failed test you have to do something like saving the test input to a file or printing it out (if it's text) before running the test.
Random testing is a bad practice a long as you don't have a solution for the oracle problem, i.e., determining which is the expected outcome of your software given its input.
If you solved the oracle problem, you can get one step further than simple random input generation. You can choose input distributions such that specific parts of your software get exercised more than with simple random.
You then switch from random testing to statistical testing.
if (a > 0)
// Do Foo
else (if b < 0)
// Do Bar
else
// Do Foobar
If you select a and b randomly in int range, you exercise Foo 50% of the time, Bar 25% of the time and Foobar 25% of the time. It is likely that you will find more bugs in Foo than in Bar or Foobar.
If you select a such that it is negative 66.66% of the time, Bar and Foobar get exercised more than with your first distribution. Indeed the three branches get exercised each 33.33% of the time.
Of course, if your observed outcome is different than your expected outcome, you have to log everything that can be useful to reproduce the bug.
I would suggest having a look at Machinist:
http://github.com/notahat/machinist/tree/master
Machinist will generate data for you, but it is repeatable, so each test-run has the same random data.
You could do something similar by seeding the random number generator consistently.
Use of random test data is an excellent practice -- hard-coded test data only tests the cases you explicitly thought of, whereas random data flushes out your implicit assumptions that might be wrong.
I highly recommend using Factory Girl and ffaker for this. (Never use Rails fixtures for anything under any circumstances.)
One problem with randomly generated test cases is that validating the answer should be computed by code and you can't be sure it doesn't have bugs :)
You might also see this topic: Testing with random inputs best practices.
Effectiveness of such testing largely depends on quality of random number generator you use and on how correct is the code that translates RNG's output into test data.
If the RNG never produces values causing your code to get into some edge case condition you will not have this case covered. If your code that translates the RNG's output into input of the code you test is defective it may happen that even with a good generator you still don't hit all the edge cases.
How will you test for that?
The problem with randomness in test cases is that the output is, well, random.
The idea behind tests (especially regression tests) is to check that nothing is broken.
If you find something that is broken, you need to include that test every time from then on, otherwise you won't have a consistent set of tests. Also, if you run a random test that works, then you need to include that test, because its possible that you may break the code so that the test fails.
In other words, if you have a test which uses random data generated on the fly, I think this is a bad idea. If however, you use a set of random data, WHICH YOU THEN STORE AND REUSE, this may be a good idea. This could take the form of a set of seeds for a random number generator.
This storing of the generated data allows you to find the 'correct' response to this data.
So, I would recommend using random data to explore your system, but use defined data in your tests (which may have originally been randomly generated data)
I just refactored some code that was in a different section of the class I was working on because it was a series of nested conditional operators (?:) that was made a ton clearer by a fairly simple switch statement (C#).
When will you touch code that isn't directly what you are working on to make it more clear?
I once was refactoring and came across something like this code:
string strMyString;
try
{
strMyString = Session["MySessionVar"].ToString();
}
catch
{
strMyString = "";
}
Resharper pointed out that the .ToString() was redundant, so I took it out. Unfortunately, that ended up breaking the code. Whenever MySessionVar was null, it wasn't causing the NullReferenceException that the code relied on to bump it down to the catch block. I know, this was some sad code. But I did learn a good lesson from it. Don't rapidly go through old code relying on a tool to help you do the refactoring - think it through yourself.
I did end up refactoring it as follows:
string strMyString = Session["MySessionVar"] ?? "";
Update: Since this post is being upvoted and technically doesn't contain an answer to the question, I figured I should actually answer the question. (Ok, it was bothering me to the point that I was actually dreaming about it.)
Personally I ask myself a few questions before refactoring.
1) Is the system under source control? If so, go ahead and refactor because you can always roll back if something breaks.
2) Do unit tests exist for the functionality I am altering? If so, great! Refactor. The danger here is that the existence of unit tests don't indicate the accuracy and scope of said unit tests. Good unit tests should pick up any breaking changes.
3) Do I thoroughly understand the code I am refactoring? If there's no source control and no tests and I don't really understand the code I am changing, that's a red flag. I'd need to get more comfortable with the code before refactoring.
In case #3 I would probably spend the time to actually track all of the code that is currently using the method I am refactoring. Depending on the scope of the code this could be easy or impossible (ie. if it's a public API). If it comes down to being a public API then you really need to understand the original intent of the code from a business perspective.
I only refactor it if tests are already in place. If not, it's usually not worth my time to write tests for and refactor presumably working code.
This is a small, minor antipattern but it so irritates me that whenever I find it, I expunge it immediately. In C (or C++ or Java)
if (p)
return true;
else
return false;
becomes
return p;
In Scheme,
(if p #t #f)
becomes
p
and in ML
if p then true else false
becomes
p
I see this antipattern almost exclusively in code written by undergraduate students. I am definitely not making this up!!
Whenever I come across it and I don't think changing it will cause problems (e.g. I can understand it enough that I know what it does. e.g. the level of voodoo is low).
I only bother to change it if there is some other reason I'm modifying the code.
How far I'm willing to take it depends on how confident I am that I won't break anything and how extensive my own changes to the code are going to be.
This is a great situation to show off the benefits of unit tests.
If unit tests are in place, developers can bravely and aggressively refactor oddly written code they might come across. If it passes the unit tests and you've increased readability, then you've done your good deed for the day and can move on.
Without unit tests, simplifying complex code that's filled with voodoo presents a great risk of breaking the code and not even knowing you've introduced a new bug! So most developers will take the cautious route and move on.
For simple refactoring I try to clean up deeply nested control structures and really long functions (more than one screen worth of text). However its not a great idea to refactor code without a good reason (especially in a big team of developers). In general, unless the refactoring will make a big improvement in the code or fix an egregious sin I try to leave well enough alone.
Not refactoring per-say but just as a matter of general housekeeping I generally do this stuff when I start work on a module:
Remove stupid comments
Comments that say nothing more than the function signature already says
Comments that are pure idiocy like "just what it looks like"
Changelogs at the top of the file (we have version control for a reason)
Any API docs that are clearly out-of-sync with the code
Remove commented-out chunks of code
Add version control tags like $Id$ if they are missing
Fix whitespace issues (this can be annoying to others though because your name shows up for a lot of lines in a diff even if all you did was change whitespace)
Remove whitespace at the end of the lines
Change tabs->spaces (for that is our convention where I work)
If the refactor makes the code much easier to read, the most common for me would be duplicate code, e.g. an if/else that only differs by the first/last commands.
if($something) {
load_data($something);
} else {
load_data($something);
echo "Loaded";
do_something_else();
}
More than (arguably) three or four lines of duplicate code always makes me think about refactoring. Also, I tend to move code around a lot, extracting the code I predict to be used more frequently into a separate place - a class with its own well-defined purpose and responsibilites, or a static method of a static class (usually placed in my Utils.* namespace).
But, to answer your question, yes, there are lot of cases when making the code shorter does not necessarily mean making it well structued and readable. Using the ?? operator in C# is another example. What you also have to think about are the new features in your language of choice - e.g. LINQ can be used to do some stuff in a very elegant manner but also can make a very simple thing very unreadable and overly complex. You need to weigh these two thing very carefully, in the end it all boils down to your personal taste and, mostly, experience.
Well, this is another "it depends" answer, but I am afraid it has to be.
I almost always break >2 similar conditionals into a switch... most often with regards to enums. I will short a return instead of a long statement.
ex:
if (condition) {
//lots of code
//returns value
} else {
return null;
}
becomes:
if (!condition)
return null;
//lots of code..
//return value
breaking out early reduces extra indents, and reduces long bits of code... also as a general rule I don't like methods with more than 10-15 lines of code. I like methods to have a singular purpose, even if creating more private methods internally.
Usually I don't refactor the code if I'm just browsing it, not actively working on it.
But sometimes ReSharper points out some stuff I just can't resist to Alt+Enter. :)
I tend to refactor very long functions and methods if I understand the set of inputs and outputs to a block.
This helps readability no end.
I would only refactor the code that I come across and am not actively working on after going through the following steps:
Speak with the author of the code (not always possible) to figure out what that piece of code does. Even if it is obvious to me as to what the piece of code is doing, it always helps to understand the rationale behind why the author may have decided to do something in a certain way. Spending a couple of minutes talking about it would not only help the original author understand your point of view, it also builds trust within the team.
Know or find out what that piece of code is doing in order to test it after re-factoring (A build process with unit tests is very helpful here. It makes the whole thing quick and easy). Run the unit tests before and after the change and ensure nothing is breaking due to your changes.
Send out a heads up to the team (if working with others) and let them know of the upcoming change so nobody is surprised when the change actually occurs
Refactoring for the sake of it is one of the roots of all evil. Please don't ever do it. (Unit tests do somewhat mitigate this).
Is vast swaths of commented out code an antipattern? While not code specifically, (it's comments) I see a lot of this kind of behaviour with people who don't understand how to use source control, and who want to keep the old code around for later so they can more easily go back if a bug was introduced. Whenever I see vast swaths of code that are commented out, I almost always remove them in their entirety.
I try to refactor based on following factors:
Do I understand enough to know whats happening?
Can I easily revert back if this change breaks the code.
Will I have enough time to revert the change back if it breaks the build.
And sometimes, if I have enough time, I refactor to learn. As in, I know my change may break the code, but I dont know where and how. So I change it anyways to find out where its breaking and why. That way I learn more about the code.
My domain has been mobile software (cell phones) where most of the code resides on my PC and wont impact others. Since I also maintain the CI build system for my company I can run a complete product build (for all phones) on the refactored code to ensure it doesnt break anything else. But thats my personal luxury which you may not have.
I tend to refactor global constants and enumerations quite a bit if they can be deemed a low refactor risk. Maybe it's just be, but something like a ClientAccountStatus enum should be in or close to the ClientAccount class rather than being in a GlobalConstAndEnum class.
Deletion/updating of comments which are clearly wrong or clearly pointless.
Removing them is:
safe
version control means you can find them again
improves the quality of the code to others and yourself
It is about the only 100% risk free refactoring I know.
Note that doing this in structured comments like javadoc comments is a different matter. Changing these is not risk free, as such they are very likely to be fixed/removed but not dead certain guaranteed fixes as standard incorrect code comments would be.