In The Guerilla Guide to Interviewing Joel says that guys who want to get things done, but are not smart will do stupid things like using a visitor design pattern where a simple array would be sufficient.
I find it hard to detect, if the design pattern suggested by the Gang of Four should be applied.
Therefore, I would like some examples from Your work experience
When is a simple approach (fixed size array) sufficient?
What is the minimum size of a piece of software that justifies the use of the GoF patterns?
When to refactor from simple-minded to GoF? Can this be done in a sensible way?
I often find that using test driven development helps guide me when faced with these questions.
When is a simple approach
sufficient? It is always sufficient
to use the simplest approach to get
the next test to pass. But knowing
when/how to refactor is the real art
form.
What is the minimum size of a
piece of software that justifies the
use of the GoF patterns? A rule of
thumb I once read is that when you
code something once, fine, when you
duplicate that code somewhere a
second time, make a note and move
on. When you find a need for the
same code a third time, it's time to
refactor to remove duplication and
simplify, and often that involves
moving to a design pattern.
When to
refactor from simple-minded to GoF?
I like what #anopres said - it's
time when you feel the pain of not
having the design pattern in place.
The pain (or code "smell") may
manifest itself in several ways.
Code duplication is the most
obvious. Refactoring books like
Fowler's Refactoring or
Kerievsky's Refactoring to
Patterns list many such pain
points/code stenches.
Can this
[refactoring] be done in a sensible
way? The trick to refactoring is to
have a suite of unit tests in place
which you have confidence in, and
then to refactor without causing any
of those tests to fail.
Refactoring, by definition, does not
change the functionality of your
code. Therefore, if your tests
continue to pass, you can have a
pretty good feeling that you didn't
break anything. Although it can be difficult, I actually enjoy this part of TDD, it's almost like a game to make changes without breaking any tests.
In summary, I would say that TDD helps guide me to write the code that is sufficient at the time, and perhaps more importantly helps me to make the changes later when inevitably requirements change, more functionality is required, etc.
Design Patterns are a consequence, not an objective. You don't think today I shall use Strategy Patterns, you just do it. Halfway through doing the third nearly identical class you stop and use a paper notebook to figure out the general case and knock up a base class that describes the shared context. You refactor the first two classes to be descendants, and this gives you a reality check and quite a few changes to your base class. Then the next thirty are a walk in the park.
It's only the next day at the team meeting that you save everyone thirty minutes of boredom by saying "I used strategy pattern. They all work the same so there's only one test program, it takes parameters to change the test case."
Intimate familiarity with patterns makes you use them reflexively, whenever the situation demands it. When people treat the use of patterns as an objective in its own right, you get stilted, ugly code that speaks of mechanism rather than purpose; the how rather than the why.
Most patterns address recurring fundamental problems like complexity mitigation and the need to provide extensibility points. Providing extensibility points when its clear they won't be needed pointlessly complicates your code and creates more failure points and test cases. Unless you're building a framework for release into the wild, solve only the problems you actually face.
Patterns are only tools and vocabulary. You write the code to be as simple, understandable and maintainable as you know how. By knowing patterns you have more allternatives at your disposal, and you have a language to discuss pros and cons of the approach before implementing it.
In either case you dont just "switch" to "using a pattern". You just keep doing what you allways do, write the code the best way you know how.
This is similar to any other design decision. Ultimately, it depends. You should learn those patterns that are useful in your language (many GoF patterns aren't needed in Lisp or Smalltalk, for example), learn their advantages and disadvantages, understand the constraints of your system, and make the choice that best fits your needs.
The best advice that I can give is to learn, learn, learn.
Switching from a simple approach to a formal design pattern is usually something that happens fairly naturally for me as a problem increases in complexity. The key is to be familiar enough with the patterns that you can recognize the tipping point and switch from the simple approach to a design pattern when it will bring the most benefit for current and future development.
For a larger, more complex project, the tipping point should be fairly early on; in many cases, before you even start coding. For smaller projects, you can afford to wait before deciding to implement a pattern.
One of the best things you can do to increase your ability to recognize when a pattern should be used is to take some time after completing a project to examine how complex your "simple" approach has become. If it would have taken you less time and effort to implement a pattern, or if the pattern would clarify what you were trying to do, you can file that knowledge away for the next time you encounter a similar problem.
When you have a problem that one of the patterns solves. The GoF book has a section in each chapter that explains what types of scenarios each pattern is appropriate for. You should not analyze each problem you have, then go look up what pattern to use. You should become familiar with the patterns so that you learn to recognize what situations call for them.
One of the nice things about the the original GoF book is that there is discussion of the scenarios where the pattern would best solve problem. Reviewing these discussions can help you determine if "it's time".
Another good place to start is with Head First Design Patterns. The exercises that illustrate the use of different design patterns are elaborate enough to offer a good learning experience. In addition the exercises are also grounded in real world scenarios, so it's never a stretch to see when the appropriate times to apply design patterns.
Related
Background:
An art teacher once gave me a design problem to draw a tiger using only 3 lines. The idea being that I study a tiger and learn the 3 lines to draw for people to still be able to tell it is a tiger.
The solution for this problem is to start with a full drawing of a tiger and remove elements until you get to the three parts that are most recognizable as a tiger.
I love this problem as it can be applied in multiple disciplines like software development, especially in removing complexity.
At work I deal with maintaining a large software system that has been hacked to death and is to the point of becoming unmaintainable. It is my job to remove the burdensome complexity that was caused by past developers.
Question:
Is there a set process for removing complexity in software systems - a kind of reduction process template to be applied to the problem?
Check out the book Refactoring by Martin Fowler, and his http://www.refactoring.com/ website.
Robert C. Martin's Clean Code is another good resource for reducing code complexity.
Unfortunately, the analogy with the tiger drawing may not work very well. With only three lines, a viewer can imagine the rest. In a software system, all the detail has to actually be there. You generally can't take much away without removing something essential.
Check out the book Anti-Patterns for a well-written book on the whole subject of moving from bad (or maladaptive) design to better. It provides ways to recover from a whole host of problems typically found in software systems. I would then add support to Kristopher's recommendation of Refactoring as an important second step.
Checkout the book, Working Effectively with Legacy Code
The topics covered include
Understanding the mechanics of software change: adding features, fixing bugs, improving design, optimizing performance
Getting legacy code into a test harness
Writing tests that protect you against introducing new problems
Techniques that can be used with any language or platform—with examples in Java, C++, C, and C#
Accurately identifying where code changes need to be made
Coping with legacy systems that aren't object-oriented
Handling applications that don't seem to have any structure
This book also includes a catalog of twenty-four dependency-breaking techniques that help you work with program elements in isolation and make safer changes.
While intellectually stimulating, the concept of detail removal doesn't carry very well (at least as-is) to software programs. The reason being that the drawing is re-evaluated by a human with it ability to accept fuzzy input, whereby the program is re-evaluated by a CPU which is very poor at "filling the blanks". Another more subtle reason is that programs convey a spaciotemporal narrative, whereas the drawing is essentially spacial.
Consequently with software there is much less room for approximation, and for outright removal of particular sections of the code. Never the less, refactoring is the operational keyword and is sometimes applicable even for them most awkward legacy pieces. This discipline is however part art part science and doesn't have very many "quick tricks" that I know of.
Edit: One isn't however completely helpless against legacy code. See for example the excellent book references provided in Alex Baranosky and Kristopher Johnson's answers. These books provide many useful techniques, but on the whole I remain strong in my assertion that refactoring non-trivial legacy code is an iterative process that requires both art and science (and patience and ruthlessness and gentleness ;-) ).
This is a loaded question :-)
First, how do we measure "complexity"? Without any metric decided apriori, it may be hard to justify any "reduction" project.
Second, is the choice entirely yours? If we may take an example, assume that, in some code base, the hammer of "inheritance" is used to solve every other problem. While using inheritance is perfectly right for some cases, it may not be right for all cases. What do you in such cases?
Third, can it be proved that behavior/functionality of the program did not change due to refactoring? (This gets more complex when the code is part of a shipping product.)
Fourth, you can start with start with simpler things like: (a) avoid global variables, (b) avoid macros, (c) use const pointers and const references as much as possible, (d) use const qualified methods wherever it is the logical thing to do. I know these are not refactoring techniques, but I think they might help you proceed towards your goal.
Finally, in my humble opinion, I think any such refactoring project is more of people issue than technology issue. All programmers want to write good code, but the perception of good vs. bad is very subjective and varies across members in the same team. I would suggest to establish a "design convention" for the project (Something like C++ Coding Standards). If you can achieve that, you are mostly done. The remaining part is modify the parts of code which does not follow the design convention. (I know, this is very easy to say, but much difficult to do. Good wishes to you.)
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I would like to know the reasons that we do refactoring and justify it. I read a lot of people upset over the idea of refactoring. Refactoring was variously described as:
A result of insufficient upfront
design.
Undisciplined hacking
A dangerous activity that needlessly risked destabilizing
working code
A waste of resources.
What are the responsible reasons that lead us to refactor our code?
I also found a similar question here how-often-should-you-refactor, it doesn't provide the reason for refactoring.
Why do we refactor?
Because there's no actual substitute for writing code. No amount of upfront planning or experience can substitute actual code writing. This is what an entire generation (called waterfall) learned the hard way.
Once you start writing the code and be in the middle of it, you reason about the way it works on a lower level you do notice things (performance, usability or correctness things) that escaped the higher design view.
Refactoring is perfecting.
Ask yourself: why do painters do multiple strokes with the brush on the same spot?
Refactoring is the way to pay the technical debt.
I'd like to briefly address three of your points.
1. "A result of insufficient up-front design"
Common sense (and several books and bloggers) tell us we should strive for the simplest, cleanest design possible to address a given problem. While it's quite possible that some code is written without sufficient work on developing an understanding of the requirements and the problem domain, it's probably more common that "poor code" wasn't "poor" when it was written; rather, it is no longer sufficient.
Requirements change, and designs have to support additional features and capabilities. It's not unreasonable to anticipate some future changes up-front, but McConnell et al. rightly caution against high-level, overly-flexible designs when there's no clear and present need for such an approach.
3. "A dangerous activity that needlessly risks destabilising working code"
Well, yes, if done improperly. Before you seek to make any significant modification to a working system, you should put in place proper measures to ensure that you're not causing any harm - a sort of "developmental Hippocratic oath", almost.
Typically, this will be done by a mixture of documentation and testing, and more often than not, the code wins out, because it's the most up-to-date description of the actual behaviour. In practical terms, this translates into having decent coverage with a unit test suite, so that if refactoring does introduce unexpected problems, these are identified and resolved.
Obviously, when you seek to refactor, you're going to break a certain number of tests, not least because you're trying to fix some broken code contracts. It is, however, perfectly possible to refactor with impunity, provided you have that mechanism in place to spot the accidental mistakes.
4. "A waste of resources"
Others have mentioned the concept of technical debt, which is, briefly, the idea that over time, the complexity of such systems builds up, and that some of that build-up has to be reduced, by refactoring and other techniques, in order to reasonably facilitate future development. In other words, sometimes you have to bite the bullet and go ahead with that change you've been putting off, because otherwise you'll be making a bad situation appallingly worse when you come to add something new in that area.
Obviously, there's a time and a place to pay off such things; you wouldn't try and repay a loan until you had the cash to do it, and you can't afford to go around refactoring willy nilly during a critical stage in development. Nevertheless, by making the decision to address some of the problems in your code base, you save future development time, and thus money, and maybe even further into the future, avoid the cost of having to abandon or completely rewrite some component that is beyond your understanding.
In order to keep a maintainable code base?
Code is more read than written, so it is necessary to have a code-base that is readable, understandable and maintainable. When you see something that is poorly written or designed, it can be refactored to improve the design of the code.
You clean your house also regularly, don't you? Although it may be considered a waste of time, it is necessary in order to keep your house clean, so that you have a nice environment to live in.
You may need to refactor if your code is
Inefficient
Buggy
Hard to extend
Hard to maintain
It all boils down to the original code not being very good, so you improve it.
If you have reasonable unit tests it shouldn't be dangerous at all.
Because hindsight is easier than foresight.
Software is one of the most complex things created by humans, so it is not easy to consider everything beforehand. For large projects it can even be impossible for the team (at least for one consisting of humans ;) ) to consider everything before they actually start developing it.
Another reason is that software isn't constructed, it's growing. That means software can and has to adapt to ever changing requirements and environments.
As Martin Fowler says, the only thing surprising about the requirements for software changing is that anyone is surprised by it.
The requirements will change, new features will be requested. This is a good thing. Enhancement efforts succeed most of the time, and when they fail, they fail small, so there is budget to do more. Big up front design projects fail often (one statistics puts the failure rate at 66%), so avoid them. The way to avoid them is to design enough for the first version, and as enhancements are added, refactor to the point where it looks like the system intended to do that in the first place. The lifespan of a project that can do this (there are issues when you publish data formats or APIs - once you go live you can't always be pristine anymore) is indefinite.
In response to the four points, I would say that a process that shuns refactoring demands:
A static world where nothing changes
so that the upfront design can hit a
non-moving target perfectly.
Will
result in ugly hacks to work around
design flaws that aren't being
refactored.
Will lead to dangerous
code duplication as the fear of
changing existing code sets in.
Will
waste resources over engineering the
problem and building large design
artifacts in anticipation of
requirements that never end up
getting built, causing large amounts
of code and complication to drag the
project down while not providing any
value.
One caveat, though. If you don't have the proper support, in an automated tool for simple cases, and thorough unit tests in the more complicated cases, it will hurt, there will be new bugs introduced, and you will develop a (quite rational) fear of doing it more. Refactoring is a great tool, but it requires safety equipment.
Another scenario where you need refactoring is TDD. The textbook approach for TDD is to write only the code you need to pass the test and then refactor it to something nicer afterwards.
...because coding is like gardening. Your codebase grows and you domain changes as time passes. What was a good idea back then often looks like a poor design now and what is a good design now may well not be optimal in the future.
Code should never be considered a permanent artifact nor should it be considered too sacred to touch. Confidence should be garnered through testing and refactoring is a mechanism to facilitate change.
While a lot of other people have already said perfectly valid reasons, here's mine:
Because it's fun. It's like beating your own time in steeplechase, having the stronger bicep in armwrestling or improving your highscore in a game of your choice.
A straightforward answer is, requirements change. No matter how elegant your design is, some requirements later on will not buy it.
Poor understanding of the requirements:
If developers don't have a clear understanding of the requirements, the resulting design and code cannot satisfy the customer. Later as the requirements become more clear, refactor becomes essential.
Supporting new requirements.
If a component is old, in most of the cases it will not be able handle the radical new requirements. It then becomes essential to go for refactoring.
Lots of bugs in the existing code.
If you have spent long hours in office fixing quite a few nasty bugs in a particular component, it becomes a natural choice for refactoring at the earliest.
Upfront: Refactoring does not need to be dangerous when a) supported by tools and b) you have a testsuite that you can run after the refactoring in order to check the functioning of your software.
One of the main reasons for refactoring is that at some point you find out that code is used by more than one code path and you don't want to duplicate (copy&paste) but reuse. This is especially important in cases where you find an error in that code. If you have refactored the duplicated code into an own method, you can fix that method and be done. If you copy&paste code around, there is a high chance that you don't fix all places where this code occurs (just think of projects with several members and thousands of lines of code).
You should of course not do refactoring just because of the sake of refactoring - then it is really a waste of resources.
For whatever reason, when I create or find a function that scrolls off the screen, I know it's time to sit back and consider whether it should be refactored or not - if I'm having to scroll the whole page to take in the function as a whole, chances are it's not a shining example of readability or maintainability.
To make insane stuff sane.
I mainly refactor when the code has suffered so much under copy + paste and a lack of architectural guideance that the action of understanding the code is akin to re-organising it and removing the duplication.
It is human to err, and you're ALWAYS going to make mistakes when you develop software. Creating a good design from the beginning helps a lot, and having skilled programmers on the team is also a good thing, but they will invariably make mistakes, and there will be code that is hard to read, tightly coupled or non-functional, etc. Refactoring is a tool to mend these flaws when they've already occurred. You should never stop working on preventing these things from happening to begin with, but when they do happen, you can fix them.
Refactoring to me is like cleaning my desk; it creates a better working environment because over time it will get messy.
I refactor because, without refactoring, it becomes harder and harder to add new features to a codebase over time. If I have features A, B, and C to add, feature C will be finished sooner, with less pain and suffering on my part, if I take time to refactor after features A and B. I'm happier, my boss is happier, and our customers are happier.
I think it's worth restating, in any conversation involving refactoring, that refactoring is verifiably behavior-preserving. If at the end of your "refactoring" your program has different outputs, or if you only think, but can't prove, that it has the same outputs, then what you've done isn't refactoring. (That doesn't mean it's worthless or not worth doing -- maybe it's an improvement. But it's not refactoring and shouldn't be confused with it.)
Refactoring is a central component in any agile software development methods.
Unless you fully understand all the requirements and technical limitations of your project you can't have a complete upfront design. In this case instead of using a traditional waterfall approach you're probably better off with an agile method - agile methods focus on adapting quickly to changing realities. And how would you adapt your source code without refactoring?
I've found code design and implementation, particularly with unfamiliar and large projects to be a learning process.
The scope and requirements of a project change over time, which has consequences on the design. It may be that after spending some time implementing your product you discover that your planned design is not optimal. Perhaps new requirements were added by the client. Or perhaps you're adding additional functionality to an older product and you need to refactor the code in order to sufficiently provide this functionality.
In my experience code has been written poorly and the refactoring has become necessary to prevent the product from failing and to ensure it is maintainable/extendable.
I believe an iterative design process, with prototyping early on is a good way to minimise refactoring later on. This also allows you to experiment with differing implementations to determine which is most suitable.
Not only that, but new ideas and methods for what you're doing may become available. Why stick with old, fallible code that could become problematic if it can be improved?
In short, projects will change overtime, which necessitates changes in structure to ensure it meets new requirements.
From my own personal experience I refactor because I find if I make software the way I want it made from first go that it takes a very long time to create something.
Therefore I value the pragmatism of developing software over clean code. Once I have something running I then begin to refactor it into the way it should be. Needless to say, the code never devolves into a piece of unreadable tripe.
Just a side note - I did my degree in software engineering after reading some material from Steve Mcconnell as a teen. I love design patterns, good code reuse, nicely thought out designs and so on. But I find when working on my own projects that designing things initially from that point of view just doesnt work unless I'm an absolute expert with the technology I'm using (Which is never the case)
Refactoring is done to help make code easier to understand/document.
To give a method a better name - perhaps the previous wasnt clear or incorrect.
To give variables more descriptive / better names.
Break up a really long method into many smaller methods representing the steps involved in solving the problem.
Move classes to a new package(namespace) to assist organisation.
Reduce duplicate code.
Does point number one even matter? If you're refactoring, the up-front design was obviously flawed. Don't waste time worrying about the flaws in the original design; it's old news. What matters is what you have now, so spend that time refactoring.
I refactor because proper refactoring makes maintenance SO much easier. I've had to maintain a TON of bad, awful code and I don't want to hand down any that I've written for someone else to maintain.
Maintenance costs of smelly code will almost always be higher than maintenance costs for sweet smelling code.
I refactor because:
Often my code is far from optimal first time around.
Hindsight is often 20-20.
My code will be easier to maintain for the next guy.
I have professional pride in the work I leave behind.
I believe time spent now can save a lot more time (and money) further down the track.
All your points are common descriptors of why people do refactor. I would say that the reason people should refactor lies within point #1: A Big Design Up Front (BDUF) is almost always imperfect. You learn about the system as you build it. In trying to anticipate what could happen you often end up building complex solutions to deal with things that never actually happen. (YAGNI - You ain't gonna need it).
Instead of the BDUF approach, a better solution is therefore to design the parts of the system you know you are going to need. Follow the principles of single responsibility principle, use inversion of control/dependency injection so that you can replace parts of your system when needed.
Write tests for your components. And then, when the requirements for your system change or you discover flaws in your initial design, you can refactor and extend your code. Since you have your unit tests and integration tests in place, you will know if and when the refactoring breaks something.
There is a difference between large refactorings (restructuring modules, class hierarchies, interfaces) and "unit" refactorings - within methods and classes.
Whenever I touch a piece of code I do a unit refactoring - renaming variables, extracting methods; because actually seeing the code in front of me gives me more information to make it better. Sometimes refactoring also helps me to better understand what the code is doing. It's like writing or painting, you extract a fuzzy idea out of your head; put a rough skeleton onto paper; then into code. You then refine the rough idea in the code.
With modern refactoring tools like ReSharper in C#, this kind of unit refactoring is extremely easy, quick & low risk.
Large refactorings are harder, break more things, and require communication with your team members. It will become clear to everyone when these need to happen - because requirements have changed so much that the original design no longer works - and then they should be planned like a new feature.
My last rule - only refactor code that you are actually working on. If code's functionality doesn't need to be changed, then it's good enough & doesn't need further work.
Avoid refactoring just for refactoring's sake; that's just refactorbating!
We are developing software with pattern recognition in video. We have 7 mathematicians who are creating algorithms. Plus we have 2 developers that maintain / develop the application with these algorithms. The problem is that mathematicians are using different development tools to create algorithm like Matlab, C, C++. Also because they are not developers the don't give much concerns for memory management or multi-threading. This one of the reason why the app. has a lot of bugs.
If in your company you have similar situation, how do you deal with it? What's the best tools you can recommend to create algorithms? What communication supposed to be between mathematicians and developers? What's in your opinion the most effective to work with high-level tools?
I am not sure whether you devs are rewriting the mathematician's stuff or if you just have to interface to it so I am not sure if my answer is of any use.
However: I work together with a bunch of phd candidates and postdocs on a machine learning library and am a student myself. In that process, I came to translate a lot of algorithms from python/numpy to C++/blas.
This process can be quite tedious - especially with numerical and stochastic algorithms, it is hard to find bugs.
So here is what I did: Get some sample inputs and calculate the results with the python code. Generate unit tests out of these for C++ and then start coding them in C++.
Checking concrete sample inputs with the outputs is essential in this setting.
I agree with Makach.
Let the guys who are creating the algorithms use the tools that they are most familiar with. Because there are two separate (and equally important) tasks to work on in this project. First, there is the creating of an efficient, elegant and appropriate mathematically sound algorithm, then there is the twistedly difficult task of translating it into CPU-speak. The mathimaticians should focus on their first task, and to make it easier for them, allow them to use the toosl they are comfortable with. In terms of man hours, it is a much more efficient use of their time to write MATLAB code, than it would be to have them learn a new programming language.
Your task is to unearth the (presumably) brilliant mathematics that are buried within the gibberish code.
That part is just a perspective on the problem at hand. Here's the actual answer.
Communication, mutual respect, and teaching/learning.
Communication & Mutual Respect
You must communicate with them often. Work closely with them and ask them questions whenever you come across something you're not sure of. This is much easier when there is mutual respect, which means that if you spend all your time criticizing their coding abilities, then they will be forced to spend all their time criticizing your math abilities. Instead, try quick learning-sessions. ("Lunch & Learn" is a fairly common tactic)
Teaching/Learning
The first and most important piece of wisdom to impart to them is commenting. Have them comment the crap out of their code. Tell them that the comments are much more important than the code quality, and that as long as their comments are right, they can leave the rest up to you guys. Because they can. They don't need to have their code look beautiful, for be the fastest, they just need it to make sense to you guys.
To continue this mutual learning scenario, if you notice some very simple very common mistakes they are making, (nothing NEARLY as complicated as multithreading) just give them a quick heads up. "That way works (or doesn't) but here's a way to do it that is a little different but it will make your lives much much easier." Encourage them to reciprocate by trying to notice which nuances or parts of their algorithms which you and your team are having difficulty with and teach a little tutorial about it.
Once you guys get the communication flowing, you'll find it easier and easier to shape their coding style to what is best for your team, and they will also find it easier to understand why you don't see it the same way they do.
Also, as mentioned by Kekoav, make sure they provide a few fully loaded test cases.
That means for something like
A -> B -> C -> D -> Solution
They would provide you all the values for A, then what it looks like at B, then what it looks like at C and so on. So that you can be certain that not only is it correct at the end, but it's also correct at every step of the way. Try to have them provide examples that are regular, and also a few of them that are unusual, so that you can be certain your code covers edge cases.
I'd recommend the devs spend a few hours getting used to Matlab, especially the Matlab debugger. If their background is CS then they'll already be familiar with vectors and matrices theoretically if not practically. Other than the matrix being the default data structure, Matlab is C-like and easy enough to interpret for translation into another language.
I have been working with a physics professor lately, and have a little experience with this(although admittedly I'm no expert).
I have had to translate a lot of Matlab code into another language. It has been difficult because a lot of(most) of the operations are absent, including when it comes to precision, and working with matrices and vectors. A good math library needs to be found, or created to fit your needs.
The best way that I have found is to do the following:
Get the algorithm to work correctly in the new language.
Create some tests to verify that the algorithm is producing desired output. Have your mathematicians verify that your converted solution in fact works, and that you have covered all bases with your tests.
Then after it is working, and you can trust your tests, optimize the algorithm to be good coding style, have good design and performance characteristics. Use your regression tests to make sure you aren't breaking anything.
I normally start with a verbatim copy of their algorithms into the other language, and then work from there, regardless of if I do a lot of tests.
It is important to get a working copy first, in case the performance is really not an issue and you need to move on to other things and can come back later to make it faster.
This is your job. How you deal with this is what identifies you as a system developer.
Communicate with your colleagues. Draw and explain, have meetings, agree upon and set standards requirements, follow your plans and talk to your project manager. Make sure that your relevant colleagues are joining up on meetings. Have 1-1 talks etc etc
You cannot blame it on the mathematicians for developers creating bugs. It's their job to worry about implementation, not the mathematicians.
I'm looking for resources that provide an actual lesson plan or path to encourage and reinforce programming practices such as TDD and mocking. There are plenty of resources that show examples, but I'm looking for something that actually provides a progression that allows the concepts to be learned instead of forcing emulation.
My primary goal is speeding up the process for someone to understand the concepts behind TDD and actually be effective at implementing them. Are there any free resources like this?
It's a difficult thing to encourage because it can be perceived (quite fairly) as a sea-change; not so much a progression to a goal but an entirely different approach to things.
The short-list of advice is:
You need to be the leader, you need to become proficient before you can convince others to, you need to be able to show others the path and settle their uncertainties.
First become proficient in writing unit tests yourself
Practice writing tests for existing methods. You'll probably beat your head on the desk trying to test lots of your code--it's not because testing is hard or you can't understand testing; it's more likely because your existing code and coding style isn't very testable.
If you have a hard time getting started then find the simplest methods you can and use them as a starting point.
Then focus on improving the testability of the code you produce
The single biggest tip: make things smaller and more to the point. This one is the big change--this is the hardest part to get yourself to do, and even harder to convince others of.
Personally I had my "moment of clarity" while reading Bob Martin's "Clean Code" book; an early chapter talks about what a clean method will look like and as an example he takes a ~40 line method that visually resembled something I'd produce and refactors it out into a class which is barely larger line-count wise but consists of nothing but bite-sized methods that are perhaps 3-7 lines each.
Looking at these itty-bitty methods it suddenly clicked that the unit-testing cornerstone "each test only tests one thing" is easiest to achieve when your methods only do one thing (and do that one thing without having 30 internal mechanisms at play).
The good thing is that you can begin to apply your findings immediately; practice writing small methods and small classes and testing along the way. You'll probably start out slow, and hit a few snags fairly quickly, but the first couple months will help get you pointed in the right direction.
You could try attending (or hosting one if there is none near you!) a coding dojo
I attended one such excercise and it was fun learning TDD.
Books are always a good resource - even though not free - they may be worth your time searching for the good free resources - for the money those books cost.
"Test driven development by example" by Kent Beck.
"Test Driven Development in Microsoft .NET" by James W. Newkirk and Alexei A. Vorontsov
please feel free to add to this list
One thing I worked through that helped me appreciate TDD more was NHibernate and the Unit of Work Pattern. Although it's specific to NHibernate and .NET, I liked the way that it was arranged. Using TDD, you develop something (a UnitofWork) that's actually useful rather than some simple "this is what a mock looks like" example.
How I learn a concept best is by putting it to use towards an actual need. I suggest you take a look at the structure of the article and see if it's along the lines of what you're looking for.
Geeks are excellent at working to metrics, whether they are good for them or not!
You can use this to your advantage. Set up a CI server and fail the build whenever code coverages drops below 50 percent. Let them know that the threshold will rise 10 percent every month until it's 90. You could perhaps use some commit hooks to stop them being able to check code in to begin with but I've never tried this myself.
Let them know the coverage by the team will be taken into effect in any performance reviews, etc. By emphasising it is the coverage of the team, you should get peer pressure helping you ensure good coverage.
This will only ensure they are testing their code, not how well they are testing their code, nor whether they are writing the tests first. However, it is strongly encouraging (or forcing) them to incorporate testing into their daily development process.
Generally, once people have something in their process they'll want to do something as easily/ efficiently as possible. TDD is the easiest way to write code with high coverage as you don't write a line of code without it being covered.
Find someone with experience and talk to them. If there isn't a local developer group, then start one.
You should also try pushing things too far to start with, and then learn when to back off. For example, the whole mock thing started when someone asked "What if we program with no getters".
Finally, learn to "listen to the tests". When the tests look dreadful, consider whether it's the code that's at fault, not your testing technique.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Many times we find ourselves working on a problem, only to figure out the solution being created is far more complex than the problem requires. Are there controls, best practices, techniques, etc that help you control over complication in your workplace?
Getting someone new to look at it.
In my experience, designing for an overly general case tends to breed too much complexity.
Engineering culture encourages designs that make fewer assumptions about the environment; this is usually a good thing, but some people take it too far. For example, it might be nice if your car design doesn't assume a specific gravitational pull, nobody is actually going to drive your car on the moon, and if they did, it wouldn't work, because there is no oxygen to make the fuel burn.
The difficult part is that the guy who is developed the "works-on-any-planet" design is often regarded as clever, so you may have to work harder to argue that his design is too clever.
Understanding trade-offs, so you can make the decision between good assumptions and bad assumptions, will go a long way into avoiding a needlessly complicated design.
If its too hard to test, your design is too complicated. That's the first metric I use.
Here are some ideas to get design more simpler:
read some programming books and articles, and then apply them in your work and write code
read lots of code (good and bad) written by other people (like Open Source projects) and learn to see what works and what does not
build safety nets (unit tests) to enable experimentations with your code
use version control to enable rollback, if those experimentations take wrong turn
TDD (test driven development) and BDD (behaviour driven development)
change your attitude, ask how you can make it so, that "it simply works" (convention over configuration could help there; or ask how Apple would do it)
practice (like jazz players -- jam with code, try Code Kata)
write same code multiple times, with different languages and after some time has passed
learn new languages with new concepts (if you use static language, learn dynamic one; if you use procedural language, learn functional one; ...) [one language per year is about right]
ask someone to review you code and actively ask how you can make your code simpler and more elegant (and then make it)
get years under your belt by doing above things (time helps active mind)
I create a design etc., and then I look at it and try and remove (agressively) everything that doesn't seem to be needed. If it turns out I need it later when I am polishing the design I add it back in. I do this over several iterations, refining as I go along.
Read "Working Effectively With Legacy Code" by Michael C. Feathers.
The point is, if you have code that works, and you need to change the design, nothing works better than making your code unit testable, and breaking your code into smaller pieces.
Using Test Driven Development and following Robert C. Martin's Three Rules of TDD:
You are not allowed to write any production code unless it is to make a failing unit test pass.
You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
In this way you are not likely to get much code that you don't need. You will always be focused on making one important thing work and won't ever get too far ahead of yourself in terms of complexity.
Test first may help here, but it is not suitable for all situation. And it's not a panacea anyway.
Start small is another great idea. Do you really need to stuff all 10 design patterns into this thing? Try first to do it "stupid way". Doesn't quite cut it? Okay, do it "slightly less stupid way". Etc.
Get it reviewed. As someone else wrote, two pairs of eyes are better. Even better are two brains. Your mate may just see a room for simplification, or a problematic area you thought was fine just because you spend many hours hacking it.
Use lean language. Languages such as Java, or sometimes C++ sometimes seem to encourage nasty, convoluted solutions. Simple things tend to span over multiple lines of code, and you just need to use 3 external libraries and a big framework to manage it all. Consider using Python, Ruby, etc. - if not for your project, then for some private use. It can change your mindset to favor simplicity, and to be assured that simplicity is possible.
Reduce the amount of data you're working with by serialising the task into a series of smaller tasks. Most people can only hold half a dozen (plus or minus) conditions in their head while coding, so make that the unit of implementation. Design for all the tasks you need to accomplish, but then ruthlessly hack the design so that you never have to play with more than half a dozen paths though the module.
This follows from Bendazo's post - simplify until it becomes easy.
It is inevitable once you have been a programmer that this will happen. If you seriously have unestimated the effort or hit a problem where your solution just doesn't work then stop coding and get talking to your project manager. I always like to take the solutions with me to the meeting, problem is A, you can do x which will take 3 days or we can try y which will take 6 days. Don't make the choice yourself.
Talk to other programmers every step of the way. The more eyes there are on the design, the more likely an overcomplicated aspect is revealed early, before it becomes too ossified in the codebase.
Constantly ask yourself how you will use whatever you are currently working on. If the answer is that you're not sure, stop to rethink what you're doing.
I've found it useful to jot down thoughts about how to potentially simplify something I'm currently working on. That way, once I actually have it working, it's easier to go back and refactor or redo as necessary instead of messing with something that's not even functional yet.
This is a delicate balancing act: on the one hand you don't want something that takes too long to design and implement, on the other hand you don't want a hack that isn't complicated enough to deal with next week's problem, or even worse requires rewriting to adapt.
A couple of techniques I find helpful:
If something seems more complex than you would like then never sit down to implement it as soon as you have finished thinking about it. Find something else to do for the rest of the day. Numerous times I end up thinking of a different solution to an early part of the problem that removes a lot of the complexity later on.
In a similar vein have someone else you can bounce ideas off. Make sure you can explain to them why the complexity is justified!
If you are adding complexity because you think it will be justified in the future then try to establish when in the future you will use it. If you can't (realistically) imagine needing the complexity for a year or three then it probably isn't justifiable to pay for it now.
I ask my customers why they need some feature. I try and get to the bottom of their request and identify the problem they are experiencing. This often lends itself to a simpler solution than I (or they) would think of.
Of course, if you know your clients' work habits and what problems they have to tackle, you can understand their problems much better from the get-go. And if you "know them" know them, then you understand their speech better. So, develop a close working relationship with your users. It's step zero of engineering.
Take time to name the concepts of the system well, and find names that are related, this makes the system more familiar. Don't be hesitant to rename concepts, the better the connection to the world you know, the better your brain can work with it.
Ask for opinions from people who get their kicks from clean, simple solutions.
Only implement concepts needed by the current project (a desire for future proofing or generic systems make your design bloated).