I am a one-man shop at the place where I work, and when I started there I had zero experience and a BS degree from a below-par school of Computer Science. On top of that, my first project at the company involved not just figuring out good design principles, it also involved learning a new language. Needless to say, my code was crappy in the beginning, and all the new features I've added since then have been hacked on top of all that crappy code. It's amazing to me that my software works as well as it does.
I have learned a TON during my employment, and I am dying to refactor my code to make it more readable so future new hires can dive in and help me with it. I also REALLY want to make it easier to add new features without having to hack stuff together. I think it would be useful to learn a framework like Prism for WPF/Silverlight, but I have a huge to-do list (since I am a one-man shop), and it looks like it will take a pretty decent amount of time just to learn how to use it.
Now I have read up a little bit on Prism to where I know the basic principles behind it. Furthermore, it wouldn't be hard to write my own code that accomplishes some of the same things that Prism is used for. I've actually done that already to some degree and I'm making good progress on making things more modular.
My question is this: should I go on writing more infrastructure code that gives me exactly what I need and no more, or should I take the time to learn something like Prism? Or maybe it could be asked like this: Should I spend time writing my own simple custom solutions, or should I spend time trying to grasp a rich, vast framework that may possibly be more complex than is necessary? And what factors should I take into account when making the decision?
I wrote my own PHP MVC framework for a recent project with exactly what I needed. It was loads of fun, taught me a lot, and an overall good experience, and I will never, ever do it again. While an excellent secondary distraction project, it highly detracted from my productivity on the main project.
Really, a lot of it depends on just how much infrastructure you will have to develop. If it's just a tiny bit that won't take more than an hour or two, go for it. If it will take significant amounts of time, use someone else's work, move on, and get your project done.
With the background you give - mostly self-educated and no peers to discuss your current development - you should absolutely check out other libraries and tools. At the very least, get new input how code can be designed and problems can be solved. You may feel that you have achieved something - and you have, congratulations - but that's a plateau, not the peak.
"I have no time to learn something new because I have so much to do"
- that's what I read in your rationale for more code.
This is a warning sign - you are moving yourself into a dangerous position. No time to learn? No time to document? No time to think of all the implications? No time to do it right? No time to train a new employee? No time to call it a day?
You won't solve this problem by learning prism, or any other library, but it's the wrong rationale.
Third, code bogs you down. Having more code to maintain makes you slower. One-man-startups can crank out hundreds, even thousands of LOC per day for days and weeks. As projects and organizations get larger, you end up with an average of a few dozen.
As a recommendation from personal experience: write bulding blocks, not frameworks. Frameworks are great when you have to make the same application with different company logos over and over. Or, as TDWTF's Alex says, the key is in the differences not the similarities.
I don't want you to stop writing code, far from it. But you are discussing a tradeoff, and from the information you've given, I would recommend to put most weight on learning new things.
If the app your writing is going to be around for a while and have to be maintained, particularly by other developers, then any time spent to learn and integrate a standard framework will be worthwhile.
It'll provide documentation for how the app is written and any developer familiar with that framework will be able to pick it up faster. It should reduce the amount of code you have to write and help you concentrate on your specific business problem and not the plumbing of writing an application.
The core issue is, how many times will you re-use the framework, saving you each time the work of re-implementing similar stuff instead? Remember, the stuff you write from scratch, if it's supposed to be any good at all, will have to be tested, validated against different environments (clients &c), and maintained -- all stuff that would come to you "for free" by using a good, actively maintained framework.
If you're going to use that framework only a couple of times, maybe the net returns are still in favor of rewriting from scratch -- but if the framework covers a field that you need in more than just a couple cases, the returns on the investment of learning to use the framework (assuming it's any good!-) vs redoing things from scratch are going to be vastly positive!
I was in a similar situation when I graduated college. I received an offer from a large company about 1.5 years into my stay at the small company. What I learned was this (may be different for you and others):
It was an awesome idea to work at a small company right out of school. I say this because you have to wear many different hats. For example you would write the code, test the code, deploy the code, write stored procs, etc. The end result is that you are familiar with the entire process from conception to whatever. That experience is critical I think.
I loved writing code. I remember the days that I would be driving home and thinking of my day spent dealing with production support issues. I was spending more time supporting customers and writing "one offs" that I wasn't mostly writing code.
Working for a big company is a bad idea right of college. When you're working for a big company they have a specific role for you, and you have specific boundaries. If you're a developer at a big company odds are that you are not deploying the application(s) to production, or tuning stored procs.
Working for a big company is a great idea after working for a small one. That's because if you work at the small company it will force you to learn about a lot more than just coding. And if you understand that you will be a better developer.
Working with good developers makes you better. When you work with a group of guys that are good you will get better. This is because each dev has a specific history that they bring to the group, and you all learn from one another. On the group that I work with mainly right now there is: an MSBuild expert, a Silverlight expert and an F# expert, and other good guys. So some of the guys learn MSBuild from me, and I learn from them. Just talking to guys who are good can make you better.
So if I was you, don't spend too much time there. Maybe 1 or 2 years, after that find a job somewhere that has some talented developers. You will be a much better dev in 5 years. I know that I am because of my move.
I'll play contrarian: YAGNI (You Aren't Going to Need It).
What if the framework
Is badly designed?
Is buggy?
Is too slow?
Will be different in two years, and the old version won't be supported?
Discussions of frameworks often assume that frameworks are great, where the reality is that frameworks vary just like anything else, and many frameworks are larded with stuff you are never going to need.
Here's some advice that I hope bears on your more specific questions:
Continue to make incremental improvements. It sounds like you are being productive with this approach and that it is paying off for you.
Learn more about the framework. Or multiple frameworks. Maybe you can try a small pilot project in, say, 1 to 4 days.
It is an honorable strategy to learn about a framework not in order to use the framework, but in order to cherry-pick the best ideas and adopt them into your own designs.
If you decide for the time being not to adopt a framework, this is an easy decision to revisit later. If you decide to adopt a framework, backing away from it later can be very expensive. It may be worth paying some extra costs up front to reduce the probability of making a very expensive mistake.
I think where I have been burned the most is by depending on somebody else's code base that changed out from under me. I call this the "every Perl script I ever wrote was broken a year later" problem. But I work on a lot of different small projects that tend to get bursty attention and have a very long lifetime compared to the time it took to create one. If you have one big project that you're working with daily for years, you can adapt more easily to changes in external frameworks.
Who has that tool?
This is what I ask to myself everytime I need to solve every problem. This is the main factor to evaluate the effort necessary for develop the tool.
When starting a large project, everyone define well known (at project scope) usefull statements; thinking to enlarge that abstraction layer depends on the frequency of the problem, the importance of the the problem solution, the effort to develop the solution.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
The question might seem trivial, but it's an actual problem: when you're working on a project, do you do any kind of architecture design before actually starting coding? Do you spend much time working together with a customer to get a detailed specs/usecases/mockups?
During coding, do you alter those architectural decisions made before? Do you go back to the customer with new set of specs/usecases/mockups?
I'm wondering, what's a good balance between all those non-coding actions and coding itself, from your experience?
Update:
Ok, so from the anwsers so far it seems like there are 2 approaches:
design early, then sit and code to avoid late fixes
minimize the design alone part, instead do iterative development (agile methodologies seem to prefer it that way).
I guess which way to go depends on the project, team and customer... am I right?
That which minimises the total time spent ;-)
It heavily depends on the kind of project, but generally speaking it's better to "waste" time over-designing and specifying requisites than finding out later that something was wrong and come the whole way back to fix it.
I read something about quantitative measurements of the impact of poor design decisions in "The Mythical Man-Month" or maybe in a book called something like "Software Requirements Pro Practices" from Microsoft Press, I think the time wasted in a late fix (near product delivery) was about 10x than in early stages.
If you do agile, design and coding are the same thing. In my experience it is good to pair program during the very first stage of the project...
Have a look at scrum, agile and waterfall. This is related to project management not programming per se.
Architecture also becomes easier once you have built enough applications within a domain or a platform. In PHP, if you use Joomla, Symfony or codeigniter then your scaffolding and architecture is already in place. Same for ASP.NET MVC.
My personal experience tells me that you should consider different factors. There's no silver bullet. My personal list follows, grown mostly by experience.
If you are developing something that is well known in details, the development team is sparse and with difficulty to communicate efficiently all together, the team has strong or huge dependencies towards the work of other teams, and what you are developing has a fundamental long term importance that will be difficult to change in the future (eg. file formats), go for a very long design phase, akin to a waterfall model. Also, you should spend a lot of design if you plan to develop a rather complex application, and you have to deeply consider all the possible interactions between features before coding. Coding takes very little time compared to design. Also, you should consider this if it somehow important to keep efficient record of how the application behaves from a very high level point of view, and if your team tends to be highly unstable, so that your knowledge stays on paper, rather than in people's brain.
if you have to implement something brand new and to do research on, you want feature as soon as possible, growing the application from fast feedback, you have a pool of geeks that work in the same room, are very committed to your cause, love programming and they are passionate to share and build together, go for agile methods.
if you are in between to the previous two cases, go for an iterative approach. I normally choose a 3 months schedule. When I code alone, I work agile-like, mostly because I have to cope with frequent disruption, so I add feature by feature. However, I release iterative, namely I don't plan to do an official, stable release before the third iteration. I want space to learn the field, do mistakes, and correct them before committing to maintain some stupid choice.
if you code in academia, you are screwed, because you have some of the issues in 1 without the manpower to accommodate them, and some of the issues in 2 without the easy communication required by agile methods.
roughly 50/50. whenever ive analysed my project schedules, it turns out about 50% of the time goes into design, project management, quality control, and auxiliry tasks. the remaining 50% is coding. if i dont see that 50/50 ratio, i worry.
mind you, this is using traditional waterfall model (which is more suited to custom-app development). agile methods are better for shrink-wrapped software in my opinion.
I would say it's roughly 50/50, no matter the "methodology" or project type. It only varies in how those 50% design are distributed. And that may depend on the project, but most of all it depends on the people who do the work, and how they are "wired". It's more a matter of psychology than methodology.
Some people (I'd say the more cautious characters) need a more detailed mental map before they start coding. If they don't have that map out of prior experience, they will need more "investigation" time up front.
Others yet like to just "jump in" into coding with only a rough mental map, and work out the details as they go.
Somewhere in between is to do the elaboration via spikes and prototypes, and develop the "big picture" on top of that running code. For me personally this tends to yield the best results, and the least waste. (After all, prototyping is, in a way, a test-first approach applied on solution ideas. You get an idea, test it out in a spike or prototype, then implement/integrate it with the main code base.)
My advice is: Find out the style that feels best to you personally, and stick to it. That's going to be pretty sure the style you are going to be most effective with.
Those two things are tightly coupled. Well at first stage, you are definitely will spend some time to make design decision. Then you will have to start coding and almost in all cases you will came up with some improvement decision for your previous design.
After all it will depend on delivery date and how much time you have at all and then to decide accordingly how you going to balance it. In general you make a startup design and then during coding you will update and change it. Also is a good practice to deeply involve your customer in design decision during development stage to force him be aware of it and how much of your time you will spend on each change.
The longer the period between when you write your specification and the time you start coding will increase the chance that requirements will change. So, to answer your question, as soon as possible....
If your suffering from too much requirement creep then I would suggest implementing smaller iterations of releases (if possible) and then creating new requirements/specifcation documents for each of these samller phases.
If you can't do this.... make sure you have a good change management process sin place.
My google-fu is failing drastically, but I recently read something to the effect of:
"Spend 6 months coding, 6 months designing and 6 months testing. The good news is, they're all the same 6 months."
It's important to design enough to have a map of what you are trying to code, and how it relates to the rest of the system. You can't just code most large projects - they're too big, and usually involve multiple components. I've done that when I was young, and you end up with a big ball of mud, or stay up all night for a week refactoring it.
What I tend to do now is design down to the package level, and assign roles to components. On large systems getting to the component selection stage can take several months, and involve some trail and prototyping coding.
Then the APIs and implementations of each package are evolved, based on what messages the functionality require, and how the clients of the packages evolve to cause the emergence of further requirements or constraints. I usually evolve an API by designing a pure interface (by writing the code for it) with unit tests for each known use case, then implement it. So there is some writing of code involved in designing - the best representation of the API is usually the code and inline documentation, and it's easiest to confirm that the client can perform the actions required to satisfy a use case ( and the code to do so is not excessively complex ) by writing code which exercises the API in that way, and that code trivially becomes a unit test for the implementation of the API when it arrives. But the code written during 'designing' isn't the code which supplies the implementation of the API. For APIs with low coupling ( so can be changed without breaking too many clients ), I'll switch between designing and implementing modes rapidly; for ones with higher coupling, I'll typically publish the API and use-case examples for peer review before committing too implementing them.
As aleemb said, this really is a project management question. I suggest you read up on several methodologies, find the useful and not-so-useful parts of each, and evaluate your own circumstances (team size/experience, customer engagement and commitment levels, what's done in your organization, schedule/budget, etc.) and come up with the best schedule you can. It really all just depends on your specific circumstances.
Think about how many people are going to be involved in writing the software.
If it's just a one-developer job, maybe take a smaller percentage for design. If you're going to have 30 people working on it, you probably want a lot bigger fraction for the design.
Getting teams of developers to write software is much like partitioning software up across multiple CPU's - you are going to get the best return for added CPU (read 'developer') when you can minimize the necessary communication between them. You sure don't want to get 10's of k-loc into your project before the developers start discussing architectural issues.
Now you could probably also make the case that, when you do a better job with the design phase, the coding will actually take less time and be less painful. Measure twice and cut once, and all that.
Also, you probably should think about the likelihood of the project being 'put on hold'; design artifacts have much better shelf life than immature code.
Depends on your chosen methodology.
Traditionally with Big Design Up Front or Waterfall you spend 90% of the time designing and 10 % of the time coding. You then spend another 90% of the time handling all the changes that the initial design missed. And another 90% of the time chasing bugs.
With modern Agile development you spend 10% of the time designing and 90% of the time coding. then another 90% handling all the changes that the customer representative forgot to mention and another 90% of the time chasing bugs.
I'm "just" a hobbyist programmer, but I find that as my programs get longer and longer the bugs get more annoying--and harder to track. Just when everything seems to be running smoothly, some new problem will appear, seemingly spontaneously. It may take me a long time to figure out what caused the problem. Other times I'll add a line of code, and it'll break something in another unit. This can get kind of frustrating if I thought everything was working well.
Is this common to everyone, or is it more of a newbie kind of thing? I hear about "unit testing," "design frameworks," and various other concepts that sound like they would decrease bugginess, make my apps "robust," and everything easy to understand at a glance :)
So, how big a deal are bugs to people with professional training?
Thanks -- Al C.
The problem of "make a fix, cause a problem elsewhere" is very well known, and is indeed one of the primary motivations behind unit testing.
The idea is that if you write exhaustive tests for each small part of your system independently, and run them on the entire system every time you make a change anywhere, you will see the problem immediately. The main benefit, however, is that in the process of building these tests you'll also be improving your code to have less dependencies.
The typical solution to these sort of problems is to reduce coupling; make different parts less dependent on one another. More experienced developers sometimes have habits or design skills to build systems in this manner. For example, we use interfaces and implementations rather than classes; we use model-view-controller for user interfaces, etc. In addition, we can use tools that help further reduce dependencies, like "Dependency injection" and aspect oriented programming.
All programmers make mistakes. Good and experienced programmers build their programs so that it is easier to find the mistakes and restrict their effects.
And it is a big deal for everyone. Most companies spend more time on maintenance than on writing new code.
Are you automating your tests? If you do not, you're signing up creating bugs without finding them.
Are you adding tests for bugs as you fix them? If you do not, you are signing up for creating the same bugs over and over.
Are you writing unit tests? If not, you are signing up for long debugging sessions when a test fails.
Are you writing your unit tests first? If not, your unit tests will be hard to write when your units are tightly coupled.
Are you refactoring mercilessly? If not, every edit will become more difficult and more likely to introduce bugs. (But make sure you have good tests, first.)
When you fix a bug, are you fixing the entire class? Don't just fix the bug; don't just fix similar bugs throughout your code; change the game so you can never create that kind of bug again.
Bugs are a big deal to everyone. I've always found that the more I program, the more I learn about programming in general. I cringe at the code I wrote a few years back!! I started out as a hobbyist and liked it so much that I went to engineering college to get a Computer Science Engineering major (I am in my final semester). These are the things that I have learned :
I take time to actually design what I am going to write and document the design. It really eliminates a lot of problems down the line. Whether the design is as simple as writing down a few points on what I am going to write or full blown UML modeling (:( ) doesn't matter. Its the clarity of thought and purpose and having material to look back at when I come back to the code after a while that matter the most.
No matter what language I write in, keeping my code simple and readable is important. I think that it is extremely important not to over complicate the code and at the same time not to over simplify it. (Hard learned lesson!!)
Efficiency optimizations and fancy tricks should be applied at the end, only when necessary and only if they are needed. Another thing is that I apply them only If I really know what I am doing and I always test my code!
Learning language dependant details helps me keep my code bug free. For instance I learned that scanf() is evil in C!
Others have already commented on the zen of writing tests. I would like to add that you should always do regression tests. (i.e. Write new code, test all parts of your code to see if it breaks)
Keeping a mental picture of code is hard at times, so I always document my code.
I use methods to make sure that there is a bare minimum dependence between different parts of my code. Interfaces, class hierarchies etc. (Decoupled design)
Thinking before I code and being disciplined in whatever I write is another crucial skill. I know people who don't format their code so its readable (Shudder!).
Reading other peoples source to learn best practices is good. Making my own list is better!. When working in a team, there must be a common set of them.
Don't be paralyzed by analysis. Write tests, then code, then execute and test. Rinse wash repeat!
Learning to read over my own code and combing it for mistakes is important. Improving my arsenal of debugging skills was a great investment. I keep them sharp by helping my classmates fix bugs regularly.
When there is a bug in my code, I assume its my mistake, not the computers and work from there. That is a state of mind that really helps me.
A fresh pair of eyes aids in debugging. Programmers tend to miss even the most obvious errors in their own code when exhausted. Having someone to show your code to is great.
having someone to throw ideas at and not be judged is important. I talk to my mom (who is not a programmer) , throw ideas at her and find solutions. She helps me bounce my ideas back and forth and refine them. If she is unavailable, I talk to my pet cat.
I am not so be discouraged by bugs anymore. I've learned to love removing bugs almost as much as programming.
Using version control has really helped me manage different ideas I get while coding. That helps reduce errors. I recommend using git or any other version control system you might like.
As Jay Bazzuzi said - Refactor code. I just added this point after reading his answer, to keep my list complete. All credit goes to him.
Try to write reusable code. Reuse code, both yours and from libraries. Using libraries which are bug free to do some common tasks really reduces bugs (sometimes).
I think the following quote says it best - "If debugging is the art of removing bugs, programming must be the art of putting them in."
No offense to anyone who disagrees. I hope this answer helps.
Note
As others Peter has pointed out, use Object Oriented Programming if you are writing a large amount of code. There is a limit to code length after which it becomes harder and harder to manage if written procedurally. I like procedural for smaller stuff, like playing with algorithms.
There are two ways to write error-free programs; only the third one works. ~Alan J. Perlis
The only way for errors to occur in a program is by being put there by the author. No other mechanisms are known. Programs can't acquire bugs by sitting around with other buggy programs. ~Harlan Mills
Obviously, bugs are a big deal to any programmer. Just look through the list of questions on Stack Overflow to see this illustrated.
The difference between a hobbyist and an experienced professional is that the pro will be able to use his experience to code in a more "defensive" way, avoiding many types of bugs in the first place.
All the other answers are great. I'll add two things.
Source control is mandatory. I'm assuming you're on windows here. VisualSVN Server is free and maybe 4 clicks to install. TortoiseSVN is also free and it integrates into Windows Explorer, getting around the VS Express limitations of no add-ins. If you create too many bugs, you can revert your code and start over. Without source control, this is next to impossible. Plus you can sync your code if you have a laptop and a desktop.
People are going to recommend many techniques like unit testing, mocking, Inversion of Control, Test Driven Development, etc. These are great practices, but don't try to cram it all into your head too quickly. You have to write code to get better at writing code, so work these techniques slowly into your code writing. You have to crawl before you walk and walk before you can run.
Best of luck in your coding adventures!
This is a common newbie thing. As you get more experience, of course, you'll still have bugs, but they'll be easier to find and fix because you'll learn how to make your code more modular (so that changing one thing doesn't have ripple effects everywhere else), how to test it, and how to structure it to fail fast, close to the source of the problem, rather than in some arbitrary place. One very basic but useful thing that doesn't require complex infrastructure to implement is to check the inputs to all functions that have non-trivial precondtions with asserts. This has saved me several times in cases where I would have otherwise gotten weird segfaults and arbitrary behavior that would have been near impossible to debug.
If bugs weren't a problem then I'd be able to write a 100,000 line program in 10 minutes!
Your question is like, "As an amateur doctor, I worry about my patients' health: sometimes when I'm not careful enough, they sicken. Is patients' health a problem for you professional doctors too?"
Yes: it's the central problem, even the only problem (for any sufficiently all-inclusive definition of 'bug').
Bugs are common to everyone -- professional or not.
The larger and more distributed the project, the more careful one must be. One look at any open source bug database (ex: https://bugzilla.mozilla.org/ ) will confirm this for you.
The software industry has evolved various programming styles and standards, which when used right, make wrong code easier to spot or limited in its impact.
Therefore, training has a very positive on code quality... But at the end of the day, bugs still sneak through.
If you're just a hobbyist programmer, learning full bore TDD and OOP may involve more time than you're willing to put in. So, going on the assumption that you don't want to put in the time on them, a few easily digestible suggestions to cut down on bugs are:
Keep each function doing one thing. Be suspect of a function more than, say, 10 lines long. If you think you can break it into two functions, you probably should. Something that will help you control this is naming your functions according to exactly what they are doing. If you find that your names are long and unwieldy then you function is probably doing too many things.
Turn magic strings into constants. That is, instead of using:
people["mom"]
use instead
var mom = "mom";
people[mom]
Design your functions to either do something (command) or get something (query), but not both.
An extremely short and digestible take on OOP is here http://www.holub.com/publications/notes_and_slides/Everything.You.Know.is.Wrong.pdf. If you get this, you've got the gist of OOP and are quite frankly ahead of a lot of professional programmers.
The prevailing wisdom seems to be that the average programmer creates 12 bugs per 1000 lines of code - depends on who you ask for the exact number, but it's always per lines of code - so, the bigger the program, the more the bugs.
Subpar programmers tend to create way more bugs.
Newbies are often trapped by idiosyncrasies of the language, and lacking experience tends towards more bugs too. As you go on, you will get better, but never will you create bug-free code... well I still have bugs, even after 30 years, but that could be just me.
Nasty bugs happen to everyone from pros to hobbyists. Really good programmers get asked to track down really nasty bugs. It's part of the job. You'll know you've made it as a software developer when you stare at a nasty bug for two days and in frustration you shout, "Who wrote this crap!?!?" ... only to realize it was you. :-)
Part of the skill of a software developer is the ability to keep a large set of interrelated items straight in his/her head. It sounds like you're discovering what happens when your mental model of the system breaks down. With practice you will learn to design software that doesn't feel so brittle. There are tons of books, blogs, etc. out there on the subject of software design. And Stack Overflow of course for specific questions.
All that said, here's a couple of things you can do:
A good debugger is invaluable. Often you have to step through your code line by line to figure out what went wrong.
Use a garbage-collected language such as Python or Java if it makes sense for your project. GC will help you focus on making things work instead of getting bogged down by maddening memory errors.
If you write C++, learn to love RAII.
Write LOTS of code. Software is somewhat of an art form. Lots of practice will make you better at it.
Welcome to Stack Overflow!
What really changed my odds against code complexity and bugs was using a coding standart - how to place brackets an so on. It may seem like just boring and useless thing but it really unifies all the code and makes it much easier to read and maintain. So do you use a coding standart?
If you're not well organized, your codebase will become your very own Zebra Puzzle. Adding more code is like adding more people/animals/houses to your puzzle, and soon you have 150 various animals, people, houses and cigarette brands in your puzzle and you realize that it just took you a week to add 3 lines of code because everything is so inter-related that it takes forever to make sure the code still executes how you want it to.
The most popular organizational paradigm seems to be Object Oriented Programming, if you can break your logic down into small units which can be constructed and used independently of each other, then you will find bugs far less painful when they occur.
The IT department I work in as a programmer revolves around a 30+ year old code base (Fortran and C). The code is in a poor condition partially as a result of 30+ years of ad-hoc poorly thought out changes but I also suspect a lot of it has to do with the capabilities of the programmers who made the changes (and who incidentally are still around).
The business that depends on the software operates 363 days a year and 20 hours a day. Unfortunately there are numerous outages. This is the first place I have worked where there are developers on call to apply operational code fixes to production systems. When I was first, there was actually a copy of the source code and development tools on the production servers so that on the fly changes could be applied; thankfully that practice has now been stopped.
I have hinted a couple of times to management that the costs of the downtime, having developers on call, extra operational staff, unsatisifed customers etc. are costing the business a lot more in the medium, and possibly even short term, than it would to launch a whole hearted effort to re-write/refactor/replace the whole thing (the code base is about 300k lines).
Ideally they'd be some external consultancy that could come in and run the rule over the quality of the code and the costs involved to keep it running vs rewrite/refactor/replace it. The question I have is how should a business go about doing that kind of cost analysis on software AND be able to have confidence in that analysis? The first IT consultants down the street may claim to be able to do the analysis but how could management be made to feel comfortable with it over what they are being told by internal staff?
We recently decided to completely rewrite large portions of our business code from scratch, and it has not gone as well as we had hoped. I've seen a lot of quotes saying you should never try to rewrite anything from scratch, and now I see why. I would recommend starting small - don't try to rewrite the whole thing at once. Identify the large problem areas and focus on refactoring small portions of the system at a time. Since there is 30+ years worth of work in the system, it will take a long time to get it back to a reasonable state. We had about 5-8 years worth of work to rewrite, and it has been difficult. I can't imagine 30+ years of work!
First, the profile of the consultant you need is very specific. Unless you can find someone who worked in a similar domain with the same languages, don't hire him.
Second, there's a 99% probability (I like dramatic numbers) the analysis will go as follow:
Consultant explores the application
Consultant does understand 10% of the application
Time's up, time for the report
Consultant advices a complete rewrite (no refactoring, plain rewrite)
So you may as well make the economy of what the consultant will cost.
You have only two solutions here:
Keep with the actual source code but determine proper methods to fix problems so that you have a very long run refactoring that is progressly made by those who know the application
Get a secondary team to make a new application to replace the old one
If I talk about a secondary team, it's because you cannot bring just one architect to make the new application and have the old team working with him:
They're too busy on the old application
There will be frictions because the newcomer will undoubtedly underestimate the task at hand
I talk from experience, believe me.
If you go the "new application" way don't put your hopes too high. You'll end up with an application that has less than half the functionalities of the current one, simply because you cannot cram 30+ years of special case and exceptional situation fixes into a freshly design software.
Oh, also, if your developers happen to tell you they have a plan, by all means, hear them out. They most probably know what they are talking about.
The first thing that comes to mind is that you are prematurely addressing the rewrite/refactor/replace argument. The first step two steps I would recommend would be:
Unit tests
QA
It's well within engineering scope to implement these. Unit tests are an essential preliminary step before any reasonable refactor or rewrite could possibly take place. By 'unit test' I mean wrap each function call with corresponding code that proves the code works for all known conditions. In complex retrofits this may not actually happen at the most granular level but any automated tests will help immensely.
And QA - have an independent (and aggressive) quality assurance team that rigorously tests beta releases before production. Their test plans and test procedures become essential for any kind of replacement effort.
Once you've got the code under control, then you are in a position where the business can reasonably consider massive changes.
Just a note about your comment about external consultants - no consultancy will ever care enough about the code to provide realistic quality assurance. QA ends up being married to the hip of business defending the company bottom line. It's an internal function ultimately and an external consultant can't provide much more than getting you started really.
I think that your description provides all of the necessary information on code quality (lack thereof). The fact that so many support resources are required also indicates the high costs involved with maintaining the existing system.
As I answered here, a good approach to consider is refactoring one piece of the system at a time until everything works at an acceptable level. I agree with Joel re not throwing away existing code (see Things You Should Never Do. Parts of your code work, so you should leave those in place whenever possible, and focus on the sections that lead to downtime.
Andy also makes a great point about starting small as well.
Another thing to try, is reviewing the processes around the system. When you do this, you should try to determine what failure situations are caused directly or indirectly by user action?, are there configuration or environment problems? If you are having trouble fixing the code directly, then you can still prop it up by dealing with external issues more effectively.
Read the book Working Effectively with Legacy Code (also see the short PDF version) and surround the code with automated tests, as instructed in that book.
Refactor the system little by little. If you rewrite some parts of the code, do it a small subsystem at a time. Don't try to make a Grand Redesign.
The code has been around for 30 years?
Development paradigms have shifted substantially in the last three decades in many ways, and most relevant to your predicament, I feel, is in terms of the amount of time (in man days) required to create something to input->process->output something.
300,000 lines of code 30 years ago, could probably fit into 100,000 lines or less today, and expending fewer man hours(?) This could seem optimistic/ridiculous to some, but on the other hand is achievable, depending on the type of application in question. You have given no indication as to the classification of system - is it a real-time manufacturing process control system of sorts with sensors and actuators tied to it? An airline booking system ? Does it post-process some backlog of data? In other words could it be rebuilt in something like Java and quickly with an agressive, smallish team? Have the requirements been documented, and if so do they need updating or redeveloping from scratch? Is human safety a factor?
Just a quick sanity check, I think whether or not you should rebuild depends on (any order means the same thing):
Number of code dudes required.
Level of expertise of said dudes.
Which languages do not fit.
Which languages do fit.
How much it costs to use chosen language(s) them in terms of hardware and software.
How much does the business depend on this to stay alive.
Is it really too much downtime, or are you just nitpicking? (maybe they really don't care, but pretend to).
Good luck with that!
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I am currently a grad student, but I was in the industry for a few years before going back to school.
I am in a class which involves teams of 4 working on fairly ambitious projects. As a result of having been in the industry, I have a lot of "software engineering" experience my fellow teammates lack (they are using SVN for the first time this semester). They are all very good programmers; but they don't have a lot of experience in building "real stuff".
Since I had a fairly concrete vision for a project, and my teammates did not, my idea is the one we will spend this semester working on. On top of that, as a result of my experience, plus the fact that I admittedly have a somewhat strong personality, I've become a de-facto team lead -- established weekly meeting times, assigned initial tasks, etc.
I want to avoid the trap of being so forceful with my ideas for what we should be doing and how we should be doing it, that my teammates feel like they have no say and become uninvolved and detached.
So here is the question:
How can I keep my team of undisciplined but talented programmers motivated while enforcing basic best practices (version control, milestones, etc) and a coherent project vision?
Edit: Thanks to everyone who answered so far. I think I've overemphasized the "software engineering" aspect of things; I'm also looking for ideas for how to encourage my teammates to contribute to the design, and feel ownership in the project which is at the moment a little bit "The SquareCog (and friends) Show!"
The best method I've found has nothing to do with code: team lunches.
Get together in an informal setting where you each talk about your problems, concerns, ideas, etc. This helps team unity in a way that very little else does.
As for the actual code side of it, minimize the amount of work they have to do to work inside the framework you want them to. If you want them to use tickets, do the actual management side of things for them -- have them tell you what the ticket is and have you do the actual legwork of managing these things. This seems like it'd take a long time, but overall it's minimal compared to the cost of poor communication and coordination. It pays off very quickly.
For version control, show them why it truly benefits them. Programmers pick up on ideas and run with them when they see they actually help them rather than just being a PITA.
I think developers are really practical people.
Play with those traits of typical developer personality:
1. Creativity
2. Curiosity
3. Practicality
Following your direct example, source control:
Most of us (I mean by my own experience) will fail to see the point in source control in the beginning (just because), so always keep them aware of the reason behind using source control.
Another thing is.. who decided to go on SVN? There are alternatives, I for one would fight to the teeth not to have SVN because I am a Git! (pun intended)
Instead of pulling them by the nose, you should/could have explained to them:
We need source control, find one you like and lets vote it out what we use to control the source.. this way there is a common ownership..and not just a follow the leader exercise.
Another thing is, be flexible in what you implement.
Draw out a plan on necessities, but try to be ready to implement them as the need arises, or as it becomes obvious to all that x, y or z practice should be implemented.
Have them need to implement the tools and resources and planning techniques you know by having them come to you for advice. (this doesn't mean you can't lay out a best practices blog internally or some other way of giving them access to this information beforehand)
Developers like to learn and grow, but we need ownership and understanding in the direction we are going.
If you try to force feed and drive them too much, both you and them will just lose motivation, enlightenment requires self driven forces.
How about the Scrum (even if you don't call it that). Gives everyone a chance to have their say, and you listen. As the forceful personality giving the others a real chance to communicate what's on their mind (not yours) is a good step towards harmony.
On top of that they will learn from your tech experience, you will learn from their ideas and enthusiasm. A good leader is always open to communication, you set the direction and vision (and you did choose the project) they come up with the clever ways of doing it.
I've been in a similar position a number of times.
Sometimes I just take charge, and be damned. Fair enough.
And many times, I resist the urge; I try to encourage my colleagues to take the lead. Sometimes this works, sometimes not.
And sometimes, I just come clean. "I seem to be taking over, as is my nature. But I don't want to railroad you guys. Anyone else fancy taking the lead? If not, are you happy with what I have already suggested? Speak up if you have any good ideas...". Again, sometimes this works, but not always.
Ultimately, you can 'lead the horse to water'... Projects need the lead, if no-one else rises to the challenge, it is better that you do.
Once you have the lead, lead by example...
The project should be interesting enough to keep them involve
the technology should be also recent
let them know this is how the industry moves and that they will gain the necessary experience to be in top of other programmers
offer prizes and punish those who break the build
rotate the positions let them test their ability to lead
offer non-monatary prizes or awards
Give them their own areas to "own." Even though they may not take pride in the project, they will want to make their own areas excel. Make they question, can their area be refactor or improved. It will make them learn new techniques or practices.
Allow them to learn by fire (in small phases) and then show them the correct way. Let them fail doing it their way, but allow time for them to do it the proper way.
Update:
Sorry to make the above sound like the team-leader would be the one in control of what is correct. It is mean to be more of a code-review that can be done by any one on the teams. They can move forward with the changes/refactor together as a team.
I am doing my masters degree currently and have had frequent group projects. It is not unusual to have only 1 or 2 members of the group doing the project. Not just from my projects but talking with other people. Basically what you said about the "SquareCog" and friends show is not unrealistic.
Really the more people you have on the group the lazier people will be. Also the more time lost communicating with them as they invent tons of ideas that they have no intention of following through with. It is well known that there is a point where extra programmers do not help the project anymore. There is only so much you can break something up. Over doing it will slow things down more than just giving a part to one person and create more dependencies.
Also the average student has a comfort zone, so even if you can get them to do some work, the will stay within their comfort zone. Someone has to leave their comfort zone for most projects (unless someone already knows the information in the class) to succeed. Most of the time I find that I am the only one willing to do that in my group, and some groups have no one. The most radical example was a 7 person project where almost no one did anything. One other guy was willing to do some light sys admin tasks and then the web design, that were within his comfort zone. One girl did some database design (and I do mean some because I basically did the design as a high level outline that she formalized with column names/data types). The rest did absolutely nothing. The class was distributed systems, so someone needed to learn JBoss (and Enterprise Java Beans), Amazon Web Services, etc... But it doesn't matter the class. In a data mining class, someone will have to figure out which techniques to use and how to use the toolkit.
Also many students are not good programmers. In fact there was someone in one of my groups who couldn't program at all. Really based on his description an MBA sounded like the right degree for him, but anyway he went through with the Masters in CS by farming out his programming to friends/contractors... Many are just terrible programmers and not just in style, they couldn't debug hello world with visual studio.... Rather than understand what went wrong they will just keep adding code until it works by coincidence.
One thing that happens quite often is that people come up with fairly ambitious projects that are not realistic for a semester. Usually I end up taking he scissors and cutting it to a barebones project and offer that once we finish the barebones part, then we can refine it and add the more advanced stuff. What almost always ends up happening is that people drag out finishing it and in the end after we get the barebones done no one wants to do anything additional.
There are 2 types of grad students. Full time grad students who take 4-5 classes, in which case they cannot afford to spend 40/80 or even 20 hour work weeks working on the project. Or part time grad students who have a day job, in which case they take 1 or 2 classes and have a full time job so they have even less time. I would say as a general estimate you can figure 6 hours of homework per graduate class (most will spend less). Assuming a normal class, probably 3-4 or that needs to be spent on studying/reading for the class. This leaves 2-3 hours per week per person to work on the project. Even getting that much would be good.
Some of the ideas floated like team lunches are not realistic at all. Many grad classes have group projects, and the full timers can't do 4 or 5 team lunches per week, that is like 5 hours of wasted time per week that could be spent on a rpoject. Also there may be money issues if you go to restaurants and expect all to buy lunch. And for someone who goes part time like me, I'm not going to do a team lunch because I work 9-6+ or 8-5 on college nights.
Probably your best bet is to find people's comfort zones and figure out tasks you can assign to them. Also to identify the freeloaders and not waste your time with them.
Also using version control for a school project seems like overkill. If the whole class is just the project maybe not. But assuming it is a normal class with lectures, exams, and homework assignments with the project done on the side, then any time spent on infrastructure is time you are not getting the project done. Really, though unrealistic for a professional environment, school projects are like start ups. Get them done, even if the code is a mess. You can always clean up later. But if you don't get it done, your grade will suffer. And in reality once it is done, I don't want to clean it up and no one else does either.... Getting everyone to use source control (unless you share a machine) would waste a lot of time with set up issues, and adjusting people to using it. i don't know what your project is. But with many graduate projects you have to do some research/experimentation and then the programming code is relatively simple. One class got me with a 5,000 line project, lucky it wasn't a group project.
Again on the project keep it simple. You can just coordinate the parts, assign the different parts and as they are done check/test it and then integrate it with your version control and leave them free to work on the project whatever is most comfortable.
Many will be happy to let you design the thing, implement the thing, and then learn absolutely nothing and just get the grade. It is their loss because they won't get the lessons of the project. But they are quite happy with squarecog and friends being just squarecog. Some will want to contribute something, but they are in the minority. If you get one of them great for you!!! Also watch out for over engineering. You have to look at things realistically. 3 hours per week per group member would be great, but I find even that is unrealistic. When a project is due sometimes you might get 5 or 10 hours per week from someone who slacked off. But you can't expect more.
How to Win Friends and Influence People has the following suggestions:
Fundamental Techniques in Handling
People
Don't criticize, condemn, or complain.
Give honest and sincere appreciation.
Arouse in the other person an eager want.
Six Ways to Make People Like You
Become genuinely interested in other people.
Smile.
Remember that a man's Name is to him the sweetest and most important
sound in any language.
Be a good listener. Encourage others to talk about themselves.
Talk in the terms of the other man's interest.
Make the other person feel important and do it sincerely.
Twelve Ways to Win People to Your Way
of Thinking
Avoid arguments.
Show respect for the other person's opinions. Never tell someone
they are wrong.
If you're wrong, admit it quickly and emphatically.
Begin in a friendly way.
Start with questions the other person will answer yes to.
Let the other person do the talking.
Let the other person feel the idea is his/hers.
Try honestly to see things from the other person's point of view.
Sympathize with the other person.
Appeal to noble motives.
Dramatize your ideas.
Throw down a challenge & don't talk negative when the person is
absent, talk about only positive.
Be a Leader: How to Change People
Without Giving Offense or Arousing
Resentment
Begin with praise and honest appreciation.
Call attention to other people's mistakes indirectly.
Talk about your own mistakes first.
Ask questions instead of directly giving orders.
Let the other person save face.
Praise every improvement.
Give them a fine reputation to live up to.
Encourage them by making their faults seem easy to correct.
Make the other person happy about doing what you suggest.
In addition to this, "Top Three Motivators For Developers (Hint: not money!)" notes the Autonomy, Mastery, and Purpose ideas that can also be great motivators for people when it comes to creative work.
I'm sure you have all been there, you take on a project where there is a creaky old code base which is barely fit for purpose and you have to make the decision to either re-write it from scratch or repair what already exists.
Conventional wisdom tends to suggest that you should never attempt a re-write from scratch as the risk of failure is very high. So what did you do when faced with this problem, how did you make the decision and how did it turn out?
It really depends on how bad it is.
If it's a small system, and you fully understand it, then a rewrite is not crazy.
On the other hand, if it's a giant legacy monster with ten million lines of undocumented mystery code, then you're really going to have a hard time with a full rewrite.
Points to consider:
If it looks good to the user, they
won't care what kind of spaghetti
mess it is for you. On the other
hand, if it's bad for them too, then
it's easier to get agreement (and
patience).
If you do rewrite, try to do it one
part at a time. A messy,
disorganized codebase may make this
difficult (i.e, replacing just one
part requires a rewrite of large
icebergs of dependency code), but if
possible, this makes it a lot easier
to gradually do the rewrite and get
feedback from users along the way.
I would really hesitate to take on a giant rewrite project for a large system without being able to release the new edition one part at a time.
Just clean up the code a little bit every time you work with it. If there isn't one already, setup a unit testing framework. All new code should get tests written. Any old code you fix as a result of bugs, try to slide in tests too.
As the cleanups progress, you'll be able to sweep more and more of the nasty code into encapsulated bins. Then you can pick those off one by one in the future.
A tool like javadoc or doxygen, if not already in use, can also help improve code documentation and comprehensibility.
The arguments against a complete rewrite a pretty strong. Those tons of "little bugs" and behaviors that were coded in over the time frame of the original project will sneak right back in again.
See Joel Spolsky's essay Things You Should Never Do. In summary, when you rewrite you lose all the lessons you learned to make your current code work the way it needs to work.
See also: Big Ball of Mud
It is rare for a re-write of anything complex to succeed. It's tempting, but a low percentage strategy.
Get legacy code under unit tests and refactor it, and/or completely replace small portions of it incrementally when opportune.
Refactor unless it is very bad indeed.
Joel has a lot to say on this...
At the very least, rewrite the code with the old code in front of you and don't just start over from scratch. The old code may be terrible, but it is the way it is for a reason and if you ignore it you'll end up seeing the same bugs that were probably fixed years ago in the old code.
One reason for rewriting at one of my previous jobs was an inability to find developers with enough experience to work on the original code base.
The decision was made to first clean up the underlying database structure, then rewrite in something that would make it easier to find full-time employees and/or contractors.
I haven't heard yet how it worked out :)
I think people have a tendency to go for rewrites because it seems more fun on the surface.
We get to rebuild from scratch!
We'll do it right this time!
etc.
There is a new book coming out, Brownfield Application Development in .NET by Baley and Belcham. The first chapter is free, and talks about these issues from a mostly platform agnostic perspective.
Repair, or more importantly, refactor. Both because Joel said so and also because, if it's your code, you've probably learned a ton more stuff since you touched this code last. If you wrote it in .NET 1.1, you can upgrade it to 3.5 SP1. You get to go in and purge all the old commented out code. You're 100x better as a developer now than when you first wrote this code.
The one exception I think is when the code uses really antiquated technologies - in which case you might be better served by writing a new version. If you're looking at some VB6 app with 10,000 lines of code with an Access database backend obviously set up by someone who didn't know much about how databases work (which could very well be you eight years ago) then you can probably pull off a quicker, C#/SQL-based solution in a fraction of the time and code.
It's not so black and white... it really depends on a lot of factors (the more important being "what does the person paying you want you to do")
Where I work we re-wrote a development framework, and on the other hand, we keep modifying some old systems that cannot be migrated (because of the client's technology and time restrictions). In this case, we try to mantain the coding style and sometimes you have to implement a lot of workarounds because of the way it was built
Depending on your situation, you might have another option: in-license third-party code.
I've consulted at a couple of companies where that would be the sensible choice, although seemingly "throwing away IP" can be a big barrier for management. At my current company, we seriously considered the viable option of using third-party code to replace our core framework, but that idea was ultimately rejected more for business reasons than technical reasons.
To directly answer your question, we finally chose to rewrite the legacy framework - a decision we didn't take lightly! 14 months on, we don't regret this choice at all. Just considering the time spent fixing bugs, our new framework has nearly paid for itself. On the negative side, it is not quite feature-complete yet so we are in the unenviable position of maintaining two separate frameworks in parallel until we can port the last of our "front-end" applications.
I highly recommend reading "Working Effectively with Legacy Code" by Michael Feathers. It's coaching advice on how to refactor your code so that it is unit testable.