As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I've been thinking about software estimation lately, and I have a bunch of questions around time spent coding. I'm curious to hear from people who have had at least a couple years of experience developing software.
When you have to estimate the amount of time you'll spend working on something, how many hours of the day do you spend coding? What occupies the other non-coding hours?
Do you find you spend more or less hours than your teammates coding? Do you feel like you're getting more or less work done than they are?
What are your work conditions like? Private office, shared office, team room? Coding alone or as a pair? How has your working condition changed the amount of time you spend coding each day? If you can work from home, does that help or hurt your productivity?
What development methodology do you use? Waterfall? Agile? Has changing from one methodology to another had an impact on your coding hours per day?
Most importantly: Are you happy with your productivity? If not, what single change would you make that would have the most impact on it?
I'm a corporate developer, the kind Joel Spolsky called "depressed" in a couple of the StackOverflow podcasts. Because my company is not a software company it has little business reason to implement many of the measures software experts recommend companies engage for developer productivity.
We don't get private offices and dual 30 inch monitors. Our source control system is Microsoft Visual Source Safe. Enough said. On the other hand, I get to do a lot of things that fill out my day and add some variety to my job. I get involved in business analysis, project management, development, production support, international implementations, training support, team planning, and process improvement.
I'd say I get 85% of my day to code, when I can focus and I have a major programming task. But more often I get about 50% of my day for coding. If production support (non coding-related) is heavy I may only get 15% of my day to code.
Most of the companies I've worked for were not actively engaged in evaluating agile processes or test-driven development, but they didn't do a good job of waterfall either; most of their developers worked like cut-and-paste cowboys with impugnity.
On occasion I do work from home and with kids, it's horrible. I'm more productive at work.
My productivity is good, but could be better if the interruption factor and cost of mental context switching was removed. Production support and project management overhead both create those types of interruptions. But both are necessary parts of the job, so I don't think I can get rid of them. What I would like to consider is a restructuring of the team so that people on projects could focus on projects while the others could block the interruptions by being dedicated to support. And then swapping when the project is over.
Unfortunately, no one wants to do support, so the other productivity improvement measure I'd wish for would be one of the following:
Better testing tools/methodologies to speed up unit testing
Better business analysis tools/skills to improve the quality of new development and limit its contributions to the production support load
Realistically, it probably averages out to 4 or 5 hours a day. Although its "lumpy" - there may be days where there could be 8 or 9 hours of it.
Of all the software developers I know, the ones that write production code (as opposed to research) 4 to 5 seems to be the max of actual coding. There is a lot of other stuff that goes on.
And to be honest there is a lot of procrastination. I find its a bit like writers block. sometimes its just hard to get started, but then a good 2 hour session is a LOT of work done. Its just all the preparation you have to go through, the experimentation to make sure you are taking the right approach. The endless amount of staring out the window and checking email etc...
I work a 37.5 hour week.
30 of those hours (80%) I am supposed to be billing our clients.
In reality I find that I use about 60% coding on actual client systems, 20% experimenting with new techniques and reading blogs, and 20% is wasted on office politics and "socializing".
Am I happy about it?
Do I wish that I could stare at the screen 30 hours a week coding on my given assignments?
Well. Since 20% of the time is used bettering myself at my craft, in the 60% that is effective coding I probably produce more than I would in 90% of my time if I didn't.
Then again, try to explain that fact to the higher ups ;)
Well, I generally come in at least
fifteen minutes late, ah, I use the
side door - that way Lumbergh can't
see me, heh heh - and, uh, after that
I just sorta space out for about an
hour.
...Yeah, I just stare at my desk; but
it looks like I'm working. I do that
for probably another hour after lunch,
too. I'd say in a given week I
probably only do about fifteen minutes
of real, actual, work.
For me, switching between projects is a big cause of procrastination. When I've just finished a project I tend to procrastinate on kicking off the next requirement assigned to me. My mind feels still like in coding mode, but I then have to estimate the expenses for creating a spec first. So I have to switch from coding to calling customers and the like, which feels uncomfortable.
What helps me most in being productive is to cut away any distraction in the first hours of the day and starting immediately with the day's most important task. I need to get into the flow as early as possible.
I recommend having a look at The Programmers’ Stone:
We know that stress impairs some cognitive functions. The loss of those functions can precisely explain why programming is hard, and show us many other opportunities to improve the ways we organize things. The consequences roll out to touch language, logic and cultural norms. Click here for the Introduction...
I spend about 40% of my day coding. 40% goes to non-coding activites (such as fighting with our sketchy build server or figuring out why NUnit failed with no error message again or trying to figure out why our code has stopped talking to the Oracle server downstaird... junk like that). The other 20% is usually spent procrastinating, or in meetings.
Am I happy with my productivity? Absolutely not. I work 7ish hours/day, and I spend about 2.5 of that coding. I would much rather be spending 5-6 hours of my day coding, with only an hour dedicated to all the other stuff (sadly, the one thing that would make that happen -- that the PM stop diddling with the build scripts every day -- isn't going to happen). Unfortunately, since I am a corporate developer, management doesn't see the time being frittered away. Because I get so much more done in that 40% of my day than most of the drones in the building get done in a week (including the PM), they think I'm productive.
#Bernard Dy: I have spent probably 30% of my career in corporate settings (am not at the moment). Usually its after some failed (or not failed, but fizzled) start up idea, or some kind of burnout/change. Its ok for a little bit, it is nice to meet people from totally different backgrounds (who would have thought that lawyers and actuaries could be so much fun to hang out with), but in the end, I just find it too hard to get up in the morning with motivation (or after a holiday dread going back) - probably for reasons like you define (just a lack of care). But its good experience and a source of ideas at the least. And you can meet brilliant people everywhere (its not just programmers who are smart - I always tried to seek out who the real brains were behind a business).
Interestingly the only time I have practiced strict agile/XP was in a corporate setting - in that case probably 7 hours a day was actual hands on code (in a pair) - I have never been so exhausted after a day of that. not sure if that is a good thing, perhaps I am just lazy.
To answer some of my own questions:
The current team I'm on does only does gross task estimation, so it's hard to track hours per days. I would say that, for my career, the time spent coding has been anywhere between 25% (mostly management) to 85%+ (working from home 4 days a week, get together for a meeting for half a day once a week). If I had to guess, though, the average is probably somewhere in the vicinity of 60%.
The biggest influence for me in time spent coding is the presence or absence of meetings. When I worked on agile projects with everybody in the same room, meetings tended to be ad-hoc and very short, so the time spent coding was very high. I also felt I spent less time -- sometimes a lot less time -- doing non-coding things when I was in a team room, because it's much easier to waste time, accidentally or otherwise, when nobody has a clear view of your monitor. :)
I do outsourcing and basically I code all day, I have two projects and I don't have much time to make anything else which it means that I can't take more work cause I could not finish anything, that is a good policy, you should take just as you can.
Remember also that that you should have spare time and very importantly is to rest enough because if you don't you won't be very productive. The key here is planning and discipline.
In my non-coding time I spent it with my wife, I also like to get out town and try not to think on my projects, the more I make this balance the more productive I am.
When I don't much work I like to read programing blogs and also I like to study programming.
And finally I would like to say that IMHO our carreer should not be seen as a work, instead you should see it like something fun.
I'm a software developer in an R&D department working 40 hour a week.
I spend like... 10% of my time actually coding.
In my non-coding hours I mostly test, evaluate, compare and put down results. I also spend a lot of time writing specification for the code I will write and researching for the code I will write, I participate in brainstorm meetings for the current projects, etc.
I could say that from my teammates (also software developers) I am the one who codes most at the moment; but in depends on which task we work at each time.
I would not quantify actually coding as working hard. If there is a good specification, a proper research and a good understanting of the project, coding is just a formality and goes on almost smoothly and quickly.
Here we have a sharred office, with two teams. We are mostly coding alone, rarely on a pair. My work changes a lot the amount of time I was coding; in the past I was spending most of my time coding, without a very good understanding of the coding. If I had a task I would immediately start coding, and re-coding each time I realised I did something wrong and so on. And it was very ineffective.
The development methodology is somewhere between prototyping and spiral now. It has clearly change the number of hour I code.
I am happy with my productivity, related to my deadlines and goals.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
As a lead developer I often get handed specifications for a new project, and get asked how long it'll take to complete the programming side of the work involved, in terms of hours.
I was just wondering how other developers calculate these times accurately?
Thanks!
Oh and I hope this isn't considered as a argumentitive question, I'm just interested in finding the best technique!
Estimation is often considered a black art, but it's actually much more manageable than people think.
At Inntec, we do contract software development, most if which involves working against a fixed cost. If our estimates were constantly way off, we would be out of business in no time.
But we've been in business for 15 years and we're profitable, so clearly this whole estimation thing is solvable.
Getting Started
Most people who insist that estimation is impossible are making wild guesses. That can work sporadically for the smallest projects, but it definitely does not scale. To get consistent accuracy, you need a systematic approach.
Years ago, my mentor told me what worked for him. It's a lot like Joel Spolsky's old estimation method, which you can read about here: Joel on Estimation. This is a simple, low-tech approach, and it works great for small teams. It may break down or require modification for larger teams, where communication and process overhead start to take up a significant percent of each developer's time.
In a nutshell, I use a spreadsheet to break the project down into small (less than 8 hour) chunks, taking into account everything from testing to communication to documentation. At the end I add in a 20% multiplier for unexpected items and bugs (which we have to fix for free for 30 days).
It is very hard to hold someone to an estimate that they had no part in devising. Some people like to have the whole team estimate each item and go with the highest number. I would say that at the very least, you should make pessimistic estimates and give your team a chance to speak up if they think you're off.
Learning and Improving
You need feedback to improve. That means tracking the actual hours you spend so that you can make a comparison and tune your estimation sense.
Right now at Inntec, before we start work on a big project, the spreadsheet line items become sticky notes on our kanban board, and the project manager tracks progress on them every day. Any time we go over or have an item we didn't consider, that goes up as a tiny red sticky, and it also goes into our burn-down report. Those two tools together provide invaluable feedback to the whole team.
Here's a pic of a typical kanban board, about halfway through a small project.
You might not be able to read the column headers, but they say Backlog, Brian, Keith, and Done. The backlog is broken down by groups (admin area, etc), and the developers have a column that shows the item(s) they're working on.
If you could look closely, all those notes have the estimated number of hours on them, and the ones in my column, if you were to add them up, should equal around 8, since that's how many hours are in my work day. It's unusual to have four in one day. Keith's column is empty, so he was probably out on this day.
If you have no idea what I'm talking about re: stand-up meetings, scrum, burn-down reports, etc then take a look at the scrum methodology. We don't follow it to the letter, but it has some great ideas not only for doing estimations, but for learning how to predict when your project will ship as new work is added and estimates are missed or met or exceeded (it does happen). You can look at this awesome tool called a burn-down report and say: we can indeed ship in one month, and let's look at our burn-down report to decide which features we're cutting.
FogBugz has something called Evidence-Based Scheduling which might be an easier, more automated way of getting the benefits I described above. Right now I am trying it out on a small project that starts in a few weeks. It has a built-in burn down report and it adapts to your scheduling inaccuracies, so that could be quite powerful.
Update: Just a quick note. A few years have passed, but so far I think everything in this post still holds up today. I updated it to use the word kanban, since the image above is actually a kanban board.
There is no general technique. You will have to rely on your (and your developers') experience. You will have to take into account all the environment and development process variables as well. And even if cope with this, there is a big chance you will miss something out.
I do not see any point in estimating the programming times only. The development process is so interconnected, that estimation of one side of it along won't produce any valuable result. The whole thing should be estimated, including programming, testing, deploying, developing architecture, writing doc (tech docs and user manual), creating and managing tickets in an issue tracker, meetings, vacations, sick leaves (sometime it is batter to wait for the guy, then assigning task to another one), planning sessions, coffee breaks.
Here is an example: it takes only 3 minutes for the egg to become roasted once you drop it on the frying pen. But if you say that it takes 3 minutes to make a fried egg, you are wrong. You missed out:
taking the frying pen (do you have one ready? Do you have to go and by one? Do you have to wait in the queue for this frying pen?)
making the fire (do you have an oven? will you have to get logs to build a fireplace?)
getting the oil (have any? have to buy some?)
getting an egg
frying it
serving it to the plate (have any ready? clean? wash it? buy it? wait for the dishwasher to finish?)
cleaning up after cooking (you won't the dirty frying pen, will you?)
Here is a good starting book on project estimation:
http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351
It has a good description on several estimation techniques. It get get you up in couple of hours of reading.
Good estimation comes with experience, or sometimes not at all.
At my current job, my 2 co-workers (that apparently had a lot more experience than me) usually underestimated times by 8 (yes, EIGHT) times. I, OTOH, have only once in the last 18 months gone over an original estimate.
Why does it happen? Neither of them, appeared to not actually know what they are doing, codewise, so they were literally thumb sucking.
Bottom line:
Never underestimate, it is much safer to overestimate. Given the latter you can always 'speed up' development, if needed. If you are already on a tight time-line, there is not much you can do.
If you're using FogBugz, then its Evidence Based Scheduling makes estimating a completion date very easy.
You may not be, but perhaps you could apply the principles of EBS to come up with an estimation.
This is probably one of the big misteries in the IT business. Countless failed software projects have shown that there is no perfect solution to this yet, but the closest thing to solving this I have found so far is to use the adaptive estimation mechanism built in to FogBugz.
Basically you are breaking your milestones into small tasks and guess how long it will take you to complete them. No task should be longer than about 8 hours. Then you enter all these tasks as planned features into Fogbugz. When completing the tasks, you track your time with FogBugz.
Fogbugz then evaluates your past estimates and actual time comsumption and uses that information to predict a window (with probabilities) in which you will have fulfilled your next few milestones.
Overestimation is rather better than underestimation. That's because we don't know the "unknown" and (in most cases) specifications do change during the software development lifecycle.
In my experience, we use iterative steps (just like in Agile Methodologies) to determine our timeline. We break down projects into components and overestimate on those components. The sum of these estimations used and add extra time for regression testing and deployment, and all the good work....
I think that you have to go back from your past projects, and learn from those mistakes to see how you can estimate wisely.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I am final year college student. I am trying to figure out how much time i should spend on coding and learning.
while (true) {
learn;
code;
}
I have two friends in my university, both studying media informatics, and both were absolute beginners in programming.
The first one reads a lot at home if he has to learn new languages for a project but has never had a private programming project.
The second one reads a bit but has his own python project. A web application for his friends, where you can bet on soccer results.
Both in comparison:The first guy is slow in programming and always stumble upon simple things and his code can be optimized (in line numbers and comments) at least by 5. And in two days he will stumble upon the same issue again...
The second guy is much faster, can easily read foreign code and languages and stumble upon one problem at most two times, the third time he used what he has learned....
So imho, doing your own project, where you code because you love it, where you work until morning to fix a bug or to finish an implementation, is the very best way to learn!
Surely you've realized that putting a finite measurement on the amount of time you should spend coding, is futile and hugely irrelevant.
Do what you want, but always try and keep up to date.
When I first started programming, it seems that I learn new things by leaps and bounds. Functions, classes, inheritance and etc. But after a while, I realize that you learn by coding. I load myself up with tonnes of reading material - Effective C++, Modern C++, but nothing beats them when I actually sat down and code.
Of course, writing your code the same way over and over again does not make you a better programmer. You have to learn to think - how do I make it reusable? less error prone? portable? immune to changes in other areas of the application? easier to maintain? Is there a better way to do this?
Eventually, the learning peaks, and what you learn are what I like to term as multipliers. It's like knowing that dirname(__FILE__) in PHP returns the current directory which an include file is in. It's like finding out what's a ORM and how by abstracting away the DB you can focus more on the code logic rather than an endless routines of writing INSERTS and UPDATEs SQL statements. It's like learning smart pointers and effective use of STL in C++, using Firebug effectively when doing JavaScript/CSS/HTML...and lots more.
So code; once you get frustrated about something ("There must be a better way to do this than now!"), search for a better way - this is how I learn, anyway.
When I was young:
Monday to Friday, 10am to 7pm, programming in office
Saturday afternoon, reading in Chapters
Monday to Saturday, 9pm to 1am, programming at home
Sunday, drive to downtown and pick up a few books from the bookstore
those were the days when Google was know as nntp
These days:
Monday to Friday, 10am to 7pm, coding in office (too bad I am on web now ;-)
9pm to 1am, coding on my MacBook Air on a few iPhone projects
Saturday and Sunday, coding for another 16 hours
too bad, Google interrupts me too much and I cannot count how many hours are spent on reading blog and pdf books ...
You have to decide that yourself. If you're constantly feeling like you should spend more time coding, then you're probably right. You should never force yourself to the point where the sight of a curly bracket makes you want to puke either. If you're sufficiently interested in programming then the amount of time you spend naturally without slacking off/burning out, will be just fine. (And if you're not, you should cut your losses as soon as possible.)
Be sure that this kind of approach does not make you less valuable of a programmer than the angry nerd in your class who spends every waking hour coding as a part of his masterplan to get back at the world.
the simple answer: do not create some sort of a schedule
why?
you never can know ahead what situation you're in during a certain time, so let's say you set it at everyday at 10am, then suddenly your dog died today at 10am, your family called you to mourn over poor Snuffel...for hours; schedule's all ruined
so what do you do?
code up; if you get tired grab a book or read an article (articles today are really juicy), if you get tired of reading and coding, play games that rattle your brain (yet entertaining, something like Civilizations IV). if you're all rested, fire up your IDE and apply what you just read about. Don't worry if you get it all messy the first time (unless you're a mad genius who certainly will kill himself if he doesn't get something right on his first try).
Note: you should probably set a time for how long you play the game, though:)
My suggestion would be to discover your strengths and if learning is among them, then you may enjoy spending a lot of time learning so do what you want here. Of course one shouldn't go so overboard that things like hygiene get sacrificed, so do try to maintain some basic standard of existing which includes the basics of cleaning your place, yourself, and that kind of thing.
For myself, I'd say that I'm almost always trying to learn something, somewhere. Maybe it is learning about how much patience I have in traffic or how well can I handle this curveball that life has thrown me by my having to do things like income taxes and discover what has changed in the software or tax laws from the previous year. If you look at life as a series of opportunities, you may learn a lot in the world.
In my humble opinion most of the time you are programming. While you program you are learning from experience. This is one type of learning. Another type of learning comes from reading books and other resources (courses, internet, Development conventions). I use books to keep up with technology and to better understand what I am doing. I read almost every day from 0.5-1.0 hour. It depends on your free time and the type of person you are.
Please take into account that there are more ways to learn: code reviews, reading other people's code and I am sure that there are more that I didn't mention here.
Anyway, good luck.
I am guessing the 'learning' here means, acquiring new tips and tricks, grapsing new technology in the market and stay up to date with the trends in technology.
From my experience it is taking around 20% time for learning, and it is mainly because I work on all latest technologies from Microsoft like WPF/Silverlight/Surface. But this % of time will really depends on your personal interest/ organizational interest and the type of career growth you are looking forward to.
And if your job is merely converting domain/business logic to the code which doesn't involve critical technology roadblocks then it might be close to 0% time you need to spend on learning.
Since you didn't present any constraints or conditions in your question then the simplest answer I can give is:
Spend as much as you want.
The very fact you have to ask, might mean you are not ideally matched to writing code. First and foremost you should love coding, and finding out how things work.
This is not a profession where you ever stand still. Totally agree with another poster, in that you should always be looking for a better way, and recognising when there is no better way.
The best software workers - the rockstars if you will - are always on. Any situation can be a teaching. For instance, consider Gregor Hohpe's article Starbucks Does Not Use Two-Phase Commit, in which he analyses how the coffee vendor uses asynchronous processing to maximize throughput of customers' orders.
Coding == Learning
In my opinion.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I lead a small group of programmers in a university setting, having just moved into this position last year. While the majority of our team are full time employees, we have a couple of people who are traditionally graduate assistants.
The competition for these assistantships is fairly intense, as they get free graduate school tuition on top of their salary while they have the job. We require that they sign up for at least a year, though we consider ourselves lucky if they stay for two. After that, they get their master's degree and move on to bigger and better things.
As you can imagine, hiring and re-training these positions is time- and resource-intensive. To make matters worse, up to now they have typically been the sole developer working on their respective projects, with me acting in an advisory and supervisory role, so wrangling the projects themselves to fight the entropy as we switch from developer to developer is a task unto itself.
I'm tempted to bring up to the administrators the possibility of hiring a full- (and long-haul) developer to replace these two positions, but for a school in a budget crisis, paying for two half-time graduate assistants is far cheaper (in terms of salary and benefits) than paying for one full-time developer. Also, since I'm new to this position, I'd like to avoid seeming as though I'm not able to deal with what I signed up for. For the forseeable future, I don't think the practice of hiring short-term graduate assistants is going to change.
My question: What can I do to create an effective training program considering that the employees may be gone after as little as a year on the job?
How much time should I invest in training them, and how much would simply be a waste of time?
How much time should they take simply getting acclamated to our process and the project?
Are there any specific training practices or techniques that can help with this kind of situation?
Has anyone dealt with a similar situation before?
Do I worry too much, or not enough?
By the way, and for the record, we do the vast majority of our development in Perl. It's hard to find grad students who know Perl, while on the other hand everybody seems to have at least an academic understanding of Java. Hence this question which I asked a while back.
Why don't you ask the students what they find difficult and make cheat sheets, lectures, etc. for the parts of the job that they have trouble with? Maybe you need to create some introductory Perl lectures or purchase some dead trees. How about a Safari subscription at O'Reilly? I'd ask the students how they prefer to learn, though, before embarking on a training project. Everyone has different learning styles.
I'd also spend some time and capital creating a culture of professional software development at work. It'll be tough since academic programmers are often neophytes and used to kludging up solutions (I'm an academic programmer, btw) but the students will thank you in the long run. Maybe you can all go out to lunch once a week to discuss programming and other topics. You might also want to take some time to do code reviews so people can learn from each other.
With high turnover you definitely need to ensure that knowledge transfer occurs. Make sure you are using source code control and that your students understand proper commenting. I'd also make the students create brief documentation for posterity. If they are getting credit, make them turn in a writeup of their progress once a semester. You can put this in a directory in the project's repository for anyone who inherits it. As mentioned in other posts, a group wiki can really help with knowledge transfer. We use Mediawiki in our group and like it a lot.
One last thing I should add is that I find it helps to keep a list of projects for new developers that relatively easy and can be completed in a month or so. They are a great way for new people to get acclimated to your development environment.
This is a relative question, and should be taken on a case-by-case basis. If the new hire already knows Perl, you don't need to go over this piece of training (yes, you could put Perl as a mandatory prerequisite, but that would significantly limit your applicant pool), and their first bit of training should be something like fixing a bug in an existing application or walking them through an application they will maintain. Though, given that the developers are only there for a year makes me think the development styles are going to vary some (if not a lot).
Getting the new person up to speed with your process is very important, as long as your process works. In this high turnover environment, you should put a strong emphasis on documentation in your process. A Wiki is a great thing to have for this documentation, since it's centralized and any of the developers can access it. Having them try to figure out how a project works by themselves (with little to no documentation) is a waste of both their time and your time.
Perhaps I'm reading too much into the question, but if your university teaches java, why are you using Perl? Wouldn't it make more sense to use the tools that your students already know? This alone would cut the learning curve significantly. [once you eliminate the legacy code of course]
other than that, try:
break the projects up into month-sized bites
overlap the internships by at least 2 months, if not 6, so the new guy can work with and be trained by the old guy(s)
document whatever repeatable processes you have (as was suggested by Mark Nold)
if the grad students are cheaper than full-time pros, quit whinin' ;-) If not, go for the pros.
Have you considered making a "three ring binder" like Macdonalds and many other high turn over industries have? Have one folder which you can print out and hand to the new hire which shows the new hire some basics of getting up and running with Perl in your environment. This should be a "hello world", plus some basic regex and array manipulation. Lastly your manual should go on to show examples of the 5 things you find yourself doing all the time.
The example code may be authenticating users against an external security system, walking through recordsets or using ghostscript to create PDFs. Whatever they are, they should cover the basics of what you meet 80% of the time. More importantly the examples should show users how you expect the code to be written for clarity (eg: naming and approach), and give them some insight into servers and software in use and other practicalities which a generic book won't show them.
You won't get the binder right first time, but since you have a high staff turn over, you'll have plenty of time to test and improve it.
On top of this i would pick a single Perl programming book and give the new user their own copy of your three ring binder, plus "Programming Perl" to keep on their first day. At a cost of $50 per hire i'm sure it's a lot cheaper than the alternatives and you'll have them flipping burgers.... i mean cutting code in no time.
My initial couple of thoughts are that you should:
hire for the position, i.e. it's Perl-centric so make that a big part of the pre-requisites. That way you don't need that piece of training as well.
invest time in the on-boarding process, maybe use a wiki so that you can easily update it to help bringing them on-board.
Edit: Some extra points:
maybe have a chat and see if Perl can be introduced into the curriculum? If not, then make it known six months before the ads go up that applicants need to know Perl. This way you'll get people who have Perl experience and who have actively demonstrated their motivation.
can you open up some small projects so that they could be done by potential candidates during this six months?
approach the design of your large-scale projects so that they can be done in a piece meal manner. This is how The ACE Components have been done iirc.
allow a specific period for documentation and review of the work done by the departing grad student.
allow an overlap period of at least a couple of weeks where the new grad student can work with the departing grad student. They can learn the development environment and they can be guinea pigs for any updates to your wiki.
Still more to come...
HTH
cheers
That is a pickle, but it is not as uncommon in the commercial sector as you would think. I heard a statistic once that the average tenure of a programmer industry-wide is about 18-24 months. Normally I would suggest getting more experienced programmers who would require less ramp-up time and only need to be trained on the problem domain/technology updates and not the basics
I think your best bet is just to ask for about 30-50% more grad students than it will take to actually perform the job to account for the learning and ramp-up time and invest in some additional resources for testing as this environment is a recipe for mistakes since everyone is learning on the job. Also, this is probably difficult given the academic schedule, but try to stagger the start dates as much as possible to maximize the overlap between employees. Pair-programming teams of new-hires/old-hires might also help increase consistency and supplement the training without sacrificing too much productivity.
How much time should I invest in training them, and how much would simply be a waste of time?
Answer: It's not the amount of time or the amount of waste, but perhaps the approach. Would it be possible to video train - video yourself training one person and provide it as training for subsequent students/developers. You can add over time, but it does reduce your time needed to go through the same process.
How much time should they take simply getting acclamated to our process and the project?
Answer: This is all dependent on the person...min. of a day max of 2-3 weeks I would guess on avg.
Are there any specific training practices or techniques that can help with this kind of situation?
Answer: Video training (home made), having the current student/developer create/update a wiki of needed info, points of interest, etc.
Has anyone dealt with a similar situation before?
Answer: Use to be a 12-18 month avg. turnover - I would imagine that it's changed now (longer), but every company has turn-over, but perhaps not forced like yours due to the resource being students.
Do I worry too much, or not enough?
Answer: Knowledge lost through transition is a key risk area...
Is the application something you could consider open-sourcing?
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
As long as there are software projects, the world is wondering why they fail so often.
I would like to know if there is a list or something equivalent which shows how many software projects fail today. Would be nice if there would be a comparison over the last 20 - 30 years.
You can also add your top reason why a software project fails. Mine is "Requirements are poor or not even existing." which includes also "No (real) customer / user involved".
EDIT: It is nearly impossible to clearly define the term "fail". Let's say that fail means: The project was more than 10% over budget and time.
In my opinion the 10% + / - is a good range for an offer / tender.
EDIT: Until now (Feb 11) it seems that most posters agree that a fail of the project is basically a failure of the project management (whatever fail means). But IMHO it comes out, that most developers are not happy with this situation. Perhaps because not the manager get penalized when a project was not successful, but the lazy, incompetent developer teams?
When I read the posts I can also hear-out that there is a big "gap" between the developer side and the managment side. The expectations (perhaps also the requirements) seem to be so different, that a project cannot be successful in the end (over time / budget; users are not happy; not all first-prio features implemented; too many bugs because developers were forced to implement in too short timeframes ...)
I',m asking myself: How can we improve it? Or do we have the possibility to improve it? Everybody seems to be unsatisfied with the way it goes now. Can we close the gap between these two worlds? Should we (the developers) go on strike and fight for "high quality reqiurements" and "realistic / iteration based time shedules"?
EDIT: Ralph Westphal and Stefan Lieser have founded a new "community" called: Clean-Code-Developer. The aim of the group is to bring more professionalism into software engineering. Independently if a developer has a degree or tons of years of experience you can be part of this movement.
Clean Code Developers live principles
like SOLID every day. A professional
developer is the biggest reviewer of
his own work. And he has an internal
value system which helps him to improve and become better.
Check it out on: Clean Code Developer
EDIT: Our company is doing at the moment a thing called "Application Development and Maintenance Benchmarking". This is a service offered by IBM to get a feedback from someone external on your software engineering process quality etc. When we get the results, I will tell you more about it.
Bad management.
Projects are not successes or failures based on some underlying feature of the project, but on whether they fulfill the needs of the users. (They can fail altogether, in which case there was a gross misstatement of what was possible.) It is mostly in the process of evaluating the feasibility and cost-benefit ratio of the project, and establishing goals, that software projects tend to fail or succeed.
There's a disconnect between people who deal with facts and things (like programmers) and people who deal with other people (like sales types and managers). To a programmer, the facts are the facts, and have to be dealt with. To a sales person, the facts are what other people think, and are changeable.
There's also differences between tangible and intangible facts. Nobody thinks that workers could build a large bridge in a month if they were really motivated; they can see all the steel and concrete and other stuff that has to be moved and fixed into position. Software is much less tangible, and lacks the physical restrictions: while it is not even theoretically possible to build the bridge within a month, it is conceivable that a team could create a large project within a month, as "all" they have to do is get everything right the first time. It is physically possible to type thousands of lines of code a day; it's just that the chance that they're usable as is is so close to zero it doesn't matter. The actual productivity of a top developer is actually pretty unimpressive in word count, compared to (say) the productivity of a journalist.
Therefore, those who are used to flexible facts don't have the imposing physical limits to remind them that things can be pushed only so far, no appreciation for what programming actually requires, and no good feel for how much productivity is realistically possible. Moreover, they know how to get their way in negotiations, much more than the average developer, so in negotiations about what's possible they tend to assume more than they can, in the real world, get.
In the meantime, software development is inherently fuzzy, because producing the physical product is trivial. I can produce a copy of software quickly and cheaply, once it's been developed. Software development is design work, pure and simple. Anything corresponding to manufacturing is ruthlessly eliminated with such things as compilers and wizards and code generation. The developer, faced with the manager who wants the impossible, finds it hard to say the impossible is actually impossible, because there's no way to say it's actually impossible. Given facts that are unknown enough to feel flexible, the person with strong negotiating skills and determination will typically get the answer he or she wants.
Given this disconnect, one might ask whose responsibility it is to bridge it. The answer is, in my opinion, clear. The responsibility for understanding how different people think belongs to the people who specialize in dealing with other people. The responsibility for coordinating different types of people belongs to the people whose job it is to coordinate these things. Therefore, managers.
Managers who do understand software development and developers, and can deal well with other managers, will do well, and their projects will generally succeed. There are still far too many of the other type in the world.
Not a direct answer, but I found the Virtual Case File to be a fascinating case study on how a big government-backed well-funded project can still tank.
You can also add your top reason why a
software project fails.
Another IEEE Spectrum Online article "Why Software Fails" examines this very question. It summarizes the major points as follows:
Unrealistic or unarticulated project goals
Inaccurate estimates of needed resources
Badly defined system requirements
Poor reporting of the project's status
Unmanaged risks
Poor communication among customers, developers, and users
Use of immature technology
Inability to handle the project's complexity
Sloppy development practices
Poor project management
Stakeholder politics
Commercial pressures
Poor planning.
Hofstadter's Law
It always takes longer than you expect, even when you take Hofstadter's Law into account.
Mismanagement.
SW project get started by throwing developers against a perceived problem. Business requirements crystallize as the project progresses. New functionality gets added while deadlines stay put. More developers are thrown in. Original project members quit or get fired. By this point too much time, money and resources is invested in the project so it cannot be canceled. As the deadline passes the project is declared finished and successful despite the obvious lack of finished product.
Come to think of it - I've jet to see a SW project fail...
Honestly, I think its because most programmers are not very good at what they do(and I don't mean just cranking out code). People on stackoverflow are probably the exception. I don't know about the rest of you but as a consultant/contract programmer I have worked in or around many places, and the ratio of mediocre or poor programmers to good ones is about 10 to 1.
One of my strengths has always been estimating accurately and then delivering on time and on or under budget - I always aim for coming in 10% under cost and on time. Then I like to tell my client that because I got things done for less $$ than expected, which of the "extras" would you like to add in?
Even a perfectly functioning product that is late and/or over budget will be considered a failure by many business managers. Programmers often focus on just the technical aspects of what they do, with little regard for the cost or deadline. You really need to do all three well for it to be deemed successful project. There are many other programmers that could code circles around me without a doubt, but for the person paying for the project, that is rarely enough.
It is because no-one seems to read anymore.
Every single reason why projects fails has been analyzed time and time again.
You only have to read three books to know why 80% of projects fail:
The Deadline: A Novel About Project Management (Tom Demarco, published 1997)
It's a great introduction and it's pretty entertaining.
Peopleware : Productive Projects and Teams (Tom Demarco, published 1987)
The Mythical Man-Month: Essays on Software Engineering (Fred Brooks, published 1975)
We as a profession simply seem to forget everything every 3-5 years (see "centralised computing is inefficient; let the clients handle it" vs cloud computing).
(From a programmers point of view - I'm not a project managemer, but I've often been involved in the process).
A number of people have mentioned that bad programmers are endemic. But I think this is true in another sense as well - we're all bad programmers in that we find it difficult to anticipate complexity, an unavoidable issue that 50 years of magic bullet estimation and planning schemes have failed to solve.
Anticipating the side effects of large projects gets exponentially more difficult as projects grow. This is a dull truism, for sure, but for me it means that on any project I've worked on where I've been involved in the estimating process I've run into some case where there's an unanticipated consequence of a design decision that causes everything to come to a grinding halt, or at least a few days of bugfixing - just something that nobody foresaw, not any sort of malpractice or stupidity. It's just the nature of a complex enough system.
Aside from the built-in uncertainty, there's also a tendency to underestimate things whose outline is known, because the fact that they have less uncertainty makes them seem simpler to implement.
So the uncertain stuff gets magnified, the clear stuff gets minimized, and what really kills you is the thing that you didn't think would be uncertain.
The number one reason: a failure of project management.
A PM's raison d'etre is to make a project succeed, ergo a project failure is their failure. Certainly there are factors beyond their control, but it's still within the PM's job description to manage that risk, and the only get out clauses should be someone higher up the food chain taking decision control (which is a terrible thing to do to a PM) or acts of god.
In my experience failures mostly occur when PM work has been fast and loose or non-existant, including when decisions start to flow from sales people and when the client starts decreeing change control. A good PM is priceless.
Failure is a judgement -- more of an accusation, really.
"The project was more than 10% over budget and time."
Which budget? Which schedule?
6 months ago, I wrote a plan saying it would take 6 months.
3 months ago, the users asked for more stuff. I gave them a plan that said it would take 9 more months.
Last month I was told that the project was 6 months over budget and therefore a "failure".
But wait. It delivered what the users wanted. It was over the "original" estimate. It was under the revised estimate. The users want more. IT wants less.
I'll approach it from a different aspect than most the rest here.
I've noticed a project slowly fail over a period of time. Sure, it's gotten better in that time too--but it still isn't profitable. In this market profitability, and being in the black, means success.
Why is it failing? I think it's simple: you get what you pay for.
Software is like a bank account, not primordial ooze. If you don't put resources into it (time, money, focus, effort) then you won't get anything out of it except failure and cost. So you must invest things into your project, and sometimes the earliest work sets the stage for years to come. You can't throw mud at your computer and expect a new mouse in two years and $10 million dollars later, so likewise there must be effort expended.
One of the biggest problems today are "budget developers" in a third-world country. I don't begrudge them their part of the market, but for a well-funded Silicon Valley startup to seek them out and get a budget foundation (framework or even prototype) is to make a poor investment in the future. This very same budget framework is what is causing my friends so much of a hassle today. It works now; it worked when it was written, but it wasn't written well and few even take the time to maintain it. Were the company to stop and rewrite the software the way it should have been written in the first place they wouldn't have all this trouble. Can they afford the time? Nope. They have to make it profitable before they can even thing of it.
As the saying goes, "I can make it: cheap, fast or good. Now, pick any two of those." Everyone wants all three, myself included. But if you don't invest the time, planning, and work required to make your project a success from the start ... then don't expect anything you can be proud of later. It'll feel like a forged Mona Lisa where you, and every other engineer like you, can see a defect here and there that shouldn't have been there from the start.
So:
Don't undertake what you cannot afford in: time, money, effort, focus, etc.
Don't skip planning!
Don't be afraid to rewrite early when it counts the most. (Later it'll be worse than a trip to the dentist, believe me.)
Don't underestimate the power of bureaucracy to prevent you from doing it right.
And don't be cheap where you should spend the most of your time. It will cost you later, guaranteed. And if not you, then someone else will take the bullet for you.
One common mistake is that sales people and technical people do not communicate sufficiently. So the salespeople sell things that are technically not feasable within budget. (And then they run with their bonus :) )
Being over budget and time is not a good definition of failure (and actually being in budget and time doesn't always mean success). Take the following examples provided by Hugh Woodward, PMP, in Expert Project Management - Project Success: Looking Beyond Traditional Project Metrics:
Sydney Opera House: With its graceful sails dominating Sydney Harbor, the Sydney Opera House is arguably one of the most recognized buildings in the world. Yet, from a project management perspective, it was a spectacular failure. When construction started in 1959, it was estimated to cost $7 million, and take four years to build. It was finally completed in 1973 for over $100 million.
[...]
Project Orion: This massive effort to develop Kodak's new Advantix photographic system was reputedly very well managed from a project management perspective. PMI recognized it as the 1997 International Project of the Year, and Business Week selected the system as one of the best new products of 1996 (Adams, 1998). But Kodak's stock price has fallen 67% since the introduction of the Advantix system, in part because it failed to anticipate the accelerating switch to digital photography.
Corporate Intranet: Finch describes a project that involved the implementation of a corporate intranet to globalize and improve communications. From a traditional project perspective, it failed to meet its success criteria, but not significantly. It was one month late and believed to have been accomplished with a small budget overrun. But both the project manager and senior management viewed the project as successful. The hardware and software had been installed successfully with a minimum of disruption, thereby providing all staff members with access to the corporate intranet. Following implementation, however, employees made only limited use of the intranet facilities. The main objective of the project was therefore not achieved. In this case, both the project manager and senior management focused on an objective that was too narrow.
[...]
Manufacturing Plant Optimization: A paper manufacturing company with five plants across North America decided to increase its manufacturing capacity by embarking on a de-bottlenecking program. A project team was formed to install the necessary equipment, and charged with completing the work in 18 months at a cost of $26 million. But almost immediately, the project team was asked to defer major expenditures until an unrelated cash flow problem was resolved. Rather than stop work completely, the team adopted a strategy of prototyping the technologies on which the de-bottlenecking program was based, and actually developed some cheaper and more effective solutions. Even when the project was authorized to proceed, the team continued this same approach. The project eventually spanned five years, but the resulting capacity increase was three times the initial commitment. Not surprisingly, the company immediately appropriated another $40 million to continue the program.
[...]
So in these examples, which ones were truly successful? Examples like the 2002 Winter Olympics and the Batu Hijau Copper Concentrator would suggest that these are truly successful because they not only met the traditional project managers' definition of success, but also met the projects sponsors' perception of success.
As we start to look at the examples
like Project Orion, the Corporate
Intranet and the Laptop Upgrade, we
notice that the traditional metrics
start to fail. These projects are
considered successes in project
managers' definition of success, but
failed at meeting the sponsors'
success criteria. The project Orion
example is quite astounding as this
project was recognized by PMI (Project
Management International) in 1997 as
the International project of the year.
Yet it did not increase Kodak's
revenue, because they did not foresee
the adoption of digital cameras.
Most interesting are the examples of
the Manufacturing Plant Optimization
and the Sydney Opera House. They both
failed to meet the traditional project
managers' success metrics but were in
fact considered successes. This is
particularly shocking when you see
that the Sydney Opera House had a
"cost overrun of 1300%" and a
"schedule overrun of 250%".
Once we realize that projects can fail
to meet the traditional metrics of
success, but still be successful to
the stakeholders, this creates a
quandary for the project manager. How
does one really define success? Is it
possible that a "Challenged" project
could be canceled that would have met
the sponsors' needs? Is it also
possible to identify a project that
should be canceled that is currently
on time, on budget and meeting the
defined needs?
What do you think of that conclusion? How do you really define success?
My answer is rather unusual from the rest of this, but quite common around here: killer bugs. I had a project go an extra two weeks (50% extra time) because of one switch in source that I didn't know about until I dug through the source code (there was nothing documented in help or on the web).
People/companies do not proudly shout about their failures, so many cases wont be ever heard.
Poor use of practices and software development methods. In my experience, one of the big reasons a project failed its that the development team use a wrong method to face the software development process. Choosing a methodology without having a good understanding of how it works and what it takes, can bring a time consuming issues to a project, like poor planning.
Also a common problem is also the use of technologies without a previous evaluation of it to understand how it can be applied, and if it brings any value to the project.
There have been some good studies done on this. I recommend this link from the Construx website (Steve McConnells company).
The Construx link above is real good!
Many projects are managed on a rosy picture of reality. Managers tend to power talk developers into optimistic estimates. But say a 20 week projects gets "talked down" to 10 weeks. The requirements phase will now be 1 week instead of 2. After 1 week, the requirements aren't finished, but the project moves on. Now you're working on shaky ground-- and on a stretched schedule!
This can be funny. Once there was this old guy in a room opposite mine. His job title was system adminstrator. But the system he was supposed to adminsiter wasn't there. It had never been finished, although management thought it had been. The guy played games for about a year before he got bored and moved on.
The funniest part? Management put up a new job opening after he left!
IT Project Failures is a blog about project failures that may have a few answers here if one wants to read about it.
Myself, I think a large part of this lands on the question of being able to state exactly what is expected in x months at $y when really the answer is much more open-ended. For example, if a company is replacing an ERP or CRM system, there is a good chance that one isn't going to get all the requirements exactly right and thus there will be some changes, bug fixes and extras that come from taking on such a large project. For an analogy consider how many students entering high school or university could state their precise schedule for all 4 years without taking any classes and actually stick to that in the end. My guess would be very few do that as some people change majors or courses they wanted to take aren't offered or other things happen that change what the expected result is but how is this captured in a project plan that we started here and while we thought we were going there, we ended up way over in spot number three.
The last statistic that I heard was that 90% of projects are either over time or over budget. So if you consider that failing, quit a bit.
The reason why it fails mainly lies on process. We as software engineers don't do a good job of gather requirements, and controlling the customer. Because building software is a "mysterious" job to people outside of IT, it is difficult for them to understand why last minute changes are difficult. It's not like building a house and clearly showing them why it isn't possible to quickly add another door to the backside of the house made of brick.
Not only software projects go over budget, or take more than scheduled time to complete. Just open the newspaper and look at public projects like bridges.
But for a software project it is far more easy to cancel everything. If a bridge or building is half finished, there is no way back. Half the infrastructure is in place. It is very visible and it takes money to remove it.
For a software project you can press Shift-Delete and nobody notices.
Its just a very hard job to do an accurate cost analysis.
Using the FBI's Virtual Case File system it comes down to poor program management. The program had pre-9/11 requirements and post-9/11 expectations. Had government management done their job there would have been contract mods.
http://government.zdnet.com/?p=2518
"Because of an open-ended contract with few safeguards, SAIC reaped more than $100 million as the project became bigger and more complicated, even though its software never worked properly. The company continued to meet the bureau’s requests, accepting payments despite clear signs that the FBI’s approach to the project was badly flawed, according to people who were involved in the project or later reviewed it for the government."
Although $105,000,000 for 700,000 lines of code comes to $150 per line of code. Not too bad.
To truly gage whether a project is really successful, the original estimate / budget is probably a poor yardstick. Most engineers tend to underestimate how much time it will take because of political pressure, and they don't know how to estimate anyway. Many managers are poor planners because they want unrealistic deadlines to please their boss, they often don't understand what's involved, and plus it's something they can look at and understand and use as a stick in meetings, as opposed to actually helping solve problems. Practically, it helps businesses make rough projections of expense for budgeting purposes, at least it gives them something the go by.
A better measurement of project success would be - are the users happy? Does it help the business make money? How fast will the money gained help recover the cost of the system? These are harder to gauge than a simple plan, though.
In the end, I've find you're better off delivering on deadline, even if it means working some overtime. It sucks, but that is the reality of it.
As stated previously the vast majority of people involved in software development to not actually understand how
ask the right questions to learn about the problem
appreciate the users goal and judge expectations
understand technology available and the related Eco structure
most of team directly/indirectly involved are poorly skilled.
do not appreciate or know when they are wrong do they can take action.
Even with perfect requirements and related definitions too many developers don't know what they are doing.
Just look at the types of questions asked here. Would you go to a doctor that asked the same equivalent question. The scarey thing is that they ask and don't know how to learn or reason.
Different agendas
Management do not really understand what a developer does, how they produce code and the difficulties encountered. All they care about is the final product delivered on time. But for a developer they care about the technical aspects, well produced code in a solution they our proud of.
Deadlines
I've often heard developers say they wish they could produce better code, and that deadlines often push them into producing something that just works, rather than good code. When these deadlines are unreallistic the problems are exacerbated.
I think this thread managed to gather the biggest group of tallented unhappy software developers, engineers, project managers, etc. that it is possible to gather in one place.
I share point of view with most of the posts stuck here and I think they come out of a lot of suffering through seeing co-workers not doing a good job when job (programming) and success is the most important part of our lives.
http://www.clean-code-developer.de/
They have an incredible cause! and their philosophy, if taken ahead, could manage to create a new layer of heroes, as the main stream of developers and IT professionals is so ROT these days.
I'm working on something similar here in Brazil, 'cos I love our profession of bringing software alive both as PM and software developer (.NET) and I can't stand people who face programming as nothing else but they way out to make big money with almost no effort.
... sure I don't consider going overnight in front of the computer geniality. but the few who matter, have done it more than once.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
What are the main reasons personal projects (software apps etc) never get to the level of competing with your salary?
To me one big problem is "on-the-fly" feature expansion, with this problem, the end only gets further and further away!
For me, it's simple: I work 8 hours a day already. I spend a few more hours a day keeping current. I have a girlfriend, some local family and a decent circle of friends. I have (gasp) non-computer-related interests and hobbies. In other words, I have a life.
So ... Time. Time is not on my side. Would that it was ... My blog might be a bit more current if there were just two more hours in every day. :)
(Originally posted by John Rudy.)
If you want your hobby to become your job you have to acquire all the other skills you need to be in business. At the end of the day your pet project has to stand on its own two feet in the real world. At the same time you are enjoying the coding you need to get yourself a concrete plan to commercialise your activity.
Most hobby projects fail to make the big time for one of two reasons:
The idea is not commercially viable
The discipline necessary to commercialise the idea is missing
Just because you are a great technologist does not mean you'll be a great businessman. You may be, but the two are not necessarily linked. It is no weakness to consider partnering with someone who has no technical skills but a good network and some proven business acumen. Quite often people like that are looking for techies too so you might find a great partnership. That person can provide the structure and commercial discipline that you probably lack if feature creep is pushing your completion backwards.
I think the primary reason is the simple work overload that most developers experience. Most personal projects take place in the evening and weekends, and as excited as most of us get about our ideas for personal projects, after 40 hours (or more) of salaried programming, it's hard for "more work" to compete with watching a game while sipping a beer or spending quality time with the family.
Different skill sets are required to start and maintain a business than to develop software. Entrepreneurship skills can be learned, but not every has the skills to make it happen. A lot of times the skills it takes to get something started and off the ground are different than the skills it takes to finish it and polish it. For me, I know that I have the creativity to make software and find ways to solve problems, but I have little interest in finding funding for a business and marketing a product or service.
Assuming that you're a developer, it's most likely due to the fact that you do not know when, or are incapable of, stopping development and focusing on other things, like marketing and sales.
Time and Losing Interest, there is always a new tool or technology that can take your attention away from completing projects.
I'm not sure if I understand your question, but here are a few answers:
Adding "on-the-fly" features isn't necessarily a bad thing. In fact, it's the expected model of Web 2.0 and Web 3.0 projects. The key is to keep them very simple, only roll them out once they've been tested, and listen to your users. If you try to dump the kitchen sink in on the first release, it will most likely be ugly, confusing, and buggy.
Being a great programmer is only a part of it. You need business skills, marketing, knowledge of the user's needs and how to meet them, artistic/design skills, and a hell of a lot of luck.
Lot's of people have great ideas. Often different people have the same ideas. Most never get implemented. Of those that do, very few of them succeed. In some cases, revolutionary products took years to convince the buyers and users that they even wanted the product. Often the people or companies behind the first few iterations failed miserably and then a third or fourth person or company finally hit the market at the right time with a right product. Apple is great at both ends of this by the way - they not only innovate (first Mac, the Newton, etc.), but they also wait until the market need grows and they sense a place to pounce in and take advantage of it (the iPod, the Mac vs. Windows issues, etc.)
Most of these bullets apply as much to software as they do to widgets and services. The big advantage that software has is lower startup costs. Just like the saying "On the Internet, no one knows you're a dog" - "When looking at a web app, the user doesn't know if you are a multi-billion dollar company or a single guy sitting in your underwear in your parent's basement." If your software is good, that is...
I'd say one of the big reasons is that by nature, personal projects don't get as much attention as your job will.
I have a slew of personal/side projects I'm working on, but they get far less of my attention that my 'real' work does because, right now, that's what's paying the bills.
If I were to take a month off and work only on my personal stuff, it'd probably be pretty cool / worth money.
developers often design for themselves instead of for their customers
developers tend to put off releasing products until things are 'perfect' - and they never will be
Weakness of mind and spirit. Build a team around your product early.
Scope creep. Concentrate on selling what you have already got: "The customer can have any color he wants so long as it's black". Henry Ford
Small feature set. Leverage features of your product by what is already available on the market.
Not enough hours spent daily. Often achieving something might depend just on simple routine, putting your time in.
Deep down I think its a lack of belief in the project. If I believed in what I was doing I would not stop in completing the project.
Desire to build an ideal product
For example: There are various ways (algorithms) to get a particular task done. But, people wait to discover that one ideal solution. Even if there are multiple solutions for the same problem already available. That ideal solution is never found.
Procrastination
Your personal software projects don't compete with your salary for one reason.
What do you do for your salary? Whatever that is -- however much you may like or dislike it -- it more valuable than your software product.
"But my day job involves a lot of stupid time-wasting meetings." So? Clearly, someone will pay you more for wasting your time in meetings than for your software products.
"But my day job forces me to waste months in useless analysis and design documents and test plans that never even get used." So? Clearly, someone thinks this activity is more important than writing software.
"How can meetings or useless documents be more valuable than software?" I don't know, but look at your experience. Companies love to pay programmers relatively large amounts of money to hang around and waste time.
Companies don't love to pay for software.
Your personal projects don't compete with your salary because your time is more valuable than your products.
The biggest reason? Because if you can write it yourself and people like it, someone else can make an open source version with much better support than you can provide alone. Why not skip the middle man and release it as open source yourself? Sure, you miss out on the direct profit, but that looks very good come hiring time.