Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Being relatively new to the software industry I have come across a question of deadline enforcement:
Back in the idyllic age of academia, the deadline was the end of the semester and the penalty was a well defined 'F' (or local equivalent). Out here in the real world we need to make code our current and future peers can work with, I face the situation where deadline comes, deadline goes, and the project is still not finished.
Now what? On one extreme we could fire everyone involved, on the other we could richly reward everyone involved.
What actions have you seen applied as 'penalty' for missed deadline, and which of these eventually resulted in more-good-code?
What project-management responses caused the project to fail outright,
What responses restored working order and resulted in code that could be maintained afterward?
What responses resulted in more-bad-code?
Deadlines are part of a fundamentally wrong idea about how to do software development. People new to, or outside of, the software development industry do not understand this:
Software is done when it is done, no sooner and no later.
If a developer has a task and a week to do it, and it looks like it will take more than a week, nothing can be done to change that. No matter how much harder the developer works, no matter how many people are added to the task, it will still take as long as it takes (in fact adding people usually makes it take longer).
Instead, read up on the agile development process. Software should be developed iteratively, and each iteration should be based on the results of the previous iteration, not on external requirements imposed.
Edit based on the extensive comments below:
I would never argue that developers cannot be held to some kind of delivery expectation. My point is in response to the specific hypothesis which the asker posed - that the nature of software development in business is somehow analogous to schoolwork, or any other kind of work for that matter. I contend that it absolutely is not. "Deadline" implies much more than a simple delivery date. It is a fixed point by which a fixed amount of work must be completed. Software simply does not work that way. I wrote a few more paragraphs explaining why, but honestly if you don't already believe that, nothing I say is going to convince you.
If you are working on a software project and it is clear you will not be able to reach your deadline, what can you do to rectify that? The answer is well-known by now: practically nothing. You can't add more people. You can't "work faster". It's just not going to get done on time. You tell the stakeholders, everyone adjusts, and keep working (or not). What, then, did the original date mean?
Anyone who claims software development is analogous to bridge-building or homework or that impending deadlines can still be met if the developers would just get their shit together and work their asses off, are deeply confused about their own profession.
Your first reaction should not be what to do in response to the missed deadline, but to analyse why you missed the deadline. The response to missing the deadline would then follow naturally from that as a consequence of the reason.
For instance, if everyone involved didn't do their job, fire them.
But if they did their job, and more, then why was it still missed? Too much other activities done by the same people? Too big a scope for the deadline (ie. unrealistic deadline). Or ... etc.
The top reason for missing a deadline in my experience is that people aren't allowed to work 100% on the project at hand, and thus any estimates you might have, although accurate on their own, aren't really useful at all. That, plus unrealistic estimates and deadlines.
Developers should never be penalized for Management's mistakes.
It's like a parent punishing a child because the parent had a bad day.
Reasoning:
Deadlines are a fact of life. People want to know how long something will take. The best we can do is estimate/guess. It is the role of management to try to figure this magical, never correct guess. When they create a deadline, they need to use the right tools (experience, ASKING FOR HELP FROM DEVELOPERS, lawyers, hr, etc)
However....
The penalty for missing a deadline should not fall back on the workers. It is the management's fault for missing deadlines. They should have said no, should have scaled back the project or should have motivated the workers better.
In a construction crew, if you piss of the workers, you start a fight. In my company, if we miss deadlines, the management gets in trouble. Not the workers. It's the manager's job to control the project and what is done. The workers are only doing what they can. The manager's are in charge of assigning roles and tasks.
I'm not saying the quality of workers isn't a factor, but the management should KNOW that! It doesn't take a genius to know that a project isn't well thought through or nicely controlled. Ask anybody if their manager has any idea what's going on and you'll find the problem.
We stopped missing as many deadlines when the managers realized it was their fault for setting/agreeing to the deadlines.
</rant>
Re: The questions:
1.What actions have you seen applied as 'penalty' for missed deadline, and which of these actually made things 'better'?
Manager has less responsibility. This person does not get promoted or publicly thanked. Most likely this person will be moved to a "less-critical' project.
2.What project-management responses caused the project to fail outright, and what responses restored working order and resulted in code that could be maintained afterward?
feature creep: manager keeps adding more stuff in the list. <- fight this off with a List of tasks ordered by priority. When you add things to the list, compare their priority with the things around it. Make new things harder to be set as "top priority."
too many bugs in the code: Manager need to require tests (atleast critical) and automation. Builds need to be standard and automatic. Real users need to see the code before it is "finished."
un-readable code: Institute peer code reviews. If someone has dirty code, ask someone to "help" them with a project.
If you have the salesman problem, where the salesman promises features that doesn't exist/work: Management needs to step in and explain the problem to that salesman. Also, not giving that salesman public affirmation for a job well done sometimes helps this.
Rather than a penalty, how about realistic estimates and rewarding on-time releases?
Inspired by the comments to my response
Maybe the question should be "How do I make realistic estimates?" For me, I use FogBugz estimation history and completion date plots. These give me data points of how long I estimated a task to take and how long it actually took. This has helped guide me to give realistic release dates in the long-run (it didn't happen overnight). I find estimating timelines to be an interative process: I
design
estimate
develop
find an shortfall in the design & iterate.
Death. Clean and simple.
Depends on whether developers set deadlines on each modification request, or whether these are set for them by management.
In the latter case, unless all your developers are sitting and playing Halo 3 all day, a missed deadline is often an indication of a mistake on the side of management or the team leads. So firing everyone wouldn't solve the problem. It might make sense to introduce better indicators into your software process so you could see that the deadline would be missed long before it happens.
If your developers do give time estimates, then I would be very careful about rewarding and penalizing developers for meeting deadlines or missing them. The result of doing this could be that they would adjust their "fudge factor" in time estimation. They would give themselves too much extra time (to reap the rewards), which messes things up if they are good at estimation. Your goal should be to get them to give good and reliable estimates, not to change the way they work to meet these estimates.
It depends on if the deadline was possible in the first place, maybe it was a fault with the planning and estimation of how long it would have taken. Make sure you know why the deadline was missed before deciding punishment
Oh, man...
First of all, there are external deadlines and internal deadlines, and they should be different.
What happens with an internal deadline is the frequency of activity increases as the deadline approaches, reaches a peak at the deadline, and then falls off as the deadline recedes. So plan the external deadline to follow the internal deadline by a couple weeks at least.
Then, make sure the deadlines are realistic. Partly you do that by involving the developers in setting them, and in deciding what will be accomplished.
Finally, I've mostly been a developer, but once when I took a stab at management, I would never want to take the latest-and-greatest version into a conference or presentation. I would want to take a version that was at least a few weeks old and that I knew where the problems were and that I could be sure would not contain unpleasant surprises.
In his wonderful book about project management - "Deadline" - Tom DeMarco tells us a story, about project manager from a western-world is managing a project in some fictional post-communist eastern European wild country (wild is a really good term, because the citizens are a bit.. uncivilized).
One day PM discovers, that something went wrong, some part of his project dramatically missed the unrealistic schedule. Previous PM established penalty for missing deadline simply by hanging responsible person on a butcher’s hook, but as schedules were unrealistic, one man already missed deadline.
So the story tells us about a day, when western-style PM is presented with a responsible person, and he should send him to be hanged on butcher’s hook. PM, as most people do, is terrified of vision of sentencing someone to cruel death simply because some was never able to finish his project in time. And – by all means – hanging this poor man does not advance the project. Since this is a fiction novel about project management, and not about tortures, our hero cancels the penalty.
But there is some big issue behind this story about hanging someone: if you set a deadline, and establish some kind of penalty for missing this deadline, the day will come, you will probably have to actually punish someone. And will you do it? No matter what the punishment will be: hanging, bonus loss, firing, breaking the deal or some fee – you may have to punish someone. Will this penalty do some good to your project? You have to answer it by yourself.
So: do not establish a penalty for missing the deadline, you will not want to execute…
As others have mentioned, before talking about penalties, start with "how do we determine whether these deadlines are realistic"?
Or as my boss once said, "We'll be happy to work a plan when you give us a plan that works".
I still think that should be on a t-shirt.
Once you've reached the point at which people have blown the deadline, you have to ask yourself (A) what the natural consequences of that are and (B) how you can best complete the task and maintain some kind of movement towards the business objectives (even if you're not running a business).
Explicitly penalizing people for blowing the deadline is unlikely to help unless they believe that they've earned it. This will not happen if the deadline was unrealistic, if there were elements of the team that were the primary points of failure, if there were serious problems with requirements, or if the majority of the team involved believes that the above factors are true.
In one case I was on a team that blew a deadline on a small deliverable by over three months - and the original deliverable due date was three months from start! We misunderstood the requirements, didn't sufficiently talk to the customer, and underestimated the time involved. Management was not at all interested in assigning blame. This was partially because it would have been counterproductive to finishing the deliverable, partially because none of us were "problem employees", and partially because management knew that we were all highly-motivated to fix the problem and satisfy the customer. So we got it done, the customer was as happy as could be expected, and we moved on with our lives, with some valuable lessons on how to avoid the situation in the future.
So far in my career I haven't seen any real penalties for missing a deadline (and I've missed plenty). I imagine it's different for companies building software or games to be sold in stores where the company has made promises to the public.
But in the custom software development realm, it's so hard to accurately estimate how long a project is going to take. And often times this fact is reluctantly accepted by companies everywhere.
No penalty. "Deadlines" and estimating have been and continue to be one of the hardest and most challenging parts of software development.
It is ridiculous to impose penalties on developers for this issue.
While I have never seen any disciplinary action or firings I have seen lots of "mandatory" overtime and peer pressure to work longer hours.
I almost got fired as a manager for telling the team that reported to me NOT to come in on weekends and work late. I know those things are detrimental to the project and to morale.
Generally the "punishment" is in the form of making people feel guilty or anxious, but I am sure there are places that do more "official" things.
The world is full of idiots. Management is no exception.
I think the question it self demonstrates a misunderstanding of the role of management and project management.
There is, unfortunately, a common perception in the minds of many with the word Manager in their title that management means putting the heal to/kicking the butts of lazy workers. It fits with those that believe in Parkinson's Law as well.
It's not. It's about making it possible for works to do their jobs - be it being the communication channel between them and some other part of the organisation, getting them resources, or running interference (moving the furniture out of the way).
To wit, the PM should already know the project/task is going to miss it's deadline. They should be asking questions, and know what's going on. They have the power to either cut tasks or increase/rebalance the resources to get the job done (or say to the sponsor, if you don't give the resources, it ain't getting done on time). And as such, the penalty goes to the PM, whether it is nothing, tongue lashing, demotion, or termination.
Sometimes the delay is unavoidable. This is why we build in contingency time. Sometimes, it's a known risk; and so long as you have a backup plan - you are OK.
As for the responses, you have four parameters: Scope, Time, Money, and Quality
Scope - you can cut to make the deadline.
Time - is fixed. You might be able to get your staff pull a week or two at 60hrs, but your productivity will begin to suffer after that. And it also costs more money if you are paying them fairly.
Money - You can buy pieces from someone else to speed up the process. You could even hire more people, if the work is disjointed enough that you don't have to have a lot of communication with the existing staff - see Brook's Law
Quality - Idealistic fools claim you can never skimp on quality. But you can. You don't have add bugs (one form of anti-quality); but you can put less quality in. Do you code your function so it can handle unlimited length strings, or is 100 characters good enough for this version? Do you make it easy for the next upgrade to bolt on a new module, or do you weld it shut and worry about adding a plug-in module when you do the next version.
Not doing these things aggressively enough (when required) will surely lead you to a failure.
It's certainly not a cut-and-dry answer. Here are some things which I weigh and things I do/encourage in order to make sure things get done on time.
1.) Set priorities properly. Projects will always have various degrees of completion. It's not a binary "done"/"not done" switch. If the highest priority things are done first, it's easier to swallow. Ideally, you should quickly get to the point where it works, but it doesn't do everything we need it to do and it doesn't look pretty. Once there, it can be released if it absolutely needs to.
2.) I've found that the best way to handle it is to make the releases as small as possible. This makes the estimates more accurate. If your boss or "the market" dictates that your estimate is unacceptable, consider assigning more developers to this task if possible. Sometimes a task can't really be divided up easily, or there's only one person familiar with the code. If it's not a high priority just tell the powers that be that it's going to take longer. Setting reasonable goals and managing expectations is key.
3.) As for motivation, rewards, and punishment... there are many doctors who have written entire books on these subjects. In my experience, giving programmers something which is challenging and letting them have some freedom to do it their way is a good start. Listening is something managers need to do well in order to succeed. If the developer is seasoned, you should be able to just explain the problem and let the developer come up with the solution. If their solution isn't as good as what you had in mind, you can suggest it and go from there. Just dictating how to do something, even for new programmers, is seldom effective. Making the developers think about things will help them be able to solve problems on their own. This is related to delegating, as that only works if the developers can do the work on their own.
4.) Reduce turnover by paying people well if they're doing well. It usually costs much more to find good people. It takes time to get familiar with a large code base and the hiring process can also help avoid spending time on people who can't cut the mustard.
5.) Ask (don't demand) if a developer can stay late/work weekends. Only do this when it's something very critical (for example a security flaw which gives user access to data they ought not be able to access; a new law/regulation passes which you must comply; etc.). If they say no, don't hold it against them. It's likely not their fault that things didn't get done; and even if it is, it's reasonable that they made plans for time when they aren't expected to be at work. If they are willing to come in, make sure they know your sincere appreciation. Compensate them well for helping out when they aren't obligated to, buying lunch doesn't cost much and it's a very nice gesture. Don't make a habit of expecting people to work late/weekends unless it's part of their contact/agreement (or if they like doing so).
6.) Understand why things are running behind schedule. Did you promise something which wasn't possible (given the people available, quality expected and time allotted)? Did some other project come up and take up resources and the deadline wasn't adjusted? Was the code just harder to do than expected? Giving time estimates is difficult. You need to plan everything out, have experience and know how long each developer will take for the task. Compensate for unexpected problems which will likely arise and give the programmer a sooner deadline than your boss or the client. It's always OK to be done early. And if you're almost always done early or on time, that one time that you missed your deadline will be more understandable if you have an explanation of some sort.
7.) Remember, it usually boils down to time, quality and money. You can generally pick any two, but the third one will need to balance the equation. So if it needs to be done quickly and on a shoestring budget, you can expect the quality to suffer. If you need it done quickly and of high quality, expect to pay a lot of money, and so on.
8.) I'd say the #1 thing which works for me is listening. If your too busy barking orders then you might not even know about problems with the company. Now just because a developer says "the code sucks, the design is terrible and we need to re-write everything if we want to get anything done in a timely manner" doesn't mean that it'll happen. But if you hear comments like that and explain that we can't afford to do this or we'll get killed in the marketplace, it'd be way too expensive. And ask what can be done to make sure things don't get much/any worse. Ask if there's a way we can clean it up over time. Can we just (re-)write one class and build new stuff based on that? Can we slowly migrate to a new design one feature/segment/module at a time? You understand where they are coming from and vice versa, you can probably solve at least some of the issues. Just remember that compromising works both ways.
9.) Negative re-enforcement seems to result in higher turnover, which is costly. Having a bunch of people who aren't familiar with your code doesn't help deadlines either. Money is one motivator, but I've quit a higher paying job to go to one where I'm happier before, and I know I'm not alone there. Free food when the team does a good job isn't really that expensive. I'm not too keen on group activities since they're either cutting into an employees time, or taking away from work time. It works sometimes, but cutting into an employees personal time so they can hang out with co-workers instead of being with their friends isn't that great of a reward. Having everyone stop working is also expensive... so it just depends on the company size, culture, etc.
Hopefully that helps answer your question. The other answers in this thread are also good suggestions... design plays a big part in how quickly code will be written.
Once a project is late, there is not much 'management' (good, bad, well meaning or malicious) can do, that will not result in the project being even later
... the only exception possibly being the removal/avoidance of exterior distractions.
If you're missing your deadlines, fix your estimates.
Taken from a corporate development standpoint...
If the deadline came from someone other than the person performing the work, review the situation to determine the cause of the overrun. In these cases, it is often related to incomplete requirements, scope creep, poor management, etc. No punishment should be given for missing a deadline that the person never provided in the first place.
If the deadline was provided or agreed upon by the individual performing the work, then that person needs to explain the factors that led to the delay. In addition, this person should be reminded to notify their supervisor, project manager, or other responsible party as soon as they are aware that a deadline may be missed. This information should not come to light after the deadline has passed. If this occurs repeatedly, your company's disciplinary process should be followed. This may involve write-ups, suspensions, or termination.
People tend to take real ownership of deadlines when they are the ones setting them. When deadlines are placed on them without their input, deadlines tend to become meaningless to the person performing the work.
Your question is inherently flawed: it assumes that punishment is the best way to manage people. In general, that people don't respond well to punishment or threats of punishment; it brings out the worst behaviors, make the motivation external, and distracts from internal motivation. Rewards and bribes (threats of reward) are the other side of the same coin, and do no better.
These forces are built in to work for hire, however, so you'll never get the best creative work out of your programmers, but you don't have to make it worse by punishing them when they miss a deadline.
Instead, meditate on the creative process, the chaos of multiple people doing creative work, and what tools are effective in managing chaos.
To manage any chaotic system, do lots of measurement and be ready to change course quickly. In the case of programming:
Take the smallest steps possible. Don't "break the task in to small steps", as you'll waste a lot time planning steps that won't work out like you planned. Chaos, remember?
Pick the smallest steps that deliver the most value.
after a short period, reevaluate your plan based on what you've learned
deliver working software to actual, real customers as soon as possible, so they can tell you what you should really be doing.
You may recognize this as the thinking behind SCRUM.
Flogging
There are two possibilities:
The deadline was missed because someone didn't do their job.
The deadline was unrealistic.
Rather than thinking in terms of penalties, I would suggest doing a post-mortem to determine what went wrong and finding ways to improve the next deadline estimate.
You ask "what should the penalty be...". It would appear you are asking from the perspective of "inside the company".
In real life, the penalties are often swift and severe - loss of business, lawsuits, bad reputation in the industry. These are the REAL penalties imposed by clients who were promised something by a certain date that was not fulfilled.
Internally, you can often do whatever you like. But once you start involving paying clients, then managing those clients becomes a critical part of the overall job.
Penalties such as I described can often be avoided (or lessened) by "on top" communication with the client. If the client wants something added (so-called feature creep), then this should immediately be answered with the impact these changes will have on the project (costs more, delivered later, whatever). The client should be encouraged to triage all such requests against their deadlines and projected costs (i.e. let the client manage feature creep, not you).
If other things change the delivery time, then as soon as you know there will be slippage, you must inform the client. If done early, clients are remarkably willing to work with you. But if you don't say anything until it's too late, they are less likely to forgive... especially should they discover you knew a significant time earlier and didn't tell them.
Cheers,
-Richard
I've seen executives leave a company shortly after some deadlines were missed. This changed everything but didn't necessarily make things better or worse. I've seen some contractual obligations like clawbacks as a way to penalize someone for missing a deadline that I'm not sure how well they work.
When one completely changes what a project is supposed to do midway through the alloted time for the project that tends to cause the initial trajectory to no longer be valid and thus the project will fail because it likely will not meet the initial deadlines within budgets. Replanning the project into short increments of at most a few months is a response that I believe is a logical direction to take a project to get good results as a lot of project may have to accomodate changing requirements which can easily change deadlines, head count or time worked.
What should the penalty be for setting an unrealistically short development timeframe against all of the advice of the developers and their leads?
Coincidentally, this seems to happen almost as often as development teams missing ship dates.
This is not really a programming question, but more of the management question.
Missed deadlines are rarely developer's fault. As a developer you should try your best to do as good work as you can, but in the end everyone is capable of only so much. If developers put in honest effort and despite this the deadline was missed, it means that the deadline was unrealistic to begin with.
Dealing with deadlines is responsibility of managers. There are different approaches but none of them include "penalizing" developers for doing their job. An important thing to understand here is the so-called project management triangle. What it means is that software project can be good (i.e. meeting requirements, good quality), fast (meeting deadlines) and cheap (headcounts, tools). The trouble is that only 2 out of these 3 properties can be chosen.
So if management want something good and fast - it is not going to be cheap.
If management want something good and cheap - it won't be fast.
And finally if management want cheap and fast - guess what, it won't be any good.
So the correct response to missed deadline depends on the chosen scenario. Good and fast requires adding some extra help, better tools, investment in above-average developers and more.
Good and cheap by definition assumes that deadlines are going to be missed (Blizzard, makers of World Of Warcraft are good example of this approach)
And finally cheap and fast usually means cutting features and releasing with bugs.
The main goal of project management is to plan how an application is going to be built, in time. You should not start your project development if you don't have a schedule showing what you're going to be doing every single day the project will last.
This way, you can detect that you're going to be late, as long as you follow the project's evolution on a regular (weekly if not daily) basis. And the earlier you know it, the sooner you can act accordingly.
You usually have two options :
Catch up (by hiring additionnal workers, working more, or removing features).
Tell your customer that something went wrong (even better : what went wrong) and you're gonna need more time.
For the second option, I'm not meaning there won't ever be penalties. But from my personal experience, as long as the customer is informed in advance and offered solutions (preferably three : give more money for additional workers/remove features to save some time/accept the project being late), they'll be open to negociation. Which is always better than conflicts :)
Perhaps the better question is if deadlines are meaningful in the face of inaccurate estimates? Businesses do a lousy job of estimating software--that is a fact. Both management and developers play a part in this and neither one seems willing to own-up to their responsibility in this problem.
But to answer your specific questions:
1.What actions have you seen applied as 'penalty' for missed deadline, and
which of these eventually resulted in
more-good-code?
The 'penalty' I've seen for missed deadlines for managers and developers range from nothing, to promotion, to simple transfer. The most severe penalties I've personally witnessed was a manager "transferred" to a less important project and for the business-unit to lose a financial bonus.
The only time I have ever seen someone fired over a missed deadline was when the employee was already going to be fired--the deadline gave the business a legal reason to fire the employee.
2.What project-management responses caused the project to fail outright?
This is a whole separate discussion on its own... but there is some inherent bias in this question--project management is at fault.
The three top things I have personally seen PM's do that sabotage a project are (in order of severity):
Ignore data/recommendations/warnings from their technical staff.
Ask for estimates early in the development process. This results in estimates with an error-bar of 10x (it'll take one month, give or take ten months).
Reject/modify/demand software estimates so that they fit an arbitrary budget and schedule. This is not to say Developers should ignore business demands--but rather the business demands need to be set equally by Developers and non-Developers.
3.What responses restored working order and resulted in code that could
be maintained afterward?
I have yet to see a functional software development organization. So the fix is usually a lot of blood, sweat, and tears from a couple of heroic developers working with a highly-capable PM who knows how to defend against politics within the company (i.e. deflect BS from their staff).
4.What responses resulted in more-bad-code?
Yelling. Cursing. Insults. (Sadly, this still happens in some workplaces)
More "project management"--either by way of people, meetings, status reports.
Getting software estimates earlier in the process so "we can plan better." Estimates need to come later when your staff has more data and a better understanding of the problem.
Coddling the developers (it's not your fault, the manager screwed up).
Coddling the project managers (it's not your fault, the developers screwed up).
Adding additional, unqualified staff to the project.
I got fired for missing a deadline, I was 98% finished with the product, external forces and deadlines that are that firm don't allow software to be developed properly. Even I can admit I wrote some poor code under the circumstances, but I also wrote some good maintainable code as well. No consideration was given for feature creep, infact no technical specifications were detailed upfront and adaptation of functionality was required as limited and buggy versions of the software became available for managements review. I could have communicated better, but when I did communicate it was emphasized that deadlines are non-negotiable.
Two obvious questions come to mind when a deadline was missed:
Was the deadline feasible?
Did external factors impact performance?
Obviously, if someone presents you with a deadline that doesn't make sense then there shouldn't be any penalty for missing the deadline. Also, if someone misses a deadline because they were called up for jury duty that also shouldn't be held against them as well.
In the event those questions don't apply then the next thing to do is to figure out what went wrong. If you based your estimate for how long something would take, and thus the deadline, on the developers estimation of how long it would take them to write the code then perhaps they were too optimistic in their responses.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
In my organization we have to handle multiple projects with different technologies,like flex, iphone, .net, php etc. The problem is I know only java.
So if a developer says me that an issue will take 2 days to solve I really cant judge whether he is right or wrong.
How to handle this situation?
One more problem is that because I don't know a particular technology so its very tough for me to say that whether a particular thing is possible or not in that technology?
I prepare project plan, documents, contact with clients these things but have almost no controls on developers because my insufficient knowledge of all those technologies
What can I do about this?
If an estimate strikes you as a bit over the top, surely the developers should be able to explain what it is that will cause the delay. Especially if you know java - they should be able to explain it to a non-tech project manager as well (after all, the project manager might be required to explain to the customer).
Other than that; trust your developers. They probably don't underestimate badly enough to get you into a tight spot, but you should always give yourself some margin as well, when communicating with the customer. If they constantly over-estimate, you should notice that after a while.
This is from Scrum but it is applicable even if you don't do Scrum. In Scrum, only the developer is allowed to give an estimate of how long it would take to do something. Managers are not allowed to give, recommend or in any way modify that estimate. So first of all trust that estimate.
But.. most programmers are inherently over-confident. If a programmer says 2 days then it may take him 3 days to complete a task. The estimate does not reflect reality (at first).
The solution to this is to keep a record of the estimate. I usually write the task and estimate down on a piece of card and post it on a big whiteboard. If the task takes longer than the estimate then tactfully remind the developer and make a mental note. Next time, before making a new estimate drop subtle (or not so subtle) hints about missed deadlines or early completion. This way the developers will slowly learn to improve their estimates.
Be cordial in all this. The aim is to get accurate, reliable estimates - not to put pressure on people. The whole point of this is to train people to make better estimates. Trust me, having an accurate estimate is more important than missing a deadline. It is relatively easy to tell a client that a project will be delayed by 1 week if by the end of that week you can actually deliver the project. On the other hand repeatedly telling the client that the project will be completed "tomorrow" will quickly make the client lose trust in you.
A few other notes:
When I started this process it took around one month for most people to actually be able to give me accurate estimates. It's not that they were intentionally lying about previous estimates. It's just that people, especially programmers, don't actually know how to make good estimates without training.
Whenever developers come up with an estimate of more than 3 days I automatically ask them to break the task to smaller subtasks such that each subtask takes only 1 or 2 days to complete. This also automatically generates milestones that you can actually track to see if the task is going well or is stuck.
Explain this process to your boss and get his support. It is very hard (but not totally impossible) to do this if your boss keeps pressuring you because you will end up pressuring your developers. Your boss must understand that time estimates can only be made by people actually spending time doing the job.
you should probably trust your developer's estimates, yet expect that they won't always be 100% accurate, remember they are estimates. It might also be a good idea to use a process that has the acceptance built in that estimates are only estimates, and either doesn't require them (e.g. Kanban), or has features built in to adapt to the nature of estimates (e.g. Scrum).
PMs shouldn't really need to know that much about the technology, as that is what the developers are for, but I understand that is not always the case, particularly where the PM also has technical responsibilities.
So judging whether or not something is possible shouldn't be your responsibility alone, this type of evaluation should be delegated practically completely to the developers, at least regarding the technical considerations. You can still provide your evaluation where business, economic, and customer considerations reflect on the possibility or otherwise of some endeavor.
In short, leverage the developer's technical knowledge where it is needed.
Befriend your development team. Explain that your job is not to boss them around or tell them how to do something and how long it should take, but to help them co-ordinate and shield them from direct interaction with the clients.
Once you are in a situation of mutual trust, describe the needs of the client and rely on the estimates provided by your developers. They're the ones who have the knowledge anyway.
Should a client ask you for an estimate on the spot, answer that it's impossible to give an exact figure without putting some thought and expertise into it. If they insist, answer a large-ish figure (at least what it would take you to do that same thing in a language you know) and tell them that you will provide the actual numbers shortly.
Maybe your process could do with some help.. from your comments above i saw:
Step 1 -the PM has to communicate with client to build the project
plan...
Step 2- build the mock ups---
Step 3--get approval of those two documents from client and management
....
Step 4-- project starts
Where do the developers come in and estimate? If you're asking them at Step 4, it's too late as you've committed to some sort of schedule with your client and management.
To ensure everyone's expectations are correct, take along a trusted developer to Step 1. Before you present a plan just ask the dev; "how long could we build this with a team made up of n of our developers?"
This has a few advantages:
Better estimates
Sets more accurate expectations with both client and dev team
Increase commitment to those estimates from the dev team
1 and 2 should be obvious, but item 3 is important as people rarely like working to estimates set by someone else. While you may not be able to use the whole dev team in your estimating, use a trusted advisor.
I also do manage some small projects, some COBOL, some MS, mostly java.
I only really know java, and my skills need updating too.
We use a scope estimating tool, really just an excel sheet with fields the dev fills in to assess the impact. This helps the developers break down the problem into smaller pieces, it forces them to really think through the steps that need to get done and this gives us a more accurate estimate. Whatever helps the developer, helps you. :)
And then when we are done we compare the estimate vs actual and baseline it for future references. We define good estimates as having a variance of 10% or less.
Don't just blindly trust the developer's estimates - you don't know where he's basing that from, no one could ever know 100% of the technology/framework or data model or business rules or such. Know what he has factored in the estimates.
I believe trust is earned. If a developer has given bad estimates before, would you trust the estimates he'll be submitting next is adequate? I definitely won't, unless I see an improvement.
I've been a developer, and I have made quite a few bad estimates too. (Hhmm maybe that's why I'm less of a developer now and more of a manager.. )
The only way to get an accurate estimate is to break the task down into small units of work. The smaller and more focused the units of work, the more accurate the task estimate.
Getting the team together to discuss exactly what each task will involve is a great way to peer-review the proposed technical solution. Often a better technical solution is hatched by the team through detailed discussion of the task. This is a great use of time, and you'll easily recover the design-peer-review time once development starts.
If you sit in on these team discussions then you will start to understand the technology a bit better, and you will start to trust your programmers more as you understand the estimating challenges from their perspective. It is also a great way to cross-train the team, as you'll often hear the phrase "I didn't know you could do that" as developers share their knowledge. Make sure the team reviews are ego-free and friendly.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
Lately I've been feeling like I'm being pulled in different directions. In my company there are a lot of forces demanding my time and I'm having a hard time deciding which direction to focus my energies.
I have the choice of several different coding projects, some of which could demand a lot more work over time, and present unknown challenges.
How do you decide what to work on next in the big picture? Grease the squeakier wheels? Low hanging fruit? (translation: easier projects)
Do you have a system for determining and reaching your goals?
We've used a value graph to identify projects based on value vs. effort, as a part of lean.
A few of the things I ask that affect what I work on :
Are any of my tasks holding up other developers or keeping other people from being able to do their work? If so, it'll probably get done first.
Do I have any tasks with deadlines coming up? If so, it might be the next candidate to work on, unless I can justify something else being worth the task slipping schedule.
Will work on any of the tasks affect (make obsolete / make easier / make more difficult) any of the other tasks I have? If so, it might get moved up.
Is there a good chance that any of the tasks will change (requirements still not concrete / other tasks out there that might affect it), making time spent on it now likely wasted? If so, it gets moved down.
Are there any things that just really bug me, that I think I can get done before anyone notices that I'm not working on whatever my boss thinks is the highest priority. (which is justified because it'll distract me from thinking about the other things).
Are there any tasks I should work on while things are still fresh in my mind?
... other than that, we go with a combination of whatever sort of fixed deadlines I have, what tasks I have that might be holding up other people, what the boss wants first (I like this boss, and I only have one giving me tasks ... in the past, I'd've answered differently), which ones seem more interesting, which ones I can get done quickly to just get them off the list, etc.
There have been times when I've had more than one manager, and I just had to put everything on a list on a whiteboard, and told them to number them (17 items, which kept growing). Management balked, but I was sick of getting bitched out at meetings week after week with stuff not being done, and having to go through the list of every 'emergency' task I was given. (and being told that any manager in the department was allowed to task me in the case of an emergency ... which was something like 30 people ... and being bitched at when I asked who got to decide if it was an emergency or not.)
When I have the choice: whatever seems to be the biggest challenge with the most fun attached.
Fun + challenge = rapid learning, to me.
And sometimes that takes me away from the technical stuff - people can be a fun challenge too.
I tend to weigh 4 items based on things when deciding what to work on next:
Is this item a requirement for something else?
Can I work on this item yet (ie, am I waiting on something for it)?
How fast/easily can I get this item done?
How interesting do I find the work required for this item?
I my current team (working on a variety of Business Intelligence software projects), we've recently started adopting a variant of classical "agile" project planning and estimating -- everybody seems to be pretty happy with it so far, including us (developers of different levels of experience), the product managers (highly technical people, typically also with some development experience, but mostly interested in the business side of things), management (pretty technical at the level we report to, but also less-technical, more-businessy directors and VPs), and other stakeholders (users and would-be users of our software). But, of course, it is early times, and we'll adjust as we go along. (In the past few years I used other variants of this in very different application areas, such as cluster management software; but I've often also used more ad-hoc, less-structured approaches).
The outline is as follows. At each iteration (we're currently on 2-week iteration cycles), the PMs choose some "elementary units of business value" that they might like to get from projects in our area -- a typical unit would be one feature, a bug fix, some optimization aspect, etc. In a small meeting with tech leads and one or two senior engineers, each unit is decomposed into engineering tasks (and dependencies among tasks are identified). In a larger whole-team meeting, the relative "cost" of each task (how much time, roughly, it will take to perform that task relative to other tasks) is collectively assessed (we're using completely abstract units of effort we call "points", though I've seen other teams use less abstract units such as "ideal engineering-days"). The costs assessed include unit testing and technical documentation.
The tasks, each with its assessed cost, go on what's called "the backlog" for the team, together with "internal restructuring" tasks (typically refactorings that will deliver no new user-observable plus, but will make further development and maintenance more productive), also with assessed costs and a summary of expected benefits (which must be expressed in ways understandable to the PMs -- fortunately, as I said, ours are highly technical people). A refactoring may also, by engineering team consensus, be deemed a prerequisite of certain business-requested tasks (e.g. "makes no sense to work further on component X until class Y, too large, is properly split, which will take N points").
The PMs now get to order the tasks in the backlog in any way they prefer, based on the business value that completing the units those tasks make up would deliver, subject to the dependency constraints. They have a good idea of roughly how many "points" the team may accomplish in a 2-week iteration (our "velocity") based on past results, so they try to make sure some business-valuable release can be performed at the end of the iteration (as opposed to having many business-valuable thingies "in flight"... but none completed and deliverable to stakeholders yet!-).
The team then devotes about 80% of its time and effort to tackling the top-priority tasks as designed by the PMs (including pair programming some of the time, for particularly urgent tasks or for situations where one team member needs to learn more about some technology or some part of the codebase, and another team member who's an expert in those is available for pairing up with them for a while). The priority order is an important indication, but it's not entirey rigid (e.g. if the top task requires extensive work in Java, and the second one requires it in Python, I may well pick the second one, as my relative productivity will be enormously higher that way -- and vice versa for a team member who's a Java guru, etc etc).
"Priority 0" aka "Code Red" issues may arise at any time, and, if they do, by definition they will take priority over any other task (and be accounted for only retroactively in the planning, to make sure velocity is assessed properly). But, as we do a pretty good job with testing, release engineering, and other quality-assurance practices, those emergencies are fortunately few and far between.
This, plus other "mandatory" ways for engineers to spend their time (training courses, all-hands meetings, quarterly performance self-evaluations and peer reviews, etc), is supposed to account for about 80% of engineers' time -- the remaining 20% being time each engineer should devote to "something completely different" ("blue-sky" exploratory projects, "egineering community" efforts, open-source contributions, etc, etc), not directly related to the projects at hand. Nobody actually measures the hours precisely, but that's still a useful guideline (I keep thinking of ways to make measurement easy and painless that I could implement in my 20% time, to help me allocate time and effort more precisely, but I haven't actually gotten any round tuits yet;-).
Easy. I ask my boss.
High value + Low risk.
Only go on high value + something with higher risk if you already have a track record in the company / credibility.
Easy: What is the highest and best use of my time.
If I am involved in a project and that isn't the answer to that question, I ask myself why I'm working on it and how soon can I finish it.
:) On Job i left this decision n my Project Leader and Team Leader, As they know better
"What is project priority"
At home, i do where i see fun, learning and challenge
When I have a choice about which job to start next I try to find a balance between two things: quick, easy fixes that are highly visible (e.g. fix the non-critical bug that a user has been complaining about) and taking on a project where I can use something I've been learning about. I find if I alternate between those types of jobs I can keep myself and my co-workers happy.
I'd look at what has the highest priority from management's perspective for an intial prioritizing of upcoming projects. If they are all priority 1 projects, then there are a few other factors that may help my decision:
Do I see how valuable the project will be to the organization? Is this the type of thing that really helps with the competitive advantage that we have?
Does there seem to be a buildup of projects of a specific size? For example, are lots of little projects being ignored for the few really big ones? If so, I may take some of the little ones that may be seen as quick wins that may help my team look good.
Do any of these projects use my strengths? This can be a bit hard to determine but it could help a lot with motivation, at least using the Marcus Buckingham interpretation of a strength.
What teams and structures are in place for the other projects? I don't think I'd want to join a project that looks like a massive train wreck about to happen. Is there enough structure so that I won't go off and do my own thing that may hurt the project's chances of success? Do I believe I could handle working with X using methodology Y and technology Z?
Those are a few of how I'd look at making the decision, along with talking to my manager as part of this is his job, right?
You should ask yourself a question. Are you pursuing a general IT career path which may or may not include your current company, or, do you intend to have a long career with your current employer?
If you intend to have a succesful IT career moving around differnet employers then, sadly, the most succesful strategy is "buzzword collecting". Identify the current/next big thing and try to get it on your CV. e.g. FInd a trivial AJAX with SOA back end project which may never go to production, this will enhance your value to future employers even if the project had little value to your current temployer.
If you plan on a long career with your current employer, the most succesful strategey would be to align your goals with the business. For instance the most critical project for the business may be upgrading an old unsexy VB/Oracle stock control package to include an ancient EDIFACT interface with a new supplier. If you are seen as a key player in the success of such a project you will rank very highly (and rightly so) in your employers esteem, and your opinions and advice will be taken seriously.
Since you didn't specify whether you talk from developer or manager perspective, I'll try to cover both.
Providing a framework for prioritisation of efforts is a management’s direct job. The immediate day-to-day prioritisation may stay with management or be handed over to developers.
The decision who should work on what and when in average company is likely to be perceived as a matter of power, control and prestige by both groups and one who makes the most prioritisation decisions as clearly more important player.
In shrewd companies, however, it is well understood that decisions have several interesting properties:
Each takes time and effort to make, which is diverted from doing the actual work.
Every decision is a trade off
To make a good trade off one who makes the decision needs all the right information in her or his disposal
Subsequently, management doesn’t have all the information to make every decision and nor they likely to have right information to make a good trade off in each case, but developers cannot spend their time doing hundreds of prioritisation decisions per day instead of producing software, nor do all necessary co-ordination.
Hence the solution is for management to create a simple framework for task assessment and prioritisation and hand it to developers who will quickly apply it on case by case basis, filling the gaps. In management lingo such framework is called strategy; it saves time by removing repetitive redundant decision making, gives focus and consistency to the efforts, and provides direction. It should be detailed enough to remove the burden of re-assessing the situation each time, but loose enough to allow developers to make right choices when it matters.
The framework itself may give very straightforward rules for making decisions or, alternatively, provide some analytical methods such as Pareto, SWOT, Cost Benefit, Expected Return analysis or Porter Five Forces etc. However, it is worth keeping the rules simple, unambiguous and as straightforward as possible.
Joel Spolsky made available to the world several very good internal software strategy documents written in plain English. Not all documents are directly to do with developing software (showing that it viable actually to have a different unrelated decision frameworks for various aspects of the company life). Also since the documents are several years apart it’s actually possible to see how these frameworks kept changing to suit the situation:
Fog Creek Compensation
Our .NET Strategy
Set Your Priorities
Fruity treats, customization, and supersonics: FogBugz 7 is here
If you're intrested in choosing what things to work on from a personal point of view one of the best advices around in my opinion is the one given by Paul Graham in his essay "What You'll Wish You'd Known".
Fundamentally as software developers we are business enablers. Your priorities should be in tandem with business priorities and be pragmatic between quick wins and larger strategic initiatives. Effort and Priority make an excellent matrix in which to score projects taking the least effort/highest priority first.
From the tone of your question it sounds like business priorities are either unclear or there is conflicting direction between stakeholders. This is the place to start and it will make your decisions much easier once it is resolved.
You really need to discuss this with the business because only they can tell you what has the highest value to them. After that, I would go after the items that carry the most risk because if something is going to cause a schedule to slip it's best to know early rather than late.
If you're having trouble with what the business priorities - usually caused by being on multiple projects with different stakeholders who all think that their project is the most important - you can try getting both stakeholders in a room to discuss which project is higher priority. Or you could delegate that negotiation to your manager as that is actually his job.
I tend to work on multiple projects at one time, so I will work on a harder project, make some headway, and when I get stuck and need to think about how to do the next part, I will go to some low-hanging fruit, so that I can continue to make headway, as I give my subconscious time to work on the harder problem.
But, it really depends on your priorities. I have never been good at just trying to impress people, so I just quietly go about trying to get work done.
If we're talking in a work environment, i go through and just prioritise things - what is mission critical, what is urgent, and then anything else just gets put into the list and it gets done in the order it comes in.
As for picking the next big project at work, i like to do what offers the greatest challenge. I had been working as a developer for a year and i had the opportunity to do some work for a very large company working with some security experts and doing things i'd never done. So i chose that, and it looks great on my resume.
In terms of personal development work (not as in self help), again i'll offer something that challenges me. It's got to be something that i haven't done before. It doesn't matter if someone else has done it - i haven't, and i can learn from that.
In the end, it all comes down to what value it holds for you, and what value it holds for the client. Luckily, i've got a few years of sales experience under my belt as well, so i can easily sell the products i need to to clients.
If your problem is one of procrastination, then perhaps you need to focus on getting rid of those jobs that you fear tackling most - or at least making some forward progress on them to reduce the stress of considering how far behind you are.
This book, by Mark Forster, provides some good tips.
Failing that, you might want to produce an iteration plan. Let everyone vote for jobs - whatever gets the most gets scheduled right away. that way every stakeholder, including yourself gets some input into scheduling.
I would ask the boss, if they wont make the decision then I would go for the project which I felt was going to be best for the company, in profit and moral of the team.
If I was torn between too projects i would go for the one that sounds like I can develop my skills more and be interested in most.
If a project sounds exciting I become more driven and determined too :)
Given the nature of your question, I'm assuming this is all work that somebody thinks you should be doing, but there clearly isn't enough time to do it all. Therefore, you are just looking for priority knowing full well that some items are likely not to get done.
Impact/Risk if the item isn't done.
Visibility - Does anybody else really care about this task
Alignment with department goals - weed out things that really aren't your job
Alignment with company goals - weed out things that aren't important to your company's business.
Enjoyment factor
Alignment with career goals - Many people would rank this item significantly higher. Depends how important your career is versus what you do today. I've rank today's enjoyment a bit higher than long term career goals. Some projects may be horrible, but they can move along your career.
I guess it depends on how much is on the list. If there's a lot of low hanging fruit that's been on the list for a while, it may be worth while to take some time and clean some of it off. That way, there would be less demands on the available time and potentially more time or incentive available to work on the big projects.
Plus it can be cathartic to be able to cross a bunch of stuff off the list.
I will usually start working on a larger project first. Then when I feel I need to step away from it for a bit, usually so I can approach it with a clear mind later, I try to kick out some of the quick one-off tasks or simple projects.
I know that isn't very descriptive, but calling the occasional audible for distraction seems to work out well for me when tackling a big project list.
I tend to look at project from a learner prospective. I tend to choose a project that will help me learn something new, I also look for "cool" and intersting as well.
On tthe other hand you can choose your next project according to where it would lead you. Ask yourself if you have a career goal that project X will help you achieve. Perhaps a high profile project is better then intersting - at least for a short while.
One way to defide is to define several key pints that matter to you (i.e. new technology, intersting etc.) and try to rate each opportunity and see which gets the higher score.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I'm time+cost estimating a semi-complex software solution, that hasn't got specific requirements in about 75% of features. I would still like to make as good estimate as possible, by getting additional data from the client. There will still be parts that may end up not being able to develop, since there's too many dependencies with other products/technologies and lack of definition. I also have a very tight schedule to produce this estimate.
There will also be other contenders on this project. Client expects a price+duration (and probably also features) and I know everyone will be off. I know that's impossible, but tell that to marketing people. Another problem is that I'm talking to middle-man and not directly to the client. I can get confidence with middle-man only, but not with the deciding client. Which is a different problem altogether.
What disclaimer/info can I put in my price plan/contract not to kill team with this project, so when project starts slipping (in terms of cost/time/features) we will be covered with some sort of payment. I would of course like to be paid by sprints or releases with relation to time, but I doubt client could be convinced in this. I'm sure we can finish this product before deadline and also create a great product, but how can I convice client to believe me?
Question
What can I do to get this project and avoid death march situation at the same time?
Any suggestion welcome!
EDIT: Outcome
In the end we (me and my co-worker) convinced the client we need at least a week to evaluate the product. So we did. We also pushed (and got) a slot for a few hours long meeting with the client to clarify any outstanding requirements' questions. So we did. Meeting was done after we made the first estimate draft, so we were sure we have all the questions to point out specifics that were either completely misunderstood or too vague to estimate. I hope we get the project, because it would mean 8 months of full time work for us, plus a reasonable pay. We'll know in about a week and a half.
Of course I also pointed out that the way we'll be delivering this product will get them exactly where they want to be with a product that will actually be what they wanted. And also that we only commit to price and time, but not functionality, because it is and will be subject to change. I think we made a good enough impression.
In this economic environment, there are a lot of companies competing for a little work. Someone is bound to give them a very sweet bid that will
Not be able to deliver on,
Kill their team with, or
Both.
When they can't deliver at the agreed price, they will start to cut down on the quality in order to deliver something and get paid.
Your challenge is to present that fact to your prospect in a professional manner, and convince them that you will work very hard to deliver at a reasonable cost, but also to deliver exactly what they need. The fact that you're going back for more detail, and the method you approach the project with (agile... but be sure and explain the business benefit to them) helps ensure that they will end up with what they really need.
Remember, they want to get the software delivered that they need at the lowest possible price.
Convince them that you will deliver exactly to their needs, and that your price is reasonable.
Welcome to the world of fixed priced development services :-)
Techniques for to win this project and avoid death march situation at the same time:
Don't underbid a project. Bid for what you think the project will take and add some percentage for things likely to go wrong.
If you are missing 75% of the detail, odds are the project will be significantly different than you currently expect. Document some reasonable detail assumptions within the outline of the defined work. When the project actually starts and the details don't match the assumption, you have an opportunity to negotiate the costs for the changes. At that time, you may also be in a better position to know how much you are over/under and attempt to compensate with this quote.
Your goal in an SOW (statement of work) should be to define enough details so that it gives you an opportunity to renegotiate the cost of changes when you know more about the project. Write these as positives, as much as possible. Note, it is unlikely that people that actually understand the project will read or understand the SOW...I base this on the point that you are given few details to quote. This means it isn't a consultative sale and neither party is really focused on building the 'right' solution.
If you can get a contract as T&M (time and materials) great. I doubt you'll get it or unable to get it without some restrictions that essentially defeat the purpose of a T&M. Your potential customers look at this as them accepting all of the risks around your abilities.
Hopefully, you aren't the first at your company to do this. Find out, historically, how projects have been and the typical result rates. Many software development groups charge an hourly rate that is significantly higher than cost...but their quotes tend to be lower and not actual hours. Customers often will argue more about the hours/days than the actual quote. Enterprises tend to be used to paying high hourly rates.
Figure out your department's expected margin (profit you need to gain from the job). This may help you to understand how much of a 'death march' you may face when your project slips.
In the SOW specify the level of detail that will be required in a specification before you begin work. While Agile and other customer focused processes take an approach that oriented at finding the best solution, they aren't designed to keep costs under control in a fixed bid environment. You will need to take a waterfall approach to requirements and then build in an agile fashion so that you can adjust along the way. The specification, like the SOW, will give you an opportunity to bill for changes. While the customer won't like this, it will put the burden and risks associated with requirements on them and not your team.
Note, to be successful with these negotiations, you needs a supportive management, sales and project management team. If you don't have that, you are bound to always be on 'death marches.' Even if you forgo quality, process, testing and other items, you'll find there's never enough time for a project.
EDIT:
Addressing the middle-men situation. I think the best course of action would be to submit a list of risks along with your bid as a courtesy to the customer. Kind of like giving them a heads-up on what their project limitations are. This will cost you some work up front but I think it could help you win the project.
you have two options
make a best-guess and double or triple your estimate (your competition is probably doing the same thing.)
explain to the customer that you can't bid work like this, and tell him that everyone else that gives him a fixed estimate is probably not being completely truthful.
At the end of the day, if you can't make money on the work, the there is not point in trying for it.
Personally, I prefer the latter, up-front and honest communication with your customers will take you farther than any bid tricks ever will.
A few things I'd say you should think about:
Assumptions: There's no one disclaimer you can add but you need to fill the gaps in the requirements with sensible assumptions and document them. Nothing major or scary, just a section in your spec/bid with a list of bullet points saying what you assumed to be true which was missing (e.g. users details will be pulled using LDAP and no admin screens will be written to cover user admin).
This gives you clarity in estimating as you now have a full scope to work from, but it also means that if the client comes back with things which are wildly different you have a fair basis to start talking about raising change requests and varying the cost. Alternatively they may come back during the negotiations saying this assumption or that one isn't true and you have more information.
Out of Scope: A specific case of assumptions - list things which you aren't including (e.g. No integration will exist with system X). Again this allows you to have a full scope and a reasonable case for potentially varying cost at a later stage.
Assumptions and out of scope are particularly applicable when things are mentioned in passing but not really followed up, or for things which they say could wait for a second phase. These are often the things the client will believe are being done as part of the main project but the project team don't.
Hopefully the thoroughness and insight from the assumptions and scope you define will help inspire confidence with the ultimate client too.
Contingency: A tricky one but you should add contingency in two ways:
(1) for specific risks. For things which might mean something takes longer than you've estimated then put in an amount to cover that weighted by the chance of it happening. Add all these up and that's your risk contingency.
(2) Shit happens contingency - unpredictable shit happens on IT projects. Add between 10% and 20% to cover this.
Whether you hide contingency from your commercial people and the client or not depends on your relationship but if it gets removed they need to understand what that means (essentially you WILL over run).
Understand the relationship between effort and cost: As a technologist your role is to provide an estimate of the effort based on the information you have. You need to then communicate that with assumptions, level of contingency and so on to your commercial team who can convert it into a monetary value. The thing to be clear with them on is that if they want to drop the cost that doesn't change the effort.
There are loads of good reasons for writing down the cost to the client (to build a relationship, because you'll end up with stuff you can reuse later and so on) but people need to understand that unless the scope changes the effort stays the same - the reduction comes out of the profit.
i have a blog article which may have a few tips in it for you:
http://pm4web.blogspot.com/2009/06/surviving-under-resourced-project.html
one of the other posters here has a good point to. there will always be someone who will offer a lower price to get the work. and the developer will suffer for it later (i.e. having to do a lot of free work to satisfy the client).
some clients need to have this experience before it clicks that you cant do IT projects on the cheap without paying some kind of price.
LM
Go for realism. Avoid promising too much, then make a point of it.
A lot of customers out there have been burnt by unrealistic offerers who fail to deliver as promised.
Emphasize the need for a specifications sprint. Convey a focus on core functionality and commitment to deliver rather than a feature bonanza. Offer a primary development phase to deliver core functionality.
Communicate the power and safety of the agile approach. Credit the customer with the ability to see good sense.
In short: Strive to come across as realistic and serious (more so than your competitors). The most important thing for any serious customer in the end is not the price, but a confidence that the product will be delivered on time and budget.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
As long as there are software projects, the world is wondering why they fail so often.
I would like to know if there is a list or something equivalent which shows how many software projects fail today. Would be nice if there would be a comparison over the last 20 - 30 years.
You can also add your top reason why a software project fails. Mine is "Requirements are poor or not even existing." which includes also "No (real) customer / user involved".
EDIT: It is nearly impossible to clearly define the term "fail". Let's say that fail means: The project was more than 10% over budget and time.
In my opinion the 10% + / - is a good range for an offer / tender.
EDIT: Until now (Feb 11) it seems that most posters agree that a fail of the project is basically a failure of the project management (whatever fail means). But IMHO it comes out, that most developers are not happy with this situation. Perhaps because not the manager get penalized when a project was not successful, but the lazy, incompetent developer teams?
When I read the posts I can also hear-out that there is a big "gap" between the developer side and the managment side. The expectations (perhaps also the requirements) seem to be so different, that a project cannot be successful in the end (over time / budget; users are not happy; not all first-prio features implemented; too many bugs because developers were forced to implement in too short timeframes ...)
I',m asking myself: How can we improve it? Or do we have the possibility to improve it? Everybody seems to be unsatisfied with the way it goes now. Can we close the gap between these two worlds? Should we (the developers) go on strike and fight for "high quality reqiurements" and "realistic / iteration based time shedules"?
EDIT: Ralph Westphal and Stefan Lieser have founded a new "community" called: Clean-Code-Developer. The aim of the group is to bring more professionalism into software engineering. Independently if a developer has a degree or tons of years of experience you can be part of this movement.
Clean Code Developers live principles
like SOLID every day. A professional
developer is the biggest reviewer of
his own work. And he has an internal
value system which helps him to improve and become better.
Check it out on: Clean Code Developer
EDIT: Our company is doing at the moment a thing called "Application Development and Maintenance Benchmarking". This is a service offered by IBM to get a feedback from someone external on your software engineering process quality etc. When we get the results, I will tell you more about it.
Bad management.
Projects are not successes or failures based on some underlying feature of the project, but on whether they fulfill the needs of the users. (They can fail altogether, in which case there was a gross misstatement of what was possible.) It is mostly in the process of evaluating the feasibility and cost-benefit ratio of the project, and establishing goals, that software projects tend to fail or succeed.
There's a disconnect between people who deal with facts and things (like programmers) and people who deal with other people (like sales types and managers). To a programmer, the facts are the facts, and have to be dealt with. To a sales person, the facts are what other people think, and are changeable.
There's also differences between tangible and intangible facts. Nobody thinks that workers could build a large bridge in a month if they were really motivated; they can see all the steel and concrete and other stuff that has to be moved and fixed into position. Software is much less tangible, and lacks the physical restrictions: while it is not even theoretically possible to build the bridge within a month, it is conceivable that a team could create a large project within a month, as "all" they have to do is get everything right the first time. It is physically possible to type thousands of lines of code a day; it's just that the chance that they're usable as is is so close to zero it doesn't matter. The actual productivity of a top developer is actually pretty unimpressive in word count, compared to (say) the productivity of a journalist.
Therefore, those who are used to flexible facts don't have the imposing physical limits to remind them that things can be pushed only so far, no appreciation for what programming actually requires, and no good feel for how much productivity is realistically possible. Moreover, they know how to get their way in negotiations, much more than the average developer, so in negotiations about what's possible they tend to assume more than they can, in the real world, get.
In the meantime, software development is inherently fuzzy, because producing the physical product is trivial. I can produce a copy of software quickly and cheaply, once it's been developed. Software development is design work, pure and simple. Anything corresponding to manufacturing is ruthlessly eliminated with such things as compilers and wizards and code generation. The developer, faced with the manager who wants the impossible, finds it hard to say the impossible is actually impossible, because there's no way to say it's actually impossible. Given facts that are unknown enough to feel flexible, the person with strong negotiating skills and determination will typically get the answer he or she wants.
Given this disconnect, one might ask whose responsibility it is to bridge it. The answer is, in my opinion, clear. The responsibility for understanding how different people think belongs to the people who specialize in dealing with other people. The responsibility for coordinating different types of people belongs to the people whose job it is to coordinate these things. Therefore, managers.
Managers who do understand software development and developers, and can deal well with other managers, will do well, and their projects will generally succeed. There are still far too many of the other type in the world.
Not a direct answer, but I found the Virtual Case File to be a fascinating case study on how a big government-backed well-funded project can still tank.
You can also add your top reason why a
software project fails.
Another IEEE Spectrum Online article "Why Software Fails" examines this very question. It summarizes the major points as follows:
Unrealistic or unarticulated project goals
Inaccurate estimates of needed resources
Badly defined system requirements
Poor reporting of the project's status
Unmanaged risks
Poor communication among customers, developers, and users
Use of immature technology
Inability to handle the project's complexity
Sloppy development practices
Poor project management
Stakeholder politics
Commercial pressures
Poor planning.
Hofstadter's Law
It always takes longer than you expect, even when you take Hofstadter's Law into account.
Mismanagement.
SW project get started by throwing developers against a perceived problem. Business requirements crystallize as the project progresses. New functionality gets added while deadlines stay put. More developers are thrown in. Original project members quit or get fired. By this point too much time, money and resources is invested in the project so it cannot be canceled. As the deadline passes the project is declared finished and successful despite the obvious lack of finished product.
Come to think of it - I've jet to see a SW project fail...
Honestly, I think its because most programmers are not very good at what they do(and I don't mean just cranking out code). People on stackoverflow are probably the exception. I don't know about the rest of you but as a consultant/contract programmer I have worked in or around many places, and the ratio of mediocre or poor programmers to good ones is about 10 to 1.
One of my strengths has always been estimating accurately and then delivering on time and on or under budget - I always aim for coming in 10% under cost and on time. Then I like to tell my client that because I got things done for less $$ than expected, which of the "extras" would you like to add in?
Even a perfectly functioning product that is late and/or over budget will be considered a failure by many business managers. Programmers often focus on just the technical aspects of what they do, with little regard for the cost or deadline. You really need to do all three well for it to be deemed successful project. There are many other programmers that could code circles around me without a doubt, but for the person paying for the project, that is rarely enough.
It is because no-one seems to read anymore.
Every single reason why projects fails has been analyzed time and time again.
You only have to read three books to know why 80% of projects fail:
The Deadline: A Novel About Project Management (Tom Demarco, published 1997)
It's a great introduction and it's pretty entertaining.
Peopleware : Productive Projects and Teams (Tom Demarco, published 1987)
The Mythical Man-Month: Essays on Software Engineering (Fred Brooks, published 1975)
We as a profession simply seem to forget everything every 3-5 years (see "centralised computing is inefficient; let the clients handle it" vs cloud computing).
(From a programmers point of view - I'm not a project managemer, but I've often been involved in the process).
A number of people have mentioned that bad programmers are endemic. But I think this is true in another sense as well - we're all bad programmers in that we find it difficult to anticipate complexity, an unavoidable issue that 50 years of magic bullet estimation and planning schemes have failed to solve.
Anticipating the side effects of large projects gets exponentially more difficult as projects grow. This is a dull truism, for sure, but for me it means that on any project I've worked on where I've been involved in the estimating process I've run into some case where there's an unanticipated consequence of a design decision that causes everything to come to a grinding halt, or at least a few days of bugfixing - just something that nobody foresaw, not any sort of malpractice or stupidity. It's just the nature of a complex enough system.
Aside from the built-in uncertainty, there's also a tendency to underestimate things whose outline is known, because the fact that they have less uncertainty makes them seem simpler to implement.
So the uncertain stuff gets magnified, the clear stuff gets minimized, and what really kills you is the thing that you didn't think would be uncertain.
The number one reason: a failure of project management.
A PM's raison d'etre is to make a project succeed, ergo a project failure is their failure. Certainly there are factors beyond their control, but it's still within the PM's job description to manage that risk, and the only get out clauses should be someone higher up the food chain taking decision control (which is a terrible thing to do to a PM) or acts of god.
In my experience failures mostly occur when PM work has been fast and loose or non-existant, including when decisions start to flow from sales people and when the client starts decreeing change control. A good PM is priceless.
Failure is a judgement -- more of an accusation, really.
"The project was more than 10% over budget and time."
Which budget? Which schedule?
6 months ago, I wrote a plan saying it would take 6 months.
3 months ago, the users asked for more stuff. I gave them a plan that said it would take 9 more months.
Last month I was told that the project was 6 months over budget and therefore a "failure".
But wait. It delivered what the users wanted. It was over the "original" estimate. It was under the revised estimate. The users want more. IT wants less.
I'll approach it from a different aspect than most the rest here.
I've noticed a project slowly fail over a period of time. Sure, it's gotten better in that time too--but it still isn't profitable. In this market profitability, and being in the black, means success.
Why is it failing? I think it's simple: you get what you pay for.
Software is like a bank account, not primordial ooze. If you don't put resources into it (time, money, focus, effort) then you won't get anything out of it except failure and cost. So you must invest things into your project, and sometimes the earliest work sets the stage for years to come. You can't throw mud at your computer and expect a new mouse in two years and $10 million dollars later, so likewise there must be effort expended.
One of the biggest problems today are "budget developers" in a third-world country. I don't begrudge them their part of the market, but for a well-funded Silicon Valley startup to seek them out and get a budget foundation (framework or even prototype) is to make a poor investment in the future. This very same budget framework is what is causing my friends so much of a hassle today. It works now; it worked when it was written, but it wasn't written well and few even take the time to maintain it. Were the company to stop and rewrite the software the way it should have been written in the first place they wouldn't have all this trouble. Can they afford the time? Nope. They have to make it profitable before they can even thing of it.
As the saying goes, "I can make it: cheap, fast or good. Now, pick any two of those." Everyone wants all three, myself included. But if you don't invest the time, planning, and work required to make your project a success from the start ... then don't expect anything you can be proud of later. It'll feel like a forged Mona Lisa where you, and every other engineer like you, can see a defect here and there that shouldn't have been there from the start.
So:
Don't undertake what you cannot afford in: time, money, effort, focus, etc.
Don't skip planning!
Don't be afraid to rewrite early when it counts the most. (Later it'll be worse than a trip to the dentist, believe me.)
Don't underestimate the power of bureaucracy to prevent you from doing it right.
And don't be cheap where you should spend the most of your time. It will cost you later, guaranteed. And if not you, then someone else will take the bullet for you.
One common mistake is that sales people and technical people do not communicate sufficiently. So the salespeople sell things that are technically not feasable within budget. (And then they run with their bonus :) )
Being over budget and time is not a good definition of failure (and actually being in budget and time doesn't always mean success). Take the following examples provided by Hugh Woodward, PMP, in Expert Project Management - Project Success: Looking Beyond Traditional Project Metrics:
Sydney Opera House: With its graceful sails dominating Sydney Harbor, the Sydney Opera House is arguably one of the most recognized buildings in the world. Yet, from a project management perspective, it was a spectacular failure. When construction started in 1959, it was estimated to cost $7 million, and take four years to build. It was finally completed in 1973 for over $100 million.
[...]
Project Orion: This massive effort to develop Kodak's new Advantix photographic system was reputedly very well managed from a project management perspective. PMI recognized it as the 1997 International Project of the Year, and Business Week selected the system as one of the best new products of 1996 (Adams, 1998). But Kodak's stock price has fallen 67% since the introduction of the Advantix system, in part because it failed to anticipate the accelerating switch to digital photography.
Corporate Intranet: Finch describes a project that involved the implementation of a corporate intranet to globalize and improve communications. From a traditional project perspective, it failed to meet its success criteria, but not significantly. It was one month late and believed to have been accomplished with a small budget overrun. But both the project manager and senior management viewed the project as successful. The hardware and software had been installed successfully with a minimum of disruption, thereby providing all staff members with access to the corporate intranet. Following implementation, however, employees made only limited use of the intranet facilities. The main objective of the project was therefore not achieved. In this case, both the project manager and senior management focused on an objective that was too narrow.
[...]
Manufacturing Plant Optimization: A paper manufacturing company with five plants across North America decided to increase its manufacturing capacity by embarking on a de-bottlenecking program. A project team was formed to install the necessary equipment, and charged with completing the work in 18 months at a cost of $26 million. But almost immediately, the project team was asked to defer major expenditures until an unrelated cash flow problem was resolved. Rather than stop work completely, the team adopted a strategy of prototyping the technologies on which the de-bottlenecking program was based, and actually developed some cheaper and more effective solutions. Even when the project was authorized to proceed, the team continued this same approach. The project eventually spanned five years, but the resulting capacity increase was three times the initial commitment. Not surprisingly, the company immediately appropriated another $40 million to continue the program.
[...]
So in these examples, which ones were truly successful? Examples like the 2002 Winter Olympics and the Batu Hijau Copper Concentrator would suggest that these are truly successful because they not only met the traditional project managers' definition of success, but also met the projects sponsors' perception of success.
As we start to look at the examples
like Project Orion, the Corporate
Intranet and the Laptop Upgrade, we
notice that the traditional metrics
start to fail. These projects are
considered successes in project
managers' definition of success, but
failed at meeting the sponsors'
success criteria. The project Orion
example is quite astounding as this
project was recognized by PMI (Project
Management International) in 1997 as
the International project of the year.
Yet it did not increase Kodak's
revenue, because they did not foresee
the adoption of digital cameras.
Most interesting are the examples of
the Manufacturing Plant Optimization
and the Sydney Opera House. They both
failed to meet the traditional project
managers' success metrics but were in
fact considered successes. This is
particularly shocking when you see
that the Sydney Opera House had a
"cost overrun of 1300%" and a
"schedule overrun of 250%".
Once we realize that projects can fail
to meet the traditional metrics of
success, but still be successful to
the stakeholders, this creates a
quandary for the project manager. How
does one really define success? Is it
possible that a "Challenged" project
could be canceled that would have met
the sponsors' needs? Is it also
possible to identify a project that
should be canceled that is currently
on time, on budget and meeting the
defined needs?
What do you think of that conclusion? How do you really define success?
My answer is rather unusual from the rest of this, but quite common around here: killer bugs. I had a project go an extra two weeks (50% extra time) because of one switch in source that I didn't know about until I dug through the source code (there was nothing documented in help or on the web).
People/companies do not proudly shout about their failures, so many cases wont be ever heard.
Poor use of practices and software development methods. In my experience, one of the big reasons a project failed its that the development team use a wrong method to face the software development process. Choosing a methodology without having a good understanding of how it works and what it takes, can bring a time consuming issues to a project, like poor planning.
Also a common problem is also the use of technologies without a previous evaluation of it to understand how it can be applied, and if it brings any value to the project.
There have been some good studies done on this. I recommend this link from the Construx website (Steve McConnells company).
The Construx link above is real good!
Many projects are managed on a rosy picture of reality. Managers tend to power talk developers into optimistic estimates. But say a 20 week projects gets "talked down" to 10 weeks. The requirements phase will now be 1 week instead of 2. After 1 week, the requirements aren't finished, but the project moves on. Now you're working on shaky ground-- and on a stretched schedule!
This can be funny. Once there was this old guy in a room opposite mine. His job title was system adminstrator. But the system he was supposed to adminsiter wasn't there. It had never been finished, although management thought it had been. The guy played games for about a year before he got bored and moved on.
The funniest part? Management put up a new job opening after he left!
IT Project Failures is a blog about project failures that may have a few answers here if one wants to read about it.
Myself, I think a large part of this lands on the question of being able to state exactly what is expected in x months at $y when really the answer is much more open-ended. For example, if a company is replacing an ERP or CRM system, there is a good chance that one isn't going to get all the requirements exactly right and thus there will be some changes, bug fixes and extras that come from taking on such a large project. For an analogy consider how many students entering high school or university could state their precise schedule for all 4 years without taking any classes and actually stick to that in the end. My guess would be very few do that as some people change majors or courses they wanted to take aren't offered or other things happen that change what the expected result is but how is this captured in a project plan that we started here and while we thought we were going there, we ended up way over in spot number three.
The last statistic that I heard was that 90% of projects are either over time or over budget. So if you consider that failing, quit a bit.
The reason why it fails mainly lies on process. We as software engineers don't do a good job of gather requirements, and controlling the customer. Because building software is a "mysterious" job to people outside of IT, it is difficult for them to understand why last minute changes are difficult. It's not like building a house and clearly showing them why it isn't possible to quickly add another door to the backside of the house made of brick.
Not only software projects go over budget, or take more than scheduled time to complete. Just open the newspaper and look at public projects like bridges.
But for a software project it is far more easy to cancel everything. If a bridge or building is half finished, there is no way back. Half the infrastructure is in place. It is very visible and it takes money to remove it.
For a software project you can press Shift-Delete and nobody notices.
Its just a very hard job to do an accurate cost analysis.
Using the FBI's Virtual Case File system it comes down to poor program management. The program had pre-9/11 requirements and post-9/11 expectations. Had government management done their job there would have been contract mods.
http://government.zdnet.com/?p=2518
"Because of an open-ended contract with few safeguards, SAIC reaped more than $100 million as the project became bigger and more complicated, even though its software never worked properly. The company continued to meet the bureau’s requests, accepting payments despite clear signs that the FBI’s approach to the project was badly flawed, according to people who were involved in the project or later reviewed it for the government."
Although $105,000,000 for 700,000 lines of code comes to $150 per line of code. Not too bad.
To truly gage whether a project is really successful, the original estimate / budget is probably a poor yardstick. Most engineers tend to underestimate how much time it will take because of political pressure, and they don't know how to estimate anyway. Many managers are poor planners because they want unrealistic deadlines to please their boss, they often don't understand what's involved, and plus it's something they can look at and understand and use as a stick in meetings, as opposed to actually helping solve problems. Practically, it helps businesses make rough projections of expense for budgeting purposes, at least it gives them something the go by.
A better measurement of project success would be - are the users happy? Does it help the business make money? How fast will the money gained help recover the cost of the system? These are harder to gauge than a simple plan, though.
In the end, I've find you're better off delivering on deadline, even if it means working some overtime. It sucks, but that is the reality of it.
As stated previously the vast majority of people involved in software development to not actually understand how
ask the right questions to learn about the problem
appreciate the users goal and judge expectations
understand technology available and the related Eco structure
most of team directly/indirectly involved are poorly skilled.
do not appreciate or know when they are wrong do they can take action.
Even with perfect requirements and related definitions too many developers don't know what they are doing.
Just look at the types of questions asked here. Would you go to a doctor that asked the same equivalent question. The scarey thing is that they ask and don't know how to learn or reason.
Different agendas
Management do not really understand what a developer does, how they produce code and the difficulties encountered. All they care about is the final product delivered on time. But for a developer they care about the technical aspects, well produced code in a solution they our proud of.
Deadlines
I've often heard developers say they wish they could produce better code, and that deadlines often push them into producing something that just works, rather than good code. When these deadlines are unreallistic the problems are exacerbated.
I think this thread managed to gather the biggest group of tallented unhappy software developers, engineers, project managers, etc. that it is possible to gather in one place.
I share point of view with most of the posts stuck here and I think they come out of a lot of suffering through seeing co-workers not doing a good job when job (programming) and success is the most important part of our lives.
http://www.clean-code-developer.de/
They have an incredible cause! and their philosophy, if taken ahead, could manage to create a new layer of heroes, as the main stream of developers and IT professionals is so ROT these days.
I'm working on something similar here in Brazil, 'cos I love our profession of bringing software alive both as PM and software developer (.NET) and I can't stand people who face programming as nothing else but they way out to make big money with almost no effort.
... sure I don't consider going overnight in front of the computer geniality. but the few who matter, have done it more than once.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
You must have heard the archetypical story of a failing/failed project:
A team of inexperienced programmers work 24x7
Bugs are fixed only to introduce new bugs
Customer is screaming that he could not even do the basic stuff (Saving/Querying) etc.
Programmers used to having the spec handed down struggle to improvise
No automated unit tests aggravate the situation
Architecture document that looked nice on paper was not followed in practice
Third party components used become bottlenecks not having been tested for fitness in the first place
Milestone after milestone missed
The team is not able to come up with a delivery date as nobody agrees as to the quantum of work actually needs to be done
No technical leadership / or a Cowboy Coder that can take on the technical issues
Now, If you were to be brought in as #10 what would be your first steps?
Update: First of all: Thanks to you all for chipping in. Well... I'm being brought in as #10. I was the original Architect anchoring the solution when we made the proposal to the client. Then, unfortunately, I couldn't take on the delivery responsibilities as I was assigned somewhere else. :)
Let's say it's a webification of an existing desktop application. I'm now being brought in as #10. Running away, sadly, is not an option. I'm sure this can still be reversed by following agile best practices and just wanted to tap the community for ideas.
The larger question perhaps is this: If the development team does not have specs but only the (baselined) code for a running application, the original solution called for looking at the code and extracting business rules on the fly. Now, the inexperienced programmers are reluctant to look at VB 6.0 code and want documents! So how do you fight this if you were to instate Agile processes?
Vyas, I feel like I could have written this question. My previous job involved resurrecting a KVM project that had failed after a year's development. Specs were in the form of a user manual and developers' experience with similar products. I ended up teaching C to 3 assembly programmers and re-architecting from scratch. We brought the product successfully to market in 4 months. (Then I resigned. Go figure.)
Some of the things I'd do again, particularly with an inexperienced team:
1. A team of inexperienced programmers work 24x7
10. No technical leadership / or a Cowboy Coder that can take on the technical issues
Give them a (short!) break from the project to "recharge." Maybe a day, maybe an afternoon, or maybe a long lunch on you. It will mark the end of the "old" project and the beginning of success.
Get their agreement to work their butts off when they return, and promise that you will be their go-to guy, cheerleader, and flak jacket. You, collectively, are a team, and your job is to forge their path, eliminate distractions, and lead them.
Plan an immediate success, no matter how small, and maintain a "can-do" attitude.
8. Milestone after milestone missed
9. The team is not able to come up with a delivery date as nobody agrees as to the quantum of work actually needs to be done
3. Customer is screaming that he could not even do the basic stuff (Saving/Querying) etc.
Take small bites! Break each piece down as far as possible, then deal with the small components. You'll identify "gotchas" early and be better able to scope the whole project.
Define your interfaces. Anytime you can isolate a chunk, do it. This allows parallel development, because you've already decided on parameters, preconditions, assumptions, what happens inside, and return values. You can stub it out, and build other modules and tests independently.
Prioritize. Focus on the defects and issues that affect the customer first. New features come last. If necessary, defer features rather than delivering buggy code.
Assign responsibilities. Volunteers are preferred, each in his/her area of expertise, but one person must be accountable for each task.
Track defects, and record everything that will help you reproduce, locate, and fix them. Document any that remain at delivery time, so the customer won't be surprised.
4. Programmers used to having the spec handed down struggle to improvise
6. Architecture document that looked nice on paper was not followed in practice
You will create the spec details as you go, each piece just before it's needed. It needn't be pretty, complete, or even written, as long as everyone understands the current task and you've got the big picture.
Discuss the implementation, one piece at a time, when the developer is ready to code it. Write the skeleton yourself if necessary, and let the team fill in the "guts." You want to keep them focused on each task, without "improvising."
Be available to answer questions as they arise. Your primary goal is to keep the team productive.
2. Bugs are fixed only to introduce new bugs
5. No automated unit tests aggr[a]vate the situation
Plan and start unit testing ASAP. If possible, enlist resources outside the team.
Fix small problems before they grow larger--or get hidden. Confidence in each small piece builds confidence in the whole.
7. Third party components used become bottlenecks not having been tested for fitness in the first place
Brainstorm solutions when you're not coding. Don't let them stop your progress if at all possible. Can you encapsulate or code around them? Replace them?
General suggestions:
Stay ahead of the team. Anticipate and try to solve problems before your team hits them. Gather any necessary information before it's needed.
Communicate constantly. Make it clear that you want no surprises, and solicit concerns, questions, status, roadblocks, etc throughout each day. Encourage collaboration and share "discoveries" across the team.
Celebrate every success. Compliment a clever solution, bring donuts when a problem is solved, demonstrate a new working feature ... anything that shows the team you appreciate them.
Get each task done, then move ahead. Don't waste time tweaking, enhancing, or reworking anything that isn't a direct barrier to success.
Keep your promises to the team, the customer, and your management.
Good luck -- please keep us posted!
Run away or find a new job. This is a death march and they need a scape goat.
Often, the death march will involve
desperate attempts to right the course
of the project by asking team members
to work especially grueling hours,
weekends, or by attempting to "throw
(enough) bodies at the problem" with
varying results, often causing
burnout.
Freeze releases, and start fixing issues with the program.... deal with the customer complaints by priority (the business side of the company can prioritize) and get the program running. Once you get the biggest issues out of the way, start cleaning up the code. Assign tasks to other developers, and start enforcing coding practices on all new code.
If you can do whatever you want, then look at what the real issues are and deal with them. If that means putting together a new team to develop the software all over from scratch, so be it. But you should try to at least fix the major bugs. Don't bother introducing new features, they only compound the problem, and a program that doesn't work and the problems aren't dealt with lose you clients.
Number 10 is obviously the worst problem, or at least the root of all others. Find someone with some creativity and ability to deliver a project, and give them free reign to do anything - including start over.
I hope you are getting paid really well. In any case, my plan would be something like these steps in the following order:
0) Stop adding features or functions across the team. Allow bugs to be addressed while the following steps are taken up to step 5, then stop bug fixing & resume feature development:
1) Apply what I call the Inverse Staffing Law: Weaker team members slow down the better and faster ones and generally a late software project needs people removed, not added. So, you need to assess the quality of the team members as individual contributors. Eliminate weaker staff from the team because presumably there are some. This is best done by reviewing their code and examining their bug fixes and figure out who is making the code worse vs. better and chop them for the team. This is not a time to mentor, you are going to need the best folks to have a change of "fixing" the situation in a optimal period of time. If you can't fire them or reassign them, have them getting coffee or something for everyone else left.
2) Assess the code itself. Identify areas of the code that are not constructed well and/or not well abstracted. If a area code is not constructed well and/or is obviously brittle at it what it is supposed to do, target it for a re-write. This feels painful at this point, but it will save you time in the long run. Recurring bugs and/or history of fixes will help identify the code that can't be salvaged. If a code area or module is fundamentally constructed well, but not abstracted well at the interface level, it should be suitable for re-factoring. This will save significant time and is useful code. Keep a list of the re-write areas, the re-factor areas, and the suitable areas.
3) Define a new reasonable architecture that you believe will result in a robust and complete solution to where you want to eventually be in features and functions. The architecture might not be optimal as starting clean, but in effect match up what you have with where you would like to be.
4) Work with the stake holders to decide what will make an acceptable first release attempting to table as many features as possible for "later" releases. Maybe you can't cut anything, but if you can, now is the time to do it.
5) Stop the background bug fixing efforts and assign the defined work out to the (remaining) team to estimate out a reasonable new implementation plan of the rest of the functionality. They need to own the schedule. Roll up the schedule and be fairly conservative. Now you have a reasonable prediction of when you could actually have something workable and robust.
6) Implement the remaining features and then harden up the release by tackling the remaining bugs. I am assuming all the normal good software development practices are observed here like source control, unit tests, etc.
7) Remove as many barriers as possible to keep the team cranking out stuff as fast possible.
8) Monitor for issues, and assist by getting your hands dirty where ever your can. Offer to take on the nastier issues to the extent you can help and still keep all members of the team as productive.
Good Luck!
This isn't about technical leadership any more, it's now about project management.
You as the technical lead will just be shifting deckchairs on the Titanic. So here's what I would do if I was the de-facto project manager.
1) Identify the project sponsors and stakeholders - both the official ones and the real ones.
2) Go to them and request that the project "goes dark" for a week.
3) If they don't agree, walk away from this project.
4) If they do agree, call a project time-out for a week - everything stops.
5) Spend that whole week talking to the important people on the project to identify the real project state.
6) Whilst engaged in those discussions, start formulating a project recovery plan, emphasising possible trade-offs between scope, schedule, budget, and personnel.
7) At the end of the week, decide which (if any) of your possible project scenarios are feasible.
8) Take the best of these scenarios back to the project sponsors and stakeholders, and start negotiating.
9) When a way forward is agreed, reboot the project and pray - possibly not in that order.
Common sense has already been pointed out to you by Maxim (Quit the death march). But if for reasons unknown you wish to persist, let me regale you with my experience in a similar situation - perhaps it might come useful.
It was my first job in a sleepy old town where good computer jobs where hard to come by and I despertely needed one immediately after college. I was hired coz the management thought i was enthusiastic enough and might be better than nothing (I offered to bring in my own comp to save them a cost of giving me a PC and offered to work for the experience alone)
The project had been abandoned by its creators due to the death march situation and had gone away after deleting all the comments in the code and performing other obfuscations. Nobody knew win32 / MFC stuff either.
I simply started studying the code on good old paper and pencil (lots of rubbing and corrections) until within 20 days time i knew the entire code including the variables by heart and what and where things where happening.
Armed with this knowledge i was able to make a critical piece working which had eluded everyone before. Of-course this was nothing but a drop in the ocean but it enabled the management to buy the clients confidence "smart fellow - got him with great difficulty - already got x working - u will have ur stuff working within y time".
Once the client was convinced and we where able to buy some time, some pressure was taken away. This got some hope back into the team and we started to hammer away for good. 6 months later i got promoted to project lead and 9 months later we had our fix shipment (lots of progress demos and a visibly more and more satisfied client in between).
As you can see, the elements of success are not directly duplicatable. But i would summarize that you need to breath some hope into the project first - show some progress and win confidence - that of your peers, management and the client. Once that is in place the technical stuff should be corrected too - there is nothing to replace this part of the equation.
If that does not seem likely, all that hard work (oh yes - lots and lots of work like you never imagined - why do you think its called a death march) would be a waste and you had better quit even before you start.
I had no choice and i was hot blooded and desperately need a job. The technical details where something icould work magic upon, and everthing just clicked into place. I really earned a lot of good will and self respect with that piece of work but in the long run its just a story i can narrate with great aplomb and nothing more except for those few in the know.
Things might be different for you but its for you to decide.
Good luck
Make sure you aren't the scapegoat
Cut scope creep
Trim functionality "requirements"
Implement a faster dev cycle (maybe Agile/Scrum/XP/whatever)
If you can, run away.
If not, you need to stop all activities that make the project unstable - including coding and fixing defects.
Assess where you are
Break up the requirements into much smaller "milestones"
Read some practical books (Mcconnell's "Software Project Survival Guide" comes to mind.
Identify all the problems and risks. Communicate all those to all involved.
Work on each piece one at a time.
Celebrate improvements and milestones as they are reached.
Good luck. Your scenario sounds pretty bad. It may not be salvageable - and things have to change to get better.
If you really had to get it on track (if bailing isn't an option)
Start off by accepting that it's a failure in management. You might then want to go on to implementing a strict but light process.
I'd suggest some form of Agile, since it's the easiest to successfully implement without a GURU, but you have to be VERY strict about it, including Pairing, Ruthless Refactoring, Reviews, Spiking functionality, Visibility, TDD, one-week cycles, 8-hour workdays (Yes, longer than 8 tends to harm productivity more than help, as you seem to have noticed)...
Don't be cutting anything out either. Parts of Agile rely on other parts--without the pairing, refactoring and testing you cannot eliminate upfront design (one of the biggest agile failures).
Don't forget about the management side of it. One week iterations to start (demo EVERY week). Constant adaptation. Very short stand-ups every day to address issues. (Keep to 15 minutes max, table longer issues, etc) Burndown charts, core-team with a client on it.
You can't just have a 15 minute meeting every week and 2 week iterations and call it Agile, but if you do it right, it just MIGHT give you a chance. You might get a GOOD agile consultant in to train you on getting started.
Also, constantly evaluate what works and what doesn't. Be prepared to fix what doesn't work. Weekly meetings to analyze that weeks' development successes and failures.
Overall it CAN work, and can bring a flailing team into line, but it's not trivial. The nicest part is that you can implement it without taking huge chunks of time out of your current development. You just keep developing, but you do it better.
Tough situation, you have zero customer trust and basically can't be successful under that situation, no matter what.
For all intents and purposes the project needs a reboot; the unfortunate fact is that incumbant shops usually don't get this oppurtunity to start over and re-evaluate everything that is there.
I hate to say it, but you need to halt development and spend a month working out what went wrong...
The result needs to be a plan for a feasible 6month - 1year delivery really making them focus on what the must-haves are and real trade studies on your third party components. And trashing the code base needs to be an option; start a new source control project and when you get to a particular module port peices that make sense and leave the garbage behind.
Agile is great and all, and a valid approach once you get a real plan in place; but its not going to fix a broken relationship with your customer... or all the junk that's already there.
Here's the summary of key learning after reading through your experiences:
Maxim
1: Make sure this is not a "Death March"
Ellie
2: Make sure what's delivered works
3: Refactor & Realgin codebase to Architecture / Best practices
4: Look at what are the real issues: Is the team technically competenet to deliver?
Kendall
5: Ensure availaibility of Technical Leadership
Bill K
6: Implement Agile Processes (At least automated unit tests if not TDD, short iterations that make progress visible)
7: Get customer buy-in
8: Be prepared to throw out what cannot work (wishful thinking aside)
Warren
9: Make sure the team memebers that remain given a chance to start over
Tim
10: Motivate team and as improvement becomes visible reward them
jsl4980
11: You need buy-in on schedule from your team (most imp.) & customer
[This raises more questions. What if your customer asks whether the team is competent enough to stick to your schedule? What if you yourself know that the timelines the team is proposing just shows their lack of understanding]
Ather
12: Is the team commited?
13: Do you formally QA?
Patrick
14: Start over, redesign and reconform to Architecture/Design best practices for modules yet to be developed.
The summary has 14 items. You can't do them all. So, what's the first step?
Here's what you have to do first -- get one thing improved.
You've got fundamental quality issues. (#2-5)
You've got architecture and component issues. (#6, 7)
You've got schedule problems. (#1, 8, 9)
You can tackle quality. Formal unit testing, heading toward TDD can help. This might be hard because architecture issues slow testing down.
You can tackle architecture. This might be harder because it will probably involve rework that will not appear deliverable. But it may fix quality issues. Or, it may be compounded by fundamental testing problems.
You can tackle schedule. Without other corrections (i.e., quality or architecture) you may not get any traction with fixing schedule issues.
I think that overall improvements in people's attitudes come from starting with one success -- any success -- as early as possible. What's the lowest-hanging fruit?
One long-standing bug? One unit test suite to find and fix that bug?
One major architectural feature? Would a diagram that everyone can post in their cube help? How about a presentation clarify things?
One new use case? One new feature that actually works?
Here's a good book on the subject:
Catastrophe Disentanglement: Getting Software Projects Back on Track
First off, be resolved that you may fail -- if you can't accept that, don't take the challenge. And that includes being a scapegoat (it does happen). Management won't look at it in those terms (i.e. they're not intentionally/consciously 'setting you up'). But that is a reality of a corporate environment; if you take on the responsibility (often with more pay than those that don't), then your head is for the block if things don't work out. You have to be ready to stick with it for the long haul too. I was once placed on a client site for 8 months to fix a waning project. And as you saw, one of the other blog-posters here spent 9 months before a release version was ready.
Now, assuming you are okay with the possibility of it going all pear-shaped in spite of your efforts, this is what I suggest:
a bug tracking system is going to be your number one best friend, it will allow you to regain a semblance of control. you can't hope to understand a complex system as a whole, so 'chunking' it will help. and a bug tracking system allows you to unitize problems and distribute them to the other guys you are working with.
you have got both technical and political challenges to deal with. the technical generally aren't so bad because you're a coder and you know how to do this. the political ones are much trickier, you're at the helm of a ship thats gone hopelessly off-course, and you're in the Bermuda triangle. the biggest challenge is often stemming the tide of negative sentiment amongst the client (e.g. client: "these cow-boys don't know what they are doing", "they promised me this and didn't deliver", "i have no confidence in these guys to any more").
for starters, apologize to the customer and tell them in concrete terms what you are doing to do to re-right their project, e.g. you: "I'm sorry about the delay on your project, I'm getting stuck into it now. I've looked at the project history, and personally, I would be angry too if I was paying good money for this system. the first thing I'm going to tackle is..." <- bingo, you've just taken responsibility for the project which means there's no turning back - its all or nothing now.
a few other people have said it here, and I agree; stop adding new features. what they haven't mentioned is that you may have to do this to keep the client happy (remember, there's a technical and political side to the challenge).
understand the business domain as best you can. read through any requirements documents you can get your hands on. you are at a massive disadvantage by coming onto the project late since you don't know what was originally discussed. the devil is in the detail. this is what sunk me on a late projects I wasn't able to salvage, everyone was on edge, and i missed a minor requirement. at the time, it wasn't a big deal and could have been corrected easily, but politically speaking, it was the straw that broke the camels back. one tactic which may help is to go out on client site for a few weeks.
understand that time is money. its not just a technical issue. the client has paid for something which isn't right or has not been delivered. your company has expended resources, possible having already used up all the project budget - the business is now losing money. and this is where the issue of new features come in again, yes - people are saying don't add them, stablise. but adding new features can be a politically helpful tactic, management will be happy because new money is coming in for off-spec work.
I'd recommend against you or your coding crew working ridiculous hours to deliver. if you normally leave at 5pm, leave at 6.30pm or 7pm instead. you and your coding boys can consistently maintain an hour or two of extra work for many weeks on end and perhaps 4-5 hours over the weekend. working until 9pm or 10pm every night will result in burn-out in roughly 2 weeks (some can go longer). after that point, your extra time on the project is doing more harm then good. in the unlikely event your boss takes issue with this, make a choice; do what they ask (i.e. work more hours), or say "I've already committed extra hours to working on this project - I'm here for the long haul and im going to get this project done if its the death of me. but that is the limit of how much time I'm willing to put in. i have other commitments to keep outside of work" <- but be ready for the consequences (remember, political situation as much as a technical one).
there are people here that are saying "stop and write a spec, stop and do this..." - I'm sorry guys, i just cant agree with you here, its unrealistic. the project is already stagnating, the last thing management or the client wants to here is "we have to stop everything and...". I've tried this before, where I've said to the client and management "the bugs will keep coming until we stop and i write up a detailed system test plan. it will take me two weeks" - the client didn't want to pay for this, and management wasn't willing to wear the cost. as it happened, the bugs kept coming.
learn to 'juggle' - you have to map out tasks ahead of time so programmers aren't waiting on you. this will generally mean you do less coding yourself. generally this is best achieved by having a project schedule before coding starts. programmers should know what they are doing next after they finish what they are currently working on, and they shouldn't be coming to ask you "what do i work on next?", they should already know.
build-in recovery utilities, especially if the software has recurring problems which are hard to pin-down. for example; it may take 12 hours to track down a bug and fix it, it may take 2 hours to put in utility (read 'hack') to fix the problem for the time-being. time and momentum are of the essessen, and unfortunately bandaid fixes may be needed.
be very observant of the clients mood. they need to know you are 'on their side' (e.g. client: "the product is unacceptable", you: "i agree, i would kick our asses to if i was in your position. all i can tell you is im on it and wont rest until its all working"). when the client is back on your site, they will actually start helping you. for instance, they may shield you from pressure from your management.
lead your guys by example. something along the lines of "I'm staying back a bit to work on the project, I'd appreciate the help if your willing to stay back too" and "i know its not our mess, but we're still going to clean it up anyway. i want the client to get some good quality software". programmers could generally care less about the company that got them into this situation, but they may care if its about one of their own or the client ('may').
many of the suggestions I've seen here assume a fairly high degree of power (e.g. 'stopping the project to restart it properly' or 'say no to new features') - you are starting the project already hamstrung, and as a programmer, you will traditionally have less power to affect change then a true manager. this doesn't mean 'give up/don't try' - it just means you are going to have to be creative and do things you don't normally do (i.e. use 'soft' or people skills).
a lot of people here are saying bail on the project, run for the hills. I have been on 3 hopelessly late projects to date. i managed to fix 2, and 1 I couldn't fix. personally, it doesn't bother me to take on a late project. after all, the worst that can happen is you get fired :)
If you were involved in the project from the beginning, I hate to say it, but the company should replace you (and the entire team).
It should be reanalyzed with a competent team with real project management processes and lead by a project manager with experience in this situation.
None of the original coders should work on the 'new project' of saving it. They can move to other projects (they don't have to be fired) but to get a fresh look at the project, everybody should be replaced.
And of course, management has to understand and be on board with the fact that the project is going to be much later than expected. If management doesn't agree with this (replace team, find experienced leadership, take a step back and start again) then #Maxim is right - get out of there.
1) The first thing I will assess is whether the people on the team are committed to the project or not? If not, it is worthless to do any other thing. Nothing can prevent the disaster unless I get a dedicated and committed team.
2) I'll make sure that there is QA on the team.
3) Come up with a reasonable plan of iterative and incremental releases to the customer. With the mess we are in, there is no way customer can get everything soon. Based on the priorities of customer, we'll deliver smaller increments of functionality to him frequently. This will keep customer engaged, a bit less-edgy since he is seeing something happening.
What ever you do, do it step by step.
First, it's not about addind features, it's about fixing the app. Don't add anything new. Just refactor. Say no to any new stuff somebody ask you to introduce in the system.
Don't try to improve the whole app. Take your team, make it focus on one aspect at the time, with the best practices you can, especially using unit test.
Use test driven development only. In that case, it will immediately show you what part of the behavior you don't understand (you can't code a test if you don't know what to test.
So here are the road map :
Identify the critical part you need to change
Isolate the code that implies this behavior
Find any occurence of this code in the rest of the code
Refactor using this knowledge and massive TDD
Integrate, test and fix until this particular part works
Go back to step
Make the situation clear to your boss : it will take time, money and will be painfull. Explain why, what you will do, and that you have no other way or it will fail AGAIN.
A above all, don't try to make it clean the first time. Refactor what you can, but don't expect to change the entire architecture of the part you are working on the first time. You will have to iterate the process on the whole application several times.
No miracle. Just method and patience.
Been there, followed these steps:
Stabilize
gather the real story: how good/bad is the codebase, how good/bad are the developers, what really needs to get done (bare bone min.), when it needs to get done by
reduce overtime (tired people, good or bad, don't work well)
remove the bad, input new/good - err on the side of replacement (many could be burnt out and appreciate even a forced change)
remove access to bad/un-required code (focus on the 20% of the code base that provides the 80% of the value)
put base code practices in place ensuring only good code is getting in (don't damage the base anymore)
Control
implement teams focused on the app components (decouple as much as possible)
put code management, release management, risk management, QA, etc. in place (build your environment so you can succeed)
get on your clients/sponsors good side - delivery a win, even if it's a somewhat stable very very small release - and then put in change management (control what gets requested)
Move forward
develop a plan (planning is essential, plans are useless according to Ike - you need to plan to find what is missing and to set a target, but don't expect to tell the future) - continuous planning is required
aggressively manage your people - good people make good product - make sure you get and retain the best
refactor over time - clean up code as you go - you may not have the luxury to fix everything at once so do it overtime to provide for a cleaner code base
move forward bravely - overtime be more aggressive with your deliveries test (but not stress) your team
Agile refactoring. Identify and prioritize what customer wants and then create the most important stuff in short sprints out of existing code. Good luck man :)