Related
Let's say you have a project that will involve two web applications (that will share DAL/DAO/BO assemblies and some OSS libraries):
a semi complex management application that uses Windows Live ID for authentication and is also capable of communicating with various notifier services (email, sms, twitter etc.), targeted notifiers being about 10% of functionality
a low to semi complex user application with less functionality but more robustness that also uses Windows Live ID for authentication
There are two of us with medium estimation capabilities and we won't be able to do it in two days even if we wanted/have to. At least it would be a far off estimate.
Questions
How long would/does it normally take you to make a reliable/valuable estimate?
What would you suggest to speed-up estimation without sacrificing accuracy?
How much slack (in terms of cost/time) would you add depending on estimation speed (when you would say: I could estimate it a bit more, because I think it's still quite off)
Since we use Agile methods (Scrum, specifically) it takes us about an hour longer than it takes the users to prioritize.
More time doesn't lead to more accuracy.
The hard part, then, is getting the users to prioritize. We hear this discussion all the time "if the whole thing isn't completed on time, it's all worthless." "Except for the XYZZY component, which does have some value." That argument can go on for hours until it's resolved that XYZZY should be first.
Generally, we try to create 4-week sprints. The first few are complicated because there's always something new. After the first two (or three) we seem to set a steady pace.
Each use case has a relatively simple, subjective valuation of how the effort it will take to finish it. Anything over one full sprint in duration has to be decomposed. Most times a few use cases are bundled into a single sprint.
The are formal ways of scoring each use case to better handle the cost and schedule issues. We don't use them because the extra effort doesn't help.
After the first two sprints,
There's new and different functionality,
The priorities have all changed,
The details of each use case have been dramatically revised.
What does "accuracy" mean when the thing you're trying to estimate changes at the end of each sprint?
One lesson learned. Parts of my company spend a long time fully defining exactly what will be delivered, and then measuring that they are delivering precisely what they want.
Customers notice this, and one said we "spend a lot of time delivering what the contract says, but it isn't what we needed."
The problem with firm up-front estimates is that they take on a life of their own. The more you "invest" in the estimating, the more the estimates seem to be a useful deliverable. They aren't useful because they're generally totally wrong. They're based on up-front assumptions that are totally wrong.
It's a bad policy to invest more time in estimating. The "accurate" answers aren't more accurate, but they are more treasured by every layer of management. As you and the customer learn, you invalidate numerous assumptions and you absolutely must re-estimate constantly.
Don't do it up front. If your contract requires you to do it up front, then make sure you have a change control provision and tell the customer that you absolutely will make changes as you go forward. As both you and the customer learn, you both must make changes.
Interesting question. I'm afraid the answer is "it really depends!" I know that's not terribly useful (although it IS true) so here are some factors:
1) Quality and completeness of requirements and their specification. This is, to me, most often the project-estimate killer. If you don't have quality requirements, you have no reasonable basis for an estimate. We use a "RUP-lite" style of product development here, so as the engineering manager I won't give anything but the coarsest-grain estimate until we've completed our "elaboration" phase and gotten sign-off from product management that the core 80% of the product features are in fact accurately covered.
2) The scope & nature of the product. Bigger/more expensive/more complicated = substantially longer to estimate. I've spent years working in telco-carrier land delivering solutions which have the normal robust carrier requirements ("5 9's" of uptime required by SLA mean you must really do a good job of solution design and failure recovery!). In that sort of environment with all the moving parts across functional areas of the business, the estimation of work is going to hinge on getting the whole picture...specifically, cross-functional dependencies and external dependencies can be a REAL killer here. That said, I've also built lots of shrink-wrapped and enterprise software, too. In those environments it's much easier as the scope is typically substantially smaller, so thus easier to estimate.
3) How "new" is this project? How "new" is the team to this product or technology set? The newer the product or team, the longer and more buffer you should allocate.
4) How specific do we need to be? If this is a "rough guess" then I'll lean on my engineering leads to provide a conservative estimate, then I'll pad that. If we need a "real" estimate (e.g., one which is used by my boss and which I'll be responsible for hitting), I'd need the input from a number of different managers and team members, who will need time to analyze the requirements and confer amongst themselves.
That can take as little as a couple days, or weeks..it all depends on the size. "Two or three days" is, frankly, not long enough for sizing anything but the most trivial of projects.
The best thing you can do to improve the quality of your estimates to to improve the quality of your requirements, and be ruthless in identifying hidden dependencies.
One final thing: FWIW, I've been doing this since '81 and I regard accurately estimating a project's duration/cost as the single most difficult/fraught with peril part of engineering management.
To make a reliable estimate you really would need to create a list of tasks to be done. Break it down into stories/tasks (even if you don't use agile) and evaluate them. This can take a lot of time already - especially the amount of research (to look for libraries to do this notifier stuff to reduce the cost). I would take at least 3 days - however 1-2 week(s) sound more reasonable for me. This doesn't seem to be a small project.
I wouldn't dare to speed-up the estimation process if you wan't to have somewhat reasonable results. Estimations are never accurate and you will just make things worse.
An option would be to make a totally rough guess within a few hours and then start to work on it already (for a week or so). At the end of the week you might be able to give a better estimation based on your current progress. However it's important to create a good prototype (which looks like crap but has a little bit of code in all of the regions).
Well, a common quote is "The price will be between 50% and 400% of our initial estimates". The reason that this quote has grown large is that it's true. Estimation accuracy largely depends on your domain-knowledge. If this is the 100th time you sold a given type of blog-engine than you're pretty sure about the estimates. However, more often than not, you don't have that indepth knowledge of the domain (If the app already exists, why create a new one?).
Agile development has become popular because people largely recognize that the traditional "waterfall" type of thinking just doesn't work for most real-world projects. You should apply the same way of thinking to your estimates. Obviously, you need some kind of selling point, but be sure to tell your customers that this information is very vague (and that no matter what any competitor will tell them, their estimates are vague as well).
You need to sell iterations, and therefore also estimate iterations. I'm sure some marketing-guy in any company will know how to handle the paperwork and the legal stuff for this, so it shouldn't be that big a deal.
Consider a 5 iterations project:
Start out by putting up milestones. These are subject to change, but will provide your vargue first estimates for the final product.
Plan iteration #1.
Estimate iteration #1, adjust total estimates accordingly.
Do iteration #1
Rinse / repeat till iteration #5
You're done :)
Reflect over your project. How did your estimates evolve? Reasons? Learn by doing :)
Most customers would rather have part-estimates and part-deliverances than some unrealistic goal that some suit sold them :)
Straying slightly from your original question but it's also important to learn from the estimates that you have produced in the past by keeping information about how long it actually took to do the things that you estimated.
We estimated 5 days to produce these pages, it actually took 10 days, and etc.
In the long term information like this will (hopefully!) enable you to produce more accurate estimates.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
As a new developer who is the only software guy on staff I have faced a few challenges but possibly the most difficult has been time estimates. I strugle every time I have to give a project estimate.
My question then is; If I do not have any experience and I don't have a fellow dev in my environment, how do I provide a solid estimate? I have read Joel Spolsky's article on Evidence Based Scheduling but how can that apply if I do not have any evidence?
I appreciate any advice on this subject.
You don't provide a solid estimate. You give as good an answer as you can, and explain that it is just a very rough estimate, and why it's so rough.
If you make it very clear that:
You can't give an accurate estimate
It's entirely reasonable that you can't give an accurate estimate because it's different work to what you've done before
You'll update the estimate as time goes on and you get to know the topic better
I think you should be okay. You need to make those things very clear though, in writing, so that you don't get held to your rough estimates later.
You are allowed to say "I don't know, I don't have enough evidence"
Then do some prototyping to get some evidence.
Then answer the question.
So you may in fact be able to give an estimate of when you will be able to give the estimate.
IMO Joel is way, way off in his article, and his conclusions and recommendations are not based on any reality. (Sorry, Joel) Fundamentally he says that you should be able to schedule your work down to units of time of hours or less before you even begin. But the reality is you don't know what those units of work are all going to be (in non-trivial systems) before you get in to the code. So you can't come up with an hour-by-hour breakdown of what you're going to do before you even pop the hood and have that breakdown reflect what actually happens with any accuracy.
Giving a project estimate is very difficult if you want that estimate to be of any value. Coming up with accurate estimates is difficult for programmers because very often you don't discover all the complexities of the project until you get under the hood.
So the solution to this is to get under the hood when coming up with estimates. For smaller projects & bug fixes this is fairly straightforward:
Replicate the bug on your machine.
Find the code that is causing the bug.
Figure out how to write the code that will fix the bug.
Estimate how long it will take you to write that code.
In finding the code you have to write you necessarily must discover most or all the complexities that would have thrown off your estimate.
The interesting thing about this method is that the time it takes to generate the estimate is very often 90% of the total time to actually do the work. You practically have to do the work in order to come up with an estimate. With bug fixes especially, the solution is often on the order of one line of code, so your estimate will end up being 5 minutes. That's fine because deadlines can be set around estimates like that.
As you get practice with this you will get better and better at "just knowing" how long things will take. At first you'll only be able to "just know" how long the smallest projects will take. But over time you will be able to estimate larger & larger projects.
I first base my estimate on my percieved complexity of the problem. How big is the problem. How many pieces might it touch or require. This gives me a general guideline. Then I always make sure I add a 15-25% fudge factor because you are going to miss something.
Finally you make it very clear that this is a rough estimate based on your understanding of the problem, and how you might solve it.
Also do not give any rough estimates in very precise increments. 4.5 hours is not a rough estimate. Half a day is a rough estimate.
Although it is very rough, I estimate on Lines of Code. This parameter, whose meaning for productivity is close to zero, still gives you an idea of the complexity of a project.
Measure the fact that on average, a developer can write circa 200, max 300 lines of code per day. Keep into account that just for coding of a single man army:
A small project of 1000 lines of (logic) code can be done in one or two weeks
An average complexity project of 10.000 lines of (logic) code could be completed in two or three months.
A large project of 100.000 lines of (logic) code requires at least a couple of years
To the logic code, you have to add the testing, which is already included in the previous estimates. To have a clue of the complexity, the Gimp is 600.000 lines of code, a kernel ranges in the million or more.
To this, add the fact that if you are working waterfall, the time you need to develop the code is actually a small part of the time needed to develop specifications and design. I would estimate a 2/3 time is for specs+design, and the remaining 1/3 goes in coding, maybe even more on the specs+design part. It is really time consuming.
So, track your estimate from the complexity, to the lines of code, consider the manpower you have and how much they can work in parallel, and add the overhead of specs+design, you will get a very rough estimate.
I suggest you the mythical man month. It is a fantastic book on this regard.
Personally, I imagine an estimate as a statistical distribution - and try to communicate the idea of standard deviation with it:
10 is 'it has a 50% chance to be between 8 and 12'
It's hard to get much more precise than that for overall project estimates. It is perfectly possible to get more precise (split into individual independent stories, collectively estimate each, and other agile practices) - but it has a cost.
(Also, an estimate should NOT be an engagement on deliverables - otherwise it gets padded to death and becomes useless)
If you refuse to give an estimate for something you have never done, you will probably do that all your life. First split the task as much as possible, this will help you clarify how you are going to do it. There is more chances you will be able to compare a fragment of the task with something you have done before. Don't hesitate to communicate your degree of certitude to your manager.
For an experienced programmer, who at least knows the system and has a set of reasonable requirements in front of them, "i don't know" is not a valid answer. If you say you don't know your PHB will go off and apply their 1337 h4x0r sk1lz and make an estimate in the order of "that sounds like a piece of cake, how about 1 hour".
You should be able to break the problem down into a series of smaller problems you've solved before and come up with a reasonable number for each part. Point out that it is very rough and could blow out considerably once you get to full analysis of the problem.
They're called 'estimates' because they're rough. You get better at estimating by doing it more and learning to draw on past experience as much as possible. Remember to factor in contingency (interruptions, task switching, possibility of being sick, possible rework, etc). Usually adding 50% makes the estimate closer to the mark.
Provide a rough estimate and be really clear about that.
Identify a strategy on how you will tackle the project. Particularly identify pieces of the system you can deliver as working intermediate releases. Pay special attention at the closest of these you would be able to release fully functional, and if possible take the rest out of scope (keep a list of these and anything that comes up, to be scheduled as a follow up project).
Use short iterations. Consider/analyze how the intermediate releases fit in 2-6 week iterations. Take into account learnings this give you, and adjust the overall estimate.
Go on with the first iteration, and apply what you learn about the assumptions you made. How off you are in the early iterations usually point to a problem in the estimates. Resist the temptation of considering the deviation in the estimates part of the initial overhead, as you will probably be delaying the point in time where you realize the estimates where off. Note that I do understand/agree the velocity of the project increases over time, but thinking about that tends to hide/delay the issues.
I do this all the time. ALmost everything I do is the first time. How do I estimate ? I guess! And then I guess again. And I keep doing that each delta-time interval that a schedule is reworked, because project plans are iterative and you only what you know when you are doing it. My guesses are pretty good tho because I have, after many many years, figured out what 'looks' easy and what 'looks hard'
Try Function Point Analysis. For CRUD stuff it gives good ballpark figures. It's main advantage is that it's based not on what you are going to implement, but on what the user has asked for. You'll need to find out what your FP productivity is, though. You can use past projects in the same language to do that.
You could use average productivity for the target language if you just can't build a historical dataset. It will give you something, not necessarily approaching reality, but will at least let you compare efforts for different projects.
Now, mind you FPA is bad on algorithmically-heavy software, and it relies on AVERAGES, meaning you'll probably overestimate or underestimate each project.
my coworker always says, first estimate the project length, then multiply it by two add 1 and then add the next highest units. so if your answer is 3 days, then you would say 7 weeks. that's a half joke, one idea would be first estimate the project and then when its finished see how far off you were, maybe you are consistently off by a multiple of 2 or 3, or whatever.
Any unknown task or job always has something which is known to a certain degree and easy to estimate. I split and I give estimates right away for the things I know and for the things I feel I know. The rest is honestly being declared as a thin spot and then we start "bargain". If work giver trusts my competence he will except my rough estimations and risks - we work together. It was never a failure, simply because I would never take a task which I can't lift or run into the ground (gut feeling?). If work giver doesn't trust me, I always recommend who to ask and where to look for a better option. Then we either work together or not. Most of the time we do, but everyone is on the safe side. I do my job, "thin spot specialist" gets his/her cut, managers are happy and customer's satisfied. Perhaps it's a bit naive, but it works for me :)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
If so why? How much?
I tend to inflate mine a little because I can be overly optimistic.
Hofstadter's Law: Any computing project will take twice as long as you think it will — even when you take into account Hofstadter's Law.
If you inflate your estimate based on past experiences to try and compensate for your inherent optimism, then you aren't inflating. You are trying to provide an accurate estimate. If however you inflate so that you will always have fluff time, that's not so good.
Oh yes, I've learnt to always multiply my initial estimation by two. That's why FogBUGZ's Evidence-Based Scheduling tool is so really useful.
Any organization that asks its programmers to estimate time for coarse-grained features is fundamentally broken.
Steps to unbreak:
Hire technical program managers. Developers can double as these folks if needed.
Put any feature request, change request, or bug into a database immediately when it comes in. (My org uses Trac, which doesn't completely suck.)
Have your PMs break those requests into steps that each take a week or less.
At a weekly meeting, your PMs decide which tickets they want done that week (possibly with input from marketing, etc.). They assign those tickets to developers.
Developers finish as many of their assigned tickets as possible. And/or, they argue with the PMs about tasks they think are longer than a week in duration. Tickets are adjusted, split, reassigned, etc., as necessary.
Code gets written and checked in every week. QA always has something to do. The highest priority changes get done first. Marketing knows exactly what's coming down the pipe, and when. And ultimately:
Your company falls on the right side of the 20% success rate for software projects.
It's not rocket science. The key is step 3. If marketing wants something that seems complicated, your PMs (with developer input) figure out what the first step is that will take less than a week. If the PMs are not technical, all is lost.
Drawbacks to this approach:
When marketing asks, "how long will it take to get [X]?", they don't get an estimate. But we all know, and so do they, that the estimates they got before were pure fiction. At least now they can see proof, every week, that [X] is being worked on.
We, as developers, have fewer options for what we work on each week. This is indubitably true. Two points, though: first, good teams involve the developers in the decisions about what tickets will be assigned. Second, IMO, this actually makes my life better.
Nothing is as disheartening as realizing at the 1-month mark that the 2-month estimate I gave is hopelessly inadequate, but can't be changed, because it's already in the official marketing literature. Either I piss off the higher-ups by changing my estimate, risking a bad review and/or missing my bonus, or I do a lot of unpaid overtime. I've realized that a lot of overtime is not the mark of a bad developer, or the mark of a "passionate" one - it's the product of a toxic culture.
And yeah, a lot of this stuff is covered under (variously) XP, "agile," SCRUM, etc., but it's not really that complicated. You don't need a book or a consultant to do it. You just need the corporate will.
The Scotty Rule:
make your best guess
round up to the nearest whole number
double that quadruple that (thanks Adam!)
increase to the next higher unit of measure
Example:
you think it will take 3.5 hours
round that to 4 hours
quadruple that to 16 hours
shift it up to 16 days
Ta-daa! You're a miracle worker when you get it done in less than 8 days.
Typically yes, but I have two strategies:
Always provide estimates as a range (i.e. 1d-2d) rather than a single number. The difference between the numbers tells the project manager something about your confidence, and allows them to plan better.
Use something like FogBugz' Evidence Based-Scheduling, or a personal spreadsheet, to compare your historical estimates to the time you actually took. That'll give you a better idea than always doubling. Not least because doubling might not be enough!
I'll be able to answer this in 3-6 weeks.
It's not called "inflating" — it's called "making them remotely realistic."
Take whatever estimate you think appropriate. Then double it.
Don't forget you (an engineer) actually estimate in ideal hours (scrum term).
While management work in real hours.
The difference being that ideal hours are time without interuption (with a 30 minute warm up after each interuption). Ideal hours don't include time in meetings, time for lunch or normal chit chat etc.
Take all these into consideration and ideal hours will tend towards real hours.
Example:
Estimated time 40 hours (ideal)
Management will assume that is 1 week real time.
If you convert that 40 hours to real time:
Assume one meeting per day (duration 1 hour)
one break for lunch per day (1 hour)
plus 20% overhead for chit chat bathroom breaks getting coffie etc.
8 hour day is now 5 hours work time (8 - meeting - lunch - warm up).
Times 80% effeciency = 4 hours ideal time per day.
This your 40 hour ideal will take 80 hours real time to finish.
Kirk : Mr. Scott, have you always multiplied your repair estimates by a factor of four?
Scotty : Certainly, sir. How else can I keep my reputation as a miracle worker?
A good rule of thumb is estimate how long it will take and add 1/2 again as much time to cover the following problems:
The requirements will change
You will get pulled onto another project for a quick fix
The New guy at the next desk will need help with something
The time needed to refactor parts of the project because you found a better way to do things
<sneaky> Instead of inflating your project's estimate, inflate each task individually. It's harder for your superiors to challenge your estimates this way, because who's going to argue with you over minutes.
</sneaky>
But seriously, through using EBS I found that people are usually much better at estimating small tasks than large ones. If you estimate your project at 4 months, it could very well be 7 month before it's done; or it might not. If your estimate of a task is 35 minutes, on the other hand, it's usually about right.
FogBugz's EBS system shows you a graph of your estimation history, and from my experience (looking at other people's graphs as well) people are indeed much better at estimating short tasks. So my suggestion is to switch from doing voodoo multiplication of your projects as totals, and start breaking them down upfront into lots of very small tasks that you're much better at estimating.
Then multiply the whole thing by 3.14.
A lot depends on how detailed you want to get - but additional 'buffer' time should be based on a risk assessment - at a task level, where you put in various buffer times for:
High Risk: 50% to 100%
Medium Risk: 25% to 50%
Low Risk: 10% to 25% (all dependent on prior project experience).
Risk areas include:
est. of requirement coverage (#1 risk area is missing components at the design and requirement levels)
knowledge of technology being used
knowledge/confidence in your resources
external factors such as other projects impacting yours, resource changes, etc.
So, for a given task (or group of tasks) that cover component A, initial est. is 5 days and it's considered a high risk based on requirements coverage - you could add between 50% to 100%
Six weeks.
Industry standard: every request will take six weeks. Some will be longer, some will be shorter, everything averages out in the end.
Also, if you wait long enough, it no longer becomes an issue. I can't tell you how many times I've gone through that firedrill only to have the project/feature cut.
I wouldn't say I inflate them, so much as I try to set more realistic expectations based on past experience.
You can calculate project durations in two ways - one is to work out all the tasks involved and figure out how long each will take, factor in delays, meetings, problems etc. This figure always looks woefully short, which is why people always say things like 'double it'. After some experience in delivering projects you'll be able to tell very quickly, just by looking briefly at a spec how long it will take, and, invariably, it will be double the figure arrived at by the first method...
It's a better idea to add specific buffer time for things like debugging and testing than to just inflate the total time. Also, by taking the time up front to really plan out the pieces of the work, you'll make the estimation itself much easier (and probably the coding, too).
If anything, make a point of recording all of your estimates and comparing them to actual completion time, to get a sense of how much you tend to underestimate and under what conditions. This way you can more accurately "inflate".
I wouldn't say I inflate them but I do like to use a template for all possible tasks that could be involved in the project.
You find that not all tasks in your list are applicable to all projects, but having a list means that I don't let any tasks slip through the cracks with me forgetting to allow some time for them.
As you find new tasks are necessary, add them to your list.
This way you'll have a realistic estimate.
I tend to be optimistic in what's achievable and so I tend to estimate on the low side. But I know that about my self so I tend to add on an extra 15-20%.
I also keep track of my actuals versus my estimates. And make sure the time involved does not include other interruptions, see the accepted answer for my SO question on how to get back in the flow.
HTH
cheers
I wouldn't call additional estimated time on a project "inflated" unless you actually do complete your projects well before your original estimation. If you make a habit of always completing the project well before your original estimated time, then project leaders will get wise and expect it earlier.
What are your estimates based on?
If they're based on nothing but a vague intuition of how much code it would require and how long it would take to write that code, then you better pad them a LOT to account for subtasks you didn't think of, communication and synchronization overhead, and unexpected problems. Of course, that kind o estimate is nearly worthless anyway.
OTOH, if your estimates are based on concrete knowledge of how long it took last time to do a task of that scope with the given technology and number of developers, then inflation should not be necessary, since the inflationary factors above should already be included in the past experiences. Of course there will be probably new factors whose influence on the current project you can't foresee - such risks justify a certain amount of additional padding.
This is part of the reason why Agile teams estimate tasks in story points (an arbitrary and relative measurement unit), then as the project progresses track the team's velocity (story points completed per day). With this data you can then theoretically compute your completion date with accuracy.
I take my worst case scenario, double it, and it's still not enough.
Under-promise, over-deliver.
Oh yes, the general rule from long hard experience is give the project your best estimate for time, double it, and that's about how long it will actually take!
We have to, because our idiot manager always reduces them without any justification whatever. Of course, as soon as he realizes we do this, we're stuck in an arms race...
I fully expect to be the first person to submit a two-year estimate to change the wording of a dialog.
sigh.
As a lot said, it's a delicate balance between experience and risk.
Always start by breaking down the project in manageable pieces, in fact, in pieces you can easily imagine yourself starting and finishing in the same day
When you don't know how to do something (like when it's the first time) the risk goes up
When your risk goes up, that's where you start with your best guess, then double it to cover some of the unexpected, but remember, you are doing that on a small piece of the project, not the whole project itself
The risk goes up also when there's a factor you don't control, like the quality of an input or that library that seems it can do everything you want but that you never tested
Of course, when you gain experience on a specific task (like connecting your models to the database), the risk goes down
Sum everything up to get your subtotal...
Then, on the whole project, always add about another 20-30% (that number will change depending on your company) for all the answers/documents/okays you will be waiting for, meetings we are always forgetting, the changes of idea during the project and so on... that's what we call the human/political factor
And again add another 30-40% that accounts for tests and corrections that goes out of the tests you usually do yourself... such as when you'll first show it to your boss or to the customer
Of course, if you look at all this, it ends up that you can simplify it with the magical "double it" formulae but the difference is that you'll be able to know what you can squeeze in a tight deadline, what you can commit to, what are the dangerous tasks, how to build your schedule with the important milestones and so on.
I'm pretty sure that if you note the time spent on each pure "coding" task and compare it to your estimations in relation to its riskiness, you won't be so far off. The thing is, it's not easy to think of all the small pieces ahead and be realistic (versus optimistic) on what you can do without any hurdle.
I say when I can get it done. I make sure that change requests are followed-up with a new estimation and not the "Yes, I can do that." without mentioning it will take more time. The person requesting the change will not assume it will take longer.
Of course, you'd have be to an idiot not to add 25-50%
The problem is when the idiot next to you keeps coming up with estimates which are 25-50% lower than yours and the PM thinks you are stupid/slow/swinging it.
(Has anyone else noticed project managers never seem to compare estimates with actuals?)
I always double my estimates, for the following reasons:
1) Buffer for Murphy's Law. Something's always gonna go wrong somewhere that you can't account for.
2) Underestimation. Programmers always think things are easy to do. "Oh yeah, it'll take just a few days."
3) Bargaining space. Upper Management always thinks that schedules can be shortened. "Just make the developers work harder!" This allows you to give them what they want. Of course, overuse of this (more than once) will train them to assume you're always overestimating.
Note: It's always best to put buffer at the end of the project schedule, and not for each task. And never tell developers that the buffer exists, otherwise Parkinson's Law (Work expands so as to fill the time available for its completion) will take effect instead. Sometimes I do tell Upper Management that the buffer exists, but obviously I don't give them reason #3 as justification. This, of course depends on how much your boss trusts you to be truthful.
Whilst answering “Dealing with awful estimates” posted by Ash I shared a few tips that I learned and personally use to spot weak estimates. But I am certain there must be many more!
What heuristics to use in the scenario when one needs to make a quick evaluation of software project estimate that has been compiled by a third-party (a colleague, a business partner or an external company)?
What are the obvious and not so obvious signs of weak software estimates that can be spotted without much detailed knowledge of task at hand?
A single person having done the estimates, rather than having used consensus based estimation (to fully understand the implied scope of requirements) such as Wideband Delphi.
Especially true if the person doing the estimation is not the person doing the implementation!! - I once worked on a project estimated by someone else as 60 days before any requirements had even been given. Lets just say I was not a happy bunny
No time for documentation.
No time for ramp-up (in terms of learning, and team size).
No list of risks, and their impact to the timescale.
No buffer for the unexpected - in terms of late breaking requirements, and risks.
No one has said it, so I will. The obvious answer is that if you have software schedule estimates then that is a sure sign of unrealistic figures. Yes, there are many methods for estimating software but none of them are accurate in any way, shape or form. What usually happens is that deadlines are set. If the task is over-estimated then extra time is spent making the result better. If the task is under-estimated then something is sacrificed to meet the delivery (like testing and features).
I know this answer isn’t what people want to believe, but estimating is always a guess. More often than not, a developer can’t even predict how much they will accomplish by the end of the day. You are expecting them to guess things months/years down the road on something that they aren’t even sure what is really involved yet.
The only practical answer to your question that isn’t prone to giving unrealistic results would be using a worksheet that comes up with guesses based on previous history at your company. Unfortunately, that will not account for tasks the estimator missed. At least this may give ballpark numbers.
Unless you develop knock offs of the same exact system over and over again, then anyone who thinks they have figured this out is fooling themselves. There are way too many variables involved.
There are two types of estimates: task estimates and project estimates. You can view these as the big and small pictures.
Project estimates are necessarily high level (granularity no smaller than days typically) and must include things like:
High level architecture;
Time for testing;
Ramp up times;
Defect processes;
Time for documentation;
Relevant training;
Assumptions;
Dependencies (eg team A can't do what they need to until team B delivers phase 1);
Critical path (which pieces are likely to determine if the project slips and by how much); and
Risks.
The more of those things that are missing, the more unrealistic (or risky) the estimates.
The second kind of a task estimate, which is typically much lower level. For this kind of estimate it should be simply a task breakdown (with no task being larger than say 5 days).
These don't tend to address the above items but some of them might be relevant, such as assumptions regarding decisions not made yet (eg production hardware). It may also be worth identifying who can and can't do the tasks due to relevant experience, background knowledge or skills (as that person or those persons may end up overcommitted).
Other posts have mentioned the testing time should equal or exceed dev time. I strongly disagree with this. I've seen 8 hour dev tasks result in 100+ hours test time and 80 hour dev tasks result in less than 2 hours of testing. In both cases the testing time was entirely reasonable. The is no absolute correlation between the two. At best, there is a loose connection.
Is the estimate what the management
wanted to be told?
Does the
estimate nicely fit in with the
planned shipment date for the next
release?
Does the management reward
people that give good news more then
people the give bad news?
Was the
estimate prepared before knowing who
would be working on the project?
Did
someone that wanted that bit of
functionally implemented prepare the
estimate?
Is there a history of
software being late?
Is it normal for
developers to be moved onto other
tasks partway though a project?
Have
some or all developers given up on
commenting on bad estimates as a
waste of time?
Count up the number of questions you get “yes” or “maybe” answers.…
If you get mostly “no” answers to the above questions, then it may be worth looking at the estimate in detail to see if it includes the tasks that other people of listed in this thread.
Wow... I really like toolkit's answer.
And I agree that any estimate at all is flawed, because it assumes that the estimator has way more of a clue for how to solve the problem than any estimator actually does when a project gets estimated. However, I think you still need to at least estimate the size of the mountain before you start. Some thought as to whether it's worth trying to do it should precede any endeavor and that's what the essence of an estimate should be.
I did come up with a few more indicators of a dangerous estimate:
No cross-reference - If the estimate can't be validated at least two different ways, it's likely to be unreliable. For example, the last estimates I've done I've been able to break down the work into small feature chunks, and consider how long it's taken our team to do similarly scoped features. Then I was able to look at the sum of these costs and see how big the scope was relative to other projects I've worked on. That's two ways to validate.
The background of the estimator - if this is a software estimate done by a hardware guy who's never written code - be very afraid. More subtle - the closer the estimator's background is to the technology and problem domain of the project, the better.
Detail - as said a few different ways in a few different posts - I like to see detail for both individual features, as well as the tasks needed to complete each feature. Most estimates I see don't show the detail in the formal presentation, but if you ask the person who did the estimate, they should have a file somewhere. Hopefully it's not the back of a paper napkin stained with beer and ketchup. :)
Documented Assumptions - any estimator will have had to make some set of assumptions about the task. These should be documented somewhere, perferably in the formal paperwork. I always get a little worried when I see a short proposal with not many documented assumptions. Either they were thought through and not communicated to the customer, or they were not thought through. I'm not honestly sure which one is worse. It goes without saying that the assumptions should also be reasonable.
Balanced Lifecycle - However the task is broken down, what's the ratio of design, code and test? How about documentation, integration with external systems and post release support? How about those extra things that are so vital (system admins, CM gurus, management effort)?
Slack - I'm sure the corporate daemons of cheapness will come and flay me, but a schedule and a cost should have some slack. If the estimate looks ambitious and agressive to an experienced eye, it is likely to be too low. Estimates are almost never too high.
One good heuristic is to see if test time is roughly the same a development time. That's a good sign for the estimate.
If they can't give you a breakdown of the estimate then that's a bad thing. Usually a sign of lots of little things that may have been forgotten. They don't need to provide the complete original breakdown, just a breakdown like:
requirements
development
testing
packaging and deployment
etc.
They should be using a standard template to calculate their estimate. They don't need a number in every column, but they do the template to list all possible tasks. That way the template can be used to jog peoples's minds when working on the estimate.
If the estimate is overly precise, e.g. 0.25 hour increments, then that, for me, is a bad smell.
If there are things missing like requirements capture, testing, deployment, and handover to any Ops group? If any of those are missing, that's the sort of thing that will come back and bite you.
Edit: One other thing to watch for is the old "perpetually 90% complete" tasks. You get progress update after progress update listing a task as "90% complete". That's not good!
HTH
cheers
Is the compiler of the
estimate available and willing to
discuss it with other senior project
members? If not, that is often a
concern.
Was the estimate sent to the
customer before the experience and
skills of the development staff are
known. Two point estimates may help
but only to some extent.
Before even getting a chance to look at the estimate, you are told that you are committed to delivering all of the functionality described by a specific date.
(Thanks for responding to my question, by the way.)
If you see one or more of these, you may have a bad estimate:
Single point estimates: an estimate should be associated with a range of possible dates or a confidence value
Insufficient granularity of tasks: a large task bucket usually indicates that the functionality is not well understood (which is especially a problem since poorly understood problems are usually under-estimated)
No expression of assumptions and/or risks
Inadequate effort allocated for commonly skipped or underestimated items (e.g. build scripts, documentation, deployment, etc.)
I agree with sateesh, I really like Software Estimation: Demystifying the Black Art by Steve McConnell. He has several checklists which are useful when reviewing and/or preparing estimates.
I totally agree with Dunk, the first sign of bad estimates is the mere presence of a large detailed upfront schedule. Estimates are exactly that, an approximation, otherwise we would call them exactimates. So they should never be used alone in the management of a project.
The most important point to consider is not the accuracy of estimates but the consistency. If a third party were doing estimates for you, then ask to see a history of their successes or failures, speak with their past clients and determine their reliability.
That all being said, from an Agile standpoint, some of the ways we attempt to gain more consistent estimates during a project are;
Use a relative sizing standard (S,M,L,XL) rather than time based (ideal days).
focus on complexity not time
Always use group estimates not single person estimates
Gather estimates frequently throughout the project, generally at the start of each sprint
use feedback from previous sprints in determination of story complexity
track velocity to give meaning to the relative sizing
frequent and early story retrospection to examine/understand any thrashing
If you are dealing with companies that use these estimation methods then, chances are you are going to receive consistent and therefore better results.
Estimates of the form 3, 6, or 12 months (basically any round numbers) reek of guessing. Usually when you guess you pick some round number bigger than you think it will take -- quarters, half a year, etc. -- are the usual suspects. I much prefer estimates in terms of actual development iterations (whatever their size).
What are the obvious and not so
obvious signs of weak software
estimates that can be spotted without
much detailed knowledge of task at
hand?
Estimates which are given without much detailed knowledge of the task at hand are generally not good.
Perhaps a general approach you could take is to check that items in the requirements correspond to those in the estimate. If you want to be very quick check the relative sizes, if there is a 100 word estimate given to a 100,000 word brief it stands no chance of being right.
Also (as others have said) check that analysis, coding, debugging, testing, integration, contingency etc are mentioned. It shows some thought has gone into it.
Having success and sign off criteria at various stages is a great sign. If they have a defined point which is 10% done at least if the estimate is wrong you know early and have a chance to adapt. If there are no checkpoints until “finish” you may not know that you are behind until that date is hit.
How familar is the person giving the estimate with the people doing the work?
I have often seen estimates where there is a generic person doing the work, even though the team is made up of individuals with very different backgrounds. Most likely the tasks and the specialities don't line up perfectly and you get a c++ serverside programmer who ends up doing either your gui or your database... Sometimes the manager of the team doesn't really appreciate the team member's strengths, so if he has been asked to come up with the estimate on his own because his team is busy on the previous project you will find that the work in question is really only suitable for part of the team (not motivating, lack of skills etc)
One other helpful way to evaluate the estimates is to compare it with the actual effort that was spent on previous projects of similar kind. The best data for the comparison would be the effort data of the previous projects that the organization has done. If there is no organizational historical data you can try to benchmark the estimate against industry wide benchmarks.
I would also say if the estimate is presented as single absolute number (say 180 days) then it is not a good sign. A single absolute number would indicate that the estimate is that the task will be finished with 100% probability on the given data. The estimates presented in a range (say 130 to 180 days) would indicate that the range in which task could be completed.
Much of what I have written above I attribute it to the book :
Software Estimation: Demystifying the Black Art
by Steve McConnell
I check the estimates against the man-power. Although not a very accurate heuristic, it's clear if some massive work has just one or two devs assigned to it, that the task was not estimated correctly
A good estimate will have a good breakdown, involving all phases of the project.
It will almost certainly not finish at a convenient date for the business.
It will include risks of various sorts.
It will be presented in terms of confidence intervals, either implicitly (10-12 months) or by using large units (about four quarters).
It will be made by somebody with responsibility for getting the project done, preferably more than one such person.
If there are delays at the start, there will be delays at the end (expressed as 10-12 months from start, or about 1Q2010 if we start now, not something like January 2010 when the project hasn't started yet).
Assumptions and dependencies will be clearly and prominently listed.
Edit: Part of this depends on the stage the project is in. An early but precise estimate is a warning sign, particularly if there is no confidence interval assigned. That reeks of a Procrustean estimate.
Also, watch for other development methodologies. A timeboxed project can have a rigid and arbitrary schedule, but the feature set will be flexible.
Any of the following:
It is one big project and there isn't a short high level strategy described
There isn't a clear, short and concise vision of what wants to be achieve with the project
The project isn't structured around business value being delivered gradually
The team is trying to give "accurate" estimates for a big project, going into (or was done with) a long analysis phase? (changes will come, and will usually affect those estimates in way that can't be easily quantified without yet more big efforts)
There are "detailed" estimates for the whole project (related to previous)
There aren't detailed estimates for the first phase, or there is something wrong in those.
"Four to six weeks", when not accompanied with a breakdown into shorter tasks...
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
My firm just got its first large-scale development project inquiry and I would like to use an Agile process. The client has a vision for the application but openly admits to having very few requirements and recognizes that we will have to charge by the hour. Because of this, I've all but sold him on an Agile approach.
The problem is that he wants a figure to budget around. I've read a number of articles that pretty much advocate against giving up an estimate because the client will budget for that number and even as requirements change, the number in their head and in the books doesn't.
I've read there are a number of ways to factor in pricing in the contract, but almost all of them (save one) incorporate an up-front number. This just seems to violate the entire set of principles of Agile development.
So my question is, if you're an Agile shop, how do you manage to circumvent this Catch-22 of Agile development?
Here's the fundamental question.
When will the client think they're done?
If they think they'll be done by June, then you put an Agile team in place. That's 4-6 people for 6 months. That's the budget. Essentially, you do the multiplication for them. team * rate * 6 months.
If they think they'll be mostly done by June, but there will be more work after that, then you're possibly looking at 9 months of work. Again, you're just doing a multiply that they could do for themselves. team * rate * 9 months.
If they think that you'll be their development team for the foreseeable future, give them a price that will get the project through to the end of the year. team * rate * 12 months.
Since each release is an opportunity to reprioritize, you should be pricing each release as as separate piece of work based on the things you will get done in that release. Since it's their priority scheme, they control what you build, they control the budget, phase by phase.
Often your client really wants to know how much a particular feature set will cost. Instead of ask that, they ask about overall budget (which is silly). Spend a lot of time on the first release showing what they get and how much that first release will cost.
Eventually, they'll see the fundamental truth.
They're buying the features from most important to least important. If they prioritize correctly, they can stop spending money at any time and have something useful.
Done is a relative term. Some projects are "done" because there's no more money. Others are done because there's no more time. Rarely (at least in software development) is a project ever done because we ran out of things to do.
For this situation, we've provided an estimate for the first chunk of work, and then let the client purchase more sprints as required to complete the work to the desired level.
As you get more of the system developed, and incorporate feedback from the client into the sprint deliverables, you will both get a better feeling for the amount of work involved, and hence the costs involved.
Edit: Cool. Way excellent! From the fact that you've sold them on an Agile approach, BTW good one!, chances are you will be able to seel them on approaching it from an agile point of view in terms of features to be implemented. Maybe have a listen to this "Intro to Scrum" podcast. You might be able to sell them on the fact that they probably won't need to have all the bells and whistles that they think tha they need right now.
HTH
cheers,
Rob
Look, there's a core fact here. You will be asked to estimate cost, contract for a particular delivery date, and commit to a full set of delivered features.
You can't do all three.
Not "you shouldn't" or "it would be wise not to"; you (for all practical purposes) can't. By which I mean that the probability of successfully doing all three is extremely small.
The best answer is to commit to a cost and schedule, and to an iterative process with quick iterations and regular feedback, and then write the agreement so that what is done unde the fixed cost and schedule is what will be delivered. That is, you keep delivering new features (and modifications) until the time and money runs out.
The truth is, even if you do sign up to all three, the best you'll ever be able to do is two out of three anyway. Might as well be open about it.
Here's how a crusty old government contractor I know recently put it: "As the prostitutes say, first you gotta get them upstairs."
That doesn't answer your question, but it's worth remembering. If they won't come upstairs without number up front (and they won't), you have to give them a number.
Anybody sponsoring a software project needs to have an idea of what kind of return they're going to get on their investment, so that they can evaluate whether or not the investment is worth making. A project may be worth doing if it costs $1m and takes 12 months. It may not be worth doing if it costs $2m and takes 24 months.
The real question is: can you do this project within a time frame and budget that makes it possible for the client to realize an appropriate return on their investment? If your answer to that question is anything but "yes," then they shouldn't hire you to do the project. Otherwise, they're just spending money and rolling the dice.
If your current answer to that question is "we don't know yet," then they shouldn't hire you to do the project. But that doesn't mean that they shouldn't hire you to find out whether or not the project is worth doing. A good consulting-firm buzzword for this is "preliminary scoping exercise."
Agile development is about managing a curve in three dimensions: money spent, calendar time, and feature set. It's a given that if the budget and schedule are fixed, the feature set must be variable. You can't know what the final feature set will be, given those constraints. But you can know if a feature set that you'll be able to produce within those constraints falls inside a range that your client will find acceptable.
You can know this, and you can find it out. Finding that out is something that your firm can do and your client can't. It's a service that you can provide to them and that they should pay you for. Avoid the temptation to do this for free and call it a cost of sales. Project scoping is every bit as much a part of professional services as software development. You're not doing this to close a sale; you're doing this to help your client make a business decision that they don't yet have enough information to make.
But first you gotta get them to come upstairs.
These answers are really great. I don't have a lot of experience to share, but I thought of an analogy:
Last year my wife and I remodeled our kitchen. The contractor proposed billing on time & materials instead of giving an estimate, because we didn't know what we'd discover with respect to plumbing, subfloor damage, and so on. We agreed, because I'm working from home and I was available continuously to answer questions. The contractor and I had a good rapport so when something came up, he felt free to knock on my office door and we'd discuss it. Decisions got made quickly and the work was done the way I wanted it. That seems similar to the Agile priority on frequent customer review.
By contrast, my wife's cousin is also remodeling his house. He's an ER doctor and he is not available on site during the remodel. So he described being regularly surprised by the contractors making major decisions without consulting him ("What? I didn't want that picture window blocked off! Tear out the wall and reframe the window the way it was!"). This is of course painfully expensive and blows the schedule out of the water. Definitely not Agile.
So one way to convince a client that a time & materials mode will work may be to assure them you'll make frequent progress reports to get their feedback. I believe this is already a core recommendation of most Agile methodologies. To begin, of course this requires the customer give their trust to the consultant.
As a separate resource, I searched and found a book on this subject: "Agile Estimating and Planning" by Mike Cohn. Read the praise quotes on that page, from an impressive set of Agile experts.
First of all, I want to say I think it's great you've sold him an on agile approach and per hour pricing. That's very hard to do.
Regarding your dilemma, I think you should present him with a rough pricing estimate and the features / scope it includes. During the development process, if you see something important coming up that will change the scope / cost, you say to the client "stop - this is great, but understand that it changes the original scope we discussed."
At this point the client has the privilege of agreeing, being aware that he will pay more than he originally thought, or halting the new development for now. Many agile projects are built in many mini-stages - for which you can give pretty accurate estimate of price / time. Before any new mini-stage commence, it's important that you and the client see eye to eye on what is up next and how much time / cost will be spent on it.
As always there's quality, functionality and time (=resources) which are your main parameters. We're not messing with quality any more, so there's only features and resources left. Fairly often you can get away with convincing that a certain amount of resources will at minimum reach a certain feature set. So as long as you can find an amount of resources that delivers a plausible feature set I believe this is enough for quite a few customers. The upside is that they can control progress while on the move, and there's always the option of backing out.
The way that I have done this is to use my current velocity and estimate a range based on rough estimates of complexity (number of stories). You would update this as you begin planning the application so that the range estimate gradually becomes better and better.
For example, give an estimate from 6mos to 2 years (if you're sure that it will take less time than 2 years. As you break it down into features, then plan for those features, you come up with better and better estimates so that after 6mos you can reliably say (absent any other changes), we'll be done in another 2-3 months (total of 8-9 months). Eventually, you get to the point where you can say, we'll be done after the next iteration (2weeks to 1month).
You need to pair this with letting the customer know that adding features will increase the time to completion, then let them decide how to proceed -- either drop the feature or increase the time. For any given project, scope and time are really the only variables that you can effectively change. Quality and resources are pretty much fixed. Actually, you can add resources, but generally you see an increase in time and decrease in quality initially so this only works if your time horizon is long.
One approach you could take is to give the client a general estimate that you believe is relatively close. Also give a percentage. The percentage will be an increase or decrease allotment in the budget due to changing requirements.
Example: General estimate of $10,000 but since the requirements are vague and the project could change course easily there is a +/- 30% price adjustment option.
This way the client can have a general idea of what to budget and you have some flexibility in charging to the project.
This is a really tough issue. See what Robert Glass has to say on the subject in his book "Facts and Fallacies of Software Engineering". (Note that Amazon has books available new for less than they're available second-hand! [as of 2009-01-05 12:20 -08:00]; also at Google books.)
If you have successfully sold the customer an on Agile approach, and billing by the hour, do not give them an estimate for how much it will cost to complete the project!. Even if they say they want it only for budgeting purposes, do not do it!.
The reason is that any estimate of cost will come to be treated as a definite commitment; no matter how many times you say it isn't, and how many times the customer says they understand that, it will be treated as a commitment. Even if the customer understands, their boss won't, and although they won't be able to legally force you to deliver for the price you said, you will come to known as a company that doesn't deliver what it promised. Providing an estimate throws away all the advantage of being agile in the first place.
What I suggest is telling them that you will spend no more than such-and-such an amount before the end of the financial year (or whatever other billing period your customer is interested in). That should be enough for them to plan financially without in any way saying that the project will be finished by then.
This was my answer to a closed question related to story points. I use this all the time for macro estimation.
When I switched to points, I decided to it only if I could meet the two following points; 1) find and argument that justify the switch and that will convince the team 2) Find an easy method to use it.
Convincing
It took me a lot of reading on the subject but a finally found the argument that convinced me and my team: It’s nearly impossible to find two programmers that will agree on the time a task will take but the same two programmers will almost always agree on which task is the biggest when shown two different tasks.
This is the only skill you need to ‘estimate’ your backlog. Here I use the word ‘estimate’ but at this early stage it’s more like ordering the backlog from tough to easy.
Putting Points in the Backlog
This step is done with the participation of the entire scrum team.
Start dropping the stories one by one in a new spreadsheet while keeping the following order: the biggest story at the top and the smallest at the bottom. Do that until all the stories are in the list.
Now it’s time to put points on those stories. Personally I use the Poker Planning Scale (1/2,1,2,3,5,8,13,20,40,100) so this is what I will use for this example. At the bottom of that list you’ll probably have micro tasks (things that takes 4 hours or less to do). Give to every micro tasks the value of 1/2. Then continue up the list by giving the value 1 (next in the scale) to the stories until it is clear that a story is much bigger (2 instead of 1, so twice bigger). Now using the value '2', continue up the list until you find a story that should clearly have a 3 instead of a 2. Continue this process all the way to the top of the list.
NOTE: Try to keep the vast majority of the points between 1 and 13. The first time you might have a bunch of big stories (20, 40 and 100) and you’ll have to brake them down into chunks smaller or equal to 13.
That is it for the points and the original backlog. If you ever have a new story, compare it to that list to see where it fits (bigger/smaller process) and give it the value of its neighbors.
Velocity & Estimation (macro planning)
To estimate how long it will take you to go through that backlog, do the first sprint planning. Make the total of the points attached to the stories the teams picked and VOILA!, that’s your first velocity measure. You can then divide the total of points in the backlog by that velocity, to know how many sprints will be needed.
That velocity will change and settle in the first 2-3 sprints so it's always good to keep an eye on that value