I refactor my and other people's code all the time.
When I work in a branch and not in Trunk, this sometimes results in some extremely painful merges, especially if I don't merge back to Trunk regularly (the code at the branch slowly shifts away from the Trunc, and when people modify Trunk I have to figure out manually how to apply this to the branch).
The solutions I know are either
Constantly merge to and from Trunk - reduces painful merges, but then why work in a branch at all?
Whenever you need to refactor something, switch to Trunk, do the refactoring there and merge to your branch - I don't find this very practical, since the actual cost of switching environments for every refactoring is huge.
What do you do?
Refactoring on a large scale needs to be done at the right time in the development timeline. If you do huge amounts of refactoring near release you'll end up hurting yourself because you'll introduce painful merges at a time when changes should be minimized. The more disruptive your refactoring will be the earlier in the development cycle it should happen (and the more special process there should be for it, e.g. stop edits to the affected files as much as possible for a period of time).
Constantly merging to and from trunk is generally good practice.
Why work in a branch at all in that case? Because you have more control (you can stop merging into trunk to stabilize it for release, for example, without stopping checkins to your development branch). Because you can place a high level of validation around merging to/from trunk without impacting checkin velocity to the development branch much.
I go with 1, make small changes when possible and check in often or else the merges become painful. Having a separate branch can make things easier if you need to work on other things at the same time or the refactoring takes more time than you originally thought. The other bonus is that it makes it easier for several people to take part in the re-factoring and you can check things in to the branch as often as you like.
I would suggest the following strategy for a scenarios where time window between releases is at least 2 months.
When you start getting close to a release, create a release branch. Release branch should be treated as no refactoring here please and i am (almost) feature complete branch. It is at this point you should start focusing your effort on stabilising the release on the release branch. Merge back any defect fixes from the release branch onto the trunk as necessary. Meanwhile the trunk is treated as perpetually open for refactoring. Also if feasible try to reduce refactoring as you get closer to a major release and accelerate it in the days immediately after one.
In case you are following a continuous release strategy (ie. a release every 1 to 2 weeks), you should not separate refactoring and coding on separate branches, unless you are doing a major surgical enhancement. In such surgical enhancement situations (which should be spaced out no less than 3 months each), drop a release from your schedule in advance whenever you intend to perform a merge, use one of the cycles for the release merge and increased testing, keep your fingers crossed and then release.
Changes need to be either quick (so not too much changes under you) or else local (so you only care about changes in a small number of places).
Otherwise the merge can be just as much work as the refactor was. As an algorithm, optimistic locking simply doesn't work when too many transactions fail and must be restarted.
Fundamentally, you cannot allow a situation where 20 programmers in a company all change the names of 50% of the methods in the code base every day. And for that matter, if multiple people are always refactoring in the same places at the same time, then they're only undoing each other's work anyway.
If programmers are spending a lot of time manually supervising merges, then present to your managers an opportunity to increase productivity by changing the way tasks are defined and assigned.
Also, "refactor the whole system to use factories everywhere" is not a task. "Refactor this one interface and its implementations to use factories" is a task.
This is where a good distributed VCS excels. But I am guessing you are committed to SVN already.
Personally, I just do the refactor and then merge as soon as possible to avoid the conflict hell. It is not the most productive method, but the least error prone.
I once had a branch that sat dormant for about 3 weeks because the feature was 'put on hold' and it was impossible to merge. I just started the feature over again in a new branch, using the old as reference for certain parts.
At the risk of being obvious, I'd say try to avoid branching altogether. The amount of overhead this causes must not be underestimated. Even when you think you can't hold off any longer (release one of system in production, release two being built but also change requests to release one) still try to find another way: Is there really no way you can isolate functionality without branching (e.g. split off a "common" project and some subprojects)? Is there really no way you can integrate all code on the head (e.g. create Strategy classes that incorporate the differences or create switches to turn new features on or off)?
If you absolutely have to branch, I'd go with option 1. Try to merge as small changes as possible and do it frequently.
Commit early, commit often.
Or in this case... Merge early, merge often.
Continuous integration is the key... 1 small batch of changes at a time...
Related
I'm considering switching from HG to Plastic SCM (http://www.plasticscm.com, mainly because it seems to offer much nicer VS integration), and they promote "task driven branching", that is, branching from mainline for every feature. This makes sense, but, I had a few questions:
They recommend not merging your tasks back to mainline after they're completed. This seems very non-intuitive, I'd have thought that one would, after testing, want to immediately merge back to tip so that you don't have to rebase later on. Not to mention, if tasks aren't merged back, and say a new release is coming up, one needs to merge in possibly hundreds of different branches, and make sure they all play nice with each other in a short period of time (testing in independence doesn't mean they'll play nice with others, imho). So, this seems like it's bound to fail, am I wrong? Do you practice this method?
Let's say I'm wrong about the above, given the following scenario: Task A, B, C. Where B, C are dependent on A being completed, would it be better to complete A, merge it back to the mainline, and then branch from there to work on B/C, or, sub-branch your initial branch(the branch you created for A). Is that even possible? Recommended? It seems slightly cleaner in my head, if the same person is implementing A, B, C. If not, obviously, merge back to mainline makes the most sense.
Let me know what you guys think!
Thanks.
In a rather good discussion about branch strategies we had recently, jgifford25's answer contained a link to what one of Subversion's developers calls the 'agile release strategy', and which looks rather similar to what the Plastic guys are suggesting - a branch per feature, with merges into release branches rather than into the trunk. I didn't think that was a good idea, and i don't think this is a good idea. I also don't think it's a coincidence that in both cases, the idea is being pushed by a SCM developer - i think those guys have a case of "everything looks like a nail", and think any process problem can be fixed with more and bigger SCM.
So why is this idea bad? Let's follow the Plastic guys' argument. They build this process around one central idea: 'keep the mainline pristine'. So far so good. They then advance a syllogism that looks like:
If you check broken code into the trunk, the build breaks
Broken builds are bad
Therefore don't check code into the trunk
The problem with this is that it completely misunderstands why broken builds are bad. Broken builds are not bad in and of themselves (although they are unhelpful, because they stall development), they are bad because they mean that someone has checked in broken code. It's broken code that's the real problem, not broken builds - it's the broken code which actually has the potential to cause damage (lost user data, lost space probes, global thermonuclear war, that sort of thing).
Their solution, then, amounts to having people check their broken code in elsewhere, so that it doesn't break the build. This pretty obviously does nothing at all to deal with the actual problem of broken code - quite the opposite, it's a way of concealing broken code. Indeed, it's not clear to me at which point the brokenness gets detected - when the task branches are finalised and merged to the release branch? That sounds like a great way of deferring difficult work to late in your release cycle, which is a very poor idea.
The real solution, rather, is quite simply not check broken code in at all. In the pursuit of that goal, a broken build is actually good, because it tells you that there is broken code, which lets you fix it. That, in fact, is the whole flipping point of the idea of continuous integration - your merge early and often into a single trunk which is the prototype of what will actually get released, so you detect problems with what you intend to release as early as possible. That absolutely requires the 'unstable trunk' model, or something isomorphic to it.
The blog post that orangepips's answer links to mentions Ubuntu's idea about process as a driver for this idea. But look at what Shuttleworth actually said:
Keep trunk pristine
Keep features flowing
Release on demand
That's my emphasis on the last point, but it's Shuttleworth's end goal: he wants to be able to cut releases at any time. A process which defers merging and testing to the release process, as the Plastic model does, cannot possibly do this.
Rather, if you want to see what a process which can do it looks like, look at what the lean guys do: one codeline, continuous integration (on a scale of hours or even minutes, rather than days or weeks), no broken code.
So, in conclusion: don't do this. Have one codeline, and check working code into it as often as you can. Simple.
PS Okay, so you might want to make release branches to stabilise and bugfix actual releases. Ideally, you wouldn't, but you might need to.
PPS And if you have a CI test suite that is too slow to run before checking in (eg functional tests which take an hour), then something you could do with any DVCS is have two repositories: a dirty one, where developers merge into, and a clean one, which is pushed to by a script which watches the dirty repository for changes, builds and tests new versions coming into it, and pushes to the clean repository if they pass. You can then run on-demand releases (for QA and so on) from the clean repository, and developers can update from the clean repository to stay current while developing. They will obviously have to update from the dirty repository immediately before merging, though.
After reading the PR, it sounds as if they advocate for a model where code is tested before it's merged into the trunk/main/base-line (see rule #4). This presupposes a suite of unit tests and that those tests cover whatever changes have been made. For most projects I've been involved with, the suite doesn't exist and likely never will fully.
In my own experience using Subversion, the trunk is pristine, but is not what releases are made from. Instead the trunk is where back and forward ports between version flow. Releases come from version branches.
From the version branches, feature branches are created - sometimes. These branches allow for frequent commits that may break things. Once a feature branch is done, it's merged into the version; inevitably there are problems to resolve when this integration occurs. Finally once a version has been built and validated, it's merged into the trunk.
So I think #1 is not realistic. As for #2, it depends. Does it seem certain that B and C will not change A? If so, merge A back then branch for B and C. But most likely I would branch A to make B and C, because likely the latter will change the former. Then once done, roll up all three.
I'm working on a project where I have a few related bugs that are fairly minor in terms of loss of functionality. They are basically minor but annoying aesthetic problems, and based on loss of functionality should be fixed eventually, but not as a top priority. However, these bugs are caused by a fundamental, baked-in design flaw that would be nightmarish to correct.
When faced with bugs that are functionally minor, but caused by baked-in design flaws, is it generally better to treat them as showstoppers to avoid painting yourself further into a corner, or treat them as low-priority bugs, continue with functionally more important stuff and hope that you'll figure out a way around them later, when the project is more mature and fixes for minor bugs are given priority?
If the issues are minor in terms of functionality loss, and that functionality loss is not relevant in the context of the software solution being developed, I'd say you should try to check how limiting it can be to other future functionalities, and if it will not have implications within sight, go forth.
I say this because typically, when you finish a software project, no matter how small and nicely conceived, you will typically have a maturity at the end that will allow you to realize flaws and changes that you could have made in the design. Basically, if at the end you had to start it again, you would always do it differently.
I would continue as though it isn't a showstopper, but start mentally preparing for the fact that you may have to do a major rewrite eventually. If you don't end needing to do it, that's a bonus. Especially if you're working in a team or managed setting, ensure that it is clearly labeled as a risk in any future developments it could affect.
Management always wants to see the return on their investment in your time. So, management wants to see a timely release and they also want to see that you fixed x number of bugs in y hours. But, that doesn't mean you need to ignore design. As you fix the bugs, prepare for a redesign of the part in which you see the flaws. After release, be prepared to present your reasoning of why it needs to be redesigned and how long it will take you to fix it. Write this down while it's fresh and while you're fixing these bugs.
In an ideal world, we would have an infinite amount of time to constantly re-work design based on changes in requirements. In the actual world, we have to weigh the cost of changing the design against the cost of continuing with the current design. You can start by quantifying the amount of time you've spent fixing bugs in the current design and comparing that to the time you estimate it would take to re-design this component.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have two projects, with identical priorities and work hours demand, and a single developer. Two possible approaches:
Deliver one project first.
Split the developer's time and deliver both later.
I can't see any reason why people would choose the second approach. But they do. Can you explain me why?
It seems to me that this decision often comes down to office politics. One business group doesn't want to feel any less important than another, especially with identical priorities set at the top. Regardless as to how many different ways you explain why doing both at the same time is a bad idea, it seems as though the politics get in the way.
To get the best product to the users, you need to prevent developer thrashing. When the developers are thrashing, the risk of defects and length of delivery times begin to increase exponentially.
Also, if you can put your business hat on, you can try to explain to them that right now, nobody is getting any value from what the completed products will deliver. It makes more sense for the business to get the best ROI product out the door first to begin recouping the investment ASAP, while the other project will start as soon as the first is finished.
Sometimes you need to just step away from the code you have been writing for 11 hours in order to stay maximally productive. After you have been staring at the minutiae of a system you have been implementing for a long time it can become difficult to see the forest for the trees, and that is when you start to make mistakes that are hard to un-make.
I think it is best to have 2-3 current projects; one main one and 1-2 other projects that aren't on such a strict timeline.
If both projects have the same priority for the company, one obvious reason is for project managers to give higher management the illusion that both of the projects are taken care of.
Consider that the two projects could belong to different customers (or be requested by different people from higher management).
No customer wants to be told to wait while a different customer's project is given priority.
"We'll leave the other one for later" is, a lot of times, not an acceptable answer, even though this leads to delays for both projects.
I believe this is related to the notion of "Perceived Responsiveness" in a software program. Even if something takes more time to do, it looks faster when it appears to be doing something, instead of idly waiting for some other stuff to complete.
It depends on the dependencies involved. If you have another dependency upon the project that can be fulfilled when the project is not 100% complete, then it may make sense to split the developer's time. For example, if your task is large, it may make sense to have the primary developer do a design, then move on to a second task while a teammember reviews the design the primary developer came up with.
Furthermore, deserializing developers from a single task can help to alleviate boredom. Yes, there is potentially significant loss in the context switch, but if it helps keep the dev sharp, it's worth it.
if you go by whats in the great and holy book 'peopleware', you should keep your programmer on one project at a time.
the main reason for this is that divided attention will reduce productivity.
unfortunately, because so many operational managements are good businessman rather then good managers, they may think that multitasking or working on both projects somehow means more things are getting done (which is impossible, a person can only physically exists in one stream of the space-time continuum at one time).
hope that helps :)
LM
I think the number 1 reason from a management standpoint is for perceived progress. If you work on more than one project at the same time stakeholders are able to see progress immediately. If you hold one project off then the stakeholders of that project may not like that nothing is being worked on.
Working on more than 1 project also minimizes risks somewhat. For example if you work on one project first and that project takes longer than expected you could run into issues with the second project. Stakeholder also most likely want their project done now. Holding one off due to another project can make them reconsider going ahead with the project.
Depending on what the projects are you might be able to leverage work done in one for the other. If they are similar then doing both at the same time could be of benefit. If you do them in sequence only the subsequent projects can benefit from the previous ones.
Most often projects are not a constant stream of work. Sometimes developers are busy and sometimes not. If you only work on 1 project at a time a developer and other team members would likely be doing nothing while the more 'administrative' tasks are taking place. Managing the time over more than one project allows teams to get more done in a shorter timeframe.
As a developer I prefer working on multiple projects as long as the timelines are reasonable. As long as I'm not being asked to do both at the same time with no change in the schedule I am fine. Often if I'm stuck on one project I can work on the other. It depends on the projects though.
I'd personally prefer the former but management might want to see progress in both projects. You might also recognise inaccurate estimates earlier if you are doing some work on both, enabling you to inform the customer earlier.
So from a development perspective 1 is the best option but from a customer service point of view 2 is probably better.
It's managing your clients expectations; if you can tell both clients you are working on their project but it will take a little longer due to other projects then to say we are putting your project off till we finish this other project the client is going to jump ship and find someone that can start working on their project now.
It's a plaecbo effect - splitting a developer between two projects in the manner you've described gives people/"the business" the impression that work is being completed on both projects (at the same rate/cost/time), whilst in reality it's probably a lot more inefficient, since context switching and other considerations carries a cost (in time and effort).
On one hand, it can get the ball rolling on things like requirement clarifications and similar tasks (so the developer can switch to the alternate project when they are blocked) and it can also lead to early input from other business units/stakeholders etc.
Ultimately though, if you have one resource then you have a natural bottleneck.
The best thing you can do for that lone developer is to intercept people( from distracting that person), and try to carry some of the burdon around requirements, chasing clarifications and handling user feedback etc.
The only time I'd ever purposely pull a developer off their main project is if they would be an asset to the second project, and the second project was stalled for some reason. If allowing a developer to split a portion of their time could help jump-start a stalled project, I'd do that. This has happened to me with "expert" developers - the ones who have a lot more experience/specialized skills/etc.
That being said, I would try to keep the developer on two projects for as little time as possible, and bring them back to their "main" project. I prefer to allow people to focus on one task at a time. I feel that my job as a manager is to balance and shift people's priorities and focus - and developers should just develop as much as possible.
There are three real-life advantages of splitting developers' time between projects that cannot be ignored:
Specialisation: doing or consulting on work that requires similar specialised knowledge in both projects.
Consistency and knowledge sharing: bringing consistency into the way two separate products are built and work, spreading knowledge accross the company.
Better team utilisation: on a rare occasion when one of the projects is temporarily on hold waiting for some further input.
Splitting time between several projects is beneficial when it does not involve a significant change in context.
Having a developer to work single-handedly on multiple software development projects negates the benefit of specialisation (there isn't any in the case), consistency and knowledge sharing.
It leaves just the advantage of time utilisation, however if contexts differ significantly and there is no considerable overlap between projects the overhead of switching will very likely exceed any time saved.
Context switching is a very interesting beast: contrary to its name implying a discreet change the process is always gradual. There are various degrees of having context information in one’s head: 10% context (shallow), 90% (deep). It takes less time to shallow-switch as opposed to fully-switch; however there is a direct correlation between the amount of context loaded (concentration on the task) and output quality.
It’s possible to fill your time entirely working on multiple distinct projects relying on shallow-switching (to reduce the lead time), but the output quality will inevitably suffer. At some point it’s not only “non-functional” aspects of quality (system security, usability, performance) that will degrade, but also functional (system failing to accomplish its job, functional failures).
By splitting the time between two projects, you can reduce the risk of delaying one project because of another.
Let's assume the estimate for both projects is 3 months each. By doing it serially, one after the other, you should be able to deliver the first project after 3 months, the second project 3 months later (i.e. after 6 months). But, as things go in software development, chances are that the first project encounters some problems so it takes 12 months instead. Or, even worse, goes into the "in use, but never quite finished" purgatory. The second project starts late or even never!
By splitting resources, you avoid this problem. If everything goes well with the second project, you are able to deliver it after 6 months, no matter how well the first project does.
The real life situations where working on multiple projects can be an advantage is in the case where the spec is unclear (every time) and the customer is often unavailable for clarification. In those cases you can just switch to the other project.
This will cause some task switching and should be avoided in a perfect world, but then again...
This is basically my professional life in a nutshell :-)
Having read Fowler's "Refactoring" for a while, I still often catch myself thinking "I should have done this in smaller steps." -- even when I did not broke my code.
Refactoring in small steps is safe, but cost time. It's a trade off between speed and risk -- I try to be strategic in choosing the way how I am refactoring.
Nevertheless: Most the time I am doing refactorings in larger steps. If I took some of Fowler's "Mechanics" section and compare how I am working, I maybe find that I often leap two or five steps forward at once. This does not mean that I am a refactoring guru. My code maybe stay for 5 - 60 minutes broken or uncompilable.
Do you refactor in smaller steps and try to produce unbroken code in shorter frequencies? And: Are you successful in doing this?
Martin Fowler seems to lean towards the small, gradual refactoring approach. However, after reading his book he does occasionally make some drastic steps but only with unit tests to back up the code.
Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth doing". However the cumulative effect of each of these transformations is quite significant. By doing them in small steps you reduce the risk of introducing errors. You also avoid having the system broken while you are carrying out the restructuring - which allows you to gradually refactor a system over an extended period of time. - Martin Fowler
I try :) The one urge I have to resist most while refactoring is actually making other changes along the way. Say I'm refactoring some code and see something unrelated in the code. I have to make a conscious effort not to go "fix" that as well. Make a note of it and move on. For one thing, it's a distraction from the task at hand. It also ends up polluting your change set so your commit message now has to document several seemingly random changes.
Yes, always. I think the real essence of refactoring is picking which steps to start with.
I find the thing with refactoring large changes in a safe manner is always to have a reasonably clear picture of where you want to go. Then consider your existing system and try to find out which pieces you can introduce that have least likelyhood of being a radical change. Then you can introduce these in a controlled and well tested manner.
So what you do is to work in the vincinity of the nastiness. Not always attacking directly from the front, but sometimes just chipping away small pieces. Usually I wait, and only go for the "big prize" after a few rounds of chipping away at minor nastiness. But I know where I want to go.
The nice thing about working this way is that you can maintain progress. You never "stop development to do refactoring". Arguably there are cases where stopping is the correct situation, but most of the time it's not.
The idea here is that if you "start" with cashing in the prize money, you will be spending the next X days doing the drudgery. And there's risk, maybe you chicken out or it doesn't work - or spend 6 months instead of a week. If you do the drudgery first, cashing in the prize will be possible with less risk. And your code will improve as you go. Sometimes you can decide that doing half the job was enough, since your understanding of the problem increases. Sometimes your idea of where you wanted to go was slightly botched, and you can realign your goal as you progress.
But its tempting to go straight for the reward.
I tend to refactor in large steps most of the time so I can see the forest from the trees. It's a "stream of consciousness" kind of programming. As long as you have your last working version safe in your source control of choice...
that's where the "red, green, refactor" approach is useful. At each stage you have the ability to verify that your code's behaviour is unchanged, and the refactoring only has to integrate the new behaviour.
The rule of thumb I use is refactor with tests and only refactor as much code as you are confident too.
At 60 minutes are you certain that your code is doing exactly what it should be. You'd need a lot of tests to pass. I would just try and get one going and then move on to the next.
If I have a clear picture of what I want to do, and if I can easily verify that I haven't broken anything afterwards, I am taking larger steps.
If the refactoring is more complicated, I try to break it down into smaller steps and do heavy testing after each one.
I usually refactor code as I change it. That is, instead of taking a piece of code and rewriting it while maintaining its function, I rewrite it towards a new functionality and in the process of doing so I improve the design of the code.
Often this means that by the time I've implemented the feature I was after I haven't done a complete and satisfactory refactoring of the old code. It is improved though, and I know I'll have the time to improve it further the next time I'm about to change its function.
For testing this means that I get to test both the refactoring and the new feature at the same time, which should save some time.
It also mean that I only spend enough time on refactoring to improve the maintenance situation required for that particular feature. This should help to avoid over engineering and/or wasting time refactoring stuff that already works and wont benefit from a better desing. By focusing only on code I would change anyway, there is also a high probability I will revisit that code in the near time to do further changes while it's in the users attention span.
Small discrete steps is what I'm most comfortable with, though at some points it can be a test of my self-control to reign in what could be a refactoring blood-bath. If I notice any improvements (no-matter how large) that could be made, I make a note of them and consider how it'd be split up into individual refactoring tasks. Plus, having a saga of changes in the commit message doesn't help.
NB. The code-base I work on is quite old, and full of those mystical bugs named after scientists. With large portions still lacking anything near even 50% test coverage it would be careless to get carried away.
Yep. I like to run the tests continually and so a chain of tiny refactors works well. I get really uncomfortable having my code broken for more than a few minutes at a time, and I generally revert if my code is broken when I go home at night, the re-write the next morning ALWAYS works better than trying to pick up where I was.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
As the title suggest...
How can I apply a scrum process to anything that doesn't work on new code and can be estimated to some degree?
How can I apply a scrum process to maintenance and emergency fixes (which can take from 5 minutes to 2 weeks to fix) type of environment when I still would like to plan to do things?
Basically, how do I overcome unplanned tasks and tasks that are very difficult to estimate with the scrum process? or am I simply applying the wrong process for this enviroment?
Basically, how do I overcome unplanned tasks and tasks that are very difficult to estimate with the scrum process? or am I simply applying the wrong process for this enviroment?
You're using the wrong process for this environment. What you need is a stack/queue management process which is separate to your planned/estimated SCRUM development process.
The reason for this is simple and twofold:
As you mention in your post, it is
often very difficult to estimate
maintenance tasks, especially where
legacy systems are involved.
Maintenance tasks in general and
legacy systems specifically have a
tendency to involve 'curly'
problems, or have a long 'tail',
where one seemingly simple fix
requires a slightly more difficult
change to another component, which in
turn requires an overhaul of the
operation of some subsystem, which in
turn... you get the point.
Quite often when dealing with
maintenance tasks, by the time you
have finished estimating, you have
also finished solving the problem.
This makes the process of estimation
redundant as a planning tool. Those who insist on dividing estimation from solving the problem
for maintenance tasks are
simply adding unnecessary overhead.
Put simply, you need a queueing system. It will have these components:
A 'pool' of tasks which have been
identified as requiring attention.
Newly raised items should always go
into the pool, never the queue.
A process of moving these tasks out of
the pool and onto the queue. Usually a combination of business/technical knowledge is required.
A queue of tasks which are clearly ordered
such that developers responsible for
servicing the queue can simply pick
from the front of it.
A method for moving items around in the queue
(re-prioritising). To allow 'jumping the queue' for critical/emergency items.
A method for delivering the completed items which does not interrupt servicing the queue. This is important because the overhead of delivering maintenance items is usually significantly lower than development work. You don't want your maintenance team sitting around for a day waiting for the build and test teams to give them the ok each time they deliver a bugfix.
There are other nuances to queue management, but getting these in place should set you on the right path.
If you have that much churn in your environment, then your key is going to be shorter iterations. I've heard of teams doing daily iterations. You can also move towards a Kanban type style where you have a queue which has a fixed limit (usually very low, like 2 or 3 items) and no more items can be added until those are done.
What I'd do is try out one week iterations with the daily stand-ups, backlog prioritization, and "done, done". Then reevaluate after 5 or 6 weeks to see what could be improved. Don't be afraid to try the process as is - and don't be afraid to tweak it to your environment once you've tried it.
There was also a PDF called Agile for Support and Operations in 5 minutes that was recently posted to the Scrum Development list on Yahoo!
No one said that backlog items have to be new code. Maintenance work, whether bug fixes, enhancements or data fixes can be put into the Product Backlog, estimated and prioritized. This is actually one of the biggest benefits of using Scrum - no more arguments with users about whether something is a bug fix or an enhancement.
With Waterfall, there's a tacit understanding that bugs are the responsibility of the developers. Somehow, they are on the hook to fix them without impacting the development of new code and features. So they are "free" to the users, but a massive inconvenience to the developers.
In Scrum, you recognize that all work takes time. There is no "free". So the developers freely accept that something is a bug but it still goes into the Product Backlog. Then it's up to the Customer to decide if fixing the bug is more important than adding new features. There are some bugs that you can live with.
As the title suggest... How can I
apply a scrum process to anything that
doesn't work on new code and can be
estimated to some degree?
On the contrary, I've heard teams find adopting Scrum easier in the maintenance phase.. because the changes are smaller (no grand design changes) and hence easier to estimate. Any new change request is added to the product backlog, estimated by devs and then prioritized by the product owner.
How can I apply a scrum process to
maintenance and emergency fixes (which
can take from 5 minutes to 2 weeks to
fix) type of environment when I still
would like to plan to do things?
If you're hinting at fire-fighting type of activity, keep a portion of the iteration work quota for such activities.. Based on historical trends/activity, you should be able to say e.g. we have a velocity of 10 story points per iteration (4 person-team 5day-iteration). Each of us spend about a day a week responding to emergencies. So we should only pick 8 points worth of backlog items for the next iteration to be realistic. If we don't have emergency issues, we'll pick up the next top item from the prioritized backlog.
CoryFoy mentions a more dynamic/real-time approach with kanban post-its in his response.
Basically, how do I overcome unplanned
tasks and tasks that are very
difficult to estimate with the scrum
process? or am I simply applying the
wrong process for this enviroment?
AFAIR Scrum doesn't mandate an estimation technique.. Use one that the team is most comfortable with.. man days / story points / etc. The only way to get better at estimation is practice and experience I believe. The more the same set of people sit together to estimate new tasks, the better their estimates get. In a maintenance kind of environment, I would assume that it's easier to estimate because the system is more or less well known to the group. If not, schedule/use spikes to get more clarity.
I sense that you're attempting to eat an elephant here.. I'd suggest the following bites
Working Effectively with Legacy code
Agile Estimating and Planning
Agile Project Development with Scrum
Treat all fixes and improvements as individual stories and estimate accordingly. My personal view is that things that take less then 10-15 minutes to fix should just be done straight away. For those that take longer, they become part of the current 'fix & improvement' iteration cycle. As with estimating regular requirements, you take a best guess as a team. As more information comes to light, and the estimates are off adjust the iteration and upcoming sprints.
It's hard to apply a iteration cycle to fixes and improvements as more often then not, they prevent the system from working as they should and the 'business' puts pressure for them to go live asap. At this point it may work out better moving to a really short iteration cycle, like one or two weeks.
You ask about how to use a process for emergencies. Try to reserve the word "emergency" for things that require hacking the code in the production environment with the users connected at the same time. Otherwise, stakeholders are likely to abuse the word and call an emergency on anything they would like to have really fast. Lack of process does not mean out of control: somebody must be accountable for declaring the emergency, and somebody (best practice is somebody else) must authorize the changes outside the normal process and then take responsibility for it.
Other than that, the suggestion of using each iteration to complete a number of fixes and improvements is probably the best way to go.
This depends a lot on the application life cycle. If this is a 'Sunset' application to be retried soon, of course, the focus would only be on fixing the top priority bugs.
If the product is 'Mature' and has a roadmap and is continuing to evolve you would have fixes and enhancements to take care of. There's a lot of impetus to keep the design clean and evolve by refactoring. This may mean periodic minor and major releases [except for eFixes - emergency fixes/hotfixes]. You can pratice agile to your hearts delight as enhancement and fixes could be story boarded and made part of your Sprint backlog. The entire list would make your Product Backlog.
Bottom line: If you want to refactor and keep your application design clean [programmers tend to take shortcut if the focus is exclusively bug fixing], it could only happen with a 'living' application. One that is evolved and updated. And agile is natural fit.
Even if you have only fixes (that it's a Turing Complete ;) or Sunset application), it helps if they can all be rolled into a sprint and rolled into production end of each sprint. If the fixes need be rolled into production as and when they're fixed, it's much more difficult to apply Scrum.
We have applied scrum in this context.
Some of the keys of success.
1. Everyone in the enterprise buy the scrum idea ( this is crucial for success )
2. Sprint of about 2 weeks (our 2-3 firsts sprint where of 1 week to understand the process)
3. Under no circumstance a point could be added to the current sprint
4. If a real emergency arise stop the sprint, do a retrospective and start a new sprint.
5. Take time for retrospection (time to share though about the last sprint and to analyze it)
6. In a sprint insert at least one task to improve the process (often added to the backlog in the retrospective); it's good for the troop morale and in the end of the day you will be in your way to have less emergency.
7. TIME BOXED daily stand up meeting
For the estimation, usually the more you estimate the more precise you become. What is good with Scrum is that each programmer pick his task and can set a new estimate if he think it's not realist. And if you still have some issue with estimation, let your team find a solution... you can be surprise with what they come with.
For the 2 weeks fix. If it's the original estimation, cut it in smaller pieces. If you did an more optimistic estimation (let say 2-3 days), the issue should rise as a blocker in the stand up meeting. Maybe somebody else have ideas about how to fix it. You can decide to do some pair programming to find a solution. Sometime, just to describe the bug to another programmer help a lot to debug. You can also delay it after other task in the sprint. The idea is to deliver fully functional tasks. If you don't have time to fix it in full and to demonstrate it, even if your at 90% done (yeah! we know what it means), you consider it not done in the sprint. In the next sprint you will be able to address it with the correct time estimation.
Finally, from what I understood, Scrum is more about having "tools" to improve your process. You start with something simple. You do small iteration. In each iteration you have a FIX TARGET to complete instead of an infinite bug list. Because you pick your tasks from the list in the planning (oppose to being assign them), you become more engage to deliver it. With the stand up meeting, you meet your pair every day with your TODO list... you want to respect your engagement you did the day before. With the iteration, you take the time to talk to each other and to identified what's going good and what's should be improve. You also take action to improve it and to continue doing what's working. Don't be afraid to change anything, even what I said ;) / even any basic of the Scrum itself... The real key is to adapt Scrum to what your team need to be happy and productive. You will not see it after one iteration, but many of them....
I'd highly recommend looking at what value sprints/iterations would give you. It makes sense to apply them when there are enough tasks to do that they need to be prioritized and when your stakeholders need to know roughly when something will be done.
In this case I'd highly recommend to combine three approaches:
schedule as many incoming tasks as possible for the next iteration earliest
use Yesterday's Weather to plan for how much buffer you need to plan for to deal with tasks that have to be dealt with immediately
use very short sprints, so as to maximize the number of tasks that can wait until at least the start of the next iteration
In fact that is true vor every Agile/Scrum project, as they are in maintenance mode - adding to an existing, working system - from iteration 2.
In case that iterations don't provide enough value, you might want to take a look at a kanban/queuing system instead. Basically, when you are finished with a task, just pull the next task from a prioritized task queue.
In my opinion it depends on how often you have a 'real' release. In our specific case we have one major release each year and some minor releases during the year.
This means that when a sprint is done, it's not immediately deployed on our production server. Most of the times a few sprints will take place before we have our complete 'project' finished. Of course we demo our sprints and we deploy our sprints on our testing-server. The 'project' in his totality will undergo some end-to-end testing and will finally be deployed to our production servers -> this is a minor release. We may decide that we will not deploy it immediately to our production server, when it's for instance dependant on other products/projects that need to be upgraden first. We then deploy this in our major release.
But when issues arrise on our production server, immediate action may be required. So, no time to ask a product owner for the goal or importance (if we even have one for sunc an issues) because it blocks our clients from working with our application. In such an urgent cases, these kind of issues will not be put into a Product Backlog or sprint but are pure maintenance tasks to be solved, tested and deployed as soon as possible and as an individual item.
How do we combine this with our Sprint? In order to keep our Team members focused on the sprint, we decide to 'opt in - opt out' our people into the Team. This means that one or more people will not be part of a Team for a certain Sprint and can focus on other jobs like these urgent fixes. The next Sprint this person will again be part of the Team and someone else will be responsable for emergency calls.
Another option could be that we foresee like 20% of the time in a Sprint for 'unplanned tasks' but this will give wrong indication about the amout of work we can do in a Sprint (we will not have the same amount of urgent fixes during each sprint). We also want our Team members to be focused on the Sprint and doing these urgent fixes in a Sprint will distract our team members. 'context-switching' will also mean time-loss and we try to avoid that.
It all depends on your 'environment' and on how fast urgent issues should be fixed.
Treat all "bug fixes" that don't have a story as new code. Estimate them, and work them as normal. By treating them as new stories you will build up a library of stories and tests. Only then can you begin to pin the behavior of the application.
Take a look at Working Effectively with Legacy Code by Michael Feathers. Here is a link to an excerpt. http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf
-Jason
I have had some success by recognizing that some percentage of a Sprint consists of those unplanned "fires" that need to be addressed. Even in a short sprint, these can happen. If the development team has these responsibilities, then during sprint planning, only stories are committed to the sprint that allow enough headroom for these other unplanned activities to occur and be handled as needed. If during the sprint, no "fires" ignite, then the tam can pull in stories from the top of the backlog. In many ways, it becomes a queue.
The advantage is that there is commitment to the backlog. The disadvantage is that there is this hole in capacity that can be communicated as an opportunity to drag the team into non-critical tasks. Velocity can also vary widely if that gap in capacity is filled with unplanned work that is not also tracked.
One way I have gotten around part of this is to create a "support" story that fills out the rest of that capacity with a task that represents the available hours the team can allocate to support activities. As support situations enter the sprint that cannot be deferred to the backlog, then a new story is created with tasks, and the placeholder support story and tasks are re-estimated to account for the new injection. If support incidents do not enter the sprint, then this support story is reduced as backlog items come in to fill the unused capacity.
The result is that sprints retain some of the desired flow and people feel like they aren't going to get burned when they need to take on supporting work. Velocities are more consistent, burndowns track normally, but there is also a record of the support injections that happen each sprint. Then during the sprint retrospective we can evaluate the planned and unplanned work and if action needs to be taken differently next sprint, we can do so. If that means a new sprint duration, or a pointed conversation with someone in the organization, then we have data to back up the new decisions.