How to handle short coding sessions? - time

I'm working on some pet projects and generally I sit around my personal computer about 22:30 or 23:00 to code. But since I try to sleep about 24:00 I don't start coding and ending up reading articles, playing some games etc.
I don't feel like I can write decent code in an hour, because the project is quite big and I don't want to randomly or carelessly hack it. Even though I use TDD, most of the time stuff I'm doing is not straight forward which requires lots of testing before getting it right.
What's your approach to these kind of issues? Do you just code later when you got enough time or do you have a different approach which allows you to code just for 30 minutes and continue later?

I generally don't write lots of code until i have the time to do it. The reason is that for me to get effective takes focus and that takes a bit of time to be correctly focused. That said those 30min slots are great for
Writing more tests: nothing like trying to get to 100% code coverage, and it's not a big waste since you are investing
Research: I spend lots of time reading blogs, looking for frameworks I can use or tools. Spending 30min finding a framework that does 80% of a feature you need is much better than spending hours trying to code it. The other factor to this is that if you implement the framework and you find it is a bad fit you are better educated in the needs which means your development will be smoother.

Well my first thought was "use unit testing", but then I read you are already using this. But I still think it's the solution to your program.
Try to make your tests as small as possible and use the "1 assert per unit test" rule to create small atomic tests. You should be able to fix several of these small tests in a 30-minute session.

Here are some things you could try:
Don't sit near the computer. Instead, take a large piece of paper and go somewhere quiet. Think about what you want to accomplish. Write down interface ideas, detailed implementation. Make a list of questions you need to solve before you can go on.
Take off a week and code away. The ratio of getting-into-flow over flow-time is just too bad for 30 minutes.
Keep a log about what you do instead of coding. Observe your emotional state.
Go to bed early and try to have your pet coding session very early in the morning.

A small tip (that I use at work too) is to stop coding in the middle of something, with an obvious big red compile error waiting.
The next time you start working, the error will actually help you to remember what on earth you were doing.
While you are working on the small problem, the big picture clears up and then you can continue designing.

Developing with such short times is difficult, but you can still get something of that time. Unit testing is one. Writing down the interface of a class is another. While coding the real stuff will take much more time, these tasks are essentially a no-brainer, and they are just an exercise in typing.
So, my suggestion is: focus on small tasks that do not require thinking and concentration, and can be completed in the timespan you have.

Never worked for me, pet projects are usually too interesting and I end up working to late hours of the night or through a weekend.
I would suggest reconsidering your priorities - if all the time you have available is one hour late at night, maybe it's better to spend it on games, articles etc. Or just hanging out. When you have a bit more time, say a lazy Sunday, spend all the time at once and actually get the sense of accomplishing something that pet projects are supposed to give.

Here's what I do with my personal projects outside of work:
1) I try to give myself a good map of my project by planning it out on paper. I diagram all of my objects, data structures, and/or SQL tables and determine what basic functions and interactions between those components are necessary. I may write some actual code during this phase if the solution is obvious, but typically not.
2) Once the big picture is in place, I prioritize the most basic and critical elements. I also try to figure out which parts will be easier to write than others.
3) After priorities have been set I start working on the easiest and most critical parts first and work gradually towards the more complex and less important components. Breaking each task down into smaller parts tends to help. For example, I may design a database table first and the next day create a data interface class that controls interactions with that table.
4) Unit testing really helps me feel a sense of accomplishment, even if 30 minutes of effort only results in a few quality lines code.
5) Keep a change log, even if it isn't very detailed. I've found my change logs to be invaluable if I work on a big project in many short spurts over an extended period of time.
Those steps right there help me the most. In the end I am able to identify small chuncks of the project that can typically be completed in about 30-60 minutes. Of course, as a project develops, I usually have to re-evaluate something and go back to the beginning for a while when I discover I left something out of the planning phases. Sometimes I need to go a bit farther and give myself a time line with some deadlines and make sure I celebrate milestones with a personal reward. If you have a tendency of staying up till the wee hours of the morning, something I struggle with, I also suggest giving yourself a coding curfew as well. I also try to make sure my "coding computer" doesn't have many distractions on it, such as games.

There's always one more bug. If not, there's one more neat feature you can add, and THAT will add more bugs. Which is one reason why I think that the use of the phrase "All you have to do..." by anyone in (or using) IT should be a hanging offense.
I can cut down on how long my coding sessions last by doing trivial things, thinking things out while away from the keyboard (the shower is best - or while in bed at 4 a.m.) and by using lightweight environments such as script languages, but "quick" coding sessions are something I long ago gave up hope on.
Just shifting mental gears into the coding mode takes time, picking up the threads of where I was before takes time, discovering that my "quick and easy solution" was neither of the above takes time. Fixing my "quick and easy solution takes time, debugging - more time, and so forth.

Related

How not to rush yourself? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I often find that I do a less than complete work on a feature, especially in the Design phase. I detect several reasons:
I'm over-optimistic
I feel the need to provide quick solutions, so sometimes I fool myself into thinking the design is fool-proof when in fact it's still full of holes, just to get the job done faster. Of course I end up paying dearly later.
I'm aware of this behavior of mine for some time, yet I still find I don't manage to compensate. Have you encountered similar problems? How do you approach solving them?
I use a couple of techniques. The first is a simple paper to-do list. In the morning I write down my tasks for the day. I try to work on a task until I can cross it off. I cross it off only when I'm done to my own satisfaction. My to-do list helps me stay focused. When an interruption comes in, I can consciously choose whether it is important enough to interrupt what I'm doing now.
The second technique I use is to give up on the idea of "done" for a design. Instead, I focus on what I've started calling "successions", where a design goes through predictable stages. Each stage supports the current functionality well and will be succeeded at some point by the next stage. This lets me do a good job, a job I can be proud of, without over-designing.
I have the intuition that there is a small catalog of such successions (like http://www.threeriversinstitute.org/FirstOneThenMany.html) that would cover most of design. In the meantime, I try to remember that "sufficient to the day are the troubles thereof".
I run into this problem a lot.
My solution is a notebook. (The old fashioned paper kind).
I write out how I'm planning on implementing the solution as an bulleted overview list, and then I try and flesh out each point on the list.
Often, during that process, I come across issues I hadn't thought of.
Of course, the 80/20 rule still applies... I still come across things when I'm actually doing the implementation that hadn't occurred to me, but with experience these tend to diminish.
EDIT: If I'm still not sure at the end of this process, I put together a throwaway prototype testbed... It's important to make sure it's throwaway, because otherwise you run the risk of including some nasty hacks in your real codebase.
It's very common to miss edge-cases and detail when you're in the planning phase of a project, especially in the software development field. Please don't feel that this is a personal failing; it's something endemic.
To counter this, many software development methodologies have emerged. Most recently there has been a shift by many development teams to 'agile' methods, where there is a focus on rapid development with little up-front technical design (after all, many complexities are only discovered when you actually begin developing). I'm currently using the Scrum system, which has been excellent in my small team:
http://en.wikipedia.org/wiki/Agile_methods
http://en.wikipedia.org/wiki/Scrum_%28development%29
If you find that your organisation will not accept what they may regard as a radical shift in approach, it may be worth investigating whether they will agree to the development of a prototype system. This means that you could code up a feature to investigate the technologies involved and judge whether it's feasible, without having to commit to full development, a quality bar, testing schedules etc. The prototype should be thrown away once the feasibility has been proved or disproved, then proper development may begin, including all that you've learned in the process.
If your problem is more related to time management, then I'd recommend the Getting Things Done approach (http://en.wikipedia.org/wiki/Getting_things_done). This is pragmatic and simple, concentrating on making you productive without overloading you with information that isn't immediately relevant to your current work. I've found that I get overwhelmed with project/feature ideas at times and it really helps to write everything down and file it for a later time when I have the resources available to work effectively.
I hope this helps and best of luck!
Communication.
The best way to not rush yourself into programming mistakes is communication. Yes, good ol' fashioned accountability. If another person in the office is involved in the process, the better the outcome. If a programmer just takes on the task without any concern for anybody else, then there is a higher possiblity for mistakes.
Accountability Checklist:
How do we support this?
Who needs to know what has changed?
Why are we doing this in the first place?
Will there be anybody who doesn't want this changed?
Will someone else understand how I did this?
How will the user perceive and use this change?
A skepticle comrad is usually good enough to help. Functional Specifications are good, they usually answer all of these thoughts. But, sometimes a conversation with another person can help you with it and you can get changes out the door faster.
I have learned, through years of mistakes (though still making them), that almost anything I want to use repeatedly, or distribute, needs to be designed properly. So getting burned enough times will end your optimism.
When getting pressure from management, I tell them I will have to put in the thought anyway, so I should do it when it's cheap. I think on paper as well, so I can actually prove that I'm doing something and it keeps my fingers on the keyboard, both of which provides a soothing effect to management. ;-)
At the risk of sounding obvious - be pessimistic. I had a few experiences where I thought "that should take a few hours" and it ended up taking a couple days because of all the little things that pop up unexpectedly.
By far the best way I've found to manage things is to (much like Andrew's answer) write out the design and requirements as a starting point. Then I go through and look for weak points in the design, gotchas and additional use cases etc. I try to look at this as a critical exercise - there's no code written yet, so this is the time to be totally ruthless and look for every weak point. Look for error conditions you'll have to handle, and whatever amount of time you think it will take to complete each feature/function, pad that amount by a lot. I've had times where I've doubled my initial estimate and still not been that far off the mark.
It's very hard as a programmer to realistically project debugging time - writing the code is easy to estimate, but debugging that into functioning, valid code is something else entirely. Therefore I find there's no exact science to it but I just pad tasks by a whole bunch, so that I have plenty of breathing room for debugging.
See also Evidence Based Scheduling which is a fascinating concept in scheduling developed by FogCreek for their FogBugz product.
You and the rest of the world.
You need more a more detailed design, more accurate estimate, and the willingness to accept that sometimes the optimal solution is not necessarily the best solution (e.g., you could code some loop in assembler to get optimal performance, but that's going to take a lot longer than just doing
for (i=1; i<=10; i++) {}
). Is the time spent doing it really worth it for an accounting package over a missile system.
I like to designing, but over time I've found that much design up front is a lot like building castles into the sky - it's too much speculation, however well-educated, missing critical feedback from actually implementing and using the design.
So today I'm much more into accepting that while implementing a design I will learn a lot of new stuff about it, and need to feed that learning back into the design. Doing that is a skill that is fun to learn, including the skills to keep a design flexible by keeping it simple, free of duplication and cohesive and decoupled, of changing the design in small, controlled steps (=refactoring), and writing the necessary extensive suite of automated tests that make this kind of changes safe.
This seems to be a much more effective approach to me than getting better at "up front design speculation" - and addtionally it makes me equally well prepared for the inevitable moment when the design needs to be changed due to a simply unforseeable change in the requirements.
Divide, divide, divide. List all the steps that will be required to finish the project, then list all the steps those steps will require to be concluded, and so on until you reach atomic items you are absolutely sure you can finish in a day or less. Add the duration of all these values to arrive at a length of time.
Then double it. Now you have a number that, if depressing, is at least somewhat realistic.
If possible "Sleep on your design" before publishing it. I find after I leave work, I usually think of things I have missed. This usually happens while I am lying in bed before falling asleep or even while showering the next day.
I also find it valuable to have a peer/friend that I trust review what I have before distributing it. Somebody else almost always sees something I didn't think of or miscommunicated.
I like to do as others stated here. Write down in pseudo code what the flow of your app will be. This immediately highlights some detailed areas that may require further attention that where not apparent up front.
Pseudo code is also readable to business users who can verify your approach meets their needs.
Using pseudo code also creates a nice set of methods that could be put to use as an interface in the final solution. Once the pseudo code is fairly tight, look for patterns and review some common GOF patterns. They do not have to be perfect but using them will sheild you from having to rewrite the code later during the revisions that are bound to come along.
Just taking an hour or two write psuedo code, yields some invaluable time saving pieces later on:
1. An object model emerges
2. The program's flow is clearly defined for others
3. It can be used as documentation of your design with some refinement
4. Comments are easier to add and will be clearer for someone else reviewing your code.
Best of luck to you!
I've found that the best way to make sure you've chosen a good design is to make sure that you understand the problem, know the limitations you have, and know what things are must-haves vs. nice-to-haves.
Understanding the problem will involve talking to the people who have the need and keeping them anchored to what needs to get done first instead of how they think it ought to get done. Once you know what actually has to happen, you can go back and talk over requirements about how.
Knowing your limitations may be quite easy: needs to run on the iPhone; has to be a web application; needs to integrate with the already-existing Java code and deployment setup; and so on. It may be quite difficult: you don't know what the potential size of your user base is (hundreds? thousands? millions?); you don't know whether you'll need to localize it (though if you're not sure, assume you will have to).
Must-haves vs, nice-to-haves: this is possibly the most difficult part. Users very often have emotional attachments to "requirements" ("It should look just like Excel") that are not actually part of the "has to happen" stuff. You often have to juggle functionality vs. desires to get an acceptable implementation. You can't always give everyone a pony.
Make sure you write all this down! Even if it evolves along the way, or the design is small, having a "this is what we're planning to do now" guide to refer to when you need ot make a decision about committing resources makes it easier to restrain yourself from implementing a really cool whiz-bang feature instead of a boring must-do.
Since you recognize that you feel the need to provide a quick solution, perhaps it will slow you down to realize that you can probably solve the problem faster and deliver it sooner if you spend more upfront time in design. For instance if you spend 3 hours designing and 30 hours writting code, it probably means that if you spend 6 hours designing you might need to only spend 10 hours writing code. (These are not actual figures just examples). You might try to quantify this for yourself on the next few projects you do. Do a couple where you behave as you normally would and see what ratio of design/codewriting/testing&debugging you actually do. Then on the next project deliberately increase the percentage of time you spend on design phase and see if it does shorten the time needed for the other phases. You will have to try for several projects on this as well to get a true baseline since the projects may be quite different. Do it as a test to see if you can improve your performance on the the other phases and thus deliver a faster product if you spend 20% more time or 50% more time or 100% more time on design.
Remember the later in the process you find the problem with a design the harder (and more time-consuming) it is to fix.

Do you refactor in small steps?

Having read Fowler's "Refactoring" for a while, I still often catch myself thinking "I should have done this in smaller steps." -- even when I did not broke my code.
Refactoring in small steps is safe, but cost time. It's a trade off between speed and risk -- I try to be strategic in choosing the way how I am refactoring.
Nevertheless: Most the time I am doing refactorings in larger steps. If I took some of Fowler's "Mechanics" section and compare how I am working, I maybe find that I often leap two or five steps forward at once. This does not mean that I am a refactoring guru. My code maybe stay for 5 - 60 minutes broken or uncompilable.
Do you refactor in smaller steps and try to produce unbroken code in shorter frequencies? And: Are you successful in doing this?
Martin Fowler seems to lean towards the small, gradual refactoring approach. However, after reading his book he does occasionally make some drastic steps but only with unit tests to back up the code.
Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth doing". However the cumulative effect of each of these transformations is quite significant. By doing them in small steps you reduce the risk of introducing errors. You also avoid having the system broken while you are carrying out the restructuring - which allows you to gradually refactor a system over an extended period of time. - Martin Fowler
I try :) The one urge I have to resist most while refactoring is actually making other changes along the way. Say I'm refactoring some code and see something unrelated in the code. I have to make a conscious effort not to go "fix" that as well. Make a note of it and move on. For one thing, it's a distraction from the task at hand. It also ends up polluting your change set so your commit message now has to document several seemingly random changes.
Yes, always. I think the real essence of refactoring is picking which steps to start with.
I find the thing with refactoring large changes in a safe manner is always to have a reasonably clear picture of where you want to go. Then consider your existing system and try to find out which pieces you can introduce that have least likelyhood of being a radical change. Then you can introduce these in a controlled and well tested manner.
So what you do is to work in the vincinity of the nastiness. Not always attacking directly from the front, but sometimes just chipping away small pieces. Usually I wait, and only go for the "big prize" after a few rounds of chipping away at minor nastiness. But I know where I want to go.
The nice thing about working this way is that you can maintain progress. You never "stop development to do refactoring". Arguably there are cases where stopping is the correct situation, but most of the time it's not.
The idea here is that if you "start" with cashing in the prize money, you will be spending the next X days doing the drudgery. And there's risk, maybe you chicken out or it doesn't work - or spend 6 months instead of a week. If you do the drudgery first, cashing in the prize will be possible with less risk. And your code will improve as you go. Sometimes you can decide that doing half the job was enough, since your understanding of the problem increases. Sometimes your idea of where you wanted to go was slightly botched, and you can realign your goal as you progress.
But its tempting to go straight for the reward.
I tend to refactor in large steps most of the time so I can see the forest from the trees. It's a "stream of consciousness" kind of programming. As long as you have your last working version safe in your source control of choice...
that's where the "red, green, refactor" approach is useful. At each stage you have the ability to verify that your code's behaviour is unchanged, and the refactoring only has to integrate the new behaviour.
The rule of thumb I use is refactor with tests and only refactor as much code as you are confident too.
At 60 minutes are you certain that your code is doing exactly what it should be. You'd need a lot of tests to pass. I would just try and get one going and then move on to the next.
If I have a clear picture of what I want to do, and if I can easily verify that I haven't broken anything afterwards, I am taking larger steps.
If the refactoring is more complicated, I try to break it down into smaller steps and do heavy testing after each one.
I usually refactor code as I change it. That is, instead of taking a piece of code and rewriting it while maintaining its function, I rewrite it towards a new functionality and in the process of doing so I improve the design of the code.
Often this means that by the time I've implemented the feature I was after I haven't done a complete and satisfactory refactoring of the old code. It is improved though, and I know I'll have the time to improve it further the next time I'm about to change its function.
For testing this means that I get to test both the refactoring and the new feature at the same time, which should save some time.
It also mean that I only spend enough time on refactoring to improve the maintenance situation required for that particular feature. This should help to avoid over engineering and/or wasting time refactoring stuff that already works and wont benefit from a better desing. By focusing only on code I would change anyway, there is also a high probability I will revisit that code in the near time to do further changes while it's in the users attention span.
Small discrete steps is what I'm most comfortable with, though at some points it can be a test of my self-control to reign in what could be a refactoring blood-bath. If I notice any improvements (no-matter how large) that could be made, I make a note of them and consider how it'd be split up into individual refactoring tasks. Plus, having a saga of changes in the commit message doesn't help.
NB. The code-base I work on is quite old, and full of those mystical bugs named after scientists. With large portions still lacking anything near even 50% test coverage it would be careless to get carried away.
Yep. I like to run the tests continually and so a chain of tiny refactors works well. I get really uncomfortable having my code broken for more than a few minutes at a time, and I generally revert if my code is broken when I go home at night, the re-write the next morning ALWAYS works better than trying to pick up where I was.

How do you bring a failing project back on track? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
You must have heard the archetypical story of a failing/failed project:
A team of inexperienced programmers work 24x7
Bugs are fixed only to introduce new bugs
Customer is screaming that he could not even do the basic stuff (Saving/Querying) etc.
Programmers used to having the spec handed down struggle to improvise
No automated unit tests aggravate the situation
Architecture document that looked nice on paper was not followed in practice
Third party components used become bottlenecks not having been tested for fitness in the first place
Milestone after milestone missed
The team is not able to come up with a delivery date as nobody agrees as to the quantum of work actually needs to be done
No technical leadership / or a Cowboy Coder that can take on the technical issues
Now, If you were to be brought in as #10 what would be your first steps?
Update: First of all: Thanks to you all for chipping in. Well... I'm being brought in as #10. I was the original Architect anchoring the solution when we made the proposal to the client. Then, unfortunately, I couldn't take on the delivery responsibilities as I was assigned somewhere else. :)
Let's say it's a webification of an existing desktop application. I'm now being brought in as #10. Running away, sadly, is not an option. I'm sure this can still be reversed by following agile best practices and just wanted to tap the community for ideas.
The larger question perhaps is this: If the development team does not have specs but only the (baselined) code for a running application, the original solution called for looking at the code and extracting business rules on the fly. Now, the inexperienced programmers are reluctant to look at VB 6.0 code and want documents! So how do you fight this if you were to instate Agile processes?
Vyas, I feel like I could have written this question. My previous job involved resurrecting a KVM project that had failed after a year's development. Specs were in the form of a user manual and developers' experience with similar products. I ended up teaching C to 3 assembly programmers and re-architecting from scratch. We brought the product successfully to market in 4 months. (Then I resigned. Go figure.)
Some of the things I'd do again, particularly with an inexperienced team:
1. A team of inexperienced programmers work 24x7
10. No technical leadership / or a Cowboy Coder that can take on the technical issues
Give them a (short!) break from the project to "recharge." Maybe a day, maybe an afternoon, or maybe a long lunch on you. It will mark the end of the "old" project and the beginning of success.
Get their agreement to work their butts off when they return, and promise that you will be their go-to guy, cheerleader, and flak jacket. You, collectively, are a team, and your job is to forge their path, eliminate distractions, and lead them.
Plan an immediate success, no matter how small, and maintain a "can-do" attitude.
8. Milestone after milestone missed
9. The team is not able to come up with a delivery date as nobody agrees as to the quantum of work actually needs to be done
3. Customer is screaming that he could not even do the basic stuff (Saving/Querying) etc.
Take small bites! Break each piece down as far as possible, then deal with the small components. You'll identify "gotchas" early and be better able to scope the whole project.
Define your interfaces. Anytime you can isolate a chunk, do it. This allows parallel development, because you've already decided on parameters, preconditions, assumptions, what happens inside, and return values. You can stub it out, and build other modules and tests independently.
Prioritize. Focus on the defects and issues that affect the customer first. New features come last. If necessary, defer features rather than delivering buggy code.
Assign responsibilities. Volunteers are preferred, each in his/her area of expertise, but one person must be accountable for each task.
Track defects, and record everything that will help you reproduce, locate, and fix them. Document any that remain at delivery time, so the customer won't be surprised.
4. Programmers used to having the spec handed down struggle to improvise
6. Architecture document that looked nice on paper was not followed in practice
You will create the spec details as you go, each piece just before it's needed. It needn't be pretty, complete, or even written, as long as everyone understands the current task and you've got the big picture.
Discuss the implementation, one piece at a time, when the developer is ready to code it. Write the skeleton yourself if necessary, and let the team fill in the "guts." You want to keep them focused on each task, without "improvising."
Be available to answer questions as they arise. Your primary goal is to keep the team productive.
2. Bugs are fixed only to introduce new bugs
5. No automated unit tests aggr[a]vate the situation
Plan and start unit testing ASAP. If possible, enlist resources outside the team.
Fix small problems before they grow larger--or get hidden. Confidence in each small piece builds confidence in the whole.
7. Third party components used become bottlenecks not having been tested for fitness in the first place
Brainstorm solutions when you're not coding. Don't let them stop your progress if at all possible. Can you encapsulate or code around them? Replace them?
General suggestions:
Stay ahead of the team. Anticipate and try to solve problems before your team hits them. Gather any necessary information before it's needed.
Communicate constantly. Make it clear that you want no surprises, and solicit concerns, questions, status, roadblocks, etc throughout each day. Encourage collaboration and share "discoveries" across the team.
Celebrate every success. Compliment a clever solution, bring donuts when a problem is solved, demonstrate a new working feature ... anything that shows the team you appreciate them.
Get each task done, then move ahead. Don't waste time tweaking, enhancing, or reworking anything that isn't a direct barrier to success.
Keep your promises to the team, the customer, and your management.
Good luck -- please keep us posted!
Run away or find a new job. This is a death march and they need a scape goat.
Often, the death march will involve
desperate attempts to right the course
of the project by asking team members
to work especially grueling hours,
weekends, or by attempting to "throw
(enough) bodies at the problem" with
varying results, often causing
burnout.
Freeze releases, and start fixing issues with the program.... deal with the customer complaints by priority (the business side of the company can prioritize) and get the program running. Once you get the biggest issues out of the way, start cleaning up the code. Assign tasks to other developers, and start enforcing coding practices on all new code.
If you can do whatever you want, then look at what the real issues are and deal with them. If that means putting together a new team to develop the software all over from scratch, so be it. But you should try to at least fix the major bugs. Don't bother introducing new features, they only compound the problem, and a program that doesn't work and the problems aren't dealt with lose you clients.
Number 10 is obviously the worst problem, or at least the root of all others. Find someone with some creativity and ability to deliver a project, and give them free reign to do anything - including start over.
I hope you are getting paid really well. In any case, my plan would be something like these steps in the following order:
0) Stop adding features or functions across the team. Allow bugs to be addressed while the following steps are taken up to step 5, then stop bug fixing & resume feature development:
1) Apply what I call the Inverse Staffing Law: Weaker team members slow down the better and faster ones and generally a late software project needs people removed, not added. So, you need to assess the quality of the team members as individual contributors. Eliminate weaker staff from the team because presumably there are some. This is best done by reviewing their code and examining their bug fixes and figure out who is making the code worse vs. better and chop them for the team. This is not a time to mentor, you are going to need the best folks to have a change of "fixing" the situation in a optimal period of time. If you can't fire them or reassign them, have them getting coffee or something for everyone else left.
2) Assess the code itself. Identify areas of the code that are not constructed well and/or not well abstracted. If a area code is not constructed well and/or is obviously brittle at it what it is supposed to do, target it for a re-write. This feels painful at this point, but it will save you time in the long run. Recurring bugs and/or history of fixes will help identify the code that can't be salvaged. If a code area or module is fundamentally constructed well, but not abstracted well at the interface level, it should be suitable for re-factoring. This will save significant time and is useful code. Keep a list of the re-write areas, the re-factor areas, and the suitable areas.
3) Define a new reasonable architecture that you believe will result in a robust and complete solution to where you want to eventually be in features and functions. The architecture might not be optimal as starting clean, but in effect match up what you have with where you would like to be.
4) Work with the stake holders to decide what will make an acceptable first release attempting to table as many features as possible for "later" releases. Maybe you can't cut anything, but if you can, now is the time to do it.
5) Stop the background bug fixing efforts and assign the defined work out to the (remaining) team to estimate out a reasonable new implementation plan of the rest of the functionality. They need to own the schedule. Roll up the schedule and be fairly conservative. Now you have a reasonable prediction of when you could actually have something workable and robust.
6) Implement the remaining features and then harden up the release by tackling the remaining bugs. I am assuming all the normal good software development practices are observed here like source control, unit tests, etc.
7) Remove as many barriers as possible to keep the team cranking out stuff as fast possible.
8) Monitor for issues, and assist by getting your hands dirty where ever your can. Offer to take on the nastier issues to the extent you can help and still keep all members of the team as productive.
Good Luck!
This isn't about technical leadership any more, it's now about project management.
You as the technical lead will just be shifting deckchairs on the Titanic. So here's what I would do if I was the de-facto project manager.
1) Identify the project sponsors and stakeholders - both the official ones and the real ones.
2) Go to them and request that the project "goes dark" for a week.
3) If they don't agree, walk away from this project.
4) If they do agree, call a project time-out for a week - everything stops.
5) Spend that whole week talking to the important people on the project to identify the real project state.
6) Whilst engaged in those discussions, start formulating a project recovery plan, emphasising possible trade-offs between scope, schedule, budget, and personnel.
7) At the end of the week, decide which (if any) of your possible project scenarios are feasible.
8) Take the best of these scenarios back to the project sponsors and stakeholders, and start negotiating.
9) When a way forward is agreed, reboot the project and pray - possibly not in that order.
Common sense has already been pointed out to you by Maxim (Quit the death march). But if for reasons unknown you wish to persist, let me regale you with my experience in a similar situation - perhaps it might come useful.
It was my first job in a sleepy old town where good computer jobs where hard to come by and I despertely needed one immediately after college. I was hired coz the management thought i was enthusiastic enough and might be better than nothing (I offered to bring in my own comp to save them a cost of giving me a PC and offered to work for the experience alone)
The project had been abandoned by its creators due to the death march situation and had gone away after deleting all the comments in the code and performing other obfuscations. Nobody knew win32 / MFC stuff either.
I simply started studying the code on good old paper and pencil (lots of rubbing and corrections) until within 20 days time i knew the entire code including the variables by heart and what and where things where happening.
Armed with this knowledge i was able to make a critical piece working which had eluded everyone before. Of-course this was nothing but a drop in the ocean but it enabled the management to buy the clients confidence "smart fellow - got him with great difficulty - already got x working - u will have ur stuff working within y time".
Once the client was convinced and we where able to buy some time, some pressure was taken away. This got some hope back into the team and we started to hammer away for good. 6 months later i got promoted to project lead and 9 months later we had our fix shipment (lots of progress demos and a visibly more and more satisfied client in between).
As you can see, the elements of success are not directly duplicatable. But i would summarize that you need to breath some hope into the project first - show some progress and win confidence - that of your peers, management and the client. Once that is in place the technical stuff should be corrected too - there is nothing to replace this part of the equation.
If that does not seem likely, all that hard work (oh yes - lots and lots of work like you never imagined - why do you think its called a death march) would be a waste and you had better quit even before you start.
I had no choice and i was hot blooded and desperately need a job. The technical details where something icould work magic upon, and everthing just clicked into place. I really earned a lot of good will and self respect with that piece of work but in the long run its just a story i can narrate with great aplomb and nothing more except for those few in the know.
Things might be different for you but its for you to decide.
Good luck
Make sure you aren't the scapegoat
Cut scope creep
Trim functionality "requirements"
Implement a faster dev cycle (maybe Agile/Scrum/XP/whatever)
If you can, run away.
If not, you need to stop all activities that make the project unstable - including coding and fixing defects.
Assess where you are
Break up the requirements into much smaller "milestones"
Read some practical books (Mcconnell's "Software Project Survival Guide" comes to mind.
Identify all the problems and risks. Communicate all those to all involved.
Work on each piece one at a time.
Celebrate improvements and milestones as they are reached.
Good luck. Your scenario sounds pretty bad. It may not be salvageable - and things have to change to get better.
If you really had to get it on track (if bailing isn't an option)
Start off by accepting that it's a failure in management. You might then want to go on to implementing a strict but light process.
I'd suggest some form of Agile, since it's the easiest to successfully implement without a GURU, but you have to be VERY strict about it, including Pairing, Ruthless Refactoring, Reviews, Spiking functionality, Visibility, TDD, one-week cycles, 8-hour workdays (Yes, longer than 8 tends to harm productivity more than help, as you seem to have noticed)...
Don't be cutting anything out either. Parts of Agile rely on other parts--without the pairing, refactoring and testing you cannot eliminate upfront design (one of the biggest agile failures).
Don't forget about the management side of it. One week iterations to start (demo EVERY week). Constant adaptation. Very short stand-ups every day to address issues. (Keep to 15 minutes max, table longer issues, etc) Burndown charts, core-team with a client on it.
You can't just have a 15 minute meeting every week and 2 week iterations and call it Agile, but if you do it right, it just MIGHT give you a chance. You might get a GOOD agile consultant in to train you on getting started.
Also, constantly evaluate what works and what doesn't. Be prepared to fix what doesn't work. Weekly meetings to analyze that weeks' development successes and failures.
Overall it CAN work, and can bring a flailing team into line, but it's not trivial. The nicest part is that you can implement it without taking huge chunks of time out of your current development. You just keep developing, but you do it better.
Tough situation, you have zero customer trust and basically can't be successful under that situation, no matter what.
For all intents and purposes the project needs a reboot; the unfortunate fact is that incumbant shops usually don't get this oppurtunity to start over and re-evaluate everything that is there.
I hate to say it, but you need to halt development and spend a month working out what went wrong...
The result needs to be a plan for a feasible 6month - 1year delivery really making them focus on what the must-haves are and real trade studies on your third party components. And trashing the code base needs to be an option; start a new source control project and when you get to a particular module port peices that make sense and leave the garbage behind.
Agile is great and all, and a valid approach once you get a real plan in place; but its not going to fix a broken relationship with your customer... or all the junk that's already there.
Here's the summary of key learning after reading through your experiences:
Maxim
1: Make sure this is not a "Death March"
Ellie
2: Make sure what's delivered works
3: Refactor & Realgin codebase to Architecture / Best practices
4: Look at what are the real issues: Is the team technically competenet to deliver?
Kendall
5: Ensure availaibility of Technical Leadership
Bill K
6: Implement Agile Processes (At least automated unit tests if not TDD, short iterations that make progress visible)
7: Get customer buy-in
8: Be prepared to throw out what cannot work (wishful thinking aside)
Warren
9: Make sure the team memebers that remain given a chance to start over
Tim
10: Motivate team and as improvement becomes visible reward them
jsl4980
11: You need buy-in on schedule from your team (most imp.) & customer
[This raises more questions. What if your customer asks whether the team is competent enough to stick to your schedule? What if you yourself know that the timelines the team is proposing just shows their lack of understanding]
Ather
12: Is the team commited?
13: Do you formally QA?
Patrick
14: Start over, redesign and reconform to Architecture/Design best practices for modules yet to be developed.
The summary has 14 items. You can't do them all. So, what's the first step?
Here's what you have to do first -- get one thing improved.
You've got fundamental quality issues. (#2-5)
You've got architecture and component issues. (#6, 7)
You've got schedule problems. (#1, 8, 9)
You can tackle quality. Formal unit testing, heading toward TDD can help. This might be hard because architecture issues slow testing down.
You can tackle architecture. This might be harder because it will probably involve rework that will not appear deliverable. But it may fix quality issues. Or, it may be compounded by fundamental testing problems.
You can tackle schedule. Without other corrections (i.e., quality or architecture) you may not get any traction with fixing schedule issues.
I think that overall improvements in people's attitudes come from starting with one success -- any success -- as early as possible. What's the lowest-hanging fruit?
One long-standing bug? One unit test suite to find and fix that bug?
One major architectural feature? Would a diagram that everyone can post in their cube help? How about a presentation clarify things?
One new use case? One new feature that actually works?
Here's a good book on the subject:
Catastrophe Disentanglement: Getting Software Projects Back on Track
First off, be resolved that you may fail -- if you can't accept that, don't take the challenge. And that includes being a scapegoat (it does happen). Management won't look at it in those terms (i.e. they're not intentionally/consciously 'setting you up'). But that is a reality of a corporate environment; if you take on the responsibility (often with more pay than those that don't), then your head is for the block if things don't work out. You have to be ready to stick with it for the long haul too. I was once placed on a client site for 8 months to fix a waning project. And as you saw, one of the other blog-posters here spent 9 months before a release version was ready.
Now, assuming you are okay with the possibility of it going all pear-shaped in spite of your efforts, this is what I suggest:
a bug tracking system is going to be your number one best friend, it will allow you to regain a semblance of control. you can't hope to understand a complex system as a whole, so 'chunking' it will help. and a bug tracking system allows you to unitize problems and distribute them to the other guys you are working with.
you have got both technical and political challenges to deal with. the technical generally aren't so bad because you're a coder and you know how to do this. the political ones are much trickier, you're at the helm of a ship thats gone hopelessly off-course, and you're in the Bermuda triangle. the biggest challenge is often stemming the tide of negative sentiment amongst the client (e.g. client: "these cow-boys don't know what they are doing", "they promised me this and didn't deliver", "i have no confidence in these guys to any more").
for starters, apologize to the customer and tell them in concrete terms what you are doing to do to re-right their project, e.g. you: "I'm sorry about the delay on your project, I'm getting stuck into it now. I've looked at the project history, and personally, I would be angry too if I was paying good money for this system. the first thing I'm going to tackle is..." <- bingo, you've just taken responsibility for the project which means there's no turning back - its all or nothing now.
a few other people have said it here, and I agree; stop adding new features. what they haven't mentioned is that you may have to do this to keep the client happy (remember, there's a technical and political side to the challenge).
understand the business domain as best you can. read through any requirements documents you can get your hands on. you are at a massive disadvantage by coming onto the project late since you don't know what was originally discussed. the devil is in the detail. this is what sunk me on a late projects I wasn't able to salvage, everyone was on edge, and i missed a minor requirement. at the time, it wasn't a big deal and could have been corrected easily, but politically speaking, it was the straw that broke the camels back. one tactic which may help is to go out on client site for a few weeks.
understand that time is money. its not just a technical issue. the client has paid for something which isn't right or has not been delivered. your company has expended resources, possible having already used up all the project budget - the business is now losing money. and this is where the issue of new features come in again, yes - people are saying don't add them, stablise. but adding new features can be a politically helpful tactic, management will be happy because new money is coming in for off-spec work.
I'd recommend against you or your coding crew working ridiculous hours to deliver. if you normally leave at 5pm, leave at 6.30pm or 7pm instead. you and your coding boys can consistently maintain an hour or two of extra work for many weeks on end and perhaps 4-5 hours over the weekend. working until 9pm or 10pm every night will result in burn-out in roughly 2 weeks (some can go longer). after that point, your extra time on the project is doing more harm then good. in the unlikely event your boss takes issue with this, make a choice; do what they ask (i.e. work more hours), or say "I've already committed extra hours to working on this project - I'm here for the long haul and im going to get this project done if its the death of me. but that is the limit of how much time I'm willing to put in. i have other commitments to keep outside of work" <- but be ready for the consequences (remember, political situation as much as a technical one).
there are people here that are saying "stop and write a spec, stop and do this..." - I'm sorry guys, i just cant agree with you here, its unrealistic. the project is already stagnating, the last thing management or the client wants to here is "we have to stop everything and...". I've tried this before, where I've said to the client and management "the bugs will keep coming until we stop and i write up a detailed system test plan. it will take me two weeks" - the client didn't want to pay for this, and management wasn't willing to wear the cost. as it happened, the bugs kept coming.
learn to 'juggle' - you have to map out tasks ahead of time so programmers aren't waiting on you. this will generally mean you do less coding yourself. generally this is best achieved by having a project schedule before coding starts. programmers should know what they are doing next after they finish what they are currently working on, and they shouldn't be coming to ask you "what do i work on next?", they should already know.
build-in recovery utilities, especially if the software has recurring problems which are hard to pin-down. for example; it may take 12 hours to track down a bug and fix it, it may take 2 hours to put in utility (read 'hack') to fix the problem for the time-being. time and momentum are of the essessen, and unfortunately bandaid fixes may be needed.
be very observant of the clients mood. they need to know you are 'on their side' (e.g. client: "the product is unacceptable", you: "i agree, i would kick our asses to if i was in your position. all i can tell you is im on it and wont rest until its all working"). when the client is back on your site, they will actually start helping you. for instance, they may shield you from pressure from your management.
lead your guys by example. something along the lines of "I'm staying back a bit to work on the project, I'd appreciate the help if your willing to stay back too" and "i know its not our mess, but we're still going to clean it up anyway. i want the client to get some good quality software". programmers could generally care less about the company that got them into this situation, but they may care if its about one of their own or the client ('may').
many of the suggestions I've seen here assume a fairly high degree of power (e.g. 'stopping the project to restart it properly' or 'say no to new features') - you are starting the project already hamstrung, and as a programmer, you will traditionally have less power to affect change then a true manager. this doesn't mean 'give up/don't try' - it just means you are going to have to be creative and do things you don't normally do (i.e. use 'soft' or people skills).
a lot of people here are saying bail on the project, run for the hills. I have been on 3 hopelessly late projects to date. i managed to fix 2, and 1 I couldn't fix. personally, it doesn't bother me to take on a late project. after all, the worst that can happen is you get fired :)
If you were involved in the project from the beginning, I hate to say it, but the company should replace you (and the entire team).
It should be reanalyzed with a competent team with real project management processes and lead by a project manager with experience in this situation.
None of the original coders should work on the 'new project' of saving it. They can move to other projects (they don't have to be fired) but to get a fresh look at the project, everybody should be replaced.
And of course, management has to understand and be on board with the fact that the project is going to be much later than expected. If management doesn't agree with this (replace team, find experienced leadership, take a step back and start again) then #Maxim is right - get out of there.
1) The first thing I will assess is whether the people on the team are committed to the project or not? If not, it is worthless to do any other thing. Nothing can prevent the disaster unless I get a dedicated and committed team.
2) I'll make sure that there is QA on the team.
3) Come up with a reasonable plan of iterative and incremental releases to the customer. With the mess we are in, there is no way customer can get everything soon. Based on the priorities of customer, we'll deliver smaller increments of functionality to him frequently. This will keep customer engaged, a bit less-edgy since he is seeing something happening.
What ever you do, do it step by step.
First, it's not about addind features, it's about fixing the app. Don't add anything new. Just refactor. Say no to any new stuff somebody ask you to introduce in the system.
Don't try to improve the whole app. Take your team, make it focus on one aspect at the time, with the best practices you can, especially using unit test.
Use test driven development only. In that case, it will immediately show you what part of the behavior you don't understand (you can't code a test if you don't know what to test.
So here are the road map :
Identify the critical part you need to change
Isolate the code that implies this behavior
Find any occurence of this code in the rest of the code
Refactor using this knowledge and massive TDD
Integrate, test and fix until this particular part works
Go back to step
Make the situation clear to your boss : it will take time, money and will be painfull. Explain why, what you will do, and that you have no other way or it will fail AGAIN.
A above all, don't try to make it clean the first time. Refactor what you can, but don't expect to change the entire architecture of the part you are working on the first time. You will have to iterate the process on the whole application several times.
No miracle. Just method and patience.
Been there, followed these steps:
Stabilize
gather the real story: how good/bad is the codebase, how good/bad are the developers, what really needs to get done (bare bone min.), when it needs to get done by
reduce overtime (tired people, good or bad, don't work well)
remove the bad, input new/good - err on the side of replacement (many could be burnt out and appreciate even a forced change)
remove access to bad/un-required code (focus on the 20% of the code base that provides the 80% of the value)
put base code practices in place ensuring only good code is getting in (don't damage the base anymore)
Control
implement teams focused on the app components (decouple as much as possible)
put code management, release management, risk management, QA, etc. in place (build your environment so you can succeed)
get on your clients/sponsors good side - delivery a win, even if it's a somewhat stable very very small release - and then put in change management (control what gets requested)
Move forward
develop a plan (planning is essential, plans are useless according to Ike - you need to plan to find what is missing and to set a target, but don't expect to tell the future) - continuous planning is required
aggressively manage your people - good people make good product - make sure you get and retain the best
refactor over time - clean up code as you go - you may not have the luxury to fix everything at once so do it overtime to provide for a cleaner code base
move forward bravely - overtime be more aggressive with your deliveries test (but not stress) your team
Agile refactoring. Identify and prioritize what customer wants and then create the most important stuff in short sprints out of existing code. Good luck man :)

Programming in a crunch

Ideally you want a schedule that's accommodating and flexible but when it comes to paying the bills and working in a business, that's rarely a luxury programmers have.
I have been fortunate to have the grace of Steve McConnell and Frederick Brooks to tell me what to do if I want to screw up my project and I take their work seriously.
And yet there are still times when your back is against the wall and you need to speed up the work. What are some "tweaks" to your process you've used to speed up delivery without sacrificing quality? Is this even possible?
I consider learning good practices to be of higher value than learning the actual code so no "be a better coder" answers. That's a given.
jpgs are faster than html pages
Without sacrificing quality, your best options are pretty much the same best options that you have when it's not crunch time: Eliminate unnecessary distractions, focus on the most important work first, obtain better hardware, and offload any work that you can reasonably offload.
Arrange my tasks in some sort of list of small independent tasks (or, tasks where the dependencies are already taken care of in the order). Then, just sit down and clear one task off my plate after another. Anything where I have to coordinate with someone else, ask a question, etc. is punted forward; I just sit and code with no distractions, until I get to a place where I need some sort of outside interaction. Then I deal with all of my coordination with other people, re-planning, and so on in one batch, and go heads down into straight coding again.
Also, prioritize and cut everything that's optional. If you get the required stuff done in time, then go back to the optional stuff, but do what is absolutely required first. This may be relaxed slightly if there are tasks that are easier to do when you're in the right context, but on the whole, try to keep the optional parts to a minimum.
Oh, and don't make these judgement calls while you're coding. When you're coding, just take one task off your list, do it, and move on to the next. As I said, batch up all of the non-coding work into chunks so it doesn't slow your coding down.
Anyhow, that's what I do when I'm up against a wall. Not sure how well it works for anyone else.
Usually, cutting features and work items is the best way to meet a tight deadline.
You can try to rush coding, but you'll almost always pay for it later - You'll spend much more time debugging and stabilizing the rushed code than you saved in the first place.
Actually, I think a lot of programmers and organizations have a lot of low-hanging fruit. Make sure you are using your time productively, know how to trade-off speed for quality appropriately, don't "gold plate" things beyond what your customer is asking for, etc.
Profile your organization's processes. What is the bottleneck slowing you down? Context-switching (i.e., multitasking) is often a real productivity-killer.
I suppose the problem with this is that when your back is up against the wall, organizational improvements like these seem like luxuries unto themselves, even though they are the things that could really help you speed up your work.
The biggest help (for me) is forcing myself to complete each single task before moving on to another one. If I need to get some bugs out of libfoo, or even write libfoo to go on to the next step, I force myself to stay on that task, then the next one, then the next one.
Most of the time that I find myself in a crunch, its because I was too much of a grasshopper, jumping around from thing to thing.
This morning I sat down and forced myself to write a bunch of unit tests, rather than working any more on the library being tested. It sucked, but now they're done, and I can really sail through finishing my library.
I'd suggest learning GTD or something like it before you get in the crunch, using to avoid a crunch if at all possible and then, if you must, get through the crunch by just doing it. Get in the zone and persevere.

Time management tricks, tools & tips [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Working with software day-to-day usually means you have to juggle project work, meetings, calls and other interrupts.
What single technique, trick, or tool do you find most useful in managing your time?
How do you stay focused?
What is your single biggest distraction from your work?
The trick the Getting Things Done system teaches is to have a trusted system you can put action items into. That way you don't have to keep "juggling". To keep with the metaphor, you can put the other balls down and have confidence that they will not be forgotten. Then you can concentrate on a single ball at a time. There are many, many other excellent tricks GTD teaches. Well worth getting the book.
I read this rule somewhere, and I use it every day...
If someone asks you to do something - if it takes less than 2 minutes, do it immediately. If it takes longer, put it on your list and come back to it.
This really works for me.
You should see this:
Randy Pausch Lecture: Time Management
It's a teacher from Carnegie Mellon who is near dying, giving his final lecture about time management. It's the best tips and tricks you can find.
To manage the general mayhem of the job, I try to use a toned down version of GTD focusing mainly on trying to maintain Inbox Zero and pushing tasks into a todo list (I use Remember the Milk for task list management).
As for maintaining flow in spite of interruptions, leaving a TDD project in a state where tests are failing tends to give you a place to jump right back in when you come back from a meeting or other interruption. Leaving a batch of uncommitted changes might serve a similar purpose -- to get your mind instantly back into the flow of the project without having to go look around to remind yourself what state things are in. Beyond that, using a fairly detailed task list for the projects at hand can help keep you on task and moving forward.
Often times, I've found my manager's manager to be the biggest distraction! :-) He likes to feel plugged in to the day-to-day work of his dev teams and frequently comes around on "walk-abouts" to see how things are going.
I find email the most distracting, so I've really cracked down on receiving certain types of email. I've unsubscribed from many mailing lists, job alerts etc. Shutting down email for a period of the day is quite useful too.
I enjoy going to the library. The quiet but busy, concentrated atmosphere basically forces you to work. The change of venue also seems to shut out some of the busyness and maybe worries you have in day-to-day life.
I use most of ZTD (http://zenhabits.net/2007/11/zen-to-done-the-simple-productivity-e-book/). GTD is too sophisticated and to big for me.
Basically, I make lists of tasks. Every morning I select three which I really need to do that day. I work on them until they're done. I struggle not to get dragged to other things.
In an office, I sometimes book a conference room and work there, distraction free. I emerge from the lair when I'm finished with the three most important tasks.
Encourage people to push their correspondence with you down the distraction chain:
Phone / Face-to-face
Instant messaging
Email
You can do this by deffering them: "I'm really busy right now, can you send me it in an email?"
This should reduce the amount of interruptions you receive allowing you to stay "in the zone" for longer periods of time, increasing your productivity.
Finally, allot time for processing emails at set times of the day. I, for example, have my email set to send and receive once every two hours. This bulking of activities allows you to get more done in the day without impacting customer relations.
My single biggest distraction is myself - I tend to go all hare-brained, chasing emails and internet links much of the time. Therefore, I'm using a simple trick to discipline myself into staying focused and on-task for larger parts of the day. The principle is to stay accountable for the use of my time:
1) Have a scheduled job in your operating system, that pops up a small messagebox every 15 minutes (in Windows, it should run the command C:\windows\system32\cmd.exe /C "start /B msg jpretori /W /V "15-minute check"")
2) Have IDailyDiary running in your system tray (a text file will work fine, too). Every time the box pops up, fill in what you've been up to the last 15 minutes.
I've caught myself with an ugly day filled with procrastination before... It's quite a good motivation to stay on-task.
Recently I've started using a great little free windows app called NextAction which you can get from here.
It's greatness comes from it's simplicity and it really helps to refocus and stay on track when dealing with all the days distractions ... email, co-workers, scrums, rss feeds, twitter, lunch, coffee breaks, etc. Having a list of what I'm working on always there on the desktop makes it very easy focus after any context switch.
Much better than pencil and paper, check it out for yourself.
NOTE: There is a more comprehensive web based 'NextAction' at code.google ... not so good for me, but maybe for others.
A single answer to all of the listed questions is David Ellen's Getting Things Done (GTD) ( "The Art of Stress-Free Productivity" )
A 45-minute presentation of the process can be found on youtube, and you can get the book on Amazon
Also you could think about the kind of things which make you want to browse the net, check your email, etc. For example, if a build I'm working on is taking too long my mind will wander.
So it actually pays off to make the build process as quick and efficient as you can make it, so you can make changes and test quickly.
I also find it helps to get enough sleep (tiredness is bad for concentration) and not to drink too much caffeine (seriously. I feel so much better after cutting down the amount of caffeine I drink. Try naturally caffeine free teas!)
(I seem to have wandered slightly off-topic into concentration there... still, I find the better I can concentrate the better I will use time!)
If you want to improve something, you first have to measure it.
I like Rescuetime. It logs all applications and websites you visit and how much time you spend there. You can tag applications/websites, i. e. with "work", "waste", "news" and get nice charts, productivity measures etc.
I find that http://www.rescuetime.com/ lets me know what I was actually doing all day, rather than what I THINK I was doing!
It also lets you put a "productive" level on each process/website you run or do so you can see how productive you are being.
The single most useful piece of time-management advice I could give is just get on and do it. If something is going to take less than 5 minutes to do, do it now.
Single most useful? http://www.nowdothis.com is AWESOME for focusing on what currrently needs to get done, and has raised my productivity by tons. (Bonus tip: Use Google Chrome to make it its own application and then make the app always be on top of other windows)
Biggest distraction? Google Reader.
(Slightly off-topic)
They says there is no such thing as time management. You can't manage an hour and get extra 20 mins out of it. Well, I recently discovered that you can. It you're listening to podcasts or watching recoded web casts, you can speed up the play speed. I found that it also helps me stay focused on the content rather than drifting off and starting check my email during the natural pauses. Then I saw Jeff's post on the same topic.
Email, IM, Skype... all those can distract. But biggest distraction is when my fellow colleague ask me why I wrote some year old algorithm this way and not that way. It brings my work to halt even if I know the answer.
To stop this interruptions, we have a 5-minute break every hour outside the office where we can talk about such problems.
There are a lot of things to do and I'm not sure you'll find any single technique to get organized and stay focused.
But...
Do list the things you have to do. Several short lists will be needed (today, later, inbox == to be sorted out, etc...). Review these lists once in the morning, and then in the evening. These related posts are worth a read: The Taste of the Day and The Trickle List
Timeboxing: allocate time in your calendar to get the tasks done
As suggested by harriyott, switching off email is kind of essential too!
This question has already been asked, so you might search for it.
I personally use Zen to done which is a simplified version of "Get Things Done". For the trusted system I host Tracks application for myself.
The best way to get through a big chunk of work while staying focused is to list your priorities on paper before you start. Trying to keep a big list in your head is a sure path to procrastination. Plus, it's a great feeling to tick off items as you finish them. Put on some music, close down your email, and get busy.
But then you have people trying to get your attention. Make sure your colleagues and clients know that you prefer to receive their queries in email rather than in person or by phone. Bugs go directly into the tracking system, without anyone having to tap you on the shoulder for each one. Sounds obvious, but stopping your work to discuss something for 5 minutes can sometimes cost you 30 minutes productivity by the time you are focused again.
You can find a great document about time management in Wouter van Oortmerssen (aka Aardappel, the developer of famouses Open Source games like cube and http://sauerbraten.org/ )
The article I'm talking about is this
I guess you're after something practical. What I do is keep my action items away from my work environment, it helps keep me focussed. I keep a pad next to my desk, I write down each action item for the day at the top and half way down start keeping notes. When I've finished a task I tick it off, anything not ticked can be carried over to the next day (if it's still relevant). Been using it for about 3 years, I find it keeps me productive and helps me remember things. I've tried all kinds of software solutions, nothing works better for me.
I have an old laptop that I remove the wireless card from and sit on a completely quiet room away from distractions. Whatever I can't get done without the internet is just leave until later. My biggest problem is that I gooel to find a solution and end up doing 30 minutes research on something a blogger has mentioned in passing. I still find it takes me a good hour before i get into the flow of not distracting myself.
On the 'how to stay focused', I think once you decide to close your email and put your phone on send, the next things to control are the sounds around you that might derail your thoughts. People talking, phones ringing, etc.
I have started putting the headphones on and surfing to http://www.simplynoise.com/. This is a noise generator that gives you the option of white, pink, or brown/red noise. It drowns out most of the audio distractions that often poke at my concentration.
Stay productive: When I'm working on a boring project and notice I don't do anything useful but reading news, I set a timer.
Simple enough, set your mobile on a 1-2 hour timer. Work during that period. When the timer rings, take a break and feel good about yourself :)
For some reason, this works (for me and a couple of other people I know)!
The single most valuable tool that I can recommend is a "todo" list.
This may take the form of a specialised app, gadget or pen and paper, however the most important thing to remember is that new tasks should be added to the bottom of the list and tasks to be started must be taken from the top - ie. don't cherry pick your tasks, as this will leave you with a task list full of time-consuming (and often boring) jobs that will begin to drag you down.
Possibly better for programmers than GTD is Time Management for System Administrators. Same basic principles (reduce interruptions, keep a list) but with a nerdier bent.
I close email and listen to soothing music. Of course, this tactic really is all about minimizing distractions.
The lecture of Randy is great, especially since he knows that he does not have much time left in this world.
Meetings are the biggest time wasters. Try to avoid them wherever you can.
I don't believe in those tools popping up every so often asking me what I'm currently doing. That's very distracting as well.
It might be good to make a time-log for for a couple of weeks, but just to understand where you are spending your time so you may be able to improve things.
I like the time management stuff by Steven Covey.
... and by the way I'm lecturer for time management for IEEE for Europe/Middle East and Africa.

Resources