Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
We are looking at hiring a software development project manager. His job is going to be concerned with running multiple dedicated project teams focused on delivery of software for external customers. He will also need to provide support to our business development unit and oversee post-implementations support of the aforementioned software. What level of hands on development experience should we expect from the applicants? Successful candidate is not expected to do any coding.
Not important. We should be focused on proven project management experience in software area.
None.
Some experience, exact technology does not matter.
Heavy experience, exact technology does not matter.
Some experience involving same acronyms as we use daily over here.
Heavy experience involving same acronyms as we use daily over here.
Some experience, mostly with technologies we do not use.
Heavy experience, mostly with technologies we do not use.
This question is regarding the best level and quality of required technical experience and is not concerned with any other skills and qualifications of a software project manager. Many thanks.
As with any position, you need to assess first and foremost what skills and experience you need on the team for you to be successful. Then hire to fill the gap for the skills that you do not already have on your team.
If you already have a team with strong technical and technical leadership skills then you don't need to hire someone who is likely to compete with the people you already have. If you are missing this, you probably want to hire a technical manager with some project planning and tracking skills.
Great project managers are those that are multidisciplinary - they are most successful where they can bridge the divide between the various stakeholders and team. The primary role of the project manager is to manage risk and facilitate communication and collaboration. As a minimum, you should look for someone that has proven experience in either your industry or with the technology space that you are playing in, otherwise they will be unable to gain the respect of the rest of the team and perform their primary role.
Which brings me to something else you should consider carefully - what is your culture? For example in a previous job, we had development leads that were very strong technically and wilful. Project managers were always relegated to second chair, and pretty much ended up as glorified MS Project admin. assistants. Anyone good did not stay long. What do you need to do to allow the type of skills you want to acquire for the team to flourish?
Most of our project managers have zero technical experience, so I'm guessing the skill sets are different enough that it's not necessary. However, they have to be bright enough to grasp/learn the concepts involved in development -- just not the implementation.
That's not to say that a technical background would be a bad thing -- it could be a "nice to have". Then again, it could possibly get in the way and they could try to control the implementation.
In my experience the very best technical managers I've had had very strong technical backgrounds (and usually were a little reluctant to trade herding code for herding coders). The worst were the the ones that were merely average programmers at best and had more of a management background.
The tentative conclusion I've drawn from this is that while not all programmers are management material, all good technical managers started out as good programmers.
Note that this answer is coming more from the perspective of hiring an engineering lead. For a project manager - someone whose job is to interface between the technical people and the customer - technical acuity is probably less of a requirement.
Some technical skill would be nice, but far more important is that they understand the functional area your company exists in. So if you sell an OS, then you probably want stronger technical skills than if you're writing banking software, for example.
Go with point 1. "Not important. We should be focused on proven project management experience in software area."
Edit: (after re-reading your intro-para) Seems what you want is a product-manager, and in support you need team-leaders on the diverse teams to handle and report on the technical issues. (Also since customer-contact is involved: a little marketing experience won't hurt!)
As an aside:
You are focusing on the wrong skill-set. You want proven administrative skill; proven organizational skill; and above all: proven people skills - (s)he must be able to communicate without antagonizing or patronizing the audience. The technical staff and programming staff will have all the necessary experience in development. (S)He must be able to manage and control these staff members effectitively.
The manager has to be able to communicate with developers. This either requires a decent technical background, although not necessarily with the same technology, or enough humility to know when the developers know more about something than the manager. I've seen both work well.
I think what I'm saying is that having respect for the developers is important, and there's two paths to it: understanding what they do, or understanding that you don't understand what they do.
Answer is "4".
Heavy experience with some technology is critical. I know the mindset is "project manager does not have to understand technology, he just manages people".
Well no, PM does not manage people: he manages project that is supposed to produce some deliverable that is acceptable at least across some desired aspects (capability, performance, reliability, security, maintainability, etc). If he can't understand technology, he's lost. Of course, he does not have to be an expert in peculiar technologies used in a project: but he has to be able to filter BS away, to question programmer's estimates (we know how those go), to feel at least technical risk here or there, to be able to formulate business ramifications of particular technologies.
In some ways I think that PM's challenges re technology are even bigger than those of programmer: he has to be genuinely interested in technology, yet he can't / should not have any technology bigotry, to be actually fair towards them (what they are actually good for and what they are actually not good for).
Read "In search of stupidity" for evidence how non-technical managers drove many tech companies into the ground.
This is excellent summary by Spolsky: http://www.joelonsoftware.com/articles/Stupidity.html
Now, the small print #1: not every programmer will make a good PM, of course. In short, control freaks, toxic personalities, egomaniacs, people who are good at coding but not at negotiating, people who are good at coding but yield to pressure too easily -- will FUBR their projects.
Small print #2: It might be possible that people with very good analytical skills might make up for lack of experience with technology. I've worked with people who were excellent business process and procedure designers, who instinctively understood how UI should be organized and what the software should be doing in this particular place and why and who could detect BS quickly even when served by domain experts but who could not program if their life depended on it.
Most has been answered already, but I'll add this:
Keep the same mindset that you would have when hiring an office manager. While the technology knowledge is important, you'll find that ambition, a will to learn, coupled with a team leader attitude will get you a better manager than looking at mostly technology knowledge. Most projects have some company/industry-specific skills that are involved and a quick learner / great leader will bridge that gap quickly.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
Is PMBOK more for after implementing software is built, delivering it to the customer while Agile or Scrum is more for building the software in the first place? Just trying to understand.
Thank you.
EDIT: My main concern is the PMBOK. They use it a lot where I work but not for development (they implement with it.) They don't develop a lot and so I have no way of asking, "Hey, what do you use for development?" I have to come up with the best plan on my own. I really don't care a whole lot about being PMP certified but if that's the best way to go to develop software using PMBOK I can justify learning it. If scrum or agile is the best way, then I'd rather use that and be successful than have a pmp by my name.
Well I can answer this question from my real world experience being both a PMP certified implementer of enterprise software solutions as well as an experienced Agile PM having managed a team of 14 in a Hybrid Agile software development project.
When implementing COTS (commercial off the shelf) software I find that the PMBOK can be followed quite closely. If followed to a "T" the PMBOK will steer you down the path of a "Waterfall" methodology. If you are not familiar with Waterfall, this is the approach where most of your project time is spent early in the project gathering requirements, performing design, estimating, etc. Build or development comes much later in the process. The reason this approach works well for software implementations is because the customer generally wants to know the cost upfront for the project. The only real accurate way to determine the project cost is to follow the waterfall approach...at least initially.
The Agile/Scrum methodologies work much better for building software. When I say build I mean the entire build process from design, development, testing, etc. I won't go into the differences between what is covered in the PMBOK, Waterfall or Agile methodologies as that is not what you asked. Agile is very much about iterative design & build, and less up-front design. In agile you want to iterate quick, and perform JIT (just-in-time) requirements gathering (using storing), design, build and testing (TDD). This reduces the amount of waste and produces usable software earlier in the project. Agile has many benefits to software development projects.
Now what I have found helpful is taking the waterfall approach as far as you need to build an accurate estimate and resource plan. Once that is complete you can switch in more of the agile processes to finish off your project.
Remember not to confuse the PMBOK with an methodology. PMBOK is a set of industry standard processes that can be followed to deliver a project; not just a software project, it could be engineering, municipal planning, etc. There are many parts of the PMBOk that are beneficial in the software development world such as: communication planning, risk planning, project close-off, etc.
It's a pretty broad subject so I hope this helps you make the appropriate decision for your project. Remember one size does not fit all.
I would personally suggest understanding Scrum and/or Agile ( whichever suits your organization best) before jumping into PMBOK.. The standards in PMBOK are more conventional and the phases are described in a slightly pseudo- waterfall method which is not as acceptable in IT industry now as before Agile and Scrum knocked at our doors... If you study PMBOK after knowing about Scrum, then its easier to filter out whats not relevant and get a better understanding of both concepts overall..
Both PMBOK and Scrum/Agile are for engineering as well as Implementation.. PMBOK covers areas other than the IT industry, that's all!
Also, Scrum is something anyone or everyone in a project team learns and implements.. but PMBOK is primarily for a Project Manager...
PMBOK is just a naff industry standard used to give the management in organization A who don't know any better the (false) peace of mind that organization B is doing things in the 'correct' PMI certified way.
Scrum as Pedro said is an agile project management technique that revolves around working in fixed iterations which is highly regarded by most software professionals. Although Scrum doesn't prescribe any engineering techniques it is commonly used in conjunction with the engineering techniques outlined in XP (Extreme Programming) such as pair programming and continuous integration.
For business reasons some companies have to at least appear to follow PMI, CMMI or ISO in order to win work, but in reality most serious software shops are practicing agile techniques such as scrum/xp/kanban behind the scenes.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 7 years ago.
Improve this question
We have a four person development team that is in need of a formalized project management system. I have a general understanding of Scrum and Kanban but it's hard to truly understand until it's been tried. We don't have the luxury of trying one for a few weeks and then switching to another so I was hoping that someone out there in a similar situation might have thoughts on which worked better for them and why. Also, any other systems for managing development that worked would be great to hear about.
Another note: there's the possibility of the team growing, of course, so we would need a system that scaled well.
Yet another note: We work on three separate software applications in Windows all of which are based on a central library which we also wrote (so i guess you could say four projects)
Both Scrum and Kanban are really process "skeletons". Neither is specific to software development. Scrum was popularised by software development organisations but is positioned as general management technique rather than a software project management technique. Kanban emerged from manufacturing and has been adapted to software development, initially by maintenance teams. Both Scrum and Kanban aim to manage the flow of units of work through the team that is doing that work, measure how fast work flows so that estimates can be made more and more accurately, and make bottlenecks highly visible so that they can be addressed.
Because neither is specific to software development, teams using Scrum and Kanban add software development practices to the process to help them incrementally and iteratively release and improve the software. Most teams, whether working within a Scrum or Kanban process, adopt the technical practices of XP and reflective practices of Crystal.
XP is basically Scrum applied to a single team plus guidelines about what makes code "high quality" and how programmers can achieve that. Crystal Clear also applies to small co-located teams but is more flexible about programming practices although it also recommends the XP practices (the book describing the process is excellent and full of invaluable advice, whatever process you decide to go with). Scrum teams also usually adopt the reflective practices of Crystal: regular "heart-beat" retrospectives and larger retrospectives after every major milestone. Kanban requires continual reflection and improvement but some teams use retrospectives too.
If you want to start applying an incremental/iterative process in a small programming team, then I think XP is a good process to start with because it sets the bar pretty high for technical capability and is very well documented. How continuous-flow and Kanban best applies to different areas of the software development industry is still being debated on the kanban-dev mailing list and elsewhere.
I would recommend also performing regular retrospectives to improve the process and adapt it to your specific situation.
The most important part is to have a reflection/retrospective mechanmism in place which facilitates continuous improvement. Start with some process model and evolve it over time for your needs. Stop doing things that are not worth doing. Keep on doing things that bring in high value. Try new things that you think could be valuable or address specific problems.
I think Scrum works for small to medium team. Compared to XP it leaves out some details, so you can borrow from XP or do something that makes sense. Either methodology you pick, you have to consider the role of chickens(customers/managers/stakeholder/domain experts) role. Sometimes you have to play the roles yourself, but many agile methodologies work because there's external pace car with grounded knowledge of the domain.
Other key aspects are the communication level among your team and some form of quality assurance mechanism. It's hard to do pair programming if you are not located in the same building. Scrum tries to get a feature to acceptance within a sprint cycle, and XP tries to get the feature integrated within the day using unit tests, code review, and continuous integration.
*) Sprint can range from 15-30 days.
What is you question ? Is it which methodology would be most suitable ?
You don't get much benefit from all the overhead that a formallised system will impose with that size of team. Instead, try a good management technique to make sure everyone is listening to each other and blocks are removed.
I've worked with a team of that sice and even bigger on two teams that shared some common libraries. Scrum worked well for us. Now I work with a team with 6 members and we use XP and I think it works as well. The first team developed a product and the influences from 'the outer space' were not that big. So longer iterations worked fine. No we develop a customer project and therefore shorter release cycles are better for us.
But SCRUM and XP are more than that. Now we use TDD and Pair-Programming (both more from the XP world). We also have daily standup meetings that are more SCRUM like. So we adoped XP and SCRUM to work for our project and our situation.
I would start with short cylces (1 week) and reviews of this cycle. It will take some time to adopt a new methodology in your team but if the members are willing to learn and change it will work.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 months ago.
Improve this question
Recently we’ve seen the emergence of so-called “Shadow IT” within many organisations. If you’re not already familiar with the term, it refers to those who manage to dodge the usual IT governance by means such as using thumb drives to share files or “unapproved” software products to achieve business tasks. Shadow IT can emerge from within technology groups but in many cases is sourced from non-tech areas such as the marketing or sales department.
What I’m really interested in is examples you have of Shadow IT within software development. Products like Excel and Access are often the culprits as their commonality means they’re easily accessible to the broader organisation. In many cases this is driven by someone who has just enough knowledge to make the software perform a business function but not quite enough to be aware of all the usual considerations required when building software for an enterprise.
What sort of cases of Shadow IT have you witnessed in the software development space? What processes have you seen unofficially addressed by this practice and just how important have these tools become? An example would be the use of a single Access database on a folder share becoming common practice for tracking promotions across the marketing department. Remember this cuts both ways; it can be extremely risky (lack of security, disaster recovery, etc) but it can result in innovation from a totally unexpected source.
Why does IT assume they should own and control all technology in the business?
The very fact that we have a name for technology that IT does not control (Shadow IT) suggests that we'd like IT to have control over all technology in an organization.
The only real reason I can think of for IT to have control is security (even then, I'd be very weary of trusting the most sensitive data to IT). Most other reasons given against business user-developed solutions are completely false. Take the reasons above: "software produced may not be well designed...", "the software may not be well supported...". Who are we kidding here? IT's track record on these fronts is simply not good enough to claim the high ground here.
Savvy business users solve their own information problems - they have been doing so long before IT existed. Anyone remember triplicate forms? Fax machines? Photocopiers? These things didn't need IT departments to govern them and they worked very well. If IT cannot solve the problem, or IT's track record has been sufficiently poor that business users have lost faith in IT, then business users will solve their own problems, using whatever means are available to them. Access, Excel, and shared drives are frequently used very successfully by business users. If IT is to stay relevant to an organization, it needs to support it's business users needs and deliver technology that people actually want to use, not just technology people use because they have to.
I have seen an organization where a multimillion dollar portal implementation promised to solve many business technology and information sharing problems. Years later, still not in production, business users gave up, and in despair developed their own solutions by outsourcing the development of a data centric web application. Guess what? It worked brilliantly and other departments are now bypassing IT and doing the same, on their own departmental budgets.
IT is a support organization for business users. This may offend some who believe IT's place to be somewhere alongside executive management in terms of its importance to the business, but IT has to deliver what the business needs, otherwise its just justifying its own existence.
The advantage is that users get exactly what they want and need, when they want and need it. Getting a request through a largish IT shop is a trying experience for a user. IT rarely has the business knowledge to let them give the business owners exactly what they are asking for, and when requests are denied or requirements amended, an explanation in plain English (or whatever language) is rarely forthcoming.
The disadvantages outweigh the benefits. Societe Generale lost billions due in part to "Shadow IT". It can cause support nightmares when an Access application, for example, becomes essential and outgrows the capabilities of the person who created it, or that person leaves. Even a poorly written Crystal Report can become so popular and widely used that it starts to drag down the database it is accessing when reporting times comes around. And if the person who wrote that report did not fully understand relational databases, it could produce bad data in some situations; data that causes bad business decisions to be made. Using a commercial (outsourced) application guarantees that the users will not get exactly what they want; there will always be compromises, and no explanation of why they were made.
The previous poster was right. Shadow IT exists because IT does not do its job well enough. There is not enough business knowledge, not enough responsiveness, and especially not enough communication. These things are why "Shadow IT" exists. The business owners paid for the machines, the admins, the dbas, and the programmers. It frustrates them when IT loses sight of that.
At the end of the day, the primary driver for most businesses is results i.e. making money. If the business sees that it can achieve the desired outputs necessary for the operation without spending thousands on software but through "shadow IT", then I can only see it being encouraged. I feel that that it is part of our job as developers to point out the pitfalls in operating in this fashion.
The pros of "shadow IT" could be
cost - less expensive
whilst the people writing the software may not be software experts, they are likely to be domain experts and have an intrinsic knowledge of how a piece of software should function.
depending on how the IT is organized, "shadow IT" may be able to respond faster to changes and business needs than the core IT can.
And the cons
software produced may not be well designed to be extensible, handle errors correctly a d all other aspects that come from experience in software development.
the software may not be well supported or, due to the way in which it has been produced, there may be no support at all.
Over time, the average person is becoming more IT savvy. Younger marketeers and finance people know that Excel and Access make them vastly more efficient. Working without them would make them feel handicapped.
I expect this trend to continue, and Corporate IT becoming more of an enabling organization. Where you make available data, help users troubleshoot their workflow, and limit them to a specific compartment for security.
What was called software development 10 years ago, will be everyman's tool 10 years from now!
There is no such thing. There are dinosaurs, and there are people who need to get work done.
If something like 'Shadow IT' happens, it is because 'Official IT' is not doing its job.
Software developers have hundreds of little and not so little applications they need to get their work done. The IT governance organisation should learn how to handle tens of updates a day, and switch to releasing daily (and patching a few times a day). Development has learned how to do that, they are next.
Sometimes I use Amazon EC2 and/or RDS when my company's resources are not enough or would take too long to provision. I pay for this out of my own pocket but do get to achieve my goals faster. All this without having to spend painful hours in meetings, trying to convince superiors or the SA-s that I really do need to do some thing or other.
In my mind, EC2 is the ultimate shadow IT. It's super easy to get going and provides me with the ultimate control.
Well, I suppose these things are everywhere. Not a big deal if it not threatens the company operation in any way.
Ya it's a big problem where I work. Architects and DBA's try to make a centralized system but these little "Shadow IT" departments make these small apps that have their own security or duplicated data... Personally, if I was the head of IT I would fire anyone who started such a project without IT support. Kinda harsh but it's important to keep the system healthy.
Most software developers have "unapproved" software on their computers. Just expect it. I'm not sure how much I have, but I'm sure I have dozens, if not hundreds of utilities that corp. IT has never even heard of on my work laptop.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Let's say you work for a huge company which suddenly decides to do custom in-house software development. Additionally, they want to be able to offer successful developments to their customers as well (if any).
Now you are in charge of it.
What would you see as most important to build a successful software development infrastructure?
flexible to future growth
flexible on used technologies (projects with c, java, .net, web, mobile, ...)
What kind of tools (source control, forge, ...), hardware (virtual, seperate dev & production, ..), processes (guidelines, code reviews, ...), etc.
UPDATE: Please don't answer that you need the right people and the right tools. This is exactly what i am looking for.. What are the right tools and what people of what type would you hire first to join your team? Think of it as you will be the lead of that development.
Set yourself up to pass the Joel Test with at least a score of 10.
I think having the right people is going to be the most important. Nothing else will matter if your programmers stink.
Someone in charge who knows what they're doing.
Obviously, there are lots of factors, but here are the ones I'd say are crucial:
Hire smart people (and pay them what they're worth)
Select good tools appropriate for the kind of development (don't go for cheap tools)
Establish version control system and policies
Establish testing mechanisms and policies
Don't be afraid to outsource the stuff you don't know how to do
Get the best people for the job. If they aren't willing to pay for the best available, or give you a hard time over your personnel budget, you're off to a bad start.
Get the right tools for the job... software, hardware, support contracts from your vendors, etc.
Establish procedure early on for your development life cycle, and make sure that you have the people in place to make use of it. This is everything from how you evaluate Opportunity Assessments to Development, Testing, and post-production support. Make sure you have the people and the tools for each part of the life cycle.
Dont try to be flexible in technologies. First start by focusing on one technology (Java, .NET, whatever...) and then move to other if you need to. You will be able to solve problems using any technologies, but it is very hard to find people good in many technologies.
At the infrastructure level, Source Control is a must. Continuous integration is a plus. Take time to put in place a standard project layout that you will be able to evolve. It make it easier for developers to switch projects. Take time to put in place a good build process (Ant, Maven, in the Java world). Integrate your build process with your IDE so that developers dont have to wait 5 minutes to deploy their project every time they want to test a code change.
I agree with Guillaume: If you want to build a department from scratch, you need to focus. You need to build your team, have everybody grow into their new responsibilities, get to know each other etc. Trying to go into too many directions at once is the direction towards failure.
So, identify the technology you want to develop in. Since the primary goal in your example is in-house development, the in-house requirements will determine your decision. Build your team with that primary goal in mind.
For in-house development, you need at least two people who already know the company and its processes. (Two because one will definitely be ill or on holidays when the first major crisis hits you). On the other hand you need some outsiders, who are not entrenched by the "we have always done it like this" mindset, who can think out of the box. Those should also be at least two people, for the reason stated above. Your job as the team leader is to balance those two groups and integrate them into a team.
For future growth, always think in terms of organic growth.
Do not increase the team size by 200 %, hire one new guy here and another guy (or gal) there. Slowly build your team.
When you take on a new project, always think of expanding your teams expertise. Try something new with every project. That can be a new source repository, an automated daily build process, a new system to write specifications or documentation, or even a different technology (for example Java when you usually develop in .Net, Delphi or C++). Just make certain you never try to make a big leap in an important project. (I once worked for a company who decided to switch from VB 6.0 to .Net for the biggest project they had ever attempted before. They survived. Barely.)
That way your department will slowly but constantly expand its capabilities. Then when the opportunity presents itself to do development for an external customer, you will already have accumulated most of the knowledge you need in order to pull it off.
Oh yes, and smacl is right, too: You need solid QA/QM if you want your department to survive long term.
Start laying out (and follwing) your QA rules from day one. Keep them as short and flexible as possible. Add what you discover to be missing, and throw out what proves to be unnecessary or impractical.
Not sure this is what you wanted to know, but I felt the need to say it ;-)
Develop a strong QA strategy, including acceptance criteria and change control. Preferably keeping it lightweight to suit internal clients. In addition understand how to carry out requirements analysis, expectation management, and resource management.
Put another way, don't just wing it to create crappy solutions that waste more time than they save and are impossible to maintain. Take time to think about what you want and need, how you can achieve it, and what it is going to cost.
I will offer an answer more focused specifically on coding and the developers / architects role in addition to the previous answers on teams, version control, qa etc. which are of course all important.
Many of your decision is very dependant on your specific business and software structure (a single product code base, SOA, many projects etc.) But in general you should always spend significant time up front developing Core Software Infrastrcuture that will pay huge dividends during the SDLC.
Software infrastruture
Coding Naming Conventions Exception
Handling strategies Logging
Strategies Settings and Configuration
Base classes and Helper Classes
General Architecture and Layers
(Presentation, Facade, Domain
Entities, Data Stores etc.)
Design Tools such as UML 2.0
Requirements
Management / End user interaction
There are tons more, but these are certainly some basics to think about. All of the successful projects I have been involved with incorporated decent software infrastructure. I will also note that many of the project that fail have a common theme... lack of a common infrastructure in place. In most cases these failed projects are lead by a non-technical person that think they can simply throw a bunch of ideas at a few programmers and expect them to deliver in a few weeks.
Bottom line, you need to invest some up front planning and prototyping to ensure success in the long term!
Good luck.
Raiford
www.blacksaber.com
The first persons you should hire should be experienced senior level professionals. Then build up from them / with their input. Add the junior people later.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
My company has recently started using Scrum; we've done 2 sprints. We're still learning, but we've definitely exposed and fixed some problems in our development process already. So in general I think it has been good for us.
In reading many of the internet musings about Scrum from evangelists, cynics and everyone in between, three common and somewhat contradictory themes have stood out to me:
Scrum implementation fails because the processes of Scrum are not followed closely enough.
Scrum implementation fails because the organization does not adapt Scrum to its own environment/culture/practices.
The processes of Scrum are not important; only the values in the Agile Manifesto matter.
Examples of these can be seen in the responses to these SO questions:
Have you had a bad experience with Scrum or Sprinting?
Is Scrum evil?
Is Agile Development Dead?
I have to admit that we're not yet following all the guidelines of Scrum: we haven't done a release at the end of the sprints, our Scrum Master doesn't want us to move tasks out of the sprint backlog near the end of the sprint so that he can see how much our planning was off (which means the burndown chart never goes to 0), and urgent customer support issues still have incredible power to disrupt everyone's planning, for a few examples.
My question is: in trying to solve these and other issues, is it better to try and be closer to the official Scrum processes, better to be closer to some of our pre-Scrum processes, or better to meditate on the principles of Scrum to try and come up with a different process altogether?
I would say that you are really missing one of the key components of agility if you don't release early and often. To the degree that you don't do this, your process is not agile and bound to suffer the same sorts of problems that traditional, plan-driven processes have. It may be that this is a temporary condition as you are just getting used to things, but you need to start releasing soon (and regularly).
You'll always have the problem with show-stoppers, but you may be able to help this by shortening your sprint length. The customer may not be able to wait a month, but they may be able to wait 2 weeks for some things. A shorter sprint length, then, may help you to defer some requests to the next sprint making them less disruptive. You also need to be upfront with the customer that the disruptions are actually causing your pace to suffer. They may voluntarily choose to wait if they know that their chosen features are being delayed by some requests.
Another observation that I would make is that, as with almost anything, it's better to start out by following the pattern as closely as you can while you are learning. Once you have a good grasp of the fundamental principles, you can then see where some principles can be bent, broken, or replaced much more clearly to improve the process. Until you really get it, the things you change may hurt or help -- you really have no idea since you don't have the experience that tells you how things ought to be working. Unless your Scrum master is really experienced, you may want to hew closer to the defined practices until you've got a few more sprints under your belt.
Almost everything I've read on Scrum says that one of the keys is to adapt the process to fit your own situation. No two development teams are the same, and different things work for different people.
The main ideas behind Scrum are:
Have a tight feedback loop from requirements to development and back to the stakeholder(s).
This allows the development team to continually verify that they are building something that's actually wanted and allows the development to be easily adjusted as requirements and expectations change. Stakeholders can add or remove features at any point and they can adjust the priority of the features as their needs change.
Keep the software in a state where it's releasable at the end of any given sprint.
That's not to say you have releases every sprint, but that you could if the customer decides they want to have the latest stuff. This also helps a development team avoid the situation of integration hell that comes from people going off and working on a piece of the project on for months at a time in isolation.
Be completely transparent with what's going on in development and everyone needs to be willing to make tradeoffs.
This is where most projects fail and where Scrum can really succeed if everyone buys into the process. So many development projects are set up to where a release has to have X features released on Y date and no flexibility in changing that. This results in half-done features and bug ridden software as the developers cram to get in all the required features on their checklist.
The reality is, unexpected things happen in software development. With open communication and willing participants in the Scrum process, customers and developers can continually evaluate the current state of the project and make educated decisions on prioritizing the work remaining on the project.
Scrum does work. Not with all teams in all situations, but it has been shown to work.
I would suggest trying to embrace textbook Scrum as much as your business environment allows, see how that works out, and then tune it.
Why does your Scrum master not want to move tasks out of the sprint backlog? Does he not 100% embrace the principles of Scrum? (I would see that as worrying in a Scrum master)
Most problems implementing Scrum are actually just problems in the team or business being exposed by the Scrum process e.g. - if your sprints are thrown out by unforeseen support issues this suggests you are not allocating enough resource to support
Every company is different, every project is different and every client is different.
I think it's just as easy to fail by following scrum (or any other methodology) too closely in an environment that doesn't fit the methodology as it is to fail because you follow scrum too loosely in a project that does fit.
At the end some generic answer in a QA site is no replacement to serious analysis of your own project, company, team and clients - there is no magic formula and you have to make your own decision.
Answer: You need to adopt both Scrum and XP together to get the full benefits of scrum.
Reasons:
The reasons are based on years of doing XP and scrum, and specifically on what I learned from Jeff Sutherland's talk (for the ACCU in London, May 2009)
Scrum is a management technique - not necessarily a software production method. Some people use scrum in other domains e.g. preparing museum exhibitions and running religious institutions... so it has the mechanisms you need to make a multidisciplinary team deliver work adaptably in small increments.
Scrum, originally included all the extreme programming practices. Jeff Sutherland actually said that he's never seen a scrum project achieve the higher orders of productivity measured for scrum without using the extreme programming practices.
Scrum and XP both come from the same background - Object-oriented programming, specifically with Smalltalk. The programmers went off and developed XP whilst the management people created scrum. You need both aspects - development practices and management practices.
The XP practices were deliberately removed from Scrum to make it easier to adopt. - Implementing the XP practices is hard and it's difficult to get them adopted quickly. Jeff actually said that Ken Schwaber removed the XP practices to help people get started with scrum. The danger now is that this minimal scrum has become all that people see and expect.
Lots of non-technical project managers now teach scrum - but they don't have the skillset to teach XP
Not all developers find the XP practices easy to adopt - they can be hard sell and it takes a few months rather than the 2 days it takes to establish basic scrum.
Scrum doesn't attempt to address the technical issues in software development. It's just a small management process.
The strength of scrum is that it doesn't get in the way by prescribing lots of unnecessary or irrelevant technical work.
The weakness of scrum is that it doesn't tell you what good technical practices to do.
Extreme Programming does address the technical issues involved in software development and it fits very well within scrum. The reason the scrum people didn't force everyone to do the XP technical practices is that it takes about 6 months to implement those tech practices, rather than the 2 days it takes to implement the most basic scrum.
Whether or not scrum is "evil" - there are certainly drawbacks with it. We discussed the uneasy relationship between XP and Scrum at length at XP Days, London, 2009: http://xpday-london.editme.com/WhereHasXpGone
Scrum is not really the problem that you are showing. Most development methodologies work, even waterfall, as much as we like to bash it, works. Scrum does make you concentrate a little more on the important things, but it won't stop people from making bad decisions like not really following the process.
The system is pretty simple at its core.
See the problem.
Define what done is.
Create a series of tasks that will get you to done.
Estimate those tasks.
Select enough of those so that you can get something done in a short period of time.
Complete the tasks.
Rinse and repeat.
OK admittedly these steps are simplified, and I haven't thrown in a scrum master and a customer. But the point is that the framework is just a basic time management strategy. If the people in your system are chaotic and not good at getting things done then scrum really won't help them.
It's better to start applying Scrum by the book, and to really understand the underlying principles and values from the Agile manifesto, prior to customize it, so that the process does not get denatured. Be sure to run retrospectives at the end of each and every iteration (Sprint) to "inspect and adapt" your process and eliminate waste.
For your Scrum Master, he can track what is removed from the current Sprint. Also Sprints are planned based on the previous Sprints achievement, not on what was previously scheduled. I do no get its point.