Agile vs Waterfall
These words have become completely overloaded when discussing product development. Lots of conversations about helping organizations improve their product development processes go sideways based on individual perspectives about the meaning of Waterfall and Agile. At this point these words don’t provide a distinction that is helpful when we are trying to figure out how to build products in organizations. This is a red-herring contradiction. When I hear someone say “we need to do that Waterfall” or “that wouldn’t work in Agile” or “we can use Agile for that project” then I want to stop and ask “what does Agile mean to you?” or “what do you mean by Waterfall?”
I want to break down this debate into a clearer discussion around specific characteristics; Type of Effort, Upfront Planning, Sequencing and Feedback, and Composition of Teams. Then talk about how understanding these characteristics helps us improve the ability to be predictable and achieve the fastest time to ROI.
Type of Effort
Product Development encompasses different types of efforts. We frequently view Agile as the model to do exploratory product development. In exploratory projects, teams are exploring new ways of solving a problem or trying to discover a new customer or market segment. Mike Cottmeyer has been calling these divergent projects. Fast time to value involves rapidly performing lots of experiments, getting feedback on your hypothesis, and adapting your approach.
At the other end of the spectrum are convergent projects. In a convergent project – there is a scope, a due date, and a budget. Agile is a very strong approach for delivering convergent projects. Organizations doing convergent projects need teams to have a pretty strong ability to make and meet commitments and they have a significant need to manage risks and coordinate dependencies. These organizations can make the trade-off decisions necessary to maximize the chances they will converge on the best outcome.
Most of the organizations we deal with need to have a pretty clear idea about what they are going to get — and when they will get it — before they decide to do a project. They are primarily performing convergent projects. You manage convergent and divergent projects differently. A frequent challenge we see is when organizations move to Agile, they treat all projects like divergent projects but they are actually in convergent projects.
Not planning isn’t an option. The question is how much planning do you need to do to be successful? It isn’t possible to be predictable if you don’t do a sufficient amount of planning. The red-herring in Agile is that you don’t do any planning. The Red-herring in Waterfall is that you get all planning done up front. Regardless of your approach to the project, base the amount of upfront planning on how much you need to do to answer the questions of, “What will I get?” and “When will we finish?”.
Sequencing and Feedback
As I stated above, most organizations need a pretty clear understanding of what they are going to get and when they will get it. We see different modes of sequencing and feedback. Many organizations accomplish predictability by sequencing the work in phases. For example, Analyze, Architect, Design, Construct, Integrate, and Test/Remediate. The intention is to make good decisions upstream that protect downstream efforts.
Testing and integration feedback, deferred until late in the project, is usually the problem. Challenges identified late put the project under duress at a point when it is difficult to respond effectively. This is the project that takes as long to complete the last 20% as it took to deliver the first 80%. In phase based projects it is very difficult to be predictable because risks tend to manifest late in a project. Time to ROI is negatively impacted when project durations run past the planned delivery.
The other form of phasing and sequencing leverages frequent delivery of working tested product. When we sufficiently plan, sequence frequent deliveries to maximize the potential for success on the project. Resolve risks and dependencies early, while there is time to respond to them. Deliver more valuable work earlier in the project. Provide the opportunity to explore options and stop after sufficiently solving the problem. We gain insight from frequent deliveries and this insight is used to make trade-off decisions to maximize return on the project. Projects using value based sequencing can be more predictable than phase based projects. Time to ROI can also be maximized in value based sequencing in those cases where a smaller subset of the potential set of features can be deployed to deliver value.
Composition of teams
We see people belong to functional groups and are matrixed into various project teams, in most organizations. It makes sense when the sequencing is in sequential phases. Individuals are often spread across multiple projects simultaneously. Using this method, we can maximize the utilization of everyone in the organization.
In another model, people belong to collaborative teams that work together to deliver working, tested product frequently. In this approach, team members have almost instant availability to each other and everything needed to deliver its next increment. People are generally dedicated to a single team so they can work collaboratively and with near immediate availability on each frequent delivery. Co-located teams help with this immediate availability.
Team composition plays a crucial role in the ability to leverage value-based sequencing. The collaborative team composition approach may have a negative impact on utilization. (I disagree with that assumption, and I’ll tell you why in a follow up blog post next week). The value gained from increased predictability and faster time to ROI often outweighs the impact of lower utilization.
Arbitrary references to Waterfall and Agile don’t provide much useful information. Be as predictable as possible and maximize time to ROI. Have clear conversations around team characteristics that are best suited to the problems we are solving. Understand if your project is convergent or divergent and determine the appropriate amount of planning. Assess if your project would benefit from phase based or value based sequencing, and whether functional separate or collaborative teams are the best way to build software.
In my experience, Agile is about collaborative teams and value-based delivery. I prefer this approach on divergent projects that require little governance and upfront planning. I also prefer this approach on convergent projects that require significant governance and upfront planning. So I am very comfortable with Agile – whether the project is a divergent or a convergent type of effort and regardless of the appropriate amount of upfront planning. Not everyone sees these the same way because not everyone has the same perspective of Agile. Let’s move away from red-herring conversations of Agile versus Waterfall and focus on meaningful characteristics.