Understanding Risk in Your Project Portfolio
A post or so ago I used the process of designing and building a house as a metaphor for how to plan an agile project. I was basically making the case that we would never spend our own money the way we are asking our stakeholders to spend their money. We would never give a home builder $500K dollars without some sort of commitment around what we were going to get for our time and money. As consumers, we want to know what to expect.
That said, I don’t think it’s reasonable for us as consumers to expect that nothing is ever going to change as we begin building and learn more about the kind of house we really want to build. We can’t go back to the builder and demand changes for free because we didn’t understand exactly what we wanted. We create high level plans, set budgetary estimates, collaborate through out the process to guide the build to successful completion and a satisfactory outcome.
Why This is Relevant to Software
Software projects are all over the place in terms of risk and uncertainty. If I am building an e-commerce site from scratch, and we are using known and well understood technologies, that is a pretty straightforward type of project. I might not know everything that I want on day one, but I can understand the major parts and pieces. I can understand the underlying implementation, create a budget, and probably come pretty close to meeting that budget at the end of the project.
This is the type of project where the metaphor of building the house makes sense. Create a high level plan, develop budgetary estimates, establish constraints, and work collaboratively with the customer to guide the solution into a successful outcome. As the consumer, I should be able to show up with my money, spend a minimal amount of time up front, and get started on the project. I should have a pretty good idea of what to expect when the project is done.
Not every project though follows this same kind of rule. Some projects are much more uncertain. Some projects are funded and neither us nor the customer know much of anything about the target product. They might know they have a need, but understanding how to solve for that need is elusive. Some projects are being built for customers that don’t even exist yet. We are creating a market and trying to figure out what we are going to sell as we go.
Some projects are built on top of legacy systems that have a ton of technical debt, poor automated testing, no ability to produce a daily build… let alone do continuous integration. Some products have so many defects and undocumented edge cases that interacting with or changing that software is an exercise in futility. How do you estimate for a project where requirements are unknown and the technology platform is virtually unknowable?
Estimating the Un-estimateable
I do believe there a projects out there which defy estimation. Projects where the requirements are truly unknown and maybe even unknowable. Projects where the risk and technical uncertainty are just too great to have any idea what it is going to take to do the project. When you couple this with the fact that people are not fungible resources and don’t burn estimates at necessarily the same rate, trying to predict anything doesn’t seem to make much sense.
It seems to me that we should look at these unknowable kinds of projects differently and apply a different approach to managing them. For the first kind, it seems reasonable to create a high level plan, look into the future to identify and mitigate risk, provide estimates, and plans, and have a pretty good idea of what we are going to get. We can still do agile, we can still inspect and adapt, we can still change requirements in the small while converging on our higher level goals in the large.
The second kind of project can’t be handled the same way. We don’t know the requirements are or what it will actually take to build them. We can’t do high level plan, or look into the future to identify and mitigate risk, estimates are nonsense, planning is an exercise in futility, and (if we are honest with ourselves) have to admit that we really don’t have any idea of what the customer is going to get for their time and money. It’s a bit of a crapshoot.
If we can fundamentally agree that both kinds of projects exist, maybe we need a different way to talk about each of them. Maybe for the ones we can predict, we use adaptive planning techniques… user stories, timeboxes, estimates, roadmaps, rolling wave planning, or whatever… to make sure we are driving up communication and collaboration, delivering early, getting feedback, and tailoring as we go. For the ones we can’t… I think we need to start use the language of investments.
Why Do We Call It a Project Portfolio?
If we use the notion of an investment portfolio as a metaphor to talk about a project portfolio… we can start to think about our collection of projects as a series of investments. In a balanced portfolio, I might have some of my money in lower yield, relatively safe, municipal bonds. I might put some of my money in a higher risk mutual fund and hope for a bigger return. I could allocate the remainder to even higher risk, higher reward mutual funds, individual stocks, or maybe even a startup.
The goal of my investment strategy is to preserve some of my capital if things go really bad, while at the same time, putting some my money at greater risk to get a faster payoff. The general principal is that the more risk I assume the more money I should make on the backside. The corollary is that the more risk I take, the greater chance I have of loosing everything. Let’s tie this back to our project discussion.
Projects are investment increments in our Project Portfolio. If I am investing in something that is well understood, in a well understood market, one where we can adequately predict outcomes, chances are I’m not the only one that can see the opportunity or has the capability to build it. I am probably in a more commoditized market and there is a pretty good chance that my long term yield on the investment will be lower. It’s important to control cost because margin isn’t as great.
What if I am investing though in something that is difficult or unknown? Chances are I won’t have the same level of confidence I’ll see a return on my investment. If I am successful, I might be able to change the world and get rich. If I fail, I might run out of money and be unable to feed my family. That is the nature of risk. I think the fundamental problem with projects and estimates is that companies are putting money in very high risk investments expecting a guaranteed return.
Managing Portfolio Risk and Return
Personally, I think we should estimate and plan for the stuff that is estimateable. Many, many software products fall into this category and with proper forward planning, risk identification, and a willingness to inspect and adapt and get continuous feedback, we would greatly increase the odds that these projects could deliver a relatively fixed scope within a set of established time and cost constraints. We’ll adapt in the small to deliver in the large… just like my house example.
The projects which can’t be done this way need to use a language of investment and risk. These are not safe investments. We are putting money at risk in hopes of getting significant return. In these projects, we are spending money to learn, and based on what we learn, we adapt and we change, and maybe we even pivot into something entirely different we never even expected. As an investor, I can continue to invest as long as I want, but I never get a guarantee.
The fundamental disconnect is when we start to blur that which is safe and that which is risky and try to use the same language to describe both. We try to use predictive techniques for stuff that needs to be adaptive and we try to converge on outcomes that need to be inherently emergent. Many companies see this and separate development from R&D. Some though are accidentally investing in R&D when they think they are doing safer product development.
When you are doing R&D, an inherently high risk investment, with the expectation of safety and safe returns… that is when we get into trouble. We get ourselves into an irrational place where we are making bets without understanding the risk profile of the bet. You can’t manage your business that way… you can’t manage ANY business that way. My take is that this fundamental disconnect is what is driving the discussion around estimating in software right now.
Quick Summary
I get really tired with the never ending debate around should we have projects in product development. I get really tired of the debate over estimating or not estimating. There are fundamentals underneath these debates which are seldom addressed. What is our domain? What is our context? What are the goals of our delivery system? What is the nature of our customer and their needs? What do they have to spend? What do they expect in return?
How I manage my consulting company is often quite different from how I recommend that companies run their product delivery. The world I live in is often quite different from the world my customers live in. We have different constraints and different variables. We have different customers with different needs and sell different products. I like this notion of looking at projects or products as investment vehicles with different risk profiles. Understand your risk and invest accordingly is good advice.
Check out the previous post, Should You Use Agile To Build Your Next Home?