Small Agile User Stories Reduce Variability in Velocity, Improve Predictability
Last Updated on Tuesday, 11 February 2014 07:37 Written by Doug Brophy Wednesday, 22 January 2014 09:15
If you estimate the size of your Agile user stories in relative story points, realize that the more large stories you have in your backlog, the less reliable will be your velocity (trend) for planning future work. The velocity trend values will be less accurate when there are bigger stories in your backlog, than when the user stories are smaller. The less accurate velocity trend is not a good basis for future planning, i.e., as a planning velocity for the next release. The reason for this lies in the nature of relative story point estimation, and there is some statistics to back it up too. Let us delve into this.
By design, relative story point estimates are increasingly less accurate for larger estimates. The Fibonacci sequence (well, a modified version of it: 1,2,3,5,8, 13, 20, 40, 100) is used as the point scale to model this. The idea is that a story estimated to be size 3 is likely to be in the small range of 2 to 5 actual points (if we were to measure actual points – which we don’t, but that’s another story). Whereas, a story estimated to be size 13 could be anywhere from size 8 to 20 in “actual” size, and so on up the scale.
This model of estimation accuracy for story size is analogous to the ”Cone of Uncertainty” (CoU) concept for software development projects. The CoU describes the exponentially decreasing uncertainty in project estimates as the project progresses. The CoU shows that the uncertainty decreases as more details about the actual requirements, design and implementation become known. “Estimates created at Initial Concept time can be inaccurate by a factor of 4x on the high side or 4x on the low side (also expressed as 0.25x, which is just 1 divided by 4).” [Construx public web page]. As the project progresses, estimates become increasingly certain until, for example, the day before the actual project completion date, that date is nearly 100% known. A diagram of the Cone of Uncertainty is shown below.
In an analogous way, early on in a project, work is not broken down into detail – we have epics and features. As the project progresses, we refine the backlog by breaking the epics and features down into agile user stories (and estimating their size). This is depicted in the diagram below.
Agile User Stories : Relative Story Point Estimates
Getting back to relative story point estimates, think about the following. The larger the estimated Agile user stories sizes, the more error there can be in that estimate compared to the “actual” size of the Agile user stories, due to the Cone of Uncertainty effect, which is modeled by the Fibonacci (or other non-linear) point scale. For example, a story estimated to be a 13 could be just a bit bigger than another story which is truly an 8, or just a bit smaller than a size 20 story. An 8-er could be anywhere from a 5 to a 13, and so on. As you can see, this estimation error is bigger for larger stories. When you add the estimated Agile user stories sizes to calculate the velocity, this error adds up. An achieved velocity of say, 30, with mostly small Agile user stories, is something different from a velocity of 30 achieved with larger Agile user stories. Which of these 30s would you have more confidence in using to forecast future velocity?
Adding the caveat, “on average”, would probably be the more accurate way to make some of the above statements. Agile user stories estimated to be size 13 could on average (meaning over many Agile user stories estimated to be 13) be anywhere from an actual size of 8 to 20. And what of this notion of “actual size”? We don’t measure that, and these are all relative sizes, right? But there must be some actual size once the story has been delivered. Whatever it is, whatever its units – Lines of Code, Function Points, some function of effort, complexity, and risk – it does exist, and all Agile user stories, once completed, have an actual size. We don’t have to know that actual size or units to make this argument about the error in the estimates and its effect on velocity.
This can be explained in terms of statistics as well. For those not interested in the math, you can skip ahead two paragraphs. I believe the case has been made for smaller Agile user stories in your backlog reducing variability in your velocity.
If we think of the error in any particular story size estimate (versus the “actual” size) as a random variable (i.e., over all Agile user stories estimated to be a particular size, say a 5), then the variance on that error is bigger for larger sized Agile user stories, due to the CoU effect. When the story sizes are summed, the variance of the error in the resulting sum (the velocity) is the sum of the variances (assuming for simplicity that the story size estimate errors are independent and normally distributed, which they probably aren’t, but my hunch is it – the variance- only gets worse/ bigger for other distributions). This statistical property may be difficult to perceive on any one Scrum-team-sized project, but think about it across an entire large organization of many, many projects over time. The statistics play out over the large number of samples. The more large Agile user stories that are in the sprint backlogs across the organization, the more variance will be in the velocity and thus the less reliable will be release commitments based on those velocities.
This strongly suggests that you (and your organization) will be much better off in terms of forecasting your rate of progress (velocity) for planning purposes if you have mostly small stories in your backlog. Not only does the math tell us this, but it passes the common sense test too. If you have a mix of small to medium and large stories in your backlog, all it takes is one or two of larger stories planned into a sprint not completing to significantly reduce your velocity from what you planned. The next sprint you may spike up in velocity due to taking credit for the large “hangover” story(s). If this pattern repeats sprint over sprint, your velocity will vary widely and predictions based on it will not be very reliable. On the other hand, if you have mostly small user stories in your sprints’ backlogs, then if the team occasionally does not finish a story this will not impact the velocity statistics significantly… the average velocity should be reliable enough for forecasting.
Ideally then, we would break all Agile user stories down to about the same small size and forget about sizing them – just count them and use count for velocity. However, in practice I have found this is not practical – there are usually constraints on how / much we can break down Agile user stories. Furthermore and perhaps more importantly, in practice we find that the discussions around the story size estimation tend to be very valuable in terms of developing a clear, shared understanding of the Agile user stories.
The moral of the story is… break down those Agile user stories, or more generally, break down your work into as small chunks as is practical. There are many benefits of working with small Agile user stories besides the reduced variability. These advantages include: increased focus, which helps prevent failure; earlier discovery / faster feedback; shorter lead time/ better throughput; and reduced testing overhead. Those are good candidates for the subject of future articles.
A corollary arising from this observation is that if you cannot break down all your Agile user stories to be relatively small, realize that velocity may not be a reliable planning parameter. You might want to also look at story cycle time (per size), for example. In addition, the average story size (velocity/num_stories) could be tracked and monitored. The idea would be that lower is better – velocity will be more reliable for planning when there is lower average story size in the historical backlog.
To help improve your user stories read 10 Tips For Better Story Estimation by Jann Thomas.Learn More
Agile Release Planning Kick-Starts Agile Adoption
Last Updated on Tuesday, 4 February 2014 11:37 Written by Doug Brophy Wednesday, 3 July 2013 07:00
Organizations just starting out in their Agile adoption are eager to begin seeing results. After proper Agile teams have been formed and trained in the basics of Agile, they are ready to begin practicing. But what is the first step in your Agile adoption, where should they start? Agile Release Planning is the answer.
Agile Adoption : Release Planning
Agile Release Planning enables teams to speed up their Agile adoption. It brings the stakeholders together in a working session focused on developing the software requirements and plans to develop the software incrementally and iteratively. Risks, issues, dependencies and assumptions are identified and addressed in the planning.
This kick-starts the Agile adoption practices, enables teams to begin delivering results quickly, and sets them up for success with the release. A recent coaching engagement provided a good example of this.
An Agile development team had been formed to build the next generation platform for a Resource and Operational Planning tool suite. A new platform with a single, common database and user interface was needed to better integrate and present the product offerings, in line with the product vision and roadmap. The legacy platform had become overly burdened with technical debt after years of ad hoc development, and became costly to support and extend.
The new team was in its forming stage. There were conflicting views and differences in understanding of the requirements for the next generation platform. There were strong differences of opinion about what and how to leverage from the existing platform architecture and code base, which was large and complex.
Most of the new team came from the legacy platform development team. But the Product Owner (PO) and Product Manager (PM), two developers and a tester were from a different but similar product suite. The PO had recently been part of a similar platform uplift program for the other product suite, and had a developer background. As such, the PO had strong opinions about how to execute this project, including on the subject of architecture and code reuse. He strongly advocated for starting over with a new clean architecture since, for example, the existing platform had lots of business logic tightly coupled with the UI.
But the legacy development team members were very concerned about doing this. They were counting on re-using a lot of the legacy code (and architecture). In their minds, that would be the fastest way to complete the project, since they were very experienced in development of the legacy platform.
Agile Adoption : Release Planning Workshop
To overcome these issues, we held a Release Planning Workshop. Representatives from Product Management, Development, and other stakeholders in the release were in attendance. Our workshop began with an intentional focus on reviewing the business goals and associated product vision and roadmap, as a precursor to talking about architecture, epics, features, and stories. This quickly made it clear to Product Management that they were not fully prepared to discuss the platform requirements in this manner.
We were able to identify some epics/ features and stories on that first day of the workshop, but it was mostly a training exercise for the team to learn what and how we do release planning to improve their Agile adoption. With this new understanding, Product Management asked for a week to go “do their homework” to fully prepare for another try at Agile release planning.
Agile Adoption : Outcomes
We reconvened a week later for “take two” of release planning. This time we made rapid progress and accomplished the goals of release planning. Notable outcomes include the following.
1) Development finally understood and accepted that Product Management wanted a new, simplified, de-coupled architecture – not just a refactoring of the existing architecture. This would be the basis for a normalization of how data entry, processing, and presentation would be done across products/ features running on the new, common platform.
2) Delivery date would be negotiable as the project progressed. Development had been assuming that there would be a fixed due date 9-12 months later, which they would be pressured to accept, even though it would be high risk to commit to a delivery date that far out. But Product Management did not have a fixed due date in mind. Instead, getting the new platform they envisioned was the priority for them. They could be flexible about the final completion date.
3) Product Management did want to see incremental milestones of progress, and fortunately, that is exactly what Agile release planning is designed to provide. The plan was for Development to deliver a release every quarter, internally only, incrementally and iteratively building up the platform with every internal release, leading up to the final product release.
4) With a well-groomed release backlog coming out of the gate, the team could begin planning and executing sprints, and gauging their velocity. This in turn would enable refining of the release plan as the project progressed.
5) The team delivered a demo by the end of the second sprint, and every sprint thereafter. They recently made their first quarterly release milestone.
Agile Adoption : Summary
For a new development team with only basic Agile training prior to release planning, within two weeks we had a mutually agreed upon viable release plan that enabled the team to begin executing sprints aligned to the first quarterly release. They had learned how to capture product requirements as user/ role and goal –centric epics, features and stories, create acceptance criteria, estimate stories in relative story points, and they were ready for the first sprint planning and execution.
Without good Agile release planning, teams often try to start sprinting with a poorly groomed backlog, with no clear direction towards delivering on release goals. This can result in wasted time and effort (money), engraining of bad habits, as well as frustration and disappointment with Agile adoption.Learn More