How to Structure Your Agile Enterprise

WRITTEN BY Mike Cottmeyer

Agile Org Structure


First of all… this is a really, really, really big topic. If we are lucky, we’ll get a start and maybe lay a foundation for future conversations. My goal in the next 1000 words or so is to at least introduce the foundational concepts, and frankly… help me see where I want to go with this. So… with all my pre-qualifications in place, let’s see what we can do.

I’ve talked about what it takes to do Scrum well and explored many of the anti-patterns that cause Scrum to fail. One of the biggest challenges to adopting Scrum is the ability to form complete cross-functional teams. Before we get into how to solve the problem, let’s first explore why it’s so hard to begin with.

Not Small Agile Teams

First of all… let me share that I have NEVER worked on a small agile team. I’ve coached many of them, but my introduction to agile was in the context of large enterprise class financial systems… things like online banking and bill payment. The kinds of systems where the company makes a penny or two on every transaction and does millions every year.

Think about what it takes to build these kind of systems. Quite often you had a hundred or so people involved… maybe more. You are dealing with a very diverse technology stack. At the time, we had people building software in .NET, Java, C, and Cobol… interfacing with an Enterprise Data Warehouse, leveraging web services, PeopleSoft, and any number of other systems.

To make things more complicated, quite often the systems we were building were actually systems of systems. Not just in the Systems Engineering sense of the word, but actually products of products where the product we were building was made up of other products, each with their own customers, product management, and deadlines.

Introduce Changes

Our job was to introduce changes in these systems, not break any of the other existing functionality, make sure we delivered on time, without getting in the way of anyone else delivering on time. Needless to say, these were pretty complicated delivery efforts and required tons of analysis, architecture, and business sponsorship across the entire organization.

All said, the core team of folks was maybe 100 people or so, but we were interfacing with an organization that was at least 400 people.

Okay… so now think about the guidance that we give for the formation of a Scrum team. A cross-functional group of 6-8 people that have everything necessary to deliver a working, tested, increment of the product. How in the world would I get 6-8 people with the skills necessary, the understanding of complex workflows, and the domain knowledge to touch this many systems?

Lack of Sub-system Level Ownership

I’m not suggesting that it’s *impossible* to do this, but I’ll tell you… most products have not been built with the underlying architecture, test harnessing, or continuous integration necessary to even take a stab at shared code ownership across such a diverse stack. At some point, the lack of sub-system level ownership would very likely erode the integrity of the overall solution.

Even suggesting this would be incredibly risky proposition and likely a non-starter in most organizations.

So where does that leave us relative to forming Scrum teams? The idea of feature based teams in organizations building these kinds of systems, at this level of scale, is not practical guidance. If you can form a team like this in your environment… stop reading now and go do it. It is clearly the simplest way to go. If not, you have some work to do to figure this out.

Forming Capability Based Delivery Teams

Most organizations are viewed as a series of capabilities. A product is effectively a capability. A feature set within a product can be a capability. Services that support multiple products can be a capability. Things like accounting, and finance, release management, and deployment can be capabilities. Performance testing and integration can be capabilities.

At the team level in any organization, rather than getting hung up on the idea of the cross-functional feature team, the idea is to form Scrum teams around capabilities. The capability based Scrum team operates just like any other textbook Scrum team, it’s just that their output is a capability that can be consumed by other capabilities within the enterprise.

Capabilities as defined here tend to have pretty well defined inputs and outputs, contracts and service levels with the rest of the organization. They typically rely on a less diverse technology stack and the level of domain knowledge necessary to iterate the product is less relative to the entire system you are trying to build.

The comparison we like to make here draws a parallel between refactoring a legacy architecture into a services oriented architecture. The goal is to get each of the services loosely coupled and highly cohesive. Similarly forming Scrum teams, we want each team to be loosely coupled and highly cohesive… just not necessarily responsible for an end-to-end slice of the entire product.

That said, if you think about it… we might just have a semantic argument at play. One man’s product is another man’s service. So from the point of view of the Scrum team, they are building a product… it’s just that the product is a service that can be consumed by the rest of the organization… building blocks if you will… that the enterprise software is based upon.

Forming Products From Collections of Services

The problem with services oriented teams is that, more often than not, our customers aren’ buying services from us (although sometimes that actually does happen)… it’s our internal product organization that is consuming the service. In this case, you may very well have a traditional Scrum team formed around a product or feature set of the product consuming the output of the services teams.

In this case, these Scrum teams operate more like a textbook Scrum team. They have a backlog, meet with the Product Owner, and produce working tested software every sprint. But get this… by definition, these teams wouldn’t have everything necessary to actually deliver the software, they are dependent on the services in order to create an end-to-end slice.

Interesting Insight

The problem with this scenario, isn’t that the product team is dependent on the services team.  The problem is that the product team and the services team are both delivering their backlog at different rates and potentially at different times. When an end-to-end feature requires both teams to deliver their piece, we create a planning dependency between the teams that is difficult to manage.

I’m clearly jumping into the governance conversation here a bit.  It’s these planning dependencies that are killing large scale agile adoptions. If you take the SAFe route, you try to fit everyone into the room to work out an integrated release plan. We’ll often do something similar but leverage a smaller group of people, running features through a Kanban system.

The idea is that we’ll manage requirements decomposition through a Kanban, but have queues to batch requirements into sprints or releases. This is, if the organization isn’t ready for a continuous flow delivery model. Either way, the challenge remains.  When you have a large, multi-team system, it’s very likely that you’ll find bottlenecks in the system. Some teams will go faster than others.

The problem with bottlenecks is that one team races ahead building services while the product is struggling to make forward progress. Quite often, it’s the products that are able to race ahead and the service creation that struggles to keep up. In both scenarios, you end up effectively with partially complete code that no one can use.

High Cohesion and Loose Coupling

To alleviate these problems between teams, teams must be highly cohesive and loosely coupled. That means that we should avoid at all costs committing to features that require more than one team to be done in the same place at the same time in order to make an integrated delivery.

Maturing agile enterprises, need to move toward a model where the product teams and the services teams operate on their own individual cadence and are able to deliver software at whatever velocity they are able to commit. In practice, this results in three rules for team-to-team interaction in a large enterprise.

Three rules for team-to-team interaction in a large enterprise

1. Product teams can only commit to delivering features based on services that currently exist. This is the least risky approach.

2. Product teams can commit to delivering features based on services that are on the near-term services roadmap. This carries some risk, but is usually pretty manageable.

3. Product teams inject dependencies, by requiring creation of services while the feature is in flight. This is the highest risk scenario.  Avoid if at all possible.

Basically, Scrum tries to solve the dependency and bottleneck problems by requiring each team to have everything necessary to deliver a working tested increment. We ask each of the team members to serve as specializing generalists.  We have people swarm on backlog items to get done and stabilize velocity. I totally agree with all of this.  It’s just not practical at scale.

We’ll explore this more when we talk about governance.  For now, let’s recap where we are.

Large organizations often can’t implement the idea of a complete cross-functional feature team. In these organizations, Scrum teams need to be organized around capabilities, products and services that must be combined to evolve delver software at enterprise scale. The more effectively we can decouple the delivery teams from each other, the less planning overhead we’ll require.

Product Owner Teams

I’ve been on record for quite a while that I believe the single product owner construct is a myth in most organizations. I get why this was created and why it should work. Without a singe voice of the business to give clear unambiguous guidance to the team, they are going to thrash. The problem is that a single person with the necessary skills and authority to create the backlog don’t often exist in companies.

We are a big fan of the Product Owner Team construct and implement it almost everywhere we coach. The idea is that to sufficiently groom a product backlog you need several points of view. We encourage the PO Team to have the product, architectural, analyst, and project management point of view.

I’m not talking about roles or people, I’m explicitly talking about the perspective brought by each of these disciplines. This is regardless of who actually does the work. When you are working in a fixed time and cost methodology, and varying scope to meet business goals, it requires more than just the business’s point of view to make the appropriate tradeoffs.

When working in a predictive-convergent organization, our job is to figure out how to solve relatively fixed business problems with somewhat pre-determined solutions. By having these four perspectives working together, you can make real time trade-offs about what to build and how to build it to optimize your chances of being successful.

Portfolio Governance Teams

Similar to the Product Owner Team structure, we find that many organizations need cross functional teams of leaders to approve the investment increments at the portfolio level of the organization. Depending on the size of the organization, this can be a team of directors and managers. I’ve implemented it as a team of the CIO, CTO, the VPs of Engineering and Marketing.

Most organizations have some sort of governance model in place… a phase gate model if you will. Again, we’ll talk more about this when I do a post on agile governance.  For now, just think about leaving your existing phase gate model in place.  Model it in a Kanban, run much smaller batches through the system, limit WIP, etc.  You get the picture.

I like cross-functional teams providing guidance on requirements decomposition. For the same reason, I like cross-functional groups of leaders providing guidance on how work is approved and flowing through the system. The leadership team can see it if delivery is block and help get the resources necessary to the teams to get things moving again.

That Still Isn’t Everyone

At this point, we’ve talked about teams formed to help work get approved and into a queue to be built.  There are teams of people responsible for requirements decomposition. There are teams responsible for high-level architecture and building products.  There are even different teams responsible for delivering services. What about the rest of the organization?

Some organizations have strategy groups, marketing, sales, integration, release management, performance testing, infrastructure, DevOps, and SCM. I’m certain there are many groups I’ve failed to mention. Should we tuck these people into Scrum teams, or maybe disband these groups? Perhaps we let the Scrum teams do everything themselves?

That might work for smaller organizations. But again, in many of the large organizations we work with, these functions are necessary, and maybe even necessarily separate. For us, this is where the Lean part of Lean-Agile program and portfolio management comes in. It doesn’t matter so much how these teams work, if they use Scrum or Kanban or Waterfall.

What matters is that these teams agree to work with the other parts of the organization in small batches. They will limit work in process and reduce cycle time. At scale, there are going to be upstream and downstream groups that have to work together to get the product you’re building into market.  The current problem de jour in the scale agile discussion is coordinating the delivery across those teams.    I don’t believe that Scrum is the answer in these kinds of organizations.


Okay… where does that leave us? When designing your agile enterprise, you’ve got to ultimately take into consideration how you are going to form teams. You have to understand how the work of those teams is coordinated. Additionally, you have to understand what you are going to measure to know you’re making progress. We explored (in somewhat gory detail today) the structure side of the problem.

I made my best case that the notion of the cross-functional feature team will break down at scale. There is just too much subject matter expertise, too much domain knowledge, and too diverse a technology stack for one team of 6-8 people to build much of the large scale software that is getting built today.

If this is your reality, that means you’ll have to consider implementing Scrum teams based on products and services. More generically, implement capabilities that you can reuse across your organization. Governance is largely coordinating work across these teams.  In the end, if you decouple teams more and have them work at their own pace, you’ll actually be more agile.

Product and Services Teams

Product teams and services teams don’t make up the whole of the agile enterprise. We have many upstream and downstream groups that have to be able to work effectively with the Scrum delivery teams to get the product into market. IMO… this is something that Scrum doesn’t have much to say about. SAFe is making a pretty good stab at it.

More generically though, the idea is to model the rest of the organization into a Lean/Agile value stream. It is to run smaller batches through the organization and limit WIP. It is to work to reduce cycle time and reduce time to market. At scale, business agility isn’t about how effective Scrum teams are.  It’s about how effectively the entire organization can get product into market and generate a return.

Hope this gives you guys some things to think about. IMO… this is the missing piece of the puzzle for many companies.

I usually try not to directly pimp our services when I write, but helping people solve this problem is a significant part of our practice. We use all sorts of techniques from interviews and collaborative sessions to a very structured approach called Capability Modeling. This allows for structured conversation and results in a heat map of your organization.  This is not only for forming teams, but for having conversations about the impact of changes you might want to make to your technology platforms. It’s proving to be a pretty powerful lever for making organizational changes at scale.

Good luck, let us know how we can help. Looking forward to your feedback.

Check out the next post, Shu Level Agile Isn’t The Same As By-The-Book Scrum.

Check out the previous post, Structure, Governance, and Metrics on Large Scale Agile Teams.

leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *

12 comments on “How to Structure Your Agile Enterprise”

  1. Kelly Waters

    Hi Mike,

    That’s a really great post, thanks for taking the time to put all that down. I agree totally with your comments. The principles of Scrum and agile for teams are still great, but as things scale up they need to be adapted…


    • Mike Cottmeyer

      You’re welcome Kelly… trying hard to get all my latest thinking out of my head and into the blog. This is stuff I have been talking about with clients for quite some time, just haven’t had the bandwidth to write much until recently.

  2. Peter Trudelle

    Great post Mike! Thanks for giving us the benefit of your experience. These organizations do need to start where they are, though I wonder how much of the existing scale and structure are inherent to the business need rather than the unintended side effects of traditional organization and management. Some structures and processes seem to encourage growth, complexity, and specialization for their own sake, leading to even small organizations where no single team can deliver value end to end on its own.

  3. Mike Cottmeyer

    I think the current state of the organization is the direct result unmanaged scale and complexity. The models we thought worked 20 years ago aren’t the same ones that we now know will work today. The parallel between transforming and organization and refactoring a legacy application I think is very strong and very powerful. That said, the approach I am recommending is a modern and consistent way of looking at organizations at scale. If I were building an enterprise class solution today, this is how I would build it from the ground up. I would pay more attention to high cohesion and loose coupling between teams, organizing teams around significant architectural and business focused capabilities, and put in the program and portfolio management constructs to make sure we are focusing on value, reducing WIP, elevating bottlenecks, and reducing cycle time as an enterprise. That said, it’s not how I am building LeadingAgile… we are different kind of business even though some of the patterns hold. One size does not fit all, but again… I don’t see this as a tradeoff or a compromise. I do see it as a valid pattern for agile at scale.

  4. Christian McMahon

    Hi Mike, Great post. I would like to republish it on my blog with full credit of course. My site is written for and followed by CIO’s / IT leadership positions. Thanks, Christian

  5. Tristan Kromer (@TriKro)

    Very good article. I like the portfolio governance team.

    I often find myself debating how to implement a cross functional team dedicated to continuous improvement of processes. A team looking at the red tape and figuring out where to slash things so that the product teams don’t get caught up in branding, legal, or other approval when deploying small scale tests or features.

  6. Mike DePaoli

    Hi Mike,

    Good post. Taking a look at natural systems can re-enforce your thinking. Natural systems are governed by simple rules, agents in the system operate within these rules, their contextual interaction creates complexity, complexity tends to migrate toward more complexity and adaptation. Overtime even a new governing rule may form.

    If Lean-Agile Organizations adopt a (or ‘the’ depending on your zealotry) set of Lean-Agile rules for their value delivery and apply them at all levels, you have a system that can be loosely coupled but tightly cohesive. In a software product development organization, it is my experience that this allows for contextual adaptation in the system but the system can still be constrained toward moving in the same direction.

    At Agile Development West – 2013 I delivered a talk what I termed LALF, the Lean Agile Leadership Framework. It provides such a set of rules for creating a fertile management / leadership culture that is conducive to a Lean-Agile product development system. IMO, especially at scale, it is important to have the culture of an organization be fertile for the type of development system that is being grown. Lean-Agile at scale also is compromised when evolved team level practice (like plants raised in a green house) are exposed to the harsh reality of a more traditional enterprise product development environments, characterized by predictive planning, large batch execution, low transparency, long cycle feedback loops and individual focused motivational philosophy. Basically a desert for cultivating human performance.

    It will be interesting to hear your thoughts applied at the ecosystem level as you evolve your thinking here in your blog.

    • Mike Cottmeyer

      One of hallmarks of our consulting practice is that we don’t lead with culture change. Culture is ultimately important for the long term success of any change initiative, but forming teams, creating policies, and defining metrics that support the delivery goals of the organization, is IMO, the most important first step. Once the lean-agile delivery framework is in place, you have a foundation from which culture can emerge and adaptation can take place. I have seen way too many change initiatives fail due to the weight of ineffective delivery models that are codified into the environment and crush the early stage cultural shifts. I just don’t believe that the right structures will initially emerge in the presence of simple rules and agile leadership… at least that has been my experience. After the initial constructs are in place, I’d tend to agree.

      BTW – I realize there is a risk I have myopically focused on one small aspect of your post… but that’s what I felt compelled to add ;-) Thanks for the comment!

  7. Mike DePaoli

    I’m not suggesting that the emergent culture should be the primary, upfront focus. It will evolve with the more tactical changes you discuss, which by the way, I agree with. The point is that the same governing rules should be introduced in management thinking else the evolving team level changes can violently collide with the established organization. Having management model the values and fundamental rules you are trying to instill at the team level, IME, is quite helpful to successfully enabling change in their teams.

    Purely hoping that the more tactical changes to team level working policy and techniques will eventually extinct the values and practices that are counter to lean-agile in the wider ecosystem is one approach, but addressing the issues more systemically, in a balanced manner, can help to mitigate the pain of the systemic change.

    There are different scopes of influence in a change initiative. I know you don’t like to only operate at the team level but to have the ear and support of senior management :-)

  8. Mike DePaoli

    One other comment. The basic rules I mention aren’t the only explicitly defined policies, they are the constraints on the system that keep the more specific policies and practices that emerge aligned to the same end. This applies at all levels of the organization.

  9. Dot Tudor

    Hi Mike,

    Great post, Mike – sorry I missed it until now! The principles of Scrum and Agile for teams really do need to scale up and we need to retain the power of incremental delivery of value. The roles we are asking for in the creation of any backlog are multidimensional: they are cross-functional and across different skills. They are also a mix of future-facing and BAU perspectives; they also represent multiple authority levels within an organization.
    DSDM has moved some way towards this in a project world, in defining Business Sponsor, Business Visionary, Business Ambassador and Business Advisor roles at different levels of authority, and having technical and business analysis skills within the governance level. SAFe has also recognized the problem in a product-enhancement world, at program and portfolio levels. I believe we also need to consider a separate style of governance to reflect at least two situations: the provision of a service, with a pipeline of enhancements; the set-up of a time-limited project, with the temporary organization of people at all management levels and across disciplines. I am looking forward to your comments on governance and to engaging with the discussion. Thanks