Digital PM Summit 2015 – An Agile Retrospective – Summary Report

Background

Earlier this week in Philadelphia, 300+ Digital Project Managers gathered for the 2015 Digital PM Summit. This is a two-day event focused on PMs who work in Digital Agencies.

I was fortunate enough to be invited to present at the conference and the workshop I led was called, “A Digital PM Agile Retrospective”. My intent with the session was to gather information that would help provide clarity around the following questions:

1. To what extent are Digital PMs using Agile practices? Are these practices being employed in a “pure” form or being modified in order to make them more valuable to the Digital context?
2. Are Agile practices helping Digital PMs (and their organizations) get their work done and deliver value for their clients. If so, which ones and how?
3. Are Agile practices failing to help Digital PMs (and their organizations) get their work done and deliver value for their clients? If so, which ones and how?
4. Are the current forms of Agile suited to working in a Digital context, or is there something missing?

The workshop was attended by 50+ participants. My assumption is that by the fact that they self-selected the session, they had some level of interest or degree of familiarity with Agile. While most responded to most questions, the number of responses was not consistent across all questions.

The time available for the session was limited to 45 minutes. Due to the time constraints, the number of questions I was able to ask was limited. It would have been advantageous to have been able to ask more questions and to have included more participants in the survey, but this was an opportunity to engage with a group of them all at once, even if only for a short time. My hope is that the information shared here will be the beginning of a discussion that will continue to evolve over time.

I should state up-front that in my analysis of the results, while I have tried to keep my commentary focused on the numbers, there are places where my own assumptions and opinions do creep in.

During the first part of the session, attendees were asked to respond to certain questions in a “dot voting” method by placing post-its on flip chart sheets organized around the room indicating their response to the question posed. Each question/topic was presented with an explanation intended to ensure all participants were at a point of shared understanding, For example, when I presented the question about self-organizing teams, I offered a definition of the term and asked the participants to use that definition when responding. Following the session I counted up the totals of all responses.

In the second part of the session, participants were also asked to work in small groups to discuss and provide information which specifically addressed the following questions:

1. What Agile practices work well for your teams?
2. What Agile practices don’t work well for your teams?
3. How does Agile fail to meet the needs of working in a Digital context?

Details on these responses are included in this post. You can also find a slide deck of the actual results on slideshare.

Retrospective Results

 

Approaches to Managing Work

Method	Response	Percentage
Waterfall	10	20%
Agile	5	10%
Conscious Hybrid	16	31%
Unconscious Hybrid	19	37%
Not Sure	1	2%

An interesting thing to note here is that 68% of the participants are using a mix of waterfall and Agile practices. Of those 31% indicated that their mix of waterfall and Agile was a Conscious Hybrid, meaning that it evolved out of first practicing a disciplined form of Agile (like Scrum) and that after introducing Agile, their organization made a conscious decision to move to a blended approach that they determined would be more effective. 37% of the participants indicated that within their organization they practiced an Unconscious Hybrid and that decisions about blending waterfall and Agile were made at the onset of the adoption of Agile for one reason or another, but without having first made an attempt at a disciplined approach to adopting a form of Agile.

In comparing this to the 2015 State of Scrum Report issued by the Scrum Alliance, only 4 % of the respondents indicated that “Scrum or Waterfall Scrum was introduced and integrated into our Waterfall method” but 63% of respondents said they were practicing Scrum and waterfall in the same organization.

Cross-Functional Teams

Participants indicated that the majority of them (70%) were working with teams that could be described as cross-functional. This was defined as teams that are “…comprised of people who collectively have all the skills needed to build the product, and each team member is willing to do more than just their own thing.”

We have those	30	70%
We Don’t Have Those	8	19%
Not Sure	5	11%

Self-Organizing Teams

Teams that are self-organizing were described as teams that are “…empowered to estimate, plan and manage their own work in whatever way they feel makes the most sense for them as a group.” I was pleasantly surprised to see that 48% of the respondents indicated that they had self-organizing teams in place.

We have those	20	48%
We Don’t Have Those	19	45%
Not Sure	3	7%

Daily Standup

A daily standup meeting was defined as an event in which the following are true:

• Teams meet once each business day for not more than 15 minutes.
• The focus of this event is for the team to sync up and plan their work for the day.
• This is NOT a status meeting for a PM/SM/PO/Manager.

We have those	15	37%
We Don’t Have Those	26	63%

What is interesting here is that if only 10% of the respondents are using Agile, that means there are another 27% who are using a hybrid method, but have found that Daily Standups are valuable in that way of working.

One interesting question raised during this portion of the session was (paraphrased) “What if you do a Daily Standup, but only during certain parts of the project?” This highlights something I had concerns about but did not have time to truly test out in the session. If an organization is using an Agile practice like Scrum, in a disciplined way, then they are having Daily Scrums (or Daily Standups) every working day. Not having them everyday would indicate that either they had not been trained properly (or learned on their own) why we do Daily Scrums every day, or that the team did not see the value in maintaining that ceremony. In my experience working with teams, that latter is normally an indicator of a poor understanding of Scrum or poor implementation of the practices.

For example, some teams decide doing them everyday is wasteful because things don’t change that much day to day. This is usually an indicator that the work has not been broken down into small enough pieces for incremental progress to be made each day.

Definition of Done

One of the most important factors in making Agile work is having a clearly defined definition of done. This is something that should be documented and understood across the teams and business. Ideally, the definition of done and definition of potentially shippable are the same, but in some organizations this is not possible. When they are not the same, the clarification of both becomes even more critical.

For example, if a team doing two-week Sprints has a definition of “done” that actually means, ready for the offshore three-week manual integration testing, then done is not equal to potentially shippable. Everyone on the team and the business has to understand this and prepare for the fact that technical debt may come back two Sprints down the road for work we are declaring done when we show it to stakeholders in the Sprint Review. From a strict sense, this is not ideal, but it is a reality in some cases.

The bottom line, and this may be even more relevant for the digital space, is that if you do not have a definition of done that is clearly understood by the organization and the client, you are unprepared for the risks that come with it. This risk may be from technical debt that comes back to haunt you or from clients who are “unhappy” and want free work from you to make things right. Unfortunately, 76% of the respondents indicated they did not have a definition of done.

We have that	10	24%
We Don’t Have That	32	76%

Potentially Shippable Work

Only 29% of the respondents stated that the work their teams do in iterations results in potentially shippable work. This may occur for a variety of reasons including teams comprised of silos of expertise who deliver to other silos, teams which do not understand their capacity and cannot plan effectively, or frequent interruptions from stakeholders trying to push work into a Sprint after planning is complete. 71% of the respondents in the room are working with teams that are not reaching a potentially shippable state at the end of the iteration. Given the importance of this to Agile in general, it is a fairly concerning number, but it does seem aligned with the response to the question about the definition of done.

Yes	12	29%
No	19	45%
We don’t work in iterations	11	26%

Retrospectives

45% of the participants practice retrospectives, 55% do not. While that may simply indicate a preference for a traditional Project Review or Post Mortem at the end of a project, it may also mean that 55% of the teams are not taking the time to meet and reflect on how to work better together. Regardless of whether they are practicing waterfall, some form of Agile, or a hybrid approach, this is a very concerning number.

We do those	19	45%
We don’t do those	23	55%

Results of Group Work

The Good

When asked what Agile practices worked well for their teams, the participants came up with the following list:

• Daily Scum
• Breaking things into Sprints
• Time boxes – predictable work and commitments
• Quick QA
• Clients like the backlog
• Good for team focus, manageable pieces
• Prioritize work daily
• Sprint planning – people actively engage – collaborate within the team
• Regular work schedule, people show up
• Weight off of PM “control”
• Demos to show progress and keep the client engaged
• Shippable work
• Self-organizing has created leaders
• Under promising and over delivering
• More retrospectives, more learning and process iteration and continuous feedback
• Learning from the end of the sprint helps the team improve as they go
• Cross Team Communication
• Scrum improves issue resolution
• More frequent feedback highlights new scope

Some of these responses (like Shippable work and Weight off PM “control”) seem to indicate a sound practice of Agile, or understanding of the value it can provide. Others, like “Under promising and over delivering” point towards a lack of clarity on how Agile practices like Scrum are designed to work.

The Bad

When asked what Agile practices do not work well for their teams, the participants came up with the following:

• Daily Scrums don’t last 15 minutes – too many tangents – more of org. stats and just one more meeting to deal with
• More stress on teams
• Some want traditional direction
• Takes “longer”
• Agree to do more than can be accomplished
• Developers/Clients don’t understand
• Don’t know when you will be done
• Hard to sell clients on the true benefits of Agile
• Learning curve, adaptation
• Difficult to know where we are
• Manage expectations/manage budgets
• UX Research – where does it fit
• Learning curve – not everyone is on the same page
• Keeping scrum short with large teams
• No Product Ownership
• Using a Scrum Board
• Backlog Management
• Estimation
• User Stories

This list is not out of the norm with teams and organizations (Digital or not) that are trying to adopt an Agile process, especially Scrum. But it does indicate that something is amiss with the actual disciplined practice. It is possible that in the organizations where these problems are present, a remedy might be as simple as training or coaching. But it is also possible that the types of relationships Digital Agencies have historically had with their clients makes resolving many of these issues more complicated and requires further adjustment to an out of the box model of something like Scrum.

The Ugly

When asked how Agile failed to meet the needs of working in a digital context, the participants came up with the following list:

• UX-Research doesn’t fit
• Requirements gathering
• Digital teams aren’t fully dedicated to one project
• Need for certitude
• Client Comfort
• Need to educate the client
• Service vs. Product
• How to Sell It
• Time spent in meetings is expensive
• Getting buy-in with a fixed budget
• Where does the bid process/proposal fit in

In many ways, this list echoes what is offered above and indicates that there may be either a poor understanding of how Agile (or Scrum) is designed to work or simply a poor practice of the chosen approach.

Following the session I had a discussion with one participant who initially expressed a concern over how to fit the research UX has to do into the context of Scrum. In walking through how they had resolved it, however, what he explained was a completely reasonable and Agile way of handling it. There was a concern that perhaps they were not doing it “right”. In considering all the information that was provided by the participants of the session, I believe it is important to keep this anecdote in mind. In some cases, yes, Agile practices are being misused and not delivering value. In some cases, there is a perception that Agile is being practiced in a disciplined manner even when it is not. The other side of that coin however, is that in some cases the practices in play are either Agile or complimentary to an Agile approach, but there is a perception that it is not being done “the right way”.

Wrapping Up

While my goal with this session was to gather information that would help provide insight on whether or not current Agile practices were helping with Digital work, the results do not give a clear answer to that question. What does seem to be highly likely though is that within the Digital context, there may be gaps in the understanding and practices of Agile which results in challenges for those trying to implement. Because so many Digital Agencies need to maintain the capabilities necessary for both Agile and Waterfall to meet the needs of their clients, this may be understandable as an accepted risk.

My two biggest questions coming out of this are:

1. How do we get more Digital Agencies (Agile or not) to begin doing  Retrospectives on a regular basis.

2. What can be done to help more Digital Agencies who are using a hybrid to move to a Conscious Hybrid rather than an Unconscious Hybrid.