The Product Backlog Is Evil and Must Be Stopped

Kill the Product Backlog – An Essay

(guest blogger Dan Greenberg)

 

Agile (a set of values and principles) and Scrum (a tangible set of roles, artifacts, and events) each aim to eliminate wasted time and make work simpler, more enjoyable, and more effective. Yet, in many cases, when I walk into environments that have undergone Agile transformations or Scrum adoptions, I get a vibe that "this Agile thing" or "this Scrum thing" has added a whole bunch of work and overhead that didn't use to have to be done. There are off-the-record complaints that people used to be able to just come in and do work and now they have to update Jira, sit through brutal release planning sessions, report velocity, attend ten million status meetings, and prepare for terrifying demos. Upon reflection, most of that has to do with someone's obsession with this thing called the Product Backlog, a term that I'm starting to wish the Scrum Guide had left out. Here is an example-based pitch for why I think we'd all be better off Product-Backlog-free.

 

Recognizing that We’re Not Special Will Make Us Special

 

In the Scrum Guide, Sutherland and Schwaber call out explicitly that Scrum is lightweight, easy to learn yet difficult to master. We gloss over the harmless commentary of "Scrum being difficult to master" because after all, my organization is not one of the ones that will have difficulty mastering it. Kind of like how 90% of automobile operators rate themselves as better than average drivers. Math makes that impossible. Forty percentage points of those drivers are incorrect by math alone. In my limited experience, most organizations are not mature enough to master Scrum and I think removing the Product Backlog from their lexicon would actually help. Let’s talk about this Product Backlog thing and then let’s do away with it altogether.

 

Designed to Make Change Easy

 

From JJ Sutherland’s The SCRUM Fieldbook:

 

“How Scrum Works.

 

First you need to understand that there are three, and only three, roles in Scrum: the Product Owner, the Scrum Master, and the Team Member. There’s no business analyst, there’s no tech lead, and there’s no senior Scrum Master – only those three roles. They make up a Scrum team that is able to independently deliver value. The team is the smallest organizational unit in Scrum. These Teams rapidly deliver value to customers in short cycles called Sprints.

 

The Product Owner, or PO, owns the “what.” What the Team is going to build or create or service to deliver or process to write or put out. The PO takes input from customers, stakeholders, the Team itself, and whoever is going to get value from whatever the Team is doing. It could be rural farmers in Uganda struggling with crop disease, or engineers building an autonomous car, or filmgoers going to see a newly released movie. The PO has to take all of that input, some of which may be contradictory, and create a vision of what the Team will do. Then – and this is often the hard part – after getting all those ideas, the Product Owner has to rank them in order from the most valuable to the least. There are no top priorities in Scrum – there is only one priority at a time. This is often hard to settle on. But that’s the way Scrum works.

 

So the PO prioritizes all the stuff that has to be done, from the most valuable to the least valuable, creating what is called the Product Backlog. This Product Backlog is a potentially infinite list of all the possible things that could be tackled by this Team. It is also a living document, constantly changing, based on feedback from customers, changing market conditions, insights, management, whatever. It is designed to make change easy.”

 

Example: Gentlemen, Start your Planning

 

Here’s the problem in my estimation. I’ll illustrate with a hypothetical example.

 

A team forms and chooses its Sprint length, say 2 weeks. The team spends four days (that’s 4 times 8 times the number of people in the room – say 12 – times the average hourly rate for each of those folks in dollars or pounds or euros) building and sizing a massive Product Backlog just so it can go to management and say “here’s everything we’re going to do over the next six months, with story points and estimates on all of it, so now you know it will take us exactly six months and we are going to deliver exactly X at the end of six months”. First of all, let me stop there – doesn’t that sound like big up-front planning, fixed scope, fixed, cost, fixed timeline, Gantt charts, project management? In other words, aren’t we already doing what Scrum was designed to eliminate?

 

Now Sprint!

 

Once all of that planning is done, the team can finally start Sprint 1. They pull in a few items from the backlog that they think they can accomplish in two weeks and begin working. After two weeks, the team regroups with a Sprint Review and a Retrospective and then moves into deciding what to work on for the next Sprint. Now, remember, we’ve learned something from Sprint 1. Maybe we realized that there was quite a bit missing from our big up front plan. Maybe we realized that a lot of our plan actually can’t be done at all. Maybe we even realized that we’re building the wrong product or going about it completely the wrong way. Whatever it was that we learned, we apply that newfound knowledge to plan Sprint 2. That’s Scrum working as designed.

 

Scrum Working as Designed

 

What almost universally happens at this point is that 90% of the work we decide is most valuable for Sprint 2 is work that either was not on the Product Backlog at all or may somewhat resemble an existing Backlog Item but needs to be significantly rewritten. Now, we either spend a day and a half as a full team (12 hours times 12 people times average hourly rate) searching through this insanely large Backlog to find exactly which items we need to edit and which we need to add OR we are short on time so we just add all of the work we now know we need for Sprint 2 and then 2-3 people (the Scrum Master, the Product Owner and maybe a manager or two) spend the next few days doing this excruciating task of updating “tickets” in the Backlog. Then management wants to know whether we’re still on track (hint: we’re not) so we have to spend more time crafting a narrative to explain what happened.

 

Imagine if every time you wanted to make a u-turn in your car, you had to call your boss and explain why you were going the wrong way. You'd probably be so exhausted eventually it'd be easier to just keep driving in the wrong direction. Or you'd decide it was just easier never to leave your house.

 

I Thought Change Was Supposed to Be Easy

 

All of that time spent trying to force-fit an obsolete Product Backlog into a changing set of circumstances actually makes change extremely difficult, while JJ Sutherland said he wants change to be easy. If we didn’t have a Product Backlog, we would be free to just write down what we wanted to do for Sprint 2 and change would be easy.

 

The missing piece is that without a Product Backlog, management doesn’t have their time estimates and their cozy Release Plan to tell them whether or not things are on track. However, I think we can decouple management’s plan from the work the team is actually doing because they aren’t really related. Management’s plan is a vestige of an antiquated way of thinking about and doing things, while the team working in a Scrum framework is more reflective of reality. So I think we can create a separate team of two or three people to create a realistic-looking plan or “Product Backlog” with point estimates and timelines that will satisfy the corporate executives. It won’t be completely accurate but the plan that the team would have put together won’t be accurate anyway because it will drastically change every Sprint. The one this separate group designs will be at least as accurate as whatever it was the team would have produced.

 

Fingers Need to Be Pointed, Heads Need to Roll

 

After six months of work, a group of people will take a look at the original Product Backlog, decide that they don’t have what was originally asked for or promised, deem the project a failure, and demand to know who is to blame. But it is not that the project is a failure – quite the contrary, in fact. It is our way of looking at it that is flawed. As we progressed through Sprint after Sprint, the team learned valuable information and adapted their plan along the way. The fact that the current state does not match the initial plan is Scrum working as designed. All that information that was learned and the pivots the team took along the journey actually indicate that the project is hugely successful. If we were contractually obligated to stick to the original plan, we would perhaps be contractually obligated to go down a path that was not wise to go down.

 

If we are more interested in following a plan than in responding to change (and welcoming changing requirements), we should not be using Scrum and should not call ourselves Agile.

 

SCRUM (NOT reinventing the wheel)

 

Want to practice actual Scrum? Try these simple steps and let me know how it works. I recommend you not put this process on top of all of your current meetings, but rather replace everything you are doing currently with only the below. Spoiler alert: this is not my version of Scrum, this is actual Scrum. I'm not reinventing the wheel.

 

 

1.     Form a Team

Sutherland and Schwaber recommend a team size of 3-9, with a preference toward smaller size. If you can feed a team on two large pizzas, you’re probably okay. The team should include a Scrum Master and a Product Owner. The Product Owner owns the product vision, coordinates with stakeholders and customers to learn what’s important and with the team to learn what’s possible. The Scrum Master coaches the team and the organization on Scrum and helps remove any impediments that are preventing the team from delivering working product. Here, the team is welcome to employ any of a number of team-building practices if they are interested: Choosing a Team Name, Planning a Social Event, Writing a Social Contract, Choosing a Definition of Done…

 

2.     Spend About a Half Hour Deciding What to Build

Don’t overthink this part. Just put down a few bullet points and maybe a drawing of roughly what the product might look like when it’s done. There’s no need to over-engineer because this vision will change every Sprint and since change is inevitable, the Product Owner will be in charge of making sure that change is easy.

 

3.     Spend About Three Hours Deciding What to Do Over the Next 2 Weeks (or 1, 3, or 4 Weeks)

A Sprint can be anywhere from 1-4 weeks long, but let’s use 2 as an example. As a team, informally write down on index cards everything you plan on doing over the next two weeks. Break work down into small chunks and write one chunk on each index card. Be brief and use easy-to-understand language that the team can easily grasp. As the team works these items, they should be more concerned with collaborating with the customer than sticking to a negotiated contract so it’s really not too important what is written on the cards. This should be an informal and quick process. You may not even need 3 hours.

 

4.     Spend Two Weeks Sprinting (or 1, 3, or 4 Weeks)

For two weeks, do what you committed to as a team. Meet once every morning briefly for 15 minutes to check in with one another and raise any questions or confusions that came up during the prior day. As you are working the Sprint, bring any questions about the index cards to the Product Owner, who if necessary will check in with the stakeholders and customers.

 

5.     Inspect the Prior Sprint and Adapt

Spend a morning hanging out as a full team, including the stakeholders, and customers. Go over the prior Sprint’s work, play around with the product as it currently stands, and have an informal chat about where the team should head next. Maybe adjust that product vision. Keep it informal. If anyone’s nervous, something’s wrong.

 

Go grab lunch as a team or just take an hour or so to relax. Then reconvene and take the afternoon to have an informal chat about how the team worked together over the prior Sprint. Reflect on ways to become more effective and healthy as a team and think about how to adapt to grow toward that ideal in the next Sprint.

 

6.     Repeat Steps 3 Through 5 As Needed