Estimation Isn’t for Everyone
The Evolution of Agility in Software Development
By John Nevins and Eric Chima
As an engineering manager, I once led a traditional Scrum team, living the mainstream Agile culture. We spent hours defining, refining, and estimating work so that we could feed it into our fine-tuned productivity machine. We were all veterans of Agile Scrum — so why did it seem like this machine wasn’t as fine-tuned as we thought?
“Ro-sham-bo, throw!” our Scrum Master would shout, as fingertip estimates darted into the air. Inevitably, a few of those numbers would match, a few wouldn’t, and then the discussion would really kick off.
“That’ll take a week,” an astute engineer would offer.
“Ah-ah! That’s a measure of time and not complexity,” I would remind the group. We would then take time to recite the ethos of complexity points, before revisiting the all-too-familiar topic of developer capabilities as they related to the estimate. Back and forth we’d go, always talking about how we described the work, rather than the actual work itself.
By the time one such session was complete, we had refined three user stories worth of work, totalling 13 points. We still needed another 10 points of work to make a full sprint at our average velocity. We agreed to meet again to be ready for Sprint Planning.
This whole process seemed very important, because these stories comprised a large body of work. It had elevated in profile enough that I needed to forecast a completion window for our stakeholders. The problem was, without knowing the estimate for each ticket, I had very little way of projecting a date without committing the team to an arbitrary deadline.
I started to look at past performance data. We were on the Fibonacci system, measuring complexity by our guts at 1, 2, 3, 5, and 8 points. 13 was too big, and 21 was unforgivable. I looked at all of the stories that crossed the finish line in the six weeks prior, and saw that the average ticket was 4 points. That’s an abomination in a Fibonacci system, I know, but the numbers don’t lie, so I went with it. I put a 4 on every ticket in the Epic, added 20% for good measure, projected a date, and handed it over.
We delivered within one week of my projection. And it wasn’t a fluke.
We continued to shape our projects that way, and delivered consistently. It got me thinking about where we spend our time in the software development lifecycle.
I really started to question the system.
I came back to the topic several years later, when my new team was struggling with velocity. It was the most senior team I had ever led, brimming with potential, but still we were stalled. We transitioned from Scrum to Kanban, trying to go lean, but the team was still spinning its wheels in meetings, debating the size of tasks we had never tried before. Refined work wasn’t making it into the “To Do” column fast enough. I couldn’t shake the sense that if we just put the time and effort we were spending on estimates into actually completing tasks, we would get a lot more done. When I suggested it to the team, I learned they felt the same way. We stopped estimating and never looked back.
By the time 2022 rolled around, I was confident that estimations were a crutch that strong teams didn’t need anymore. As long as your engineers are skilled at breaking down problems — a key muscle for any Agile team — you can keep the work flowing. If a ticket takes too long, you can learn from it and break it down better next time. And if you need to do some long-term planning, well, all you need is a little math.
The Work and the Metawork
Since the Agile Manifesto was first signed back in the early 2000s, software teams have built upon new norms of working. At that point, they were no longer bound to the waterfall method of …
Requirements — Design — Implementation — Test (and fix!) — Deliver
… but rather could hedge risk by fitting that entire cycle into two weeks. The struggle is real — I once worked on a waterfall-based team that operated on a one-year cycle, and that test phase really gets frantic when six months worth of work is supposed to function without issue.
Now, throughout the industry, software teams plan their calendar around the same familiar ceremonies: refine, estimate, plan, retro, and repeat. There’s a reason these processes are so commonplace: They keep teams organized and they get work done. But every hour spent in a ceremony is 2.5% of a developer’s work week gone. Gathering six people in a meeting works out to be surprisingly expensive, and most engineers would rather be doing the work than sitting on a videoconference talking about it. The cruft of ceremonies, discussions, and meetings has spawned a term — “metawork” — has made its way to Urban Dictionary.
Eliminating estimations, then, isn’t about revolutionizing Agile. It’s about simplifying it. As Agile moves deeper into its 20s, methods that seemed groundbreaking in 2001 are familiar and easy now. Many developers have been breaking down work and iterating for their entire careers. If your team has been working together for a while, they should be used to cracking chunks of their problem space into manageable tickets. And once that problem has been solved, a few key metrics can track and even predict how quickly that work is finished.
A New Velocity
We’ll come right out and say it: Velocity is still the single most important metric to your team. Traditionally, that’s tracked in terms of story points, but it doesn’t have to be. Velocity is simply a measure of throughput, so why not leverage your issue tracking system and track the number of work items you complete each week? If your team finishes six tickets per week, and they’re breaking down tickets well, that’s your velocity right there.
“But wait,” you say. “Breaking down tickets isn’t perfect! Some of our stories are 8s and some are 3s!”
Sure they are. But in the long run, they’ll all average, so why spend the time to debate each individual ticket? What matters most is the predictable output of the team, and that’s going to be the same regardless of the number you assign to the task. If you find your velocity fluctuating too much, that just means you need to get better at breaking down problems.
Let’s make the example more concrete: If 80% of your stories are 3s and the other 20% are 8s (as an example), on a backlog of 10 tickets, your average story size is 4.
If you put a 4 on every story, or ticket, you’ve still reflected the accuracy of the backlog on average. If you complete 20 points every 2 weeks, on an average story size of 4 points, you’re going to complete 10 story points every week, regardless of the relative size of the individual task.
So, if you skip this whole exercise and put a 4 on each ticket, you’ve saved yourself the hassle. The number on each ticket is starting to feel less useful, right? Let’s just go ahead and delete it, because if your team has broken its work down efficiently, you don’t need it. Your velocity is now the number of tickets per week, or throughput.
If you’re worried that your team isn’t reducing its stories well enough, just turn to a concept from the Kanban method of Agile: Cycle Time. Track the time your stories spend between In progress and Done and you can tell how well you’re breaking down tickets. If the average cycle time of a task increases from, say, 36 hours to 7 or 10 days, use that as a catalyst for a conversation about work breakdown in those all important team retrospectives.
It’s important to note, though, that the throughput metric applies to teams. An individual does not complete a certain number of points per week. This is an important distinction, because you don’t want your developers to feel judged by the system, or to try to game it by engineering the tickets they take on. Remember, at the end of the day, it’s not about the number of story points or even the number of tickets, but rather getting real value in the hands of users. Velocity is a tool for planning projects, not a scorecard for evaluating employees.
Planning Projects
Story estimations and velocity measurements all boil down to one thing, right? At some point in a project, someone with a manager title (product, project, engineering or otherwise) is going to ask, what is the level of effort? Or, put more plainly, how long is this going to take?
In that sense, it’s easy to see why those manager types would lean on estimations. Precision feels better when you know you’re going to be judged on it, even if it’s the faux precision of story points. But you can plan a project just as easily using the hard data of your team’s past performance.
To start, calculate your team’s velocity. Say, for example, that your team has completed six tickets per week over the last six months. If you break down your new project, and see that it has 35 tickets, you can reasonably say that it will take about six weeks to complete. Then, you’ll want to add in a margin of error for phenomena like scope growth and debugging. You can base that buffer on your team and your experience with past projects. It might feel like a bit of a guess at first, but after you complete a few projects, you’ll learn just how much inflation tends to happen. It’s data, just like anything else.
Above all, this is an iterative process, just like everything in Agile. The more you do it, the better your data will be, and the more accurate your projections. As people join or leave the team, it might take a few cycles to refine your velocity again — but that happened with estimations, too. The important thing is that you can get better at it without any extra effort from your developers — they get to focus on the work.
The Final Word
Meta has dubbed 2023 the Year of Efficiency, though for many of us it feels like we’ve been in the Age of Efficiency for a while now. As leaders of Agile teams, our role is to point engineers toward more of the work that matters and less of the work that doesn’t. Here at The Times, as a subscriber-based business, we have an obligation to our readers to ship meaningful value early and often. That extends from our newsrooms to our engineering teams, and we’ll embrace any approach that aids in that agenda. For some of our teams, eliminating estimations is a strategy that works.
With that in mind, we’ll admit that this method isn’t for everyone. If your team is still building the muscle for work breakdown, struggling with process discipline, or is new to Agile altogether (where have you been?), the traditional approach might still be best for you. And teams that work on very tight, week-to-week deadlines might find that they need a bit more precision than we’re suggesting.
But for most teams, we think, eliminating estimations will be an easy win. And even if your team isn’t ready quite yet, that doesn’t mean they can’t be soon. Agile is designed for iteration and experimentation. Give it a try and see how it goes, then make changes that work best for your team. We’re guessing you won’t have many engineers clamoring to get that meeting back on the calendar.
We wish you all the best in your Agile adventures. And with that, we’re going to slide this post into the Done column.
Eric Chima is a senior technical product manager for the Web Platforms team at The New YorkTimes. He has worked with Agile teams for thirteen years in web, IP telephony and virtual collaboration. His team stopped using estimations this year.
John Nevins is a Senior Engineering Manager who leads the GraphQL Platform team for The New York Times. Throughout his career, he has led software teams in a variety of industries and methods and considers himself a student of the evolving ways of working. He stopped using estimations with his teams years ago.
