Nobody, anywhere, has ever produced good and reliable estimates.

Every estimate is wrong, but every estimate also inevitably turns into pressure to deliver "on time", and then disappointment and blame lands on the people who “fail” to meet an arbitrary deadline.

I do not want implicit blame and unmet expectations to become business-as-usual at Griffin. I want us to do better than that.

The truth is, if Griffin could reliably produce accurate estimates for software development, we wouldn’t need to be in banking: we’d already have a multi-billion dollar company. But we can’t, so I’m writing this post instead. I’m going to talk about language, constraints, and things that are in theory knowable, and then offer some practical advice for giving better answers to the perennial question: “How long will it take?”

Terminology

• Estimate: A prediction of how long a project will take. Estimates are frequently expressed as a single date or fixed period of time, e.g. “this project will be done in three weeks.”
• Target: A business objective with a target date. “We need to deliver Confirmation of Payee by July 1st, so that we're compliant with regulations.” A target may be desirable or even mandatory, but that does not mean it’s achievable.
• Commitment: A promise to deliver defined functionality at a specific level of quality by a certain date. “We promise to deliver an MVP of Confirmation of Payee by the regulatory deadline of July 1st”. Commitments don't have to correspond to estimates.
• Forecast: The same as an estimate, but with fewer connotations around commitments and targets. If you fail to accurately predict the weather, nobody thinks that means you should have controlled the weather better. (This dilutes the blame problem, but doesn’t really address the underlying issue, which is that all estimates/forecasts are wrong.)

Being precise about language is important. When we collapse these concepts into each other, we end up with conflicting expectations and communication breakdowns. It happens all the time. We present our estimates based on a target set by our Board and not on our actual knowledge of how long things take. Or conversely, we hear an estimate and implicitly convert it into a commitment in our head. “I think the project will take four weeks” does not mean “the project will be done in four weeks”, and it certainly doesn’t mean “you can tell customers that we will deliver it in four weeks”.

All software projects are complex

We’ve all heard about (or experienced first-hand) kitchen remodels that took twice as long as expected and came in massively over budget. But then we turn around and act like estimates for shipping major software features can be made both accurate and narrow, if only we tried a little harder.

This becomes more absurd when you consider the fact that every kitchen remodel is, in theory, pretty similar. The same people building to roughly the same specs, using the same tools and materials and techniques they use on every project.

By contrast, building software is always novel. If there was already code that solved the problem, we could just copy and paste the files. If the problem were easy to understand and fix, it would already be done, or wouldn't be a problem at all.

Software estimates almost always grow over time because complex projects are about wading into the unknown. All complex projects require learning new information and skills. The more we learn, the more variables we have to account for. Every variable pushes the upper bounds of how long a project could take.

Upper bounds are not knowable

The main reason that estimates are useless is that lower bounds are fairly easy and knowable, whereas upper bounds are not knowable. This is a strong claim, so let’s break it down.

Humans are pretty good at lower bounds. If you ask someone with no cynicism how long a project will take, they will usually answer with a decent estimate assuming nothing goes wrong.

Of course, something always goes wrong. So, why do we always default to the lower bound?

There are lots of reasons: we don’t like grappling with uncertainty, we like to tell people what they want to hear, we’re optimistic and have faith that our team will work hard and deliver on time!

But the main reason we don’t is because we can’t: when we get into “what could go wrong”, there are just too many variables, including things like “a senior engineer got hit by a bus” and “war”. And even if we could account for all the variables, the resulting upper bound would be too large to be useful for planning.

At Griffin, it took us five years to secure our full banking license. In the 2010s, we were watching challenger banks secure theirs in as little as two. So, why did it take us so long? Is it because we’re dumber or lazier than other banks? No, it’s because in 2017 the possible impact of the Brexit referendum was still a looming unknown and no one had ever heard of “SARS-Covid-19”. By 2019, when we submitted the first draft of our application to the regulators, their perspective on challenger banks had shifted significantly, and a year later we were adapting to living with a global pandemic.

If David and I (and our investors) knew in 2018 that we wouldn’t be a fully authorised bank until 2024, I’m not sure Griffin would exist. To be honest, I’m not sure whether that’s an argument for or against estimating projects.

What does a useful estimate look like?

Useful estimates must be unbiased and analytical. They should be based on what we know to be true, not what we want to be true. They should not be tied to targets and they should not be conflated with or substituted for commitments. "But our customers/Board/investors want it" does not make it possible to produce a good estimate, and also doesn't imply that we should give a bad one.

As mentioned above, estimates are usually expressed as a single date: “This feature will be done by the end of the month.” Something this narrow will never be accurate, even for relatively straightforward projects (see: the kitchen remodel).

If we’re trying to be more realistic and manage expectations, we might express our estimate as a range: "That feature will take between three and five weeks". Most project managers will hear this and think “great, so four weeks!” and also treat five weeks as the absolute upper bound.

But, since upper bounds are unknowable, good estimates should be communicated as probability distributions (i.e. a date range with a confidence percentage). For example, “we predict this feature will take 4-8 weeks to complete with 50% confidence.”

Note that this says nothing at all about the longest possible amount of time it might take. Eight weeks is only the upper bound 50 times out of 100 (i.e. P50).

Failure is always an option, and “when will it be done?” always contains a small chance of “sometime between a year from now, and never”. The bigger the project gets, the more probable “never” becomes. So, most of the time we talk about P50 estimates, because under P99 estimates‍—‌which necessarily include things like “Covid” and “business failure” - nobody would ever commit to starting a project. If we want to ship six major projects in 2025, business failure is a remote possibility for each one of those, but that level of paranoia is not helpful for figuring out what we can accomplish.

Epistemology

For estimates to be unbiased, they have to be based on things we can know, i.e. the current state of things. We can’t predict the future, we can only react to it as it happens: at which point, it becomes the new current state of things and we can use it to adjust our estimates. In other words, we can ignore the things we don’t know until we know them.

Note that even things we can know and control to a certain extent‍—‌our priorities and resourcing‍—‌can change quickly and at will. It is entirely possible for a feature to be on the roadmap for "six months from now" forever. I have personally seen features stay on the backlog for five years.

Things you can't know

• People. When they will go on holiday, get sick, get promoted, move teams, go on parental leave, quit, etc.
• Regulation. How much it will change next year and how that will impact us.
• Production incidents. How often they will occur, how disruptive they will be, and how long they will take to fix.
• Investor sentiment. When it will shift, making it easier or harder to raise, and possibly requiring us to change our business strategy.
• Customer feedback. Whether it will be positive or negative, what we will need to do in response to it, and how high it will be on the priority list.
• Audit findings. What we will need to do about them, and how urgently.
• Security exploits. When they will be released in the wild or how disruptive they will be.

Things that are knowable

• Priority. We can usually know the relative priority between two features, and we can see what number a project is on the priority list right now. (Though of course, this can change quickly: “We need to ship X by April to stay compliant, so Y is being bumped down.”)
• People. How many people are assigned to a task right now.
• The current state of a project. “We are shaping a solution.” “We have a solution that we’re happy with, but haven’t started building.” “We are three weeks into building.” “We are in UAT.”
• The size of the project. Bigger projects have higher tail risk than smaller projects.
• Overall current velocity. How many PRs shipped last week
• Lower bounds on start date. "We will not start working on this feature until at least Q2."
• Lower bounds on work duration. "This feature will take at least a month."

Defence against the dark arts

While all of the above may seem sensible, most of us still face constant pressure to offer up unrealistic commitments based on bad estimates. I’ve set out a few practical steps for breaking the cycle below.

Make fewer commitments about the future

• Commitments always incur a cost, so don't hand them out for free. Only make commitments when there is a corresponding large benefit, e.g. closing a deal with a big customer, or a potentially huge cost for not committing, e.g. regulatory fines.
• Don't stack commitments. Each team gets at most one commitment, and it must be the thing they're currently working on. Zero commitments is ideal.
• A team should only ever be working on the Most Important Thing. The next thing isn't the Most Important Thing until they're done with the current thing. By definition, there cannot be two Most Important Things.
• Don't make commitments until you've actually started working on the thing. Completion still isn't guaranteed, but it much more likely that we will change our minds about work that hasn't started, rather than down tools on a project that’s already underway. If we’ve started building, it also means we probably have a good handle on complexity, which means our estimates are a bit more accurate.

Throw out your Gantt chart

Gantt charts are ubiquitous and almost completely useless for software projects. A useful estimate is always a probability distribution, but those aren't representable on the chart. Similarly, Gantt charts can’t accommodate an honest representation of lower and upper bounds. They are also the ultimate visual conflation of estimate-target-commitment, and they create bad estimates within bad estimates (”This task is 70% done”, “Task A will be finished on Friday, which means Task B will start on Monday”).

Trying to forecast a project that has not started yet is folly. Neatly stacking up estimates for each stage of an upcoming project in a Gantt chart is the business equivalent of astrology: it might give us a sense of purpose and control, but it has no basis in reality.

As a general heuristic, the farther we are into a project, the more likely our forecasts are to be accurate. So, your best chance of accurately forecasting how long a project will take is to diligently track the speed at which the team is working, and adjust your estimates regularly based on this.

The process might look something like this:

• Break down the entire projects down into tickets. Use the smallest sensible ticket size. Count the number of tickets, e.g. “This project is 20 tickets."
• The total number of tickets will almost certainly grow as the team works. "The project is now 25 tickets, and we have completed ten of them.”
• Track progress on things that are done. A task is either done or not done. If you can meaningfully say a ticket is “50% done”, that ticket is actually two tickets.
• Estimate your lower bound based on the current completion rate on tickets: "The team shipped ten tickets in five working days, therefore the project will take at least a week and a half." Adjust accordingly as ticket completion rates either slow or speed up.
• If ticket opening rates are slowing, and ticket completion rates are speeding up, you are likely close to completing the project.

Don't do waterfall. Iteration on the requirements, scope, etc. are all fine, but the state you're in today is knowable, and the decision to make another pass can be tracked.

Note that completion rates are unique to both teams and projects. You cannot use completion rates from one to estimate another, because some types of work are harder. And you can't compare team speeds because their ticket sizes, skills, experience, domain knowledge and interruption load are all not comparable.

“What decision would you make with that information?”

When people ask you for an estimate, they are almost always asking the wrong question. So when someone asks “when?” or “how long?”, answer with another question: "what decision would you make with that information?" This question should be easy to answer, and it means you can give the person the information they need without creating commitments or trying to predict the future.

For example:

Sales: When will X be done?

Product: What decision would you make with that information?

Sales: A prospect wants to sign, but they need X within the next quarter.

Product: X will take at least six months, and we won’t be starting until at least Q3.

Sales: Why can’t you start sooner?

Product: Because we are building Y for Current Big Customer, and we need Z before end of summer due to a new regulation that’s coming in.

Sales: Okay thanks, that will help me set expectations.

OR

Product: How long would it take to build Y?

Engineering: Why do you want to know that?

Product: The executive team wants to see the business case.

Engineering: What does the revenue forecast look like?

Product: £50k in year 1.

Engineering: It will definitely cost more than that to build that feature.

Delivering without commitment

“But Allen, without commitments, how can you ever deliver anything? What’s stopping you from endlessly navel-gazing and never shipping?”

We’ve all heard of “good, cheap, fast, pick two”. We usually talk about it as though it’s a difficult trade-off, but it is in fact a best case scenario. (Less talked about is the very real possibility of “bad, expensive, and slow”.)

Griffin is a bank, which means everything we do must meet a minimum quality bar. Like all tech companies, we want to ship features as quickly and cheaply as possible above the quality bar we set for ourselves. When we make a commitment to ship a feature by a specific date, that forces us to choose: “If the feature is not ready on Date, what happens?” It pressures us towards shipping low-quality features “on time”, over shipping high-quality features later.

The prevalent myth is that pressure to ship will somehow reduce the time it takes to develop a high-quality feature. That simply by virtue of existing, a commitment will make a sticky problem easier to solve, or stop a team member getting sick at the worst possible moment. In all my years leading teams of hard-working, talented developers, I have never once seen this bear out in practice.

Shipping high-quality features at a sustainable velocity has nothing to do with deadlines, arbitrary or otherwise. In my experience, getting to “good and fast” is about hiring highly skilled people who care about the work, prioritising ruthlessly and iterating quickly, and thoughtfully cutting scope to get the minimum lovable product into the customer’s hands sooner rather than later.