Why Tech Debt Estimates Are Always Wrong

Debt Work Is Harder to Estimate Than Feature Work

A feature has a specification. You may not know exactly how long it takes, but you know what "done" looks like before you start, and the surface area is bounded by the thing you agreed to build.

Debt work inverts that. The job is to find out what is actually in there. You are estimating the cost of removing something whose shape you cannot see until you start removing it, and the discoveries are not edge cases around the work -- the discoveries are the work. The undocumented consumer of the API you were going to delete. The batch job that depends on the column ordering. The behavior that looks like a bug and turns out to be a customer contract.

Feature estimation asks "how long will it take to build what I described?" Debt estimation asks "how long will it take to find out what I do not know?" The second question has no answer of the same kind, and pretending otherwise is where the error starts.

The Planning Fallacy and the Inside View

The underlying mechanism was described by Daniel Kahneman and Dan Lovallo in Management Science. Their abstract states it directly: "Decision makers have a strong tendency to consider problems as unique. They isolate the current choice from future opportunities and neglect the statistics of the past in evaluating current plans." The consequence follows in the next line: "Overly optimistic forecasts result from the adoption of an inside view of the problem, which anchors predictions on plans and scenarios."

That is exactly what a team does at a whiteboard. It builds a scenario -- extract this module, swap that adapter, migrate the callers, delete the old path -- and adds up the steps. The scenario is coherent, the arithmetic is correct, and the estimate is still far too low, because the scenario contains only the steps you can currently imagine. Every unimagined step is missing from the total, and unimagined steps are precisely what debt work consists of.

Reference Class Forecasting

The outside view has an operational form. Bent Flyvbjerg's paper in the Project Management Journal presents it as "a promising new method called 'reference class forecasting'" built on the decision theory that won the 2002 Nobel prize in economics, and describes it as achieving "accuracy by basing forecasts on actual performance in a reference class of comparable projects." It was developed for infrastructure megaprojects, where optimistic forecasting is measured in billions rather than sprints, and it transfers to engineering work without modification.

The method is deliberately unglamorous. Identify a class of past work comparable to what you are about to do. Look at what that class actually cost, not what it was estimated to cost. Adjust your forecast toward that distribution. The hard part is not the arithmetic; it is admitting that your specific, carefully reasoned plan is weaker evidence than a boring list of what happened before.

"It Is Just a Small Refactor" Is the Most Expensive Sentence in Engineering

The word doing the damage is "just." It signals that no investigation is required, and it therefore prevents the investigation that would have revealed the actual size. The sentence is usually spoken honestly, by someone who has read the code and not the callers.

It is expensive for a second reason. A small estimate gets approved casually, without a checkpoint, a stakeholder, or a scope agreement. So when the work turns out to be four times larger, there is no mechanism to catch it -- the team is three weeks in, the sunk cost argument is loaded and ready, and nobody has the standing to stop it. Large estimates get scrutiny, which is a form of protection. Small estimates get waved through, which is not.

Discovery Is a Line Item, Not an Overhead

Most debt estimates are estimates of the change. Nobody prices the finding out, even though on legacy work the finding out is routinely the larger half: reading code nobody has touched in years, tracing callers, reproducing behavior, writing characterization tests to pin down what the thing currently does before you dare alter it.

Give discovery its own line. It makes the estimate legible to a stakeholder, because "two weeks to understand the blast radius, then an estimate for the change" is a sentence a non-engineer can evaluate. And it makes the eventual overrun attributable: you can see whether the change was harder than expected or whether the system was simply more tangled than anyone knew, and those two failures call for completely different corrections next time.

Timeboxes and Spikes: The Honest Answer to Irreducible Uncertainty

Some uncertainty cannot be reasoned away before the work starts. When that is the case, an estimate is a guess wearing a number, and the honest response is to stop estimating and start buying information.

A spike is a fixed purchase of learning: three days to establish whether the extraction is feasible and what it would touch, with a written finding at the end. A timebox is a fixed purchase of attempt: two weeks on this, then we stop and decide regardless of state. Both convert an open-ended risk into a known, capped cost, which is the thing a stakeholder actually wanted from the estimate in the first place.

Estimate in Ranges and Confidence, Not Points

A single number is a claim of precision you do not possess, and the moment it leaves your mouth it becomes a commitment. A range carries the uncertainty with it, and uncertainty is genuine information that a planner can use.

Attach a confidence level, because a range without one is still ambiguous. "Three to eight weeks, and I am about eighty percent confident it lands inside that" tells a stakeholder two things: the likely cost, and that there is a real chance of the upper end. It also converts the later conversation from "you were wrong" into "we landed in the tail we discussed", which is the difference between a blameless outcome and a broken relationship.

Communicating Uncertainty Without Sounding Evasive

Engineers avoid ranges because ranges sound like hedging, and hedging sounds like incompetence. The fix is not more confidence. It is giving the stakeholder something concrete to hold on to in place of the false precision you took away.

Three habits carry most of the weight: name the reference class out loud so the number has visible provenance; commit to a date when the range narrows rather than to the range itself; and report movement early, because the cost of a bad estimate is almost entirely in how late it is discovered. For framing this in terms a budget owner responds to, see selling to management.

Track Your Own Error: The Only Calibration That Works

Every technique above depends on one input: knowing what your past estimates were and what the work actually took. Most teams do not have this, because the estimate is overwritten or forgotten the moment the ticket moves, and so the same error is repeated indefinitely with no feedback signal at all.

The instrument is embarrassingly simple. Record the original estimate, keep it immutable, record the actual, and look at the ratio in aggregate every quarter. Within a couple of quarters you will know your own multiplier, and it will be specific to your team, your codebase, and your category of work -- which is worth far more than any industry rule of thumb.

Punish Missed Estimates and You Buy Silence

Estimates are forecasts. A forecast that is treated as a promise stops being a forecast, because the person producing it now has an incentive that has nothing to do with accuracy.

What follows is predictable. Padding goes underground, so numbers inflate but nobody can say by how much, and the organization loses the ability to reason about capacity at all. Risk stops being reported, because raising a concern early looks like pre-excusing a failure -- so problems surface at the deadline instead of the moment they were known. And scope quietly shrinks to protect the date: tests get skipped, the migration ships half-finished, and the debt you were paying down comes back with interest.