Optimism bias and the planning fallacy: budgeting by best case

The Sydney Opera House was budgeted at $7 million for a 1963 completion. It opened in 1973 at $102 million. Your kitchen renovation, your 'three months of expenses' emergency fund math, and your plan to pay off the card by June are all running the same software. Daniel Kahneman and Amos Tversky named it the planning fallacy: humans systematically underestimate the time, cost, and risk of their own plans — even when they know that similar plans, including their own past ones, virtually always ran over.

Two biases in a trench coat

The planning fallacy rides on optimism bias — the robust finding that roughly 80% of people rate themselves less likely than average to experience job loss, illness, divorce, or financial shocks, which is statistically impossible. Layered on top is how we plan: from the inside. You imagine the steps of your project going right, one after another, and the imagined movie becomes the estimate. What the inside view can't see is everything that isn't a step — the contractor who disappears, the transmission that dies in month three, the sick week, the permit delay. No single surprise is likely. Some surprise is nearly certain. Best-case plans price in zero surprises, which is why they aren't estimates at all — they're fiction with a spreadsheet.

Worse, the fallacy survives experience. Kahneman's most quoted admission is that knowing about the planning fallacy doesn't stop him from committing it. Your last three projects ran late and over budget, and your next estimate will still be rosy — because memory files each overrun as a one-off caused by bad luck, not as data about how your estimates perform.

The fix: reference-class forecasting

The correction, developed from Kahneman's work and used for major infrastructure projects in several countries, is called reference-class forecasting. The move: stop asking 'how will my project go?' (inside view) and ask 'how did projects like mine actually go?' (outside view). Your project is not special. That sentence is the entire method, and resisting it is the entire disease.

1. Define the reference class: kitchen renovations, first-time car ownership, households like yours running a monthly budget.
2. Get base rates: real renovations commonly run 30–50% over quoted price; cars cost roughly 50% above the payment once insurance, fuel, and repairs land; your own last six months of actual spending is the base rate for your budget.
3. Anchor on the base rate, then adjust only for hard evidence your case differs — a fixed-price contract counts; feeling organized doesn't.
4. Check your calibration: compare your last three estimates to what actually happened, and apply your personal overrun percentage to the next one.

Budget-by-best-case: the household version

The planning fallacy's most expensive form isn't a renovation — it's the ordinary monthly budget built for a month that never happens: no car repair, no wedding gift, no copay, no annual subscription renewal you forgot. Then a normal month arrives, the budget 'fails,' and the whole system gets abandoned by March. The budget didn't fail. The forecast did.

• Budget from your actual last-six-months average, not from an aspirational month. Cut from reality, not from fiction.
• Add a 10–15% 'life happens' line — not padding, but the base rate of surprise, pre-funded.
• Convert 'irregular' into 'scheduled': list every annual and semi-annual cost (insurance, registrations, holidays, subscriptions), divide by twelve, and fund it monthly. Most budget 'emergencies' are just annual bills with good camouflage.
• Apply the outside view to timelines too: whatever date you project for the debt payoff or the down-payment goal, present the 25%-longer version to yourself as the real plan. Hitting a realistic target sustains motivation; missing a fantasy one ends it.
• For any big decision, run a premortem: 'It's a year from now and this plan failed — what happened?' The question unlocks the risk list optimism was hiding.

Base rates worth taping to your monitor

Reference-class forecasting only works if you have reference numbers. These are the overrun rates that show up again and again in project research and consumer data — treat them as starting anchors to adjust from, not gospel.

The bottom line

You will keep writing best-case scripts — the bias survives awareness, expertise, and your last three overruns. So stop trusting the script and start trusting the base rate: budget from what things like this actually cost, pad by the overrun rate history suggests, and pre-fund the surprises that arrive on schedule every year. The plan that assumes some things go wrong is the only plan that survives things going wrong — and things going wrong is the reference class for being alive.