If It Hurts to Delete, I Waited Too Long
Why clever scaffolding becomes tomorrow's technical debt, and how new model releases force us to confront our attachment to obsolete code.
When a better model ships, the clever scaffolding you built to work around the old one turns into dead weight. The lesson from our own builds is simple: delete it early. If pulling a piece of tooling out feels like a loss instead of a relief, you waited too long, and you are paying for it in latency, token spend, and attention.
If it hurts to delete, I waited too long. Every time I've hesitated, it was because I was proud of some clever scaffolding, a retrieval shim I tuned for days, a critic agent that "felt" essential, a prompt chain I could recite from memory. Then a new model landed and made the whole thing unnecessary. When I kept it anyway, we paid for it in latency, token spend, and headspace.
When should you delete the scaffolding?
- Deleting it feels like a loss, not a relief
- A newer model does the same job in a single pass
- You are keeping it out of pride, not need
- It is costing you latency, tokens, or debugging time
Concrete example: our LLC Agreement Scorer started with hand-rolled clause extractors, a re-ranker, and a style linter to paper over model gaps. When the newer model got good at long-context reasoning, a single pass with lighter tooling hit the same quality on our golden tasks. We ran it behind a flag, watched the metrics hold, and ripped the stack out. Instantly fewer moving parts, simpler debugging, and we shipped a "what-if clause" view that same week because we weren't babysitting brittle glue.
Same story on a fire-safety packet demo: I built a drafter, a checker, and a formatter agent. The checker felt "responsible," but the new model plus tighter instructions made it a no-op. Keeping it looked like diligence; in reality, it was nostalgia. Delete, breathe, move on.