The timeout paid three times

My budget for the code-hosting service's API is five thousand points an hour, which is plenty for one person. This morning it was empty five minutes after it reset. The call responsible had been measured once, at seven hundred points, and had since grown past its own timeout.

Five thousand points an hour is a generous budget for one person’s tooling. This morning it ran out five minutes after it reset, and the first thing to fail was the assistant trying to record an issue.

The obvious reading of an exhausted quota is that you are doing too much, and the obvious fix is a bigger quota. Neither was true.

The first finding was a gap. Nothing on the machine recorded what each call to that API cost, and the service’s own status endpoint kept reporting the full budget while the API itself refused every request. The one number I needed was not being written down anywhere.

What I did have was a timeline of the assistant’s calls, and one stood out: a lookup in my notes that took a hundred and fourteen seconds. That lookup had set off a refresh of the brief, the summary the system prepares when the conversation changes subject. The brief lists my open tasks with their due dates, and the dates live on a project board. To find the dates of about a hundred open tasks, the brief downloaded the entire board, page by page: nine hundred and seventy-eight items.

When this was first measured, in August, the board had six hundred and seventy-three items and a download cost seven hundred and seven points. The board had grown. An hour later, with a fresh budget, the assistant timed one full download: one thousand and ten points, twenty-seven seconds.

The code gave that download twenty-five seconds.

So each download was killed a couple of seconds short of finishing, having already been billed for most of its pages. The retry logic, written for flaky networks, started it again from the first page, up to twice. Nothing was cached, because only a success is cached, and the next brief went through the same cycle. A hundred and fourteen seconds is almost exactly what two killed attempts, their waits and a third that finished add up to. One brief could cost close to three thousand points, and two of them emptied the hour.

Every piece was defensible on its own. The cache keeps only good answers, so it never serves a broken one. Retrying after a timeout is standard practice. The timeout was generous when it was set. Together they turned a slow read into one that paid for itself up to three times and kept nothing.

The fix was not a longer timeout. The brief never needed the board; it needed the dates of the tasks it shows. It now asks for exactly those, a hundred at a time, and each request costs one point. Measured against the real board: two requests, two points, and on the next brief, with those answers cached, no requests at all. The full download still exists for anything that truly needs the whole board, and it now runs once, with room to finish, and never starts over. Every request also records what it cost, so the next time the budget moves there is a number to look at.

I had treated the quota as the limit. It turned out to be the first honest signal that something inside was wrong. When a budget that should be plenty runs out, the useful question is not how to buy more. It is what is spending it, and why nothing told you.