It deleted before it wrote

I re-ran seven books through my distiller. A replace step threw away the old notes, then failed on the very next line, six times over. And one book came back with fewer notes than it started with, which turned out to be the correct answer.

My system distils the books I own into notes I can cite, chapter by chapter, with a locator so I can open the source at the right page. I had improved the parsing and wanted the whole shelf redone, so I wrote a small script to feed each book back through.

The pipeline replaces: it clears the old notes for that book, then writes the new ones. My script handed each book the copy the system had already stored, which meant the copy step was asked to copy a file onto itself. That raises an error, and the error came after the clearing. Six books, 989 notes, gone in the time it takes to read this paragraph.

They were recoverable, because the sources are still on disk and the whole point of the pipeline is that it can rebuild from them. What is not recoverable is the assumption I had been carrying, that a replace is a safe operation because it is one word.

It is two operations wearing one name. The repair was not a guard against my particular mistake; it was moving the clearing to the last possible moment, immediately before the write, so that anything which can fail has already failed while the old notes are still there. The old copy does not go away until the new one exists. I also made the copy step notice when the source and the destination are the same file and do nothing, which is the specific bug, but the ordering is the general fix.

Then the shelf finished and one book was odd. Where the others multiplied, the classic on investing came back with 195 notes where it had 218. Fewer notes after an improvement is exactly the shape of a regression, so I went to look.

The old 218 came from three enormous chapters cut into slices by byte count. Each slice was handed to the distiller as if it were a unit of thought, and each one dutifully restated the same principle in slightly different words, because that is what the surrounding text was doing. The new 195 are chapters. The count went down and the content went up, and the reason the other books grew is that their chapters were being found for the first time: one went from 46 units to 241.

Two things I keep. A destructive step and a constructive step under a single verb will eventually run apart, and the ordering is the only thing that saves you, not care.

The second is that more is not the direction of quality. I very nearly treated a smaller number as a failure, when the smaller number was the first honest count I had. If the unit is wrong, every metric built on it is confident and useless.