I had a page rebuilt this week and ran an audit over it before promoting it anywhere. Bilingual, two colour themes, eight sections, the usual surface. The audit came back with thirteen findings and nothing critical: contrast ratios under threshold, a heading level skipped, a focus ring that vanished on one background, a couple of layout breaks at heights nobody designs for.
Good result. I worked through the list, fixed what it found, and wrote the summary line: twelve of thirteen closed, one deferred with a reason.
Then I had a second pass check the work, and it could confirm ten.
The two it could not confirm
Neither of the two was a lie, exactly, and that is the part worth writing down. Both were fixes I had genuinely made, in the sense that I had changed the code in the direction the finding pointed. What I had not done was re-measure the thing the finding was actually about.
One was a contrast ratio. The finding named a token whose colour failed against its background. I changed the token. I did not re-run the ratio, because the new colour was visibly darker and visibly darker is obviously better. It was better. It was also still under threshold, by a margin small enough that no eye catches it and any calculator does.
The other was a focus state. The finding said the ring disappeared against one particular surface. I gave it an outline offset, looked at it, and it looked fine. On the surface I happened to be looking at. The reported failure was on the other one.
So both were closed by inspection, and both findings were specifically about things inspection cannot settle. That is not a coincidence. An audit exists to catch what looking does not catch. So the findings most likely to survive a fix are exactly the ones you will be most tempted to verify by looking.
Why the writer cannot be the checker
I have written before about reviewing my own work and missing things because the artifact I reviewed was structurally incapable of showing the problem. This is the same family, and it has a cleaner statement.
When you fix a finding, you form an intent: make this contrast pass. You then edit toward the intent. When you go back to check, you are not reading the code cold. You are reading it against the intent you just held, and the intent is the most available thing in your head. The code says "I changed this colour," and your intent supplies the rest of the sentence: "...so it passes now." The check confirms the change, not the outcome, and the two feel identical from inside because the change is the only thing you actually did.
A second pass has no intent to supply. It reads finding one, goes to the number, and the number is the number. It does not know what I meant.
This is the thing I keep re-learning in different costumes. The reviewer has to be someone other than the producer, and someone other is not a courtesy or a second opinion. It is the only way to strip out the context that makes a partial fix read as a complete one.
The arithmetic that bothered me
Twelve reported, ten confirmed. Call it an 83% self-report accuracy, which sounds respectable until you notice what it means for the summary line.
The line said twelve of thirteen. Its whole purpose was to be the thing I could read next month instead of re-running the audit. That is what a status line is for: a cache. If the cache is 83% accurate, then next month I am not looking at a page with one known gap. I am looking at a page with three, two of which I have written down as closed and will therefore never check again.
The error does not sit still, either. A wrong "fixed" is worse than a wrong "open", because "open" stays on the list and gets looked at eventually, while "fixed" leaves the list forever. A false close is permanent in a way a false open is not. Two of them a week compounds into a page that reports clean and measures otherwise, and the report is the only thing anyone reads.
What I changed
Two rules, and one thing I decided not to do.
-
A finding closes on the measurement it was opened on, not on the edit. If the finding is a ratio, the close is a ratio. If it is a state on a specific surface, the close is that state on that surface. Same instrument, both directions. "I changed the colour" is a description of work, not evidence, and the two have been sitting in the same sentence for as long as I have been writing these.
-
The verifier is a separate pass that never sees the fix list as authored. It gets the original findings and the current code, not my summary of what I did to bridge them. If it has my summary it will grade my summary, which is the failure I already have.
The thing I decided not to do is add more findings, or a stricter audit, or a third opinion. The audit was good. It found thirteen real problems and got zero of them wrong. Nothing upstream of the fixes was broken.
What was broken was the one step everybody skips because it feels like it has already happened: measuring the thing again, after. It costs less than the original audit and I had been treating it as optional, because by that point I already knew the answer.
I did not. I knew the intent.