Publishing a failure dataset without publishing your clients

We publish our production failure ledger as a dataset. The useful part of a failure record is the mechanism, and the mechanism is written in prose by whoever hit the problem. Prose written for colleagues names people.

The first version of the dataset covered 359 records. We anonymised it with a denylist: client slugs mapped to pseudonyms, a list of named staff, and patterns for keys, paths, hostnames, customer ids and session ids. A gate ran every pattern over the output and refused to write the file on a hit. It held.

The second version covers 4,308 records. The same approach does not hold, and the way it fails is worth writing down, because the failure is silent.

A denylist is a list of the people you remembered

The gate only catches names it already knows. So we stopped testing against the list and tested against the data: a scan for capitalised two and three word sequences across every prose field, with technical vocabulary filtered out.

It returned 431 distinct candidates.

431

Reading them is uncomfortable. Client staff. Business contacts. Prospects our outbound system had written to. A job applicant, named in a record about a retrieval bug, because the record was about retrieving the wrong candidate.

Our denylist named about a dozen people. It was not close, and nothing about the gate passing on version 1.0 had suggested it ever would be.

The scan also found three leaks the pattern gate could not have caught:

A column nobody thought of as prose. Our system field is a short slug like vercel or vitest, so it was never scrubbed. On three records it was a client name.

A client missing from the tenant map entirely. They were onboarded after version 1.0 was built, so no rule existed to catch them, and their company name was sitting in a record title.

Truncated identifiers. The rule for payment-processor customer ids required eight characters after the prefix. Our records quote ids shortened for readability, with four characters and an ellipsis. Those sailed through a gate built to stop exactly them.

The rule we replaced it with

Three changes, in order of how much they matter.

Stop publishing the prose. The dataset now carries structured fields for every record plus the title, and nothing else. Layer, signal, detection, prevention, closure, severity, dates, recurrence. That is what the study analyses anyway. The free text was carrying the research value of a sentence and the privacy risk of a page.

Gate the titles by shape, not by name. Titles turned out to be almost entirely technical. A scan over all 4,308 of them returned 88 name-shaped candidates, of which two were real people and two were client companies. So titles are publishable, but only under an inverted rule: every name-shaped sequence must appear on a reviewed allowlist of technical terms, and anything else is replaced with a placeholder and reported. Unrecognised is redacted rather than passed. A new client next quarter stops the build instead of shipping.

Fix the three leaks, and make the gate honest. The system column is scrubbed. The id rule matches from three characters for prefixes that cannot be an English word. The missing client is in the map. And one pattern in the gate was reporting a false positive on an environment variable that happened to contain the same four characters as a customer id prefix, which we anchored, because a gate that cries wolf is a gate people learn to skim.

What this costs

Version 1.0 published a redacted cause sentence per record. Version 1.1 does not, and that is a real loss. A reader can no longer pull a row and see the mechanism in the practitioner's own words.

We think it is the right trade and we are not pleased about it. The alternative was to hand-review 431 strings and bet that we had not miscategorised one. We would have lost that bet eventually, and the cost of losing it is not a correction notice, it is a real person's name on a public page attached to a record about a system failing.

The part that generalises

If you are publishing anything derived from internal incident records, the question is not whether your redaction list is good. It is whether your list can ever be complete, and for free text written by colleagues the answer is no.

So the test is not "did the gate pass". It is "what would this gate not have been able to see". Run a scan that does not consult your list. Ours took an afternoon and changed what we were willing to publish.

The dataset, the codebook and the full method are at the Agent Failure Index.