The Records Mountain: Migrating Legacy Documents Without Poisoning the New Chart

When the contract for the new documentation system is signed, the real work begins in administration. The archive holds three decades of records: hanging files in the basement, plus the yearly batches since the first digitization — tens of thousands of PDF files with names like 20240117_scan_0042.pdf, an inbox folder full of faxes, discharge letters, findings, prescriptions. The scanning service provider reliably delivers pages. What it does not deliver is the answer to the only question that counts: whose document is this?

Somewhere in the stack lies a discharge letter for Mr. M., born March 3, 1951. In the same stack: documents for Mr. M., born March 30, 1951. Same last name, same referring general practitioner. A human who sees both cases side by side spots the difference in seconds. A batch tool that auto-assigns above 95 percent similarity occasionally turns the two into one.

The letter then sits in the wrong chart. There, it looks entirely unremarkable — and the better the new chart works in daily use, the further the error travels.

The real work is called matching

The occasions for such migrations are everyday ones: a system change, a merger, a new site whose referral backlog has to be worked through — or the wish to start with a verified chart without leaving three decades of history behind. And in almost every case, the migration is planned as a scanning project and fails as a matching project. Splitting stacks into individual documents and reading out the text are well-solved problems today — what machine extraction delivers and where its limits lie is described in Structured data extraction from faxes (in German). Identity matching, by contrast, is the step at which a migration project can cause damage that is later almost impossible to find.

Because as soon as systems work onward from the chart, a misassigned document does not remain an isolated filing error. In a chart from which physician letters are drafted, coding suggestions generated and completeness checks run, every one of these steps inherits the error. The foreign diagnosis travels into the next letter, and the next letter itself becomes a source — a mechanism we described in How two correct findings become an invented diagnosis (in German). The word "poisoning" in the title is therefore meant literally: a single misassigned document can contaminate an otherwise clean chart, and the contamination grows with every use.

Then there is the economics. Legacy migrations are paid by the page, and a considerable share of the stock has no clinical value. The extreme example is expert-opinion and social-court files, of the kind that arise in medical expert assessment (in German): 600 to 1,200 pages per file, and in our experience over 90 percent of the pages are non-clinical — cover letters, deadline extensions, administrative correspondence [1]. Whoever sends the entire stack through expensive deep processing processes ten pages to gain one relevant one.

And finally, the human side. If a migration requires tens of thousands of matches and a human is supposed to confirm each one, confirmation becomes routine: someone who reviews 500 proposals in a day, 499 of which are correct, eventually clicks without looking. Ergonomics calls this confirmation fatigue; in everyday project work it means that critical scrutiny slackens precisely when the stacks are at their largest. A process that plans for humans as the control instance must plan for attention wearing out.

What a clean migration needs

Five steps can be distinguished in vendor-neutral terms. None of them is demanding on its own; it becomes risky when one of them is skipped under deadline pressure or silently automated.

1. Splitting: the stack is separated into individual documents — machine-proposed, precise to the page range, correctable by humans.
2. Pre-sorting: clinically relevant pages are separated from administrative correspondence before paid deep processing begins.
3. Matching: identity matching relies on administrative attributes — name, date of birth, insurance number, case number. Clinical content is no matching signal: the fact that a document mentions diabetes must never pull it toward the patient who happens to have diabetes.
4. Confirming: a human confirms every match before anything enters a patient chart.
5. Retracting: errors must be fully reversible — including everything derived from them in the meantime.

The critical point sits in step four. Every vendor will promise it; the question is whether it can be bypassed. A confidence threshold above which matches are accepted automatically, an admin role with bulk approval, a configuration flag for "migration mode": each of these shortcuts turns human control into a recommendation.

How aiomics builds the pipeline

Single-case upload — documents into an existing chart, with verification and provenance — is in production at aiomics. The batch pipeline for stacks and legacy migration is specified and planned; its matching engine is to run in shadow mode first and prove its proposal quality before its proposals reach users. The design decisions behind it are fixed and can be named today.

The human gate cannot be bypassed. No batch size, no admin role, no confidence score and no configuration flag can accept a match automatically; the chart's write path accepts only confirmations set by authenticated humans. This is verified with a deployment test on every release. A "confirm all" button does not exist — as a documented design decision, not a missing feature.

Confirmation shows evidence. The proposed patient and the document's identity fields stand side by side, with discrepancies highlighted. Near-ties — the two Mr. M.s — are never presented as a preselected default.

Attention is measured. In the first months of a migration there are random-sample second reviews. Sustained blind acceptance above 99 percent counts as a warning signal and triggers a review of the process.

Errors are retractable. A match can be withdrawn with one click, including the complete withdrawal of all field values derived from it; the provenance of the event is preserved — it remains traceable later what was in the chart when, and why. And for the gray zone: documents that cannot be assigned to anyone with confidence are parked and remain visible until someone decides. Parked documents never expire silently.

Behind these four decisions stands a risk classification: wrong patient assignment is modeled as the highest damage class in the aiomics platform — higher than any extraction error. That is why the matcher's output is consistently called a "match proposal". Identity is confirmed by humans.

One addition is planned and labeled as such: the pre-sorting of pages into clinical and non-clinical before paid processing — for a 900-page social-court file, the difference between 900 processed pages and the roughly 90 that are clinically relevant. And to draw the boundary: aiomics does not write master data in the hospital information system (KIS); where document and master record contradict each other, on the date of birth for example, the conflict goes to the source for correction.

What to measure any vendor against

1. Is there any path to accepting matches without human confirmation — threshold, special role, configuration flag, API? The defensible answer is a no, together with a description of how that is tested.
2. What does the confirmation interface show? A percentage invites click-through. Demand identity fields side by side with discrepancies marked.
3. How does a misassigned document come out again — and what happens to everything derived from it? "Delete" is not a sufficient answer if fields, drafts or suggestions have been created in the meantime.
4. Does the system measure confirmation behavior? Random-sample second review and a warning signal at near-total blind acceptance show that the vendor treats confirmation fatigue as a reality.
5. Which pages do you pay for? Ask about pre-sorting before paid processing — for expert-opinion and court files, it decides the bulk of the cost.

If a system change or an archive migration is coming up at your facility and you want to think through the matching question in advance, write to us. In our weekly briefing Visite (German; English edition Grand Rounds is in preparation), we write regularly about how reliable patient records are built.

Sources

1. Size and composition of social-court and expert-opinion files (600–1,200 pages per file, of which over 90 percent non-clinical pages): anonymized experience figures from processing such files, as of July 2026, to be read as context.
2. The described mechanisms of the batch pipeline follow the internal product specification of aiomics (as of July 2026); the functionality in question is labeled as planned in the text.