A medical algorithm can be rewritten while it is reading your scans, and the public record will not tell your doctor whether the new version is any better than the one that was approved. That is not a thought experiment. A team of researchers went into the Food and Drug Administration’s own files and counted what happens after clearance, and what they found is a regulatory paper trail that goes quiet the moment the product ships. The woman in the scanner has no idea which version of the software is looking at her tissue. In most cases, neither does the radiologist sitting two rooms away. The agency approved one product. The vendor has been shipping several.
Viet-Thi Tran — He Read the Regulator’s Own Paperwork and Found the Story Stops at Approval
Viet-Thi Tran, a physician and epidemiologist at Université Paris Cité and the Paris hospital system AP-HP, worked with colleagues at Columbia University’s Mailman School of Public Health on the audit published Nov. 5 in JAMA Network Open. His team took 163 AI and machine-learning devices cleared through the 510(k) pathway, with first clearances between May 2015 and December 2021, then tracked every related entry in the FDA database through March 31, 2025. That produced 352 filings to read. Fifty-six of the devices, 34.4%, received at least one rewrite of the algorithm after approval, spread across 106 entries, ranging from a single update to thirteen on one device. The median wait before the first rewrite was 1.8 years. JAMA Network Open
Approval, it turns out, is a photograph. Medicine thought it was buying a warranty.
The 510(k) Pathway — A Clearance Describes Software on One Day and Says Nothing About the Next
The 510(k) route clears a device as substantially equivalent to something already on the market, and it frequently requires no fresh clinical trial. The summary attached to that clearance is often the only public evidence a hospital committee ever sees before it signs a contract. U.S. Food and Drug Administration
So the file is the evidence. And the file is about the version that existed in the room that day, not the one that will be running in the scanner three winters from now.
Seventy-Five Percent — Most Rewrites Changed What the Device Was For, Not Just How Well It Worked
Forty-two of the 56 updated devices, 75%, received at least one change that altered how the device is actually used. Of the 95 usage changes the reviewers recorded, 72 (75.8%) changed what the algorithm consumes — the study’s own example is a model that had been reading electrocardiogram waveforms and then started taking blood pressure data as well. Another 23 (24.2%) added new capabilities, such as an alerting function grafted onto an imaging tool. Only 9 devices (16.1%) had updates confined to plain performance tuning. Nearly nine in ten of the devices in the cohort, 89%, were built to analyze medical images. Medical Daily
A tool that starts digesting a second kind of input is not the same tool with a newer engine. It is a different tool wearing the same nameplate, and the nameplate is what the purchase order says.
Forty-Three Percent — Nobody Wrote Down Whose Scans Were Used to Test the New Version
Here is where the record gets thin enough to see through. Across the 56 updated devices, 24 (42.9%) carried no information in any related entry about the patients or datasets used to evaluate the updated version. Fifteen (26.8%) never stated a sample size. Eight (14.3%) never reported metrics or results at all. And 48 of the 56, 85.7%, reported no clinical or organizational benefit — no improved detection, no faster read, nothing a patient would notice in a body rather than in a spreadsheet. Eight devices did report a benefit. That is the entire affirmative case, fourteen percent of the field.
The industry produced a number, the regulator filed it, and the part where anybody got better appears to have been left in a drawer.
The PCCP Loophole — December 2024 Quietly Made the Next Wave of Rewrites Invisible
In December 2024 the FDA finalized guidance on predetermined change control plans, a mechanism that lets manufacturers describe planned modifications in advance and then implement them without a new submission so long as they stay inside the authorized envelope. U.S. Food and Drug Administration Those updates do not appear in Tran’s dataset, because there is no modification request to find. Which means 34.4% is a floor, not a count. The true figure is higher, and no one outside the manufacturers can say by how much.
The paperwork got lighter. The software did not get simpler.
Patients — The Only People in This Dataset Without Names Are the Ones It Was Built to Protect
The FDA has cleared more than 1,000 AI and machine-learning medical devices, most through this single pathway, and the agency’s public list was last updated Sept. 4, 2026. U.S. Food and Drug Administration Most of what is on it looks at pictures of human interiors: mammograms, CTs, chest films in a rural emergency department at two in the morning. No patient is told when the software version changes. No consent form covers it. The person whose images were used to check the rewrite is not identified in the filing either, so there is no one to ask what the new version did to them.
Tran’s team is careful about the limits, and the limits matter. The audit used public documents only, and it cannot establish that any device performed worse after an update. It could not see the updates that skip the process entirely. The study was funded through a French public investment grant, and the authors report no conflicts. All of that is in the letter, and all of it is honest. It is also the indictment: after a decade of clearance and hundreds of filings, the honest answer to “does the updated version still work as well for patients” is that the public record does not say, and nobody has been required to make it say.
An AI-mekna that reads a scan is a medical opinion with a serial number. Right now the serial number is the only part anyone can audit.
The FDA cleared 163 algorithms, the vendors rewrote more than a third of them, and the file went silent at the exact moment the question started to matter. Silence is not a safety record.
Sources: JAMA Network Open, Medical Daily, U.S. Food and Drug Administration.