The Transparency Gap: How Missing Data in FDA-Approved AI Devices Risks Patient Health

Despite FDA principles for AI/ML medical device transparency, manufacturers often underreport crucial data on demographics and safety. This gap creates significant risks of algorithmic bias, exacerbating health disparities and compromising patient care.

Mateo Rossi

June 4, 2026 · Updated October 2, 2026 · 4 min read

The U.S. Food and Drug Administration has established guiding principles for transparency in artificial intelligence and machine learning (AI/ML) medical devices, aiming to ensure their safety and promote health equity. However, independent analyses of publicly available data reveal a significant gap between these regulatory ideals and the information manufacturers provide. Research shows that crucial data on patient demographics and device safety is consistently underreported for hundreds of FDA-approved AI products, creating a substantial risk of algorithmic bias that could amplify health disparities and compromise patient care.

The FDA's Stated Principles for AI/ML Device Transparency

The FDA’s Center for Devices and Radiological Health (CDRH) regulates AI-enabled medical devices under the Federal Food, Drug, and Cosmetic Act, employing a risk-based approach that considers a device's intended use and technological characteristics. A central pillar of this oversight is transparency, which the agency defines as the degree to which appropriate information about a device—including its intended use, development, performance, and logic—is clearly communicated to relevant audiences. According to the FDA, this clarity is essential for safe and effective use, as it supports informed decision-making, helps identify bias, and ultimately promotes health equity.

The agency’s principles call for clear documentation of an algorithm's design, the data used to train it, and its decision-making logic, in line with what it calls Good Machine Learning Practices (GMLP). During a public workshop on the topic, stakeholders including patients, providers, and software developers recommended a focus on the data sources used to train and validate AI software. They specifically urged the FDA and manufacturers to provide demographic information about the populations involved in testing. This aligns with the FDA's own stated concerns that algorithms trained on data with limited patient and site diversity may not generalize well to clinical practice across the United States.

The FDA's expectations extend across the "total product life cycle," not just the initial approval. This means developers are expected to monitor real-world performance and have a predetermined change control plan to manage updates without needing a full new submission for every modification. This regulatory flexibility is intended to support innovation while maintaining safety. A key part of this ongoing oversight is the principle that users, such as clinicians, should receive clear and essential information. As the FDA states, transparency supports "health equity through identification of bias" and increases "confidence in MLMD use."

A comparison of FDA principles for AI/ML device transparency against the documented reality of manufacturer reporting and the resulting risks.
FDA Principle for Transparency	Documented Reality (from Research)	Consequence/Risk
Training Data Characterization	The FDA emphasizes transparency regarding data sources, bias in training data, and the inclusion of diverse populations to ensure algorithms are robust and generalizable.	An analysis as of December 2024 found "significant gaps" between FDA guidance and publicly available data. A separate scoping review of 950 approved devices found that demographic and socioeconomic characteristics are consistently underreported in approval documents.	Underrepresentation in training data can amplify existing racial and gender disparities. This can lead to flawed models that produce incorrect scoring or diagnoses, thereby exacerbating health inequities.
Post-Market Performance & Safety	The FDA expects manufacturers to have change-management plans and to monitor real-world performance to ensure continued safety and effectiveness, reporting on "device modifications and updates."	Researchers found that FDA reporting data remains inconsistent, with irregular reporting of safety considerations and potential adverse events in device approval documents.	Without consistent safety and performance data, it is difficult for regulators and healthcare providers to monitor for "model drift" or performance degradation, increasing risks to patient safety.

The Concrete Risks of Opaque AI in Medicine

The gaps in data transparency for AI-enabled medical devices are not merely administrative shortcomings; they create tangible risks for patients and health systems. AI models can be flawed due to problems with the data used to develop them. When training datasets are not representative of the diverse populations on which the devices will be used, these flaws can perpetuate and even worsen existing health inequities.

Research shows that the underreporting of demographic characteristics in FDA submissions directly exacerbates the risk of algorithmic bias. An AI tool trained predominantly on data from one demographic group may underperform when used on patients from other groups, leading to inaccurate diagnoses or risk scores. This can result in what one device's safety report described as the "misuse of the device to analyze images from an unintended patient population." That device, the Brainomix 360 e-ASPECTS, also highlighted potential risks of incorrect scoring by the algorithm and outright device failure. Without consistent reporting, it is difficult to know how many other devices carry similar unstated risks.

This lack of transparency also places a significant burden on healthcare providers. The FDA advises that AI performance should be monitored in the specific environment where it is being used. However, as a study in npj Digital Medicine notes, this is "a difficult aim to implement without transparency on how the model was tested and evaluated." Without access to the underlying data on a device's validation, clinicians cannot confidently assess whether a tool is appropriate for their specific patient population, undermining their ability to ensure safe and equitable care. They are left to trust a "black box" tool without the information needed to verify its local accuracy.