AI's Medical Data Bias

Inaccurate race and ethnicity data within electronic health records (EHRs) presents a growing problem as AI is further integrated into healthcare. The inconsistent collection and classification of this data by hospitals and healthcare providers can lead to biased AI systems, which in turn perpetuate racial disparities in healthcare outcomes, diagnoses, and treatment plans. AI algorithms trained on skewed datasets may reinforce existing inequalities, eroding trust between patients and healthcare providers, especially among marginalised communities. Addressing these challenges requires standardised data collection practices, transparent reporting, and careful consideration of data quality in AI development to ensure equitable healthcare delivery.

To mitigate these issues, healthcare systems should adopt best practices for race and ethnicity data collection, including patient self-reporting, staff training, and transparent processes. Developers of medical AI models should also disclose how their race and ethnicity data were collected, enabling critical assessment of the safety and reliability of AI-based healthcare tools. Open dialogue and continuous improvement in data handling are vital steps toward achieving significant advancements in equitable AI applications in healthcare. The EU AI Act mandates that datasets used for high-risk AI systems must be representative and error-free, highlighting the urgency of accurate patient race and ethnicity data.