Mitigating Skin Tone Bias in Vital Sign Monitoring with AI

Variations in skin tones leads to biased results from oximeter devices, leading to inaccurate vital signs measurements for darker-skinned patients.

At PHUSE Connect 2024, I presented a solution for deriving less biased readings from medical devices when it came to performance across various skin tones. This was part of the Analytics Risk-based Monitoring stream, to a global audience of Clinical and Data Scientists and leaders in Pharmaceuticals and Healthcare. You can see my presentation materials here. The full paper is available to read here.

Pulse oximetry is a critical tool in modern healthcare, providing non-invasive measures of arterial oxygen saturation (SaO2). They are used in clinical trials to monitor oxygen saturation levels in study participants across various research areas in order to gauge safety and efficacy in randomized controlled trials for new drugs with respiratory or cardiovascular effects, medical device trials, or experimental treatments, such as those targeting low oxygen levels in illnesses like acute respiratory distress syndrome or traumatic brain injury.

A systematic review found that pulse oximetry (SpO2) overestimates oxygenation more commonly in patients of racial and ethnic minority groups, leading to variations in treatment. When comparing oxygen saturation measures across races, one particular study found significant overestimation from Black patients compared to White patients. These findings are consistent with other studies addressing pulse oximeters in vulnerable populations.

In a trial setting, placing Quality Tolerance Limits (QTLs) on SpO2 levels and designating oxygen saturation estimation as a Key Risk Indicator (KRI) enables enhanced patient safety and data integrity. This approach works effectively with continuous access to oxygen saturation levels, crucial for responsively modifying protocols and facilitating timely medical interventions.

Darker-skinned patients often 3x as likely to have hypoxemia missed by pulse oximeters. Even a small bias of 1% overestimating SaO2 can spike hidden hypoxemia Tendency to overestimate oxygen saturation by up to 4%, worsening as patient SaO2 levels drop. Historical clinical data suffers from systematic disparities leading to underrepresentation of marginalized groups However, a machine learning model trained on SaO2 levels with respect to fairness can mitigate this In this case, the machine learning model presents less bias (over or underprediction) of SaO2 levels than pulse oximeter readings, as well as signaling patient groups with high measurement error from device

By using AI to evaluating whether QTLs have been met or exceeded, thus improving the data quality of oxygen saturation measures. This advancement promises improved oximetry accuracy, informing QTLs that more effectively counteract bias. Clinicians stand to gain a tool for more precise interventions, elevating patient care and mitigating risks associated with flawed oximetry, particularly vital in managing ARDS, TBI, and preventing critical hypoxic episodes.

Beyond just pulse oximeters, several other racial disparities have been observed in healthcare. When comparing Black patients to White patients, there is a 40% Higher Breast Cancer Mortality among women. There is worse MS (multiple schlerosis). Prognosis and a 22% Higher Mortality from Melanoma despite underrepresentation in public skin lesion images. These measurements also risk capturing unwanted biases across vital monitoring, image diagnostics and skin-based treatments benefit from a data-driven approach. Such conditions may benefit from an approach comparable to the solution for biased SpO2 readings outlined in this dataset, when it comes to measurements that could impact their treatment or standard of care. This could involve using QTLs that are derived from or compared against calibrated measurements from ML models trained on historical, benchmark, and even real world data.

Kimani Toussaint is an optics expert whose lab at Brown University creates precision techniques to image and assess biological tissues. HR and his doctoral student Rutendo Jakachira work on optical technology in order develop a next-generation pulse oximeter they hope will work well on patients of all skin tones, not just those with lighter skin. It may take some time before this device is ready, and much more before it becomes commercially viable. It is neither cheap nor environmentally neutral to produce such devices on that scale, so perhaps a software or hybrid solution, using both optical technology and machine learning, could bring about a new era of reliable pulse oximetry.

Carry on reading to learn more about bias and fairness in data and AI…