Reporting of Fairness Metrics in Clinical Risk Prediction Models: A Call for Change to Ensure Equitable Precision Health Benefits for All

This repository includes tables and figures from the body of the corresponding article and documents from its appendix. Abstract Background: Clinical risk prediction models integrated in digitized healthcare informatics systems hold promise for personalized primary prevention and care, a core goal of precision health. Fairness metrics are important tools for evaluating potential disparities across sensitive features—such as sex and race/ethnicity—in the field of prediction modeling. However, fairness metric usage in clinical risk prediction models remain rare and rarely empirically evaluated. Objective: We seek to assess the uptake of fairness metrics in clinical risk prediction modeling through an empirical evaluation of popular prediction models for two diseases. Methods: We conducted a scoping literature review in November 2023 of recent high-impact publications on clinical risk prediction models for cardiovascular disease (CVD) and COVID-19 using Google Scholar. Results: Our review resulted in a shortlist of 23 CVD-focused articles and 22 COVID-19 focused articles. No articles evaluated fairness metrics. Of the CVD articles, 26% used a sex-stratified model, and of those with race/ethnicity data, 92% had data from over 50% one race/ethnicity. Of the COVID-19 models, 9% used a sex-stratified model, and of those that included race/ethnicity data, 50% had study populations that were more than 50% one race/ethnicity. No articles stratified their models by race/ethnicity. Conclusion: Our review shows that the use of fairness metrics for evaluating differences across sensitive features is rare, despite their ability to identify inequality and flag potential discrimination. We also find that data remains largely racially/ethnically homogeneous, demonstrating an urgent need for diverse data collection. We propose an implementation framework to initiate change in practice and call for better connections between theory and practice when it comes to fairness metric research and clinical risk prediction so that we can create a more equitable prediction world for all. Key words: Clinical risk prediction; precision health; cardiovascular disease; COVID-19