| # | Name | Source | Disease | Type of Source | Application Scope | Type of Bias | Short Description | Impact On |
|---|---|---|---|---|---|---|---|---|
| 1 | Implicit gender bias in the diagnosis and treatment of type 2 diabetes: A randomized online study | A. Skvortsova, S. H. Meeuwis, R. C. Vos, H. M. M. Vos, H. van Middendorp, D. S. Veldhuijzen, A. W. M. Evers (2023), Implicit gender bias in the diagnosis and treatment of type 2 diabetes: A randomized online study, https://onlinelibrary.wiley.com/doi/10.1111/dme.15087 | Diabetes | Scholarly publication | Treatment response | Representation bias | This study examined how implicit gender biases (IGBs) among general practitioners (GPs) influence diagnostic and treatment decisions in type 2 diabetes care. Ninety-nine GPs completed Implicit Association Tasks assessing biases related to lifestyle and communication, alongside clinical vignette evaluations featuring male or female patients. Results showed that female GPs held stronger lifestyle-related IGBs, while all GPs demonstrated communication-related IGBs. Patient gender also affected diagnostic certainty, with lower confidence in diagnosing diabetes in female cases. The findings highlight how unconscious gender biases can shape clinical decision-making, underscoring the need for further research into their impact on diabetes care. | Gender bias affecting medical decision-making |
| 2 | Systematic review and meta-analysis of performance of wearable artificial intelligence in detecting and predicting depression. | Abd-Alrazaq, A., AlSaad, R., Shuweihdi, F., Ahmed, A., Aziz, S., & Sheikh, J. (2023). Systematic review and meta-analysis of performance of wearable artificial intelligence in detecting and predicting depression. Npj Digital Medicine, 6(1), 1–16. https://doi.org/10.1038/s41746-023-00828-5 | Depression | Scholarly publication | Diagnosis, Prediction | Representation bias | The systematic review and meta-analysis found that wearable AI systems demonstrate promising performance in detecting and predicting depression. However, substantial variability exists among algorithms and devices, thereby indicating that performance can vary significantly. | Both race and gender: the identified disparities across different algorithms and devices suggest that certain demographic groups may be underrepresented or inadequately served by current wearable AI systems. This variability underscores the need for further research to enhance the generalisability and fairness of these technologies in clinical practice. |
| 3 | Gender Bias in AI's Perception of Cardiovascular Risk | Achtari M, Salihu A, Muller O, Abbé E, Clair C, Schwarz J, Fournier S. Gender Bias in AI's Perception of Cardiovascular Risk. J Med Internet Res 2024;26:e54242 (doi:10.2196/54242) | Cardiovascular | Scholarly publication | Risk assesment | Historical Bias, Evaluation bias | The study investigated gender bias in GPT-4’s assessment of coronary artery disease risk and showed that there was a substantial shift in the perception of risk between men and women when a psychiatric comorbidity was added to the vignette, even when they presented identical complaints. | Gender: Women were assessed as having a lower risk of CAD when concurrently having a psychiatric condition. |
| 4 | Cardiovascular Disease Screening in Women: Leveraging Artificial Intelligence and Digital Tools | Adedinsewo, Demilade A., Amy W. Pollak, Sabrina D. Phillips, et al. ‘Cardiovascular Disease Screening in Women: Leveraging Artificial Intelligence and Digital Tools’. Circulation Research 130, no. 4 (2022): 673–90. https://doi.org/10.1161/CIRCRESAHA.121.319876. | Cardiovascular | Scholarly publication | Disease prevention, Public health, Surveillance | Historical Bias, Representation bias, Measurement bias | Provides a broad overview of the key opportunities for cardiovascular screening in women while highlighting the potential applications of artificial intelligence, along with digital technologies and tools. Underlines the fact that health care providers, clinical guidelines committees, and health policy experts are not yet sufficiently equipped to optimize the collection of data on women, use or interpret these data, or develop approaches to targeting interventions. Has a specific section on AI bias and addresses specific underrepresented subgroup characteristics: youth, Pregnancy and Peripartum, Middle Age and Menopause, Older Age. Also, includes clinical guideline and public heath implications of AI. | Gender |
| 5 | Age, sex and race bias in automated arrhythmia detectors, Journal of Electrocardiology | Alday, E.A.P, Rad, A.B, Reyna, M.A, Sadr, N, Gu, A, Li, Q., Dumitru, M., Xue, J., Albert, D., Sameni, R., Clifford, G.D. Age, sex and race bias in automated arrhythmia detectors, Journal of Electrocardiology, 2022 Sep-Oct:74:5-9. DOI: https://doi.org/10.1016/j.jelectrocard.2024.01.006 | Cardiovascular | Scholarly publication | Diagnosis, Prediction | Historical Bias, Representation bias, Aggregation bias | Very few studies have examined diagnostic algorithmic bias (as a function of sex, age, and race) in any meaningful manner, comparing across algorithms, databases and assessment metrics. The machine learning algorithm falsely concluded that Black patients were healthier than equally sick White patients, which led to Black patients being less likely to be referred to case management despite being equally sick as White patients. In the case of ECG, there are not many studies looking at bias across algorithms, databases, and metrics. Most studies look at ECG features and prognostics. The aim of this study is to compare the performance by sex, age, and race of 56 ECG automatic classification algorithms on over 130,000 electrocardiograms (ECGs) using a wide range of metrics, and propose a bias reduction machine-learning model design to help improve health equity. | Gender, Age, Race |
| 6 | Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias | Alipour, M., & Alipour, A. (2025). Algorithmic Bias in AI-Based Diabetes Care: Systematic Review of Model Performance, Equity Reporting, and Physiological Label Bias. InfoScience Trends, 2(5), 33–46. https://doi.org/10.61186/ist.202502.05.04 | Diabetes | Scholarly publication | Diagnosis, Risk assesment, Treatment response | Representation bias, Measurement bias, Evaluation bias | This study analyzed AI-based glucose prediction models for diabetes care, focusing on performance disparities, fairness reporting, and physiological label bias. The study finds that many models exhibit performance disparities across different racial and ethnic groups, often due to underrepresentation in training datasets. Physiological label bias was also identified, where certain physiological markers are overrepresented or underrepresented in datasets, leading to skewed model predictions. | Race: biases are a source of skewed predicitons across racial groups, giving rise to under- or over-identification of diabetes risk and inappropriate treatment recommendations and affeecting the reliability of biomedical AI with regards to diasgnosis, riska assessment and treatment response. |
| 7 | Salud para quién? Interseccionalidad y sesgos de la inteligencia artificial para el diagnóstico clínico. Anales del Sistema Sanitario de Navarra | Amaya-Santos, Sua, Jiménez-Pernett, Jaime, & Bermúdez-Tamayo, Clara. (2024). ¿Salud para quién? Interseccionalidad y sesgos de la inteligencia artificial para el diagnóstico clínico. Anales del Sistema Sanitario de Navarra, 47(2), e1077. Epub 07 de octubre de 2024. https://dx.doi.org/10.23938/assn.1077 | Not Sure/Other | Scholarly publication | Diagnosis, Public health | Evaluation bias | It addresses the ethical and equity challenges involved in the rapid integration of Artificial Intelligence (AI) into clinical diagnosis, warning that algorithms, being trained on data that reflect historical inequalities, can perpetuate gender, race, and class biases that disproportionately affect marginalized groups. To face this challenge, the article emphasizes the need for an intersectionality approach to understand how the overlap of these identities exacerbates the risk of biased diagnosis. It concludes with a call to establish rigorous ethical and regulatory frameworks to ensure that the implementation of AI in the health system promotes universal health equity rather than exclusion. | Race, Gender, Socioeconomic groups |
| 8 | Race, Racism, and Risk Prediction for Cardiovascular Disease | American Heart Association, (2024). Race, Racism, and Risk Prediction for Cardiovascular Disease, https://newsroom.heart.org/news/race-racism-and-risk-prediction-for-cardiovascular-disease | Cardiovascular | Webpage | Management and planning | Measurement bias | The article discusses how traditional cardiovascular risk-prediction models, such as the Pooled Cohort Equations, have used race as a variable, which can unintentionally reinforce biological misconceptions and perpetuate inequities. To address this, the AHA developed PREVENT™, a new tool that predicts 10- and 30-year risks for heart attack, stroke, and heart failure without including race. Instead, it uses a broader range of factors—such as kidney and metabolic health indicators and a social deprivation index based on ZIP code—to better capture social determinants of health. Validation studies show that PREVENT maintains strong predictive accuracy across racial and ethnic groups. The article emphasizes, however, that removing race from predictive models does not eliminate structural racism or health disparities, underscoring the need for continued efforts to account for social and systemic factors influencing cardiovascular risk. | Race: Although the article argues that removing race from predictive models does not eliminate structural racism or health disparities, underscoring the need for continued efforts to account for social and systemic factors influencing cardiovascular risk, the exclusion of race might have an impactthe accuracy of the results |
| 9 | Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity | Amponsah D, Thamman R, Brandt E, James C, Spector-Bagdady K, Yong CM. Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity. Curr Cardiovasc Risk Rep. 2024 Nov;18(11):153-162. doi: 10.1007/s12170-024-00745-6. Epub 2024 Aug 20. PMID: 40144330; PMCID: PMC11938301, https://pmc.ncbi.nlm.nih.gov/articles/PMC11938301/ | Cardiovascular | Webpage | Diagnosis | Other | The articles reviews how artificial intelligence (AI) can both advance and threaten racial and ethnic equity in cardiovascular health. Authors explain that while AI tools can identify disparities, improve diagnosis, and expand inclusion in research, they can also reinforce bias if trained on non-representative data or deployed without equity safeguards. The authors propose three pillars for equitable AI in cardiology: 1) Diverse, representative data that include social determinants of health. 2) Ongoing bias auditing and subgroup validation to ensure fair performance. 3) Inclusive design and governance, with diverse developers and ethical oversight. They conclude that AI is not a cure-all for disparities, but with intentional, equitable design and monitoring, it can become a powerful tool to reduce cardiovascular health inequities. | Race |
| 10 | Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity | Amponsah, D., Thamman, R., Brandt, E., James, C., Spector-Bagdady, K., & Yong, C.M. (2024). Artificial Intelligence to Promote Racial and Ethnic Cardiovascular Health Equity. Current Cardiovascular Risk Reports, 18(11), 153–162. https://link.springer.com/article/10.1007/s12170-024-00745-6 | Cardiovascular | Scholarly publication | Diagnosis | Representation bias | This review discusses how AI in cardiology can both mitigate and exacerbate racial and ethnic inequities. It highlights examples such as an AI tool for identifying aortic stenosis patients that improved overall care but still showed residual delays for Black patients, and ECG models that underperform in Black women due to non-diverse training data. The paper outlines solutions like bias audits, inclusive data collection, workforce diversity, and stakeholder engagement. | Black and minority ethnic patients, women, and underrepresented groups in cardiology care and research |
| 11 | Bias and Fairness in AI-Based Mental Health Models. Ladoke Akintola University of Technology | Barnabas Barnty, Oloyede Joseph & Emmanuel Ok (2025). Bias and Fairness in AI-Based Mental Health Models. Ladoke Akintola University of Technology. https://www.researchgate.net/publication/389214235_Bias_and_Fairness_in_AI-Based_Mental_Health_Models | Depression | Scholarly publication | Diagnosis | Representation bias | The paper examines bias and fairness issues in AI-based mental health applications, including diagnostic tools, chatbots, and suicide risk prediction models. It reports how unrepresentative datasets lead to misdiagnosis and unequal outcomes across gender and racial groups, and presents mitigation strategies such as diverse datasets, fairness metrics, and human-in-the-loop approaches. | Women, racial and ethnic minorities, non-Western populations, lower socioeconomic groups |
| 12 | Bias in AI-Driven Diabetes Prediction Models: Challenges, Impacts, and Mitigation Strategies | Bhimavarapu U. (2025). Bias in AI-Driven Diabetes Prediction Models: Challenges, Impacts, and Mitigation Strategies. https://www.igi-global.com/chapter/bias-in-ai-driven-diabetes-prediction-models/376753 | Diabetes | Scholarly publication | Other | Other | Diabetes is a chronic condition that requires early detection and effective management to prevent complications and improve quality of life. As advancements in artificial intelligence (AI) and machine learning (ML) continue to play a significant role in healthcare, diabetes prediction models have emerged as valuable tools for identifying individuals at risk. However, the effectiveness of these models is often impacted by bias, which can distort predictions and lead to inaccurate or unfair outcomes. This study explores the various types of bias that can affect diabetes prediction models, including data bias, sampling bias, label bias, algorithmic bias, and confirmation bias. We also examine the consequences of bias in predictive healthcare models, particularly in diabetes detection, and discuss how biased models can perpetuate health inequities. Finally, we provide recommendations for mitigating bias, such as using diverse and representative datasets, employing fair algorithms, validating models on external datasets, and continuously monitoring and updating models. | |
| 13 | Racial bias in AI-mediated psychiatric diagnosis and treatment: a qualitative comparison of four large language models | Bouguettaya, A., Stuart, E.M. & Aboujaoude, E. (2025). Racial bias in AI-mediated psychiatric diagnosis and treatment: a qualitative comparison of four large language models. npj Digit. Med. 8, 332. https://doi.org/10.1038/s41746-025-01746-4 | Not Sure/Other | Scholarly publication | Diagnosis, Treatment response | Representation bias, Learning bias, Evaluation bias | Ten psychiatric patient cases representing five diagnoses were presented under three conditions: race-neutral, race-implied, and race-explicitly stated. The study finds that large language models (Claude, ChatGPT, Gemini, NewMes-15) often proposed inferior treatments when patient race was explicitly or implicitly indicated, though diagnostic decisions demonstrated minimal bias. NewMes-15 exhibited the highest degree of racial bias, while Gemini showed the least. | Race: racial biases undermine the fairness and reliability of AI devices in psychiatric diagnosis and treatment, creating unequal predictive performances across racial groups. This can lead to under- or over-identification of mental health conditions and the provision of less effective or riskier treatments for certian racial groups. |
| 14 | Addressing Artificial Intelligence Bias in Retinal Diagnostics. Translational Vision Science & Technology | Burlina, P., Joshi, N., Paul, W., Pacheco, K.D., Bressler, N.M. Addressing Artificial Intelligence Bias in Retinal Diagnostics. Translational Vision Science & Technology February 2021, Vol.10, 13. Addressing Artificial Intelligence Bias in Retinal Diagnostics. https://doi.org/10.1167/tvst.10.2.13 | Diabetes | Scholarly publication | Diagnosis | Representation bias, Measurement bias, Aggregation bias | This study evaluated generative methods to potentially mitigate artificial intelligence (AI) bias when diagnosing diabetic retinopathy (DR) resulting from training data imbalance or domain generalization, which occurs when deep learning systems (DLSs) face concepts at test/inference time they were not initially trained on. Findings illustrate how data imbalance and domain generalization can lead to disparity of accuracy across subpopulations, and show that novel generative methods of synthetic fundus images may play a role for debiasing AI. New AI methods have possible applications to address potential AI bias in DR diagnostics from fundus pigmentation, and potentially other ophthalmic DLSs too. When considering retinal diagnostics of DR, this study demonstrated that situations of data imbalance and domain generalization can affect the performance of AI diagnostics algorithms applied to a task, such as DR referable versus not referable classification, and result in AI bias for individuals of presumed diverse racial/ethnicity/origins partitioned along darker-skin versus lighter-skin populations, assuming the presumed skin pigmentation relates, on average, to the concentration of melanin within uveal melanocytes and subsequently on retinal coloration. The results suggest the potential benefit of certain generative methods that alter specific image markers to allow the augmentation of the diagnostic DLS and obtain parity with respect to accuracy to address this potential AI bias in retinal diagnostics. | Gender, Age, Origin |
| 15 | Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases | Cau, R., Pisu, F., Suri, J. S., & Saba, L. (2024). Addressing hidden risks: Systematic review of artificial intelligence biases across racial and ethnic groups in cardiovascular diseases. European Journal of Radiology, 111867. https://doi.org/10.1016/j.ejrad.2024.111867 | Cardiovascular | Scholarly publication | Diagnosis, Prediction, Risk assesment | Representation bias, Evaluation bias | The review found that many AI models for cardiovascular diagnosis and prognosis show disparities in accuracy and clinical performance across racial and ethnic groups. It also showed that while AI has promise for clinical use, the presence of biases undermine its reliability in diverse populations. | Race: biases can lead to unequal diagnostic accuracy and poorer treatment recommendations for underrepresented groups, predominantly racial and ethnic groups, exacerbating health disparities. They undermine trust in AI-based medical tools and limit their safe adoption in real-world, hetergeneous clinical settings. |
| 16 | Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis | Cavagnolli, Gabriela, Ana Laura Pimentel, Priscila Aparecida Correa Freitas, Jorge Luiz Gross, and Joíza Lins Camargo. ‘Effect of Ethnicity on HbA1c Levels in Individuals without Diabetes: Systematic Review and Meta-Analysis’. PLOS ONE 12, no. 2 (2017): e0171315. https://doi.org/10.1371/journal.pone.0171315 | Diabetes | Scholarly publication | Diagnosis | Aggregation bias, Learning bias, Deployment bias | The study found that HbA1c levels are systematically higher among Black, Asian, and Latino individuals compared to White individuals without diabetes, indicating a potential risk of ethnicity-related misclassification and bias in diabetes diagnosis. | Race: Non Whites with no diabetes have higher values of HbA1c compared to Whites who have diabetes. Patients may be misdiagnosed depending on the way the data is collected, the training features are selected, or the system is deployed (e.g., trained with data from an ethnic group and applied to a different one). |
| 17 | Cedars-Sinai Study Shows Racial Bias in AI-Generated Treatment Regimens for Psychiatric Patients. Cedars-Sinai Study Shows Racial Bias in AI-Generated Treatment Regimens for Psychiatric Patients | Center, C.-S. M. (2025, June 30). Cedars-Sinai Study Shows Racial Bias in AI-Generated Treatment Regimens for Psychiatric Patients. Cedars-Sinai Study Shows Racial Bias in AI-Generated Treatment Regimens for Psychiatric Patients ; Cedars-Sinai Medical Center. https://www.cedars-sinai.org/newsroom/cedars-sinai-study-shows-racial-bias-in-ai-generated-treatment-regimens-for-psychiatric-patients/ | Depression | Webpage | Diagnosis, Treatment response | Representation bias, Learning bias, Evaluation bias, Deployment bias | The study used hypothetical psychiatric cases and ran them through multiple large language models. When the patient was characterized as African American, the recommended treatments differed from those given when race wasn't indicated. | Race: if AI systems systematically under-treat or mismanage psychiatric conditions in Black patients, this can worsen outcomes, reduce trust, and widen health disparities. |
| 18 | Will Artificial Intelligence and ChatGPT Replace the Clinical Doctor? Hellenic Journal of Cardiology | Chrissos, D. (2023). Will Artificial Intelligence and ChatGPT Replace the Clinical Doctor? Hellenic Journal of Cardiology / EKE Magazine, 64(4), 268-276. Athens: Hellenic Society of Cardiology. https://www.hcs.gr/wp-content/uploads/2024/03/11.-%CE%9C%CF%80%CE%BF%CF%81%CE%BF%CF%8D%CE%BD-%CE%B7-%CE%A4%CE%B5%CF%87%CE%BD%CE%B7%CF%84%CE%AE-%CE%9D%CE%BF%CE%B7%CE%BC%CE%BF%CF%83%CF%8D%CE%BD%CE%B7-%CE%BA%CE%B1%CE%B9-%CF%84%CE%BF-ChatGPT-%CE%BD%CE%B1-%CF%85%CF%80%CE%BF%CE%BA%CE%B1%CF%84%CE%B1%CF%83%CF%84%CE%AE%CF%83%CE%BF%CF%85%CE%BD-%CF%84%CE%BF%CE%BD-%CE%BA%CE%BB%CE%B9%CE%BD%CE%B9%CE%BA%CF%8C-%CE%B9%CE%B1%CF%84%CF%81%CF%8C-1.pdf | Cardiovascular | Scholarly publication | Management and planning | Deployment bias | Greek article in the Hellenic Journal of Cardiology discussing applications of AI and ChatGPT in medicine. It reviews benefits in diagnosis, patient monitoring, and medical education while acknowledging ethical concerns, bias, lack of transparency, and the need for empathy and critical judgment. The author concludes that AI cannot replace clinicians but can complement them if used responsibly. | All patient groups |
| 19 | Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare | Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., & Mavridis, N. (2020). Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. Npj Digital Medicine, 3(1). https://doi.org/10.1038/s41746-020-0288-5 | Not Sure/Other | Scholarly publication | Diagnosis, Risk assesment | Representation bias, Measurement bias, Aggregation bias, Learning bias, Evaluation bias | The review found that many biomedical AI technologies currently neglect sex and gender differences, which introduces undesirable biases and discriminatory outcomes when deployed in health setting. Moreover, it argues that failing to account for sex/gender dimensions limits the potential of AI to deliver personalised, equitable healthcare. | Gender: these biases risk producing suboptimal medical decisions or incorrect inferences for underrepresented sex or gender groups, thereby exacerbating health inequalities. They undermine trust in AI-based biomedical tools and impede teh realisation of fair and effective precision medicide. |
| 20 | Assessing racial bias in type 2 diabetes risk prediction algorithms | Cronjé HT, Katsiferis A, Elsenburg LK, Andersen TO, Rod NH, Nguyen T-L, et al. (2023) Assessing racial bias in type 2 diabetes risk prediction algorithms. PLOS Glob Public Health 3(5): e0001556. https://doi.org/10.1371/journal.pgph.0001556 | Diabetes | Repository | Prediction-based surveillance | Other | This study examined whether commonly used type 2 diabetes risk prediction models exhibit racial bias between non-Hispanic White and non-Hispanic Black adults. Using data from nearly 10,000 participants in the NHANES surveys (1999–2010), researchers evaluated three models: the Prediabetes Risk Test (PRT) from the National Diabetes Prevention Program, the Framingham Offspring Risk Score, and the ARIC Model. Predicted diabetes risks from each model were compared with observed risks from the U.S. Diabetes Surveillance System to assess calibration by race. All models were found to be racially miscalibrated. The Framingham model tended to overestimate diabetes risk for non-Hispanic Whites and underestimate it for non-Hispanic Blacks, while both the PRT and ARIC models overestimated risk for both groups, with a stronger overestimation among Whites. These calibration errors suggest that current diabetes risk models may favor non-Hispanic Whites in preventive care decisions—potentially leading to overdiagnosis and overtreatment in this group—while underestimating risk among Black individuals, who may then be underprioritized for prevention and treatment. The findings highlight the need to reassess and recalibrate diabetes prediction tools to ensure equitable and accurate risk assessment across racial groups. | Race |
| 21 | Bias in medical AI: Implications for clinical decision-making | Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024). Bias in medical AI: Implications for clinical decision-making. PLOS digital health, 3(11), e0000651. https://doi.org/10.1371/journal.pdig.0000651 | Not Sure/Other | Scholarly publication | Diagnosis | Measurement bias, Deployment bias | The paper analyzes multiple forms of bias that emerge throughout the AI lifecycle — including measurement bias (data capture errors), label bias (human-imposed misclassification), and deployment bias (mismatch between development and clinical use). It demonstrates how these biases collectively lead to unequal diagnostic performance and treatment outcomes. | Women, older adults, and ethnic minorities |
| 22 | Fairness-Aware Hyperparameter Optimization | Cruz, A. (2020). Fairness-Aware Hyperparameter Optimization. Repositorio-aberto.up.pt. https://repositorio-aberto.up.pt/handle/10216/128959 | Not Sure/Other | Scholarly publication | Surveillance, Other | Historical Bias, Learning bias, Deployment bias | Master's dissertation focused on studying biases and potential discrimination in the use of artificial intelligence systems related to gender, age, ethnicity, or geographic location. Current bias mitigation approaches are often model-dependent, metric-dependent, and invariably introduce complexity to real-world machine learning pipelines. In this work, the author e proposes Fairband, a method for fairness-aware hyperparameter optimization. They prove that it is both possible and effective to navigate the fairness-utility tradeoff by acting solely on the hyperparameters, enable finding fairness-aware models without any oversight, as well as targeting specific fairness-utility tradeoffs. | Gender, Age, Ethnicity |
| 23 | Digital health tools for the passive monitoring of depression: a systematic review of methods | De Angel V., Lewis S., White K., Oetzmann C., Leightley D., Oprea E., Lavelle G., Matcham F., Pace A., Mohr D., Dobson R., Hotopf M. (2022). Digital health tools for the passive monitoring of depression: a systematic review of methods. DOI: 10.1038/s41746-021-00548-8 | Depression | Scholarly publication | Diagnosis | Representation bias, Measurement bias, Aggregation bias, Evaluation bias | This systematic review examines studies linking passive data from smartphones and wearables to depression, identifying key methodological flaws and threats to reproducibility. It highlights biases such as representation, measurement, and evaluation bias, stemming from small, homogenous samples and inconsistent feature construction. Although gender and race are not explicitly discussed, the lack of diversity in study populations suggests potential demographic bias. The review calls for improved reporting standards and broader sample inclusion to enhance generalizability and clinical relevance. These improvements are essential for ensuring that digital mental health tools are equitable and reliable across diverse populations. | Demographic bias |
| 24 | Deep Learning Discovery of Demographic Biomarkers in Echocardiography | Duffy, G., Clarke, S.L., Christensen, M., He, B., Yuan, N., Cheng, S., & Ouyang, D. (2022). Deep Learning Discovery of Demographic Biomarkers in Echocardiography https://arxiv.org/abs/2207.06421 | Cardiovascular | Scholarly publication | Diagnosis, Imaging | Representation bias | Using >530,000 echocardiogram videos from Cedars-Sinai and Stanford, deep learning models were trained to predict age, sex, and race. The models could predict age and sex accurately but struggled to generalize race, with race prediction highly influenced by confounding features such as sex. This shows that apparent “race detection” by AI often reflects dataset bias rather than biological difference. | Racially diverse patient populations (risk of misinterpretation and biased modeling) |
| 25 | Artificial intelligence in healthcare: Applications, risks, and ethical and societal impacts | EPRS | European Parliamentary Research Service, Scientific Foresight Unit (STOA), PE 729.512 (June 2022). Artificial intelligence in healthcare: Applications, risks, and ethical and societal impacts, https://www.europarl.europa.eu/thinktank/en/document/EPRS_STU%282022%29729512#:~:text=4,and%20ethical%20and%20societal%20impacts | Not Sure/Other | Stakeholder Group | Public health | Representation bias, Measurement bias, Learning bias | The EU report identifies and clarifies the main clinical, social and ethical risks posed by AI in healthcare, more specifically: potential errors and patient harm; risk of bias and increased health inequalities; lack of transparency and trust; and vulnerability to hacking and data privacy breaches. | "Gender and Race: in the United States, existing research has demonstrated that doctors do not take Black patients' complaints of pain as seriously nor do they respond to them as quickly as they do for their White counterparts (Hoffman et al., 2016). Persistent in most countries around the world, to varying degrees, is yet another example of common bias embedded in healthcare systems: gender-based discrimination. Once again, in the domain of pain management, studies have pointed to the increased psychologisation or invisibilisation of female patients when reporting pain (Samulowitz et al., 2018). / A study published in Science in 2019 showed that an algorithm used in the United States to help in the referral process of patients who need extra or specialist care was shown to discriminate against Black patients (Obermeyer et al., 2019). The authors of the study explained that with the algorithm, 'at a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%'. A Canadian study in 2020 evaluated the degree of fairness of state-of-the-art deep learning algorithms used to detect abnormalities such as fractures, lung lesions, nodules, pneumonia, etc. in chest X-ray images (Seyyed-Kalantari et al., 2020). The study showed that the highest rate of underdiagnosis was in young females (age: 0-20), in Black patients, and in patients on public health insurance for low-income people and households. Furthermore, patients with intersectional identities (for example, a Hispanic female patient on low- Artificial intelligence in healthcare 21 income health insurance) suffered the highest rates of underdiagnosis. The authors concluded that 'models trained on large datasets do not provide equality of opportunity naturally, leading instead to potential disparities in care if deployed without modification' (Seyyed-Kalantari et al., 2020). " |
| 26 | Minding the Gaps: Neuroethics, AI, and Depression. Non- profit Quarterly | Gemma Boothroyd. Minding the Gaps: Neuroethics, AI, and Depression. Non- profit Quarterly, March 24, 2025. https://nonprofitquarterly.org/minding-the-gaps-neuroethics-ai-and-depression/ | Depression | Webpage | Customized therapy, Diagnosis, Disease prevention, Treatment response | Historical Bias, Learning bias | In this post, the author highlights the benefits and potential issues regarding the use of AI in depression diagnosis/treatment, focusing on the prevalent gender, racial and ethnicity biases. She mentions that, given the historical, inherent biases in society generally and healthcare specifically, AI-driven advancements are not going to serve minority groups as a matter of course. Unless they are tailored to represent and serve all communities equally, they will exacerbate existing biases and disparities. | Gender; Race and ethnicity |
| 27 | The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media | Gierschmann, L. (2024). The Role of Gender: Gender Fairness in the Detection of Depression Symptoms on Social Media. Studenttheses.uu.nl. https://studenttheses.uu.nl/handle/20.500.12932/47734 | Depression | Scholarly publication | Diagnosis, Prediction, Risk assesment | Representation bias, Evaluation bias | The study found that the BDI-Sen dataset used for depression symptom detection on social media exhibits gender bias, with machine learning models such as mentalBERT showing predictive disparities that generally favour male users. Although bias mitigation techniques like data augmentation reduced the bias, they did not eliminate it completely. | Gender: the gender bias affects the fairness and reliability of AI systems in detecting depression symptoms, leading to unequal predictive performance across genders. This can result in under- or over-identification of depression symptoms in certain groups, thereby compromising the validity of such systems for clinical or mental health monitoring. |
| 28 | Gender Bias in Diagnosis, Prevention, and Treatment of Cardiovascular Diseases: A Systematic Review | Hamid A., Beckett R., Wilson M., Jalal Z., Cheema E., Al-Jumeily D., Coombs Th., Ralebitso-Senior K., Assi S. (2024). Gender Bias in Diagnosis, Prevention, and Treatment of Cardiovascular Diseases: A Systematic Review. https://assets.cureus.com/uploads/review_article/pdf/219684/20240318-4800-1qz1ppt.pdf | Cardiovascular | Scholarly publication | Customized therapy, Diagnosis, Disease prevention, Risk assesment, Treatment response | Historical Bias, Representation bias, Measurement bias, Aggregation bias | This systematic review investigates gender bias in the diagnosis, prevention, and treatment of cardiovascular diseases (CVDs). It finds that women are often underdiagnosed due to milder or misinterpreted symptoms and are less likely to be referred for diagnostic tests or specialist care compared to men. Women also receive fewer cardiovascular medications, except for antihypertensives and anti-anginals, and perceive themselves at lower risk of CVD. These disparities are linked to inadequate awareness among healthcare professionals about gender-specific manifestations of CVD. The study highlights the need for improved clinical practices and education to address gender bias and improve outcomes for women. | Gender |
| 29 | AI and Mental Healthcare – ethical and regulatory considerations | Hannah Gardiner, Natasha Mutebi (31 January 2025). AI and Mental Healthcare – ethical and regulatory considerations. https://post.parliament.uk/research-briefings/post-pn-0738/ | Depression | Stakeholder Group | Management and planning | Representation bias | The report discusses the ethical and regulatory considerations of using artificial intelligence in mental healthcare in the UK. | Racial / Cultural: Bias in AI tools (algorithmic bias) can stem from various places including AI tools being trained on biased datasets and outputting discriminatory outcomes (PN637),99,143,147–152 or developers making biased decisions in the design or training of AI tools.150,153. For example, mental health Electronic health record (EHR) data is susceptible to cohort and label bias.154 This can occur because culture-bound presentations of mental disorders, combined with a lack of transcultural literacy among clinicians, often lead to both over- and under-diagnosis.155. People can also exhibit bias when using AI tools, such as over-relying on55,143,156,157 or mistrusting55 AI outputs (Table.4). All these biases can be conscious or unconscious.k |
| 30 | For fair and equal healthcare, we need fair and bias-free AI | Henk van Houten / Former Chief Technology Officer at Royal Philips (2020), For fair and equal healthcare, we need fair and bias-free AI, https://www.philips.com/a-w/about/news/archive/blogs/innovation-matters/2020/20201116-for-fair-and-equal-healthcare-we-need-fair-and-bias-free-ai.html | Not Sure/Other | Webpage | Public health | Other | An article that describes how although Artificial intelligence (AI) has the potential to make healthcare more accessible, affordable, and effective, it can also inadvertently lead to erroneous conclusions and thereby amplify existing inequalities. Mitigating these risks requires awareness of the bias that can creep into AI algorithms – and how to prevent it through careful design and implementation. | General Impact |
| 31 | Predictive Accuracy of Stroke Risk Prediction Models Across Black and White Race, Sex, and Age Groups | Hong, C., Pencina, M. J., Wojdyla, D. M., Hall, J. L., Judd, S. E., Cary, M., Engelhard, M. M., Berchuck, S., Xian, Y., D’Agostino, R., Howard, G., Kissela, B., & Henao, R. (2023). Predictive Accuracy of Stroke Risk Prediction Models Across Black and White Race, Sex, and Age Groups. JAMA, 329(4), 306. https://doi.org/10.1001/jama.2022.24683 | Cardiovascular | Scholarly publication | Disease prevention, Prediction, Risk assesment | Learning bias, Evaluation bias | The study aims "to compare the performance of stroke-specific algorithm with pooled cohort equations developped for atherosclerotic cardiovascular disease for the prediction of new-onset stroke across different subgroups (race, sex, and age) and to determine the added value of novel machine learning techniques" (p. 306) The research aims to estimate the risk of new-onset stroke in the following 10 years. | Findings show that the ability to rank the risk was weaker for Black individuals then for White individuals for men and women. It means that the agreement between predicted and observed risk is poor for this subgroup.This risk was even more weaken for Black individuals older than 60 years old.The risk is for higher-risk individual to lack appropriate therapy and for the lower-risk individuals to be overtreated. The research shows that applying novel machine learning algorithm did not improve the performance and the discriminative accuracy. |
| 32 | Digital Mental Health Technology - Regulation and Evaluation for Safe and Effective Products | https://assets.publishing.service.gov.uk/media/6866572fadfe29730ea3a9d5/MHRA_guidance_on_DMHT_-_Device_characterisation_regulatory_qualification_and_classification.pdf | Not Sure/Other | Other | Other | Other | Potentially relevant, to read | - |
| 33 | Inteligência artificial: uma revolução ao serviço da saúde | https://eco.sapo.pt/2024/09/18/inteligencia-artificial-uma-revolucao-ao-servico-da-saude/ | Not Sure/Other | Webpage | Diagnosis, Disease prevention, Health promotion, Surveillance | Historical Bias, Aggregation bias, Learning bias | Portuguese media article about the debated topics of the Bio-Med AI Summer School, 2024 in Lisbon. This conference featured renowned speakers from the Portuguese healthcare, law, technology and ethics sectors and they talked about the role of artificial intelligence tools in healthcare, from the latest innovations to applications in scientific research and clinical practice, to the AI act. They also mentioned one case of an LLM tool used for mental health assessment (MentaLLaMA). | Gender; race |
| 34 | Gender Equality and Artificial Intelligence: Navigating the EU Policy Frameworks for a Feminist Future | https://genderfiveplus.org/shaping-a-feminist-future-for-ai-gender-five-plus-explores-risks-opportunities-and-ethical-pathways-in-the-eu/ | Not Sure/Other | Other | Other | Other | This paper explores the gender implications of AI, revealing both significant risks of perpetuating discriminatory practices and substantial opportunities for advancing gender equality. The scope of the research is limited to AI applications governed by European legislation, and it finds that while the regulatory efforts of the EU represent a significant commitment, challenges remain. | Gender |
| 35 | 1557 Final Rule Protects Against Bias in Health Care Algorithms | https://healthlaw.org/1557-final-rule-protects-against-bias-in-health-care-algorithms/ | Not Sure/Other | Legal case | Customized therapy, Treatment response | Evaluation bias, Deployment bias | The department of Health and Human Services (HHS) has provided a final rule regarding the section 1557 of the Affordable Care Act (ACA). This section concerns Nondiscrimination in Health Programs and Activites. The last rule implemented concerns the protection against the use of some algorithms in health care. Algorithms used in Medicaid and health care are know as source of bias, discriminatory and mistakely deny care and benefits needed. Therefore, the final rule prohibits discriminatory for patient decision care supported by tool. | The final rule requires from covered entities using these tools to put "reasonable efforts to mitigate the risk of discrimination". Challenges remain in term of implementation due to the undefined terms and many tools used in heatlh care will not fall under the definition of "patient care decision support" but efforts and further actions are needed as algorithms continue to evolve. |
| 36 | FDA Proposes Updated Recommendations to Help Improve Performance of Pulse Oximeters Across Skin Tones | https://www.fda.gov/news-events/press-announcements/fda-proposes-updated-recommendations-help-improve-performance-pulse-oximeters-across-skin-tones | Not Sure/Other | Other | Disease prevention, Public health, Surveillance | Representation bias, Aggregation bias, Evaluation bias | The Food and Drug Administration proposes recommendations to improve accuracy of pulse oximeters across different skin tones. Studies show pulse oximeters may perform differently between individuals with lighter and darker skin tone. | The key elements of the FDA's draft recommendations are the following ones ; collecting clinical data to assess device accuracy across different skin pigmentations ; more clinical study participants ; using objective and subjective methods for more accuracy of the device ; if the performance is accurate, it is essential to provide appropriate labelling in order to assist patients in identifying reliable options. |
| 37 | Equity in medical devices: independent review - summary report | https://www.gov.uk/government/publications/equity-in-medical-devices-independent-review-final-report/equity-in-medical-devices-independent-review-summary-report | Not Sure/Other | Other | Health promotion, Public health | Representation bias, Measurement bias, Learning bias, Evaluation bias | The Department of Health and Social of the UK has provied a report of equity in medical devices.The use of AI can introduce bias, particularly related to gender, race, and individuals from disadvantaged socio-economic backgrounds. One of the natural bias is unrepresentative datasets in AI alogirthms. Moreover, the review shows that bias in AI can come from ; how health problems are selected and prioritised ; how data is selected and used ; how outcomes in healthcare are defining and ranking ; how the algorithms are developed and tested ; how the impacts of the device are managed. Then, authors' recommendation are based on group session with patients, national leaders of health professions, regulators and developers of medical devices. | The author advocates for the creation of a government taskforce to provide oversight for Large Language Models (LLMs), in order to assess health equity in the application of med. They are making the following recommendations ; 8) NHS organisations and the public should co-design process for AI using devices to ensure equity, fariness and transparency and mitigate discrimination. 9) government should support commission to promote better understanding of equity in AI-assisted medical devices. 10) Individuals using AI should be transpartent of data used during the process. 11) "Stakeholders accross the device lifecycle should work together to ensure" best practice to reduce bias. 13) NHS should ensure "deployment of equitable AI-enabled medical devices in the health service" 14) AI-related research should prioritise diversity and inclusion 15) regulators should have the necessary ressources to handle the process of AI and manage its potential impact on equity. Moreover, authors suggest to improve equity in datasets as minorities as underrepresented. |
| 38 | The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review,Ethnicity & Health | Hussain,S.A, Bresnahan,M. & Zhuang,J. (03 Nov 2024): The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review,Ethnicity & Health, https://doi.org/10.1080/13557858.2024.2422848 | Not Sure/Other | Scholarly publication | Public health, Other | Measurement bias | This scoping review examined racial and ethnic bias in artificial intelligence health algorithms (AIHA), the role of stakeholders in oversight, and the consequences of AIHA for health equity. | |
| 39 | A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection | Ive, J., Bondaronek, P., Yadav, V., Santel, D., Glauser, T., Cheng, T., Strawn, J. R., Agasthya, G., Tschida, J., Choo, S., Chandrashekar, M., Kapadia, A. J., & Pestian, J. (2024). A Data-Centric Approach to Detecting and Mitigating Demographic Bias in Pediatric Mental Health Text: A Case Study in Anxiety Detection. ArXiv.org. https://arxiv.org/abs/2501.00129 | Depression | Scholarly publication | Diagnosis, Prediction, Risk assesment | Representation bias, Measurement bias, Learning bias, Evaluation bias | The study examines classication parity across sex and finds that female adolescents are systematically under-diagnosed mental health disorders: their model’s accuracy was ~4 % lower and false negative rate ~9 % higher compared to male patients. The source of the bias resides in the textual data, namely notes corresponding to male patients tended to be on average 500 words longer and had distinct word usage. To mitigate this, the authors introduce a de-biasing method, based on neutralizing biased terms (gendered words and pronouns) and reducing sentences to essential clinical information. After correcting, diagnostic bias is reduced by up to 27%. | Gender: Linguistically transmitted bias—ensuing from word choice and gendered language—consistently leads to the under-diagnosis of mental health disorders among female adolescents. This bias critically undermines the impartiality of medical diagnosis and treatment. |
| 40 | Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions, Computers in Human Behavior: Artificial Humans | Jerlyn Q.H. Ho, Andree Hartanto, Andrew Koh, Nadyanna M. Majeed, Gender biases within Artificial Intelligence and ChatGPT: Evidence, Sources of Biases and Solutions, Computers in Human Behavior: Artificial Humans, Volume 4, 2025, 100145, ISSN 2949-8821, https://www.sciencedirect.com/science/article/pii/S2949882125000295?via%3Dihub | Not Sure/Other | Webpage | Other | Other | The document explores gender biases in Artificial Intelligence (AI) systems, particularly focusing on ChatGPT and other generative AI models. It highlights how AI systems can perpetuate societal prejudices, leading to unequal treatment in areas like hiring, education, and healthcare. Key sources of bias include: 1) Biased Training Data: AI models often learn from datasets that reflect societal biases, such as gender stereotypes, underrepresentation of marginalized groups, and historical biases. 2) Chatbot Architecture: Biases can be embedded in the design and development of AI systems, including word embedding, aggregation bias, evaluation bias, algorithmic bias, and overfitting. 3) User Feedback Loop: Interactions with users can reinforce biases, such as presentation, ranking, and popularity biases, as well as negative feedback loops. | Gender |
| 41 | Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches | Junias, O., Kini, P., & Chaspari, T. (2025). Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches. ArXiv.org. https://arxiv.org/abs/2509.25795 | Depression | Scholarly publication | Diagnosis | Representation bias, Learning bias | The study found that large language models (LLMs) outperform traditional deep neural network (DNN) embeddings in automated depression detection and show reduced gender bias, through ravial disparities remain. Among DNN fairness-mitigation techniques, the worst-group loss provided the best balance between overall accuracy and demographic fairness, while fairness-regularised loss underperformed. | Race: The identified biases affect the fairness and diagnostic reliability of AI systems for mental health assessment, particularly by disadvantaging underrepresented racial and gender groups, in this research mainly Hispanic participants. Such disparities risk perpetuating inequities in automated mental health screening and could undermine trust and validity in clinical or public health applications. |
| 42 | Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning | Kamulegeya, L. H., Okello, M., Bwanika, J. M., Musinguzi, D., Lubega, W., Rusoke, D., Nassiwa, F., & Börve, A. (2019). Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning (p. 826057). bioRxiv. https://doi.org/10.1101/826057 | Not Sure/Other | Scholarly publication | Diagnosis, Disease prevention, Prediction, Risk assesment, Treatment response | Representation bias | The article presents findings from beta-testing an AI-powered dermatological algorithm called Skin Image Search, by online dermatology company First Derm on Fitzpatrick 6 skin type (dark skin) dermatological conditions. | Race |
| 43 | Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change? | Katie Palmer & Usha Lee McFarling, (2024), Doctors use problematic race-based algorithms to guide care every day. Why are they so hard to change?, https://www.statnews.com/2024/09/03/embedded-bias-investigation-health-equity-clinical-algorithms/ | Not Sure/Other | Webpage | Diagnosis | Representation bias | For decades, U.S. hospitals and medical societies have relied on race-based clinical algorithms — tools that guide doctors’ decisions by assigning risk points based partly on a patient’s race. Following the George Floyd protests in 2020, the medical community faced growing pressure to eliminate racial bias in clinical practice. Studies revealed that race-adjusted calculators — used in kidney function, lung capacity, and UTI risk assessments — often underdiagnosed and harmed patients of color, especially Black girls. Despite early reforms by some hospitals and medical organizations, many race-based algorithms remain in use nationwide. Opponents of removing race entirely argue that it can still provide valuable predictive information — for example, in breast cancer or surgical risk calculators — though critics say this perpetuates the false idea that race is a biological category rather than a social construct. | Race |
| 44 | Race, Sex, and Age Disparities in the Performance of ECG Deep Learning Models Predicting Heart Failure. Circulation: Heart Failure | Kaur, D., Hughes, J. W., Rogers, A. J., Kang, G., Narayan, S. M., Ashley, E. A., & Perez, M. v. (2024). Race, Sex, and Age Disparities in the Performance of ECG Deep Learning Models Predicting Heart Failure. Circulation: Heart Failure, 17(1). https://doi.org/10.1161/CIRCHEARTFAILURE.123.010879 | Cardiovascular | Scholarly publication | Disease prevention, Prediction, Risk assesment | Learning bias, Evaluation bias | The study conducted an analyzis of intersectional disparities in terms of race, sex, and age in the performance of ECG deep learning models to predict heart failure. The primary model used for the study was trained the incidence of heart failure, from 12 ECG data, for the next 5 years. | The results show that the model performed significantly worse for Black patients, below 40 years old compared to other racial groups of this age; and there was more differences with young Black women. It also perfomed worse for older patients. The model performed worse as it indicates a greater underdiagnosis for this subgroups."This study serves to highlight the need to consider the differential performance of machine learning models among demographic subrgroups" (p. 22)Identifying intersectional biases is necessary to improve algorithms and thus limiting existing disparities in health outcome. |
| 45 | Domain Adversarial Training for Mitigating Gender Bias in Speech-based Mental Health Detection | Kim, J.-W., Yoon, H., Oh, W., Jung, D., Yoon, S.-H., Kim, D.-J., Lee, D.-H., Lee, S.-Y., & Yang, C.-M. (2025). Domain Adversarial Training for Mitigating Gender Bias in Speech-based Mental Health Detection. https://arxiv.org/abs/2505.03359 | Depression | Scholarly publication | Diagnosis | Representation bias | Developed a domain adversarial training (DAT) method to reduce gender bias in AI models for depression and PTSD detection using speech data (E-DAIC dataset). DAT improved F1-scores up to +13% and reduced gender gaps in detection accuracy, improving generalization across male and female participants. | Male and female patients, especially female underrepresentation effects |
| 46 | Novel artificial intelligence applications in cardiology: Current landscape, limitations, and the road to real-world applications | Langlais, É. L., Dupont, J., Martin, A., & Nguyen, P. (2022). Novel artificial intelligence applications in cardiology: Current landscape, limitations, and the road to real-world applications. Journal of Cardiovascular Translational Research, 16(3), 513–525. https://doi.org/10.1007/s12265-022-10260-x | Cardiovascular | Scholarly publication | Customized therapy, Disease self-management, Prediction, Other | Representation bias, Evaluation bias | This article reviews recent advances in artificial intelligence (AI) applications in cardiology, focusing on supervised and unsupervised learning methods for diagnosis, risk prediction, and personalized care. It highlights current uses in ECG, imaging, mobile health, and medical notes, discusses limitations such as bias and data access, and outlines strategies for integrating AI into clinical practice. The review also addresses ethical and practical challenges, emphasizing the need for transparency, standardization, and collaboration to fully realize AI's potential in cardiovascular medicine. | Sex, gender, race |
| 47 | A Systematic Study of Race and Sex Bias in CNN-Based Cardiac MR Segmentation | Lee, T., Puyol-Antón, E., Bram Ruijsink, Shi, M., & King, A. P. (2022). A Systematic Study of Race and Sex Bias in CNN-Based Cardiac MR Segmentation. Lecture Notes in Computer Science, 233–244. https://doi.org/10.1007/978-3-031-23443-9_22 | Cardiovascular | Scholarly publication | Imaging | Representation bias, Learning bias | The authors found that CNN-based segmentation models exhibit performance disparities across racial and sex groups when trained on imbalanced datasets, with underrepresented groups receiving lower segmentation accuracy. They systematically varied the class (sex/race) imbalance and showed that increasing imbalance worsens bias, while more balanced training mitigates but does not eliminate the disparity. | These biases lead to worse anatomical delineation and measurement accuracy in underrepresented demographic groups, which can reduce clinical reliability and lead to unfair outcomes in medical imaging applications. The unequal performance risks reinforcing systemic inequities in health care when AI tools are deployed broadly, undermining trust and equitable access to diagnostic assistance. |
| 48 | Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction | Li, F., Wu, P., Ong, H. H., Peterson, J. F., Wei, W.-Q., & Zhao, J. (2023). Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction. Journal of Biomedical Informatics, 138, 104294. https://doi.org/10.1016/j.jbi.2023.104294 | Cardiovascular | Scholarly publication | Disease prevention, Prediction, Risk assesment | Representation bias, Learning bias, Evaluation bias | The study aims to evalute wheter machine learning-based predictive models for CVD risk produce same predictions across race and grender. Moreover, it also aims to evaluate if bias mitigation methods can reduce bias in this model. | The study highlights that gender bias was more important then race bias; there was a lower positive prediction rate in female group then in male group. Moreover, the research evaluates the effect of bias mitigation approach and found that resampling (2 ways ; 1) to have a balance sample size and 2) to have same classe distribution across the group) was more effective for reducing bias for gender groups than for racial groups only by case proportion. There was an imbalanced of proportion of CVD diagnoses for the women, which was lower than for men. This research emphazises the importance for these models to give fair and accurate predictions ; also the importance on collecting and preprocessing health data which could affect the performance of the date. |
| 49 | An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs | Li, Z., Keel, S., Liu, C., He, Y., Meng, W., & Scheetz, J. (2018). An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care, 41(12), 2509–2516. https://doi.org/10.2337/dc18-0147 | Diabetes | Scholarly publication | Diagnosis | Representation bias, Measurement bias, Evaluation bias | The study trained its AI model for diabetic retinopathy detection entirely on retinal images from Chinese patients and validated it on smaller, uneven samples of Malays, Caucasian Australians, and Indigenous Australians. Differences in pigmentation and imaging protocols across these groups, combined with limited subgroup data, introduce potential racial bias. Despite claiming multiethnic generalizability, the evidence for equal performance across populations is weak. | Race |
| 50 | Gender Bias in AI's Perception of Cardiovascular Risk | Margaux Achtari, Adil Salihu, Olivier Muller, Emmanuel Abbé, Carole Clair, Joëlle Schwarz , Stephane Fournier, J Med Internet Res. (2024), Gender Bias in AI's Perception of Cardiovascular Risk, https://pmc.ncbi.nlm.nih.gov/articles/PMC11538872/ | Cardiovascular | Other | Disease prevention | Other | The study investigated gender bias in GPT-4's assessment of coronary artery disease risk by presenting identical clinical vignettes of men and women with and without psychiatric comorbidities. Results suggest that psychiatric conditions may influence GPT-4's coronary artery disease risk assessment among men and women. | Gender bias affecting medical decision-making |
| 51 | Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care | Marshall H. Chin, MD, MPH1; Nasim Afsar-Manesh, MD, MBA, MHM2; Arlene S. Bierman, MD, MS3 et al. (2023). Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care. https://doi:10.1001/jamanetworkopen.2023.45050 | Not Sure/Other | Scholarly publication | Diagnosis, Public health | Representation bias, Measurement bias | This article describes guiding principles for health care algorithms and key operational considerations. Conscious decisions by algorithm developers, algorithm users, health care industry leaders, and regulators can mitigate and prevent bias and proactively advance health equity. It emphasizes the need to address algorithmic bias in healthcare AI systems, particularly those used for diagnosis, treatment, and resource allocation. They highlight how biased algorithm can lead to inequitable outcomes, especially for Black patients. A lack of demographic representation in training data, including race, gender, and geography, undermines model fairness and generalizability. The proposed framework calls for transparency, explainability, and authentic community engagement throughout the algorithm life cycle. It also stresses the importance of monitoring performance across diverse populations and balancing fairness with technical performance. | Race |
| 52 | Research Frameworks towards Health Equity | Matos, J. (2023). Research Frameworks towards Health Equity. Repositorio-aberto.up.pt. https://repositorio-aberto.up.pt/handle/10216/150994 | Not Sure/Other | Scholarly publication | Customized therapy, Treatment response | Representation bias, Measurement bias, Evaluation bias | This thesis explores the need to prioritize equity concepts in health research, analyzing the potential biases of AI in healthcare, with a particular focus on racial and ethnic disparities. This work also addresses the need to create decision support systems that are safe, reliable, and improve clinical practice for all patients. | Race and Ethnicity |
| 53 | Artificial intelligence bias in the prediction and detection of cardiovascular disease | Mihan, A., Pandey, A. & Van Spall, H.G.C. Artificial intelligence bias in the prediction and detection of cardiovascular disease. npj Cardiovasc Health 1, 31 (2024). https://doi.org/10.1038/s44325-024-00031-9 | Cardiovascular | Webpage | Other | Other | The scientific report discusses the potential of artificial intelligence (AI) in identifying individuals at risk of cardiovascular disease (CVD) for early intervention. However, it highlights the issue of AI bias, which can arise during the development, validation, and implementation of algorithms. Such biases can lead to poor performance in marginalized groups, exacerbating healthcare inequities based on age, sex, race, ethnicity, and socioeconomic status. | Marginalized groups, exacerbating healthcare inequities based on age, sex, race, ethnicity, and socioeconomic status. |
| 54 | Mitigating the risk of artificial intelligence bias in cardiovascular care | Mihan, A., Pandey, A., & Van Spall, H. G. (2024). Mitigating the risk of artificial intelligence bias in cardiovascular care. The Lancet. Digital health, 6(10), e749–e754. https://doi.org/10.1016/S2589-7500(24)00155-9 | Cardiovascular | Scholarly publication | Diagnosis, Imaging | Evaluation bias, Deployment bias | The article explores how AI models for cardiovascular imaging and diagnosis may underperform when deployed in clinical environments that differ from their training context. It stresses the importance of external validation and continuous monitoring to prevent bias at deployment. | Cardiac patients in hospitals with differing demographics or imaging protocols. |
| 55 | Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images | Miller, R. J., Zhang, Y., Ahmed, H., Lee, T., & Patel, S. (2022). Mitigating bias in deep learning for diagnosis of coronary artery disease from myocardial perfusion SPECT images. European Journal of Nuclear Medicine and Molecular Imaging, 50(2), 387–397. https://doi.org/10.1007/s00259-022-05972-w | Cardiovascular | Scholarly publication | Diagnosis | Representation bias | This study investigates how representation/selection bias in training data affects the performance of deep learning models used to diagnose coronary artery disease (CAD) from myocardial perfusion SPECT scans. The researchers found that models trained mainly on high-risk patients (those referred for invasive angiography) tended to overestimate CAD probability, particularly in low-risk and female patients. To address this, they tested data augmentation strategies that added more normal or low-risk cases to the training set. The augmented model achieved better calibration and fairness, reducing overestimation bias and improving diagnostic accuracy across patient groups. The paper emphasizes the need for balanced, representative datasets to ensure equitable AI performance in clinical cardiology. | Sex |
| 56 | Fairness And Bias Mitigation in AI Models for Diabetes Diagnosis: A Comparative Evaluation of Algorithmic Approaches | Mohd Hamdan, M. D. H., Ab Jabal, M. F., Abdul Rahman, S., & Kapi, A. Y. (2025). Fairness And Bias Mitigation in AI Models for Diabetes Diagnosis: A Comparative Evaluation of Algorithmic Approaches. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 17(3), 27–34. https://doi.org/10.54554/jtec.2025.17.03.004 | Diabetes | Scholarly publication | Diagnosis, Risk assesment | Representation bias, Measurement bias, Learning bias, Evaluation bias | The study compares three fairness-aware strategies to allegedly reduce to reduce algorithmic bias in AI model. They authors find that none of the methods individually achieves balance between accuracy, fairness, and interpretability. They suggest a hybrid approach combining multiple mitigation strategies to control bias and achieve balanced outcomes at the same time. | Race & gender: both race and gender biases can undermine the reliability and fairness of AI models. The three mitigation strategies presented strive to reduce these discrepancies, but the fact that no single strategy seems sufficient suggests that bias may still linger in modern medical research. |
| 57 | Trustworthy and ethical AI-enabled cardiovascular care: A rapid review | Mooghali, M., Stroud, A. M., Yoo, D. W., & Van Spall, H. G. C. (2024). Trustworthy and ethical AI-enabled cardiovascular care: A rapid review. BMC Medical Informatics and Decision Making, 24, 247. https://doi.org/10.1186/s12911-024-02653-6 | Cardiovascular | Scholarly publication | Diagnosis, Prediction, Surveillance, Other | Representation bias | The article examines how AI is being applied in cardiovascular medicine for diagnosis, prediction, monitoring, and treatment. It highlights that while AI holds great promise for improving cardiac care, it also raises major ethical and fairness concerns. The authors identify key issues such as representation and equity bias, where unbalanced datasets lead to poorer accuracy for certain racial, gender, or socioeconomic groups, as well as risks around privacy, accountability, informed consent, and patient harm. These biases can reinforce existing health disparities and reduce trust among patients and clinicians. The study calls for stronger regulatory oversight, the inclusion of diverse populations in AI development, and transparent, explainable systems to ensure that cardiovascular AI tools are used responsibly and equitably. | Race, sex, gender |
| 58 | Bias discovery in machine learning models for mental health | Mosteiro, P., Prieto, A., Solares, C., García, M., & Fernández, Á. (2022). Bias discovery in machine learning models for mental health. Information, 13(5), 237. https://doi.org/10.3390/info13050237 | Depression | Scholarly publication | Prediction | Representation bias, Evaluation bias | The article examines how AI can unintentionally reproduce social and demographic biases when applied to mental health prediction. Using benzodiazepine prescriptions as a proxy for conditions such as depression and anxiety, the study analyzes machine learning models trained on patient data to identify systematic disparities. It finds that women are more frequently predicted to receive such treatments, reflecting gender bias, while the models perform less accurately for minority ethnic groups, indicating representation and evaluation bias. The AI models here are not used to prescribe drugs but rather to predict treatment likelihoods, revealing how bias in healthcare data can lead to inequitable AI performance in the context of depression-related care. | Gender |
| 59 | Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases | Muzammil, M.A, Javid, S, Afridi, A.K., Siddineni, R., Shahabi, M., Haseeb, M., Fariha, F.N.U., Kumar, S., Zaveri, S., Nashwan, A.J. Artificial intelligence-enhanced electrocardiography for accurate diagnosis and management of cardiovascular diseases. Journal of Electrocardiology Volume 83, March–April 2024, Pages 30-40. https://doi.org/10.1016/j.jelectrocard.2024.01.006 | Cardiovascular | Scholarly publication | Diagnosis, Disease prevention, Prediction, Public health | Historical Bias, Representation bias, Measurement bias, Deployment bias | Electrocardiography (ECG), improved by artificial intelligence (AI), has become a potential technique for the precise diagnosis and treatment of cardiovascular disorders. The use of AI in cardiology, however, has several limitations and obstacles, despite its potential. The effective implementation of AI-powered ECG analysis is limited by issues such as systematic bias. Biases based on age, gender, and race result from unbalanced datasets. A model's performance is impacted when diverse demographics are inadequately represented. Potentially disregarded age-related ECG variations may result from skewed age data in training sets. ECG patterns are affected by physiological differences between the sexes; a dataset that is inclined toward one sex may compromise the accuracy of the others. Genetic variations influence ECG readings, so racial diversity in datasets is significant. Furthermore, issues such as inadequate generalization, regulatory barriers, and interpretability concerns contribute to deployment difficulties. The lack of robustness in models when applied to disparate populations frequently hinders their practical applicability. The exhaustive validation required by regulatory requirements causes a delay in deployment. Difficult models that are not interpretable erode the confidence of clinicians. Diverse dataset curation, bias mitigation strategies, continuous validation across populations, and collaborative efforts for regulatory approval are essential for the successful deployment of AI ECG in clinical settings and must be undertaken to address these issues. To guarantee a safe and successful deployment in clinical practice, the use of AI in cardiology must be done with a thorough understanding of the algorithms and their limits. In summary, AI-enhanced electrocardiography has enormous potential to improve the management of cardiovascular illness by delivering precise and timely diagnostic insights, aiding clinicians, and enhancing patient outcomes. Further study and development are required to fully realize AI's promise for improving cardiology practices and patient care as technology continues to advance. | Gender, Age |
| 60 | AI fails to detect depression signs in social media posts by Black Americans, study finds | Nancy Lapid, (March 28, 2024). AI fails to detect depression signs in social media posts by Black Americans, study finds. https://www.reuters.com/business/healthcare-pharmaceuticals/ai-fails-detect-depression-signs-social-media-posts-by-black-americans-study-2024-03-28/ | Depression | Webpage | Disease prevention | Representation bias | A recent U.S. study published in PNAS found that artificial intelligence models analyzing social media posts can detect signs of depression in white Americans but are far less accurate for Black Americans, underscoring the dangers of using AI trained on non-diverse data in healthcare. According to co-author Sharath Chandra Guntuku from Penn Medicine, these differences suggest that prior AI models and language-based assessments have largely overlooked racial diversity. While the researchers noted that social media analysis should not be used for diagnosis, it may still help assess risk or monitor mental health trends in communities. | Race: racial bias because it used health care spending as a proxy for medical need. |
| 61 | Exploring Gender Bias in AI for Personalized Medicine: Focus Group Study With Trans Community Members | Nataly Buslon, David Cirilli, Oriol Rios, Simon Perene del Rosario, (Vol 27, 2025). Exploring Gender Bias in AI for Personalized Medicine: Focus Group Study With Trans Community Members. https://www.jmir.org/2025/1/e72325/ | Not Sure/Other | Other | Health promotion | Aggregation bias | This paper investigates how artificial intelligence (AI) can be applied to personalized medicine for trans individuals—an area largely neglected in precision health research. Using a participatory, communicative methodology involving trans community members, the study identifies key barriers such as data bias, privacy concerns, and the lack of trans-specific health data. It also highlights opportunities for inclusive AI development through community-led data initiatives and ethical frameworks that respect gender diversity. The findings emphasize the need for trans-inclusive, culturally competent AI systems to ensure equitable access to the benefits of precision medicine. | Gender identity impact |
| 62 | Addressing bias in big data and AI for health care: A call for open science | Norori, N., Hu, Q., Aellen, F.M., Faraci, F.D., & Tzovara, A. (2021). Addressing bias in big data and AI for health care: A call for open science. Patterns, 2(10), 100347. https://www.cell.com/patterns/fulltext/S2666-3899(21)00202-6?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2666389921002026%3Fshowall%3Dtrue | Not Sure/Other | Scholarly publication | Diagnosis | Representation bias | This article analyses data, algorithmic, and human bias in AI systems for healthcare. It illustrates how underrepresentation in datasets, especially regarding gender and racial minorities, leads to systematic errors such as misdiagnosis (e.g., melanoma, cardiovascular disease, oxygen saturation). It calls for open science practices to mitigate bias through inclusive datasets, transparency, and participatory approaches. | Women, Black patients, elderly, LGBTQ+ individuals, and other underrepresented patient groups. |
| 63 | Assessing and Mitigating Bias in Medical Artificial Intelligence. Circulation: Arrhythmia and Electrophysiology | Noseworthy, P. A., Attia, Z. I., Brewer, L. C., Hayes, S. N., Yao, X., Kapa, S., Friedman, P. A., & Lopez-Jimenez, F. (2020). Assessing and Mitigating Bias in Medical Artificial Intelligence. Circulation: Arrhythmia and Electrophysiology, 13(3). https://doi.org/10.1161/circep.119.007988 | Cardiovascular | Scholarly publication | Diagnosis, Risk assesment | Representation bias, Measurement bias, Evaluation bias | The study evaluated a deep learning model's performance in detecting low left ventricular ejection fraction using 12-lead ECGs across different racial/ethnic groups. The model, originally trained on a predominantly non-Hispanic White cohort, demonstrated suprinsingly successful performance across various racial/ethnic subgroups. | Race: the study highlights the cruciality of employing different training datasets to mitigate potential biases in biomedical AI, as opposed to homogenous populations that can generate disparties in AI performance. |
| 64 | Sociodemographic biases in medical decision making by large language models | Omar, M., Soffer, S., Agbareia, R. et al. Sociodemographic biases in medical decision making by large language models. Nat Med 31, 1873–1881 (doi: 10.1038/s41591-025-03626-6) | Not Sure/Other | Scholarly publication | Diagnosis, Treatment response | Evaluation bias | In this study, the authors performed a large-scale evaluation on 9 LLMs across 32 sociodemographic groups. They found that cases labeled as Black or unhoused or identifying as LGBTQIA+ were more frequently directed toward urgent care, invasive interventions or mental health evaluations. Similarly, cases labeled as having high-income status received significantly more recommendations for advanced imaging tests, while low- and middle-income-labeled cases were often limited to basic or no further testing. | Gender and minorities. |
| 65 | Gender and racial bias unveiled: clinical artificial intelligence (AI) and machine learning (ML) algorithms are fanning the flames of inequity | Otokiti A., Shih H., Williams K. (2025). Gender and racial bias unveiled: clinical artificial intelligence (AI) and machine learning (ML) algorithms are fanning the flames of inequity. https://doi.org/10.1093/oodh/oqaf027 | Not Sure/Other | Scholarly publication | Other | Representation bias | This systematic review highlights a critical gap in the transparency and inclusivity of clinical AI/ML models. Among nearly 400 studies analyzed, 84% failed to report racial composition and 31% omitted gender data in their training datasets, raising concerns about bias and limited generalizability. The study calls for standardized demographic reporting and greater use of open data to ensure ethical, equitable, and safe deployment of AI in healthcare. | Gender, race, geographical location |
| 66 | Inteligência Artificial no setor da saúde: desafios jurídicos e regulação | Paixão Cansado, Marta, (2024), Inteligência Artificial no setor da saúde: desafios jurídicos e regulação, No 187, GEE Papers, Gabinete de Estratégia e Estudos, Ministério da Economia, https://EconPapers.repec.org/RePEc:mde:wpaper:187 | Not Sure/Other | Other | Other | Representation bias, Learning bias | In this institutional paper, the author highlights current issues associated withthe use of AI in medicine, including its potential biases. She discusses key measures to counteract these issues, such as establishing rigorous requirements for these systems, as well as their maintenance and periodic evaluation. She also focus on developing AI systems based on ethical considerations, promoting collaboration between professionals from different fields, and providing essential training for healthcare professionals in the use of AI systems, so that they can recognize and act on potential flaws and errors. | Sex; Gender; Race |
| 67 | A Narrative Review on Ethical Considerations and Challenges in AI-Driven Cardiology. Annals of Medicine & Surgery | Patel, D., Chetarajupalli, C., Khan, S., Khan, S., Patel, T., Joshua, S., & Millis, R.M. (2025). A Narrative Review on Ethical Considerations and Challenges in AI-Driven Cardiology. Annals of Medicine & Surgery, 87, 4152–4164 https://journals.lww.com/annals-of-medicine-and-surgery/fulltext/2025/07000/a_narrative_review_on_ethical_considerations_and.26.aspx | Cardiovascular | Scholarly publication | Diagnosis | Representation bias | The review explores ethical and fairness challenges in AI-driven cardiology, focusing on bias, privacy, accountability, and transparency. It reports how training datasets lacking minority and low-socioeconomic representation lead to unequal diagnostic accuracy, especially in hypertension, diabetes-related heart disease, and coronary assessment (FFR-CT, echocardiography). It calls for diverse datasets, fairness audits, and transparency frameworks. | Minority ethnic groups, women, and low-income patients disproportionately affected by cardiovascular disease. |
| 68 | The need for ethnoracial equity in artificial intelligence for diabetes management: Review and recommendations | Pham, Q., Wiljer, D., Cafazzo, J., & et al. (2021). The need for ethnoracial equity in artificial intelligence for diabetes management: Review and recommendations. Journal of Medical Internet Research, 23(2), e22320. https://doi.org/10.2196/22320 | Diabetes | Scholarly publication | Disease self-management, Prediction, Risk assesment, Other | Representation bias, Measurement bias, Evaluation bias | This review highlights how artificial intelligence (AI) systems for diabetes management risk perpetuating ethnoracial inequities because most existing studies fail to include or report diverse participant data. By re-analyzing 141 prior AI-for-diabetes papers, the authors found that only 7% mentioned race or ethnicity, and most datasets were White and Western. They warn that such representation bias can lead to inaccurate predictions and unequal care for minority groups. To address this, they propose a five-question screening tool researchers and clinicians can use to assess ethnoracial equity when developing or evaluating AI-based diabetes interventions. The paper calls for inclusive, transparent, and equitable AI to ensure emerging digital health tools benefit all populations. | Race |
| 69 | Enhancing Fairness and Accuracy in Diagnosing Type 2 Diabetes in Young Adult Population | Pias, T. S., Su, Y., Tang, X., Wang, H., Faghani, S., & Yao, D. (2025). Enhancing Fairness and Accuracy in Diagnosing Type 2 Diabetes in Young Adult Population. IEEE Journal of Biomedical and Health Informatics, 1–10. https://doi.org/10.1109/jbhi.2025.3616312 | Diabetes | Scholarly publication | Diagnosis, Risk assesment | Representation bias | The authors find a “digital ageism”, namely that traditional machine-learning models consistently fail to diagnose type 2 diabetes among the younger adult subgroup (ages 30-44), largely because that subgroup is a minor share of the dataset. Alternatively, the authors propose separate training channels per 5-year age band (e.g., 30–34, 35–39, 40–44), observing that this approach substantially boost deteection of type 2 diabetes between young adults. | Age: the bias operates explicitly along the age variable. Young adults are particularly exposed to misdiagnoses of type 2 diabetes, which undermines the effectiveness of AI-based diagnostic and treatment methods. |
| 70 | Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation | Puyol-Antón, E., Bram Ruijsink, Piechnik, S. K., Neubauer, S., Petersen, S. E., Razavi, R., & King, A. P. (2021). Fairness in Cardiac MR Image Analysis: An Investigation of Bias Due to Data Imbalance in Deep Learning Based Segmentation. 413–423. https://doi.org/10.1007/978-3-030-87199-4_39 | Cardiovascular | Scholarly publication | Imaging | Representation bias | The authors foudn that deep learning models for segmenting cardiac MRI images show performance disparities across demographic groups, with worse segmentation accuracy in underrepresented groups. They demonstrate that date umbalance (e.g. fewer images of certain sexes or demographic categories) contributes to these fairness issues. | Both race and gender: the bias means that certain patient subgroups may receive less accurate anatomical measurements or diagnostic information from AI systems, undermining clinical trust and effectiveness. It may exacerbate healthcare inequalities because underrepresented groups get systematically poorer support from AI-based imaging tools. |
| 71 | The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations | Quynh Pham, Anissa Gamble; Jason Hearn; Joseph A Cafazzo, Journal if Medical Internet Research Vo 23 (2021), The Need for Ethnoracial Equity in Artificial Intelligence for Diabetes Management: Review and Recommendations, https://www.jmir.org/2021/2/e22320/ | Diabetes | Webpage | Treatment response | Other | Diabetes disproportionately affects ethnoracial minority populations, yet artificial intelligence (AI) interventions for diabetes management often overlook these disparities. This study conducted a secondary analysis of 141 articles from a 2018 review on AI for diabetes care to evaluate ethnoracial representation. Only 10 studies (7.1%) reported any ethnoracial data, with most samples overwhelmingly White and limited inclusion of other groups. These findings highlight a critical gap in the equitable design of AI-based diabetes tools. To address this, we propose a questionnaire to assess ethnoracial equity in AI research, emphasizing the urgent need to prevent systemic biases from being embedded in emerging health technologies. | Ethnoracial representation and equity. |
| 72 | Key language markers of depression on social media depend on race | Rai, S., Stade, E. C., Giorgi, S., Fodeh, S. J., Ungar, L. H., & Guntuku, S. C. (2024). Key language markers of depression on social media depend on race. Proceedings of the National Academy of Sciences, 121(14), e2319837121. https://doi.org/10.1073/pnas.2319837121 | Depression | Scholarly publication | Prediction | Representation bias, Measurement bias, Evaluation bias | The article examines how AI models that analyze social media language to detect depression can reflect racial bias. It shows that linguistic markers commonly associated with depression, such as first-person pronoun use and negative emotion words, predict depression severity in White individuals but not in Black individuals. The study further finds that machine learning models trained on language data perform significantly worse for Black participants, even when trained on their own language, revealing issues of representation and evaluation bias. The authors conclude that current AI models for mental health detection risk misdiagnosing or underdetecting depression in Black populations, underscoring the need for racially inclusive model design and validation. | Race |
| 73 | Addressing AI Algorithmic Bias in Health Care | Ratwani, R. M., Sutton, K., & Galarraga, J. E. (2024). Addressing AI Algorithmic Bias in Health Care. JAMA, 332(13), 1051–1052. https://doi.org/10.1001/jama.2024.13486 | Not Sure/Other | Scholarly publication | Risk assesment | Other | The paper discusses a widely used US health management algorithm that used healthcare cost as a proxy for clinical need. Because Black patients historically incurred lower costs due to unequal access, the model systematically underestimated their care needs, illustrating proxy and label bias. | Black patients |
| 74 | Artificial intelligence in diabetes management: Transformative potential, challenges, and opportunities in healthcare | Sarma, A. D., & Devi, M. (2025). Artificial intelligence in diabetes management: Transformative potential, challenges, and opportunities in healthcare. Hormones, 24, 307–322. https://doi.org/10.1007/s42000-025-00644-4 | Diabetes | Scholarly publication | Diagnosis, Disease self-management, Risk assesment, Surveillance, Treatment response | Representation bias | The article reviews how AI is being used across the diabetes care spectrum, screening/diagnosis (e.g. retinopathy imaging), risk prediction and prevention, treatment optimization (insulin dosing/medication management), real-time self-management via CGM/wearables, and even genomics/drug discovery, highlighting global applications including underserved settings. It underscores equity concerns and bias risks stemming from unrepresentative datasets (demographic/representation bias), uneven data quality and infrastructure (measurement/deployment bias), and limited transparency (evaluation/interpretability bias), which can disadvantage groups by race/ethnicity, gender, geography, or socioeconomic status. The purpose of AI here is to personalize and scale diabetes care while warning that responsible, privacy-preserving, and fair implementation is essential to avoid amplifying existing disparities. | Gender, race, socioeconomic status |
| 75 | Bias Mitigation in Primary Health Care Artificial Intelligence Models: Scoping Review | Sasseville, M., Ouellet, S., Rhéaume, C., Sahlia, M., Couture, V., Després, P., Paquette, J.-S., Darmon, D., Bergeron, F., & Gagnon, M.-P. (2025). Bias Mitigation in Primary Health Care Artificial Intelligence Models: Scoping Review. Journal of Medical Internet Research https://www.jmir.org/2025/1/e60269 | Not Sure/Other | Other | Diagnosis, Risk assesment | Representation bias | Scoping review of 17 studies on bias mitigation in primary health care AI models. Race and sex were the most frequently examined attributes. Identified four main strategies: data preprocessing (relabeling, reweighing), data sourcing, human-in-the-loop tools, and ethical frameworks. Representation bias was most common, especially underrepresentation of Black and female patients. | Black patients, women, low socioeconomic status groups, and other underrepresented populations. |
| 76 | Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia | Scheetz J., Koca D., McGuinness M., Holloway E., Tan Z., Zhu Z., O’Day R., Sandhu S., MacIsaac R., Gilfillan C., Angus Turner A., Stuart Keel S., He M.(2021). Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia. https://www.nature.com/articles/s41598-021-94178-5 | Diabetes | Scholarly publication | Diagnosis, Imaging, Public health | Representation bias, Measurement bias, Evaluation bias | This study evaluates the diagnostic performance and user experience of an AI-assisted diabetic retinopathy screening system in Australian endocrinology and Indigenous healthcare settings. It explicitly includes Aboriginal Medical Services clinics, addressing racial bias by testing the AI tool in a population with higher disease burden and lower screening rates. While the study highlights the importance of equitable access and performance in underserved communities, it does not explore gender or sex bias. The findings support the feasibility and accuracy of AI-assisted screening in diverse clinical environments, with high satisfaction among patients and clinicians. | Race |
| 77 | Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations | Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182. https://doi.org/10.1038/s41591-021-01595-0 | Cardiovascular | Scholarly publication | Diagnosis | Representation bias, Evaluation bias, Deployment bias | The study found that AI systems applied to chest X-ray imaging consistently underdiagnose diseases in under-served populations, with especially high underdiagnosis rates in intersectoral subgroups (for example, Hispanic female patients). | Race & gender: this underdiagnosis bias may delay or deny necessary medical care to marginalised patients, reinforcing existing health disparities. It undermines the trustworthiness and fairness or AI diagnostic tools, threatening equitable access to medical treatment. |
| 78 | Artificial intelligence for diabetes care: Current and future prospects. The Lancet Diabetes & Endocrinology | Sheng, B., Pushpanathan, K., Guan, Z., Lim, Q. H., Lim, Z. W., Yew, S. M. E., Goh, J. H. L., Bee, Y. M., Sabanayagam, C., Sevdalis, N., Lim, C. C., Lim, C. T., Shaw, J., Jia, W., Ekinci, E. I., Simó, R., Lim, L.-L., Li, H., & Tham, Y.-C. (2024). Artificial intelligence for diabetes care: Current and future prospects. The Lancet Diabetes & Endocrinology, 12(8), 569–595. https://doi.org/10.1016/s2213-8587(24)00154-2 | Diabetes | Scholarly publication | Diagnosis, Disease self-management, Surveillance | Representation bias | This review paper examines how AI is being applied across the diabetes care continuum, from screening, diagnosis, and prediction to treatment, self-management, and complication monitoring. The authors highlight AI’s capacity to enable personalised, data-driven care, improve glycaemic control, and extend access to low-resource settings. However, they also stress critical challenges such as algorithmic and representation biases, particularly across ethnic, gender, and socioeconomic groups, which risk reinforcing inequities in diagnosis and treatment. The paper calls for ethical and inclusive AI development, transparent validation across diverse populations, and stronger regulatory frameworks to ensure that the benefits of AI in diabetes care are distributed equitably worldwide. | Race, gender, socioeconomic groups |
| 79 | Digital health, gender and health equity: invisible imperatives | Sinha, Chaitali & Schryer-Roy, Anne-Marie. (2018). Digital health, gender and health equity: invisible imperatives. Journal of public health (Oxford, England). https://pmc.ncbi.nlm.nih.gov/articles/PMC6294032/ | Not Sure/Other | Other | Public health | Other | Digital health technologies, including eHealth and mHealth, are improving access to life-saving health services in low- and middle-income countries. Despite the positive narrative, there is a lack of studies focusing on health equity and gender dynamics in digital health interventions. The document presents findings from implementation research projects across Africa, Asia, Latin America, and the Middle East, highlighting three main themes: 1) Digital health can enhance health equity. 2) Gender and power analyses are crucial for effective interventions. 3) Digital health can improve accountability within health systems. Theme 1: Positive Influence on Health Equity: Addressing health equity is essential in the context of the Sustainable Development Goals (SDGs). Research shows that digital health interventions can positively impact marginalized groups, such as ethnic minority women in Vietnam and people living with HIV in Burkina Faso. Theme 2: Importance of Gender and Power Analyses: Digital health initiatives often operate within contexts of existing gender inequalities and power imbalances. Gender analysis, particularly from an intersectional perspective, is necessary to understand and address these dynamics. Studies indicate that mHealth interventions can enhance the social status and agency of health workers, as seen in Ethiopia, while also empowering pregnant indigenous women in Peru by providing access to health information. Theme 3: Strengthening Accountability: Digital health has the potential to enhance accountability among community members and health workers. Research from Lebanon and Ethiopia demonstrates that mHealth solutions can improve health service delivery and foster bidirectional accountability between health workers and the communities they serve. Engaging local communities in health governance is crucial for the success of these interventions. Concluding Remarks: The findings emphasize the need for health equity and gender analysis in digital health methodologies. The evidence suggests that well-designed digital health interventions can contribute to achieving the SDGs and ensuring that no one is left behind. However, attention to existing gender and power dynamics is vital to avoid perpetuating inequalities. The document advocates for integrating context-specific analyses into the design and implementation of digital health initiatives to maximize their effectiveness and equity. | Health equity, gender and power dynamics |
| 80 | Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation Proceedings of the Seventh | "Sogancioglu, G., Mosteiro, P., Salah, A.A., Scheepers, F., Kaya,H. Fairness in AI-Based Mental Health: Clinician Perspectives and Bias Mitigation Proceedings of the Seventh AAAI/ACM Conference on AI, Ethics, and Society (AIES 2024) https://research-portal.uu.nl/ws/files/243475617/31732-Article_Text-35796-1-2-20241016.pdf " | Depression | Scholarly publication | Diagnosis, Public health, Risk assesment, Other | Representation bias | There is limited research on fairness in automated decision making systems in the clinical domain, particularly in the mental health domain. The study explores clinicians’ perceptions of AI fairness through two distinct scenarios: violence risk assessment and depression phenotype recognition using textual clinical notes. They engage with clinicians through semi-structured interviews to understand their fairness perceptions and to identify appropriate quantitative fairness objectives for these scenarios. Then, they compare a set of bias mitigation strategies developed to improve at least one of the four selected fairness objectives. The findings underscore the importance of carefully selecting fairness measures, as prioritizing less relevant measures can have a detrimental rather than a beneficial effect on model behavior in real-world clinical use. | Gender |
| 81 | Racial Bias Found in a Major Health Care Risk Algorithm | Starre Vartan, (2019). Racial Bias Found in a Major Health Care Risk Algorithm, https://www.scientificamerican.com/article/racial-bias-found-in-a-major-health-care-risk-algorithm/ | Not Sure/Other | Webpage | Prediction | Representation bias | "A study published in Science revealed that a widely used U.S. health care risk-prediction algorithm showed racial bias because it used health care spending as a proxy for medical need. Although intended to identify patients who would benefit from high-risk care management programs, the algorithm underestimated the needs of Black patients. Even when Black and white patients had similar medical costs, Black patients had more chronic illnesses, meaning the system was less likely to qualify them for additional care. The bias arose because Black patients often spend less on health care due to lower income, reduced access to services, and experiences of implicit bias that lower trust in providers. As a result, using cost as a stand-in for health needs reinforced existing inequalities." | Race: racial bias because it used health care spending as a proxy for medical need. |
| 82 | Artificial Intelligence in mental health and the biases of language based models | Straw I, Callison-Burch C. Artificial Intelligence in mental health and the biases of language based models. PLoS One. 2020 Dec 17;15(12):e0240376. doi: 10.1371/journal.pone.0240376. PMID: 33332380; PMCID: PMC7745984. | Depression | Scholarly publication | Other | Historical Bias, Representation bias, Learning bias | This literature review evaluated bias in existing Natural Language Processing (NLP) models used in psychiatry and found significant biases with respect to religion, race, gender, nationality, sexuality and age. | Gender, race |
| 83 | Can AI fight sex and gender bias in healthcare? | "Sue Haupt, Bronwyn Graham, Jane Hirst (2024). Can AI fight sex and gender bias in healthcare? https://www.unsw.edu.au/newsroom/news/2024/10/can-ai-fight-sex-and-gender-bias-in-healthcare- " | Not Sure/Other | Webpage | Public health | Other | Artificial Intelligence (AI) is transforming healthcare by supporting diagnosis, treatment planning, and medical innovation. However, AI systems are often developed using male-dominated datasets and design frameworks, leading to significant sex and gender biases that can endanger women and nonbinary patients. These biases arise from both the coding language used in AI and the data sets that underpin machine learning models. When AI is trained on incomplete or gender-skewed data, it risks perpetuating misdiagnoses, unequal treatment recommendations, and systemic healthcare inequities. | Gender |
| 84 | The Biomedical Applications of Artificial Intelligence: An Overview of Decades of Research | Sweet Naskar, Sharma, S., Ketousetuo Kuotsu, Halder, S., Pal, G., Saha, S., Mondal, S., Biswas, U. K., Jana, M., & Bhattacharjee, S. (2025). The Biomedical Applications of Artificial Intelligence: An Overview of Decades of Research. Journal of Drug Targeting, 1–85. https://doi.org/10.1080/1061186x.2024.2448711 | Not Sure/Other | Scholarly publication | Diagnosis, Management and planning, Prediction | Representation bias, Learning bias | The authors find that AI holds great promise for improving accuracy, diagnostic efficiency, and therapeutic decision-making in biomedicine, but real-world adoption is often hindered by biases and lack of generalisability. They also observe that many published AI applications in biomedicine inadequately address fairness, transparency, and robustness across diverse populations. | Both race and gender: these biases can lead AI biomedical tools to perform poorly or unfairly on underrepresented groups, thus reinforcing existing health inequalities. They also reduce trust, limit clinical applicability, and threaten the ethical deployment of AI in healthcare. |
| 85 | Addressing bias: artificial intelligence in cardiovascular medicine | Tat, E., Bhatt, D.L., & Rabbat, M.G. (2020). Addressing bias: artificial intelligence in cardiovascular medicine. The Lancet Digital Health, 2(12), e635–e636. https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30249-1/fulltext | Cardiovascular | Scholarly publication | Diagnosis | Representation bias | The commentary discusses how AI algorithms in cardiology, including echocardiography and cardiac MRI applications, risk reproducing existing healthcare inequalities. It highlights gender and racial bias due to underrepresentation of women and Black patients in datasets and clinical trials, leading to misdiagnosis and unequal care. Examples include biased triage algorithms and predictive models favoring White, high-income patients. | Women, black and minority ethnic patients, and low-income populations. |
| 86 | Promises and challenges of digital tools in cardiovascular care | The Lancet Digital Health Editorial (2024). Promises and challenges of digital tools in cardiovascular care. The Lancet Digital Health, 6(10), e673–e675 https://www.thelancet.com/journals/landig/article/PIIS2589-7500(24)00194-8/fulltext | Cardiovascular | Scholarly publication | Diagnosis | Representation bias | The editorial highlights how digital tools and AI in cardiovascular medicine—such as cardiac imaging and large language models—offer transformative potential but risk reinforcing existing biases if training data lack diversity. It calls for diverse research teams, inclusion of minority populations, and equitable access to digital innovations in low- and middle-income countries. | Racial and ethnic minorities, patients from low- and middle-income countries, and underrepresented groups in cardiovascular research. |
| 87 | Developing personalized algorithms for sensing mental health symptoms in daily life | Timmons A., Tutul A., Avramidis K., Duong J., Carta K., Walters S., Jumonville G., Carrasco A., Freitag G., Romero D., Ahle M., Comer J., Narayanan S., Khurd I., Chaspari T. (2025). Developing personalized algorithms for sensing mental health symptoms in daily life. https://www.nature.com/articles/s44184-025-00147-5?fromPaywallRec=false | Depression | Scholarly publication | Prediction | Representation bias, Measurement bias, Aggregation bias | This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data. It finds that these tools underperform in larger, more diverse populations because the behavioral patterns used to predict depression are inconsistent across demographic and socioeconomic subgroups. Specifically, the AI models often misclassify individuals from certain groups—such as older adults or those from different racial or gender backgrounds—as being at lower risk than they actually are. The authors emphasize the need for tailored, subgroup-aware approaches to improve reliability and fairness in mental health prediction tools. This work highlights the importance of addressing demographic bias to ensure equitable AI deployment in mental healthcare. | Race, age, and gender. |
| 88 | Head of MediaNew AI algorithm uses mammograms to accurately predict cardiovascular risk in women | Tina Wall (2025), Head of MediaNew AI algorithm uses mammograms to accurately predict cardiovascular risk in women. https://www.georgeinstitute.org/news-and-media/news/new-ai-algorithm-uses-mammograms-to-accurately-predict-cardiovascular-risk-in-women | Cardiovascular | Webpage | Prediction | Other | Researchers at The George Institute for Global Health, in collaboration with the University of New South Wales and University of Sydney, have developed a machine learning model that predicts women’s cardiovascular disease (CVD) risk using only mammograms and age. Published in Heart, the model performs as accurately as traditional risk calculators that require multiple clinical data points. | Gender |
| 89 | Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study | Travis Zack, Eric Lehman, Mirac Suzgun, et al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024 Jan;6(1):e12-e22. doi: 10.1016/S2589-7500(23)00225-X. | Not Sure/Other | Scholarly publication | Diagnosis, Management and planning, Risk assesment, Treatment response | Historical Bias, Representation bias, Evaluation bias | In this study, GPT-4 exhibited racial and gender bias across clinically relevant tasks, including the generation of cases for medical education, support for differential diagnostic reasoning, medical plan recommendation, and subjective assessments of patients.It was found to exaggerate known disease prevalence differences between groups. | Gender and race: as an example, GPT-4 tended to overrepresent stereotypes of diseases, such as sarcoidosis in Black patients and hepatitis B in Asian patients. |
| 90 | Racial disparities in continuous glucose monitoring-based 60-min blood glucose predictions in people with type 1 diabetes | Tytgat, N., Marques, P., Gho, J., Stefański, B., & Cornelissen, F. (2023). Racial disparities in continuous glucose monitoring-based 60-min blood glucose predictions in people with type 1 diabetes. PLOS Digital Health, 2(9): e0000918. https://journals.plos.org/digitalhealth/article?id=10.1371%2Fjournal.pdig.0000918&utm | Diabetes | Scholarly publication | Diagnosis | Representation bias | The study analysed whether machine learning models predicting 60-minute blood glucose levels (using CGM data) performed equally well for Non-Hispanic White vs Non-Hispanic Black patients. They found that as the proportion of White participants in training increased, performance improved for White patients but dropped for Black patients, indicating racial bias in prediction accuracy. | Black patients with type 1 diabetes (lower predictive performance). |
| 91 | Implications of Bias in Artificial Intelligence: Considerations for Cardiovascular Imaging | "van Assen M., Beecy A., Gershon G., Newsome J., Trivedi H., Gichoya J. (2024). Implications of Bias in Artificial Intelligence: Considerations for Cardiovascular Imaging. DOI: 10.1007/s11883-024-01190-x " | Cardiovascular | Scholarly publication | Diagnosis, Public health, Other | Evaluation bias, Other | Article examines an implicit and explicit biases in cardiovascular disease care, with attention given to their presence in datasets that may not be apparent to model developers. This paper is a comprehensive guide for AI development teams to understand assumptions in datasets and chosen metrics for outcome/ground truth, and how this translates to real-world performance for cardiovascular disease. Disparities in cardiovascular disease outcomes across gender and racial groups, unequal treatment of marginalized populations, and imbalances in clinical trial representation have been reviewed. Additionally, it highlites bias in cardiovascular related AI literature, and summarises mitigation strategies currently employed by AI systems. | Gender, race, marginalized groups |
| 92 | Fairness and bias correction in machine learning for depression prediction across four study populations | Vien Ngoc Dang, Cascarano, A., Mulder, R. H., Cecil, C., Zuluaga, M. A., Jerónimo Hernández-González, & Karim Lekadir. (2024). Fairness and bias correction in machine learning for depression prediction across four study populations. Scientific Reports, 14(1). https://doi.org/10.1038/s41598-024-58427-7 | Depression | Scholarly publication | Prediction, Risk assesment | Representation bias, Learning bias | The study found that standard machine learning approaches often exhibit biased behaviours in predicting depression across different populations. It also demonstrated that both standard and novel post-hoc-bias mitigation techniques can effectively reduce unfair bias, though no single model achieves equality of outcomes. | Both race and gender: the identified biases can reinforce structural inequalities in mental healthcare, particularly affecting underserved populations. This underscores the importance of analysing fairness during model selection and transparently reporting the impact of debiasing interventions to ensure equitable healthcare applications. |
| 93 | AI-based diabetes care: risk prediction models and bias concerns | Wang, S.C.Y.. et al. (2024). AI-based diabetes care: risk prediction models and bias concerns. NPJ Digital Medicine https://www.nature.com/articles/s41746-024-01034-7?utm | Diabetes | Scholarly publication | Risk assesment | Representation bias | Review of 40 studies on AI risk-prediction for T2DM. Notes that many models lack external validation, and there are concerns about under-representation by race, age, gender, leading to fairness risks. | Older adults, women, racial/ethnic minorities |
| 94 | Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models | "World Health Organisation:Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models 25 March 2025 | Publication https://www.who.int/publications/i/item/9789240084759 " | Not Sure/Other | Other | Public health, Risk assesment | Representation bias, Other | WHO is issuing this guidance to assist Member States in mapping the benefits and challenges associated with use of LMMs for health and in developing policies and practices for appropriate development, provision and use. The guidance includes recommendations for governance, within companies, by governments and through international collaboration, aligned with the guiding principles. The principles and recommendations, which account for the unique ways in which humans can use generative AI for health, are the basis of this guidance. | Race & Gender and income |
| 95 | Chatbots and Diabetes: Is There Gender Bias? Journal of Patient Experience | "Wu, G., Tewari, S., Wong, A., Chung, E., Chim, I., Hoang, B., Mansubi, N., Shams, A., Del Buono, M., Srinivasan, S., Paliath-Pathiyal, H., & Khan, O. (2025). Chatbots and Diabetes: Is There Gender Bias? Journal of Patient Experience, 12. https://doi.org/10.1177/23743735251380954 " | Not Sure/Other | Scholarly publication | Health promotion, Management and planning, Other | Representation bias, Learning bias, Evaluation bias | The study examines LLM chatbot responses to diabetes-related prompts and compares them accorss gender user identities. While factual accuracy was comparable, responses showed gender bias in the quality of communication (i.e. bot's tone, empathy, and readability). Some replies to female users were less empathetic or used different phrasing. | Gender: the gender bias affects the user's experience with chatbots. Because the bot’s tone, style, or empathetic framing may vary by gender, female (or male) users might receive less supportive or less well-tailored explanations. This can discourage trust, comprehension, or ease of usage of LLMs. |
| 96 | Deconstructing demographic bias in speech-based machine learning models for digital health | Yang M., El-Attar A., Chaspari T. (2024). Deconstructing demographic bias in speech-based machine learning models for digital health. Frontiers in Digital Health. http://DOI: 10.3389/fdgth.2024.1351637 | Diabetes | Scholarly publication | Disease prevention, Prediction, Risk assesment | Representation bias, Measurement bias, Aggregation bias, Evaluation bias | This study investigates algorithmic bias in AI tools that predict depression risk using smartphone-sensed behavioral data. It finds that the model underperforms across several demographic subgroups, including gender, race, age, and socioeconomic status, often misclassifying individuals with depression as low-risk. For example, older adults and Black or low-income individuals were frequently ranked lower in risk than healthier younger or White individuals. These biases stem from inconsistent relationships between sensed behaviors and depression across groups. The authors emphasize the need for subgroup-specific modeling to improve fairness and reliability in mental health AI tools. | Gender, race, age, and socioeconomic status |
| 97 | Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121 | "Yousufi M., Damasevičius R., Maskeliunas R. (2024). Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121 https://doi.org/10.3390/brainsci14101018 " | Depression | Scholarly publication | Diagnosis | Measurement bias, Learning bias | "Depression and anxiety are common, often co-occurring mental health disorders that complicate diagnosis due to overlapping symptoms and reliance on subjective assessments. Standard diagnostic tools are widely used but can introduce bias, as they depend on self-reported symptoms and clinician interpretation, which vary across individuals. These methods also fail to account for neurobiological factors such as neurotransmitter imbalances and altered brain connectivity. Similarly, clinical AI/ML models used in healthcare often lack demographic diversity in their training data, with most studies failing to report race and gender, leading to biased outputs and reduced fairness. EEG offers a promising, objective approach to monitoring brain activity, potentially improving diagnostic accuracy and helping address biases in mental health assessment. | Race, gender |
| 98 | Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions | Zainab Al-Zanbouri, Sharma, G., & Raza, S. (2024). Equity in Healthcare: Analyzing Disparities in Machine Learning Predictions of Diabetic Patient Readmissions. 660–669. https://doi.org/10.1109/ichi61247.2024.00105 | Diabetes | Scholarly publication | Prediction, Risk assesment | Representation bias, Learning bias, Evaluation bias | The study compared several machine learning models for predicting hospital readmissions among diabetic patients and evaluated their fairness across demographic subgroups. Gradient Boosting Machine (GBM) and Generalized Linear Model (GLM) demonstrated the most equitable and accurate performance across gender, race, and age categories. However, disparities persisted: African American and Hispanic patients experienced higher false positive rates. | "Race: The higher false positive rates for African American and Hispanic groups indicate lingering racial bias that can lead to unequal treatment or over-monitoring of these populations. Gender: Gender-based disparities were largely mitigated in GBM and GLM, suggesting that with mindful model design and evaluation AI systems can achieve equitable outcomes across genders in diabetes risk assessment. |
| 99 | ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management | "Zhang, A., Yuksekgonul, M., Guild, J., Zou, J., & Wu, J. C. (2023). ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management. ArXiv.org. https://doi.org/10.48550/arXiv.2311.14703 " | Cardiovascular | Scholarly publication | Diagnosis, Management and planning, Treatment response | Representation bias, Measurement bias, Learning bias, Evaluation bias | The study found that specifying patients as female, African American, or Hispanic resulted in ChatGPT 3.5 providing less guideline-recommended medical management for ACS (Acute Coronary Syndrome, a cardiovascular disease). These biases correlate with known clinical disparities in ACS outcomes. Interestingly, prompting ChatGPT to explain its reasoning before providing an answer improved clinical accuracy and reduced gender and racial biases. | Gender & Race: both the gender and race bias lead to unequal treatment recommendaitons for female and African American and Hispanic patients. This can result in severe under-treatment or mistreatment of ACS. |
AEQUITAS Databaseinfo@innovationhive.eu2026-07-21T13:04:58+00:00
