Could machine learning models be used to predict early pregnancy depression in racial/ethnic minority women?

In a recent study uploaded to the medRxiv* pre-print server, researchers built and evaluated machine learning models to predict depression in pregnant women using data from electronic medical records.

Their study cohort included mostly low-income Hispanic and Black patients from the University of Illinois Hospital and Health Sciences system. Their results revealed that while machine learning can predict mental health problems in early pregnancy, its predictive performance is poor for low-income minority women.

Study: Predicting Prenatal Depression and Assessing Model Bias Using Machine Learning Models. Image Credit: NicoElNino/Shutterstock.com

*Important Notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be considered conclusive, guide clinical practice/health-related behaviors, or treated as established information.

Perinatal depression and its associated risk factors

Perinatal depression (PND) is a subset of mental illnesses affecting women during pregnancy and up to one year after childbirth. This is a growing concern, especially in the United States (US), where PND affects 10-20% of pregnant women.

The incidence of PND has been reported to more than triple between 2000 and 2005, with black (2 times) and Hispanic (5 times) women being much higher than their non-Hispanic white counterparts.

The COVID-19 pandemic has further exuberated the PND, with 27-32% of American women affected. Research has shown that PND leads to several complications unrelated to mental health, including premature labor, reduced infant birth weight, increased length and cost of hospital stay, and increased maternal morbidity and mortality.

Infants are observed to suffer from a significantly increased risk of inadequate cognitive development, underdeveloped socio-emotional behavior and impaired stress responses. Research has further reported infant growth retardation and an increased risk of future mental disorders in the children of women with PND.

The incidence of perinatal depression has been associated with many environmental factors, including unplanned pregnancies, negative childhood experiences, previous mental health issues, and lack of social support. While minorities have been at higher risk than white women, reports suggest social stigma makes them less likely to be screened for PND or seek professional help.

Machine learning (ML) models have been shown to predict pregnancy outcomes using electronic medical records (EMRs). However, previous studies using the ML on the PND have focused on predicting postpartum depression and have been conducted on cohorts of middle-class white women, largely ignoring racial or economic minorities.

This should introduce a predictive bias into ML models, reducing their ability to assess EMR data from minorities, including black and Hispanic women.

About the study

In this preprint, researchers developed ML models to predict and assess the severity of depression in women of color. The researchers collected EMR data from women who received obstetric care from the University of Illinois Hospital and Health Sciences (UIHealth) system from 2014 to 2020.

The data was skewed toward black (51%) and Hispanic (29%) women. In contrast, non-Hispanic whites (9%) and Asians and Native Americans (10%) are racial minorities in this data set.

Of the 5,875 people initially included, the researchers identified 2,414 women who met their selection criteria – complete EMR data for the Patient Health Questionnaire-9 (PHQ-9; this is a presence and severity of depression) and first obstetric visit before 24 weeks of pregnancy. The researchers used the PHQ-9 scores to assign study cohorts – women with scores of 1 to 4 (low depression) were the control, while those with scores of 9 and above formed the case group.

The set of variables used in training the ML model included 29 major classes of prescription drugs, race, and health insurance (an indicator of financial status). Demographic (employment status, marital status) and lifestyle (smoking and alcohol consumption) variables were used for model selection and fit.

Several models, including XGBoost, Random Forest, and Elastic Net models, were tested, after which Shapley’s values ​​were used to identify the variables most contributing to perinatal depression.

Shapley’s values ​​are a game-theoretic approach to assessing the individual contributions of variables to an observed outcome (in this case, perinatal depression and its severity).

Finally, the researchers used their ML model to assess the risk of perinatal depression, both in the study and control cohorts.

Study results

According to the study criteria, the 2,414 women included were divided into 657 cases and 1,757 controls. To account for the inherent bias given unbalanced cohort sizes, the researchers used 400 randomly selected case-control pairs to train each of the 20 models developed.

Statistical analyzes of the raw EMR data revealed that 81% of the study cohort included low-income Black and Hispanic women. Black women showed statistically higher unplanned pregnancies and unemployment status than other ethnic groups.

Their probability of being single was just as high. Lifestyle and health choices (unplanned pregnancy and smoking) appeared to play a role in the incidence of depression regardless of ethnicity.

The researchers identified the Elastic Net model as the best of the 20 models developed. While the Random Forest model matched the Elastic Net model for predicting depression in race-independent simulations, the latter showed significantly reduced computation time and was therefore used for training and evaluation.

Of the more than 600 variables in the EMR dataset, the ML model identified marital status, unplanned pregnancies, age, employment status, insurance policy, and tobacco use as the most predictive of the PND.

“…our model also identified features that have not been previously associated with the severity of depressive symptoms in pregnancy, or that have only been reported in a few studies. For example, we found that symptoms high levels of depression were positively associated with self-reported levels of pain, diagnosis of asthma, carrying a male fetus (82), use of antihistamines, analgesics, or antibiotics, and lower platelets in the blood.

The ML model further revealed that the severity of PND was most strongly associated with self-reported pain levels and previous mental disorders, the former being highest in black women.

Finally, model performance tests on case and control cohort data revealed that although the model was able to predict depression and severity with moderate accuracy (50-66%) in simulations independent of race, sensitivity was significantly higher for high-income white women (85%) compared to black women (70%), although the sample size was skewed in favor of the latter.

conclusion

In the current preprint, researchers constructed, selected, and tested the sensitivity of machine learning models to predict perinatal depression. They identified the Elastic Net and Random Forest models as the most accurate, with the latter being used in the tests given its lower computational requirements.

Although the sample size was skewed toward lower-income minorities (Black and Hispanic women), the model’s accuracy was higher for high-income white women (85% versus 66%).

Accuracy notwithstanding, this research suggests that ML models can be used to identify EMR in the early stages of pregnancy. This could improve maternal and infant health if integrated into obstetric care practices.

*Important Notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be considered conclusive, guide clinical practice/health-related behaviors, or treated as established information.