Utilizing Machine Learning, Predictive Modeling, and Dashboards for the Real-Time Identification of Patients Most Likely to Leave Substance Use Disorder Inpatient Treatment AMA

Introduction

Adherence to treatment for patients diagnosed with substance use disorder (SUD) is a constant issue. Patients with SUD are at an increased risk of discharging against medical advice (AMA) as compared to patients without SUD. (1) AMA discharges correlate with increases in thirty-day mortality rates, and cause an increased variance in bed occupation which raises costs and is detrimental to revenue.1-4 Shepard Health hypothesizes that a predictive model can be developed to identify patients at higher risk for leaving AMA and the associative causes for this course of action.

Methods

A chart review was conducted utilizing electronic data sources at a behavioral health and SUD inpatient hospital, to identify risk factors associated with patients leaving AMA from SUD treatment. Utilizing logistic regression and machine learning algorithms, we developed models to predict the probability of a patient leaving AMA.

Results

There were more than 7,000 eligible patients discharged during the period of the study. Utilizing logistic regression analysis, the following variables produced the most accurate model with a prediction accuracy of 91%:

•       Age

•       Days in treatment

•       Wait time

•       Preliminary dosage amount

•       Time to first dose

•       First COWS total score

•       Difference between first and last COWS total score

•       Most recent COWS total score

•       Admission day 

•       Time from admission to first COWS assessment

Six classificatory machine learning algorithms were tested: logistic regression, random forests, extreme gradient boosting, support vector machines, lasso/elastic-net regularized generalized linear models, and neural networks. The Extreme Gradient Boosting model had the bests results with an 0.93 area under curve value. The following variables produced the most accurate model:

•       Age 

•       Days in treatment

•       First medication dose amount 

•       Second medication dose amount 

•       Opioid Diagnosis 

•       Alcohol Diagnosis 

•       Last COWS total score 

For both methods of modeling, days in treatment is a highly influential variable. Upon further analysis, patients were significantly more likely to leave AMA in Day 2 to Day 4. This result aligns with similar findings around readmissions, highlighting the existence of a period of time when interventions are critical to ensure positive outcomes.

Discharge AMA Rates by Number of Days Patient is in Treatment: With Trend Line of Best Fit

Discussion

To minimize the risk of patients leaving AMA, providers must identify associative risk factors and implement mitigating interventions. There is limited data available on AMA risk factors, and the data that exists is mostly linked to patient demographics. Healthcare facilities cannot control patient demographics, so risk factors linked to operations and treatment protocols are essential to eliminating unplanned discharges and reducing occupancy volatility.

Analyzing risk factors for AMA via machine learning provides key indicators where clinical and operational interventions can be implemented. An initial intervention implemented was a real-time dashboard for providers and administrators to monitor patients’ risk for AMA and assign resources and interventions to address patient needs.

*DIT: Days in Treatment variable

Conclusion

Utilizing machine learning and statistical modeling provides valuable information to help define thresholds and benchmarks and inform clinical providers so they can continue to provide the highest quality and most financially sustainable care.

1.        Ti L, Ti L. Leaving the hospital against medical advice among people who use illicit drugs: a systematic review. Am J Public Health. 2015;105(12):2587–2587

2.        Southern WN, Nahvi S, Arnsten JH. Increased risk of mortality and readmission among patients discharged against medical advice. Am J Med. 2012;125(6):594–602.

3.        Choi M, Kim H, Qian H, Palepu A. Readmission rates of patients discharged against medical advice: a matched cohort study. PLoS ONE. 2011;6(9):e24459.

4.       Glasgow JM, Vaughn-Sarrazin M, Kaboli PJ. Leaving against medical advice (AMA): risk of 30-day mortality and hospital readmission. J Gen Intern Med. 2010;25(9):926–92