Power of machine learning algorithms for predicting dropouts from a German telemonitoring program using standardized claims data

Background: Statutory health insurers in Germany offer a variety of disease management, prevention and health promotion programs to their insurees. Identifying patients with a high probability of leaving these programs prematurely helps insurers to offer better support to those at the highest risk of dropping out, potentially reducing costs and improving health outcomes for the most vulnerable. Objective: To evaluate whether machine learning methods outperform linear regression in predicting dropouts from a telemonitoring program. Methods: Use of linear regression and machine learning to predict dropouts from a telemonitoring program for patients with COPD by using information derived from claims data only. Different feature sets are used to compare model performance between and within different methods. Repeated 10-fold cross-validation with downsampling followed by grid searches was applied to tune relevant hyperparameters. Results: Random forest performed best with the highest AUC of 0.60. Applying logistic regression resulted in higher predictive power with regard to the correct classification of dropouts compared to neural networks with a sensitivity of 56%. All machine learning algorithms outperformed linear regression with respect to specificity. Overall predictive performance of all methods was only modest at best. Conclusion: Using features derived from claims data only, machine learning methods performed similar in comparison to linear regression in predicting dropouts from a telemonitoring program. However, as our data set contained information from only 1,302 individuals, our results may not be generalizable to the broader population.