(W1030-02-10) Development of a Predictive Model for Vancomycin-Associated Nephrotoxicity Using the Area under the Concentration-Time Curve of Vancomycin: A Machine Learning Analysis

Purpose: The primary side effect of vancomycin (VCM) is nephrotoxicity, which affects 5-43% of patients treated with VCM.¹ Nephrotoxicity associated with VCM can increase patient mortality.² Consequently, it is crucial to accurately assess the risk of nephrotoxicity. Recently, several studies sought to develop machine learning models with VCM trough concentrations, concomitant medications, and other background factors to predict VCM-associated nephrotoxicity.^3-7 However, these machine learning models have notable limitations. First, they do not address the issue of imbalanced data, given the relatively low incidence of nephrotoxicity (13.9–15.7%) in the Japanese population.^3-7 Generally, machine learning models can ignore the minority class, and this oversight can lead to the misclassification of most patients as not at risk of nephrotoxicity. Second, the existing models do not utilize the area under the concentration-time curve (AUC) of VCM to predict VCM-associated nephrotoxicity. With recent updates in guidelines recommending a shift from trough-guided to AUC-guided therapeutic drug monitoring (TDM) to prevent nephrotoxicity more effectively,^8-9 it is necessary that machine learning models also adapt to incorporate AUC data. In this study, we aimed to develop a new model for predicting VCM-associated nephrotoxicity to address these limitations.

Methods: We retrospectively analyzed the medical records of patients who received intravenous VCM at our hospital from August 2017 to July 2021. Fig. 1 illustrates the workflow of the construction of the machine learning model. First, we randomly divided the dataset into a 60:40 ratio for train and test data. Second, we developed machine learning models using AUC of VCM and other patient background factors based on the following machine learning algorithms: lasso regression, support vector machine, complementary naïve Bayes classifier, random forest, and AdaBoost. We utilized the synthetic minority oversampling technique (SMOTE) to address the issue of imbalanced data and performed hyperparameter tuning using a 5-repeated 3-fold cross-validation on the training data. Finally, we compared the performance of our developed machine learning models to that of the conventional model (AUC-guided TDM; AUC ≤ 600 μg·h/mL⁸). Recognizing the typical lack of interpretability in machine learning models, we used Shapley additive explanations (SHAP)¹⁰ to visualize the significant features. We evaluated the following nine patient background factors using SHAP: AUC at steady state, age, sex, body mass index, creatinine clearance, residence in the intensive care unit, mechanical ventilation, diabetes, and cumulative number of concomitant intravenous nephrotoxic agents.

Results: Data from 270 patients were collected: 162 were used for model building and 108 for performance evaluation. The random forest model achieved the highest F1 score with SMOTE, with an F1 of 0.436 and a sensitivity of 0.632. In contrast, the conventional model yielded an F1 value of 0.255 and a sensitivity of 0.316. SHAP plots revealed that patients receiving a higher cumulative number of concomitant intravenous nephrotoxic agents had a higher risk of nephrotoxicity than those receiving low doses (Fig. 2).

Conclusion: This study is the first to develop a machine learning model for predicting VCM-associated nephrotoxicity using the AUC of VCM. This model enables prediction with fewer missed cases of nephrotoxicity than conventional AUC-guided TDM.

References: 1） Antimicrob Agents Chemother. 2013; 57(2):734-44.
2） Infect Dis Ther. 2020;9(1):89-106.
3） J Eval Clin Pract. 2017;23(6):1240-1246.
4） J Eval Clin Pract. 2019;25(1):163-170.
5） PLoS One. 2020;15(7):e0236789.
6） Antibiotics (Basel). 2020;9(12):920.
7） Br J Clin Pharmacol. 2022;88(7):3241-3255.
8） Pharmaceutics. 2022;14(3):489.
9） BMC Infect Dis. 2021;21(1):153.
10) Adv Neural Inf Process Syst. 2017; 30: 11-21.

Acknowledgements: This study was approved by the Ethics Committee of the Faculty of Medicine, Tokyo Medical and Dental University. (authorization number: M2021-146)


Fig. 1. Workflow of the machine learning model building process.


Fig. 2. SHAP plot　
The significance of nine risk factors on VCM-associated nephrotoxicity is presented. Each dot represents a patient's output (risk of nephrotoxicity). The vertical axis indicates the average magnitude of each feature's impact on the output ranked in descending order. Higher-ranked features have a greater impact on the output. The horizontal axis shows the magnitude and direction of variation in the risk of VCM-associated nephrotoxicity compared to the average patient, with 0 as the reference point. Positive values indicate a high risk of VCM-associated nephrotoxicity, while negative values indicate a low risk. The color of each dot represents the feature value. Red and blue dots indicate high and low feature values, respectively.