In China, although antiretroviral therapy is available for PLWHs, opportunistic infections and HIV-associated comorbidities are still the main reasons for admission to the ICU, which increases the socioeconomic burden and reminds us of timely prediction of the occurrence of serious comorbidities to reduce the risk of ICU admission. An intelligent healthcare system based on ML models for predicting specific risk factors has potential value in clinical work [11,12,13], including ICU admission. ML-model-based intelligent healthcare systems can help physicians identify the risk of ICU admission in a timely manner, reduce the likelihood of developing serious illnesses, and develop preventive strategies to alleviate patient suffering and reduce healthcare costs [34].
In this study, we used a fixed time-window approach to analyze the risk of ICU admission in HIV patients. A fixed observation window provides a relatively stable and representative observation interval. For example, we selected data from January 2009 to December 2020 as the observation window, during which we systematically collected the clinical data and regression information of all HIV patients who met the inclusion criteria. This fixed observation period allowed us to clearly observe the trends in ICU admissions in this group over a longer period of time, as well as the association of various risk factors associated with them. Some studies have shown that fixed observation windows help to capture the phases of disease development and provide a reliable database for the construction of prediction models, enabling us to optimize algorithms based on risk factors that improve the accuracy of predictions.
The multiple ultiple imputations by chained equations (MICE) algorithm was chosen over methods such as simple average imputation mainly because of the significant advantages of the MICE algorithm in dealing with complex datasets [35, 36]. MICE iteratively fits a predictive model for each variable with missing values conditional on other variables in the data, which takes into account correlations between variables and more realistically reflects the data distributions and underlying relationships [37]. Simple average imputation may cause the distribution of the variables to change, resulting in biased predictive models. From clinical studies, missing data are often not completely random and are affected by a variety of clinical factors such as disease severity, etc. The MICE algorithm can more accurately simulate the real data situation and retain the intrinsic relationship between clinical variables, thus providing a more reliable data base for subsequent machine learning models and enhancing the accuracy and robustness of machine learning models in predicting the risk of ICU admission for HIV-infected patients.
Machine learning applications in medicine have potential value in the diagnosis, treatment, and prognosis of various diseases. An artificial neural network (ANN) [38] is a machine learning algorithm based on the structure and behavior of neurons in the brain, which has a significant impact on the prediction of chronic disease progression and therapeutic effect [39,40,41]. Consistent with Pienaar et al.‘s study on mortality prediction in pediatric intensive care units [42], our study of 1,530 AIDS patients (7:3 training-to-testing ratio) confirmed the superiority of ANN over LOG/RF/KNN/SVM/XGB models using the best performance metrics (lowest Brier score, highest AUROC/AUPRC). Morgan et al. [43] indicated that with an increase in sample size, machine learning models presented more accuracy and stability, less over-fitting, and more consistency with internal validation. As the sample size increased, the metrics in the machine learning algorithms enhanced model stability and generalizability, including the Brier score, AUROC, and AUPRC.
The Brier score [27] refers to the mean square deviation between the predicted probability and the observed outcome and has a value ranging from 0 to 1. Lower Brier scores indicated better model performance. Receiver operating characteristic curve (ROC) and precision-recall curve (PRC) analysis [28,29,30] are more appropriate and preferable techniques for assessing the diagnostic and predictive accuracy of a disease, with values ranging from 0 to 1, and a larger area under the curve indicating better performance. In this study, the five machine learning models, LOG, RF, KNN, SVM, and XGB, were not optimal owing to their higher Brier scores and smaller ROC-AUC and PR-AUC compared with the ANN model (Brier score = 0.034, ROC-AUC = 0.961, PR-AUC = 0.895).
Traditional statistical models such as logistic regression have many drawbacks in predicting the risk of ICU admission in HIV patients. Traditional models are usually based on the assumption of linearity, and it is difficult to accurately fit such a complex pattern because the immune system is impaired by HIV infection, which involves changes in the viral load, and the effects of various comorbidities, such as lung infections and cardiovascular diseases, which are not simply linearly related. In contrast, machine Learning (ML) is expected to show greater predictive power, with ML algorithms being able to automatically learn complex patterns in data without having to assume a particular form of data relationship, such as Deep Neural Networks, which can mine features and associations hidden behind large amounts of data by their multilayered neuron structure [44]. Several studies [45, 46] have indicated that machine learning models significantly outperform traditional statistical models in predicting event risk and prognosis scenarios, providing a more reliable basis for clinical decision-making, helping healthcare professionals intervene in advance, and optimizing the allocation of healthcare resources.
This study is the first ML model developed in China to predict ICU admission risk. Previous studies have shown the risk factors [47, 48] and mortality rates [49] of HIV-infected patients admitted to intensive care units (ICUs); however, few studies in China have predicted the importance of risk factors for ICU admission of HIV-infected patients using machine learning algorithms. Chiang et al. [50] indicated that respiratory failure was the most common reason for ICU admission for PLWHs, and a low CD4 cell count was an important cause of death among PLWHs who stayed in the ICU. Akgün et al. [9] indicated that PLWHs, especially those diagnosed with different respiratory system infections, were more prone to be admitted to the ICU and had higher mortality rates than the general population. Kozhevnikova et al. [51] indicated that pneumocystis carinii pneumonia(PCP) was the main cause of respiratory failure among PLWHs, which increased the likelihood of ICU admission among these patients. In addition, various opportunistic infections, central nervous system diseases, and AIDS-related cancers are reasons for ICU admission [48]. In this study, based on the LASSO regression algorithm, we also found that respiratory failure, multiple opportunistic infections in the respiratory system, AIDS-defining cancers, baseline viral load, PCP, baseline CD4 cell count, unexplained infections, cardiovascular diseases, non-AIDS-defining events, Mycobacterium avium complex infections, multiple opportunistic infections of the CNS were significantly correlated with ICU admissions, 6 ML algorithms were further evaluated and the optimal ML model, ANN, was established, which can be used to develop diagnostic, treatment and prevention strategies for PLWHs and to reduce likelihood of ICU admission and healthcare costs.
Traditional AI models, particularly deep neural networks, are often seen as ‘black boxes’ that make accurate predictions; however, it is difficult to understand how they get these results. Explainable Artificial Intelligence (XAI) [52,53,54,55] refers to a range of methods and techniques designed to make the decision-making processes of AI models human-understandable, aiming to sense the inner workings of the black box, determine which input features have the greatest impact on the system’s outcomes or outputs, and ultimately make the system interpretable and decipherable. A variety of well-established XAI methods and techniques, such as Local Interpretable Model-agnostic Explanations (LIME) [56], Gradient Class Activation Mapping (GRAD-CAM) [57], Shapley Additive ExPlanation (SHAP) [31, 32], and Deep Learning Important FeaTures (DeepLIFT) [58] have been employed for prediction tasks across various domains.
Machine learning models have high accuracy; however, their intrinsic prediction process is not visible, and they are black-box models. To enhance the explainability of the models, this study introduced Shapley (SHAP) values for feature importance analysis. The Shapley value was calculated based on the marginal contribution of each participant to the outcome, taking into account all possible permutations of participants; however, the Shapley value also has limitations, such as a large amount of calculation [59]. Figure 4 depicts a summary plot of the SHAP values for the features, with the SHAP value for each feature plotted according to its positive or negative contribution. Positive and negative values on the x-axis indicate a positive or negative impact on the target variable, respectively. Thus, the main reasons for the increased risk of ICU admission in HIV-positive patients, in descending order, were diagnosis of respiratory failure, multiple opportunistic infections in the respiratory system, AIDS-defining cancers, baseline viral load, PCP, baseline CD4 cell count, unexplained infections, cardiovascular diseases, non-AIDS-defining events, M. avium complex infections, and multiple opportunistic infections of the CNS. The current SHAP analyses provide personalized analyses and allow for precise and specific management of the disease in PLWHs.
The immune system of patients with HIV is compromised, and once signs of respiratory failure appear, their condition may progress extremely rapidly and dangerously, resulting in admission to the ICU [47]. The early recognition of respiratory failure allows doctors to quickly adjust oxygen therapy strategies if signs of respiratory failure are detected ahead of time, thereby improving oxygenation and avoiding adverse reactions caused by hypoxia. In terms of airway management, as patients with HIV are at risk of sputum viscosity and coughing due to weakness and prolonged bed rest, doctors may be able to assess the patient’s airway condition in advance and administer nebulization therapy earlier to prevent sputum from obstructing the airway, thereby reducing the risk of exacerbation of lung infections and respiratory failure. In terms of pharmacological interventions, early recognition of respiratory failure allows physicians to adjust the synergistic use of anti-HIV medications with other symptomatic therapies more rationally. In conclusion, early identification of respiratory failure in patients with HIV allows physicians to take intervention measures in advance at various key medical stages; reduces the difficulty of treatment, morbidity, and mortality after admission to the ICU; and improves the overall success rate of life-saving treatment.
In terms of advantages, we first used machine learning classification models to predict individual risk factors for ICU admission, which helped adequately assess the risk of ICU admission in PLWHs. Second, internal validation was conducted on the predictive model using 11 years of clinical data and different ML algorithms, indicating good generalization, accuracy, and confidence in the predictive model.
This study had several limitations. First, this was a retrospective cohort single-center study, and clinical bias was inevitable. Second, model overfitting continues to be a significant problem. Overfitting describes the phenomenon where highly predictive models on training data do not generalize well to future observations, which limits the practical application of the model in real clinical situations [60]. In terms of ethics and implementation, integrating machine learning tools into clinical workflows must address patient data privacy breaches and physician skepticism. This study was not externally validated, and the conclusions of this study should be further investigated using a random and representative sample of the population in a multicenter study.
In terms of data, hospitals and clinics from different regions pool their resources to create larger and more heterogeneous datasets. By incorporating a wider range of patient characteristics and follow-up data, models can be trained to better derive the ICU risk associated with HIV patients. Develop clinician-friendly interfaces or applications that can display personalized risk trajectories using waterfall plots (SHAP values over time) and embed treatment recommendations through pop-ups linked to guidelines for early prevention in high-risk HIV patients. Clinical pilots, such as anonymous risk scores, can also be conducted in collaboration with hospitals to test ANN models in prospective trials, reduce delays in ICU admissions, and address stigma.
In this study, we established machine learning models that significantly contribute to the prediction of risk factors for ICU admission among PLWHs in China. The predictive models would help healthcare professionals rationalize the diagnosis, treatment, and care plans for PLWHs, consequently lessening their suffering and promoting early recovery. They could also be used to assess the likelihood and risk of ICU admission for HIV-positive patients.