Abstract
The ISARIC4C consortium developed and internally validated the 4C Score for prediction of mortality only in hospitalized patients. We aimed to assess the validity of the 4C Score in mortality prediction of patients with COVID-19 who had been home isolated or hospitalized.
This retrospective cross-sectional study was performed after the first wave of COVID-19. Data of all PCR-positive COVID-19 patients who had been discharged, hospitalized, or died were retrospectively analyzed. Patients were classified into four risk groups according to the 4C Mortality Score. A total of (506) patients were classified as follows: low (57.1%), intermediate (27.9%), high (13%), and very high (2%) risk groups. Clinical, radiological, and laboratory data were significantly more severe in the high and very high-risk groups compared with other groups (p<0.001 for all). Mortality rate was correctly estimated by the model with 71% sensitivity, 88.6% specificity, and area under the curve of 0.9. The mortality rate was underestimated among the very high-risk group (66.2% vs 90%). The odds of mortality were significantly greater in the presence of hypoxia (OR 2.6, 95% CI 1.5 to 4.6, p<0.001) and high respiratory rate (OR 5.3, 95% CI 1.6 to 17.9, p<0.007), C reactive protein (CRP) (OR 3.5, 95% CI 1.8 to 6.8, p<0.001), and blood urea nitrogen (BUN) (OR 1.9, 95% CI 1.3 to 3.1, p<0.002). Other components of the model had non-significant predictions. In conclusion, the 4C Mortality Score has good sensitivity and specificity in early risk stratification and mortality prediction of patient with COVID-19. Within the model, only hypoxia, tachypnea, high BUN, and CRP were the independent mortality predictors with the possibility of overlooking other important predictors.
Significance of this study
What is already known about this subject?
The ISARIC4C consortium developed the 4C Score to predict the mortality among hospitalized patients with COVID-19.
It was based on a prospective cohort study that included 74,944 consecutive patients across 260 hospitals across England, Scotland, and Wales.
The 4C Mortality Score includes eight predictors of mortality: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, Glasgow Coma Scale, urea serum level, and C reactive protein level.
The 4C Mortality Score ranges from 0 to ≥15 and it divides patients into four risk groups: low (0–3), intermediate (4-8), high (9-14), and very high-risk groups (≥15). The corresponding predicted mortality is as follows: 1.2% for the low, 9.9% for the intermediate, 31.4% for the high, and 61.5% for the very high-risk groups.
What are the new findings?
The ISARIC4C score was mainly internally validated, and it predicted mortality only in hospitalized patients.
In our study, we retrospectively applied this score on all confirmed COVID-19 cases which are either home isolated or hospital admitted.
Moreover, we assessed its external validity.
As far as we know, the validity of the score in mortality prediction has been not previously assessed in Saudi Arabia.
Significance of this study
How might these results change the focus of research or clinical practice?
The 4C Mortality Score performed relatively well in early risk stratification of patients with COVID-19 with good sensitivity and specificity in mortality prediction.
Within the score, hypoxia, tachypnea, high BUN, and CRP were the significant independent predictors of mortality.
There is a possibility to consider other important early predictors especially D-dimer. Still, there is an urgent need for a better scoring system that can be used in early stages to help in decision-making to stratify those at high risk that may require more careful assessment and earlier intervention.
Introduction
COVID-19 is a pandemic infectious disease caused by the SARS-CoV-2 coronavirus.1 To date, COVID-19 is still claiming the lives of thousands of people worldwide. While most of the cases present with mild to moderate symptoms require no special treatment, a minority develop acute respiratory distress syndrome, multiorgan failure, or have fatal outcomes.2 The case mortality rate is highly variable, ranging from about 0.5% to 10%, and it is reported to be higher than 20% in hospitalized patients.2
One of the most important questions to be addressed in this pandemic is “what are the risk factors for severe illness or death?”.3 Currently, identification of clinical and laboratory markers to rapidly and accurately stratify the risk is warranted. This identification could guide and support clinical decision-making. At the onset of the pandemic, there was a lack of COVID-19-specific risk stratification tools. During the crisis, multiple prognostic scores have been launched by different researchers. The MuLBSTA score by Guo et al 4 considered multilobar infiltrates, lymphocytic count ≤0.8×109/L, bacterial infection, smoking status, hypertension, and age ≥60 years as the poor prognostic factors. The COVID-19‐Gram Critical Illness Risk Score,5 the Quick COVID-19 Severity Index,6 and the COVID-19 Severity Index7 were all constructed to early identify hospitalized patients with an increased risk of critical illness and transfer to intensive care unit (ICU).
One mortality predictor score of hospitalized patients with COVID-19 was developed by the ISARIC4C consortium and was termed as the Coronavirus Clinical Characterization Consortium Mortality Score (4C Mortality Score).8 It was based on a prospective cohort study that included 74,944 consecutive patients across 260 hospitals. However, the use of these scoring models was mostly for hospitalized or critical patients, with none addressing patients ahead of admission. Moreover, they all have been internally but not externally validated. We still need a rapid and accurate tool to early stratify patients infected with COVID-19 in clinical practice. Therefore, this study aimed to, first, determine the performance characteristics and mortality prediction of one of the available prognostic models, the 4C Mortality Score, in patients with COVID-19 at the time of diagnosis for both hospitalized and non-hospitalized patients. Second, to investigate potential components of this prognostic tool in mortality prediction.
Patients and methods
Study design
This was a retrospective cross-sectional study using an electronic medical record review. Clinical data sources were from King Khalid Hospital in Hail and Ibn Sina College Hospital in Jeddah.
Inclusion and exclusion criteria
The study included medical records of all adult patients with confirmed COVID-19 infection who were diagnosed during the first wave (from March 2, 2020 when the first case in Saudi Arabia was confirmed until the end of August 2020 when there was a marked drop in reported cases). They were either home isolated or admitted to hospitals including those admitted to ICU. A confirmed case of COVID-19 was defined by a positive reverse transcriptase-PCR (RT-PCR) assay of a nasopharyngeal swab associated with compatible clinical manifestations. Incomplete electronic records were excluded from the analysis.
Data collection
Demographic, clinical, laboratory, radiological, and outcome data were collected by using a prespecified case report form. The demographic data included age, gender, nationality, and occupation. Clinical data included smoking status, history of travel or contact, and comorbidities in addition to the main presenting symptoms, signs, and admission data (home isolation, hospitalization, ICU). Laboratory data, ECG, and radiological reports were also extracted from the electronic medical records. Laboratory results included basic investigations and inflammatory markers. All clinical and laboratory information was collected starting from the first day of the presentation. Treatment received during home isolation or hospitalization was also collected. Treatment included lopinavir/ritonavir, anticoagulants, immunomodulatory therapy such as dexamethasone and interferon 1B, oxygen therapy, and ventilatory support (invasive or non-invasive mechanical ventilation). Outcome data included mainly mortality during the period of home isolation or hospitalization.
Scoring system: the 4C Mortality Score
We choose the 4C Mortality Score because it is valid, simple, easy-to-use, freely available, could be applicable retrospectively, and it is based on commonly available parameters at presentation. It includes eight predictors of mortality: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, Glasgow Coma Scale (GCS), urea serum level, and C reactive protein level.8 Scoring was performed as follows: 0 for 18–49 years old, 2 for 50–68 years old, 4 for 70–79 years old, and 7 for ≥80 years old, 1 for males, 0 for females, 1 for 1 comorbidity, 2 for ≥2 comorbidities, 0 for respiratory rate <20, 1 for 20–29 scores, 2 for ≥30, 0 for oxygen saturation >92%, 2 for <92%, scores 2 for GCS <15, 0 for urea <7 mmol/L, 1 for 7–14 mmol/L, 3 for >14 mmol/L, 0 for CRP <50 mg/dL, 1 for 50–99 mg/dL, and 2≥100 mg/dL.
The 4C Mortality Score ranges from 0 to ≥15 and it divides patients into four risk groups: low (0–3), intermediate (4–8), high (9–14), and very high-risk groups (≥15). The corresponding predicted mortality is as follows: 1.2% for the low, 9.9% for the intermediate, 31.4% for the high, and 61.5% for the very high-risk groups.
Ethical issue
The authors ensured confidentiality of all obtained data from patients’ medical records. The study received ethical approval from the Ethics Committee for Research at Ibn Sina National College for Medical Studies, Jeddah, Saudi Arabia as well as the local institutional review board in Hail health affairs via review according to KACST GCP regulations (IRB registration number H-08-L-074).
Statistical analysis
Statistical data were analyzed, and figures were constructed by SPSS software (V.22.0; SPSS). Data were reported as the number and the frequency for categorical variables or the median and the IQR for abnormally distributed numerical variables. For comparisons of the four risk groups, the χ2 test and the non-parametric test (Kruskal-Wallis H test) were used according to the tested variables. The diagnostic accuracy of the likelihood of mortality (sensitivity, specificity, and area under the curve) of the 4C Mortality Score was performed using the receiver operating characteristic (ROC) curve. Logistic regression analysis was performed to ascertain the effects of the eight components of the 4C Mortality Score on the likelihood of mortality from COVID-19 with the estimation of the exponential beta which was considered as the OR and the 95% CI. All significance tests were two tailed and were conducted at a minimum of 0.05 level.
Results
A total of 573 medical records of patients with COVID-19 were reviewed and 67 records were excluded due to incomplete data. Finally, a total of 506 medical records were included; 67% were aged less than 50, 67.2% were females, 43.8% were Saudis, 22.3% were smokers, 34.7% were healthcare workers, and 20.2% had jobs. According to the 4C Mortality Score, they were classified into low 289 (57.1%), intermediate 141 (27.9%), high 66 (13%), and very high 10 (2%) risk groups. There were statistically significant differences between all groups regarding their age, gender, nationality, occupation, and the number of comorbidities. The old jobless Saudi males with comorbidities predominated significantly in the high-risk groups (p<0.001 for all) (table 1).
Baseline demographic characteristic data of the different COVID-19 risk groups
There was a statistically significant difference in the clinical and radiological characteristics of patients in different risk groups. Mild symptoms like sore throat, myalgia, gastrointestinal symptoms, and runny nose predominated significantly in the low and intermediate-risk groups, while severe symptoms like cough (p<0.008) and dyspnea (p<0.001) predominated in the high-risk groups. Similarly, bilateral involvement of the lung was statistically significantly higher in high-risk groups compared with other groups (p<0.001 for all) (table 2).
Baseline clinical and radiological characteristics of patients with COVID-19 in different risk groups
Leukocytosis, lymphopenia, thrombocytopenia, and inflammatory markers were significantly higher in the very-high risk group compared with other groups (p<0.001 for all) (table 3).
Baseline laboratory characteristics of patients with COVID-19 in different risk groups
Out of 506, 233 (46%) patients were hospitalized with 42 (8.3%) needed ICU on admission. About 84% of the low-risk group were isolated at home, while the remaining 15.9% were admitted to the hospital. Most of the intermediate group were hospitalized (68.7%) with 8% admission to ICU. While 9.6% of the high-risk group were isolated at home, 61.6% were hospitalized and 28.8% admitted to ICU. Ninety per cent of the very high-risk group was admitted to the ICU. Patients received different modalities of therapy according to the local Saudi Ministry of Health protocol.9 Of note, dexamethasone was given to seven cases and interferon-beta 1b was given to one patient in the low-risk group (table 4).
Admission, drugs, and mortality of patients with COVID-19 in different risk groups
The total mortality recorded was 31 cases (6.1%) with no reported mortality in the low-risk group. Mortality increased significantly across other risk groups (4.4%, 19.7%, and 90% respectively) with only one patient from the very high-risk group who survived. Of note, this patient was an elderly hospitalized, non-Saudi woman with hypertension and chronic renal disease. She was not mechanically ventilated. Compared with actual mortality, the 4C overestimated the mortality in all groups except the very high-risk group, where there was underestimation (1.2% vs 0%, 9.9% vs 4.4%, 34.9 vs 19.7%, 66.2% vs 90%, respectively) (p<0.001) (table 4).
The ROC curve of the 4C estimated mortality at cut-off value of 2.5 estimated mortality (table 5) which corresponds to the high-risk group. It showed a good predictivity with 71% sensitivity, 88.6% specificity, and 0.9 area under the curve (95% CI 0.859 to 0.954, and p<0.001). The score correctly estimated 95.3% of the mortality (figure 1 and table 5).
The receiver operating characteristic (ROC) curve for sensitivity, specificity and area under the curve (AUC) for the estimated predicted mortality by the 4C Mortality Score.
Coordinates of the overall ROC curve
The logistic regression model of the prediction of the mortality using the 4C individual scores was statistically significant, χ2(8)=142.8, p<0.001. The model explained 64% (Nagelkerke R2) of the variance in mortality and correctly classified 97.4% of cases. The risk of mortality was significantly greater in the presence of hypoxia (OR 2.6, 95% CI 1.5 to 4.6, p<0.001) and high respiratory rate (OR 5.3, 95% CI 1.6 to 17.9, p<0.007), CRP (OR 3.5, 95% CI 1.8 to 6.8, p<0.001), and BUN (OR 1.9, 95% CI 1.3 to 3.1, p<0.002) (table 6).
Significant predictors of COVID-19 mortality within the 4C Mortality Score (multivariate logistic regression analysis)
Discussion
In this study, patients with COVID-19 from two centers in Saudi Arabia were classified retrospectively using the 4C Mortality Score into four distinct risk groups, where the severity and mortality of the disease increased progressively across them. The 4C Score showed a good sensitivity and specificity with an underestimation of mortality among the very high-risk group. The risk of mortality was significantly greater in the presence of four out of eight components of the 4C Score with the possibility of overlooking other significant predictors.
For the first COVID-19 peak, Saudi Arabia recorded a total of 316,700 confirmed cases on August 31, 2020, with a total of 3897 recorded deaths and a 1.26% fatality rate with the maximum peak on June 7.9–11 This death rate is much lower than our results (6.1%) which could be simply explained by our study design as we extracted data without randomization and almost half of our patients were hospitalized. However, in support of our results, the mortality rate among hospitalized patients reached up to 20%.2
In this study, the 4C Score provided an acceptable specificity and sensitivity in mortality prediction among all patients. This will support its external validity as it had been validated internally.8 Compared with our study, an Italian study12 reported nearly similar sensitivity (88.1% vs 71%) but lower specificity (55.9% vs 88.6%). The same study considered this score as the most accurate mortality predictor compared with other scores like COVID-19‐Gram Critical Illness Risk Score,5 National Early Warning Score,13 and Quick COVID-19 Severity Index.6 Unfortunately, the 4C Score underestimated mortality risk among the very high-risk group with overestimation in other risk groups. Overestimation but not underestimation of risk among our patients is expected as the scoring model is based mainly on data of the admitted patients.
The 4C Score supported the physicians’ decision for admission with few exceptions as 16% of the low-risk group were hospitalized and 10% of the high-risk group were isolated at home. This may reflect improper risk stratification either by physicians or by the scoring model. The founders of the score recommended against its use for admission decision.8 The 4C score supported the physicians’ decision for treatment as well. However, dexamethasone was given to seven cases in the low-risk group. Again, this could reflect a problem in the risk stratification by physicians as WHO recommended the use of corticosteroids in the most seriously ill patients.14
Among its eight components, only hypoxia, tachypnea, high blood urea nitrogen (BUN), and CRP were the significant independent predictors of mortality. In our study, the respiratory rate was the best predictor and had a high probability in the prediction of mortality. Miller et al 15 found that early evaluation and monitoring of respiratory rate may be a useful indicator of prognosis and could be a part of the protocol used by medical professionals to identify individuals who could be home isolated or hospital admitted.
Results showed that BUN levels were increasing readily throughout the groups, indicating a worsening of renal function is proportional to the disease severity regardless of their pre-existing renal function status. Many autopsy data showed virus particles present in renal endothelial cells that may directly cause endothelial damage. The virus directly infects renal tubular epithelium and podocytes causing mitochondrial dysfunction and acute tubular necrosis.16 Other postulated mechanisms are renal hypoperfusion due to the low cardiac output caused by left ventricular failure, as well as right ventricular failure caused by sepsis from COVID-19 pneumonia leads to renal congestion.17
Although acute phase reactants, including CRP, are considered non-specific, they are reported as sensitive markers of acute COVID-19 disease. In this study, CRP levels were progressively increased with disease severity. Many studies showed that an elevated CRP was associated with poor outcomes and needs for ICU care.18 The cut-off values for elevated serum CRP varied widely among the studies. Many studies suggested elevated CRP (⩾10 mg/L) levels are associated with poor outcome.19
Oxygen saturation of ≤92% was one of the most powerful predictors in our study. Petrilli et al found that SaO2 <88% on admission was associated with mortality.20 In our study, the very high-risk group showed early hypoxia on admission, suggesting their late presentation to the hospital. This implies that we need a better, faster way to recognize hypoxemia in the community setting, which becomes challenging in the context of the “silent hypoxemia” that many patients with COVID-19 experience early in the course of the disease.21
Despite old males dominated significantly in the high-risk groups and the 4C model assigned an increased risk value for older aged male patients, age and gender were insignificant mortality predictors in our results. This might be explained by the inclusion of 67.2% females younger than 50 years in our study. Cunningham et al 22 demonstrated that age has no significant correlation with the severity and mortality in patients with COVID-19. They concluded that age may not be considered as an independent factor in the scores dataset or needed to be calibrated with referencing to comorbidity. On the other hand, Peckham et al 23 found that while males and females are at equivalent risk of infection, male sex is associated with the development of severe disease and death.
In this study, the number of comorbidities did not show significant mortality prediction. Inclusion of the number and not the type of comorbidity may be the reason for its limited predictability in the 4C model. Among our patients, hypertension was the most prevalent significant comorbid risk factor in all groups, especially among the two high-risk groups (84% and 100%, respectively). Similarly, in the Lombardy region of Italy, among 1591 ICU COVID-19 patients, 49% of them had hypertension.24 The causal relationship between hypertension and COVID-19 or its severity may be related to associated comorbidities and aging.
In our study, GCS was not much affected at presentation as only three cases in the very high-risk group showed an abnormality in the level of GCS. The impairment of consciousness in patients with COVID-19 is multifactorial in a context that typically combines sepsis, severe hypoxemia, multiorgan failure, ICU complication, and toxic or metabolic encephalopathy. All these complications are usually late in the disease course.
As pointed out by Wynants et al, “unreliable predictors could cause more harm than benefit in guiding clinical decisions”.25 Many other hematologic, biochemical and immune biomarker abnormalities, which are not included in the score, are associated with severe illness and mortality in COVID-19, and therefore, they might aid in the early classification of risk and prediction of mortality.26 In our study, the lymphocytic count, the D-dimer, lactate dehydrogenase (LDH), ferritin, troponin, and creatine phosphokinase levels all showed significantly higher levels across risk groups. Of particular interest the International Society of Thrombosis and Hemostasis recommended that patients with COVID-19 with markedly elevated D-dimer levels should be considered for hospitalization early in the course of the diseases irrespective of the severity of clinical presentation.27 Unfortunately, the ISARIC 4C model did not assess predictors such as LDH or D-dimer due to their limited availability.8 The most important outcome of early identification and risk stratification of patients is to choose the best therapeutic intervention as well as prevention of the side effect of unnecessary drugs. Unfortunately, our results showed that there were both an overestimation and underestimation of treatment in some groups as dexamethasone was given to 11 cases and interferon-beta 1b was given to one patient in the low-risk groups. Moreover, one case of very high-risk group was not admitted to ICU. A recent systematic review and meta-analysis study on pregnant patients with a confirmed COVID-19 infection suggests that overuse of some drugs specially antibiotics was associated with adverse pregnancy outcomes.28 The same study concluded that avoiding unnecessary treatments in pregnant women with COVID-19 by risk stratification may improve maternal and clinical outcomes.
Limitation
This study had some potential limitations to be considered while interpreting the results. First, the retrospective cross-sectional study design is commonly carried out during outbreaks and epidemics but still some confounders could be missed causing bias. We were unable to capture all relevant data or review all follow-up data. Also, we were unable to ensure good compliance with the local management protocol, especially among the home isolated patients. Prediction of mortality was made retrospectively with knowledge of outcome data which introduces bias. Second, the small sample size from only two centers could limit the generalization of the results to the whole kingdom.
Conclusion
In conclusion, the 4C Mortality Score can early characterize patients with COVID-19 with good sensitivity and specificity in mortality prediction. Within the score, hypoxia, tachypnoea, high BUN, and CRP were the significant independent predictors of mortality. Future prospective studies with larger sample sizes from multicenters are warranted to validate our results and to support the external validity of the score system on different COVID-19 strains.
Data availability statement
Data may be obtained from a third party and are not publicly available. All data relevant to the study are included in the article or uploaded as supplementary information. The authors confirm that the data supporting the findings of this study are available within the article and its supplemental material.
Ethics statements
Patient consent for publication
Ethics approval
This study has been approved by the Ethics Committee for Research at Ibn Sina National College for Medical Studies (H-04–24122020), Jeddah, Saudi Arabia as well as the local institutional review board in Hail health affairs via review according to KACST GCP regulations (IRB registration number H-08-L-074).
Acknowledgments
The authors would like to thank Mr Ahmed Islam Badawe for his technical support in data extraction and proofreading.
Footnotes
Contributors Study conception and design: RAEM and IS. Acquisition of data: EMA and HSAJ. Analysis and interpretation of data: IS, HMM, KH, and EBR. Drafting of manuscript: RAEM, IS, HMM, and EMA. Critical revision: IS, RAEM, HMM, EMA, HSAJ, and EBR. Guarantor: RAEM.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
This article is made freely available for personal use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.
https://bmj.com/coronavirus/usageReferences
