Correspondence on ‘Prospective predictive performance comparison between clinical gestalt and validated COVID-19 mortality scores’ by Soto-Mota et al ========================================================================================================================================================== * Héctor David Meza-Comparán * COVID-19 * prognosis * risk * research Dear Editor, I read the article ‘Prospective predictive performance comparison between clinical gestalt and validated COVID-19 mortality scores’ with great interest.1 The authors compared various COVID-19 mortality prediction models validated in Mexican patients — LOW-HARM, MSL-COVID-19, Nutri-CoV, and neutrophil-to-lymphocyte ratio (NLR) —, qSOFA, and NEWS2 against clinical gestalt to predict mortality among COVID-19 patients admitted to a tertiary hospital, concluding that clinical gestalt was non-inferior. I would like to comment on some issues with this article. It is unclear what “clinical gestalt” meant in the study since no formal definition was provided by the authors other than study procedures. Others have defined clinical gestalt as “a physician’s unstructured estimate”2 or an “overall clinical impression”.3 Additionally, it is not clear how the authors selected the prediction models to be evaluated. They mentioned that three models validated in datasets including Mexican patients were included; however, in the absence of clear inclusion criteria, other models validated in Mexican patients could have been left out. Thus, I performed a systematic search within the COAP search engine and LILACS of studies published to November 5, 2021 (figure 1). Nine studies describing 17 validated COVID-19 mortality prediction models within the Mexican population were identified (table 1),4–12 four of which were evaluated by Soto-Mota and colleagues (LOW-HARM, MSL-COVID-19, Nutri-CoV, and NLR).4 7 Therefore, the authors did not evaluate a number of the important prediction models validated in Mexican patients to predict mortality. ![Figure 1](/https://d3hme472k3gd2d.cloudfront.net/content/jim/70/4/972/F1.medium.gif) [Figure 1](/content/70/4/972/F1) Figure 1 Systematic search flowchart of studies included and reasons for exclusion. The search within COAPa was performed by using the keywords and Boolean operators (mortality) AND (mexico) OR (mexican). Within LILACSb, the keywords and Boolean operators (COVID-19) AND (mortality) AND (mexico) OR (mexican) were used; an affiliation country filter for “Mexico” was also applied in the latter case. These searches retrieved 778 records (610 and 168, respectively), of which 193 studies were retained for abstract and full-text screening. Nine studies describing 17 validated COVID-19 mortality prediction models within the Mexican population were identified. aCOAP is a daily-updated database with SARS-CoV-2 and COVID-19 published articles from PubMed, EMBASE and PsycINFO, and preprints from medRxiv and bioRxiv (further information at [https://ispmbern.github.io/covid-19/living-review/](https://ispmbern.github.io/covid-19/living-review/)). bLILACS is one of the most important and comprehensive databases of scientific information in Latin America and the Caribbean with more than 880 thousand records of peer-reviewed journals, thesis and dissertations, government documents, annals of congresses and books (further information at [https://lilacs.bvsalud.org/en/](https://lilacs.bvsalud.org/en/)). View this table: [Table 1](/content/70/4/972/T1) Table 1 COVID-19 mortality prediction models validated in Mexican patients, identified through the systematic search. Although the authors mentioned the median years of hospital experience (which could include medical internship and social service in Mexico) in medical residents who performed predictions, disclosing their corresponding postgraduate year (PGY) would have been important, since confidence of predictions was generally low in this study — only ~35% had >80% confidence of prediction. While they argued that “with the COVID-19 pandemic, clinicians of all levels of training started their learning curve at the same time”, senior residents are less likely to be under-confident compared with junior residents.13 Furthermore, the statement “no score was significantly better than clinical gestalt predictions” might be questionable, due to concerns regarding sample size. An inadequate sample size could have led to the inability to detect differences, especially since the authors used *easyROC* — an open web calculator that estimates, among others, sample sizes for non-inferior ROC comparisons — to estimate the sample size for their study. Of note, *easyROC* requires an input for the “smallest difference” between tests’ AUC, not the “maximal AUC difference” as the authors report. Most important is the fact that *easyROC* was not developed to estimate sample sizes to evaluate non-inferiority between prognostic predictive models; instead, it was developed to compare diagnostic test models.14 Finally, it is worthwhile mentioning that while in younger patients obesity is the strongest risk factor for short-term mortality,15 chronological age remains the single most important predictor of in-hospital COVID-19 mortality.9 ## Ethics statements ### Patient consent for publication Not applicable. ### Ethics approval This study does not involve human participants. ## Acknowledgments I would like to thank Javier Mancilla-Galindo and Ashuin Kammar-García for their invaluable comments and recommendations regarding the manuscript, as well as Vianey Fragoso-Saavedra for her support. ## Footnotes * Twitter @HectorMezaMD * Contributors HDM-C assumes sole responsibility for the drafting, writing and revising of the manuscript. * Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. * Competing interests None declared. * Provenance and peer review Not commissioned; externally peer reviewed. This article is made freely available for personal use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained. [https://bmj.com/coronavirus/usage](https://bmj.com/coronavirus/usage) ## References 1. Soto-Mota A , Marfil-Garza BA , Castiello-de Obeso S , et al . Prospective predictive performance comparison between clinical gestalt and validated COVID-19 mortality scores. J Investig Med 2022;70:415–20.[doi:10.1136/jim-2021-002037](http://dx.doi.org/10.1136/jim-2021-002037) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34620707 [Abstract/FREE Full Text](/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamltIjtzOjU6InJlc2lkIjtzOjg6IjcwLzIvNDE1IjtzOjQ6ImF0b20iO3M6MTg6Ii9qaW0vNzAvNC85NzIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 2. Lucassen W , Geersing G-J , Erkens PMG , et al . Clinical decision rules for excluding pulmonary embolism: a meta-analysis. Ann Intern Med 2011;155:448.[doi:10.7326/0003-4819-155-7-201110040-00007](http://dx.doi.org/10.7326/0003-4819-155-7-201110040-00007) pmid:http://www.ncbi.nlm.nih.gov/pubmed/21969343 [CrossRef](/lookup/external-ref?access_num=10.7326/0003-4819-155-7-201110040-00007&link_type=DOI) [PubMed](/lookup/external-ref?access_num=21969343&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) [Web of Science](/lookup/external-ref?access_num=000295893800005&link_type=ISI) 3. Dale AP , Marchello C , Ebell MH . Clinical gestalt to diagnose pneumonia, sinusitis, and pharyngitis: a meta-analysis. Br J Gen Pract 2019;69:e444–53.[doi:10.3399/bjgp19X704297](http://dx.doi.org/10.3399/bjgp19X704297) pmid:http://www.ncbi.nlm.nih.gov/pubmed/31208974 [Abstract/FREE Full Text](/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjkvNjg0L2U0NDQiO3M6NDoiYXRvbSI7czoxODoiL2ppbS83MC80Lzk3Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 4. González-Flores J , García-Ávila C , Springall R , et al . Usefulness of easy-to-use risk scoring systems rated in the emergency department to predict major adverse outcomes in hospitalized COVID-19 patients. J Clin Med 2021;10:3657.[doi:10.3390/jcm10163657](http://dx.doi.org/10.3390/jcm10163657) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34441957 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 5. Elghamrawy SM , Hassanien AE , Vasilakos AV . Genetic-based adaptive momentum estimation for predicting mortality risk factors for COVID-19 patients using deep learning. Int J Imaging Syst Technol 2021. doi:[doi:10.1002/ima.22644](http://dx.doi.org/10.1002/ima.22644). [Epub ahead of print: 13 08 2021].pmid:http://www.ncbi.nlm.nih.gov/pubmed/34518740 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 6. Quiroz-Juárez MA , Torres-Gómez A , Hoyo-Ulloa I , et al . Identification of high-risk COVID-19 patients using machine learning. PLoS One 2021;16:e0257234.[doi:10.1371/journal.pone.0257234](http://dx.doi.org/10.1371/journal.pone.0257234) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34543294 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 7. Bello-Chavolla OY , Antonio-Villa NE , Ortiz-Brizuela E , et al . Validation and repurposing of the MSL-COVID-19 score for prediction of severe COVID-19 using simple clinical predictors in a triage setting: the Nutri-CoV score. PLoS One 2020;15:e0244051.[doi:10.1371/journal.pone.0244051](http://dx.doi.org/10.1371/journal.pone.0244051) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33326502 [CrossRef](/lookup/external-ref?access_num=10.1371/journal.pone.0244051&link_type=DOI) [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 8. Karaismailoglu E , Karaismailoglu S . Two novel nomograms for predicting the risk of hospitalization or mortality due to COVID-19 by the naïve Bayesian classifier method. J Med Virol 2021;93:3194–201.[doi:10.1002/jmv.26890](http://dx.doi.org/10.1002/jmv.26890) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33599308 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 9. Mancilla-Galindo J , Vera-Zertuche JM , Navarro-Cruz AR , et al . Development and validation of the patient history COVID-19 (PH-Covid19) scoring system: a multivariable prediction model of death in Mexican patients with COVID-19. Epidemiol Infect 2020;148:e286.[doi:10.1017/S0950268820002903](http://dx.doi.org/10.1017/S0950268820002903) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33239114 [CrossRef](/lookup/external-ref?access_num=10.1017/S0950268820002903&link_type=DOI) [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 10. Kimura-Sandoval Y , Arévalo-Molina ME , Cristancho-Rojas CN , et al . Validation of chest computed tomography artificial intelligence to determine the requirement for mechanical ventilation and risk of mortality in hospitalized coronavirus disease-19 patients in a tertiary care center in Mexico City. Rev Invest Clin 2021;73:2.[doi:10.24875/RIC.20000451](http://dx.doi.org/10.24875/RIC.20000451) 11. Wollenstein-Betech S , Cassandras CG , Paschalidis IC . Personalized predictive models for symptomatic COVID-19 patients using basic preconditions: hospitalizations, mortality, and the need for an ICU or ventilator. Int J Med Inform 2020;142:104258.[doi:10.1016/j.ijmedinf.2020.104258](http://dx.doi.org/10.1016/j.ijmedinf.2020.104258) pmid:http://www.ncbi.nlm.nih.gov/pubmed/32927229 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 12. Márquez-Salinas A , Fermín-Martínez CA , Antonio-Villa NE , et al . Adaptive metabolic and inflammatory responses identified using accelerated aging metrics are linked to adverse outcomes in severe SARS-CoV-2 infection. J Gerontol A Biol Sci Med Sci 2021;76:e117–26.[doi:10.1093/gerona/glab078](http://dx.doi.org/10.1093/gerona/glab078) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33721886 [CrossRef](/lookup/external-ref?access_num=10.1093/gerona/glab078&link_type=DOI) [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 13. Fleming M , Vautour D , McMullen M , et al . Examining the accuracy of residents' self-assessments and faculty assessment behaviours in anesthesiology. Can Med Educ J 2021;12:17–26.[doi:10.36834/cmej.70697](http://dx.doi.org/10.36834/cmej.70697) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34567302 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom) 14. Goksuluk D , Korkmaz S , Zararsiz G , et al . EasyROC: an interactive web-tool for ROC curve analysis using R language environment. R J 2016;8:213–30.[doi:10.32614/RJ-2016-042](http://dx.doi.org/10.32614/RJ-2016-042) [CrossRef](/lookup/external-ref?access_num=10.32614/RJ-2016-042&link_type=DOI) 15. Vera-Zertuche JM , Mancilla-Galindo J , Tlalpa-Prisco M , et al . Obesity is a strong risk factor for short-term mortality and adverse outcomes in Mexican patients with COVID-19: a national observational study. Epidemiol Infect 2021;149:e109.[doi:10.1017/S0950268821001023](http://dx.doi.org/10.1017/S0950268821001023) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33913410 [PubMed](/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fjim%2F70%2F4%2F972.atom)