Abstract
To examine how to increase research career outcomes among medical graduates, we analyzed the impact of the Research Scholarly Concentration at The George Washington University School of Medicine and Health Sciences. Residency placement, subsequent scholarship, and career outcomes were compared among 670 graduates who participated in the elective Clinical and Translational Research Scholarly Concentration or no Concentration between 2009 and 2018. We conducted a retrospective study of residency match (highly selective vs less selective), job type (academic vs non-academic), and postmedical school publications (any vs none). We compared the outcomes between Research Scholarly Concentration graduates and those with no Concentration, matched by graduation year (n=335). For Research Scholarly Concentration graduates, we examined the association between research outcomes and duration of research experience before medical school (n=232). Research Scholarly Concentration graduates were more likely to place in a highly selective residency (40.2% vs 21.6%, p<0.0001), 68% more likely to publish after medical school (OR=1.68, 95% CI 1.10 to 2.58), and almost four times as likely to have taken an academic health center job (OR=3.82, 95% CI 1.72 to 8.46) than graduates with no Concentration. Surprisingly, the length of research experience before medical school was not associated with these outcomes among Research Scholarly Concentration graduates. This suggests that a medical school Research Scholarly Concentration is effective in training physician researchers and should be available to both novices and research-experienced matriculants. These data suggest how other medical schools might plan Scholarly Concentration programs to improve research outcomes among medical graduates.
Significance of this study
What is already known about this subject?
Many medical schools have Scholarly Concentration programs that expose students to areas of study outside the standard curriculum.
Surveys show Research Scholarly Concentration participants report high satisfaction rates, increased research productivity, improved confidence in the ability to carry out research, and increased interest in future participation in research.
Few studies analyzed objective measures of research outcomes.
What are the new findings?
Research Scholarly Concentration graduates were more likely to place in a highly selective residency, more likely to publish after medical school, and almost four times more likely to take an academic health center job than graduates with no Concentration.
The duration of previous research experience before medical school was not associated with these research outcomes.
Relatively brief research exposure during medical school fosters continued postgraduate research.
How might these results change the focus of research or clinical practice?
Medical school Research Scholarly Concentrations can increase research outcomes, even when the graduates have limited prior research experience.
Research during medical school may be more important than previously recognized in leading to careers as clinician-investigators.
Other medical schools may use these results to plan Scholarly Concentration programs that improve research outcomes.
Introduction
Many medical schools have designed Scholarly Concentration (SC) programs to enrich students’ experiences and expose them to concentrations of study beyond the core curriculum. SC participants report high satisfaction rates,1–7 increased healthcare perspective and leadership experience,8 increased research productivity,4 9 10 improved confidence in the ability to carry out research,5 11 and increased interest in future participation in research.4 6 12 13 Because SC programs vary widely in focus and structure, it has been challenging to understand their impact on student outcomes. In addition, it is not clear whether to focus resources on structured ‘deep dives’ for experienced students, or on more heterogeneous ‘exposure’ models that anticipate participation by both naïve and experienced researchers.
Medical schools have created similar programs that take advantage of the summer between medical school year 1 (MS1) and medical school year 2 (MS2) for major activities, supported by longitudinal lectures, workshops, and elective clerkship activities. At The George Washington University (GW) School of Medicine and Health Sciences (SMHS), there are nine elective SCs, including the Clinical and Translational Research (CTR) which involves a mentored basic, clinical, or translational research project. Participation in the CTR follows an application and introductory evidence-based medicine-specific and program-specific lectures in MS1. The major experiential mentored research project in the summer between MS1 and MS2 is followed by an abstract and poster presentation, and continued research is encouraged in short clerkship experiences in years 3 and 4. A final scholarly project is due at the end of the fourth year and emphasizes national presentations and publications. Goals for the program include the opportunity to engage in mentored, inquiry-based research; to apply the scientific method; to experience professional communication skills in reporting research results; to evaluate evidence important to clinical practice; and to understand contributions by members of a research team. Faculty and students articulate additional long-term goals such as matching in competitive residencies and embarking on careers at academic health centers. Successful completion of the program is accompanied by designation on the Doctor of Medicine (MD) diploma. The GW CTR SC began in 2009 and remains popular, with about 20% of first-year medical students selecting this concentration.
The CTR SC is part of a larger effort at GW to promote and sustain the development of MD clinician-investigators, and additional opportunities are often combined with the CTR SC. The GW Mentored Experience To Expand Opportunities in Research (METEOR) Program is a competitive prematriculation research fellowship opportunity for under-represented-in-medicine students admitted to the MD Program that feeds into the CTR SC. The WT Gill Summer Fellowship and other internal fellowships or extramural funding support select summer research internship opportunities at the GW, Children’s National Health System, and the Washington DC Veterans Affairs Medical Center, and many of these are earned by CTR SC students. Some students pursue a joint 7-year BA/MD program of the GW Columbian College of Arts and Sciences and the SMHS, or the 8-year BS/MD program of St Bonaventure University and the SMHS.
To determine the impact of the CTR SC on research outcomes, we compared milestones in a clinical research career including residency matches, publications, and subsequent careers between CTR SC graduates and graduates who were in no SC. We also examined the role of premedical school research duration in contributing to outcomes among CTR SC graduates. This analysis was designed to better understand the research outcomes achieved by medical graduates as we consider steps for program enhancement and training of future MD-trained researchers.
Materials and methods
Study population
The names of all graduates in the GW CTR SC since its inception in 2009 were collected from the Offices of Professional Enrichment and Student Affairs (n=335), and a similar sample of GW medical graduates who elected no SC in the same years was collected (n=335). Graduates with no SC each year were selected randomly by choosing every third graduate when sorted alphabetically.
Student premedical school research characteristics
For 236 graduates in the classes of 2013–2018, the original CTR SC applications were available, including a curriculum vitae (CV) reflecting research and educational activities before medical school matriculation. Research experience was quantified as full-time months, using several assumptions. Unless otherwise specified, (1) summer research was considered full-time 2.5 months in duration, and (2) academic year research was considered 9 months at 10 hours per week, or equivalent to full-time 2.25 months for undergraduates and 20 hours a week for master’s students. For postbaccalaureate research not associated with a degree, research was full time for the months indicated. Research experience was included if it related to the sciences, medicine, or public health. Three graduates had very long previous research experience before beginning medical school (57, 66, and 69 months). For the tests of statistical associations, these graduates were recoded to the next highest research duration (43.25 months) to avoid a disproportionate influence on the results. The undergraduate institution of each student with a CV was recorded and categorized for research strength using Carnegie Basic Classifications.14
Medical specialties
Medical specialty residencies were collected from public sources as intermediate outcomes. Residencies were coded to match the American Board of Medical Specialties member boards. If a graduate had a transitional year followed by a residency, only that residency was included. Two graduates matched to dual-board residencies and were excluded from the analysis. Residencies were then categorized as ‘highly selective’ versus ‘not highly selective.’ The mean United States Medical Licensing Examination (USMLE) step 1 scores for those who successfully matched in 2018 were used to categorize ‘highly selective’ residencies, with a mean score of 235 or higher, and the relative risk comparing CTR versus no SC graduates was determined.
Academic occupations
Public sources, such as Google, LinkedIn, and Doximity, but excluding social media sites, were used to collect occupational information in February 2018. Post-training ‘job titles’ and ‘job organizations’ were used to identify positions in academic and private practice sectors. Following this general approach, a second search was made for each graduate with specific queries for ‘professor’, ‘instructor’, and ‘faculty’ to address academic health center appointments. Jobs were classified as academic if they included these terms. Because these sites include self-reported information, university faculty web pages were used to validate academic appointments. Over 90% of academic appointments were confirmed by university websites, and 9% of academic appointments were confirmed using additional non-university web pages. If a graduate appeared to have multiple positions that included an academic position, they were considered academic. Note that this analysis represents a snapshot in time for each individual, as public information about jobs often lacks a posting date by individuals and institutions, and graduates may have undertaken subsequent positions after the query date.
Publications
Scopus was used to obtain scholarly publications for each graduate, in a search done in April–June 2018. Documents categorized as editorials, errata, and book chapters were excluded from publication lists because they are not peer-reviewed, but these lists did contain clinical case reports. In a separate analysis, publication types from the graduating classes of 2009 and 2010 were examined, and 13% of their records consisted of case reports. Publications were further segregated by when they were published in graduates’ education/career using general criteria. Any publications up to fall of the second year of medical school were considered a result of research performed before medical school. Publications from spring of the second year of medical school through 1 year postmedical school were considered to result from research in medical school. Publications from 2 to 3 years postgraduation were scrutinized for author affiliations to determine whether they were the product of medical school or postmedical school research. Publications appearing beyond 3 years after graduation were considered as postmedical school.
Statistical analyses
SAS V.9.4 was used for bivariate and multivariable analyses. The first set of analyses compared CTR SC with no SC graduates. A χ2 test was used to compare the proportion of graduates who matched to a highly selective residency. Logistic regression was used to compare the odds of having any postmedical school publication, with graduation year as a covariate. Logistic regression was also used to compare the odds of an academic (vs non-academic) job placement. Only 2009–2012 graduates in post-training jobs were included, and graduation year was used as a covariate.
The second set of analyses involved only the 236 CTR SC graduates for whom CVs were available and tested the effect of premedical school duration of research experience, measured in full-time-equivalent research months, on residency match and any postmedical school publication. Logistic regression was used to analyze these outcomes, with graduation year included as a covariate for any postmedical school publication. CVs were not available for 2009–2012 graduates; therefore, the impact of premedical school research months on their subsequent job placement could not be assessed.
Results
Overall, 335 graduates from CTR SC from 2009 to 2018 were compared with the same number of graduates in each year with no SC (figure 1). More CTR SC graduates participated in the BA/MD or BS/MD program (54 CTR SC vs 28 no SC) and the METEOR program (6 CTR SC vs 0 no SC). CTR SC graduates were also more likely to take a year-out research fellowship than graduates with no SC (18 people in the CTR SC compared with 7 graduates with no SC).
CTR SC graduates had more publications than their no SC peers at every career stage tested (figure 2). Most graduates had 0–1 publications overall (CTR SC: 0–62 total publications with a median of 1; no SC: 0–42 total publications with a median of 0; figure 3). CTR SC graduates were also more likely to have any publications, as almost two-thirds (66%) of CTR SC graduates had at least one publication, compared with 44% of graduates with no SC. Career scholarship also differed, as CTR SC graduates were 68% more likely to have authored or coauthored a publication following medical school graduation, when adjusted for graduation year (OR=1.68, 95% CI 1.10 to 2.58, n=653; figure 2). As a group (n=85), ‘high-publisher’ graduates with five or more total publications were likely have completed the Research SC (62% of high publishers) and taken a research year-out (9% of high publishers, compared with 3% of those with four or fewer total publications). Of the three graduates in our study with the highest number of publications (50 or more), all completed the Research SC, one completed a research year-out, two completed the BA/MD program, none published before medical school, and all matched with highly selective residencies (surgery, radiology, and urology); today, one is in an academic position and two are still in training. Thus, it seems likely that the medical student research experience had a durable impact on scholarly output.
Undergraduate experience before medical school may influence MD research careers. The majority of CTR SC graduates came from undergraduate institutions with high research strength (69% from doctoral institutions). Of the graduates who came from institutions with lower research strength (baccalaureate and master’s institutions), 14% went on to academic careers (compared with 11% in academic careers from doctoral institutions). Although the impact of BA/MD and BS/MD was not evaluated independently of other factors, more than half (66%) of these dual-degree graduates in our study participated in the CTR SC. Graduates of these dual-degree programs tended to have more total publications, more research years out, and a prevalence of academic careers, although these numbers and differences were small.
Among the 236 CTR SC graduates for whom CVs were available, 85% reported research activity as an undergraduate or postbaccalaureate (figure 4). Among those conducting research, the median duration of research prior to medical school was 7.5 months (IQR: 4.1–16.0). To our surprise, the number of premedical school research months was not associated with postmedical school publications (OR=0.98, 95% CI 0.93 to 1.03, p=0.36, n=232) or with residency match (OR=1.00, 95% CI 0.97 to 1.02, p=0.80, n=232).
More CTR SC graduates matched to residencies in diagnostic radiology, dermatology, surgery, otolaryngology, urology and ophthalmology than no SC graduates, and fewer CTR SC graduates matched to pediatrics, family medicine, emergency medicine, obstetrics and gynecology than no SC graduates (figure 5). Research SC graduates were almost twice as likely to be placed in a highly selective residency compared with graduates with no SC (40.2% vs 21.6%, p<0.001, n=657). Among 2009–2012 graduates who were no longer in training positions (n=119), CTR SC graduates were almost four times as likely to have taken an academic health center job versus a non-academic job, compared with graduates with no SC (figure 6; OR=3.82, 95% CI 1.72 to 8.46).
Discussion
SCs have become a common approach to stimulate medical student inquiry. Many SC programs involve in depth study beyond the medical school curriculum in a variety of concentration areas, and programs may be elective or mandatory.15 16 Concentrations often include faculty mentorship, an experiential activity in the summer between MS1 and MS2, follow-on activities, and a required scholarly paper or presentation.15–18 Despite their ubiquity, it has been difficult to attribute outcomes to specific SC program features, in part because little was known about the input characteristics of participants, the efficacy of specific program interventions, and the outcomes of participants. The goals for Research SCs often emphasize the qualitative benefits of research to the participant, such as heightened analytical skills, ability to work in teams, enhanced independent direction, and ability to apply knowledge to clinical care. The actual outcomes of SC participation, however, may take years to be observed, and are often not rigorously evaluated.17 The Scholarly Concentration Collaborative of 31 institutions may be in a position to collate best practices and outcomes among multiple institutions. The use of a logic model as proposed by Havnaer et al 17 may help to align goals, interventions, and evaluation outcomes across programs. This study was designed to assess the outcomes of our CTR SC so that we might enhance the program in the future and improve research training of medical graduates.
Participation in the SC program was correlated with increased research outcomes, including more publications after medical school and almost four times higher likelihood of obtaining a job in an academic health center. This is in line with studies showing that MDs who conduct research during medical school are more likely to conduct postgraduate research and publish throughout their careers.10 12 13 19 20 A previous study suggests that a passion for research is the strongest predictor of future careers in research.21 Satisfaction with an SC program has been shown to increase desire to continue in research,6 and ‘exposure’ models of research experience may be adequate to foster postgraduate research.
We were surprised to find that the duration of research experience before medical school was not correlated with CTR SC graduate outcomes. About 85% of our CTR SC graduates reported a median of 7 months of research before medical school, similar to the Association of American Medical Colleges (AAMC) reports of research experiences nationally.22 The GW CTR SC encourages both individuals with substantial research activity as well as those with limited research experience to pursue SC. This range of experience poses significant curricular challenges for the SC, as some students need and request research skill-building support, while other more experienced students seek focused projects and career advice. However, similar outcomes from both research-experienced and research-naïve medical school graduates suggest that the CTR SC is equally or more effective than research experience occurring at earlier career stages in providing the benefits of publication, residency, and academic outcomes.
This study also highlights the conflicting motivations within students, which can lead to strains with their research mentors. Alberson et al 1 suggest that medical students appear to fall into two groups with different goals for their research activities. Those students interested in career-long research valued both skill-related and accomplishment-related goals (e.g., learning to create and present a poster, or develop a research question and appropriate methods). By contrast, other students placed value on accomplishment-related goals (such as publishing a manuscript or giving a poster at a national meeting). Despite mixed reports on the influence of research experience on residency match,9 23–27 some students may pursue research experience not for a research career, but as a means to enhance their residency applications.21
When considered broadly, Research SCs may offer an important opportunity to address the shortage of clinician-investigators. For example, medical students might discover a passion and aptitude for inquiry-based research and wish to pursue it in a career. Previous studies have found mixed impact of research on academic careers. One study found that neurosurgery residents were more likely to choose academic careers if they had published prior to residency.28 Another study found that, although research during medical school leads to increased involvement in postgraduate research, it did not lead to a higher rate of academic appointments.13 Although we found that CTR SC graduates were more likely to place in a highly selective medical specialty and academic appointments, it was not clear whether the CTR ‘caused’ the residency match, or whether the CTR was selected by ‘stronger’ students. Research exposure may clarify career choices or residency selections, and students may have matched to more competitive residencies for reasons indirectly related to the CTR SC, such as stronger letters of recommendation.
Practically, a career as a clinician-investigator requires more training and experience than can be accomplished in the short period of medical school. Additional research experiences and structured education that includes training in experimental design, biostatistics, and epidemiology, such as that offered through dual or subsequent MPH or MSci programs, are important to develop the skills necessary for clinician-investigator careers. We can do more to link students to continued opportunities for research during residency29 and to research career outcomes.30 31 The present study supports the use of longitudinal SC ‘exposure’ models for both research-experienced and research-naïve medical students that lead to careers in academic health centers. Given the current scarcity of clinician-investigators, these early interventions in medical school may be important in training future investigators. The marked increase in academic physicians we observed among CTR SC graduates, however, suggests that participation in a research SC may be a strong first step.
Acknowledgments
The authors wish to thank Michelle Armstrong, Pamela Bopp, and Cynthia Powell for assistance with data collection.
Footnotes
Contributors AKH contributed substantially to the conception and design of the work, as well as data acquisition, analysis and interpretation, drafted and revised the manuscript, gave final approval of the version to be published, and agrees to be accountable for all aspects of the work. LR and AA contributed substantially to the design of the work, data analysis and interpretation, revised the manuscript critically for important intellectual content, gave final approval of the version to be published, and agree to be accountable for all aspects of the work. LR also contributed substantially to data acquisition. JA contributed substantially to the acquisition of data, revised the manuscript critically for important intellectual content, gave final approval of the version to be published, and agrees to be accountable for all aspects of the work.
Funding This publication was supported by Award Number UL1TR001876 from the NIH National Center for Advancing Translational Sciences. Its content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.
Competing interests None declared.
Ethics approval The project was reviewed by the GW Institutional Review Board and considered ’exempt' (#051827).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Unpublished data are not available.
Patient consent for publication Not required.