Abstract
Clinical research is a discipline prone to the use of technical terms that may be particularly at risk for misunderstanding given the complex interpretation that is required. In this century, what is happening with the word ‘pragmatic’ when describing a randomized controlled trial (RCT) with medicines deserves a public reflection. Explanatory trials are conducted in ideal conditions to assess the comparative efficacy of interventions and are useful to explain whether interventions work. Pragmatic trials are those conducted in a way that resembles usual clinical practice conditions to assess the comparative effectiveness of interventions in a manner directly applicable for decision-makers. This, however, did not prevent 36% of authors of placebo-controlled, or prelicensing trials to identify their medicines RCTs as pragmatic in the title of their articles. The current situation is such that scientific literature has accepted that ‘pragmatic’ can convey the original meaning—that obtained in trials mimicking usual clinical practice—and a distorted one—that is focused on streamlining any trial procedure. Those involved in clinical trials should emphasize the importance of precision in the use of terms when describing RCTs through standardized solutions when possible. Unless clinical trial stakeholders agree when it would be correct to label an RCT as pragmatic, in a short period of time the term will be in danger of becoming meaningless. It is suggested that the Enhancing the Quality and Transparency of Health Research (EQUATOR) network, the Consolidated Standards of Reporting Trials (CONSORT) group and the International Committee of Medical Journal Editors (ICMJE) could address this topic and provide a consensus way forward.
‘When I use a word,’ Humpty Dumpty said, in rather a scornful tone, ‘it means just what I choose it to mean—neither more nor less.’ ‘The question is,’ said Alice, ‘whether you can make words mean so many different things.’—Lewis Carroll (‘Through the Looking-Glass’, 1871)1
Most readers would agree that the use of precise terms should be an important aim of any individual when communicating an idea, a feeling, a description, or a reasoning. The use of precise language has important benefits for the interlocutors. Many words are invented and defined within a specific knowledge area, such as medicine or epidemiology. How these terms are used may help or, conversely, prevent, a fluent and coherent interpretation of what individuals aim to convey. This, however, seems to be underappreciated by some authors of medical literature. Just recently, and regarding two different medical subjects—clinician distress2 and COVID-193—authors called for the precise use of language to fully convey important messages.
Semantics in clinical research
Clinical research is a discipline prone to the use of technical terms that may be particularly at risk for misunderstanding given the complex and nuanced interpretation that is required. One example is the incorrect use of the terms efficacy and effectiveness that many investigators, authors and even regulators use as synonyms.4 Another that has been recently highlighted is that high-quality randomized controlled trials (RCTs) should be considered as the ‘best available’ design to ensure high internal validity, rather than the very widely term ‘gold’ standard, since there are some trial aspects that are not correctly conducted.5
In clinical trials, however, one important example of a word that has been incorrectly interpreted and used for almost 20 years is ‘blinding’. Blinding (or masking) is a critical feature of RCTs since it prevents bias by keeping one or more key trial individuals unaware of allocated interventions. These trial individuals could be subjects (patients or healthy volunteers), investigators (healthcare providers), data collectors, outcome assessors and data analysts. In some RCTs, several of these four latter functions may be conducted by the same individuals. For instance, in some RCTs, investigators are also data collectors and outcome assessors. Clinicians6 and investigators7 vary greatly in their interpretations of textbook definitions of single-blind, double-blind and triple-blind RCTs. In fact, the Consolidated Standards of Reporting Trials (CONSORT) statement8 requires that RCTs should report whether blinding was done, who was blinded and how blinding was achieved. In other words, to be precise and prevent misinterpretations, CONSORT recommends that authors specifically explain in detail who was masked to allocated interventions. Despite 585 journals and over 50% of core medical journals listed in the Abridge Index Medicus on PubMed endorsing the CONSORT statement,9 a recent survey of RCTs authors still found different interpretations of the different levels of blinding.10 So, for instance, ‘single-blind’ could mean that either the subject or the outcome assessor is blinded; ‘double-blind’ could have different meanings such as that both subjects and investigators, or investigators and care providers are masked to interventions.10 Therefore, even if it has been agreed through CONSORT that authors should describe what the level of blinding actually means in their RCT, authors use this terminology—in some instances in an incorrect way10—without providing detailed information in the article.
More than completely distinct meanings in one word
In this century, what is happening with the word ‘pragmatic’ when describing an RCT assessing medicines deserves a public reflection: the original meaning is becoming distorted to a degree that currently, from our perspective, just some of the RCTs self-labeled as pragmatic could be considered as such.11 Many trials with indisputable non-pragmatic characteristics are being self-tagged as pragmatic in a conscious or subconscious attempt to grant them attributes that do not correspond to reality. Currently, gathering real-world evidence is increasingly considered a requirement for an intervention aiming to be broadly accepted. ‘Pragmatic’ has become an almost magical word synonymous with real-world evidence generation. In this context, ‘pragmatic’ provides the external validity badge to the trial and conveys broader generalizability to trial results (box 1).
Real-world evidence: external validity of randomized controlled trial results. Generalizability and applicability.26
External validity refers to whether the results of a trial could be used to patients other than those who participated in the trial. It is captured in two concepts, generalizability and applicability.
Generalizability characterizes the extent to which the study results from a specific trial population related to the broader population from which the sample was obtained.
Applicability relates to extending the results of a trial to another population in a distinct setting.
Both concepts, generalizability and applicability, are of interest when discussing real-world evidence. However, it is common to use the term generalizability when referring to other populations and settings.13
RCTs were classified as pragmatic and explanatory more than 50 years ago.12 Pragmatic trials are those conducted in a way that resembles usual clinical practice conditions to assess the comparative effectiveness of interventions, in a manner directly applicable for decision-makers; explanatory trials are conducted in ideal conditions to assess the comparative efficacy of interventions, and are useful to explain whether interventions work.3 12 Yet, most RCTs have both pragmatic and explanatory features. Currently, it is well accepted that there is a continuum between these two extremes, pragmatic and explanatory.13 14 The issue is, when should an RCT be reasonably considered and labeled as pragmatic? When assessing medicines a prelicensing trial would routinely assess efficacy and a postlicensing trial could assess either efficacy or effectiveness depending on the degree of pragmatism of the trial.3 Labeling an RCT as pragmatic requires the explicit will to do so, by default, when this term is not mentioned, the trial is considered explanatory, and does not require to be labeled as such.
A pragmatic RCT12–16: (a) is aimed to identify which of the available interventions that are compared is better; (b) the primary endpoint should be patient centered; (c) should mimic the real world, with the normal number of procedures, tests and periodicity of visits; (d) be run in several sites, thereby ensuring a full range of investigators and a heterogeneous sample of subjects; and (e) data should be analyzed in an intention-to-treat fashion. The results will be generalizable to target populations of many settings and will be useful for decision-makers (eg, patients, clinicians, policymakers). With these requirements, pragmatic RCTs assessing medicines can only be conducted with commercially available products that were prescribed according to the terms of the marketing authorization or when their use was evidence based. In other words, only those phase 4 RCTs that fulfill the above-mentioned characteristics could, from our perspective and, honoring the definition of Schwartz and Lellouch,12 be appropriately labeled as pragmatic. This, however, did not prevent 36% of authors of placebo-controlled, prelicensing or conducted in a single center to identify their medicine RCTs as pragmatic in the title of their articles.11 Furthermore, 45% of reports in which RCTs were labeled as pragmatic did not provide a single reason justifying the pragmatic label.17 It seems that many authors felt that they can describe their RCTs as ‘pragmatic’ (that has an unequivocal appeal), since it conveys the generalizability of the results, without providing any reason supporting it.
The use of the term pragmatic as a descriptor of randomized controlled trials
‘Pragmatic’ is increasingly used in articles to accompany other terms that help describe an RCT assessing medicines such as the use of masking, use of placebo, one or more numbers of participating sites and, sometimes, the phase of clinical development. Of these five attributes, the last three are usually clearly defined. As it has been mentioned earlier, this is not the case with ‘blinding’ nor with ‘pragmatic’. This latter term has become en vogue in recent years.11 Many authors have labeled their RCT as pragmatic even when, given the absence of the use of this term in the available sources (eg, full protocol, registry, published protocol), investigators should have thought about it when drafting the manuscript of the trial results: this has happened both in RCTs that evaluated experimental medicines18 or assessed a new indication of a marketed medicine.19
Scientific literature has accepted that ‘pragmatic’ can convey the original meaning (obtained in trials mimicking usual clinical practice) and a distorted meaning (focused on streamlining any trial procedure).20 As such, an RCT with only one pragmatic feature, such as streamlined eligibility criteria to create a representative sample of the patient population, could not be characterized as pragmatic.21 The generalizability of the results can only be claimed when the trial has several pragmatic features, not only one. A number of authors considered that, in addition to eligibility, there are between five and nine more characteristics that should be evaluated to know how pragmatic is an RCT.13 16 22 23 The Pragmatic Explanatory Continuum Indicator Summary 2 (PRECIS-2) tool is increasingly used to conduct this assessment. It has nine domains that should be assessed and scored to know whether a trial is closer to the pragmatic extreme or to the explanatory extreme of the continuum.13 Broad eligibility criteria were, however, the single reason used by 30% of authors that justified the use of the term ‘pragmatic’ in the title of their RCTs.17
Looking forward
Those involved in clinical trials should emphasize the importance of precision in the use of terms when describing RCTs through standardized solutions when possible. An example solution was summarized when dealing with blinding: by describing which individuals were blinded to the allocated treatments, the term becomes clearly defined. Unfortunately, although the solution was agreed on,8 many authors are still using adjectives such as ‘double-blind’ with different meanings and without providing further explanations.10 The problem with the use of the term ‘pragmatic’ is pressing since it is not easy to envisage a solution that could be widely accepted. We argue that the researchers should agree to use the term pragmatic only according to the original definition.12 The main difficulties would be (a) to agree a minimum threshold of the degree of pragmatism, that could be widely accepted to tag an RCT as pragmatic; and (b) how to assess the degree of pragmatism, that could be conducted with the PRECIS-2 tool. While this is not an easy undertaken, there are proposals that could be useful as a starting point for discussion and debate.11 Adding a prefix, such as ‘quasi’ (eg, quasi-experimental), could provide some flexibility on how to label RCTs with different degrees of pragmatism.
While a solution is reached it is important that funders, research ethics committees and journal editors play a key role by requesting investigators and authors to use the word pragmatic correctly in their trial funding applications, protocols and manuscripts, respectively. Authors should provide the PRECIS-2 nine domains assessments and scores as supplemental information to the submitted protocol or manuscript.11 Although the PRECIS-2 tool still has some subjectivity, it is a systematic way to assess nine critical RCT characteristics.
Unless clinical trial stakeholders agree when it would be correct to label an RCT as pragmatic, in a short period of time the term will be in danger of becoming meaningless, as happened previously to a word as ‘prospective’. It is suggested that the Enhancing the Quality and Transparency of Health Research (EQUATOR) network,24 the CONSORT group14 and the International Committee of Medical Journal Editors (ICMJE)25 could address this topic and provide a consensus way forward.
Ethics statements
Ethics approval
Not required
Footnotes
Contributors RDR wrote the first draft of the manuscript. RJM and FRR provided comments and edits throughout the drafting process for important intellectual content. All authors approved the final version of the manuscript and are accountable for all aspects included in it.
Funding This work required no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests RDR has received honorarium from Palex Medical for giving a lecture on low-risk pragmatic trials. RJM receives research support from Amgen, AstraZeneca, Bayer, Cytokinetics, American Regent, Medtronic, Merck, Novartis and Sanofi; honoraria from Abbott, Amgen, AstraZeneca, Bayer, Boston Scientific, Cytokinetics, Janssen, Medtronic, Merck, Novartis, Roche, Sanofi and Vifor; and has served on an advisory board for Amgen, AstraZeneca, Cytokinetics, Merck, Novartis and Boehringer Ingelheim.
Provenance and peer review Not commissioned; externally peer reviewed.