Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It collects and distributes cleaned trial data, ratings, and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological analyses to evaluate the effects of treatment across trials of different levels of pragmatism.
Background
Pragmatic studies are increasingly recognized as providing real-world evidence to support clinical decision-making. The term "pragmatic" however, is a word that is often used in contradiction and its definition and measurement require clarification. Pragmatic trials must be designed to inform policy and clinical practice decisions, rather than confirm a physiological or clinical hypothesis. A pragmatic study should try to be as similar to real-world clinical practice as is possible, including the participation of participants, setting up and design, the delivery and execution of the intervention, and the determination and analysis of outcomes and primary analyses. This is a key difference from explanatory trials (as described by Schwartz and Lellouch1) that are intended to provide a more complete confirmation of a hypothesis.
Truely pragmatic trials should not be blind participants or clinicians. This can lead to bias in the estimations of the effect of treatment. The trials that are pragmatic should also try to attract patients from a wide range of health care settings, so that their results can be compared to the real world.
Additionally, pragmatic trials should focus on outcomes that are vital to patients, like quality of life or functional recovery. This is particularly relevant for trials involving invasive procedures or those with potential serious adverse events. The CRASH trial29 compared a 2 page report with an electronic monitoring system for hospitalized patients suffering from chronic cardiac failure. The trial with a catheter, on the other hand, used symptomatic catheter associated urinary tract infection as its primary outcome.
In addition to these features pragmatic trials should reduce the trial procedures and data collection requirements in order to reduce costs. 프라그마틱 환수율 , pragmatic trials should aim to make their findings as applicable to current clinical practices as possible. This can be achieved by ensuring that their analysis is based on the intention to treat approach (as described in CONSORT extensions).
Many RCTs which do not meet the criteria for pragmatism, but contain features contrary to pragmatism, have been published in journals of different kinds and incorrectly labeled pragmatic. This could lead to misleading claims of pragmaticity and the usage of the term needs to be standardized. The development of the PRECIS-2 tool, which offers an objective and standard assessment of pragmatic features, is a good first step.
Methods
In a practical trial, the aim is to inform policy or clinical decisions by demonstrating how an intervention would be integrated into everyday routine care. This differs from explanation trials that test hypotheses about the cause-effect relationship in idealised settings. Therefore, pragmatic trials could be less reliable than explanatory trials and may be more susceptible to bias in their design, conduct and analysis. Despite their limitations, pragmatic research can provide valuable information to make decisions in the healthcare context.
The PRECIS-2 tool evaluates the level of pragmatism that is present in an RCT by assessing it across 9 domains, ranging from 1 (very explanatory) to 5 (very pragmatic). In this study, the recruitment, organization, flexibility in delivery, flexible adherence and follow-up domains were awarded high scores, however, the primary outcome and the procedure for missing data were not at the limit of practicality. This suggests that it is possible to design a trial using high-quality pragmatic features, without compromising the quality of its outcomes.
However, it is difficult to determine how practical a particular trial is since pragmatism is not a binary quality; certain aspects of a trial may be more pragmatic than others. Moreover, protocol or logistic modifications during the course of the trial may alter its score in pragmatism. Additionally 36% of 89 pragmatic trials identified by Koppenaal and colleagues were placebo-controlled, or conducted prior to licensing, and the majority were single-center. This means that they are not as common and are only pragmatic if their sponsors are tolerant of the lack of blinding in these trials.
Additionally, a typical feature of pragmatic trials is that the researchers attempt to make their findings more relevant by analyzing subgroups of the trial sample. This can result in imbalanced analyses and lower statistical power. This increases the possibility of omitting or misinterpreting differences in the primary outcomes. This was a problem in the meta-analysis of pragmatic trials as secondary outcomes were not adjusted for differences in covariates at baseline.
Additionally the pragmatic trials may have challenges with respect to the collection and interpretation of safety data. This is due to the fact that adverse events are typically reported by participants themselves and are prone to delays in reporting, inaccuracies or coding deviations. It is therefore crucial to improve the quality of outcomes assessment in these trials, ideally by using national registries rather than relying on participants to report adverse events in the trial's database.
Results
While the definition of pragmatism doesn't require that all clinical trials be 100% pragmatic there are benefits of including pragmatic elements in trials. These include:
Incorporating routine patients, the results of the trial can be more quickly translated into clinical practice. However, pragmatic trials may also have disadvantages. The right type of heterogeneity, like could allow a study to extend its findings to different patients or settings. However, the wrong type can reduce the assay sensitivity and, consequently, reduce a trial's power to detect minor treatment effects.
Several studies have attempted to classify pragmatic trials using a variety of definitions and scoring methods. Schwartz and Lellouch1 developed a framework to distinguish between explanatory studies that support the physiological hypothesis or clinical hypothesis, and pragmatic studies that guide the choice for appropriate therapies in real world clinical practice. The framework was comprised of nine domains that were evaluated on a scale of 1-5 which indicated that 1 was more lucid while 5 was more practical. The domains were recruitment setting, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 was built on the same scale and domains. Koppenaal and colleagues10 developed an adaptation of this assessment called the Pragmascope that was easier to use in systematic reviews. They found that pragmatic reviews scored higher on average across all domains, however they scored lower in the primary analysis domain.
This distinction in the primary analysis domain could be explained by the fact that most pragmatic trials analyze their data in an intention to treat method while some explanation trials do not. The overall score for systematic reviews that were pragmatic was lower when the areas of management, flexible delivery and follow-up were merged.
It is important to note that the term "pragmatic trial" does not necessarily mean a low-quality trial, and there is an increasing rate of clinical trials (as defined by MEDLINE search, however it is neither specific or sensitive) that employ the term "pragmatic" in their abstract or title. These terms may indicate a greater awareness of pragmatism within titles and abstracts, but it's not clear whether this is evident in the content.
Conclusions
As the value of real-world evidence becomes increasingly popular, pragmatic trials have gained traction in research. They are randomized trials that evaluate real-world treatment options with new treatments that are being developed. They are conducted with populations of patients closer to those treated in regular medical care. This approach could help overcome the limitations of observational research that are prone to biases that arise from relying on volunteers, and the limited availability and coding variability in national registries.
Other advantages of pragmatic trials include the ability to use existing data sources, as well as a higher likelihood of detecting meaningful changes than traditional trials. However, pragmatic tests may still have limitations which undermine their validity and generalizability. The participation rates in certain trials may be lower than anticipated due to the health-promoting effect, financial incentives, or competition from other research studies. Practical trials are often restricted by the necessity to enroll participants on time. Certain pragmatic trials lack controls to ensure that observed variations aren't due to biases in the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatic. They evaluated pragmatism using the PRECIS-2 tool, which consists of the eligibility criteria for domains as well as recruitment, flexibility in adherence to interventions and follow-up. They found that 14 of these trials scored pragmatic or highly pragmatic (i.e. scoring 5 or more) in one or more of these domains, and that the majority were single-center.
Trials that have high pragmatism scores tend to have more lenient criteria for eligibility than traditional RCTs. They also have populations from various hospitals. The authors argue that these characteristics can help make pragmatic trials more effective and applicable to daily practice, but they don't necessarily mean that a trial using a pragmatic approach is free from bias. The pragmatism principle is not a definite characteristic; a pragmatic test that doesn't have all the characteristics of an explanatory study could still yield valuable and valid results.