Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It collects and distributes cleaned trial data, ratings and evaluations using PRECIS-2. This allows for diverse meta-epidemiological analyses that compare treatment effect estimates across trials of various levels of pragmatism.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. The term "pragmatic", however, is a word that is often used in contradiction and its definition and evaluation require clarification. Pragmatic trials are intended to inform clinical practices and policy choices, rather than confirm a physiological hypothesis or clinical hypothesis. A pragmatic trial should also try to be as similar to actual clinical practice as possible, such as the participation of participants, setting up and design, the delivery and execution of the intervention, and the determination and analysis of the outcomes, and primary analysis. This is a major distinction from explanation trials (as described by Schwartz and Lellouch1) that are intended to provide a more thorough confirmation of an idea.
Truly pragmatic trials should not blind participants or clinicians. This can lead to an overestimation of the effect of treatment. The pragmatic trials also include patients from different health care settings to ensure that their results can be generalized to the real world.
Additionally, pragmatic trials should focus on outcomes that are important for patients, such as quality of life or functional recovery. This is particularly relevant when it comes to trials that involve invasive procedures or those with potential for dangerous adverse events. The CRASH trial29, for example was focused on functional outcomes to compare a two-page report with an electronic system for monitoring of patients in hospitals suffering from chronic heart failure, and the catheter trial28 utilized symptomatic catheter-associated urinary tract infections as the primary outcome.
In addition to these aspects pragmatic trials should reduce the trial's procedures and data collection requirements to reduce costs. Additionally pragmatic trials should strive to make their results as applicable to real-world clinical practice as they can by making sure that their primary method of analysis is the intention-to-treat approach (as described in CONSORT extensions for pragmatic trials).
Despite these guidelines, many RCTs with features that challenge the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This can lead to false claims of pragmatism, and the use of the term must be standardized. The development of the PRECIS-2 tool, which provides an objective and standard assessment of pragmatic features is a good initial step.
Methods
In a pragmatic research study the aim is to inform clinical or policy decisions by showing how an intervention could be integrated into routine treatment in real-world settings. This differs from explanation trials, which test hypotheses about the cause-effect connection in idealized situations. Consequently, pragmatic trials may be less reliable than explanatory trials, and could be more susceptible to bias in their design, conduct and analysis. Despite their limitations, pragmatic studies can be a valuable source of information for decision-making within the context of healthcare.
The PRECIS-2 tool evaluates the level of pragmatism that is present in an RCT by assessing it across 9 domains, ranging from 1 (very explanatory) to 5 (very pragmatic). In this study, the recruit-ment, organization, flexibility in delivery and follow-up domains were awarded high scores, however, the primary outcome and the procedure for missing data were not at the limit of practicality. This suggests that it is possible to design a trial using excellent pragmatic features without harming the quality of the outcomes.
However, it is difficult to determine how pragmatic a particular trial really is because the pragmatism score is not a binary quality; certain aspects of a study can be more pragmatic than others. Moreover, protocol or logistic changes during an experiment can alter its score in pragmatism. Additionally, 36% of the 89 pragmatic trials discovered by Koppenaal and colleagues were placebo-controlled, or conducted prior to licensing and most were single-center. They are not in line with the standard practice and can only be referred to as pragmatic if their sponsors accept that these trials aren't blinded.
Another common aspect of pragmatic trials is that researchers try to make their results more relevant by analyzing subgroups of the sample. This can lead to unbalanced comparisons with a lower statistical power, thereby increasing the likelihood of missing or misinterpreting the results of the primary outcome. This was a problem in the meta-analysis of pragmatic trials as secondary outcomes were not adjusted for covariates that differed at the baseline.
In addition, pragmatic trials can also present challenges in the gathering and interpretation of safety data. This is due to the fact that adverse events are typically reported by participants themselves and prone to delays in reporting, inaccuracies or coding deviations. It is therefore crucial to improve the quality of outcomes ascertainment in these trials, and ideally by using national registry databases instead of relying on participants to report adverse events on the trial's database.
Results
While the definition of pragmatism doesn't require that all clinical trials are 100% pragmatic there are benefits when incorporating pragmatic components into trials. These include:
Incorporating routine patients, the trial results can be translated more quickly into clinical practice. However, pragmatic trials have disadvantages. For example, the right type of heterogeneity can help a trial to generalise its findings to a variety of settings and patients. However the wrong type of heterogeneity can reduce assay sensitiveness and consequently reduce the power of a study to detect even minor effects of treatment.
find out this here have attempted categorize pragmatic trials using various definitions and scoring methods. Schwartz and Lellouch1 created a framework to discern between explanation-based studies that prove a physiological hypothesis or clinical hypothesis and pragmatic studies that guide the choice for appropriate therapies in real world clinical practice. The framework was composed of nine domains scored on a 1-5 scale, with 1 being more informative and 5 was more pragmatic. The domains included recruitment setting, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 was based on a similar scale and domains. Koppenaal et al10 devised an adaptation of this assessment, dubbed the Pragmascope that was easier to use in systematic reviews. They discovered that pragmatic reviews scored higher on average in all domains, but scored lower in the primary analysis domain.
The difference in the analysis domain that is primary could be due to the fact that most pragmatic trials analyze their data in the intention to treat manner however some explanation trials do not. The overall score was lower for systematic reviews that were pragmatic when the domains of organisation, flexible delivery, and follow-up were merged.
It is important to note that a pragmatic trial does not necessarily mean a poor quality trial, and there is an increasing rate of clinical trials (as defined by MEDLINE search, but it is neither sensitive nor specific) which use the word "pragmatic" in their title or abstract. The use of these words in abstracts and titles may suggest a greater awareness of the importance of pragmatism however, it is not clear if this is manifested in the content of the articles.
Conclusions
As the importance of evidence from the real world becomes more widespread and pragmatic trials have gained momentum in research. They are randomized clinical trials which compare real-world treatment options instead of experimental treatments in development, they include patient populations which are more closely resembling the ones who are treated in routine medical care, they utilize comparisons that are commonplace in practice (e.g. existing drugs) and depend on the self-reporting of participants about outcomes. This method has the potential to overcome the limitations of observational studies that are prone to biases that arise from relying on volunteers and limited availability and coding variability in national registry systems.
Other advantages of pragmatic trials are the ability to use existing data sources, as well as a higher likelihood of detecting meaningful changes than traditional trials. However, they may still have limitations that undermine their reliability and generalizability. The participation rates in certain trials may be lower than anticipated because of the healthy-volunteering effect, financial incentives or competition from other research studies. Many pragmatic trials are also restricted by the necessity to enroll participants on time. In addition certain pragmatic trials do not have controls to ensure that the observed differences aren't due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified RCTs that were published between 2022 and 2022 that self-described as pragmatism. The PRECIS-2 tool was used to determine pragmatism. It includes areas such as eligibility criteria and flexibility in recruitment as well as adherence to interventions and follow-up. They discovered 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials with a high pragmatism rating tend to have higher eligibility criteria than traditional RCTs that have specific criteria that aren't likely to be found in the clinical environment, and they include populations from a wide variety of hospitals. The authors claim that these characteristics could make pragmatic trials more meaningful and applicable to everyday clinical practice, however they do not guarantee that a pragmatic trial is free of bias. The pragmatism characteristic is not a fixed characteristic; a pragmatic test that does not possess all the characteristics of an explanatory study can still produce reliable and beneficial results.