Debate

DOI: 10.4244/EIJ-E-23-00023

Observational studies play little role in guiding evidence-based medicine: pros and cons

Philippe Gabriel Steg1,2, MD; Laurent J. Feldman1, MD, PhD; Elmir Omerovic3, MD, PhD

Introduction

In evidence-based medicine (EBM), randomised controlled trials are often considered the gold standard for generating high-quality evidence. However, observational studies are also commonly used to investigate the effectiveness of interventions or strategies in a real-world scenario as they can explore rare outcomes and long-term effects, yield more generalisable results, and cost less. On the other hand, observational studies can be subject to biases and confounding which can be assessed statistically, but may, in the end, limit the validity of their findings. Whether observational studies play an important role in guiding evidence-based medicine is still a matter of debate.

Pros

Philippe Gabriel Steg, MD; Laurent J. Feldman, MD, PhD

Observational studies are subject to bias and confounding related to measured but also – and more importantly – unmeasured variables. Attempts to adjust for potential confounders by multivariable adjustment, propensity matching and other methods may help minimise biases but can never guarantee elimination of bias. This is true even when observational studies are very large and allow for extensive adjustment: a large confounded observational study is just that. In fact, medical literature is replete with examples of converging multiple large observational studies suggesting the effectiveness of medical interventions which were subsequently disproven by well-conducted, large randomised trials1. Examples include – but are not limited to – the misleading conclusions of observational studies regarding an apparent cardiovascular benefit of hormone replacement therapy in postmenopausal women which was not confirmed in later randomised trials. Another well-known example is the association of premature ventricular contractions with mortality in acute myocardial infarction patients and the apparent potential benefit of antiarrhythmic agents, which were subsequently found to increase mortality in this population when tested in the randomised Cardiac Arrhythmia Suppression Trial (CAST; ClinicalTrials.gov: NCT00000526). Closer to home is the example of the potential benefit of emergency coronary angiography with a view to coronary intervention in survivors of out-of-hospital cardiac arrest. Even though observational studies have suggested a very large benefit of immediate angiography, no fewer than 6 subsequent randomised controlled trials and a network meta-analysis in this setting failed to demonstrate a benefit of the emergency angiography approach, which, in fact, may be detrimental in terms of logistics, costs, and the management of these patients2.

Critics of randomised trials often point out their limitations: they can be poorly generalisable because of extensive selection of the participants34, they can yield apparent contradictory results, and they can be extremely costly to implement. All these criticisms are valid: randomised trials are not inherently perfect and can suffer from severe flaws in their design, conduct or interpretation. However, it is inescapable that, compared to observational studies, randomised trials dramatically reduce the potential for bias and confounding. Importantly, streamlining and improving clinical trials is critical to reducing their costs and improving the generalisability of their conclusions. The model of the registry-based randomised trial, used so elegantly by Swedish cardiology investigators, is one important tool to achieve these objectives5. Likewise, new trial designs such as platform trials will help speed up discovery and reduce costs.

Does this mean observational studies are worthless? Absolutely not. Observational studies remain critical to describe the clinical characteristics, risk factors, management and outcomes of medical conditions, including the adoption of evidence-based therapies which have been tested and proven effective in randomised controlled trials, or the identification of gaps between evidence and practice. This is an important contribution to our evidence base. In certain difficult settings, such as rare diseases, observational studies are often the only tool available to accrue evidence, which is better than nothing at all. However, looking back at 40 years of continuous progress, it is striking that cardiology is one of the areas of medicine where randomised trials have been the most instrumental to advancing care. Whether one considers the benefit of myocardial reperfusion in acute myocardial infarction; the benefits of antithrombotics in acute coronary syndromes, percutaneous coronary intervention or atrial fibrillation; the role of lipid-lowering therapy in secondary prevention; or the advent of multiple effective treatments for the management of heart failure; there is no doubt that cardiology is the poster boy for randomised trials…

Conflict of interest statement

P. Steg reports research grants from Amarin, Bayer, Sanofi, and Servier; is on the clinical trials steering committee, CEC, or DSMB for Amarin, AstraZeneca, Bayer, Bristol-Myers Squibb, Idorsia, Novartis, PhaseBio, Pfizer, Sanofi, and Servier; has received consulting or speaking fees from Amarin, Amgen, BMS/Myokardia, Merck, Novo Nordisk, and Regeneron; and is Senior Associate Editor for Circulation. L. Feldman has no conflicts of interest to declare.

Cons

Elmir Omerovic, MD, PhD

Evidence-based medicine is “a clinical practice method characterised by the careful, explicit, and judicious use of the best available evidence to treat individual patients,” according to the original definition6 (Figure 1). From this perspective, observational studies (OS) are an inseparable part of EBM; hence, the argument that “OS play a small role in EBM” is invalid.

The hegemony of randomised controlled trials (RCT) and their misuse in medicine has contributed to an “illusion of knowledge” and led many physicians and decision-makers to become misguided and stop thinking beyond the RCT7. Indeed, EBM was neither conceived nor advanced solely relying on RCTs but, instead, integrated individual clinical expertise with the best available external clinical evidence from systematic research where OS assume a prominent role6.

In recent years, there has been notable progress in the field of causal inference (CI) methods89, which constitute a critical component of OS. The increasing interest of decision-makers in real-world evidence derived from high-quality OS is also boosted by the modern advancements in machine learning of “big data” and artificial intelligence. The notion that OS are crucial to establishing causality in medicine should not be controversial in the 21st century. For instance, OS can help establish natural experiments, identify and adequately adjust for confounding variables, and provide real-world evidence on the effectiveness of interventions. Additionally, OS can complement RCTs, allowing for the study of larger and more diverse populations at lower costs and over shorter periods. OS can also better estimate individual treatment effects, enabling more personalised and precise healthcare interventions based on individual patient characteristics. In principle, there is nothing that RCTs can do that OS – based on state-of-the-art CI methods – cannot do to produce valuable knowledge for medical decision-making45. From the point of view of how we gain knowledge, there should be no “gold standard” or “hierarchies of evidence” in EBM10. Hierarchies are unscientific because the profession is collectively absolved from reconciling results across studies; the OS is assumed to be incorrect merely because there was no randomisation. This is a conscious disregard of useful knowledge. In the social sciences and economics, this line of thought about OS has been accepted for a long time10. It’s time to apply this rationale to medicine.

RCTs have a higher potential for proving causal links than observational studies, assuming all theoretical assumptions are met. Nonetheless, because such situations are rare, RCTs, like OS, have substantial limitations in the real world. Despite their significance, RCT limitations have received insufficient attention in scholarly debate. One critical disadvantage of RCTs is their “local” effect, which may prevent causation from being generalised beyond the bounds of the controlled trial setting10. In addition, RCTs may not capture the long-term effects of treatments or be feasible for investigating rare diseases or outcomes.

Postapproval OS offer significant value in countering the profit-driven motives of the medical-industrial complex. Currently, pharmaceutical and medical device manufacturers plan and conduct most RCTs, which may bias the results toward their products’ interests while neglecting patient and societal demands7. Thus, OS may serve as a counterbalancing tool, especially when incorporated into national quality-of-care registries with continuous monitoring of healthcare outcomes. Incorporating OS into national quality-of-care registries can ensure a comprehensive assessment of the long-term impact of medical products on patients. These registries provide valuable data on the effectiveness and safety of medical interventions, facilitating the development of evidence-based guidelines to optimise patient care. OS can also identify previously unknown risks and benefits associated with medical products, enhancing patient safety and quality of life.

In conclusion, a more balanced and inclusive approach to EBM is necessary, where RCTs and OS are recognised for their respective strengths and limitations and used in complementary ways to better inform healthcare decision-making. This could involve funding and conducting more OS, encouraging collaborations between researchers who specialise in RCTs and those who specialise in OS, or developing guidelines for designing and implementing diverse study designs.

Figure 1. What is evidence-based medicine? Evidence-based medicine (EBM) extends beyond the domain of randomised controlled trials (RCT). Regrettably, the dominant position of RCTs in modern medicine and healthcare systems has fostered an “illusion of knowledge,” leading to unwarranted patient harm and resource wastage. Observational trials have historically been, and continue to be, a crucial component of EBM in the promotion of human health and well-being.

Conflict of interest statement

The author has no conflicts of interest to declare.

Volume 20 Number 1
Jan 1, 2024
Volume 20 Number 1
View full issue


Key metrics

On the same subject

10.4244/EIJV12I5A90 Aug 5, 2016
Meta-analysis: don’t do what I did
Serruys PW
free

10.4244/EIJV13I4A59 Jul 20, 2017
Registries, reality and complete revascularisation
Dauerman H and Rambod M
free

10.4244/EIJV13I16A306 Mar 20, 2018
It seemed like a good idea at the time
Ormiston J and Webster M
free

10.4244/EIJV12I9A177 Oct 20, 2016
Let’s talk BIG DATA
Wijns W and Barbato E
free
Trending articles
187.45

State-of-the-Art Review

10.4244/EIJ-D-21-01034 Jun 3, 2022
Management of in-stent restenosis
Alfonso F et al
free
170.15

Translational research

10.4244/EIJ-D-21-00824 May 15, 2022
Bench test and in vivo evaluation of longitudinal stent deformation during proximal optimisation
Toth GG et al
free
159.45

Expert review

10.4244/EIJ-D-21-00690 May 15, 2022
Crush techniques for percutaneous coronary intervention of bifurcation lesions
Moroni F et al
free
127.03

State-of-the-Art

10.4244/EIJ-D-22-00776 Apr 3, 2023
Computed tomographic angiography in coronary artery disease
Serruys PW et al
free
108.75

Viewpoint

10.4244/EIJ-E-22-00007 May 15, 2022
TAVI at 20: how a crazy idea led to a clinical revolution
Eltchaninoff H et al
free
X

The Official Journal of EuroPCR and the European Association of Percutaneous Cardiovascular Interventions (EAPCI)

EuroPCR EAPCI
PCR ESC
Impact factor: 6.2
2022 Journal Citation Reports®
Science Edition (Clarivate Analytics, 2023)
Online ISSN 1969-6213 - Print ISSN 1774-024X
© 2005-2024 Europa Group - All rights reserved