DOI: 10.4244/EIJV13I14A259

Evidence-based medicine, transparency and reproducibility in research, and challenges for peer review

Robert A. Byrne, MB, BCh, PhD, Deputy Editor

It seems that hardly a week goes by without an article appearing discussing the challenges facing biomedical research1,2. At the same time, the importance of clinical research and evidence-based medicine for decision making continues to grow2. In cardiology, in particular, many of the treatments that we recommend for our patients are supported by high-quality evidence showing favourable risk-benefit profiles against alternative or placebo treatments. Clinical practice guidelines have become an integral part of daily practice in cardiology, with committees regularly reviewing the totality of evidence in a given area and identifying gaps where further study is needed. In a recent survey of EuroIntervention readers, 92% of respondents reported reading articles to guide professional activity at least weekly (data on file).

Coupled with this increasing awareness of the importance of evidence-based medicine, technological advances over the last decades have made data collection and analysis more accessible to researchers and clinicians. In addition, the availability of important new therapies in interventional cardiology – drug-eluting stents, transcatheter heart valves, novel antiplatelet and antithrombotic therapies, mechanical circulatory support systems, to name but a few examples – has fuelled a rapid increase in the number of clinical investigations. Most journals in this field continue to experience year-on-year growth in the number of manuscripts that are submitted. EuroIntervention is no exception: the number of papers submitted in 2017 was more than 12% higher than in 2016 and represents a record high for our journal.

In parallel, there has been an explosion in the number of journals publishing research in cardiovascular medicine, meaning that increased opportunities to publish also exist. Indeed, high rates of growth in the output of scientific manuscripts present not just opportunities but also significant challenges. These challenges can be seen as part of the wider debate in biomedical research concerning research quality, transparency in reporting, and reproducibility of observations3,4. Perhaps some consideration of these issues recently prompted Milton Packer to write somewhat apocalyptically that “[we live in] an era where the number of opportunities to publish greatly exceeds the number of valid observations”3.

The natural tension between the triad of growing reliance on evidence-based medicine, increasing volume of scientific output, and increasing opportunity for scholarly publication places ever more responsibility on editorial processes and peer review. With the exception of the largest and most successful academic journals, most of this critical work is carried out on a voluntary basis by unpaid experts. In fact, most of us, I’m sure, have at one time or another experienced bewildered looks from friends or acquaintances not involved in research pursuits, when explaining how and why we invest a not inconsiderable amount of our free time in peer review activities. It is important to ensure that this model remains sustainable in a time of increased output. In some respects, the issues arising form part of a larger debate on what some have termed the “staggeringly profitable business of scientific publishing”5. Critics of the current model note that researchers – often funded by governmental sources – produce documents at no cost to the publisher, which are reviewed and corrected by experts on a voluntary basis. The end product is then sold back to scientists with subscriptions often paid for by government-funded institutions such as university libraries5. The issues, of course, are complex - perhaps a topic for another day.

Contemporary peer review is challenging and requires time and attention to detail. Even at the level of the individual manuscripts, the amount of data provided to review is sometimes overwhelming. Many papers include supplementary material with a large amount of data. Concerns exist that data in such appendices are less thoroughly inspected and errors less likely to be identified. As one writer in Nature recently commented, “Although I am a seasoned reviewer, I find it difficult to wade through the increasing amount of data in papers, and often encounter material where I am not an expert. If this trend continues, it will be necessary to take mini-sabbaticals to review papers”6.

While acknowledging the challenges that exist, we should also take stock of the important progress that has been made over the last decade or so in terms of standards in the conduct and reporting of studies. First, the checklist approach to data reporting has enabled an improvement of standards. Perhaps the best known are the CONSORT checklists for randomised clinical trials7. The PRISMA criteria for reporting of systematic reviews and meta-analyses are another well-known example, but checklists for the reporting of data from observational studies are also extremely valuable and probably greatly underutilised8. The website of the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network (www.equator-network.org) aggregates many of these resources in a single location and currently contains some 389 guidelines, which are paywall-free8. Second, initiatives from the International Committee of Medical Journal Editors (ICJME) provide guidelines on uniform requirements for the conduct, reporting and publishing of scholarly work. These manuscripts – now known as Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals – have been regularly updated, particularly since 20059. ICJME initiatives to promote prospective registration of research protocols have played an important role in ensuring transparency in relation to trial conduct and publication10. Moreover, in terms of accountability and in view of the complexities surrounding collaborative research between multiple authors and institutions, detailed guidance on authorship of reports is provided by the ICJME, with recent extensive discussion of this topic in one journal11. Thirdly, in terms of research transparency, guidance has recently been published on the issue of data sharing, where authors of clinical trials, for example, will be mandated to ensure that processes are put in place to make original patient-level data freely available in online data repositories12. Although this is a thorny issue that seems far from resolved, the advantages for transparency and reproducibility are clear.

In parallel to the increasing burdens of peer review, for a number of reasons (including the increasing opportunities for publication), pressure on journals to review and decide on manuscripts in as short an amount of time as possible is increasing. Of course, rapid review is not an unreasonable expectation from authors. Efficient workflows, intelligent document management systems and timely turnaround are integral elements of any well-run journal. In this respect, we hope that EuroIntervention has made some strides. For example, median time to first view has fallen from 3.3 days in 2015 to 1.9 days in 2017. However, a floor exists, beneath which more rapid turnaround of decisions means compromise in terms of review quality. The extreme end of the spectrum is pre-publishing – a process becoming increasingly common in many branches of science, where observations are published rapidly online without first undergoing the rigours of peer review. Ultra-rapid review and pre-publishing are probably poorly suited to clinical research, where published findings may have an important impact in terms of decision making for physicians and patients. Indeed, as Valentin Fuster recently asked, “Who Truly Benefits From Pre-Publishing and Rushing Publishing?: Not Patients, Nor Clinicians”13. Looking to the future, editors will continue to have to strike the right balance between expediting novel observations regarding new treatments or devices – be they benefits or unexpected safety issues– and allowing adequate time for the rigours of review and rebuttal.

As we come to terms with an era where the expectations of the community in evidence-based medicine are great, the volume of scientific output is high, and the pace of electronic communication and data dissemination is rapid, we rely more than ever on the imperfect process that is peer review and editorial oversight. To all of you who contribute to the process at EuroIntervention on an ongoing basis we remain extremely grateful.


References

Volume 13 Number 14
Feb 20, 2018
Volume 13 Number 14
View full issue


Key metrics

Suggested by Cory

10.4244/EIJV6I2A29 Jun 30, 2010
Duplicate meta-analyses on coronary bifurcation strategies: when more is less?
Biondi-Zoccai G
free

10.4244/EIJV9I11A211 Mar 20, 2014
Is evidence actually evidence?
Cummins P and Serruys PW
free

10.4244/EIJV12I2A23 Jun 10, 2016
Looking into the publishing orbuculum: the year 2020
Cummins P and Serruys PW
free

Jun 30, 2010
Scientific societies and clinical trials
Di Mario C et al
free

10.4244/EIJV11I9A197 Jan 20, 2016
Listen, understand and act - developing modern scientific engagement
Cummins P and Serruys PW
free

10.4244/EIJV9I6A106 Oct 25, 2013
ESC Journals working together
Cummins P and Serruys PW
free

EXPERT REVIEW

10.4244/EIJ-D-16-00918 Mar 20, 2018
Controversies surrounding authorship of manuscripts by industry employees: academic and industry perspectives
Smilowitz N et al
free
Trending articles
225.68

State-of-the-Art Review

10.4244/EIJ-D-21-00426 Dec 3, 2021
Myocardial infarction with non-obstructive coronary artery disease
Lindahl B et al
free
105.78

Expert consensus

10.4244/EIJ-E-22-00018 Dec 4, 2023
Definitions and Standardized Endpoints for Treatment of Coronary Bifurcations
Lunardi M et al
free
77.85

State-of-the-Art

10.4244/EIJ-D-23-00840 Sep 2, 2024
Aortic regurgitation: from mechanisms to management
Baumbach A et al
free
68.7

Clinical research

10.4244/EIJ-D-21-00545 Sep 20, 2022
Coronary lithotripsy for the treatment of underexpanded stents: the international; multicentre CRUNCH registry
Tovar Forero M et al
free
47.8

NEW INNOVATION

10.4244/EIJ-D-15-00467 Feb 20, 2018
Design and principle of operation of the HeartMate PHP (percutaneous heart pump)
Van Mieghem NM et al
free
45.3

Clinical research

10.4244/EIJ-D-18-01126 Aug 29, 2019
New-generation mechanical circulatory support during high-risk PCI: a cross-sectional analysis
Ameloot K et al
free
X

The Official Journal of EuroPCR and the European Association of Percutaneous Cardiovascular Interventions (EAPCI)

EuroPCR EAPCI
PCR ESC
Impact factor: 7.6
2023 Journal Citation Reports®
Science Edition (Clarivate Analytics, 2024)
Online ISSN 1969-6213 - Print ISSN 1774-024X
© 2005-2024 Europa Group - All rights reserved