What should clinical research be? Should it be an exclusive endeavour of the enlightened elites? Or should it be the mass product of countless independent individuals? Actually, it depends on the quality of the specific research product and on its practical use. The same applies to systematic reviews and meta-analyses. I wish only a few skilled institutions could control all current and future meta-analyses in interventional cardiology. I also wish I could be co-author of many of them! But the truth is: reality is different, and we must adapt.
Heras et al highlight in this issue the uncanny occurrence of four different systematic reviews published within a very short time from four independent groups on the same topic, namely coronary bifurcation strategies1-5. This phenomenon, which we have described in the past in detail and also commented twice in this very journal6-8, raises a number of important issues, which are relevant for researchers, clinicians, and patients alike.
First, why did this happen? As previously clarified, clinical research is a collective endeavour based on competition and gratification of the first ever to describe a novelty. Systematic reviews share with other types of research these features. Thus, multiple investigators may simultaneously conduct similar experiments/reviews and submit for publication almost at the same time. This occurrence is not uncommon, has been described elsewhere in detail,6 and its incidence is rising given the increase in the number of medical journals worldwide and the reduction in time from submission to publication of scientific manuscripts.
Second, what are be consequences of this phenomenon? Heras et al share with us their “worrying considerations… in that there has been a waste of reviewers’ and readers’ time and published pages”. Whereas duplicate meta-analyses may confound the busy reader by providing conflicting results, they also offer the very same reader the opportunity to understand in greater detail how the reviewing process works, and which meta-analyses are to be trusted and why. In addition, systematic reviews, even when duplicate, do an excellent job for journals. In 2006 we described 10 meta-analyses published in less than two years on the same topic (namely acetylcysteine for the prevention of contrast associated nephropathy)6. More than five years later, all of the 10 systematic reviews significantly outperformed the corresponding journal average citations (p<0.01 at both parametric and non-parametric tests), demonstrating that publishing a systematic review yields well for a journal, even if there are several similar ones already published (Table 1; Figure 1).
Figure 1. Association between normalised citations in Institute for Scientific Information Web of Science and impact factor of the corresponding journals for 10 duplicate systematic reviews and meta-analyses published between 2003 and 2005 on acetylcysteine for the prevention of contrast associated nephropathy. Systematic reviews consistently and significantly outperformed the journal impact factor in terms of citations.
Third, what can we do about it? We could ban all systematic reviews and meta-analyses from medical journals, as was wily suggested by some opinion leaders. Or, we could desist from taking any action against this phenomenon. Our recommendation is to create an international web registry to enable prospective registration of systematic reviews, in as much as the Cochrane Collaboration does for its own reviews and clinicaltrials.gov does for clinical trials.9 Journal editors could require the registration number of meta-analyses under evaluation, and concomitantly check on such registry for duplicates, before committing to publication. However, there is no such registry yet, and we might still have to wait a number of years before any suitable one is developed.
Winston Churchill once stated “Democracy is the worst form of government except all the others that have been tried”. Unfortunately, peer review and the current approach to publishing systematic reviews is also the worst form possible, except all the others that have been tried.
