Abstract
Aims: We investigated using meta-analytic techniques, whether, and to what degree, single or multicentre study design affects clinical outcomes in randomised controlled trials examining the efficacy of adjunctive devices to prevent distal embolisation during acute myocardial infarction (AMI).
Methods and results: We searched electronic databases, conference proceedings, and internet-based sources of information to identify relevant studies through March 2009. The pooled summary effect was estimated with a random effects model. Subgroup and meta-regression analyses were conducted to examine the impact of single or multicentre design on trial outcomes compared with other variables. A total of 25 randomised trials (5,919 patients) were included in the analysis. The major sources of heterogeneity in trial outcomes were single or multicentre design, type of device used, study size, study region, and presence of conflicts of interest, of which the most influential source of heterogeneity was single or multicentre design (p-values of regression coefficient on meta-regression analyses were 0.09 for mortality, 0.001 for incomplete ST-segment resolution, and 0.07 for impaired myocardial blush grade, respectively).
Conclusions: Single or multicentre study design has a significant impact on outcomes in trials examining the efficacy of adjunctive devices in AMI.
Abbreviations
AMI: acute myocardial infarction;
MBG: myocardial blush grade;
PCI: percutaneous coronary interventions;
STR: ST-segment resolution.
Introduction
The thrombus formation after atherosclerotic plaque rupture with subsequent closure of coronary arteries is the major cause of acute myocardial infarction (AMI)1. The distal embolisation of atheromatous and thrombotic debris is common during primary percutaneous coronary intervention (PCI) and is associated with decreased myocardial tissue perfusion, larger infarct size, and increased mortality2,3. In the last decade, great efforts have been made to develop adjunctive devices to remove thrombus and to limit distal embolisation during PCI.
The results of trials examining the treatment effects of adjunctive devices have not been consistent. Recent meta-analyses showed that the use of adjunctive devices in addition to standard PCI during AMI produce heterogeneous clinical outcomes4,5. Because the different types of devices differ in construction and require unique techniques for effective removal of thrombus, subgroup analysis was conducted according to the type of devices. Subgroup analysis showed that thrombus aspiration catheters offer survival benefit compared with PCI alone, whereas mechanical thrombectomy devices increase mortality and embolic protection devices have a neutral effect.
However, the difference in clinical outcomes can also arise from the difference in operator or institution expertise. It has been shown that high-volume angioplasty centres offer a lower mortality rate and faster procedure time in AMI compared with low-volume centres6. Consequently, study design, whether conducted either at a single centre or at multiple centres, may yield different conclusions when examining such highly-skilled procedures. In addition, single-centre trials, compared with multicentre trials, may be prone to considerable bias as the processes of randomisation, blinding of outcome assessors, or reporting of trial outcomes may be less strictly controlled.
We hypothesised that single or multicentre study design affects the results of randomised controlled trials examining the efficacy of adjunctive devices during AMI. We conducted this systematic review using meta-analytic techniques to examine the difference of study characteristics between single and multicentre trials, to evaluate the impact of study design on treatment outcomes, and to critically review whether the current evidence supports the general use of adjunctive devices in AMI.
Method
Data sources and study selection
Ovid MEDLINE, Ovid MEDLINE Daily Update, Ovid MEDLINE In-Process & Other Non-Indexed Citations, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews were searched by two investigators (YI, JAC) through March 2009. A list of exact search terms is presented in the online supplement, Table 1.
We also reviewed reference lists from relevant reviews and conference proceedings from major international cardiology meetings (including the American Heart Association, American College of Cardiology, and European Society of Cardiology meetings). Furthermore, the TCT (http://www.tctmd.com), EuroPCR (www.europcr.com), ACC (www.acc.org), AHA (http://www.americaheart.org), and ESC (www.escardio.org) websites were searched for oral presentations and expert slide presentations between January 2006 and December 2008. The search was limited to human studies and combined with a randomised controlled trial filter7. There were no language restrictions.
Studies were included if they were randomised controlled trials examining the efficacy of the use of adjunctive mechanical devices compared with standard PCI in patients with AMI. Studies were excluded if the adjunctive mechanical devices were used in saphenous vein grafts; duplicate data from other studies was used; or there were insufficient data for meta-analysis.
The following information was abstracted from eligible studies: year of publication, number of trial centres, population characteristics (sex, age, smoking, and location of occluded artery), source of funding, adjunctive device used, procedure time, door-to-balloon time, pain-to-balloon time, and results of clinical outcomes. The primary outcomes of interest were mortality rate, post PCI impaired myocardial blush grade (MBG) (defined as MBG of less than 3), and incomplete ST-segment resolution (STR) (defined as less than 70%, or 50% if70% was not available). Study quality was assessed and scored as either “yes” or “unclear” by using the following criteria: adequate randomisation methods using both adequate sequence generation and adequate allocation concealment, blinding of outcome assessors (outcome assessments include clinical outcomes, electrocardiogram, and coronary angiography), and disclosure of withdrawals and drop-outs8. The two investigators (YI, JAC) extracted the data and assessed quality for each study in duplicate and independently. Any discrepancies were resolved by discussion.
Statistical synthesis
The relative risk and 95% confidence interval (CI) were used as summary statistics for the comparison of dichotomous variables. The weighted mean difference and 95% CI were used as summary statistics for continuous variables. The DerSimonian and Laird random effects model was used to combine values from single studies because included studies were heterogeneous in many respects, particularly in the usage of adjunctive mechanical devices8. To assess for heterogeneity across studies, the Cochrane Q statistic was calculated9. In addiction, the I2 statistic was used to quantify heterogeneity on a scale of 0% to 100%, in which larger values indicate greater heterogeneity8.
Publication bias was assessed using visual inspection of a funnel plot and the Begg’s test. We also performed a non-parametric “trim and fill” method to adjust the effect of possible publication bias in this meta-analysis10. This method imputes “missing” studies, adds them to the analysis as though they actually exist, and then re-computes a pooled estimate.
In an attempt to examine the effect of single or multicentre design on trial outcomes compared with other variables, a protocol-based subgroup analysis was conducted. The results were stratified according to single or multicentre design, study region (Europe, Asia, and USA or elsewhere), type of adjunctive device used (thrombus aspiration catheters, mechanical aspiration devices, or embolic protection devices), study size, presence of conflicts of interest, and each domain of quality assessment. The adjunctive devices were categorised using the same definition as a previous meta-analysis5.
The meta-regression analysis was conducted to investigate whether and to what degree single or multicentre design, compared with other variables, explains the heterogeneous clinical outcomes across the studies. First, univariate meta-regression analysis was conducted to investigate how much each variable accounts for the amount of heterogeneity. Then, multivariate meta-regression analysis was conducted to determine which variables are the major sources of heterogeneity. We included categorical variables used in subgroup analyses and the following continuous variables in the model: number of patients, number of centres, number of patients per centre, mean age, publication year, percentage of male patients, percentage of smokers, percentage of glycoprotein IIb/IIIa inhibitor use, percentage of patients with anterior wall infarction, percent difference of direct stenting, mean difference of procedure times, pain-to-balloon time, and door-to-balloon time.
All analyses were conducted in STATA version 10 (StataCorp, College Station, TX, USA). All tests were 2-sided with a significance level of p<0.05 except for meta-regression analysis in which the significance level of p < 0.10 was chosen a priori. We attempted to conform to QUOROM (Quality of Reporting of Meta-analyses) statement in the report of this systematic review11.
Results
Search results and study selection
Our search identified 627 potentially relevant articles, of which 25 trials with a total of 5,919 patients met our inclusion criteria (Figure 1)12-35.
Figure 1. Flow chart for study selection. PCI: percutaneous coronary intervention; SVG: saphenous venous graft.
Among them, PIHRATE trial was the only trial which has yet to be published in full paper. The results were presented in European Society of Cardiology Congress 2008. The study characteristics in each trial are presented in Table 1. A single centre design was more commonly used in Europe, whereas a multicentre design was more commonly used in Asia, the United States and elsewhere (online supplement, Figure 1).
Study quality was similar between single and multicentre trials in all five components except for one, where blinding of clinical outcome assessors was documented in only two (18%) single-centre trials compared with in six (50%) multicentre trials (P=0.05) (Online supplement, Figure 2 and Table 2).
Figure 2. Subgroup analysis for mortality.
Difference in the treatment effects of adjunctive devices in acute myocardial infarction was observed when results were stratified by single or multicentre design or type of devices used.
In addition, the pooled mean door-to balloon time was shorter at 55 minutes in single centre trials than at 96.8 minutes in multicentre trials (online supplement, Figure 3). In comparison, between single or double centre versus multicentre trials, it was significantly shorter at 32 minutes in single or double centre trials than at 123 minutes in multicentre trials (P<0.001) (online supplement Figure 3).
Figure 3. Subgroup analysis for incomplete ST-resolution.
Difference in the treatment effects of adjunctive devices was significant when results were stratified by single or multicentre design, type of devices used, study regions, study size, or presence of conflicts of interests.
Statistical pooling
Overall, a meta-analysis of 25 trials suggested that the use of an adjunctive devices did not improve the survival of patients with AMI (relative risk: 0.78, 95%CI 0.57 to 1.05, p=0.676) with no evidence of heterogeneity (I2=0%) (online supplement, Figure 4).
Figure 4. Subgroup analysis for impaired myocardial blush grade.
Difference in the treatment effects of adjunctive devices was observed when results were stratified by single or multicentre design, or type of devices used.
The use of adjunctive device significantly improved the risk of incomplete ST-resolution (STR) (relative risk 0.77, 95% CI 0.68 to 0.87, p<0.001) (online supplement, Figure 5) and impaired myocardial blush grade (MBG) (relative risk 0.75, 95%CI 0.66 to 0.84, p<0.001) (online supplement, Figure 6), but significant heterogeneity was present (I2=69.2% for incomplete STR and I2=79.6% for impaired MBG).
Figure 5. Meta-regression analysis for the association between the number of centres in a trial and the trial result.
Meta-regression analysis shows that an increase in the number of centres in a trial by one increases the risk of post-PCI incomplete ST-resolution by 1.4% (95%CI 0.48% to 2.28%). The size of each trial corresponds to the inverse variance of the log-transformed relative risk, and thus, is related to the statistical weight of the study.
Visual inspection of the funnel plot revealed asymmetry for both incomplete STR and MBG<3. Begg’s test was significant for publication bias (p<0.001 for both incomplete STR and MBG<3). The trim-and-fill method, which imputed the missing studies and recalculated the pooled relative risk, still showed significant benefit of adjunctive devices for the risk of incomplete STR and impaired MBG with relative risk of 0.82 (95%CI 0.72 to 0.94, p=0.003) and 0.83 (95%CI 0.73 to 0.93, p=0.002), respectively.
Subgroup analyses
The results of subgroup analyses are presented in Figures 2-4. Overall, the difference in the treatment effects was observed when results were stratified by single or multicentre design, type of device, study region, and presence of conflicts of interest. The effect of single centre design on the treatment effect was more significant for the outcomes of incomplete STR and impaired MBG compared with other variables. The pooled relative risk of incomplete STR was 0.53 (95%CI 0.41 to 0.68) in single centre trials compared with 0.94 (95%CI 0.87 to 1.01) in multicentre trials. Similarly, the pooled relative risk of impaired MBG was 0.62 (95%CI 0.50 to 0.78) in single centre trials compared with 0.84 (95%CI 0.74 to 0.96) in multicentre trials.
When results were stratified by type of device used, the use of catheter aspiration devices significantly improved mortality with relative risk of 0.56 (95%CI 0.36 to 0.87, p=0.011), but the use of mechanical aspiration devices and embolic protection device did not (Figure 2). The superiority of catheter aspiration devices over mechanical aspiration devices or embolic protection devices was less significant for the outcomes of incomplete STR (Figure 3) and impaired MBG (Figure 4).
Meta-regression analyses
On univariate meta-regression analyses, single or multicentre design, type of device used, study size, study region, and presence of conflicts of interest were shown to be major sources of heterogeneity (Table 2). Among them, single or multicentre design was the only variable significantly associated with the trial results for all clinical outcomes including mortality (p=0.087 for regression coefficient), incomplete STR (p=0.001), and impaired MBG (p=0.07) with explained variance of 19%, 32%, and 22%, respectively (Table 2). This suggests that single or multicentre study design is the most influential source of heterogeneity. Similarly, the number of centres in a trial was significantly associated with the trial results for incomplete STR (p=0.005) and for impaired MBG (p=0.046) with explained variance of 29% and 22%, respectively. An increase in the number of centres in a trial by one increases the risk of incomplete STR by 1.4% (95%CI 0.48% to 2.28%) (Figure 5) and the risk of impaired MBG by 1% (95%CI 0.02% to 1.97%).
On multivariate meta-regression analyses, single or multicentre design was the most influential source of heterogeneity for the risk of incomplete STR (p=0.01 for regression coefficient) and impaired MBG (p=0.08) respectively, whereas type of device used was the most influential source for the risk of mortality (p=0.056). On the other hand, there was no significant association between the treatment effects and other variables. The associations with the percent difference of direct stenting, the mean difference of procedure time, and door-to-balloon time were not examined because a limited number of trials reported these results.
Discussion
The present systematic review showed that single or multicentre study design has a significant impact on the results of trials examining the use of adjunctive devices in addition to standard PCI during AMI. Overall, single or multicentre study design, type of device used, study size, study region, and presence of conflicts of interest were shown to be the major sources of heterogeneity in trial outcomes. Among them, single or multicentre study design was the only variable significantly associated with the trial results for all clinical outcomes. An increase in the number of centres in a trial by one increases the risk of post-PCI incomplete STR by 1.4% (95%CI 0.48% to 2.28%) and the risk of post-PCI impaired MBG by 1% (95%CI 0.02% to 1.97%).
The present review, compared with previous meta-analyses, disclosed the important weakness in current evidence for the use of adjunctive devices in AMI. Previous meta-analyses conducted subgroup analysis based on only type of devices used and showed the superiority of thrombus aspiration catheters over mechanical thrombectomy devices or embolic protection devices4,5. In contrast, the present review hypothesises that the heterogeneous trial outcomes arise from many variables including study design given the difference in study quality and in the level of expertise of the operator for the use of the adjunctive devices between trials. The more detailed subgroup and meta-regression analyses showed that single or multicentre study design also significantly affects the clinical outcomes in trials examining the adjunctive devices. It should be noted that multiple systematic reviews on the same clinical topic may produce conflicting results as less rigorous reviews may report positive conclusions more frequently36.
This systematic review suggests that current evidence does not support the general use of thrombus aspiration catheters in AMI at this moment given that the favourable outcomes of thrombus aspiration catheters are mainly derived from single-centre trials, not multicentre trials. In fact, the pooled relative risk of mortality with thrombus aspiration catheter was 0.52 (95%CI 0.32 to 0.85, p=0.009) in seven single centre trials but 0.78 (95%CI 0.29 to 2.08, p=0.62) in three multicentre trials, showing the remarkable difference of study outcomes between them.
There are several explanations for the impact of single or multicentre design on the trial results. First, the difference in study quality between them may explain the observed difference of clinical outcomes. For example, there was a difference in documentation of blinding of clinical outcome assessors (18% of single-centre trials compared with 50% of multicentre trials: p=0.05). As trials with high risk of bias may overestimate the treatment effect, the difference in the number of biased trials may lead to biased results favouring the pooled results of single-centre trials.
Second, small-study effects or publication bias, which was confirmed by the asymmetry of the funnel plot and Begg’s test, may overestimate the treatment effect of single-centre trials as the number of study participants was typically smaller in single-centre trials. Meta-regression analyses, however, indicated that the single or multicentre design is far more influential than study size alone on the trial results.
Third, the difference may come from the difference in operator experience. Trials which showed a survival benefit of adjunctive devices were typically conducted at single centres located in Europe with high volume and experienced operators. In contrast, the pooled results of three trials conducted in the Unites States or elsewhere, all of which were multicentre trials, did not show any clinical benefit of adjunctive devices. As the adjunctive devices during AMI have been used more frequently in Europe than other countries, the difference in experience may yield the different results across regions with favourable results in European trials, most of which were single-centre trials. Similarly, the fact that high-volume centres tend to conduct single centre trials whereas low-volume centres conduct multicentre trials may cause the biased results.
Finally, the difference in institutional expertise may exist between centres involved in the trials. For example, the pooled door-to-balloon time was significantly shorter at 31 minutes in single or double centre trials than at 122 minutes in multicentre trials. This suggests that most centres involved in multicentre trials did not achieve the recommended door-to-balloon time of less than 90 minutes, above which mortality of patients with AMI was shown to increase regardless of baseline risks37. Subsequently, multicentre trials may fail to show the beneficial effect of adjunctive devices due to dilution of worse outcome from less specialised centres conducting novel procedures.
As suggested in previous meta-analyses, the treatment outcomes of adjunctive devices also differ according to the type of device. The subgroup analysis showed that thrombus aspiration catheters among adjunctive devices significantly improved the survival of patients with AMI. The meta-regression analysis also suggested that the type of adjunctive device accounted for a significant amount of heterogeneity. However, the significant association of type of device with treatment outcome suggests that the efficacy of adjunctive devices is operator- or institution- dependent. The additional meta-analysis among fourteen trials, which reported procedure time between adjunctive device group and standard PCI group, shows that the weighted mean difference of procedure time is significantly shorter for thrombus aspiration catheter at 4.41 minutes (95%CI 2.74 to 6.08), compared with embolic protection devices or mechanical thrombectomy devices at 11.4 minutes (95%CI 9.88 to 12.93) or 13.8 minutes (95%CI 9.58 to 18.04), respectively (online supplement, Figure 3). This may suggest that the more complicated the procedure with the use of adjunctive devices is, the less likely the trial will show favourable results.
It has been shown that inconsistencies in operator technique resulted in pronounced inter-centre variability in treatment outcomes in a trial examining the new procedures in PCI38. However, most multicentre trials cited in this review neither mention selection criteria for trial centres nor perform the statistical analysis adjusted for centres. Future clinical investigators should choose stringent criteria for inclusion of trial centres and perform statistical adjustments for the potential bias arising from inter-centre variability when conducting multicentre trials examining such highly-skilled procedures.
Limitation
The interpretation of subgroup and meta-regression analysis in a meta-analysis must be used with caution because it is well known that increasing the number of additional analyses in meta-analysis can substantially raise the false-positive rates. In addition, this systematic review was limited to the analysis of aggregate data from published trials. Thus, we could not adjust for confounding variables or explore sources of heterogeneity with regard to patient and angiographic characteristics less reported in included studies. Individual-level data are required to fully assess which subset of patients, such as those with angiographically confirmed thrombus, will benefit from adjunctive devices in addition to standard PCI.
Conclusions
This systematic review showed that single or multicentre study design has a significant impact on the results of trials examining adjunctive devices in addition to standard PCI in AMI. Inter-centre variability of study quality, operator experience, and institution expertise may have lead to the different outcomes with the use of adjunctive devices. Although a recent large single-centre trial and subgroup analyses in previous meta-analyses have suggested a survival benefit of thrombus aspiration catheters among adjunctive devices, it should be confirmed by larger, multicentre, randomised controlled trials to recommend the general use of these devices in AMI. In addition, future clinical investigators should choose stringent criteria for inclusion of trial centres and perform statistical adjustments for the potential bias arising from inter-centre variability.
Acknowledgements
The authors’ responsibilities were as follows—YI: selected studies, extracted and analysed the data, performed the statistical analyses, wrote the manuscript, and takes responsibility for data integrity and accuracy of data analysis; JAC: selected studies, extracted and analysed the data, and contributed to the writing of the manuscript; NM: contributed to the writing of the manuscript: SRB: supervised the present study, contributed to the data interpretation and writing of the manuscript.
References
