External validity of a contemporaneous primary percutaneous coronary intervention trial in patients with acute ST-elevation myocardial infarction: insights from a single-centre investigation

Abstract

Aims: Randomised controlled trials (RCTs) represent the most robust source of evidence-based medicine. However, the generalisability of RCTs is limited by the inclusion of selected populations. We sought to assess the external validity of a contemporary trial including patients with ST-elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention (PPCI).

Methods and results: Patients presenting to Bern University Hospital during the inclusion period of the COMFORTABLE AMI trial were divided into three groups: RCT participants (41%), eligible not included (17.5%), and excluded patients (41.5%). Major adverse cardiac events (MACE) were defined as one-year death and myocardial infarction. RCT participants compared with RCT-eligible patients had comparable baseline characteristics and outcomes; however, excluded patients differed in risk and had higher rates of MACE (HR 3.63, 95% CI: 2.03-6.48, p<0.001), death (HR 6.23, 95% CI: 2.93-13.24, p<0,001) and definite/probable stent thrombosis (HR 3.63, 95% CI: 1.79-7.36, p<0.001). Inability to provide consent was the most frequent exclusion criterion and was independently associated with an increased risk for MACE (HR 6.85, 95% CI: 3.97-11.81, p<0.001).

Conclusions: In this single-centre investigation, results from the COMFORTABLE AMI trial appeared applicable to a broad representation of RCT-eligible patients. However, patients excluded from the trial represented a higher-risk population with impaired clinical outcomes and a lower adherence to cardiovascular medication.

Introduction

Randomised controlled trials (RCTs) represent the most robust source of evidence-based medicine and assume the highest level of support in clinical practice guidelines1. Notwithstanding this, RCTs include selected patient populations, and it remains uncertain whether the results of these studies pertain to a broader spectrum of patients encountered in daily clinical routine2. While the assessment of specific populations under controlled conditions increases the internal validity, it may jeopardise the external validity of the established results3. All-comers trials aim to recruit a wide spectrum of patients including those at higher risk for cardiac events, which should result in higher event rates and a more rapid recruitment. However, a previous single-centre report suggested that, even with an all-comers design, only approximately half of potentially eligible patients may be enrolled2.

The COMFORTABLE AMI (Comparison of Biolimus Eluted From an Erodible Stent Coating With Bare Metal Stents in Acute ST-Elevation Myocardial Infarction; NCT00962416) trial, was an all-comers RCT which included 1,161 patients with acute ST-elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention (PPCI)4. The use of a biolimus-eluting stent (BES) was associated with a significant reduction of the primary endpoint at one year with a sustained effect over an additional year of follow-up5. The results of this and a second PPCI trial6 were the basis for a change of the ESC guidelines for myocardial revascularisation in STEMI patients (i.e., a class IA indication for the use of newer-generation DES)1.

The aim of the present study was to assess differences in baseline and procedural characteristics and long-term clinical outcomes between patients included in the COMFORTABLE AMI trial, patients who were eligible but not included in the trial, and those excluded from trial participation due to formal exclusion criteria at the centre contributing the largest patient number.

Methods

STUDY DESIGN AND PATIENT POPULATIONS

Details of the design of the COMFORTABLE AMI trial have been published elsewhere7. This multicentre, assessor-blinded, superiority RCT was performed at 11 sites in Europe and Israel between September 2009 and January 2011. Patients were assigned to treatment with a BES (BioMatrix™; Biosensors Europe SA, Morges, Switzerland) or a bare metal stent (BMS) (Gazelle™; Biosensors Europe SA, Morges, Switzerland).

The Bern PCI Registry (NCT02241291) is a prospective registry which includes all consecutive patients undergoing PCI at Bern University Hospital. All cardiovascular events are adjudicated by two independent physicians. Of note, adverse event monitoring, data management and event adjudication were identical between patients included in COMFORTABLE AMI and those included in the Bern PCI Registry.

All consecutive patients fulfilling the criteria of STEMI and undergoing PPCI at Bern University Hospital within the inclusion period of COMFORTABLE AMI were included in this analysis, and divided into three groups: a) study participants (RCT), b) eligible but not included patients (eligible), and c) excluded patients (non-eligible). The study complied with the Declaration of Helsinki and was approved by the institutional ethics committee.

DATA MANAGEMENT

Data were stored in a dedicated database (Cardiobase; Clinical Trials Unit, and Department of Cardiology, Bern University Hospital, Switzerland). Complete follow-up was available for 96% of COMFORTABLE participants, 91.5% of eligible not included and 90.1% of excluded patients. COMFORTABLE AMI events were independently adjudicated by a blinded clinical events committee (CEC).

STUDY ENDPOINTS

Similar clinical outcome definitions were applied for COMFORTABLE AMI patients and the Bern PCI Registry. The primary endpoint was the composite of all-cause death and any myocardial infarction at one year. Myocardial infarction was defined according to the extended historical definition8. Other endpoints followed the definitions of the Academic Research Consortium9.

STATISTICAL ANALYSIS

For the purposes of the present study, data from all patients undergoing PPCI for STEMI treated in our institution during the time period of the COMFORTABLE AMI trial were analysed. Continuous variables are expressed as means±standard deviation or medians with interquartile ranges. P-values were derived using t-tests in the former and Mann-Whitney U tests in the latter. Categorical data are expressed as frequencies and percentages and are compared using the χ2 and Fisher’s exact tests. For nested variables in lesion-level data, p-values from general and generalised mixed models were used. Life-table estimates for events were calculated per group, and we used Cox regressions for comparisons. We report hazard ratios (HR) with 95% confidence intervals. In case of zero events, we reported continuity corrected risk ratios (RR) with 95% confidence intervals and Fisher’s test p-values. Logistic regression was used for the identification of risk factors. All p-values shown are from two-sided tests, and the level of statistical significance was set at 0.05. Analyses were performed using Stata 12 (StataCorp LP, College Station, TX, USA).

Results

During the time period for inclusion of the COMFORTABLE AMI trial, 608 patients underwent PPCI for STEMI at Bern University Hospital. One patient declined participation in the registry. Out of 607 patients, 252 (41.5%) did not meet eligibility criteria for the trial; 106 (17.5%) were eligible not included patients; 249 (41%) were included in the trial.

BASELINE CHARACTERISTICS

Baseline characteristics are summarised in Table 1. With the exception of fewer male patients and more patients presenting with Killip III class in the eligible compared to the RCT group, no differences in baseline characteristics were noted. Compared with RCT participants, non-eligible patients were older, more frequently had a history of previous MI, PCI, gastrointestinal bleeding, previous malignancy, and more often suffered from chronic obstructive lung disease and anaemia.

PROCEDURAL CHARACTERISTICS

Procedural characteristics are summarised in Table 2. We observed a gradient in the number of lesions treated per patient, with the highest number of lesions treated in the non-eligible group. The need for haemodynamic support was not different between RCT participants and eligible patients, but was more frequent in the non-eligible patients; similarly, a lower LVEF was recorded in the non-eligible group.

CARDIOVASCULAR MEDICATIONS

Medication use is summarised in Table 3. We observed differences in antiplatelet agents used for loading, with more frequent use of clopidogrel alone in eligible and non-eligible, and of clopidogrel and subsequent prasugrel loading in RCT participants10. In a dedicated analysis for DAPT adherence at one year (Figure 1), excluding patients with reasons which were expected to interfere (e.g., use of coumadin), we observed a remaining risk increase for DAPT cessation in patients not enrolled in the study.

Figure 1. Dual antiplatelet therapy (DAPT) cessation within one year. First stop recorded as time-to-DAPT cessation. Patients meeting the following criteria were excluded from this analysis: in-hospital death (31), without DAPT at discharge (19), lost at follow-up (45), oral anticoagulation at discharge (13), history of bleeding diathesis (7), and planned surgery with DAPT interruption (11). Blue: study participants (SP); green: eligible not included patients (EP); red: non-eligible patients (NEP); USD: number of patients with uncertain stop dates.

IN-HOSPITAL AND ONE-YEAR OUTCOMES

We observed no differences between RCT participants and eligible patients for any cardiovascular endpoints at both time points (Table 4). At discharge, non-eligible patients were associated with a higher risk of MACE compared with RCT participants (HR 2.63, 95% CI: 1.57-4.41, p<0.001), which was mainly driven by differences in the rate of all-cause death (HR 2.98, 95% CI: 1.65-5.38, p<0.001) (Figure 2A). Results at one year were consistent (HR 3.63, 95% CI: 2.03-6.48, p<0.001), driven by the higher risk of all-cause death (HR 6.23, 95% CI: 2.93-13.24, p<0.001). A landmark analysis at 30 days revealed no interaction by time (Figure 2B). Non-eligible patients were not associated with an increased risk of bleeding as assessed using the BARC, TIMI and GUSTO classifications.

Figure 2. Kaplan-Meier curves and landmark analysis. A) All-cause death or myocardial infarction at one year. B) Landmark analysis at 30 days and between 30 days and one year for all-cause death or myocardial infarction. Blue: study participants, green: eligible not included patients, red: non-eligible patients.

PREDICTORS OF EVENTS AMONG RCT NON-ELIGIBLE PATIENTS

Exclusion criteria that were independently associated with a significantly higher rate of MACE were inability to provide informed consent (HR 6.85, 95% CI: 3.97-11.81, p<0.001), increased baseline bleeding risk (HR 3.59, 95% CI: 1.66-7.77, p=0.001), and mechanical complications of acute MI (HR 8.73, 95% CI: 2.05-37.07, p=0.003) (Figure 3).

Figure 3. Effect of individual exclusion criteria on all-cause death and myocardial infarction at one year. aEvents in RCT participants and RCT-eligible patients. bContinuity corrected risk ratio (95% CI) with Fisher’s exact test.

Discussion

The principal findings of this single-centre investigation of the external validity of a contemporaneous PPCI trial are as follows:

1. In-hospital and one-year cardiovascular event rates were comparable between STEMI patients enrolled in the COMFORTABLE AMI trial and RCT-eligible but not included patients, attesting to the external validity of the results of the original trial.

2. Baseline characteristics were comparable among RCT participants and RCT-eligible patients, whereas RCT-ineligible patients carried a higher burden of comorbidities and procedural factors.

3. STEMI patients fulfilling exclusion criteria carried a higher cardiovascular mortality, without difference in the risk of myocardial infarction or repeat revascularisation.

4. RCT participants underwent more structured invasive and pharmacological interventions and showed an increased adherence to recommended cardiovascular medication including DAPT when compared with non-trial participants.

5. Inability to provide informed consent was the most frequent exclusion criterion and was associated with a seven times higher risk of death or myocardial infarction.

External validity is defined as the reasonable applicability of the results of a study to a definable group of patients in a particular clinical setting in routine clinical practice3. Rothwell et al proposed six criteria for the assessment of external validity3. We previously reported details on the selection of patients, outcome measures, follow-up quality and adverse effects of the treatment4. The present analysis focuses on the characteristics of randomised versus non-randomised patients and differences between the trial protocol and routine clinical practice at the centre contributing the largest patient number of the trial.

All patients included in this analysis were screened by a dedicated physician (24 hours/7 days) for enrolment into the PPCI trial. Pre-specified exclusion criteria were met by 41% of patients and an additional 18% either declined participation or were not included, as factors were present that precluded randomisation despite not being predefined as exclusion criteria. In addition, factors unrelated to the patients may have prevented timely consent and randomisation, such as treatment of multiple STEMI patients at the same time, unavailability of both stents, or fatigue of the operator during night hours. Overall, two thirds of eligible patients were actually included in the RCT. A comparable inclusion rate (60%) was reported in the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial, a contemporaneous all-comers RCT in STEMI patients investigating the impact of thrombus aspiration on mortality at 30 days11. Conversely, the recently published How Effective are Antithrombotic Therapies in Primary Percutaneous Coronary Intervention (HEAT-PPCI) trial was able to randomise 95% of patients scheduled for an emergent angiography12. The high inclusion rate was largely the consequence of a policy of delayed consent, preserving the inclusion of critically ill patients who were unable to provide consent before the procedure.

Patients who were not eligible for RCT inclusion showed a significantly higher baseline risk profile in this analysis. It is noteworthy that one third was unable to provide informed consent owing to a critical condition including a sixfold more frequent presentation with Killip class IV compared with trial participants, which in turn is one of the strongest predictors of mortality among STEMI patients13. As a result, the present study indicates that in-hospital and one-year outcomes are considerably impaired among RCT-ineligible patients. Similar findings have been reported in a recent sub-analysis of the Harmonizing Outcomes with Revascularization and Stents in Acute Myocardial Infarction (HORIZONS-AMI) trial14. Exclusion due to late presentation (>24 hours) was not infrequent. Although PPCI may be considered for this subset of patients, several factors (i.e., residual ischaemia, intermittent coronary flow, or collateral flow) may influence the benefits expected from the intervention, preventing to some degree the comparability with patients receiving early invasive treatment15.

Eligible not included patients have shown a higher baseline risk when compared to included patients in previous studies, denoting the impact of clinical selection beyond eligibility criteria16. We identified conditions that may have influenced the decision to enrol the patient such as multiple comorbidities, doubtful future adherence (i.e., drug addiction, psychiatric disorders), among others, in the eligible not included group. Both the beneficial impact of experimental interventions or therapies and a closer medical attention have been suggested as reasons for improved outcomes in RCT participants16,17. Indeed, a more aggressive antiplatelet treatment was observed in RCT participants (i.e., more frequent use of glycoprotein IIb/IIIa inhibitors and prasugrel), as well as a higher frequency of patients on DAPT throughout one year. Specifically, we observed that real-life STEMI patients tend to stop DAPT earlier than initially prescribed, a finding suggesting that adherence to DAPT requires monitoring by the treating GP or cardiologist.

Despite these differences, a similar baseline risk and comparable cardiovascular event rates were observed in this all-comers RCT among RCT participants and eligible not included patients, attesting to the high degree of external validity. Considering the impact of COMFORTABLE AMI (together with EXAMINATION) on the current revascularisation guidelines, fulfilling the requirements for external validity is of great importance. In agreement with our data, the TWENTE all-comers trial confirmed similar clinical outcomes in study patients and eligible not included NSTEMI patients17.

While there is no evidence against the applicability of the results in non-eligible patients, or a pathophysiological basis to support such conclusions18, it seems prudent to rely on RCTs and clinical guidelines for decision making. Notwithstanding this, the increasing complexity of patients undergoing interventional procedures today often compels us to an individualised approach. The effect of drugs and devices may differ in subsets of patients largely excluded from RCTs. To improve the representation of critically ill patients within RCTs, postponing the informed consent to a time point after the random treatment allocation may be the only solution, something that certainly requires intense ethical consideration.

Limitations

Our results have to be interpreted in view of the following limitations. First, despite the multicentre nature of the COMFORTABLE AMI trial, we performed our analysis based on data from a single centre that contributed the largest number of patients. The availability of a prospective registry and the unified criteria for definition of events at Bern University Hospital have favoured this approach, and this strategy has been utilised in previous studies2,14. Second, reasons for eligibility beyond exclusion criteria were not prospectively recorded, and we were not able to provide the reasons that prevented randomisation for the majority of eligible, non-randomised patients. Third, the use of drug-eluting stents was underrepresented in the RCT population; however, limitations inherent to the sample size prevented stratification by stent type for the purpose of comparisons. Fourth, longer follow-up may identify differences that cannot be detected in a one-year period. However, Kaplan-Meier curves suggest that the findings of the current analysis may prevail over time. Finally, as a post hoc analysis, the present study should be considered exploratory and hypothesis-generating.

Conclusions

Contemporaneous RCTs in patients presenting with ST-elevation myocardial infarction, with less stringent exclusion criteria, provide an appropriate representation of eligible not included patients if conducted using an all-comers approach. Excluded patients represent a higher-risk population, and caution should be applied when extrapolating results from RCTs to this subset of patients. Inability to provide written consent is the most frequent reason for exclusion and associated with a substantially increased mortality.

Funding

S. Windecker and P. Jüni were supported by the Swiss National Science Foundation (33CM30-124112 and 310030-118353). E. Spitzer received a Research Fellowship from the European Association of Percutaneous Cardiovascular Interventions (EAPCI) of the European Society of Cardiology and a research grant from the Spanish Society of Cardiology. Thomas Zanchin is supported by the Swiss National Science Foundation (323530-171146). The COMFORTABLE trial was supported by an unrestricted research grant from Biosensors Europe SA, Switzerland.

Conflict of interest statement

Clinical Trials Unit Bern, which is part of the University of Bern, has a staff policy of not accepting honoraria or consultancy fees. S. Windecker has received research contracts to the institution from Abbott, Boston Scientific, Biosensors, Biotronik, Cordis, and Medtronic. P. Jüni is an unpaid steering committee or statistical executive committee member of trials funded by Abbott Vascular, Biosensors, Medtronic, and St. Jude Medical. L. Räber has received research contracts to the institution from St. Jude Medical. The other authors have no conflicts of interest to declare.