Impact of bleeding assessment and adjudication methodology on event rates and clinical trial outcomes: insights from the HORIZONS-AMI trial

Abstract

Aims: Bleeding is a major safety outcome in cardiovascular trials. The present study assessed the impact of the adjudication process of bleeding events on three-year outcomes in the Harmonizing Outcomes with Revascularization and Stents in Acute Myocardial Infarction (HORIZONS-AMI) trial.

Methods and results: HORIZONS-AMI enrolled 3,602 patients with ST-segment elevation myocardial infarction undergoing primary percutaneous coronary intervention. An independent CEC reviewed 445 potential bleeding events identified from three sources: 339 site-reported (SR), 35 CEC-identified, and 71 database (DB)-triggered events based on programmatic identification of a decline in haemoglobin of ≥3 g/dL or in haematocrit by ≥9%; of those, 383/445 (86.1%) met the protocol definition of major bleeding. By multivariable analysis, CEC-confirmed bleeding was an independent predictor of cardiovascular death (hazard ratio [HR] 2.84, 95% confidence interval [CI]: 1.81-4.45, p<0.0001) and all-cause death (HR 2.70, 95% CI: 1.92-3.79, p<0.0001) at three years. Non-CEC-confirmed bleeding was also a predictor of cardiovascular death (HR 3.45, 95% CI: 1.47-8.11, p=0.005) and all-cause death (HR 2.41, 95% CI: 1.11-5.23, p=0.03) at three years.

Conclusions: In the HORIZONS-AMI trial, adjudication of bleeding via a centralised CEC process resulted in identification of a larger number of events than were SR. All CEC-confirmed bleeding events were independently predictive of three-year cardiovascular and all-cause mortality. The association of non-CEC-confirmed bleeding with mortality merits further investigation.

Abbreviations

CEC: clinical events committee

DB: database

GPI: glycoprotein IIb/IIIa inhibitor

GUSTO: Global Use of Strategies to Open Occluded Coronary Arteries

SR: site-reported

TIMI: Thrombolysis In Myocardial Infarction

Introduction

Bleeding is a primary safety measure in cardiovascular randomised controlled trials, and accurate assessment is essential to characterise the risk-benefit trade-off of invasive techniques and antithrombotic therapies. Given the complexity of bleeding definitions and the need for avoidance of investigator bias, site-reported (SR) events are often reviewed and adjudicated by an independent clinical events committee (CEC) to ensure they meet the pre-specified protocol definition of a bleeding endpoint; however, there is no clear evidence demonstrating that this approach is superior to accepting all SR events1-3. Moreover, non-SR bleeds may also be identified during CEC review of ischaemic or other non-bleeding events or suspected by routine surveillance for haemoglobin reductions. The extent to which these alternative sources contribute to the total bleeding rate, and whether events so identified have a similar prognostic impact to SR bleeding is unknown. Given the increasing cost of clinical trials, if centralised CEC-based bleeding event assessment offers little incremental value compared to site reporting, expenses may be reduced by not employing a CEC; conversely, if CEC assessment identifies substantially more events than investigators, and/or if those events carry more prognostic weight, use of a CEC may be cost-effective by increasing trial accuracy and power and reducing sample size.

We therefore reviewed the bleeding event adjudication process in the randomised Harmonizing Outcomes with Revascularization and Stents in Acute Myocardial Infarction (HORIZONS-AMI) trial to assess the frequency and relative risk of CEC-confirmed SR, CEC-identified and database (DB)-triggered bleeding events on subsequent cardiovascular and all-cause mortality as well as the frequency and impact of non-CEC-confirmed bleeding.

Methods

STUDY DESIGN

Details of the rationale, design, and principal results of the HORIZONS-AMI trial have been published previously4. In brief, HORIZONS-AMI was a prospective, open-label trial that compared procedural anticoagulation with bivalirudin alone versus heparin plus a glycoprotein inhibitor (GPI) in 3,602 patients with ST-segment elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention (PCI). Institutional approval and subject informed consent were obtained at all sites. Eligible patients were randomised again in a 3:1 ratio to paclitaxel-eluting (TAXUS™) or bare metal Express™ stents (both Boston Scientific, Marlborough, MA, USA). Two primary endpoints were pre-specified: protocol-defined major bleeding (not related to coronary artery bypass grafting) and net adverse clinical events, defined as the composite of death, reinfarction, target vessel revascularisation, stroke or major bleeding. The protocol major bleeding endpoint was defined as intracranial or intraocular haemorrhage, haemoglobin decrease ≥4 g/dL without or ≥3 g/dL with an overt bleeding source, reoperation for bleeding, blood transfusion, or access-site bleeding with a ≥5 cm diameter haematoma or requiring intervention. Bleeding was also adjudicated by the Thrombolysis In Myocardial Infarction (TIMI) and Global Use of Strategies to Open Occluded Coronary Arteries (GUSTO) scales. All primary endpoint and stent thrombosis events were adjudicated by an independent CEC consisting of three interventional and three non-invasive cardiologists coordinated by the Cardiovascular Research Foundation (New York, NY, USA) by review of original source documents (clinical and laboratory reports), applying strict protocol-specified definitions. The CEC was blinded to treatment assignment and adjudication decisions were reached via consensus.

SOURCE AND ASSESSMENT OF BLEEDING EVENTS

The CEC reviewed pertinent medical documentation and adjudicated all potential bleeding events according to the pre-specified definitions. All bleeds reviewed by the CEC were categorised as either meeting the protocol definition of major bleeding (i.e., “CEC-confirmed”) or not meeting the protocol definition of bleeding (“non-CEC-confirmed”). Potential bleeding events were identified from three possible sources. 1) SR bleeding events were identified by study investigators, research coordinators, and independent monitors. The CEC evaluated all SR events by reviewing supporting source documents collected by independent study monitors (J. Tyson & Associates in the USA, D-TARGET in Europe, and Tango in South America). SR events adjudicated by the CEC as meeting the protocol definition of major bleeding are termed “SR CEC-confirmed bleeds,” while SR events that did not meet the protocol definition of major bleeding are termed “SR non-CEC-confirmed bleeds”. 2) The CEC also identified bleeding events not otherwise SR (“CEC-identified bleeding”) during source document review for assessment of non-bleeding endpoints (e.g., death or repeat revascularisation), which were then adjudicated to determine if they met the protocol-defined bleeding endpoint criteria. These events, if not previously reported by site investigators, are termed “CEC-identified CEC-confirmed bleeds” or “CEC-identified non-CEC-confirmed bleeds” depending on whether they were or were not confirmed by the CEC as meeting the protocol definition for bleeding, respectively. 3) Potential bleeding events were also identified by an algorithmic DB-triggering mechanism based on programmatic identification of a decline in haemoglobin of ≥3 g/dL or in haematocrit by ≥9% compared to baseline, which triggered site queries and collection of source documents for CEC adjudication (“DB-triggered bleeding”), if not previously SR or CEC-identified. These events are termed “DB-triggered CEC-confirmed bleeds” and “DB-triggered non-CEC-confirmed bleeds” depending on whether or not they were subsequently confirmed as meeting the protocol definition for bleeding. For the present analysis, three mutually exclusive groups of CEC-confirmed bleeding events were thus defined based on their original source: SR, CEC-identified, and DB-triggered.

STATISTICAL ANALYSIS

Baseline characteristics were compared using the χ2 or Fisher’s exact test for categorical data and one-way analysis of variance for continuous data. The incidence of bleeding at 30 days and three years was compared between bivalirudin and heparin plus GPI groups using χ2 or the Fisher’s exact test for each of the bleeding event definitions. The association of CEC-confirmed and non-CEC-confirmed bleeding events with subsequent mortality was assessed using Cox proportional hazards regression. Additional covariates in these models included age, sex, diabetes mellitus, prior myocardial infarction, Killip classification (I versus >I), infarct artery (left main or left anterior descending artery versus other), baseline haemoglobin, baseline creatinine clearance, baseline TIMI flow, and randomisation arm. The proportional hazards assumption was tested and found to be tenable for all covariates. All statistical tests were two-tailed. Statistical significance was set at a level of 0.05. All analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC, USA).

Results

BLEEDING OUTCOMES ACCORDING TO SOURCE

During the three-year course of the trial, a total of 445 potential bleeding events were identified; following CEC review, 383 of these 445 events (86.1%) were adjudicated as meeting the protocol definition of major bleeding, of which 316 (82.5%) were initially SR, 35 (9.1%) were identified by the CEC, and 32 (8.4%) were identified by the DB-triggering algorithm (Figure 1). Considering all bleeding definitions (Supplementary Table 1), SR events comprised anywhere from 72.6% to 96.9% of the total adjudicated bleeding events. Of note, no CEC-categorised “non-CEC-confirmed bleeds” (i.e., reviewed events that did not meet the protocol definition of major bleeding) were adjudicated as TIMI or GUSTO bleeds.

Figure 1. Potential bleeding endpoints identified by the site investigators, research personnel, and monitors (site-reported [SR]), the clinical events committee during review of other endpoint events (CEC-identified), and the database-triggering process (DB-triggered). All events were adjudicated by the CEC through review of original source documents. A potential bleeding event assessed as a non-bleed by the CEC is denoted as a “non-CEC-confirmed bleed”.

CORRELATES OF PROTOCOL-DEFINED BLEEDING

Baseline clinical and angiographic characteristics of patients according to bleeding events are shown in Supplementary Table 2. Compared to patients with no bleeding, those with CEC-confirmed or non-CEC-confirmed bleeds were older, more frequently female, and more frequently had hypertension, diabetes, history of smoking and lower baseline creatinine clearance. Heart failure and left main or left anterior descending infarct artery were more frequently observed in patients with bleeding compared to patients without bleeding. Among patients with CEC-confirmed bleeding, clinical and angiographic characteristics did not differ between source groups except for baseline haemoglobin levels and creatinine clearance.

Table 1 shows the 30-day and three-year rates of CEC-confirmed and non-CEC-confirmed events in patients randomised to heparin plus a GPI versus bivalirudin. All CEC-confirmed and SR bleeding events were significantly reduced with bivalirudin compared to heparin plus a GPI at 30 days and three years, whether defined by the protocol, TIMI, or GUSTO definitions; similar point estimates were present for the smaller number of CEC-identified and DB-triggered events. Non-CEC-confirmed events were also numerically lower in the bivalirudin arm compared to the heparin plus GPI arm.

IMPACT OF BLEEDING ON MORTALITY

The proportion of deaths and cardiovascular deaths at three years in patients with any CEC-confirmed protocol-defined bleeding, non-CEC-confirmed bleeds and no reported bleeding is displayed in Supplementary Table 3. Patients with CEC-confirmed bleeds and non-CEC-confirmed bleeds had higher three-year rates of cardiovascular and all-cause mortality compared to patients with no reported bleed.

By multivariable analysis (Table 2), CEC-confirmed bleeding was an independent predictor of three-year cardiovascular death (model 1a, HR 2.84, 95% confidence interval [CI]: 1.81-4.45, p<0.0001) and all-cause death (model 2a, HR 2.70, 95% CI: 1.92-3.79, p<0.0001). When the source of bleeding was entered into separate models assessing cardiovascular death (model 1b) and all-cause death (model 2b), SR and CEC-identified CEC-confirmed bleeds, but not DB-triggered CEC-confirmed bleeds, were predictive of cardiovascular and all-cause death. Non- CEC-confirmed bleeds were also significant predictors of cardiovascular and all-cause death. Sensitivity analyses including all SR and DB-triggered bleeds (CEC-confirmed and non-CEC-confirmed) showed a significant association between these bleeds and all-cause death (HR 2.49, 95% CI: 1.78-3.47, p<0.001) and cardiovascular death (HR 2.60, 95% CI: 1.73-4.11, p<0.0001).

Discussion

The major findings from the present analysis from the HORIZONS-AMI trial, in which the frequency and prognostic impact of CEC-confirmed and non-CEC-confirmed bleeding events in patients with STEMI undergoing primary PCI were examined, are as follows. 1) Although the majority of major bleeding events were originally site-reported, a substantial number of additional events were detected during CEC review of non-bleeding events and by an automated DB-triggering algorithm. 2) The randomised study conclusions were similar when only SR versus all CEC-confirmed bleeds were used in comparing bivalirudin to heparin plus GPI. However, utilisation of all CEC-confirmed bleeds substantially increased the event rate in both arms. 3) All CEC-confirmed bleeding events, and particularly SR and CEC-identified, were independently predictive of three-year cardiovascular and all-cause death. 4) Non-CEC-confirmed bleeds were also predictive of three-year cardiovascular death and all-cause death.

Bleeding occurs commonly in patients with acute coronary syndromes undergoing invasive management. Although major bleeding has been strongly linked with adverse outcomes5-7, the frequency and relative risk of bleeding varies from study to study3, given the variety of bleeding definitions in use8 and inconsistent methodology and rigour in tracking and assessing bleeding events9. In the HORIZONS-AMI trial, a robust process of centralised event adjudication was utilised to evaluate potential bleeding outcomes. In addition to CEC review of bleeding events reported by the sites, the CEC sought potential bleeding episodes during review of non-bleeding events (e.g., repeat revascularisation), as well as through an algorithmic DB-triggering process seeking reductions in haemoglobin levels greater than a set threshold. Identification of a potential bleeding event from any of these three sources triggered in-depth review of source documents to determine whether it met the pre-specified protocol definition of major bleeding (which utilised clinical and laboratory-based criteria) or alternative bleeding definitions based on clinical (GUSTO) or laboratory (TIMI) scales. Addition of CEC-identified and DB-triggered events increased the total number of CEC-confirmed protocol-defined major bleeding events by 21.4%, with the increment contributed to almost equally by the two processes. Similar patterns were noted when bleeding rates were assessed according to the TIMI and GUSTO scales, although the overall bleeding rate and the proportion of non-SR events varied somewhat by bleeding definition, consistent with a recent report suggesting a more robust identification of bleeding based on the GUSTO compared to the TIMI scale10. The identification of a substantial proportion of non-SR bleeds reinforces the previously described limitations of relying solely on site sources to identify events3. Although, in retrospect, HORIZONS-AMI was sufficiently large that the SR bleeding events alone would have resulted in similar study conclusions, identifying additional endpoint events as a rule increases study power and may decrease the sample size required during the trial design phase. Accordingly, other studies have shown that centralised adjudication may increase event rates compared to investigator reporting without a significant effect on reported study outcomes11-14. The present study confirms and extends these results to a broader definition of prognostically significant major bleeding and is unique in reporting the relative rates and associated mortalities of SR vs. CEC-identified vs. DB-triggered bleeds. The fact that DB-triggered CEC-confirmed bleeds were not related to subsequent mortality is also worth noting. Thus, use of a CEC may allow smaller studies, potentially reducing study costs (or at least offsetting the costs for the CEC). Further, our data may alternatively suggest that more rigorous site training may improve identification of potential bleeding events by the sites and eliminate the need for centralised adjudication and database triggering; however, screening for site-reported events is not typically performed by physicians and, further, site monitors and investigators often lack expertise related to endpoint definitions, as suggested by the observed 6.5% rate of SR events deemed non-events by the CEC. These limitations would probably hinder a significant improvement in site reporting.

Perhaps more importantly, as demonstrated in our study, use of a CEC is required to ensure that the protocol definition of bleeding is actively met and may minimise bias and variability in reporting, particularly in unblinded studies and FDA-regulated trials, as well as allow more accurate cross-validation of reported results across studies. For example, Guimaraes and colleagues recently showed that the rates of bleeding based on claims data were significantly lower than adjudicated bleeding15. Conversely, relying on transfusion data as the sole evidence to define bleeding (without requiring evidence of active bleeding) inadvertently increased bleeding rates, indicating inaccuracies in event reporting without use of an independent CEC15.

In the present analysis, CEC-confirmed bleeding was associated with three-year cardiovascular and all-cause death. The hazard between bleeding and subsequent mortality was similar in magnitude for bleeds sourced by the sites and the CEC (the latter not identified by the sites), supporting the role of the CEC in searching for such events. Conversely, DB-triggered CEC-confirmed bleeds were not independently predictive of death. A potential explanation for this paradox is that algorithmic triggering largely identifies non-overt bleeding, the mechanisms of which may be innocuous (e.g., blood loss during catheter exchange or post-PCI haemodilution). Alternatively, laboratory-based reductions in haemoglobin without overt bleeding may be secondary to or confounded by clinical conditions or treatments included in the multivariable model, such as renal insufficiency or randomised treatment with heparin plus a GPI; however, as only 32 isolated DB-triggered events met the protocol definition for major bleeding, larger studies are required to determine whether algorithmic identification of such events is useful.

Importantly, though the CEC rejected 6.5% of proposed SR events and 56.3% of DB-triggered events after source document review, non-CEC-confirmed bleeds remained independently associated with cardiovascular and all-cause mortality, with adjusted hazard ratios of similar magnitude to those noted for CEC-confirmed bleeding. When reporting bleeding events that do not reach threshold levels of the protocol definition, sites may integrate other symptoms or signs leading to the belief that the bleed is clinically important; however, only 62 events were categorised as non-CEC-confirmed bleeds and thus a spurious finding cannot be excluded. The association of these events with mortality may be due to unmeasured confounders such as insufficient supporting source documentation for confirmation of a bleeding endpoint definition. In addition, non-CEC-confirmed bleeds have a variety of origins, either falling outside the standardised bleeding definitions, or having insufficient supporting documentation or early interventions mitigating the severity of the event. Larger studies are warranted to examine the prognostic impact of SR and DB-triggered bleeds that do not reach the threshold for a CEC-confirmed event.

Limitations

This present post hoc analysis should be considered hypothesis-generating. Nonetheless, data collection and CEC processes in the HORIZONS-AMI trial were robust, and all bleeding definitions were pre-specified. Second, HORIZONS-AMI was conducted before implementation of the Bleeding Academic Research Consortium (BARC) definitions16. As a result, we cannot assess whether the study results would have varied by utilisation of these definitions, although the overlap in the BARC criteria with the protocol definition and the consistency of the findings with the TIMI and GUSTO scales suggest that significant differences would be unlikely. Third, the exact reasons why the CEC judged that some SR events did not meet criteria for protocol-defined events were not collected. Fourth, HORIZONS-AMI did not include a formal cost substudy; as such, the potential cost implications of the present study cannot be directly determined. Finally, as previously noted, the numbers of CEC-identified and DB-triggered CEC-confirmed events as well as non-CEC-confirmed bleeds were modest, and we cannot exclude the impact of unmeasured confounders.

Conclusions

In the HORIZONS-AMI trial, adjudication of bleeding events via a robust centralised CEC process resulted in identification of a larger number of events than reported by the sites alone. All CEC-confirmed bleeding events, particularly SR and CEC-identified, were independently predictive of three-year cardiovascular and all-cause mortality. Non-CEC-confirmed events were also associated with subsequent mortality, an observation that merits further investigation.

Conflict of interest statement

R. Mehran has received research funding to the institution from AstraZeneca, Bayer, Beth Israel Deaconess, BMS, CSL Behring, Eli Lilly/DSI, Medtronic, Novartis Pharmaceuticals, and OrbusNeich, and consultant funding to the institution from Abbott Laboratories, Boston Scientific, Medscape, Spectranetics, The Medicines Company (spouse) and Abiomed (spouse). She holds equity (<1%) in Claret Medical and Elixir Medical, is on the executive committee of Janssen Pharmaceuticals and Osprey Medical, and is a DSMB member of Watermark Research Partners. G. Stone reports grants from Abbott Vascular, paid fees to the Cardiovascular Research Foundation for study trial processes, during the conduct of the study; personal fees from Medical Development Technologies, St. Jude, Ablative Solutions, Claret, Sirtex, Matrizyme, Amaranth, BackBeat Medical, Miracor, Neovasc, V-wave, Shockwave, Valfix, TherOx, Reva, Vascular Dynamics, Robocath; other from Aria, Biostar family of funds, MedFocus family of funds, Ancora, Cagent, Qool Therapeutics, SpectraWave, and Caliber, outside the submitted work; and Columbia University, his employer, receives royalties from Abbott Vascular for sale of the MitraClip. The other authors have no conflicts of interest to declare.

Supplementary data

Supplementary Table 1. Rates of CEC-confirmed bleeding events within 3 years according to original source.

Supplementary Table 2. Baseline characteristics in patients with and without bleeding during 3-year follow-up according to event type.

Supplementary Table 3. Proportion of deaths occurring within 3 years in patients with CEC-confirmed protocol-defined bleeding, non-CEC-confirmed bleeding and no reported bleeding event.

To read the full content of this article, please download the PDF.