Derivation and external validation of a novel risk score for prediction of 30-day mortality after percutaneous coronary intervention

Introduction

Percutaneous coronary intervention (PCI) is one of the most widely utilised medical procedures in the USA. It is performed in nearly one million patients annually1. Increasingly, more of these procedures are performed in patients presenting with acute coronary syndromes (ACS) rather than stable angina pectoris. Numerous risk scores have been proposed to predict in-hospital or 30-day mortality after PCI2^,3^,4, but nearly all have been derived from cohorts treated before the year 2000; even an updated version of the Mayo Clinic Risk Score (MCRS) included patients treated between 2000 and 20055. Improved stent technology, better adjunctive pharmacology, and more widely used intravascular imaging have improved outcomes for patients undergoing PCI in the past decade. In addition, most of the prior risk scores were derived from quality assurance registries with suboptimal data auditing. We therefore sought to derive a contemporary risk score to predict 30-day mortality after PCI in patients with and without ACS from high-quality randomised trial data.

Methods

Individual patient-level data from 21 randomised trials examining different types of stent and anticoagulation strategies performed and reported between 1999 and 2016 were pooled in a centralised database at the Cardiovascular Research Foundation (New York, NY, USA)6. The list of trials and selected characteristics appears in Supplementary Table 1. Institutional review board or ethics committee approval for each study was secured by the trial investigators during their conduct. Baseline characteristics and adjudicated events from each study were tabulated. Cardiac events, such as death, spontaneous myocardial infarction (MI), target lesion revascularisation, and target vessel revascularisation were adjudicated by independent committees in each study according to protocol-specific definitions utilising original source documents. Stent thrombosis was adjudicated according to the Academic Research Consortium definite or probable definitions7. Studies performed before this definition was instituted were re-adjudicated after publication8.

The studies compared different types of stent (bare metal stents, first-generation drug-eluting stents [DES] [sirolimus-eluting and paclitaxel-eluting], and second-generation DES [biolimus-eluting, everolimus-eluting, and zotarolimus-eluting]), various antithrombotic regimens (heparin with or without glycoprotein IIb/IIIa inhibitors or bivalirudin), and randomised patients with different clinical presentations (stable coronary artery disease [CAD] and ACS, including ST-segment elevation myocardial infarction [STEMI], non-STEMI, and unstable angina). The endpoint of 30-day all-cause mortality was available in every study.

STATISTICAL ANALYSIS

Categorical variables were compared using the χ2 or Fisher’s exact test. Continuous variables are presented as mean±standard deviation or median with interquartile range and were compared by analysis of variance. From the pooled randomised trial database we constructed a logistic regression model for 30-day death using the following candidate variables: age (>75 versus <55, 55-75 versus <55 years), sex, diabetes mellitus, current smoker, prior PCI, prior coronary artery bypass grafting, prior MI, clinical presentation (ACS versus no ACS), stent type (bare metal stents versus first-generation DES versus second-generation DES), pre-PCI Thrombolysis In Myocardial Infarction (TIMI) flow (2-3 versus 0-1), lesion location in the left main (LM) or left anterior descending (LAD) artery versus others, number of treated lesions, and post-PCI TIMI flow (3 versus 0-2). The regression coefficient estimates for each significant variable (p<0.05) were transformed into integers, and the overall score was calculated for each patient. The score was characterised by its discrimination or accuracy (Harrell’s C statistic)9 and calibration (Hosmer-Lemeshow goodness-of-fit test –for which lower χ2 values and higher p-values signify better calibration)10. For validation, the regression model was then applied to the patients treated in the all-comers Assessment of Dual AntiPlatelet Therapy With Drug Eluting Stents (ADAPT-DES) registry11. A two-sided alpha of 0.05 was used for all statistical testing. Analyses were performed using SAS, version 9.4 (SAS Institute, Inc., Cary, NC, USA).

Results

The 21 studies enrolled 33,370 patients, of whom 24,532 (73.5%) were suitable for this analysis and served as the derivation cohort (Supplementary Figure 1). The most common reason for exclusion was lack of recorded TIMI flow data in the treated arteries. The validation cohort consisted of 8,547 out of 8,582 patients (99.6%) enrolled from ADAPT-DES in whom all model variables were present. A comparison of the baseline characteristics of the patients included in the derivation and validation models is shown in Supplementary Table 2. The validation cohort had substantially less favourable baseline and angiographic characteristics than the derivation cohort, except for a lower incidence of ACS and smoking.

By 30 days, there were 120 deaths (0.49%) in the pooled DES cohort and 19 deaths (0.22%) in the ADAPT-DES cohort. The baseline characteristics of the derivation and validation cohorts by vital status at 30 days are shown in Table 1 and Table 2. In the derivation cohort, patients who died by 30 days were older and more likely to have diabetes mellitus and a lesion in their left main or left anterior descending artery (p<0.01 for all). They were more likely to present with ACS, have more lesions treated with first-generation DES, and not achieve optimal angiographic results compared with survivors (p<0.01 for all). Similar findings were observed in the validation cohort.

The variables significantly associated with 30-day death and the derived score assigned to each of them are shown in Table 3. In the derivation cohort, the median [interquartile range] score was 5 [3, 6] and the mean±standard deviation score was 4.9±2.4. The minimum score was 0, and the maximum was 16. Discrimination (C statistic=0.848 [0.812, 0.885], p<0.0001) was very good, and calibration (χ2=6.53, p_HL=0.68) was good (Figure 1A). For the median score of 5, the sensitivity was 93% and specificity was 48%. In the validation cohort, the median score was 4 [3, 6], the mean score was 4.5±1.9, and the range was 0 to 12. Discrimination (C statistic=0.828 [0.724, 0.933], p<0.0001) was very good, and calibration (χ2=9.73, p_HL=0.31) was satisfactory (Figure 1B). There was a graded increase in mortality in both the derivation and validation cohorts with higher scores. In the derivation data set, 30-day mortality was substantially higher in patients with a score ≥5 (the median) compared with <5 (0.86% [111 deaths in 12,895 patients] versus 0.08% [9 deaths in 11,637 patients], respectively, p<0.0001), representing an approximately tenfold increase. Similarly, 30-day mortality was substantially higher in the validation data set in patients with a score ≥5 compared with <5 (0.29% [15 deaths in 4,179 patients] versus 0.08% [four deaths in 4,368 patients], respectively, p<0.01) (Figure 2).

Figure 1. Discrimination of the 30-day mortality score. A) Derivation cohort. B) Validation cohort. CI: confidence interval

Figure 2. Thirty-day mortality according to median score <5 versus ≥5 in the derivation and validation cohorts. The pooled DES cohort was used for score derivation and the ADAPT-DES cohort was used for score validation. DES: drug-eluting stents

Discussion

The present study describes a novel risk score for prediction of 30-day mortality after PCI, derived from a large cohort (~25,000 subjects) of individual patient data pooled from 21 randomised PCI trials rather than from state or institutional quality assurance registries as typically used in the past. The score, derived for 30-day rather than in-hospital mortality, incorporates eight readily available clinical and angiographic variables and, for the first time, includes post-procedural parameters. The score was then validated in a large-scale, multicentre, prospective study of PCI in an all-comers population. The main findings are: (i) clinical variables such as advanced age, diabetes mellitus, presentation with ACS, and prior MI significantly contributed to early mortality after PCI, whereas prior revascularisation did not; (ii) angiographic variables, such as lesion location and suboptimal flow after PCI, were also important predictors of early mortality; (iii) the discrimination in both the derivation and validation cohorts was very good, and calibration was satisfactory; (iv) among patients undergoing PCI with a score <5 (the median, ~47% of patients), fewer than one of 1,250 (0.08%) died within 30 days, providing a high level of prognostic reassurance, while a score ≥5 was associated with one death among every 135 patients treated, with a progressively greater risk as the score increased.

Wu et al2 derived a score for prediction of risk of in-hospital death after PCI from the New York State PCI registry database. In-hospital mortality was 0.70% (321 deaths in 46,090 patients treated in 2002). Their score incorporated nine clinical variables, ranged between 0 and 40 points, and assigned the highest risk to cardiogenic shock (9 points), acute stent thrombosis causing acute MI (9 points), age >75 years (5 points), current heart failure (4 points), and end-stage renal disease (4 points). Diabetes mellitus was not an independent predictor of in-hospital mortality in their model, and post-PCI elements were not included. The discrimination of this score was similar to ours (C statistic=0.886), although the calibration was lower (p_HL=0.12).

A single institution registry (MCRS) published a similar analysis in 20075. They used seven variables for derivation of the score: age (up to 4 points), cardiogenic shock (9 points), MI within 24 hours of PCI (4 points), systolic dysfunction (up to 3 points), renal dysfunction (up to 4 points), heart failure (2 points), and peripheral vascular disease (2 points). The maximum score was 29 points, and diabetes was not a predictor of in-hospital mortality. The C statistic was 0.89. The score was validated in a larger data set of >300,000 patients in the National Cardiovascular Data Registry (NCDR)12. The C statistic was again 0.89, and the model was recalibrated for better precision for very low and very high scores. Singh et al13 further verified the accuracy of the MCRS and New York State score for 30-day mortality in a subsequent cohort of nearly 5,000 patients treated between 2007 and 2010. The discriminatory ability of the scores was very high (0.88-0.92) and similar.

Brener et al14 applied the two scores –MCRS and NYS score– in a single institution PCI cohort of 3,165 cases performed between 2005 and 2007. The C statistic values for the two scores in this cohort (mostly low-risk patients) were 0.83 and 0.82, respectively.

Recently, investigators from Sheffield, UK, derived a 5-variable (cardiogenic shock, procedural urgency, history of renal disease, diabetes mellitus, and age) risk score for 30-day mortality after PCI from a cohort of 6,522 patients treated from 2007 to 201315. Validation was performed in a subsequent cohort internally and externally. The C statistic was 0.82, but the score utilised a complex equation for calculation rather than simple integer coefficients.

NCDR re-evaluated the in-hospital PCI mortality prediction risk score in a cohort of more than 1.2 million patients (derivation and validation) treated between 2009 and 2011 at 1,252 PCI centres16. The full model includes age, STEMI, cardiogenic shock, cardiac arrest, diabetes mellitus, peripheral arterial disease, cerebrovascular disease, body mass index, chronic kidney disease, heart failure class, ejection fraction, prior PCI, and chronic lung disease, and had a C statistic of 0.925. The simplified model included only age, STEMI, cardiogenic shock, chronic lung disease, heart failure class, chronic kidney disease, and cardiac arrest, and maintained excellent discrimination (C statistic=0.910).

In contrast to these analyses from registries without rigorous data validation and monitoring, our score was derived from a large cohort of patients randomised in clinical trials with independent event adjudication and centralised and fully verified data collection and monitoring. Unlike previous scores, the present analysis also included angiographic variables, incorporating the procedural outcome in the risk assessment for 30-day death. Of note, diabetes mellitus emerged as an independent predictor of 30-day mortality in our model (in contrast to most prior scores), consistent with our understanding of the importance of diabetes in coronary artery disease and PCI17.

The present score may provide clinical utility in routine practice. The discriminatory ability of a score represents its capacity to distinguish between patients at low versus high risk for an adverse event. The median score value of 5 separated patients into categories of risk nearly tenfold different. In contrast, the precision of the score measures its ability to predict the rate of events in a certain population. This ability is modulated by the population in which the score is tested, by the prevalence of patients with extreme scores and by advances in care between the derivation and validation cohorts. As such, some of the scores discussed in this paper had to be calibrated for a lower rate of observed events.

Limitations

Despite these strengths, we recognise the limitations inherent in the present study. Patients enrolled in RCTs rarely exhibit high-risk characteristics which may affect early mortality after PCI. The comparison of these patients with those enrolled in the all-comers ADAPT-DES registry highlights this fact (Supplementary Table 2).

We did not have data on left ventricular systolic function in nearly half of the patients and chose not to include it to maintain a derivation cohort as large as possible. Data on renal dysfunction –an important contributor to death in patients with coronary artery disease– were also not available. We note that the calibration of the score for the very high scores is suboptimal, but this is probably due to the low number of patients with such high scores. Equally important, there were no patients with cardiogenic shock enrolled in these trials for obvious reasons, eliminating our ability to analyse this important parameter (prominently featured in other scores) in our data set.

We believe our data and new score complement the existing literature and add simplicity to a complex field. The score does not guide physicians on whether PCI should be performed, but rather, by using parameters from the procedure itself, predicts which patients are at heightened risk and need further optimisation of medical care, closer follow-up or change of strategy with respect to repeat revascularisation.

Conclusions

A clinical score based on eight readily available clinical and angiographic preprocedural and post-procedural variables has been developed and validated and had excellent discrimination for 30-day mortality after PCI across a wide range of clinical syndrome acuity and coronary artery severity. Use of this relatively simple score may be of prognostic utility in patients undergoing PCI, providing reassurance for low-risk patients and identifying those at high risk in whom close monitoring and other interventions may be warranted.

Appendix. Study collaborators

Ori Ben-Yehuda, MD, NewYork-Presbyterian Columbia University Medical Center, New York, NY, USA and Clinical Trials Center, Cardiovascular Research Foundation, New York, NY, USA; Giora Weisz, MD, Montefiore Medical Center, Bronx, NY, USA and Clinical Trials Center, Cardiovascular Research Foundation, New York, NY, USA; Timothy D. Henry, MD, Minneapolis Heart Institute Foundation at Abbott Northwestern Hospital, Minneapolis, MN, USA and The Carl and Edyth Lindner Center for Research and Education at The Christ Hospital, Cincinnati, OH, USA; David A. Cox, MD, CVA Brookwood Baptist Hospital, Birmingham, AL, USA; Peter L. Duffy, MD, Reid Heart Center, FirstHealth of the Carolinas, Pinehurst, NC, USA; Thomas D. Stuckey, MD, LeBauer-Brodie Center for Cardiovascular Research and Education/Cone Health, Greensboro, NC, USA; Thomas McAndrew, PhD, Clinical Trials Center, Cardiovascular Research Foundation, New York, NY, USA.

Guest Editor

This paper was guest edited by Alec Vahanian, MD, PhD; Department of Cardiology, Hôpital Bichat-Claude Bernard, and University Paris VII, Paris, France.

Funding

This investigator-sponsored pooled study was funded by Abbott Vascular. The ADAPT-DES study was sponsored by the Cardiovascular Research Foundation and funded by research grants from Boston Scientific, Abbott Vascular, Medtronic, Cordis, Biosensors, The Medicines Company, Daiichi Sankyo, Eli Lilly, Volcano, and Accumetrics.

Conflict of interest statement

P.W. Serruys has received personal consultancy fees from Abbott Laboratories, Biosensors, Cardialysis, Medtronic, Micell, Sino Medical Sciences Technology, Philips/Volcano, Xeltis, and HeartFlow. P.C. Smits has received grants and personal fees from Abbott Vascular, grants and personal fees from Terumo during the conduct of the study, and grants and personal fees from St. Jude outside the submitted work. C. von Birgelen has received institutional research grants provided by Abbott Vascular, Biotronik, Boston Scientific and Medtronic to the research department of Thoraxcentrum Twente outside the submitted work. R. Mehran has received grants from AstraZeneca, Bayer, Beth Israel Deaconess, BMS, CSL Behring, DSI, Medtronic, Novartis Pharmaceuticals, OrbusNeich and Abbott Laboratories, personal fees from Abbott Laboratories, and other from Abbott Laboratories, Spectranetics/Philips/Volcano Corp., Janssen Pharmaceuticals, Boston Scientific, Medscape/WebMD, Siemens Medical Solutions, Abiomed, and The Medicines Company, personal fees and other from PLx Opco/PLx Pharma Inc, and from Regeneron Pharmaceuticals, and other from Roivant Sciences Inc, Medtelligence (Janssen), BMS, Watermark Research Partners, Claret Medical and Elixir Medical outside the submitted work. A.J. Kirtane has received grants from Medtronic, Abbott Vascular, Boston Scientific, Abiomed, CathWorks, Siemens, Philips, ReCor Medical and Spectranetics outside the submitted work. B. Witzenbichler has received grants from the Cardiovascular Research Foundation during the conduct of the study. M.J. Rinaldi has received other from Abbott, Boston Scientific, Cordis, and Edwards outside the submitted work. D.C. Metzger has received other from Abbott Vascular outside the submitted work. P.L. Duffy has received personal fees from Philips Volcano outside the submitted work. G.W. Stone reports that Columbia University, his employer, receives royalties from Abbott for the sale of the MitraClip. The other authors have no conflicts of interest to declare. The Guest Editor is a consultant for Edwards Lifesciences.