Validation of the updated logistic clinical SYNTAX score for all-cause mortality in the GLOBAL LEADERS trial

Introduction

The logistic clinical SYNTAX score (LCSS) was developed and internally validated in a pooled database from seven stent trials with more than 6,000 patients1. By combining age, creatinine clearance (CrCl), left ventricular ejection fraction (LVEF) and anatomic SYNTAX score in a core model, the LCSS predicted one-year all-cause mortality after percutaneous coronary intervention (PCI). An extended model was developed by including six variables in the core model. The LCSS has been revised and updated to predict all-cause mortality at one and two years2.

To date, no external validation in an all-comers population has been performed. Only two studies evaluating its performance are limited to patients with acute coronary syndrome (ACS) or left main lesions3^,4. Notably, with the improvement in PCI outcomes over time, it is uncertain whether the score is still up to date. Therefore, we aimed to validate the updated LCSS externally in the real-world population of a large contemporary PCI trial.

Methods

The GLOBAL LEADERS trial was a randomised, open-label trial comparing two strategies of antiplatelet therapy in all-comer patients undergoing PCI at 130 sites in 18 countries (NCT01813435). The design of the trial has been described previously5 and is summarised in Supplementary Appendix 1.

This present analysis was pre-specified in the protocol and aimed at predicting two-year all-cause mortality in the first 4,006 consecutive patients with an available anatomic SYNTAX score in the early phase of the trial to verify the correctness of the assumed two-year all-cause mortality - a major component of the primary composite endpoint. The anatomic SYNTAX score was analysed off-line by an independent core lab (Cardialysis, Rotterdam, the Netherlands) blinded to the treatment allocation6. Patients with prior coronary artery bypass grafting (CABG) were excluded from this analysis as they were not included in the developmental cohort of the LCSS. In cases of ST-segment elevation myocardial infarction (STEMI), the anatomic SYNTAX score was calculated using the angiogram performed prior to wiring, as described previously7. The GLOBAL LEADERS trial was approved by the institutional review board at each participating institution. All patients provided informed consent. The study complied with the Declaration of Helsinki and Good Clinical Practice.

UPDATED LOGISTIC CLINICAL SYNTAX SCORE

The updated LCSS and the predicted two-year all-cause mortality were calculated using the method described previously by our group2. The updated LCSS combined the prognostic value of the anatomic SYNTAX score with clinical characteristics and comorbidities including age, CrCl, LVEF, body mass index, diabetes, peripheral vascular disease (PVD) and SYNTAX-like characteristics to predict two-year all-cause mortality after PCI. Patients with missing variables required to calculate the score were excluded. Details and definitions of the variables used in this study are shown in Supplementary Table 1.

OBJECTIVE AND STUDY ENDPOINTS

The primary objective was to evaluate the predictive performance of the updated LCSS for two-year all-cause mortality. The primary endpoint was all-cause mortality. Definitions of other outcomes reported in this study are shown in Supplementary Appendix 2.

STATISTICAL ANALYSIS

Continuous variables are presented as mean±standard deviation (SD) or median and interquartile range (IQR) according to the distribution, while categorical variables are expressed as counts and percentages. Baseline characteristics of the patients in the present study were compared with the population in the developmental cohort of the updated LCSS. Independent t-tests or Mann-Whitney U tests were used to compare continuous variables and chi-square tests were used to compare categorical variables.

Baseline characteristics and outcomes are reported for each quintile of the patients categorised according to the updated LCSS. Testing for linear trend between quintiles was performed by the Cochran-Armitage test for categorical variables and the generalised linear model with updated LCSS class as a co-variable for continuous variables. The Kaplan-Meier method was used to estimate the cumulative rates of clinical events in the quintiles and the log-rank test was performed to examine the differences between quintiles.

The area under the receiver-operating characteristic curve, which equals the C-statistic when the outcome is binary, was calculated for all-cause mortality and other outcomes. In general, the discrimination is considered outstanding if the C-statistic is ≥0.9, excellent if the C-statistic is ≥0.8 and <0.9, acceptable if the C-statistic is ≥0.7 and <0.8, poor if the C-statistic is >0.5 and <0.7, and no discrimination if the C-statistic=0.58. A more recent approach refers to a C-statistic <0.60 as poor discrimination, 0.60 to 0.75 as possibly helpful discrimination, and more than 0.75 as clearly useful discrimination9.

Agreement between observed and expected (predicted) all-cause mortality was assessed by calibration plot. Quintiles of the patients were depicted in the calibration plot augmented by a locally weighted scatterplot smoothing over the logical range of predicted probability10. Calibration-in-the-large (model intercept) and calibration slope were evaluated by fitting the calculated linear predictor in all patients with all-cause mortality as the outcome in the logistic regression model. Intercept of 0 and slope of 1 indicate perfect prediction10. Negative and positive intercepts indicate overestimation and underestimation, respectively.

The Brier score was reported as an overall measure of performance. A Brier score of 0 reflects a perfect model whereas a score of 0.25 suggests a non-informative model10. The predictive performance was evaluated in the overall population, high or non-high-risk patients according to the anatomic SYNTAX score (>22 or ≤22), and in patients with clinical and angiographic characteristics meeting the inclusion criteria of the TWILIGHT study (NCT02270242) (Supplementary Table 2). Statistical analyses were performed in R, version 3.4.2 (R Foundation for Statistical Computing, Vienna, Austria).

Results

PATIENTS AND BASELINE CHARACTERISTICS

In the first 4,006 consecutive patients, six patients withdrew consent and requested data deletion from the database, 32 patients did not receive PCI, and 275 patients had previous CABG, leaving 3,693 coronary angiograms to be forwarded to the core lab for evaluation of the anatomic SYNTAX score. From these, the anatomic SYNTAX score could be calculated in 3,473 cases; in the remaining 220 patients, the non-target vessel was not documented. After excluding 202 patients with incomplete clinical information, the updated LCSS was available in 3,271 patients with a mean age of 64.3 years (SD 10.5), mean anatomic SYNTAX score of 12.0 (SD 8.3), median CrCl of 88.8 ml/min with an interquartile range of 70.2 to 112.2 ml/min, mean LVEF of 54.8% (SD 10.7), and mean body mass index of 28.1 kg/m² (SD 4.5). The prevalence of diabetes, established PVD and SYNTAX-like population was 22.9%, 6.1%, 22.2%, respectively. These variables were used for the updated LCSS calculation.

In comparison with the developmental cohort, age and frequency of female gender, diabetes, hypertension, and hypercholesterolaemia were similar, whereas the frequency of previous stroke, previous MI, established PVD and current smoking was lower in the present study (Table 1). CrCl was higher while the LVEF and anatomic SYNTAX score were lower in the present study. There were higher rates of patients with STEMI and unstable angina, and lower rates of patients with stable angina and NSTEMI in the developmental cohort than in the population in the present study. Baseline clinical characteristics according to updated LCSS quintiles are shown in Supplementary Table 3.

DISCRIMINATION AND CALIBRATION

The C-statistic of the updated LCSS for two-year all-cause mortality after PCI was 0.71 (95% confidence interval [CI]: 0.64-0.77). Discriminative ability is poor or non-existent for other outcomes (Supplementary Table 4).

The updated LCSS systematically overestimates two-year all-cause mortality as demonstrated by the negative intercept (Figure 1). The calibration slopes indicate a weaker association between predicted and observed two-year all-cause mortality in the validation cohort compared with the developmental population. In the first to fourth quintile, predicted probabilities of two-year all-cause mortality (ranging from 1 to 4%) are close to the line of identity for the agreement between predicted and observed two-year all-cause mortality. The Brier score of the updated LCSS for two-year all-cause mortality is 0.0247.

Figure 1. Calibration plot for the updated logistic clinical SYNTAX score for all-cause mortality at two years after PCI. Triangles represent five quintiles of patients with mean predicted probability and mean observed all-cause mortality with 95% confidence interval. The distribution of the predicted probabilities is displayed in the histogram.

The C-statistic of the updated LCSS for two-year all-cause mortality in the high-risk population according to anatomic SYNTAX score or TWILIGHT’s inclusion criteria is higher than the non-high-risk population, whereas the Brier score is lower (and thus better) in the non-high-risk than in the high-risk population (Figure 2).

Figure 2. C-statistic and Brier score of the updated logistic clinical SYNTAX score in high-risk patients versus non-high-risk patients according to the treatment strategy.

OUTCOMES ACCORDING TO QUINTILES AT TWO YEARS

All-cause mortality at two years was 2.66% (87 patients). The updated LCSS discriminates all-cause mortality, net adverse clinical events (NACE) and patient-oriented composite endpoints (POCE) in the very high quintiles at two years (Figure 3, Supplementary Figure 1). However, the cumulative curves of survival, NACE and POCE among the very low, low, intermediate and high score quintiles are almost superimposed and not discriminated. Table 2 shows other outcomes at two years in each quintile.

Figure 3. Kaplan-Meier curves for all-cause mortality at two years according to the updated logistic clinical SYNTAX score quintiles.

Discussion

The main findings are as follows. 1) The overall discriminative ability of the updated LCSS for two-year all-cause mortality is either borderline acceptable according to the general cut-off proposed by Hosmer et al8, or possibly helpful according to the recent approach by Alba et al9: by stratifying patients using quintiles, the score can identify patients at very high risk for two-year all-cause mortality after PCI. 2) Although the score systematically overestimated two-year all-cause mortality in the GLOBAL LEADERS population, calibration in the first four quintiles of the patients, who had a predicted two-year all-cause mortality ranging from 1 to 4% in four of the five quintiles, is appropriate and close to the identity line between the observed and predicted all-cause mortality.

Two issues may be responsible for the findings. First, the outcomes after PCI in the GLOBAL LEADERS trial were different from the developmental cohort of the LCSS.

The updated LCSS was developed from patient-level data from seven coronary stent trials (SIRTAX, ARTS-II, STRATEGY, MULTISTRATEGY, LEADERS, SYNTAX, RESOLUTE all-comers)2 which had a pooled two-year all-cause mortality of 4.6%, which is higher than the overall two-year all-cause mortality (2.99%) seen in the GLOBAL LEADERS trial5. Contemporary improvements in PCI techniques and post-procedural medications may have led to this reduced mortality. Optimal pharmacological treatment following PCI, such as using potent antiplatelet therapy (ticagrelor) and guideline-directed medical therapy (high-intensity statin and risk factor control), have been shown to reduce mortality11^,12. In the developmental cohort, clopidogrel was the main P2Y₁₂ inhibitor used in dual antiplatelet therapy after PCI, whereas in the GLOBAL LEADERS trial the majority of patients received ticagrelor either as monotherapy or as part of standard dual antiplatelet therapy. In the GLOBAL LEADERS trial, more than 90% of patients received statin on discharge and statin usage remained as high as 90% at two years, while in the SYNTAX trial 81% and 78% of patients received statin therapy at discharge and at one year, respectively13. The developmental cohort was recruited before 2008 when new-generation drug-eluting stents (DES), fractional flow reserve and image-guided PCI were not yet recommended in the guidelines or universally reimbursed. In the developmental cohort, the majority of populations were still treated with bare metal stents (BMS) or first-generation DES, and the same biolimus A9-eluting stent was only used in one of the seven trials. Second-generation DES have been shown to reduce adverse events after PCI when compared with BMS or first-generation DES14. However, it may be argued that compared with BMS they have not demonstrated a mortality benefit.

Second, there were differences in the clinical characteristics and comorbidities of the patients enrolled in the developmental cohort and the present study: the prevalence of poor prognostic features such as prior stroke, previous MI, and a clinical presentation of STEMI was higher in the developmental cohort. Although these features were not included in the updated LCSS, they have been associated with poor outcomes including mortality after PCI15^,16. The mean anatomic SYNTAX score in the present study was comparable to other all-comer PCI trials (Supplementary Table 5); however, it was lower than the mean anatomic SYNTAX score in the developmental cohort.

IMPACT OF THE UPDATED LOGISTIC CLINICAL SYNTAX SCORE IN CURRENT PRACTICE

Unbiased, evidence-based and reliable information is mandatory for the informed consent before PCI17. Precision medicine is growing and will reach the patient community. The updated LCSS is a quantified instrument which serves this purpose well by providing the objective risk of the individual patient undergoing PCI.

In clinical practice, the patients undergoing PCI vary from a young adult with a single lesion to an elderly patient with poor renal function, low LVEF, multiple comorbidities and complex lesions. The mortality risk in these patients can range from very low to extremely high. Patients at very high risk need more intensive pharmacological therapy and aggressive risk factor modification than patients at lower risk. The updated LCSS can objectively stratify the risk and consequently optimal care should be delivered to these patients.

A dedicated impact study to evaluate the effect of the LCSS in guiding tailor-made care in patients undergoing PCI could fill this evidence gap.

POTENTIAL TRIAL IMPACT OF A RISK MODEL IN A LARGE ONGOING TRIAL

In our study, the patients in the highest updated LCSS quintile had many high-risk clinical features and comorbidities such as advanced age, female gender, established PVD, diabetes, chronic kidney disease, left main and multivessel coronary artery disease which correspond roughly to the inclusion criteria of the TWILIGHT study. In the very high-risk quintile, the two-year all-cause mortality was 6.27% with a POCE and NACE rate of 20.18% and 23.56%, respectively. By selecting clinical variables and anatomic SYNTAX score in the highest risk quintile of the updated LCSS, those patients at highest risk can be identified to provide an enriched population where the benefits of novel therapies or strategies could be more easily demonstrated in a clinical trial. Of note, in the present study, we did not see any differences between the experimental and reference treatment arms in terms of all-cause mortality, POCE and NACE.

The updated LCSS derived from baseline characteristics obtained at enrolment from the first quarter of patients enrolled consecutively in the trial provided the initial estimated overall mortality of the GLOBAL LEADERS population. The sample size of the trial was derived from the composite rate of all-cause mortality and new Q-wave MI of 5% at two years in the LEADERS study5. The overall two-year all-cause mortality in the LEADERS study was 4.86%. Since the observed two-year all-cause mortality among the 3,271 patients enrolled in the present study was 2.66% (Figure 4), the GLOBAL LEADERS trial was at risk of being statistically underpowered. If the two-year mortality had been evaluated after enrolment of one fourth of the cohort using the updated LCSS, the anticipated two-year all-cause mortality would have been 3.50%, which would have been an incentive for recalculating the sample size or prolonging follow-up until the appropriate expected event rate was reached, thereby lessening the risk of an underpowered trial. In addition, other strategies such as extending the follow-up period until achievement of the desired event rate or even modifying the primary endpoints as in the ISCHEMIA study (NCT01471522) might have been considered. In the GLOBAL LEADERS trial, the steering committee was nevertheless reassured by the somewhat high incidence of new Q-wave MI that compensated for the lower than predicted all-cause mortality.

Figure 4. All-cause mortality at two years among the population in the LEADERS trial, developmental cohort of the updated logistic clinical SYNTAX score and the present study.

Limitations

Our analysis was performed in the first 3,271 consecutive patients out of 15,991 patients in the GLOBAL LEADERS trial. However, the consecutiveness of the enrolled patients with anatomic SYNTAX score calculation performed by an independent core lab blinded to the outcomes should have avoided serious selection bias in this population.

Differences in clinical characteristics between the first quarter of the study population and the last three quarters led to consideration of an adjustment. However, we restrained ourselves from performing this because, in a study with sequential cohorts, time is the main confounding factor that cannot be intrinsically and properly adjusted.

The effect of the antiplatelet strategies was not taken into account given that in the present cohort there was no statistical difference in all-cause mortality at two years between patients in the experimental and reference treatment arms (experimental vs reference; 2.3% vs 3.0%, p-value 0.23).

Conclusions

The overall discriminative ability of the updated LCSS for two-year all-cause mortality is either borderline acceptable or possibly helpful. The score can identify patients at very high risk of two-year all-cause mortality after PCI. Calibration in the majority of patients is appropriate. The score is potentially useful in selecting enriched high-risk populations.

Guest Editor

This paper was guest edited by Alec Vahanian, MD, PhD; Department of Cardiology, Hôpital Bichat-Claude Bernard, and University Paris VII, Paris, France.

Conflict of interest statement

P. Chichareon reports receiving a grant from Biosensors, outside the submitted work. Y. Onuma reports being a member of the advisory board of Abbott Vascular. R. Modolo has received research grants from Biosensors outside the submitted work and from the Sao Paulo Research Foundation (FAPESP – grant number 2017/22013-8). R.J. van Geuns reports receiving grants and personal fees from Abbott Vascular, and grants from Boston Scientific, outside the submitted work. J.J. Piek reports non-financial support from membership of the medical advisory board of Abbott Vascular, personal fees and non-financial support for being a consultant for Philips/Volcano, outside the submitted work. E. Spitzer reports institutional grants from the European Cardiovascular Research Institute, during the conduct of the study. P. Jüni serves as an unpaid member of the steering group of trials funded by AstraZeneca, Biotronik, Biosensors, St. Jude Medical and The Medicines Company. C.W. Hamm serves on the advisory board of and reports speaker’s fees from Medtronic. G. Steg reports grants and personal fees from Servier, during the conduct of the study, grants and personal fees from Bayer/Janssen, Merck, Sanofi, and Amarin, and personal fees from Amgen, Bristol-Myers Squibb, Boehringer-Ingelheim, Pfizer, Novartis, Regeneron, Lilly, and AstraZeneca, outside the submitted work. P. Vranckx has received personal fees from AstraZeneca and The Medicines Company during the conduct of the study and personal fees from Bayer Health Care, Terumo, and Daiichi Sankyo outside the submitted work. M. Valgimigli has received personal fees from Abbott, AstraZeneca, Chiesi, Bayer, Daiichi Sankyo, Terumo, Alvi Medical, and Amgen, and grants from the Swiss National Foundation, Terumo, Medicure, Abbott, and AstraZeneca, outside the submitted work. S. Windecker’s institution has research contracts with Abbott, Amgen, Bayer, Biotronik, Boston Scientific, Edwards Lifesciences, Medtronic, St. Jude Medical, Symetis SA, and Terumo outside the submitted work. P.W. Serruys reports personal fees from Abbott Laboratories, Biosensors, Cardialysis, Medtronic, Micell Technologies, Sinomedical Sciences Technology, Stentys, Svelte Medical Systems, Philips/Volcano, Xeltis, StentIt, and HeartFlow outside the submitted work. The other authors have no conflicts of interest to declare. The Guest Editor is a consultant for Edwards Lifesciences.