Abstract
Aims: Primary percutaneous coronary intervention (PCI) is a widely practised therapeutic procedure to treat ST-elevation myocardial infarction (STEMI). However, a significant proportion of patients undergoing primary PCI suffers from adverse events, such as incomplete myocardial reperfusion. It is currently unknown to which degree these adverse events are operator related.
Methods and results: We investigated inter-operator variation using objective safety and efficacy endpoints during primary PCI for STEMI. All PCIs were performed by six experienced interventional cardiologists as part of a randomised single centre trial. The primary endpoint of this study was optimal myocardial reperfusion (myocardial blush grade 3 [MBG]). All 1,071 patients enrolled in the Thrombus Aspiration during Percutaneous coronary intervention in Acute myocardial infarction Study (TAPAS) were included in this analysis. In the six operator groups, the rate of the primary endpoint MBG 3 ranged between 29.2% and 55.5%. The variable for operators remained significantly associated with MBG 3 after adjustment for baseline and procedural differences. There were no statistical differences observed with regard to safety endpoints.
Conclusions: This study illustrates the observation that even in a controlled setting significant inter-operator variation may exist in the efficacy of primary PCI. This study supports the routine collection of high-quality datasets to evaluate and improve individual operator competence and skills.
Introduction
Percutaneous coronary intervention (PCI) is a widely practised therapeutic procedure to treat coronary heart disease. In Europe, more than 1,100,000 PCIs are performed yearly, while in the United States that number is more than 1,300,000.1,2 Due to rapid improvement and innovation of catheter techniques, devices and co-medication, PCI is nowadays considered safe and suitable as reperfusion therapy for patients with acute ST-elevation myocardial infarction (STEMI).1-5 This is mainly based on the low incidence of major adverse cardiac and cerebral events (death, reinfarction, revascularisation and stroke) during and after primary PCI.5,6 However, a significant proportion of patients treated with primary PCI for STEMI suffer from more hidden, but nevertheless serious adverse events, such as bleeding requiring blood transfusion,7-9 distal embolisation,10-14 and incomplete myocardial reperfusion.10,11,15 Although, patient related factors play a major role in the occurrence of these complications,7-15 it is currently unknown to which degree these complications are also operator related.
Previous published studies and medical audits have investigated operator dependence of outcome after primary PCI with rates of in-hospital mortality and/or other major adverse events as endpoints, and found almost no differences across operators.6,16-21 As the incidences of these events are low, the analyses had limited statistical power to conclude whether there is significant operator dependence of outcome in the context of primary PCI. In addition, interpreting individual outcome data requires stratification of outcomes based on operator versus other factors, such as system of care, hospital quality, temporal factors, patient related factors, procedural characteristics and adjunctive therapies. Because of incomplete data sets, studies were often not able to fully adjust their data for possible confounders.6,16,17,22 Finally, data collection and analyses were often not done by independent investigators, which may have induced bias in the reported results.6,16-21
We therefore thought to investigate inter-operator variation using objective safety and efficacy endpoints during primary PCI for STEMI. All PCI’s were performed by six experienced interventional cardiologists as part of a large randomised controlled single centre trial.23,24 All endpoints were analysed by a core laboratory or independent investigators.25 The primary hypothesis was that even in the controlled setting of a randomised controlled trial with experienced interventional cardiologists, significant inter-operator performance variation exists in terms of efficacy and safety.
Methods
The University Medical Center Groningen (UMCG) is a large tertiary referral centre in the northern part of The Netherlands. The UMCG has a high volume PCI centre with a 24-hour on-call duty schedule. Annually, around 2,000 PCIs (1,250 elective and 750 emergency PCIs) are performed by six interventional cardiologists: R.L. Anthonio, MD, PhD; A.F.M. van den Heuvel, MD, PhD; G.A. Jessurun, MD, PhD; B.J. de Smet, MD, PhD; E.S. Tan MD, PhD and F. Zijlstra MD, PhD. All operators had a total individual case load of >1,500 PCIs before the start of the study period. During the study period they all performed an annual number of >250 PCIs.
Patients
We included all 1,071 patients who were enrolled in the Thrombus Aspiration during Percutaneous coronary intervention in Acute myocardial infarction Study (TAPAS). The study design, methods and results of the TAPAS-trial have been reported.21-23 The TAPAS-trial was a single-centre, prospective randomised open trial with blinded evaluation of endpoints, which investigated whether thrombus aspiration was superior to conventional treatment during PCI in patients with STEMI who were candidates for primary PCI. The primary endpoint of the trial was post-procedural myocardial blush grade (MBG).
Figure 1. Trial flow diagram.
All patients admitted to the UMCG during the study period were assessed for eligibility. The inclusion criteria were, symptoms suggesting acute myocardial ischaemia >30 minutes, time from symptom onset <12 hours, and ST-segment elevation >0.1 mV in two or more leads on the ECG. Exclusion criteria were rescue PCI after thrombolysis, known existence of a concomitant disease with life expectancy <6 months, and no informed consent.
Randomisation
Before initial coronary angiography, patients were randomly assigned in a 1:1 ratio to either thrombus aspiration or conventional treatment. Randomisation was done using a computerised voice-response system. The randomisation code was developed by means of a number generator used to select randomly permuted blocks of three to six patients. Randomisation was stratified by interventional cardiologists to achieve a balanced group assignment with regard to both treatment group and operator performing the PCI. All included patients were treated by one of the six interventional cardiologists. The name of the interventional cardiologist performing the procedure was prospectively registered in the main database.
Treatment
In patients randomised to thrombus aspiration, the Export Aspiration Catheter (Medtronic Corporation, Minneapolis, MN, USA) was used to establish antegrade flow before stenting. When necessary for stent delivery, balloon dilatation was performed before stenting. In conventional patients, balloon angioplasty was followed by stent placement. However, in both groups, the use of balloon dilatation as well as the use of (additional) stents was finally left over to the operators decision.
All patients were pre-treated with aspirin (a loading dose of 500 mg), heparin (5,000 IU) and clopidogrel (loading dose of 600 mg), all administered directly after electrocardiographic confirmation of STEMI. Unless contraindicated, patients received weight-adjusted glycoprotein IIb/IIIa-inhibitor (GPI) (abciximab) during the procedure and additional ACT-guided heparin. Arterial sheaths were removed with manual compression after PCI, no arterial puncture closing devices were used during the study period. Therapy after PCI was performed according to current guidelines.4
Quality evaluation and improvement programme
Our programme for quality evaluation and improvement was according to the guidelines of the American College of Cardiology/American Heart Association/Society for Cardiovascular Angiography and Interventions (ACC/AHA/SCAI).18 Quality of care was assessed in weekly meetings in which interventional cardiologists and research fellows reviewed and evaluated all performed primary PCI’s during that week (including all patients included in the TAPAS-trial). Two times a year, all complications occurring during PCI were discussed and evaluated. In addition, clinical follow-up of all patients undergoing PCI (STEMI and non-STEMI) in the UMCG were collected on a per operator basis and evaluated.
Figure 2. Control chart for odds ratio per operator for optimal myocardial perfusion after adjustment for baseline and procedural differences. Dotted lines represent 99.8% confidence intervals.
Endpoints
The primary endpoint was optimal myocardial reperfusion assessed by MBG. Secondary endpoints were effective thrombus aspiration, TIMI 3 flow post PCI, delta haemoglobin, the use of blood transfusions and intraprocedural complications. All endpoints were analysed by the core laboratory or independent blinded investigators.
MBG is a well established surrogate endpoint for myocardial reperfusion and is strongly related with death and other major adverse events after primary PCI.15,23,24 Retrieval of thrombotic material after thrombus aspiration is an objective marker for efficacy of thrombus aspiration. Delta haemoglobin and the use of blood transfusions reflect bleeding complications during PCI. Bleeding complications are associated with longer hospital stays, higher health costs and may be also related with adverse outcome.7-9
Endpoint assessment
MBG and TIMI flow grade were assessed as previously described.15,26 All coronary angiographic data were analysed at an independent core laboratory (Cordinamo, Wezep, The Netherlands). Effective thrombus aspiration was determined by histopathological examination, performed by pathologists unaware of PCI-operator and clinical data.25 Aspirated material during thrombus aspiration was placed in formalin, and fixed for 24 hours. Histological sections were cut and stained with hematoxylin-eosin for examination with a light microscope (x100). Samples were classified into effective or no effective aspiration based on the presence of atherothrombotic material. Delta haemoglobin was calculated using the haemoglobin level at admission and one day after admission in patients who did not receive blood transfusions or underwent revascularisation (rePCI or CABG) within this period. The use of blood transfusions was defined as any gift of packed red blood cells given at the day of admission or one day after. Bleeding was subdivided in major and minor bleeding according to the TIMI bleeding score.26 Major bleeding was defined as a reduction of haemoglobin of 5 g/dl or more from study entry to the time of the lowest haemoglobin level within 10 days. Minor bleeding was calculated similar and was defined as a reduction of haemoglobin of 3 to 5 g/dl. Intraprocedural complications included intraprocedural deaths, strokes, CABG within 24 hours after diagnostic angiography, flow limiting dissections and side branch occlusions. We re-analysed all coronary angiograms and hospital records of patients who underwent CABG within seven days after diagnostic angiography, to investigate whether they were possibly related or caused by complications occurred during the index procedure.
Statistical analysis
Data are presented as frequency / number of patients with available data on the specific variable (percentage), mean ±SD or as median (interquartile range [IQR]). Comparisons across the six operator groups were performed by the Pearson chi-square test for categorical variables, one-way ANOVA for normal distributed continuous variables and Kruskal-Wallis test for not-normal distributed continuous variables. Multivariate regression analyses were used to correct for possible confounders. The following baseline variables were introduced in the multivariate analysis: age, gender, total ischaemic time, diabetes, three vessel vessel disease, TIMI flow, previous myocardial infarction, systolic blood pressure and heart rate at admission, anterior myocardial infarction and off hour arrival at cathlab (to correct for differences in temporal factors, system of care before arrival and level of catheterisation lab staff occupation). In addition, the following procedural variables were used to correct for possible procedural differences: number of stents implanted, balloon used (both pre- and post-dilatation), effective thrombus aspiration, GPI used and TIMI flow 3 post-procedure. In the regression analyses, the categorical variable operator was coded using the simple contrast method; each operator (except the reference operator) was compared to the reference operator. The operator with the best performance for the primary endpoint was chosen as the reference operator. Predicted probabilities per patient after multivariate analysis were used to draw a control chart for the primary endpoint. All p-values were 2-tailed. Statistical significance was set at <0.002 (≈3SD), because several studies have demonstrated that in the setting of comparing performance, a threshold of ≈3SD limits is more adequate to find special-cause variation in practice.27,28 Analyses were performed using SPSS software version 16.0.1 (SPSS, Chicago, Il, USA).
Results
Only a minority (7.8%) of the patients assessed for eligibility, were excluded. The six operators each performed between 150 and 233 primary PCIs of the study. There were no differences in baseline clinical and angiographic characteristics across operator groups, except for the variable off hour arrival at catheterisation lab (Table 1).
Procedural variation
Procedural differences were present across operator groups with regard to the frequency of periprocedural GPI administration, use of balloon dilatation and stent implantation (Table 2).
Efficacy endpoints
Myocardial blush grade
MBG 3 was present in 381/980 (38.9%) of the patients in which post procedural MBG could be assessed. In the six operator groups the rate of MBG 3 ranged between 29.2% and 55.5% (Table 3).
After adjustment for baseline and procedural characteristics in multivariate analysis, the variable operator remained a strong predictor of the primary endpoint MBG (Table 4).
EFFECTIVE THROMBUS ASPIRATION AND TIMI FLOW 3 POST-PROCEDURE
Thrombus aspiration was effective in 331/454 (72.9%) patients. There was a significant difference in rate of effective thrombus aspiration between operator groups, ranging from 55.4% to 85.7%. No difference was found regarding post-PCI TIMI flow 3 (Table 3).
Safety endpoints
INTRAPROCEDURAL COMPLICATIONS
The incidence of intraprocedural complications was low (see Table 3). No deaths, strokes or flow-limiting dissections occurred during the primary PCI procedures. A total of 29 patients underwent a CABG within seven days after diagnostic angiography. Thirteen (44.8%) of these CABGs were performed with 24 hours. None of the CABGs were caused by or related with intraprocedural complications. Side branch occlusions were visible in nine patients after stent implantation. No significant differences were present across operator groups with regard to these complications.
BLEEDING COMPLICATIONS
Data on delta haemoglobin was available in 971 patients. The median decrease in haemoglobin was 0.6 g/dL (IQR 0.2 to 1.3) (Table 3). Blood transfusions were given to 55/1047 patients (5.3%). No inter-operator variation was present regarding bleeding complications.
Discussion
This analysis illustrates that even during primary PCI performed by experienced operators, and despite strict adherence to current guidelines in the setting of a randomised controlled trial, significant variation may exist with respect to the rate of optimal myocardial reperfusion. These differences remained significant after multivariate adjustment. There were no statistical differences observed with regard to safety endpoints.
It is currently a matter of discussion whether individual outcome data should be used as method to evaluate and improve the quality of heathcare. Important for this discussion is the availability of appropriate data to investigate the presence and impact of inter-operator variation on procedural outcome. Our study setting allowed us to perform an accurate analysis on the impact of the operator on safety and efficacy. We investigated inter-operator variation in the setting of a randomised controlled single centre trial, therefore process of care, hospital quality and (co)therapy was comparable in all included patients. In addition, as a result of the acute nature of primary PCI, operator specific outcome data was not influenced by operator dependent patient selection due to selective withholding or referral of procedures in patients at higher risk, this in contrast with previous analyses.6,29,30
Previous studies investigated individual operator performance in terms of differences in mortality or other major adverse events.6,16-21 For example, the New York State Department of Health reports risk adjusted mortality data for individual operators since 1994.6 Given the low in-hospital/30-day mortality rates (0.63% in non-emergency and 3.27% for emergency PCIs performed during 2003-2005) these analyses had often limited statistical power to detect inter-operator differences. Further, as previously mentioned, data collection and analyses were often not done by independent investigators (in this case, the operator him or herself), which will have induced bias in the reported results.6,16-21
Analyses on clinical endpoints are certainly mandatory, however they should be accompanied with objective and sensitive surrogate endpoints. MBG and the secondary endpoints used in our study are important markers for efficacy and safety during primary PCI, and were analysed by either a core lab or blinded investigators. These endpoints are simple and not time consuming to collect and analyse, and therefore useful for evaluation of every day practice. In addition, the recent introduction of software for computer-assisted assessment of MBG and ST-segment resolution, further facilitates the routine use of these surrogate endpoints for assessing efficacy of reperfusion therapies.31,32
So what are the possible explanations of the observed inter-operator variation in efficacy? We adjusted for possible differences in patient case mix, temporal factors and process of care (e.g., total ischaemic time; on- versus off-hours).22 The residual variation suggests difference in operator performance. Well investigated is the idea that the experience of an interventional cardiologist (operator volume) is inversely proportional to the incidence of procedure-related complications during PCI.18-21 However, it is not likely that “experience” played a major role in our study, as all operators were experienced and performed >250 PCIs/year. Further, there can be differences in opinions about the optimal approach during primary PCI. This may have caused inter-operator variation with regard to the use of co-medication, balloon dilatation and coronary stents. Several studies have demonstrated that the extent and the complexity of the procedure (higher number, duration, and pressure of balloon inflations and stent implantations) is associated with the occurrence of distal embolisation and impaired myocardial reperfusion.10-14 The additional analyses performed in our study suggest that these procedural differences may partly explain the under/above performance of some operators. However, also after multivariate correction for procedural characteristics (including effective thrombus aspiration) there remained significant inter-operator variation with regard to impaired myocardial reperfusion. This suggest that differences in factors for which we were not able to account for, including most likely differences in technical skills, have contributed to the observed inter-operator variations.
The ACCF/AHA/SCAI Clinical Competence Statement 2007 proposes interventionalists to participate in local, regional and/or national PCI registries to monitor and improve both hospital and individual operator performance.18 Our results strongly support this recommendation and demonstrate that the systematic collection, reviewing and reporting of data provides important feedback with regard to safety and efficacy of health care, and is mandatory even among experienced operators. Further, essential requisites for the investigation of inter-operator variation is the careful selection of endpoints and high quality data sets, collected and analysed by independent investigators.
Limitations
Our study suffers from several limitations: With regard to the operator, we did not adjust for differences in physician characteristics (age, years of experience and total caseload). In our opinion this is not necessary, as all operators had a sufficient total case load and number of PCIs performed per year to label them as experienced and assume that learning curves have flattened out. Secondly, we adjusted for important baseline and procedural characteristics. However, we can not rule out that our results were influenced by other confounders (such as quality of care before arrival at our hospital, experience of catheterisation staff, etc.). Thirdly, although all operators had trained with thrombus aspiration before the start of the study period – and we corrected for effective thrombus aspiration in additional analyses – the use of new devices such as thrombus aspiration may have augmented the observed inter-operator variation in this study. Finally, inter-operator variation was investigated only in terms of functional endpoints. However, as the magnitude of the inter-operator variation in our study is comparable with the effect size of thrombus aspiration on myocardial reperfusion in TAPAS (MBG 3 rate was 32.2% in conventional PCI versus 45.7% in the aspiration group),23 the impact of operator variation during primary PCI is certainly of clinical relevance.
Conclusion
Our study illustrates that even in a very controlled setting significant inter-operator variation may exist in the efficacy of primary PCI for STEMI. This study supports the routine collection, reporting and reviewing of high-quality datasets to evaluate and improve individual operator performance.
Acknowledgements
We would like to thank T. Svilaas, Dr. I.C. van der Horst, Dr. M.W. Nijsten, Dr. G.F. Diercks and Dr. A.J. Suurmeijer for their valuable contributions to this study.
Current Controlled Trials number, ISRCTN16716833
The TAPAS-trial was supported by an unrestricted grant from Medtronic (for the costs of the angiographic analyses by the core laboratory). All other costs were covered by the Thorax Center of the UMCG. Medtronic had no role in the study design, collecting, analysis, interpreting of data, writing of the report and the decision to submit the paper for publication.
