Abstract
Aims: Current quality measures of percutaneous coronary intervention (PCI) procedures are based on the incidence of major adverse cardiac events (MACE). This crude marker ignores the many clinical nuances that make for sound decision making in PCI. We have established a prospective peer review audit tool to determine the quality of PCI within our cardiac network, which consists of five PCI hospitals serving a population of 1.4 million people in Sussex, UK.
Methods and results: Analysis of 10% of all PCI cases selected at random each month by a non-clinical audit manager is made by a rotating panel of two PCI operators and one cardiac surgeon. Each PCI case is assessed for anatomical suitability, lesion severity, strategic appropriateness and final outcome. Panel findings were reported back to the operator and the audit manager. A total of 326 cases were assessed by the review committee. Results were disseminated to individual operators. Coronary anatomy and lesion severity were considered appropriate for PCI in 94.2% and 96.0% of cases, respectively. Appropriateness of strategy was confirmed in 86.2% and the outcome considered satisfactory in 90.8%. A total of 242 subsequent cases were analysed to assess practice trends. This analysis demonstrated a statistically significant improvement in clinical decision making with respect to appropriateness of strategy (from 86.2% to 92.6%; p=0.004).
Conclusions: Prospective peer review of percutaneous coronary intervention cases by a rotating regional committee is valuable in ensuring procedural quality.
Introduction
The major adverse cardiac event (MACE) rate has been a longstanding measure of quality in percutaneous coronary intervention (PCI)1-3. However, the introduction of stent platforms and effective antiplatelet regimes has blunted this tool, and MACE rates for stable coronary artery intervention have fallen as low as 1% . The use of MACE as a quality discriminator is therefore of questionable value.
As PCI has become safer with low levels of major in-hospital complications such as death, Q-wave myocardial infarction (MI) or emergency coronary artery bypass graft (CABG), there has been a need for newer, more discerning measures of quality. Against this background we established a prospective ongoing peer-review programme to assess appropriateness of case selection, procedure strategy and outcome.
Revascularisation strategies have been the subject of a vast number of clinical trials, and there is a strong evidence base supporting decision-making processes with respect to clinical management in many patients. For example, the initial question for patients with ischaemic heart disease is whether PCI should be done at all. There is little data to support PCI in asymptomatic patients yet it is a practice that is relatively widespread, perhaps more so in healthcare programmes that reimburse physicians for procedures4. Similarly, it could be asked whether PCI is being performed in the wrong patients. A diabetic patient with left main stem disease could well be better served with a coronary artery bypass operation5,6. In such instances the question arises as to whether the case has been discussed at a multidisciplinary meeting with a cardiothoracic surgeon in order to provide a balanced view of the risk-benefit ratio of different strategies.
Once a decision to perform a PCI is made, the appropriateness of the strategy employed requires scrutiny. In many instances, failure or success depends on accurate anatomical and, if necessary, physiological assessment, proper use of equipment, including adequate guide catheter support, guidewire choice, as well as lesion preparation, balloon and stent use, and an appreciation of a requirement for other technology e.g., distal protection devices, thrombectomy catheters and adjunctive pharmacological therapies7-11. These decisions are based on clinical judgement and experience as well as evidence-based medicine and there may be a number of equally appropriate strategies.
Regardless of the approach undertaken, it is necessary to assess whether the outcome was satisfactory. In patients with complex disease, it may be clear from the outset that a perfect result will not be achieved with PCI, but intervention may still have been appropriate for symptomatic relief of angina. In scenarios such as these, as well as those unexpected cases in which a satisfactory result cannot be achieved, it is valuable to identify whether there is anything to learn.
Methods
Our cardiac network serves a population of 1.4 million and consists of five PCI centres, one of which is within a cardiothoracic surgical facility. The surgical centre’s PCI centres have four operators and the non-surgical centres have 11 operators. The audit included operators from the various centres who had sessions at the surgical facility (operator volume range was 90-382 for the year 2009).
Every month 10% of all PCI cases performed within the network are reviewed by a committee, consisting of two interventionists (one from within the surgical centre and the other from one of the four non-surgical PCI centres) and a cardiothoracic surgeon, who were blinded to the operator details.
Cases are chosen at random by a non-clinical audit officer. This is achieved by allocating all cases with a consecutive number and then using a random number generator programme to identify cases for review (12-15 cases in an average month).
There were a few caveats to the randomisation process:
– It was agreed that each interventionist should have at least one case reviewed per month, so that if the randomisation did not generate a case, the process continued until one did. There was no clinician involvement in this.
– Cases performed by the committee members for that month would not be included.
It was agreed that each randomised case review would be assessed with respect to:
– Was the coronary anatomy suitable for intervention?
– Was the coronary lesion sufficiently severe to warrant treatment?
– Was adjunctive technology used if there was some doubt, e.g., pressure wire/ intravascular ultrasound (IVUS)?
– Was the PCI strategy chosen appropriate?
– Was the outcome satisfactory?
Each of these factors was assessed by the committee, with reference to the clinical correspondence and notes. The completed feedback form for each case was given to the individual operator, and copies of all forms were kept by the non-clinical audit officer. This allowed communication between committee members and the operator, and the decisions of the committee could be altered depending on the detail of that communication.
Following review of 326 cases, the results were examined for the network as a whole, as well as on an individual basis. This enabled comparison of an individual operator’s interventional practice with others within the network. To ensure anonymity, the results were analysed independently of the clinicians, and presented as percentages rather than actual numbers (higher volume operators would be easily recognised). The results were assessed with respect to:
– Anatomical suitability for PCI –reasons for unsuitability varied according to the panel discussion, and related either to lesion severity, multiplicity, calcification, tortuosity, and with reference to the clinical notes.
– Lesion severity was visually assessed.
– Appropriate interventional strategy –inappropriate strategy was assessed by the reviewing panel and related to the techniques used according to current best practice. For example, direct stenting of a heavily calcified vessel or failure to postdilate a significantly constricted stent would have been deemed inappropriate strategies according to panel consensus.
– Satisfactory outcome was defined as TIMI 3 flow in the relevant vessel with <30% residual stenosis.
Results were presented to the operators in an anonymised form. They were informed which graph reflected their practice, which they could compare to other interventional operators. After a further 242 cases, the results were re-analysed and compared to the initial findings to identify any changes in practice.
Statistical analysis
Categorical variables are presented as absolute and relative frequencies (%). The comparison between the two groups with respect to anatomical suitability, lesion severity, strategic appropriateness and outcome was assessed using a logistic regression model, which allows for differences between the operators and between the two phases. Odds ratios (OR) are presented with 95% confidence intervals (CI) and p<0.05 was considered significant.
Results
After 326 cases had been reviewed the initial analysis was performed. At the time of inception of the process, ten operators were assessed. The mean number of cases reviewed per operator was 33 (range 20-51) (Figure 1).
Figure 1. The initial analysis of the first 326 patients. Each operator is highlighted and results presented as a percentage of the total cases reviewed. Operator 5 and 8 were most likely to have their cases challenged by the review committee with respect to anatomical suitability, lesion severity, appropriateness of strategy and satisfactory outcome. Conversely, operator 10 was least likely to be challenged by the review committee.
SUITABILITY OF CORONARY ANATOMY FOR PCI
Of the 326 cases, 307 (94.2%) were assessed as having suitable anatomy for intervention, 13 (4.0%) cases were felt to be borderline and six (1.8%) cases were deemed unsuitable.
LESION SEVERITY
Three hundred and thirteen (96.0%) were felt to be angiographically severe enough to warrant intervention and 13(4.0%) were felt to be borderline. No lesions were felt to be of insufficient severity to require PCI.
STRATEGY APPROPRIATENESS
The strategy was felt to be appropriate in 281 (86.2%) cases, questionable in 31(9.5%) and inappropriate in 14 (4.3%).
OUTCOME
The outcome was satisfactory in 296 (90.8%) cases, borderline in 16 (4.9%) and unsatisfactory in 14 (4.3%).
Figure 2 shows the subsequent analysis after a further 242 cases. The mean number of cases per operator reviewed at this time was 24.2 (range 16-42). Of these, 229 (94.6%) were felt to be anatomically suitable whilst 13 (5.7%) were not (six borderline; seven unsuitable). Two hundred and thirty-seven (97.9%) of the lesions were felt to be angiographically severe enough to warrant intervention, three (1.2%) were considered borderline and two (0.8%) were not felt severe enough to necessitate revascularisation. Of the interventional strategies used, 224 (92.6%) were felt to be appropriate, 11 (4.5%) were questionable and seven (2.9%) were inappropriate. With respect to outcome, this was satisfactory in 212 (87.6%) cases, borderline in 14 (5.8%) and unsatisfactory in 16 (6.6%).
Figure 2. After the initial analysis, each operator was made aware of his/her individual results against their peers. After a further 242 cases, the analysis was repeated. Operators 5 and 8 were less likely to be challenged by the review committee when compared to the initial results. There are insufficient numbers for any statistical comparison to be made between operators.
Figure 3 demonstrates the difference in outcome markers for the group as a whole between the two datasets. The second set of cases reviewed showed little difference as regards suitability for PCI (94.2% to 94.6%; p=0.687; OR=1.17 [0.55-2.50]), and it was more likely that the committee felt the interventional strategy was appropriate in the second round of analyses (86.19% to 92.6%; p=0.004; OR=2.38 [1.31-4.33]). There was no major difference seen as to whether the lesion was felt to be sufficiently severe (p=0.138; OR=2.25 [0.77-6.55]), or whether the outcome was satisfactory (p=0.260; OR=0.73 [0.43-1.26]).
Figure 3. The combined difference in outcome markers for the network PCI operators as a whole. The second set of case reviews led to an improvement in anatomical suitability with an appropriate strategy. There was no difference detected in lesion severity or outcome score.
Discussion
Evaluation of the quality of PCI needs to extend beyond recording individual and institutional volumes, or the incidence of major adverse clinical events in contemporary interventional practice. Successful percutaneous intervention requires not only technical skill but also appropriate clinical judgement with respect to the risk–benefit ratio of PCI versus surgical intervention or medical management. Since we started this process, the European Society of Cardiology have published guidelines about appropriate myocardial revascularisation strategies (REF), and it may be if units were to start an audit process now, these could be incorporated into an analysis12. Our process analysed decisions based on numerous factors including anatomy, operator experience and comorbidity.
The audit tool we have developed ensures an equitable review system. As the committee changes each month, and all interventionists are involved in the mechanism, there is inclusive ownership of the process and all operators have a chance to review each other’s work, and learn from it. The recording and presentation of the results by an audit officer rather than a clinician also provides a fair environment for dispassionate discussion of cases.
Presently the majority of coronary angiograms are undertaken away from a surgical centre; consequently revascularisation decisions are frequently made by an individual operator and not always within the umbrella of a multidisciplinary team. At the outset, when establishing this model, it was felt that the presence of a cardiac surgeon on the review committee was important. The National Confidential Enquiry into Patient Outcomes and Death (NCEPOD) report, the Heart of the Matter (2008), has encouraged multidisciplinary discussion of cases, particularly if there is multivessel disease. The involvement of a surgeon in this process has also provided the added advantage of improving understanding of what is possible percutaneously, with an objective view of outcomes. This appears to have led to increased cross-referral, whereby cardiac surgeons increasingly refer high-risk patients, referred initially for coronary artery bypass grafting, to interventionists. This reflects a consolidation of relationships, which is central to a cohesive revascularisation service.
It was encouraging to note that the committee largely felt that coronary anatomy was suitable for PCI. There were only a few cases identified as being insufficiently severe to warrant any form of revascularisation by stenting. Historically, PCI has encompassed an element of subjectivity based on the clinical scenario and the angiographic images, but now with access to detailed physiological and anatomical assessment within the catheter laboratory using pressure wire and intravascular ultrasound13,14, there is scope for a much more informed decision to be taken about the appropriateness of intervention.
An awareness that potentially any case may be scrutinised by the audit panel promotes thoughtful decision-making prospectively, especially in borderline cases. On occasion, comments by the review team led to an exchange of differing opinions with the operator, and at times a lack of final agreement, but the process of review allowed an opportunity for reflection, which has to be welcomed, and encourages an atmosphere of good medical practice15.
Some operators were more likely to have poorer outcomes than others. This observation needs careful analysis as the potentially misleading conclusion might be reached that a poorer success rate implied a less able operator. In the majority of cases it simply reflected the complexity of the cases e.g., some operators had a higher rate of chronic total occlusion cases, or were undertaking cases where there had been a previous failed attempt.
In over 90% of cases, there were comments made about the case by the committee, which reflected a range of opinion, and allowed the operator to review the case in light of the comments received. Many of the comments were complimentary, some less so, but as all PCI operators take their turn on the review committee all had ownership of the process rather than some individuals being “in charge”. Further representation on the review committee by a non-interventionist, may potentially have provided a broader perspective, but coordinating the busy schedules of so many committee members in order for the review to take place, on such a regular basis, was felt to be too challenging.
From the outset, there were no guidelines in place about what to do if an operator were to fall regularly outside acceptable practice. However, if an operator was performing PCI on patients with no evidence of ischaemia and insignificant lesions on a regular basis, this would become apparent, and likewise, at the other end of the spectrum, if an operator was performing PCI on patients with “surgical” disease without prior multidisciplinary discussion, this would also be identified. Hopefully, the operator would recognise that patterns of practice did not satisfy peer review, and adjust accordingly. If poor practice continued despite feedback, then other governance mechanisms would have to be activated.
Quality is difficult to measure in medical practice. Within this model it is achieved by review of one’s peers, and it is this element that could be translated to many areas of clinical practice. Obviously within the domain of PCI there are accurate angiographic records of how procedures are performed, which allows for objective assessment to take place. It is possible that a similar model, which has as its mainstay the principle of retrospective peer review, could be created for other specialities16. Many assessments take place when there are clinical concerns about poor practice. If processes such as those outlined in this paper were in place across medical and surgical practice, it would identify areas that might need attention well before any action was required to remedy poor clinical behaviour.
The key element of this process engenders a spirit of reflective practice that, ultimately, improves patient care.
Acknowledgements
We would like to thank Isabel McGregor and Gemma Baldock-Apps for their assistance in providing angiographic images and data from the peripheral hospital sites, which is central to providing a true representation of PCI practice across the network.
Conflict of interest statement
The authors have no conflicts of interest to declare.
Appendix
