Video motion analysis in live coronary angiography differentiates levels of experience and provides a novel method of skill assessment

Abstract

Aims: Video motion analysis (VMA) uses fluoroscopic sequences to derive catheter and guidewire movement, and is able to calculate 2D catheter-tip path length (PL) on the basis of frame-by-frame pixel coordinates. The objective of this study was to validate VMA in coronary angiography as a method of skill assessment.

Methods and results: Forty-seven coronary interventions performed by 10 low- (<1,000 cases; group A), five medium- (1,000-4,000; group B) and six high- (>4,000; group C) experience-volume cardiologists were prospectively recorded and analysed using VMA software. Total PL was calculated and procedure, fluoroscopy times, and radiation dose were recorded. Comparisons of PL were made between groups of experience. Groups A, B and C performed 24, 14 and 6 paired (right and left coronary) cannulations, respectively. Calculation of PL was possible in all recorded cases and significantly correlated with procedure (p=<0.001, rho=0.827) and fluoroscopy times (p=<0.001, rho=0.888). Median total path length (combined right and left coronaries) was significantly shorter in group C which used 3,836 pixels of movement (IQR: 3,003-4,484) vs. 10,556 (7,242-31,408) in group A (p=<0.001) and 8,725 (5,187-15,150) in group B (p=0.013).

Conclusions: VMA in coronary angiography is feasible and PL is able to differentiate levels of experience.

Abbreviations

LCA: left coronary artery

PL: path length

RCA: right coronary artery

VMA: video motion analysis

Introduction

The rapid evolution of endovascular techniques and technology in the last two decades and reduced working hours require robust assessment tools to assess progress in training and to demonstrate competence within a training period. Patients have increased expectations and therefore outlying performers should be identified in every endovascular intervention, e.g., in cardiac catheterisation, peripheral arterial intervention and in other interventional procedures.

According to the 2006 UK national coronary angioplasty audit, diagnostic coronary angiography (CA) is the most frequently performed endovascular procedure, with 215,575 diagnostic coronary angiographies undertaken in one year1. Morbidity and mortality for this procedure vary between 0.8-4.6% and 0-0.2%, respectively, according to international data2. More recent data looking specifically at PCI indicate that 92,445 such procedures were performed across 118 PCI centres in the UK3. Despite the large volume of these procedures being performed annually, investigation into suitable metrics to define proficiency in this procedure remain largely restricted to radiation dose area product (DAP) and procedure and fluoroscopy times2,4-6, although consensus exists regarding ideal reference levels for DAP (45 Gy·cm2), fluoroscopy time (6.5 mins)7 and contrast usage (100 ml)5. International recommendations have defined that trainees have to perform between 150 and 300 coronary angiograms under supervision prior to independent practice6,8,9.

Virtual reality simulators have the potential to assess technical competence in coronary angiography by automated registration of performance parameters10. Although these metrics have demonstrated good construct validity in terms of training level11,12 and experience10,13-15, they are still regarded as surrogate markers of skill to evaluate the quality of performance. Rating scales have been developed in an attempt to assess the quality of endovascular performance: these include generic16 and procedure-specific rating scales14, most of which are derived from Objectively Structured Assessment of Technical Skill (OSATS). Although these assessment tools have been extensively validated, they are time-consuming because of the post hoc analyses by at least two experts. Additionally, assigning numerical values to qualitative statements predisposes these assessments to a degree of subjectivity17. Similar to the endovascular rating scales described above, global and procedure-specific qualitative rating scales have been developed for the assessment of invasive cardiology skills11,18. Lipner et al have validated a scoring matrix based on cognitive decision-making processes at key points in simulated PCI which incorporates management of crisis scenarios19.

Video motion analysis (VMA) is a relatively new method of assessing skill in catheter-based intervention and may be able to assess technical ability reliably in simulated carotid artery stenting20. This technology utilises fluoroscopy sequences to provide motion-descriptive data on a semi-automatic basis, including number of movements and distance travelled (path length [PL]).

The primary objective of this study was to investigate whether VMA can differentiate levels of experience in real patients during diagnostic coronary angiography. Secondary objectives were to correlate VMA data with previously validated quantitative markers of skill and to investigate the impact of access route (radial versus femoral access) in the trainees still on their learning curve, and anatomy (right versus left coronary artery) on VMA data.

Methods

STUDY DESIGN AND PARTICIPANTS

The fluoroscopy screens of 50 coronary angiographies performed in a single coronary catheterisation suite (Toshiba Infinix CF-i™; Toshiba Medical Systems, Otawara, Japan) were prospectively recorded on consecutive working days. Digital camera set-up was standardised with respect to angulation (0°) and distance (50 cm) from the fluoroscopy screen. All angiographies were performed at 0 magnification factor. Prior to starting any recordings, 21 participants were subdivided into three groups based on coronary angiography experience, using stages of the standard United Kingdom cardiology training programme as reference: group A (low volume, <1,000 coronary angiography cases, first two years of higher specialist training), group B (medium volume, 1,000-4,000 cases, last four years of higher specialist training), and group C (high volume, >4,000 cases, all levels post completion of training including senior fellows and consultants). Both femoral and radial access routes were included in this cohort, and all cases were either elective diagnostic coronary angiographies or elective PCI procedures. Cases of emergent PCI for myocardial infarction were excluded. In order to reduce anatomical heterogeneity in this series, three post-CABG internal mammary artery angiographies were excluded from final analysis, leaving 47 videos that were ultimately used for analysis. In these cases, coronary anatomy was significantly altered as a result of surgical intervention and would have significantly affected the procedural metrics recorded. Of the 47 retained videos, 13 were of elective PCI. Patient consent was obtained for all anonymised fluoroscopy screen recordings according to institutional research compliance office guidance. Formal ethical approval was not required since this study obtained only anonymised recordings of fluoroscopic footage. At no stage were patient-specific data or identifiers used. At no point were treatments or investigations outside the remit of standard clinical care administered.

CATHETER TRACKING SOFTWARE

The catheter tracking software has been described extensively previously20. In brief, a software package was created that allows video-file editing and frame-accurate analysis of fluoroscopic video sequences. The user uploads the video file for analysis onto the software platform and commences catheter or guidewire tip tracking by selecting the tip of the endovascular tool using the mouse cursor and placing a tracking marker. The initial tracking point was standardised and defined as the point at which the catheter or guidewire crosses the inferior border of the head of the clavicle. The software then estimates the catheter tip’s position in subsequent frames using a semi-automatic tracking scheme that probes the immediate pixel field around the tracking marker for a similar pattern of pixellation. The software generates pixel coordinates for each frame based on the catheter tip’s position, and calculates 2D movement (path length [PL]) using cumulative frame-by-frame coordinate data, which is expressed as movement in pixels (Figure 1). In all videos, the distalmost tip of the guidewire/catheter interface (either the guidewire or the catheter) was tracked. In order to assess inter-rater reliability, the analysis software was applied to a video of a guidewire being threaded through a silicone-tubing model of standardised length (Figure 2). Five different participants tracked the same video three times, and Cronbach’s α was used to test for inter-rater reliability.

Figure 1. Data acquisition and calculation of distances between pixels. A) The active software tracking the catheter tip. On each frame, a set of pixel coordinates (x,y) is automatically generated. B) The software records the frame number (orange ring), the pixel coordinate in the x dimension (blue ring) and the pixel coordinate in the y dimension (green ring). The black ring indicates the calculation performed on Microsoft Excel™, with columns D and E representing the difference in pixels in x and y terms between the current and preceding frames. A Pythagorean calculation gives us the total distance (hypotenuse) in pixels travelled by the catheter tip between the two sets of pixels (red ring). C) Example of catheter trajectory.

Figure 2. Bench-top validation. A) The standard shape composed of silicone tubing embedded into a lightweight KAPA® foam board (300 mm in length). B) Appearance of the shape on the fluoroscopy screen.

TASK ANALYSIS

A single blinded assessor performed the post hoc video analysis of all cases using the catheter-tracking software. Strict anatomical start (inferior margin of the head of the clavicle) and endpoints (first injection of contrast into the coronary ostium) were defined for each cannulation attempt to standardise the data acquisition.

Path-length values, fluoroscopy time (FT), procedure time (PT) and dose area product (DAP) were collected for each cannulation attempt, in both radial and femoral access cases, according to this anatomical definition. In cases where only one vessel was cannulated by a particular individual, PT, FT and DAP were collated for that attempt only, with PT being the cannulation time for that particular vessel.

Phases of active movement were tracked; tracking was not performed during phases of angiographic image manipulation (such as C-arm rotation) in order to avoid incorporation of movement artefact into the final analysis. If a trainee started a cannulation attempt that was abandoned and the supervisor took over, the trainee’s cannulation attempt was discounted and the supervisor’s attempt was coded as a high volume of experience attempt for future analysis. Consequently, some cases were counted as “unpaired”; the supervisor would cannulate a given side, and the trainee the other. If the operator cannulated both coronaries in one patient, these cases were counted as “paired”. Total PL was calculated by combining the PL values for each coronary vessel.

STATISTICAL ANALYSIS

Data were analysed using the Statistical Package for Social Sciences, Version 22.0 (IBM Corp., Armonk, NY, USA). Each cannulation attempt was treated as a separate data point with regard to the variables recorded. Data were found to be non-Gaussian, so non-parametric tests were used. The Kruskal-Wallis test was used to compare the total PL across the three groups of experience and between right and left coronary vessel catheterisation. Predefined post hoc comparisons between each group of cardiologists were performed with the Mann-Whitney U test. Correlations between PL and procedure time, fluoroscopy time and DAP were evaluated using Spearman’s rank correlation coefficient. Comparisons between right and left coronary arteries were performed for the paired cannulations using the Wilcoxon signed-rank test.

Results

Of the 47 retained videos, 13 were of elective PCI. It was feasible to track coronary catheters and guidewires with minimal manual adjustment of the tracking marker. Figure 3 gives a typical graphical representation of catheter trajectories overlaid onto an angiographic image. Testing for inter-observer reliability yielded a Cronbach’s α of 0.979.

Figure 3. Catheter tracking - graphical representation of catheter paths in right and left coronary artery cannulation. A) Low volume of experience. B) Medium volume of experience. C) High volume of experience.

PATH-LENGTH DIFFERENCES BETWEEN GROUPS OF EXPERIENCE

Groups A, B, and C performed 24, 14, and 6 paired cannulations, respectively. A stepwise decrease in total PL, with a progressive narrowing of interquartile ranges was observed with increasing experience (p=0.001). Median path length was significantly reduced in group C which used a total of 3,836 pixels of movement (IQR: 3,003-4,484) vs. 10,556 (7,242-31,408) in group A (p=<0.001), and 8,725 (5,187-15,150) in group B (p=0.013) (Figure 4).

Figure 4. Total path length. Box plot representing PL taken (y-axis) to complete cannulation of bilateral coronaries across groups of experience (x-axis) – Kruskal-Wallis, p=0.001. The whiskers represent extreme values and the stars the outliers.

PATH-LENGTH DIFFERENCES IN RELATION TO SIDE OF CORONARY CANNULATION

A total of 48 left and 45 right coronary cannulations were performed when pooling all “paired” and individual “unpaired” cannulations. Similarly, a stepwise reduction in PL was noted with increasing experience for right coronary artery cannulation (Figure 5). Across all operators, path length for catheterisation of the LCA was significantly shorter compared to the path length to cannulate the RCA, requiring 2,625 (1,802-4,755) vs. 4,956 pixels (2,931-9,159) (p=0.023), respectively. Median path length for RCA cannulation was significantly reduced in group C which used 2,308 (1,405-2,630) vs. 5,591 pixels (4,004-9,823) in group A (p=<0.001) and 4,812 (2,332-12,960) for group B (p=0.015). There were no significant differences in path length for LCA cannulation between group A which used 3,201 pixels (2,149-5,553), group B which used 2,666 pixels (1,462-4,668), and group C which used 2,068 pixels (1,683-6,733) (p=0.206).

Six cannulation attempts were performed on arteries with abnormal origin, based on angiographic appearance. Four of these were performed by operators in group A, and two by operators in group B. In two of these cases (one from group A, and one from group B), the cannulations were completed by operators from groups B and C, respectively, resulting in reduced path-length recordings.

Figure 5. Right coronary artery. Box plot representing PL taken (y-axis) to complete cannulation of the right coronaries across groups of experience (x-axis) - Kruskal-Wallis, p=0.001. The whiskers represent extreme values and the stars the outliers.

CORRELATION OF PATH LENGTH WITH PROCEDURE TIME, FLUOROSCOPY TIME AND RADIATION DOSE (DAP)

Across all operators, median procedure time was 6.9 minutes (IQR: 5.4-11.05 min), fluoroscopy time 2.6 minutes (1.7-51 min), and DAP 7.4 Gy·cm2 (1.7-11.7 Gy·cm2). A strong correlation (Figure 6) of PL with PT (rho=0.827, p=<0.001) and FT (rho=0.888, p=<0.001) was noted. A moderate correlation was observed with DAP (rho=0.524, p=<0.001).

Figure 6. Metric correlations. Scatter plots showing correlation of PL with procedure time (A), fluoroscopy time (B) and radiation dose (DAP) (C). PL is given on the x-axis and conventional metrics on the y-axis. Individual values are shown by the red dots.

PATH-LENGTH DIFFERENCES IN RELATION TO ACCESS ROUTE

In the trainee group (<4,000 cases), median path length for cannulation of both coronaries via the femoral access route was 9,893 pixels (IQR: 6,385-12,419) versus 19,099 pixels (7,166-32,184) for the radial access route (p=0.104). In the high-volume/expert group (>4,000 cases), the median path length for both coronaries via the femoral access route was 3,524 pixels (2,458-4,615), versus 4,155 pixels (3,398-7,206) for the radial access group (p=0.686).

Discussion

Physicians should have the appropriate knowledge and skills to perform endovascular procedures safely. Path-length calculations may objectively assess quality of performance as shown by our institution in simulated carotid interventions20. In that pilot study, PL was able to differentiate between levels of experience showing a stepwise reduction in PL with progressive experience.

This study has shown for the first time that VMA software can be used in live coronary angiography to perform semi-automated tracking of live fluoroscopic sequences. Furthermore, PL is a valid assessment parameter in coronary angiography with a clear stepwise reduction of PL with progressive experience.

To our knowledge, this is the first clinical study to describe video-based tracking of guidewires and catheters in live fluoroscopic sequences during endovascular procedures. Other endovascular tool tracking mechanism have been developed, including MR-based21, electromagnetic-based22,23 and ultrasound-based24 modalities. Specifically in coronary angiography, Omary et al25 have described real-time MR-based tracking of cardiac catheters in a porcine model, although these required extensive modification of both the catheter and guidewire in order to facilitate accurate tool registration. This technology has the potential advantage of providing objective information in real time and performance feedback to trainees and assessors using fluoroscopic footage as a medium.

In this study, PL strongly correlated with PT and FT, and held a moderate correlation with radiation dose. The strong correlation of PL with FT exists because the software only tracked phases of active catheter movement, which is only possible during screening periods. Likewise, Jensen et al2 have shown a clear learning curve for FT. These observations are supported by extensive research into simulator-derived metrics which have so far only identified PT and FT as construct-valid performance metrics14-16.

In contrast, the weaker correlation of DAP with experience5, and with PL in this series may be explained by the number of variables which influence DAP readings; radiation dose is widely regarded as a product of a number of variables, including X-ray beam quality, X-ray beam limitation devices, X-ray geometry, and fluoroscopic and acquisition imaging dose rate settings26. In the present study, efforts were made to limit the influence of these variables by recording data from one particular angiography suite only.

Similar observations have been described by Räder et al5, who found a weak relationship between radiation dose and experience on linear regression, and by Hillock et al6, who found that lowering of radiation dose with experience was less predictable as compared to FT.

Correlation of PL with PT was slightly weaker than with FT; although the effect of experience on efficiency of movement was examined in this study, the relationship between experience and speed of decision making in cannulation strategy remains to be clarified. It is possible that relatively inexperienced operators use excessive movements in a “trial and error” fashion, whilst expert operators potentially spend more time mentally evaluating cannulation strategies rather than manipulating catheters, thus reducing total movement but increasing procedure time marginally.

The observation that the cannulation of the RCA was harder as determined by path-length measurement in the entire paired cannulation cohort is potentially explained by the anatomy of coronary take-off in “normal” adults. Congenital abnormalities excluded, the RCA ostium is normally oriented anteriorly as compared to the LCA which originates laterally27-29. We observed a stepwise reduction in PL for cannulation of the RCA with progressive experience; however, there were no differences across the groups for LCA cannulation. The bulk of all coronary cannulations in this series occurred with the C-arm oriented at a 0° angulation; RCA cannulation may present a technically more challenging task for more inexperienced operators, possibly because of the requirement to torque the angled catheter anteriorly with imaging aligned in an AP plane, or because junior trainees may simply be less aware of normal coronary anatomy and the need to adapt cannulation strategy according to the position of a target vessel. The data suggest that anatomical factors affect PL values and efficiency of cannulation, and that the right coronary potentially represents a discriminator in terms of skill assessment in coronary cannulation; experts are able to reduce their PL irrespective of the side targeted.

Based on these findings, it is interesting to note that the number of cases required to achieve the PL values of the experienced interventional cardiologists and to reduce the variability in performance (as indicated by interquartile ranges) may be significantly higher than previously thought. Most cardiological societies recommend 150-300 cases as primary operator prior to performing diagnostic coronary angiography independently. Likewise, Räder et al found that 300-600 cases were required, across a range of metrics (FT, PT, DAP, and contrast volume), to achieve expert levels of performance in a retrospective analysis of 4,200 coronary angiograms5, while Jensen et al stated that only 150 cases were required to reach expert performance2. It is possible that, due to the prospective and sequential gathering of data in this series, smaller numbers in the medium and high volume of experience groups may not accurately reflect this subpopulation of cardiologists in terms of PL. Nonetheless, PL data in this study strongly correlated with validated metrics (PL and FT) that were used in the studies described above, supporting the observed distributions of PL across the groups’ experience. The learning curve plateau for coronary angiography may not actually occur at between 300 and 600 cases, and the achievement of proficiency and expertise may require a significantly larger volume of experience.

Limitations

Potential weaknesses in this study relate to the fact that the software records movement in 2D only, whilst advancing a catheter around the aortic arch and cannulation of coronary vessels clearly involves movement in the x, y and z planes. Whilst this clearly results in a deficit of recorded movement data which may have been recorded by using 3D tracking mechanisms, one might argue that this is counterbalanced by the ability to analyse live cases without having to modify endovascular tools. Furthermore, the majority of interventional cardiology cases are performed using 2D fluoroscopy. We have carried out further as yet unpublished research that demonstrates that the 2D deficit in recorded movement data is quite small and predictable when compared to 3D electromagnetic tracking. Phases of fluoroscopy table or C-arm movement controlled by a technician or radiographer were not analysed in order to avoid incorporation of “non-operator-dependent” PL data, which may have resulted in the loss of valuable movement data. Further work is in progress to refine the software and address this limitation.

Recording live cases to evaluate skills has a number of drawbacks. Firstly, it is not possible to control anatomical variability, as is possible in simulated cases, in terms of factors such as arch tortuosity, calcification, thrombus and coronary ostial stenosis. Work is underway to quantify the effect of anatomical complexity on this new metric. The finding that some of the anatomically challenging cannulations were completed by experienced clinicians, resulting in reduced path lengths, supports the argument that experienced operators are able to minimise their path-length scores irrespective of challenging anatomy. Secondly, the performance of trainees may have been influenced by advice or orders from supervising cardiologists. These variables may have had an impact on the broad interquartile ranges in the low- and medium-volume groups, but the ability to overcome anatomical complexity may be a hallmark of true expertise and may be reflected by the narrow IQR in the high-volume group.

Conclusions

This study has demonstrated that video motion analysis is feasible in live diagnostic coronary angiography, and that PL is able to differentiate between groups of experience. VMA data correlate well with previously validated endovascular performance metrics, and may form a valuable, objective adjunct in endovascular skills assessment. The side of coronary cannulation affects VMA data, and may be a reflection of subtle variation in anatomical difficulty. The attainment of true expertise in this procedure may require a larger volume of experience than previously thought.

Funding

The research was funded/supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Imperial College Healthcare NHS Trust and Imperial College London.

Conflict of interest statement

The authors have no conflicts of interest to declare.