Diagnostic accuracy of optical flow ratio: an individual patient-data meta-analysis

Introduction

Intracoronary optical coherence tomography (OCT) imaging provides super resolution in vivo, allowing detailed evaluation of coronary lumen, plaque characteristics, and stent expansion and apposition1. Fractional flow reserve (FFR) is the current reference standard for the functional evaluation of coronary artery stenosis2. An OCT-based morpho-functional evaluation approach based on a single catheter could reduce procedural complexity, time and cost, while enabling the use of both coronary imaging and physiology for percutaneous coronary intervention (PCI) guidance and optimisation.

Recently, a novel method for the fast computation of FFR from OCT images, optical flow ratio (OFR), has been developed3. The diagnostic performance of the OFR has been validated in de novo lesions, in-stent restenosis and native lesions immediately after PCI34567.

In this systematic, individual patient data (IPD) meta-analysis, we aimed to evaluate the diagnostic performance of the CE (European conformity)-marked OFR algorithm (Pulse Medical) with wire-based FFR as the reference standard.

Methods

Study selection, data extraction, analysis strategy, and data reporting were performed according to the PRIMSA guidelines for IPD meta-analysis. The authors of all the included studies were contacted for patient-level data for this IPD meta-analysis. This meta-analysis was registered in PROSPERO (CRD42021287726).

Search strategy

PubMed, Medline and Embase were searched for eligible studies using a combination of keywords including “optical coherence tomography”, “optical flow ratio” and “fractional flow reserve”. We applied no restrictions on language or publication period. A detailed search strategy is listed in Supplementary Table 1. The literature search was performed on 1 May 2022.

Eligibility criteria

Prospective and retrospective studies with paired assessments of the OFR and wire-based FFR in patients with coronary artery disease (CAD), including both chronic and acute coronary syndromes, were considered. An additional criterion for eligibility was the use of the same CE-marked OFR algorithm, ensuring that the potential interstudy variation was not explained by software-related aspects. Duplicates, case reports, editorials, reviews or meta-analyses, and personal communications were excluded. Two independent observers performed the initial duplicate removal, screened the titles and abstracts, selected eligible studies following full text reviews, extracted the data, and performed quality scoring. Discrepancies were resolved by the involvement of a third reviewer.

Data integrity

The data were double-checked for data integrity and correspondence with the individual published papers for key results related to completeness, accuracy and consistency. The reproduced results were confirmed by each study’s principal author before data pooling.

Risk of bias in individual studies

The QUADAS-2 (Bristol University) system was used to assess the applicability and bias for each of the included studies.

Ethics

All sites obtained approval from local institutional review boards for the individual studies. Written informed consent was provided by the patients or waived by the institutional review boards as appropriate.

Outcome measures

The primary outcome was the vessel-level diagnostic concordance of OFR with wire-based FFR as reference standard in the overall population. Secondary outcome measures included vessel-level numerical agreement between OFR and FFR in the overall population: sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), positive likelihood ratio (LR+), negative likelihood ratio (LR-) and a hierarchical summary of receiver-operator characteristics curve (HSROC) for OFR to predict wire-based FFR in the overall population. They also included identification of independent predictors for increased OFR‒FFR difference using predefined clinical and lesion characteristics: age, sex, body mass index (BMI), hypertension, hyperlipidaemia, current smoker, diabetes, minimal lumen area (MLA), left anterior descending artery (LAD) lesion location, a high frame rate (100 frames/s classified as a low frame rate, while 180 frames/s classified as a high frame rate), a low pullback speed (18 mm/s and 20 mm/s classified as a low pullback speed while 36 mm/s classified as a high pullback speed), previous myocardial infarction (MI) and previous PCI.

OCT imaging

Details of OCT measurements were reported in all included studies34567. In summary, OCT imaging was performed using frequency-domain (FD) OCT systems (ILUMIEN, OPTIS, or C7-XR; Abbott), with the Dragonfly or Dragonfly Duo (also Abbott) imaging catheters, or the FD-OCT OPTIS system (LightLab Imaging) with the Dragonfly Duo and Dragonfly OPTIS catheter (Abbott). The fibre probe was pulled back at 18 mm/s, 20 mm/s or 36 mm/s within the stationary imaging sheath. Cross-sectional images were generated at a rotational speed of 100 frames/s or 180 frames/s. The cases from one centre were acquired with an F-1 system and a T-1 catheter (Forssmann Medical Co. Ltd.,), both with analogous technical characteristics to the above-mentioned system except for a rotation speed of 100 frames/s and a pullback speed of 18 mm/s.

OFR analysis

In all included studies, OFR analyses were performed by certified OFR analysts, blinded to clinical data and FFR values. OFR values at the distal position of the analysed vessel were obtained for comparison with wire-based FFR34567. OFR analyses for vessels with de novo lesions, in-stent restenosis, and native vessels immediately after stenting were all included for evaluation of the primary and secondary outcome measures. A representative case example of OFR analysis is shown in the Central illustration and Moving image 1.

Central illustration. A representative case example of OFR analysis. A) Coronary angiography immediately after PCI for a lesion mid-LAD. B1-B4) Correspond to the four positions (dashed lines) in (A). C) 3D reconstruction of both the lumen and stent struts from OCT pullback image. D) Co-registration between OFR pullback curve and lumen diameters for the reconstructed vessel. 3D: three-dimensional; LAD: left anterior descending artery; OCT: optical coherence tomography; OFR: optical flow ratio; PCI: percutaneous coronary intervention

Statistical analysis

Both one-stage and two-stage approaches (if applicable) were applied for this IPD meta-analysis. The one-stage approach analysed the IPD from all included studies in a single step, using a nested random-effect model accounting for study-level, centre-level, and patient-level clustering and heterogeneity. In the two-stage approach, first the IPD were used to obtain aggregate data (e.g., diagnostic performance of OFR against FFR) for each study separately, and second, these aggregate data were combined using a random-effect inverse variance-weighted meta-analysis. For the sake of succinctness, results derived from the one-stage approach were mainly presented throughout the manuscript, while results derived from the two-stage approach were presented in the Supplementary data. The diagnostic performance of OFR against FFR was analysed on a per-vessel level and repeated on a per-patient level to explore the effect of multivessel disease. In patients with paired OFR and FFR interrogated in multiple vessels, the vessel with the lowest FFR value with the corresponding OFR value was used. All analyses were performed for the overall population (pre- and post-PCI vessels combined), and then for pre-PCI vessels only and post-PCI vessels only, separately. For vessels before PCI, the diagnostic accuracy of the OFR was defined as the proportion of vessels with both OFR and FFR values of 0.80 or>0.80, while the cut-off value of 0.90 was used for both OFR and FFR in vessels immediately after PCI to identify physiologically suboptimal stenting results. A concordance correlation coefficient was used for evaluation of the correlation between FFR and OFR. Bland-Altman plots together with scatter plots were used to show the distribution of FFR and OFR. Subgroup analyses were performed regarding the numerical difference between OFR and FFR, including pre-PCI versus post-PCI vessels, and European/American versus Asian centres. Univariate and multivariate logistic regression analyses were used to identify the potential clinical and procedural characteristics that were associated with OFR values that deviated from FFR values by at least 0.10. Potential publication bias was tested using Deek’s funnel-plot asymmetry test for the main analysis. All analyses were performed by using MedCalc version 20 (MedCalc) and STATA version 14 (StataCorp). A two-sided p-value of <0.05 was considered statistically significant.

Results

Study selection

Supplementary Figure 1 summarises the study flow. A total of 16 studies were identified with 10 of the studies entering title and abstract screening after the removal of duplicates. Finally, 7 articles were selected for full text review, of which 5 were included in the analysis. All 5 studies provided individual patient data. The studies showed high methodological quality with low risk of bias rated by QUADAS-2 (Supplementary Figure 2).

Study and patient characteristics

The main findings of all the included studies are listed in Table 1. Baseline clinical and lesion characteristics of a total of 574 patients with 626 vessels are listed in Table 2. Among them, paired pre-PCI OFR and FFR was available in 404 vessels with de novo lesions or in-stent restenosis, while paired post-PCI OFR and FFR was available in the remaining 222 vessels. The median vessel-level FFR was 0.86 (0.79-0.93) in the overall population, 0.83 (0.76-0.89) in vessels with de novo lesions or in-stent restenosis, and 0.92 (0.88-0.96) in vessels post-PCI. A total of 171 (42.3%) vessels with paired pre-PCI measurements were physiologically significant as identified by FFR ≤0.80. A total of 85 (38.3%) vessels with paired post-PCI measurements were physiologically suboptimal as identified by post-PCI FFR ≤0.90. Deek’s funnel plot asymmetry test (Supplementary Figure 3) showed that there was no significant publication bias (p=0.64).

Table 1. Study characteristics.

Table 2. Baseline clinical and lesion characteristics.

Correlation and agreement between OFR and FFR

The overall vessel-level concordance correlation coefficient between OFR and FFR was 0.86 (95% CI: 0.84-0.88) (Supplementary Table 2), showing increased scatter along with lower FFR values (Supplementary Figure 4). The pooled vessel-level mean difference of OFR and FFR in the overall population was 0.002±0.05 with heterogeneity of I²=0.0% (Supplementary Figure 5). The vessel-level numerical agreement of OFR and FFR was independent of baseline clinical parameters and centre region (Figure 1, Supplementary Figure 6). The vessel-level concordance correlation coefficient and pooled mean difference of OFR and FFR were further tested in pre-PCI and post-PCI vessels separately, with results shown in Supplementary Table 2 and Supplementary Figure 4-Supplementary Figure 8. Sensitivity analysis showed that the numerical difference of OFR and FFR was not influenced by the removal of each of the 5 studies (Supplementary Table 3).

Figure 1. Subgroup analysis, presented as the numerical difference between OFR and FFR by vessel level. Small MLA ≤1.88 mm² for pre-PCI [4] and small MLA ≤3.28 mm² for post-PCI [7]. BMI: body mass index; CI: confidence interval; FFR: fractional flow reserve; LAD: left anterior descending artery; MI: myocardial infarction; MLA: minimal lumen area; OFR: optical flow ratio; PCI: percutaneous coronary intervention

OFR distribution according to FFR ranges

The distribution of OFR data according to different ranges of FFR values is shown in Supplementary Figure 9. The majority of OFR observations were located in the FFR range 0.60 to 0.90.

Diagnostic accuracy of OFR

The overall vessel-level diagnostic concordance of OFR and FFR was 90% (95% confidence interval [CI]: 87%-92%) in the overall population, using a cut-off value of 0.80 in pre-PCI vessels and a cut-off value of 0.90 in post-PCI vessels (Supplementary Table 4, Supplementary Table 5). The pooled diagnostic performance estimates in the overall population were as follows: sensitivity 84% (95% CI: 79%-88%), specificity 94% (95% CI: 91%-96%), positive predictive value 90% (95% CI: 86%-93%), negative predictive value 89% (95% CI: 86%-92%), positive likelihood ratio 13.51 (95% CI: 9.06-20.14), and negative likelihood ratio 0.17 (95% CI: 0.13-0.23). The diagnostic concordance of OFR and FFR was 91% (95% CI: 88%-94%) in pre-PCI vessels, and 87% (95% CI: 82%-91%) in post-PCI vessels. Other diagnostic measures in both pre- and post-PCI data are shown in Supplementary Table 5. The forest plot results by two-stage analyses are illustrated in Supplementary Figure 10-Suppementary Figure 15. HSROC curves for vessel- and patient-level data separately are shown in Supplementary Figure 16 and Supplementary Figure 17.

Patient-level analysis

At the patient-level, the diagnostic concordance of OFR and FFR was 90% (95% CI: 87%-92%) in the overall population, 91% (95% CI: 88%-94%) in pre-PCI vessels, and 87% (95% CI: 82%-91%) in post-PCI vessels. Other patient-level diagnostic performance estimates are listed in Supplementary Table 6.

Prediction of OFR−FFR disagreement

In the overall population, multivariate regression analyses demonstrated that the MLA was an independent predictor of an OFR ≥0.10 lower than FFR (odds ratio [OR] 0.39, 95% CI: 0.18-0.82; p=0.013) and a low pullback speed (OR 7.02, 95% CI: 1.68-29.43; p=0.008) was an independent predictor of OFR ≥0.10 higher than FFR (Table 3, Table 4). Multivariate regression analyses were repeated in pre-PCI vessels only and a low pullback speed was an independent predictor of an OFR ≥0.10 higher than FFR (Table 3, Table 4). The distribution of cases with OFR−FFR ≥0.10 or OFR−FFR ≤−0.10 stratified by ranges of FFR values is listed in Supplementary Table 7. In post-PCI vessels, stent length was not significantly associated with OFR−FFR ≥0.10 (OR 0.92, 95% CI: 0.70-1.22; p=0.566) or OFR−FFR ≤‒0.10 (OR 0.79, 95% CI: 0.50-1.12; p=0.297) by univariate regression analysis.

Table 3. Predictors of OFR−FFR ≥0.10 by univariate and multivariate analyses at the vessel-level.

Table 4. Predictors of OFR−FFR ≤−0.10 by univariate and multivariate analyses at vessel level.

Discussion

The following points summarise the main findings of the present study: (1) OFR had a high diagnostic accuracy with wire-based FFR; (2) there was good correlation and numerical agreement between the OFR and FFR, with increased scatter along with lower FFR values; (3) centre-level data analysis revealed that the agreement between OFR and FFR was independent of geography, and (4) OCT-derived low MLA and low pullback speed were independent predictors of an increased numerical difference (>0.10) between OFR and FFR.

The overall good correlation and numerical agreement between OFR and FFR can be attributed to the following factors. The novel algorithm of computational FFR8 was applied to the OCT images with super in vivo resolution, which provided precise lumen dimensions as the geometric boundary3. Instead of using a linear tapering reference lumen, the area of the side branch ostium was used to compute a step-down reference vessel function across bifurcations3. This provided a more accurate reconstruction of the normal coronary lumen and an estimation of downstream blood flow. In the presence of implanted stents, all struts were detected and reconstructed in 3D using a deep learning-based algorithm9, incorporated with lumen geometry for pressure drop computation. The described approach produced a pressure pullback curve along the reconstructed vessel allowing for differentiation of diffuse and focal disease (Central illustration).

To combine physiological and morphological assessments using a single OCT catheter, several similar concepts have been previously proposed10111213, with most models predominantly relying on computational fluid dynamics and having the inherent limitations of a long computational time12. Recently, Seike et al developed a new approach based on fluid dynamics equations and shortened the computational time to 10 minutes11. However, only single-vessel models are used, while the fractal flow in bifurcations was not taken into account. Sufficient clinical validations for the above-mentioned methods are still lacking. OFR has emerged as an appealing tool in simultaneous physiological and morphological assessments, given the fast computational time of around 1 minute3 and high overall diagnostic accuracy of 90%.

Most of the studies included in this meta-analysis applied OFR in addition to the angiography-based quantitative flow ratio (QFR)8141516. Fundamentally, both QFR and OFR adapt fluid dynamics equations for the estimation of FFR but with fundamental differences related to vessel geometry reconstruction and the estimated blood flow rate3. The lumen reconstruction related to OFR is more accurate than that of QFR given the resolution of intravascular imaging. However, compared with the QFR’s reconstruction algorithm, including the estimation of hyperaemic flow based on contrast flow8, OCT provides static images with no dynamic parameters available for flow velocity estimation. Specifically, in the OFR algorithm, patient-specific volumetric flow is derived from a fixed hyperaemic flow velocity of 0.35 m/s multiplied by the reference vessel size17. The combination of a geometrical reconstruction based on high-resolution intravascular imaging and a fixed flow rate in the form of OFR appears to outperform QFR4. The presented bias and imprecision (standard deviation [SD]±0.05) are at a level not previously presented in similar populations for alternative solutions based on coronary computed tomography angiography (CTA) and invasive angiography1618. More importantly, the minimal user interaction related to OFR measurements appears to have helped overcome a major limitation in terms of reproducibility for image-based physiology19. The test-retest repeatability of OFR approaches results from FFR reproducibility studies (SD of ±0.02 for intra-observer agreement)320. With the acceptance of QFR in a global scope, OFR may grow into a complementary solution in cases with the need for culprit or non-culprit plaque evaluation and for planning purposes in patients with an a priori high likelihood of PCI4. In post-PCI settings, OCT visualises coronary lumen, stent struts and plaque compositions in detail, providing the rationale for the evaluation of PCI efficacy and PCI optimisation7. Admittedly, in comparison with the QFR, limited outcome data exist validating the prognostic value of the OFR in either pre- or post-PCI scenarios. The only evidence comes from a recent study showing that OFR in combination with a novel morphological index, lipid-to-cap ratio (LCR), allows for the identification of a subgroup of patients with a 43-fold higher risk of recurrent cardiovascular events in non-culprit vessels after acute coronary syndromes21. Further prognostic data would help justify the clinical use of the OFR. In addition, the clinical application of the OFR naturally depends on the penetration of intracoronary imaging itself, which is still limited in many areas. However, computational approaches exist for a broad variety of imaging modalities, thus, also allowing for the use of intravascular ultrasound (IVUS) that could have advantages over OCT in specific patients and lesion types (e.g., ostial lesions)1622.

The current multicentre and multiregion clinical database provides robust diagnostic performance estimates informing on the reliability of OFR across Asian and US/EU regions. Our results apply to a broad range of patients (due to the inclusion of both stable angina and acute coronary syndrome) with mean FFR values in line with previous large all-comer invasive physiology studies23. Moreover, the results provide a foundation for the identification of factors affecting the accuracy of the OFR. Firstly, our results show that a high frame rate has limited influence on the OFR’s accuracy, while a higher pullback speed appears to be favourable to reduce large differences between the OFR and FFR (>0.10). One of the possible explanations for the latter is that a lower pullback speed might be more subject to cardiac motion artefact. It increased the risk of longitudinal discontinuity between consecutive OCT frames, leading to inaccuracies in 3D lumen reconstruction and pressure drop computation. Secondly, an increase in the MLA decreases the risk of a large OFR−FFR difference (>0.10). The clinical importance of the reduced precision of computational FFR in severe stenoses is still not clear, since most of the tight stenoses would have been stented. However, more evidence is needed on the combined use of OCT and OFR in populations, including more severe lesions intended for PCI where the mean FFR is likely to approach 0.7024.

Limitations

Although the sample size of this analysis is increased by meta-pooling, the proportion of prospective and retrospective studies is not balanced, with only one prospective study. As each included study has a different emphasis, not all potential factors of interest could be included in the current study (e.g., plaque composition, microcirculatory dysfunction, tandem lesions, diffuse disease). We were not able to perform a subgroup analysis according to lesion location (proximal/middle/distal) because such information was not reported by all the included publications. In all the included studies, the OFR values at the distal position of the analysed vessel were obtained for comparison with wire-based FFR. In clinical practice, OCT catheters are usually placed less distally than pressure wires6. Although vessels with stenosis between the positions of the distal optical sensor and pressure sensor had been excluded in all studies, a pressure drop might still be seen in certain non-stenotic but atherosclerotic vessels25. This could be a possible source of deviation of OFR values from FFR values. Future OCT imaging systems with faster pullback speeds and longer pullback lengths might reduce the discrepancy between the OFR and FFR. The heterogeneity index I² was high for several pooled estimates including sensitivity, mainly caused by a relatively different level of diagnostic estimates found by Ding et al7 compared with the other 4 studies (Supplementary Figure 10-Supplementary Figure 15). Of note, in post-PCI datasets, as enrolled by Ding et al7, the OFR and FFR values showed a narrow range and skewed distribution towards normal values, with a mean FFR of 0.92 and a mean OFR of 0.93, which introduced a higher risk of finding false negative results7. Therefore, the overall high pooled estimates should be interpreted with caution that variation could exist in the diagnostic performance in populations with different levels of physiology. Nevertheless, the sensitivity analysis showed that the numerical difference of the OFR and FFR was not influenced by the removal of any of the 5 studies. Lastly, prognostic data of OFR are still limited, despite a recent study finding that OFR independently predicted non-culprit vessel related major adverse cardiovascular events at 2-year follow-up, with an area under the curve (AUC) of 0.838 (95% CI: 0.806-0.866)21. Thus, the good diagnostic concordance between OFR and FFR found in the current meta-analysis should be considered as only hypothesis-generating. Future studies investigating the prognostic value of OFR are warranted. It would also be interesting to explore the risk stratification function based on a combination of OFR and other factors including high-risk plaque features (e.g., thin cap, edge dissection, stent underexpansion, etc.) in a future study.

Conclusions

This meta-analysis based on individual patient data demonstrated high diagnostic accuracy for OFR using wire-based FFR as the reference. The OFR has the potential to provide an improved integration of intracoronary imaging and physiological assessment for accurate evaluation of coronary artery disease in the catheterisation laboratory.

Impact on daily practice

The OFR enables functional and morphological assessment of coronary artery stenoses including an evaluation of stent expansion and apposition with the use of a single catheter. The increasing application of OCT has the potential to optimise coronary interventions.

Funding

S. Tu would like to acknowledge the support from the Natural Science Foundation of China (No. 82020108015 and 81871460). W. Wijns would like to acknowledge support from the Science Foundation Ireland Research Professorship Award (15/RP/2765).

Conflict of interest statement

S. Tu reports research grants from and is a consultant for Pulse Medical. W. Wijns reports institutional research grants and past honoraria from MicroPort; is a senior advisor of Rede Optimus Research and Corrib Core Laboratory; and is a co-founder of Argonauts, an innovation facilitator. The other authors have no conflicts of interest to declare.

Supplementary data

To read the full content of this article, please download the PDF.