Abstract
Aims: We aimed at updating the evidence coming from randomised and observational studies of patent foramen ovale (PFO) closure compared to medical therapy in patients with cryptogenic stroke (CS).
Methods and results: Comparative studies of PFO closure versus medical therapy published or presented through March 2013 were identified. Data from 2,303 patients in three randomised clinical trials (RCTs) and from 2,231 patients in 11 observational studies were included. In RCTs, the stroke hazard ratio (HR) for PFO closure versus medical therapy was 0.62 (95% confidence interval [CI]: 0.34-1.11; p=0.10 in the random effects model) with no significant heterogeneity or systematic bias. There was no significant difference in transient ischaemic attacks (TIA) (HR 0.77, 95% CI: 0.46-1.32; p=0.34) and no study-related deaths occurred. Pooling trials of the AMPLATZER PFO occluder device resulted in a significant reduction of stroke (HR 0.44, 95% CI: 0.20-0.95; p=0.04). Procedural success, new onset atrial fibrillation and cardiac thrombus were observed more frequently with the STARFlex compared with the AMPLATZER device. In observational studies, with high potential for baseline confounders, PFO closure was found to reduce the risk of recurrent stroke significantly (HR 0.23, 95% CI: 0.11-0.49; p<0.01 in the random effects model), with no significant effect on TIAs.
Conclusions: In RCTs, unlike observational studies, PFO closure compared with medical therapy failed to achieve a statistically significant reduction in recurrent stroke. However, pooling RCTs of the AMPLATZER PFO occluder device yielded a statistically significant reduction in stroke over medical treatment that may warrant further investigation.
Introduction
Approximately 800,000 people experience a stroke each year in the United States, and an additional 200,000 to 500,000 have a transient ischaemic attack (TIA)1. About 25-40% of these events are of undetermined nature, and are commonly termed cryptogenic strokes (CS)2. Patent foramen ovale (PFO), an anatomic variant of the atrial septum found in about 25% of the general population and 44-66% of patients with CS, has been identified as a potential pathway for paradoxical embolism of platelet aggregations, thrombi, gas bubbles or other particulate matter from the venous circulation to the brain, or even a place for embolic thrombus formation in situ1,2. Although several retrospective studies have suggested the implication of PFO in the pathogenesis of CS3-5, the latter is probably a multifactorial phenomenon, and prophylactic closure of an incidentally discovered PFO is therefore not recommended. In contrast, the risk of recurrent thromboembolism after CS has stimulated research for effective preventive therapies, including transcatheter PFO closure. At present, however, there is very little robust evidence on the efficacy of closure devices in the treatment of patients with CS, and the decision to close a PFO is left to clinical judgement.
Recently, randomised clinical trials have been published that compared PFO closure with medical therapy in patients with CS6-8. Because each of these trials missed the primary endpoint and generated hypotheses mostly based on secondary endpoints, there is a scientific and clinical rationale for increasing the statistical power beyond that of individual studies to address the important question of whether there are risk or benefit signals with PFO closure in patients with CS. Given this background, we performed a systematic review and comprehensive meta-analysis to examine the relative effectiveness of PFO closure compared with medical therapy in patients with CS randomised in clinical trials. To address the issue of entry and retention bias, which may be introduced in randomised studies by the preferences of investigators and patients, especially in a population where the chance of PFO being an innocent bystander is high, we also performed an updated meta-analysis of observational studies comparing PFO closure with medical therapy.
Methods
LITERATURE SEARCH
We searched MEDLINE, EMBASE and Cochrane databases for eligible studies, with no language restrictions. The terms used for research included “patent foramen ovale”, “PFO”, “atrial septal aneurysm”, “ASA transcatheter closure”, “heart septal defects”, “interatrial shunt,” “recurrent thromboembolism”, “recurrent stroke”, and “recurrent TIA”. Additional sources included www.clinicaltrials.gov, www.clinicaltrialresults.org, www.tctmd.com, www.cardiosource.com, Google Scholar and abstracts/presentations from major cardiovascular meetings. Furthermore, we searched the reference lists of relevant studies and reviews, editorials and letters.
We restricted our analysis to studies published or presented at a major cardiovascular meeting through March 2013 that met all the following inclusion criteria: 1) study population of patients with CS; 2) comparison of PFO closure vs. medical therapy; and 3) both efficacy and safety outcomes were reported with follow-up ≥1 year. Reviewers were not blinded to journal, author, or institution of publication. Because of the varying lengths of follow-up and varying baseline risk among studies, the measure of association derived was a relative risk rather than absolute risk or event rate. Hazard ratios (HR), odds ratios (OR) or relative risks with confidence intervals (CI) were either directly abstracted or derived on the basis of the reported event rates, as previously described9. The most updated or most inclusive data for a given study were chosen for abstraction.
DATA EXTRACTION
Studies were identified in two independent searches (performed by G.M. and L.V.) with conflicts adjudicated by a third reviewer (D.C.)6-8,11-21. Efficacy endpoints were death, non-fatal stroke and TIA. Safety endpoints were major bleeding, new onset atrial fibrillation and procedure or device-related cardiac thrombus.
STATISTICAL ANALYSIS
Separate analyses of RCTs and observational studies were pre-specified. The internal validity of the identified studies was appraised as per the Cochrane Collaboration recommendations10. Observational studies were found to be seriously limited by validity threats, including a moderate to high risk of selection, performance and attrition bias (Online Table 1). Because the degree of bias and heterogeneity among observational studies was frequently so significant as to suggest prudent interpretation, we refrained from incautious conclusions based on non-randomised studies and limited our inference and conclusions to the meta-analysis of RCTs. For completeness, characteristics of CS patients included in observational studies and quantitative syntheses of efficacy outcomes from non-randomised studies are mentioned here and reported more extensively in Online Table 2 and Online Table 3.
For RCTs, all studies meeting inclusion/exclusion criteria were included in the primary meta-analysis6-8. Data from RCTs were analysed according to the intention-to-treat principle. A separate analysis was run for trials investigating the STARFlex® septal closure system (NMT Medical Inc., Boston, MA, USA) and the AMPLATZER® PFO occluder (St. Jude Medical, St. Paul, MN, USA) device. For observational studies, all studies meeting inclusion/exclusion criteria were included in the primary meta-analysis11-21. For observational studies reporting unadjusted, adjusted, or propensity-matched data, the highest-quality estimate was picked for the overall meta-analysis (using the following rank order: propensity matched>adjusted>unadjusted). Additional sensitivity analyses confined to observational studies with ≥100 or ≥200 patients enrolled or follow-up extending to ≥3 or ≥4 years were pre-specified.
The results of all studies were combined using a random effects model to minimise heterogeneity between groups, and confirmed by a fixed effects model to avoid overweighting of small studies and to account for differential loss to follow-up in both treatment arms, if any22,23. Outcomes with zero events were included to provide a more conservative and generalisable estimate of the effect size. A two-tailed alpha of 5% was used for hypothesis testing. Statistical heterogeneity was assessed with Cochran’s Q via a chi-square test and quantified with the I2 test24. The influence of single studies was examined by excluding individual studies, and testing for systematic bias was performed using funnel plots and Begg’s test25. The statistical analysis was performed using Comprehensive Meta-Analysis V2.0 (Biostat, Englewood, NJ, USA).
Results
SEARCH RESULTS
Figure 1 illustrates the overall search strategy. A total of 2,407 potentially eligible studies were identified, 14 of which met the pre-specified inclusion criteria. Three studies were RCTs enrolling 2,303 patients and 11 were observational studies reporting data from 2,231 patients. Among RCTs, the CLOSURE I trial investigated the STARFlex septal closure system, while the RESPECT and PC trials evaluated the AMPLATZER PFO occluder device (Table 1). The primary endpoint definitions varied slightly across the studies, as well as regimen and dosage of antithrombotic drugs in the medical groups (Table 1). Aspirin with or without clopidogrel was the most common regimen used for thromboprophylaxis in the immediate post-PFO closure period.
Figure 1. Flow diagram of the study selection process. PFO: patent foramen ovale; RCT: randomised clinical trial
PROCEDURAL OUTCOMES AND DEVICE PERFORMANCE
Overall, the PFO closure procedure was successful in 93% of patients (95% CI: 91%-94%), but numerically lower rates of procedural success were noted with the STARFlex device in the CLOSURE I trial (89%) compared with the AMPLATZER device in the pooled weighted analysis of the RESPECT and PC trials (96%; 95% CI: 94%-97%). At six months, effective closure (defined as procedural success with a grade 0 or 1 residual shunt) was in total documented in 90% of patients (95% CI: 0.88%-0.92%) and occurred in 86% of patients treated with the STARFlex device and 94% (95% CI: 92%-96%) of patients treated with the AMPLATZER device. Complete closure (defined as procedural success with a grade 0 residual shunt) was reported at six months in the two trials using the AMPLATZER device, being estimated at 76% (95% CI: 0.73-0.80).
DEATH
No death at 30 days or death from neurological causes beyond 30 days had occurred in any of the trials. Cumulatively, seven deaths (three cardiovascular, four non-cardiovascular) in the device group and six (all non-cardiovascular) in the medical therapy group occurred post randomisation (p=NS). All these events were adjudicated as non-study-related.
RECURRENT STROKE
The stroke HR for PFO closure versus medical therapy was 0.62 (95% CI: 0.34-1.11; p=0.10) in the random effects and 0.62 (95% CI: 0.36-1.08; p=0.09) in the fixed effects model with no significant heterogeneity (I2=8%, p=0.34) (Figure 2). In addition to the protocol definition, the PC trial also reported on stroke by using a more contemporary endpoint definition, similar to that used in the RESPECT trial. However, adopting a similar stroke definition in the PC and RESPECT did not significantly alter the point estimate nor did it achieve formal statistical significance (HR 0.60, 95% CI: 0.35-1.03; p=0.06). There was a trend towards systematic bias across the trials (Begg’s test, p=0.12). The stroke estimate became statistically significant in the pooled analysis restricted to the RESPECT and PC trials using the AMPLATZER PFO occluder device (HR 0.44, 95% CI: 0.20-0.95; p=0.04) (Table 2).
Figure 2. Meta-analyses of recurrent stroke (A) or transient ischaemic attack (TIA) (B) with PFO closure vs. medical therapy in randomised clinical trials. Boxes represent the individual study estimates; lines, 95% confidence intervals.
RECURRENT TRANSIENT ISCHAEMIC ATTACKS
The TIA HR for PFO closure versus medical therapy was 0.77 (95% CI: 0.46-1.32; p=0.34) in both the random effects and the fixed effects models with no significant heterogeneity (I2=0%; p=0.95) or systematic bias apparent across the studies (Begg’s test, p=0.60) (Figure 2). No individual study unduly influenced this outcome, which remained not significant in analyses restricted to the STARFlex or AMPLATZER devices (Table 2).
ADVERSE SAFETY EVENTS
Study-related major bleeding episodes were reported in 1.2% and 0.7% of patients in the PFO closure and medical therapy arms, respectively (OR 1.67, 95% CI: 0.44-6.30 in the random effects model; p=0.45). New onset atrial fibrillation occurred more frequently among patients undergoing PFO closure than those on medical therapy (4% vs. 1%, OR 3.72, 95% CI: 1.41-9.79 in the random effects model; p=0.01), with larger point estimates noted for the STARFlex device (OR 9.20, 95% CI: 2.7-30.9; p<0.001) compared with the AMPLATZER device (OR 2.25, 95% CI: 1.03-4.94; p=0.04). A procedure or device-related cardiac thrombus was detected at six-month transoesophageal echocardiography in 0.7% (95% CI: 0.3%-1.5%) of patients undergoing PFO closure, more commonly with the STARFlex device (1.1%) than with the AMPLATZER device (0.4%, 95% CI: 0.1%-1.2%).
PFO CLOSURE IN OBSERVATIONAL STUDIES
Details of observational studies are reported in Online Table 2. Quantitative syntheses of efficacy outcome data of PFO closure compared with medical therapy in observational studies are reported in Online Table 3 and Online Figure 1. Briefly, PFO closure was found to reduce the risk of recurrent stroke significantly (0.23, 95% CI: 0.11-0.49; p<0.01 in the random effects model), with no significant heterogeneity or apparent systematic bias (Begg’s test, p=0.42). In contrast, there was no significant effect of PFO closure in reducing death or recurrent TIA.
Discussion
The primary findings from this systematic review and meta-analysis of PFO closure versus medical therapy are the following. First, in RCTs, PFO closure showed a nominal 38% reduction in the hazard of recurrent stroke, but this reduction did not achieve formal statistical significance. Conversely, a significant reduction in recurrent stroke was observed after exclusion of the CLOSURE I trial with the STARFlex device, by pooling data from trials of the AMPLATZER PFO occluder device. Second, although in observational studies the use of PFO closure versus medical therapy was associated with a significant 77% reduction in stroke, most of these comparisons are fraught with validity threats and a high level of heterogeneity. Overall, the collective data from RCTs and observational studies appear consistent in indicating at a minimum that PFO closure is widely adopted into clinical practice despite the lack of a conclusive evidence base.
In a previous meta-analysis of 10 observational studies from Agarwal et al, PFO closure was associated with a significant 75% reduction in recurrent neurological events compared to medical therapy26. Lack of randomisation in the studies included in that meta-analysis resulted in poor internal validity, with a high chance for strong selection, attrition and performance bias in most of them. Reliance on registries can lead to incorrect conclusions due to the influence of unassessed confounding variables, while randomisation may provide a more unbiased estimation of the effects of a treatment. Notably, another meta-analysis from Kitsios et al included the CLOSURE I trial but results from the PC and RESPECT trials were not available at the time of publication27. By including all the available RCTs, our meta-analysis represents a contemporary synthesis of randomised data on the topic of PFO closure. We also provided an updated meta-analysis of observational studies to account for the entry and retention bias typical of PFO trials (e.g., closure devices are used off-label and high-risk patients are preferentially treated outside of the trial) and provide the reader with a full contemporary picture of PFO closure in the experimental setting and in the real world. This included studies that were not pooled in the previous meta-analyses, including a recent propensity-matched study from Wahl et al21. The study from van de Wyngaert et al differed from the other observational studies in that it compared retrospective data on stroke incidence for a group of patients that subsequently underwent PFO closure with prospective data on recurrent stroke in the same population16. Notably, the exclusion of each of these studies did not significantly alter the summary estimates.
CLOSURE I, RESPECT and PC were designed as superiority trials, and were individually powered for a composite rate of death, stroke and/or TIA6-8. There were some differences among the trials with respect to study design, population included and device tested. The follow-up period was longer in RESPECT and PC compared with CLOSURE I, and RESPECT did not randomise patients who had only a TIA or a lacunar stroke. All trials missed their primary endpoint, but there were directional trends towards the reduction of secondary endpoints, including stroke, with RESPECT suggesting a benefit when evaluating the “as-treated” cohorts. Notably, in the PFO closure arm of the RESPECT, three patients with recurrent ischaemic stroke did not have a device in place at the time of the event. It should be noted that RESPECT and PC suggested a benefit of PFO closure only after two years and with a modest absolute reduction. However, a long-lasting protective benefit of PFO closure cannot be underestimated, since patients <60 years, like those included in RCTs, are at risk over an extended period of time. Longer follow-up is therefore required for this kind of evaluation.
Because recurrent stroke is an unlikely occurrence in patients with CS, and because the number of events in each trial was small, a meta-analysis of the three RCTs has the potential to increase the individual power of each individual study to ascertain if a true treatment effect exists with PFO closure over medical therapy. Their limitations notwithstanding, RCTs and meta-analyses remain the most informative setting and the best available evidence for decision making in a field of great importance but plenty of uncertainties. While there were signals of benefit with observational studies in reducing stroke, this was not incontrovertibly demonstrated by each RCT and the present quantitative synthesis of 2,303 randomised subjects.
A stratified analysis suggested that pooling RCTs of the AMPLATZER PFO occluder device yields a borderline significant reduction in stroke. This may be explained by the finding that in the CLOSURE I trial the rate of effective PFO closure with the STARFlex device was appreciably lower than in the RESPECT and PC trials with the AMPLATZER device combined (86% vs. 96%). In addition, the two devices may differ in terms of thrombogenicity28,29, and new onset atrial fibrillation episodes were more commonly detected in the CLOSURE I (6%)6-8. Our findings therefore support the understanding that PFO closure devices may differ at least in terms of safety, with the AMPLATZER device associated with a low number of adverse events in both the RESPECT and PC trials, a notion that should be taken into account when interpreting results from contemporary studies or planning further clinical testing in the field. In this regard, a patient-level analysis of the RESPECT and PC trials would be the ideal setting to elucidate the safety and efficacy profile of the AMPLATZER PFO occluder in patients with CS. Because the available RCTs did not encompass a homogeneous population (Table 1), pooling the available evidence at the patient level is also warranted to allow defining the clinical niche that is more likely to benefit from the device. For example, in the RESPECT trial, patients with substantial right-to-left shunts seemed more likely to derive a benefit from PFO closure, a clinically plausible finding that deserves further investigation. Differences in the composition of the primary endpoints of each trial (e.g., the primary endpoint of the RESPECT trial did not include TIA, and including all-cause death in the primary endpoint of the PC trial was not specific of the studied condition) raised a note of caution and prevented us from combining subgroups of different RCTs at the study level.
While other trials are ongoing (NCT00738894, NCT00562289), current challenges of trial design in PFO closure include low event rate (<2% recurrent stroke/year), need for very long follow-up to accumulate events, concurrent off-label implantation (which has significantly limited the ability for PFO closure devices to be studied in RCTs), and the understanding that, in a large proportion of patients, despite having a PFO and a CS, other mechanisms might come into play. Clinical clues to paradoxical embolism that have to be taken into account when linking PFO and recurrent stroke include a Valsalva manoeuvre or dyspnoea at the onset, a previous history of deep vein thrombosis, pulmonary embolism or varicose veins, a history of sleep apnoea, and a history of prolonged sitting, such as on a long car ride, airplane ride, or working for long hours at a desk30. A set of criteria for identifying PFO-related stroke, such as the absence of vascular risk factors like hypertension, diabetes, smoking and older age, is currently under investigation in the Risk of Paradoxical Embolism (RoPE) study to identify those patients who may have the greatest potential benefit from PFO closure31.
Limitations
The conclusions drawn from this meta-analysis are obviously subject to the limitations and differences of the original included studies themselves. Variation in study design, endpoint definitions and publication bias are limitations of all meta-analyses. In addition, studies of PFO closure versus medical therapy have small event rates, short follow-up periods, and they might differ in the entry definition of stroke. In contrast, unblinded and imbalanced referral for adjudication of endpoints may have favoured the closure device arms in some cases.
In the RESPECT trial, difference in the dropout rate between the medical therapy group and the device group resulted in unequal duration of exposure to the risk of recurrence. To account for the varying lengths of follow-up, we preferentially pooled relative measures of association rather than absolute risks or crude event rates. It should also be emphasised that the low stroke rate observed in both the closure and medical arms of the individual studies and the present meta-analysis may explain the observed lack of statistically significant difference, despite the evidence of a consistent numerical reduction, as the reflection of a type II error. Since the long-term advantage of PFO closure may accrue over time, a fair assessment versus medical therapy is likely to require longer follow-up than in the present study.
Finally, the summary main effect of a meta-analysis can depend on post hoc, data-driven design choices. To minimise this risk, we reported on the STARFlex and AMPLATZER devices combined and separately, and chose individual outcome measures, rather than heterogeneous combined endpoints, for the primary analysis. That being said, the included studies represent an inclusive up-to-date attempt to pool the available literature in this field and to derive summary-level estimates. The inclusion of systematic data aggregation from the available RCTs meaningfully extends and complements the findings from previous meta-analyses, which relied solely on observational studies or incorporated only one RCT26,27.
Conclusions
Despite the signal of a 38% numerical reduction in stroke, PFO closure compared with medical therapy failed to achieve a statistically significant reduction in stroke and TIA in the individual and pooled analysis of three available RCTs. However, pooling trials of the AMPLATZER PFO occluder device displayed a superior safety profile than that shown in the only available trial of the STARFlex device, and yielded a statistically significant reduction in stroke over medical treatment that may warrant further investigation. The observed benefit of PFO closure coming from a pooled analysis of the most updated observational literature is probably biased by inherent limitations of studies including unadjusted data. Large-scale RCTs with long-term follow-up and patient-level pooling are required to ascertain the true effects of PFO closure compared with medical therapy with respect to the occurrence of stroke and/or TIA, and eventually to define the best clinical niche for PFO closure.
Conflict of interest statement
The authors have no conflicts of interest to declare.
Online Figure 1. Meta-analyses of recurrent stroke (A) and TIA (B) with PFO closure vs. medical therapy in observational studies. Boxes represent the individual study estimates; lines, 95% confidence intervals.
