VARC endpoint definition compliance rates in contemporary transcatheter aortic valve implantation studies

Abstract

Aims: The Valve Academic Research Consortium (VARC) endpoint definitions were established to standardise the reporting of clinical outcomes following transcatheter aortic valve implantation (TAVI). It remains unclear, however, to what extent and in which manner these definitions are applied. Therefore, we sought to investigate the utilisation and adherence to VARC guidelines since their introduction in 2011 across peer-reviewed TAVI-related publications.

Methods and results: We performed a systematic literature review to identify TAVI-related manuscripts published between February 2011 and February 2014. Manuscripts were categorised into three groups: a “compliant” group of manuscripts using only VARC-defined endpoints, a “non-compliant” group of manuscripts with only non-VARC-defined endpoints, and a “mixed compliant” group of manuscripts with both VARC- and non-VARC-defined endpoints. Multivariate analyses were performed to identify predictors of VARC use. Among 5,023 published manuscripts, 498 were included in the final analysis. At least one VARC definition was used in 275 (54%), while 223 (43%) did not use any VARC definitions. After publication of the first VARC manuscript (VARC-1, January 2011), VARC use increased from 31% (n=15) at six months to 69% (n=84) at 36 months. Following the publication of VARC-2 (October 2012), VARC-1 use declined (from 58% [n=47] to 36% [n=24]), while VARC-2 use increased from 4% (n=3) at six months to 35% (n=23) at 18 months. Of the manuscripts using VARC, 49 (10%) were classified as compliant and 226 (46%) as mixed compliant. The following endpoints were more often defined using VARC vs. non-VARC: myocardial infarction (64% vs. 36%); stroke (56% vs. 44%); bleeding (79% vs. 21%); vascular complications (70% vs. 30%); acute kidney injury (63% vs. 37%); reintervention (67% vs. 33%); and composite endpoints (52% vs. 48%). Mortality, valve dysfunction, TAVI-related complications, and quality of life were more often defined using non-VARC criteria.

Conclusions: Implementation of VARC criteria in peer-reviewed manuscripts has increased over time. There remain, however, a considerable number (43%) of publications that do not report outcomes according to VARC. These data will inform the future development of VARC criteria.

Introduction

The Valve Academic Research Consortium (VARC) was established to select and standardise the reporting of clinical endpoint definitions for research in transcatheter aortic valve implantation (TAVI)1. First published in January 2011 (VARC-1)2, the VARC criteria provide a framework to evaluate and compare clinical outcomes uniformly among patients undergoing TAVI or surgical aortic valve replacement (SAVR). These criteria were updated in October 2012 (VARC-2)3, to reflect contemporary standards and guidelines. The adoption and appropriate use of these criteria are understood to be a key component in the establishment of a robust evidence base in the ongoing evaluation of transcatheter heart valve therapies.

Although some evidence suggests that VARC criteria have been adopted in peer-reviewed TAVI-related publications4, the extent of this adoption and appropriate use of these recommendations have not been formally evaluated. Moreover, it remains unclear whether the VARC-2 criteria have indeed replaced the original VARC-1 criteria. Thus, we sought to investigate the utilisation and adherence to VARC guidelines since their introduction in 2011 across peer-reviewed TAVI-related publications.

Methods

LITERATURE SEARCH

All relevant English language TAVI-related manuscripts published between February 2011 and February 2014 were systematically searched in BioMed Central, Google Scholar, and PubMed. This search was prepared in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines5. The following list of keywords was used: transcatheter aortic valve implantation, transcatheter aortic valve replacement, transcatheter heart valve, TAVI, TAVR, heart valve prosthesis implantation, aortic stenosis, transfemoral, transaortic, and transapical. Additional manuscripts were sought by a manual search of secondary sources, including references from primary papers and related articles. The title or abstract was first independently screened by two reviewers (M. Erlebach and D. Mylotte), and then all potentially suitable manuscripts were examined in detail. Meta-analyses, review articles, case reports, and small case series (n<20) were excluded. All relevant manuscripts were then examined in detail for the terms “Valve Academic Research Consortium” or VARC. Definitions according to the first VARC publication are further referred to as “VARC-1”2, while definitions according to the second VARC publication are referred to as “VARC-2”3.

Definitions

All manuscripts were first categorised according to the primary subject of interest, as defined in the study aims or objectives. In cases where the subject of interest was not clearly stated, the primary subject of interest was derived from the title and reported outcomes. Uncertainties in classification were resolved by consensus. Eight different categories were identified. These were: general outcome; imaging (including echocardiography, CT and MRI); acute kidney injury; stroke; vascular complications; bleeding complications; risk assessment; and other specific outcomes, including anticoagulation, haemodynamics, blood parameters, infection, in-hospital or intraprocedural complications, and valve sizing.

For each included manuscript, all reported endpoints were individually examined and classified as either VARC-defined or non-VARC-defined. All endpoints and their subcategories described in the VARC criteria were assessed: mortality; myocardial infarction (MI); stroke; bleeding; acute kidney injury; vascular/access-site complications; valve dysfunction; other TAVI-related complications; composite endpoints; and other endpoints (including endpoints suggested in VARC, such as NYHA classification, quality of life, reintervention or rehospitalisation as well as endpoints not contained in VARC). Classification of a VARC endpoint required an explicit mention of the VARC criteria or a citation of the VARC manuscripts in the study methodology or labelling of graphs or tables. In cases where only specific endpoints were labelled as being defined by VARC (rather than all endpoints), other endpoints were assumed to be non-VARC. If the definition of a VARC endpoint was not consistent with the definition provided in either the original or updated VARC manuscripts, depending on the timing of the publication, this discrepancy was noted and the endpoint was labelled as non-VARC.

Manuscripts were then categorised according to their VARC usage: 1) a “compliant” group consisting of manuscripts with only VARC-defined endpoints; 2) a “non-compliant” group consisting of manuscripts with only non-VARC-defined endpoints; and 3) a “mixed compliant” group consisting of manuscripts with both VARC- and non-VARC-defined endpoints. Manuscripts that reported specific endpoints not included in or defined by VARC were excluded from further analyses.

Additional data extracted from each manuscript included author affiliation with the VARC guidelines, scientific journal, number of patients in the study, publication date, and whether VARC-1 or VARC-2 was reported.

Statistical analysis

Discrete data are presented using counts and percentages, and differences were assessed using the chi-square or Fisher’s exact test, where appropriate. Continuous data are reported as mean±standard deviation (SD) and compared using the Student’s t-test. Logistic regression analyses were performed to determine the impact of study, manuscript, and journal characteristics on the use of VARC. The following variables were entered: author VARC association, multicentre study, prospective study, number of patients in the study, study topic (general outcome, stroke, bleeding/vascular/kidney complications, imaging, risk assessment, conduction, and other), and journal impact factor. A backward Wald multivariate model was built with an entry and exit p-value of 0.10. A two-sided p-value of <0.05 was considered to be statistically significant. All analyses were performed using SPSS, Version 21 (IBM Corp., Armonk, NY, USA).

Results

MANUSCRIPTS

The literature search yielded 5,023 published manuscripts. After exclusion of duplicates, reviews, meta-analyses, case reports, and small case series, data on endpoints were extracted from 514 original studies (Figure 1). Among them, 264 were published before the VARC-2 publication and 250 after the VARC-2 publication. Sixteen studies (3.1%) were excluded because they reported endpoints not included in VARC: cost-effectiveness (n=8), specialised echocardiographic parameters (n=6), and alternative imaging (n=2). Among the 498 remaining publications that were included in the current analysis, 275 (55.2%) used a VARC definition for at least one endpoint, while 223 (44.8%) did not use any VARC definition.

Figure 1. Flow diagram for systematic inclusion of studies. VARC: Valve Academic Research Consortium

VARC EVOLUTION

Of the 275 publications reporting VARC, 229 (83.3%) reported VARC-1 definitions, 44 (16.0%) reported VARC-2 definitions, and two (0.7%) reported both VARC-1 and VARC-2.

After the first VARC publication in January 2011, VARC use increased from 30% (n=15) in the first six months to 69% (n=84) at three years (Figure 2A). After the publication of VARC-2, VARC-1 use declined (from 58% [n=47] to 36% [n=24]), while VARC-2 use increased from 4% (n=3) at six months to 35% (n=23) at 18 months (Figure 2B). After the publication of VARC-2, VARC use increased from 62% (n=50) during the first six months to 74% (n=49) after 12-18 months.

Figure 2. VARC usage after publication of the VARC-1 (A) and VARC-2 (B) definitions.

STUDY CHARACTERISTICS AND VARC COMPLIANCE

The majority of published studies were single-centre investigations (n=359, 72%), with a mean sample size of 341±805 patients. Among these studies, the research objective was most commonly a description of overall patient outcomes (n=244, 49%). Specific outcome reporting (n=96, 19%) and imaging (n=61, 12%) were also commonly reported (Table 1).

Overall, 49 (9.8%) publications were classified as VARC-compliant, 223 (44.8%) as VARC non-compliant, and 226 (45.4%) as mixed compliant. The mixed compliant group consisted of 96 publications reporting both VARC and non-VARC definitions for one endpoint, and 130 publications that used VARC definitions for only a proportion of endpoints.

ENDPOINTS

The most commonly reported endpoint was mortality (n=423, 85%), followed by TAVI-related complications (n=366, 73.5%), and stroke (n=312, 62.7%) (Table 2). The least commonly reported endpoint was quality of life (n=22, 4.4%), followed by rate of rehospitalisation (n=52, 10.4%), and reintervention rate (n=61, 12.2%). Authors more commonly used VARC than non-VARC definitions for the following endpoints: MI (64% vs. 36%), stroke (56% vs. 44%), bleeding complications (79% vs. 21%), vascular complications (70% vs. 30%), acute kidney injury (63% vs. 37%), reintervention (67% vs. 33%), and composite endpoints (52% vs. 48%). Non-VARC definitions were more often used for mortality, valve dysfunction, TAVI-related complications, NYHA classification and quality of life.

MULTIVARIATE REGRESSION ANALYSIS

Several independent predictors of VARC use were identified (Table 3). Studies with a focus on bleeding, vascular or kidney complications were strongly associated with VARC use (OR 4.92, 95% CI: 1.86-13.00), while imaging and conduction studies were inversely predictive of VARC use (OR 0.10 and OR 0.18, respectively). A higher journal impact factor was also identified as an independent predictor of VARC use (OR 1.09, 95% CI: 1.04-1.14). Interestingly, a prospective study design was inversely correlated with VARC adoption (OR 0.56, 95% CI: 0.36-0.88).

Discussion

Since the introduction of the VARC criteria in 2011, an increasing number of studies report outcomes according to VARC definitions. Nonetheless, almost half (44%) of all manuscripts in our study did not use the VARC criteria or used the VARC definitions selectively (46%). Indeed, 10% of publications reported all outcomes according to VARC. While VARC use was associated with articles published in higher impact medical journals, the rare use of a number of VARC endpoint definitions suggests that the VARC criteria require careful revision to increase adoption further.

SPECIFIC ENDPOINTS

Several VARC definitions have been widely accepted and are frequently reported in the literature. Bleeding and vascular complications, stroke, MI, and acute kidney injury are among the most widely applied definitions. In contrast, VARC definitions of mortality, valve dysfunction, TAVI-related complications and quality of life are used less frequently in the literature. While one may argue that an endpoint such as mortality does not require a specific definition, the definition of cardiac or non-cardiac mortality and the specific time points suggested in VARC do have important implications for comparing or pooling results from different studies. Similarly, the limited reporting of VARC-defined TAVI-related complications is probably because authors believe that these complications do not require a specific definition. Valve dysfunction, on the other hand, seems to be an endpoint that has, to date, not been widely used. Reporting of this endpoint requires specific and substantial data input, and may therefore not be feasible for many centres with fewer resources. It is also possible that the community remains sceptical regarding the proposed definition. This finding identifies a potential area for further improvement, either by increasing the awareness of this complication amongst the TAVI community, by reinforcing the need for systematic reporting of such a complication, or by future iteration of the current VARC definition.

VARC ADOPTION

The application of the VARC criteria was discernible within months of the initial publication, 31% at six months. As expected, adoption of the VARC criteria increased over time and, most recently, three quarters (74%) of published manuscripts on TAVI used VARC definitions. While these most recent data are reassuring, further improvements and refinements in the scope and clarity of endpoint definitions may increase VARC adoption. Simplification of echocardiographic assessment, clearer criteria for conduction disorders and other TAVI-related complications, and perhaps a consensus concerning endpoint definition may lead to a further increase in VARC adoption.

FACTORS INFLUENCING VARC USE

We identified several factors independently influencing VARC usage. Studies on conduction disorders and imaging were inversely associated with VARC use. It is likely that adoption in these specific areas may have been affected by chronology: these subjects became areas of considerable research after the publication of VARC-2 and thus may not be adequately addressed in the published document. This implies that updated VARC definitions may be required to address fully the requirement of research into these subjects of interest. Indeed, this is particularly true for imaging studies detailing the application of a variety of imaging modalities on important subjects such as paravalvular leak and valve dysfunction. Moreover, the absence of VARC definitions for appropriate or inappropriate TAVI sizing is likely to limit the application of the criteria in this context.

The observation that VARC use was inversely related to prospective study design was unexpected. It is likely, however, that this finding is explained by the fact that VARC definitions were not available at the time when these studies were designed. Retrospective study design is clearly more adaptable to the introduction of new endpoint definitions. Recent prospective studies have appropriately reported outcomes according to VARC6.

VARC was more often used in journals with higher impact factors. This finding suggests that study groups producing high-quality research have already adopted and complied with VARC recommendations. Similarly, higher impact journals are increasingly emphasising the need for standardised endpoints, and consider VARC usage as a sign of a well-designed and appropriately conducted study. The association of studies reporting vascular complications, bleeding complications, or acute kidney injury with increased VARC use demonstrates the extensive applicability of these definitions.

FUTURE IMPLICATIONS

It remains unclear why specific VARC definitions are not systematically reported, particularly when other outcomes in the same publication are reported according to VARC (mixed compliant group). As we continue to develop new valves, case numbers increase and the potential patient pool is enlarged, certain aspects of the TAVI procedure, such as paravalvular leak, valve sizing and conduction disturbances, become more important for ensuring a satisfactory outcome. Standardised endpoints allow study result comparison and support further progress in TAVI procedures.

The need for new endpoints and the improvement of already existing endpoints was already implemented in the VARC-2 criteria. Echocardiographic definitions were elaborated, quality of life assessments were included, endpoint definitions were updated and composite endpoints were modified and recommended. The understanding of the present usage and limitations of the current VARC definitions may allow the VARC to improve endpoint definitions further and optimise adherence in updated VARC-3 endpoints. As these are already in progress, it will be interesting to assess the development in the criteria. Perhaps simplification and elaboration of currently not well used criteria will increase their application in future.

STUDY LIMITATIONS

In this analysis, publications were only considered as VARC-compliant when there was a specific mention of VARC, which may have negatively influence the rate of VARC-compliant publications. Some publications will have used VARC definitions without mentioning VARC, and therefore the actual rate of VARC compliance may be higher. Due to the nature of this study, certain results may be overemphasised. As endpoint definitions were often not clearly stated or merely explicitly mentioned for only a few endpoints, there is the possibility that the intention of the author was not correctly understood. This again mirrors the need for clear statement of definition usage and the need for standardisation.

Conclusions

Implementation of the VARC criteria in peer-reviewed publications has increased over time. However, in our study there remained a considerable number (44%) of publications that did not report outcomes according to the VARC criteria, and several endpoints in particular lacked adoption of VARC criteria. These findings should be taken into account when developing future iterations of the VARC criteria.

Guest Editor

This paper was guest edited by Alec Vahanian, MD, PhD, Département de Cardiologie, Hôpital Xavier Bichat, Faculté Paris Diderot, DHU FIRE, Paris, France.

Conflict of interest statement

The Guest Editor has carried out low-level consultancy work for Medtronic. D. Mylotte is a proctor and consultant for Medtronic and MicroPort. P. Genereux has received speaker fees from Abbott Vascular and a research grant from Boston Scientific, is a consultant for and has received speaker fees and a research grant from Cardiovascular Systems Inc., is a consultant and proctor and has received speaker fees from Edwards Lifesciences, and is a consultant for Saranas and Pi-Cardia. S. Windecker receives research grants from Abbott, Biotronik, Biosensors, Medtronic, Edwards Lifesciences, and St. Jude Medical. R. Lange is a consultant for and on the scientific advisory board of HighLife Medical, and is a consultant for and on the scientific advisory board of Medtronic. N. Piazza is a consultant for HighLife Medical, and is a proctor and consultant for and on the scientific advisory board of Medtronic. The other authors have no conflicts of interest to declare.