Editorial Comment

doi:10.1016/j.e

60

Appropriateness Criteria for Echocardiography: AnImportant Step Toward Improving Quality

R. Parker Ward, MD, Chicago, Illinois

The dramatic increase in the use of cardiovascular diagnostic imagingover the last decade is undeniable. Diagnostic imaging servicesreimbursed under Medicare’s physician fee schedule have grownmore rapidly than any other type of physician service.1 Although anaging population with an increasing burden of cardiovascular diseasecertainly contributes to this increase, the current dearth of clearevidence that cardiovascular imaging improves patient outcomes andthe marked geographic variation in imaging use have fostered con-cerns about potential overuse of cardiac imaging services.2,3 Notsurprisingly, echocardiography procedures are a major contributor tothe totality of cardiac imaging, accounting for approximately half ofall cardiac imaging services among Medicare beneficiaries.4 Althoughrecent increases in the use of echocardiography seem to be less thanother cardiac imaging modalities (echocardiography accounted for57% of Medicare expenditures for all cardiac imaging in 1999,decreasing to 48% in 2004),4 it remains clear that guidance is neededfor physicians and reimbursement agencies to help determine arational approach to the use of diagnostic imaging in the delivery ofhigh quality care.

In an effort to address these issues, the American College ofCardiology Foundation, in conjunction with imaging subspecialtysocieties, has published Appropriateness Criteria (AC) for a variety ofimaging modalities, now including the American College of Cardiol-ogy Foundation/American Society of Echocardiography AC forTransthoracic and Transesophageal Echocardiography.5 As withother AC documents, the development of the AC for echocardiog-raphy used a standardized methodology, combining available evi-dence with expert opinion, to identify common indications forechocardiographic procedures and to determine their level of appro-priateness. The indications in the AC for echocardiography are“purposefully broad” and attempt to “identify common scenariosencompassing the majority of clinical practice.”5 The authors cautionthat the AC should not be “considered substitutes for sound clinicaljudgment or practice experience.”5 Implicit in this methodology isthat these criteria will need to be tested in clinical practice, both toidentify potential areas for revision and to help define how theymightbe best implemented to maximize impact on the delivery of highquality medical care.

In this issue of the Journal of the American Society of Echocardiogra-phy, 2 important new studies evaluate the application of the AC forechocardiography on the current clinical practice of transthoracicechocardiography (TTE). Martin and Picard6 report on the applica-

Editorial Comments published in the Journal of the American Society of Echo-cardiography (JASE) reflect the opinions of their author(s), and do not necessarilyrepresent the views of JASE, its editors, or the American Society of Echocardi-ography.From the Noninvasive Imaging Laboratories, Section of Cardiology, Departmentof Medicine, University of Chicago Hospitals, Chicago, Illinois.

Reprint requests: R. Parker Ward, MD, Associate Professor of Medicine, Univer-sity of Chicago, 5841 S. Maryland Avenue, MC 6080, Chicago, IL 60637 (E-mail:pward@medicine.bsd.uchicago.edu).

0894-7317/$36.00

cho.2008.11.016

tion of these criteria on inpatient use of echocardiography on anacademic general medical service, whereas Kirkpatrick and col-leagues7 study these criteria in the context of an academic outpatientTTE practice. These studies, in addition to a recently reportedexperience of both inpatients and outpatients from our academicpractice,8 illustrate the many strengths of the AC for echocardiogra-phy, suggesting these criteria represent an important first step towarddefining the rational use of echocardiography in the delivery ofhigh-quality care. However, these reports also illustrate a number oflimitations, suggesting revisions to this document will be required.

COMMON THEMES

It is most striking that all 3 currently published studies of the AC forechocardiography find remarkably similar rates of inappropriate andappropriate transthoracic echocardiograms (TTEs). Among thoseTTEs in each study with an indication addressed in the AC forechocardiography (thus removing unclassifiable patients), inappropri-ate studies accounted for 11% to 13%, whereas 87% to 89% ofstudies were found to be appropriate.6-8 Although this may simplyreflect similarities among these 3 academic practices, it does suggestsome consistency in the application of the AC to assess global rates ofappropriateness. In addition, it suggests that referral for echocardiog-raphy for inappropriate indications (as currently identified in the AC)is relatively uncommon in these practice settings (Table 1).

IS CLINICAL APPLICATION OF THE APPROPRIATENESSCRITERIA FEASIBLE?

These 3 reports also illustrate the feasibility of the application of theAC for echocardiography, a critical finding because these documentswill inevitably be used for evaluations of health care delivery and forreimbursement determinations. What is clear, however, is that thisprocess is not without challenges. For example, each of these studiesnotes that the application of the AC requires subjectivity, and eachidentifies a number of clinically ordered TTEs with indications thatcould reasonably be ascribed to more than one of the indicationslisted in the AC. As might be expected, methodologic differencesbetween the studies seem to affect the implications of this subjectiv-ity. Two of these studies used a single investigator for determinationof indication and level of appropriateness,6,7 and one study used theconsensus of 2 investigators.8 Although interobserver variability wasreported to be modest in a small post hoc sample by Kirkpatrick andcolleagues,7 prospective indication determination in a large sampleby 2 independent investigators was found to be excellent, particularlyfor overall level of appropriateness.8 In sum, although challenges existwith applying the AC for echocardiography, present data suggest thatusing reasonable assumptions, these criteria can be feasibly applied toclinical practice. This seems particularly true for determination ofoverall rates of appropriateness in a particular clinical practice orimaging laboratory. However, if these criteria are to gain widespreaduse as either a gateway for performance of an individual TTE study orfor reimbursement determinations, it will be critical that the subjec-

tivity in the process is recognized and flexibility incorporated.

Journal of the American Society of Echocardiography 61Volume 22 Number 1

NONCLASSIFIABLE STUDIES: WHAT CAN WE LEARN?

Although the preceding analysis of overall levels of appropriateness inthe recent studies seems favorable, it neglects TTEs with indicationsthat are nonclassifiable or not addressed by the AC for echocardiog-raphy; a group that may well add to the number of inappropriatestudies. This group also provides the most fertile ground for identify-ing potential revisions to the AC for echocardiography. On thesurface, the 3 recent reports demonstrate a wide range in the numberof TTEs with nonclassifiable or not addressed indications (2%-35%).6-8 Although variability is expected given the different method-ologies and assumptions used, closer inspection reveals some similar-ities. For example, only a small fraction of inpatient TTEs arenonclassifiable by the AC for echocardiography (2%6, 6%8), whereasa larger portion of outpatient TTEs (35%7, 15%8) cannot be classi-fied. This might be expected given that many of the indications in theAC for echocardiography specifically address symptoms or a “changein clinical status,”5 which are the norm among patients admitted tothe hospital. However, it also likely reflects less available documen-tation for accurate classification of a study’s true indication in theoutpatient setting. What is clear is that a selective focus on theoutpatient practice of echocardiography will be most useful as thesecriteria are refined in an effort to more fully encompass the broadclinical practice of echocardiography.

More specifically, these studies identify some common individualindications for echocardiography that are not clearly addressed in thecurrent AC. For example, the most common indication not addressedin the AC for echocardiography in 2 of the recent studies waspreoperative evaluation before noncardiac surgery.7,8 Other com-mon indications that were not classifiable included valvular heartdisease and heart failure with a severity or frequency of follow-up thatdid not fit within the AC indications as written.7,8 It should be notedthat all 3 recent reports come from large academic tertiary carecenters (Massachusetts General Hospital,6 University of Pennsylva-nia,7 University of Chicago8), which likely affects the rates of non-classifiable studies and limits the overall applicability of these results.For example, a significant fraction of TTEs with indications related toniche programs, such as atrial fibrillation ablation follow-up7 orsolid-organ transplant programs,8 were identified. Although the ACdo not intend to address all possible indications for echocardiography,it is clear is that a few additional common indications will be needed

Table 1 Summary points

● Recent studies of the AC for TTE suggest current referral rates forinappropriate studies are not high (11%-13%)6-8

● The application of the AC in clinical practice is feasible, particularlyfor assessing overall rates of appropriateness with a clinicalpractice or imaging laboratory.

● Given limitations identified in application of the AC, caution isadvised before widespread application of the AC as a gateway toperformance of individual TTE studies or for individual TTEreimbursement determinations.

● Revisions to the AC will be needed to add common indications forTTE that are not currently addressed and to emphasize thatappropriateness of a study implies that it will change or clarifyclinical management or decision making.

● Additional research in a variety of practice settings is needed toassess fully the ability of the AC to encompass the broad clinicalpractice of TTE.

AC, Appropriateness Criteria; TTE, transthoracic echocardiography.

to better encompass the broad practice of echocardiography. Further

research in a variety of practice settings will be critical to help identifythese additional indications.

ARE ALL INAPPROPRIATE STUDIES INAPPROPRIATE?

Although it is remarkable that all the recent reports find a similarlylow overall rate of inappropriate studies, there are findings that raisequestions about whether some inappropriate studies may still haveimportant clinical implications. For example, the 2 studies that re-ported echocardiography findings according to appropriateness levelfound that inappropriate studies had a surprisingly high and remark-ably similar rate of important new or unexpected echocardiogramabnormalities (20%7, 17%8). Whether these incidental findings jus-tify performance of a study can certainly be debated. However, thisfinding raises the possibility that either not all inappropriate indica-tions are truly inappropriate or the process of ascribing indicationswithin the AC for echocardiography may not capture the entirety ofthe clinical circumstances that led to the ordering of the study. Anexample of the latter was demonstrated in our study at the Universityof Chicago. Patients with known heart failure who returned within 1year for a repeat echocardiogram but had “no change in clinicalstatus” were deemed inappropriate as per the AC.5 However, asubset of these patients had a left ventricular ejection fraction lessthan 35% and were returning for reevaluation of left ventricularfunction after an initial trial of medical therapy or coronary revascu-larization to determine candidacy for implantable cardioverter defi-brillator placement, an indication that current guidelines would sup-port.8 Although this may simply represent inflexibility in theindication assignment process, it serves as an illustration of how“purposefully broad” and inclusive application of the AC can neglectimportant clinical details that may alter the assessment of the trueclinical appropriateness of the study.

DOES THE ORDERING PROVIDER AFFECTAPPROPRIATENESS?

Barbier and colleagues9 previously reported that family physiciansand specialists demonstrate no difference in the adherence to guide-lines for TTE ordering.9 However, guidelines represent a differentstandard than AC. Guidelines suggest that in a particular clinicalcircumstance a study is indicated, whereas AC merely suggest that astudy for a particular indication would be generally acceptable orreasonable.5

In the recent studies of the AC for echocardiography, it seems thatordering provider characteristics have only a modest effect on theappropriateness of TTE studies. Kirkpatrick et al7 found no significantrelationship between the ordering physician specialty or the experi-ence level of ordering providers and the level of appropriateness.7

Martin and Picard6 noted only a slightly higher frequency of appro-priate studies when cardiovascular consultants were involved in thecare of the patient. In a larger sample, a statistically significant,although modestly lower, frequency of inappropriate studies orderedby cardiac specialists compared with noncardiac specialists (9% vs13%, P � .04) was noted.8 Although these results hint that noncar-diac specialists may represent an initial target for education about theappropriate use of echocardiography, this conclusion is tempered bythe populations studied to date. All have been in academic practices,and among outpatient studies a higher percentage of studies were

ordered by cardiovascular specialists (48%-53%)7,8 compared with

62 Journal of the American Society of EchocardiographyJanuary 2009

estimates of the United States as a whole (29%).3,4 Thus, furtherresearch in broad populations is needed to fully endorse this strategy.

HOW DO WE MEASURE THE TRUE APPROPRIATENESSOF A DIAGNOSTIC IMAGING TEST?

The authors of the AC recognize that it is challenging to determinethe true clinical appropriateness of a diagnostic imaging test. Theyspecifically caution that the AC should not be “considered substitutesfor sound clinical judgment or practice experience.”5 With rareexceptions, this difficulty arises because of the lack of clear clinicaltrial evidence linking an imaging-based management strategy toimproved patient outcomes. Thus, for determination of appropriate-ness status within the AC, technical panels were forced to rely on aheavy dose of expert opinion. Ideally, the ACwould now be tested inclinical practice in an attempt to demonstrate that appropriate studiesimprove outcomes or change patient management, whereas inappro-priate studies do not. Unfortunately, the current studies provide cluesabout the difficulties in reaching this standard.

Two of the recent studies attempted to address the issue of thetrue appropriateness of an echocardiogram by reporting rates ofabnormal findings on the TTE according to appropriateness sta-tus.7,8 By using this surrogate, Kirkpatrick and colleagues7 foundno significant relationship between level of appropriateness andnew or unexpected TTE abnormalities, whereas at the Universityof Chicago we found that new clinically important echocardio-gram abnormalities were significantly more common among ap-propriate studies compared with inappropriate studies.8 This dif-ference is likely due to differences in sample size and the largedisparity in TTEs with a prior study for comparison (78%7 vs36%8), thus resulting in fewer new findings overall in Kirkpatricket al’s study.

The real question is whether these abnormalities are relevant indiscussions of appropriateness of an echocardiographic study. Forexample, a patient with shortness of breath would have an appropri-ate indication for echocardiography, and a normal study may wellprompt a pulmonary workup and diagnosis that lead to an improve-ment in symptoms or patient outcomes. Such an example wouldmeet all reasonable measures of clinical appropriateness, but thissecondary level of clinical decision making would be challenging totrack accurately in any clinical trial. On the other hand, consider apatient with known hypertensive left ventricular dysfunction who isadmitted with a heart failure exacerbation and markedly elevatedblood pressure after neglecting to take blood pressure medications.This patient may well have a new echocardiogram finding of wors-ening left ventricular dysfunction or mitral regurgitation, but thiswould not likely change the management plan of resumption ofblood pressure control. Martin and Picard6 observed that clearprecipants (e.g. dietary noncompliance) were present in a highpercentage of tests ordered for heart failure exacerbations (37%)suggesting that despite an appropriate indication, the benefits ofechocardiography were negligible.

These examples highlight the subtleties and challenges of theassessment of true clinical appropriateness and suggest that incorpo-rating additional clinical details into the AC for echocardiographymight be useful. However, it should be recognized that the utility of

the AC will diminish if they become unwieldy. They are, and should

be, designed to leave room for reasonable clinical judgment. Simplerevisions that can be applied broadly might be useful and certainlypreferable to an exponential increase in the number or complexity ofindividual indications. One revision that might partially address theseissues would be to add a clear statement to the list of assumptions inthe AC for echocardiography: “It is assumed that a study done for anyindication can only be considered to be appropriate if it would beexpected to change or clarify clinical management or decision mak-ing.” This would fit within our clinical sense of appropriatenesswithout adding unnecessary layers to what is already a challengingprocess.

CONCLUSIONS

As both imagers and health care providers, improving quality for ourpatients has long been focused primarily on improving the diagnosticaccuracy of our imaging modalities and finding additional diseasestates or patient populations who might benefit from our evolvingtechniques. The success of these efforts has fostered dramatic in-creases in the use of cardiac imaging services. Although these validgoals continue, the new frontier in improving imaging quality involvesappropriate patient selection and use of cardiac imaging procedures.The AC represent an important first step in this new direction towardimproving quality. They are an effort by cardiovascular and imagingspecialists to define truly appropriate use of cardiac imaging proce-dures. Although lessons can be learned and improvements areneeded, the continued commitment of the imaging community to theappropriate use of imaging procedures will give patients and payersconfidence in the necessity and benefits of the cardiac imaging testswe provide.

REFERENCES

1. Medicare Payment Advisory Commission (MedPAC). Report to Congress:Medicare Payment Policy, March 2005. Washington DC: MedPAC 2005:154-1704.

2. Hendel RC. Utilization management of cardiovascular imaging: pre-certi-fication and appropriateness. J Am Coll Cardiol Imaging 2008;1:241-8.

3. Douglas PS. Quality in echocardiography: choosing to succeed. J Am SocEchocardiogr 2008;21:1016-7.

4. Pearlman AS, Ryan T, Picard MH, Douglas PS. Evolving trends in the useof echocardiography: a study of Medicare beneficiaries. J Am Coll Cardiol2007;49:2283-91.

5. Douglas PS, Khandheria B, Stainback RF, Weissman NJ, Peterson ED,Hendel RC, et al. ACCF/ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007appropriateness criteria for transthoracic and transesophageal echocardi-ography. J Am Soc Echocardiogr 2007;20:787-805.

6. Martin NM, Picard MH. Use and appropriateness of transthoracic echo-cardiography in an academic medical center: a pilot observational study.J Am Soc Echocardiogr 2009;22:48-52.

7. Kirkpatrick JN, Ky B, Rahmouni HW, Chirinos JA, Farmer SA, Fields AV,et al. Application of appropriateness criteria in outpatient transthoracicechocardiography. J Am Soc Echocardiogr 2009;22:53-59.

8. Ward RP, Mansour IN, Lemieux N, Gera N, Mehta R, Lang RM. Prospec-tive evaluation of the clinical application of the ACCF/ASE appropriate-ness criteria for transthoracic echocardiography. J AmColl Cardiol Imaging2008;1:663-71.

9. Barbier P, Alimento M, Berna G. Clinical utility of guidelines base echo-cardiography: a prospective study of outpatient referral patterns at a

tertiary referral care center. J Am Soc Echocardiogr 2008;21:1010-15.