From the Non

of Medicine U

R.M.L., R.P.W

Funding: This

Echocardiogr

Reprint reque

University of

MC6080 Chic

0894-7317/$3

Copyright 201

doi:10.1016/j.

Use of a Web-Based Application of the AmericanCollege of Cardiology Foundation/American Societyof Echocardiography Appropriateness Use Criteriafor Transthoracic Echocardiography: A Pilot Study

Nicole M. Bhave, MD, Ibrahim N. Mansour, MD, Federico Veronesi, PhD, Rabia R. Razi, MD,Roberto M. Lang, MD, FASE, and R. Parker Ward, MD, FASE, Chicago, Illinois

Background: Clinical application of the American College of Cardiology Foundation Appropriate Use Criteria(AUC) represents a potentially feasible alternative to third-party pre-certification for imaging procedures andwill soon be required as part of the accreditation process for imaging laboratories. Electronic tools that rapidlyapply the AUC are needed in clinical practice. We developed and tested a web-based application of the AUCto track appropriateness of transthoracic echocardiography (TTE).

Methods: Indications for outpatient TTE studies performed in a university hospital echocardiography labora-tory were assessed prospectively at the point of service using a prototype web-based AUC application (EchoAUC App). The Echo AUC App was developed on the basis of our own prior published data regarding indica-tion frequency to minimize time and screens required for completion. Echo AUC App-determined indicationswere comparedwith blinded investigator-determined indications based on review of relevantmedical records.Echo AUC App characteristics, including Echo AUC App entry time, were recorded.

Results: Of the 258 studies enrolled, Echo AUC App-determined TTE indications were Appropriate (A) in 77%(n = 198), Inappropriate (I) in 9% (n = 23), and Not Classified (NC) by the AUC in 14% (n = 37). Agreementbetween Echo AUC App- and investigator-determined classifications was excellent (94%, kappa statistic0.83). Mean Echo AUC App study entry time was 55 seconds (range 25-280 seconds).

Conclusion: The use of an electronic application allows rapid and accurate implementation of the AUC for TTEat the point of service. Such an application could be installed in echocardiography laboratories to track appro-priateness in accordance with soon-to-be-implemented accreditation requirements. Further study of thisEcho AUC App at the point of order may provide an alternative to third-party pre-certification procedures.(J Am Soc Echocardiogr 2011;24:271-6.)

Keywords: Appropriateness, Quality, Echocardiography

The last decade has seen a dramatic increase in the use of cardiovas-cular imaging. According to a review of Medicare data from1999-2004, echocardiography accounted for approximately half ofall spending on cardiac imaging services.1 This finding is not surprising,given that echocardiography is noninvasive, widely available, anduseful in a broad range of clinical scenarios. As imaging expenditurescontinue to grow, physicians and echocardiography laboratories faceincreasing scrutiny from federal and private payers. To help providers

-Invasive Imaging Laboratories, Section of Cardiology, Department

niversity of Chicago, Chicago, Illinois (N.M.B., I.N.M., F.V., R.R.R.,

.).

study was sponsored by a grant from the American Society of

aphy.

sts: R. Parker Ward, MD, FASE, Associate Professor of Medicine,

Chicago Medical Center, 5841 S. Maryland Avenue, RM 611B,

ago, IL 60637 (E-mail: pward@medicine.bsd.uchicago.edu).

6.00

1 by the American Society of Echocardiography.

echo.2010.12.027

and reimbursement agencies determine a rational approach to echo-cardiography use, the American College of Cardiology Foundation(ACCF) and American Society of Echocardiography (ASE) havepublished Appropriate Use Criteria (AUC) for transthoracic echocar-diography (TTE) and transesophageal echocardiography.2

Since publication, the AUC have rapidly been adopted as a usefultool to guide the appropriate use of imaging procedures. TheIntersocietal Accreditation Commission has announced that trackingand reporting of appropriateness according to the AUC will soon be-come a requirement for cardiovascular imaging laboratories as part ofthe accreditation process.3 The AUC have also emerged as a poten-tially feasible alternative to the expensive and inefficient process ofpre-certification for imaging procedures. Although cumbersome forall imaging modalities, pre-certification procedures are particularlyimpractical for TTE, given that it has the highest volume of all cardiacimagingmodalities. As a result, there is great interest in exploring waysto incorporate the AUC for echocardiography into order entrysystems to improve the quality and efficiency of patient care.

The application of the AUC for TTE has been studied extensivelyin a variety of clinical settings4-10 and has been found to be accurate,reproducible, and feasible. However, the manual processes studied to

271

Table 1 Patient and ordering physician characteristics fortransthoracic echocardiograms

No. of TTEs (%) (N = 258)

Female patient 137 (53%)

Prior TTE 90 (35%)

Within the preceding year 15 (17%)

>1 year ago 75 (83%)

Referring physician specialtyCardiology 142 (55%)

Internal medicine/family practice 83 (32%)

Abbreviations

ACCF = American College ofCardiology Foundation

ASE = American Society ofEchocardiography

AUC = Appropriate Use

Criteria

TTE = Transthoracic

echocardiography

272 Bhave et al Journal of the American Society of EchocardiographyMarch 2011

date are time-consuming andcumbersome, and practitionersnot thoroughly familiar with theAUCmay have to review the en-tire list of indications beforesettling on the most germane.An automated decision-tree ap-plication of the AUC is neededto allow rapid appropriatenessdeterminations and may lead tomore widespread use of theAUC in clinical practice.

Other 33 (13%)

TTE, Transthoracic echocardiogram.

In an effort to address this need, we designed a prototype web-based, automated application (Echo AUC App) that classifies theindications for TTE according to the AUC and tested the feasibilityof its implementation in routine clinical practice. We hypothesizedthat the Echo AUC App would allow rapid and accurate clinicalapplication of the AUC.

MATERIALS AND METHODS

Outpatients aged $18 years who were referred to the University ofChicago Medical Center for TTEs from September 2009 toDecember 2009 were eligible for inclusion in this prospective study.The study was approved by the institutional review board, and allpatients provided informed consent. For each echocardiogramincluded, basic patient demographics and pertinent clinical informa-tion necessary to determine the primary study indication were en-tered prospectively into a prototype web-based AUC application(Echo AUC App). Enrollment occurred at the point of service, beforeimage acquisition. The prototype Echo AUCApp is an online applica-tion of the 2007 ACCF/ASE AUC for TTE that provides a rapiddecision-tree algorithm to determine indication number (1-51) andlevel of appropriateness (Appropriate or Inappropriate) of each studyas defined by the AUC. The decision-tree algorithms in the EchoAUCApp were developed on the basis of previously published AUCimplementation studies at our institution, which established the fre-quency of studies ordered for each indication number in the AUCand defined critical branch points in the application of the AUC, tominimize the number of screens and time required to determinethe appropriateness of each TTE. The Echo AUC App includes a pre-liminary screen that queries the user for patient demographic informa-tion (age, gender), ordering physician specialty, and absence/presenceand date of prior TTE, followed by the decision-tree algorithm, witha final screen stating the resulting appropriateness determination:Appropriate, Inappropriate, or Unclassified (i.e., not addressed inthe 2007 AUC for TTE).

EchoAUCApp users were sonographers whowere performing thetransthoracic echocardiograms and physician investigators, none ofwhom had significant prior exposure to the Echo AUC App. Beforeinitiation of the study, all Echo AUCApp users were given a universallogin and password and a brief 15-minute tutorial on the use of theEcho AUC App, including an orientation to the type of informationrequired to complete the Echo AUC App for each TTE. All usersalso practicedwith the EchoAUCApp by entering two to five practiceTTEs before study initiation. All Echo AUC App users had access toonline or written study requisitions, in addition to the hospital’s elec-tronic medical record and ordering systems. For each TTE entered inthe study, Echo AUC App response characteristics were recorded,

including number of screens required to arrive at a final appropriate-ness determination. The time to complete the application, from log-onto final appropriateness determination, was recorded automatically bythe Echo AUC App for all studies. In addition, a separate physicianinvestigator, blinded to the Echo AUC App-derived appropriatenessresults, manually determined the AUC indication number and appro-priateness classification for each TTE from hospital records and studyrequisitions, using a methodology we have previously validated.6

Echo AUC App-derived indication numbers and appropriatenessdeterminations were then compared with the manually derived indi-cation numbers and appropriateness determinations in an effort toevaluate the accuracy of the Echo AUC App.

Statistical Analysis

Comparisons between Echo AUC App-derived and manuallyderived AUC study indications and AUC appropriateness determina-tions were performed with chi-square tests for categoric data andStudent t test for continuous data using a two-tailed P value < .05for statistical significance, or kappa statistics as appropriate.Interobserver variability in the determination of indication number/category and level of appropriateness (Appropriate, Inappropriate,Unclassified) is expressed as percent agreement between the EchoAUC App-derived and manually derived determinations and withthe use of kappa statistics. For indication number/category determina-tions, interobserver comparisons represent the frequency with whichthe reviewers agreed on indication numbers or categories (i.e., indica-tions 1-51, Unclassified). For level of appropriateness determinations,interobserver comparisons represent the frequencywith which the re-viewers selected indications with matching levels of appropriateness(A vs. I) or agreed that studies were Unclassified, although they didnot necessarily choose identical indication numbers.

RESULTS

A total of 258 TTEs were included in the study. The study andprovider characteristics are shown in Table 1. The mean age of thepopulation was 59618 years, and 53% of TTEs (n = 137) were per-formed on women. A prior TTE had been performed in 35% of cases(n = 90). Among patients with prior TTEs, 17% (n = 15) of priorstudies had been performed within the preceding year, whereas83% (n = 75) had been performedmore than 1 year before the indexTTE. Cardiology was themost frequent referring specialty, accountingfor 55% (n = 142) of TTEs enrolled, with Internal Medicine/PrimaryCare the next most common (32%, n = 83). Overall, the most

Table 2 Most common indications for transthoracic echocardiogram

Indication as listed in the AUC for echocardiography

No. of studies (%)

(N = 221 classifiable studies)

1. Symptoms due to suspected cardiac cause, including but not limited to dyspnea,shortness of breath, lightheadedness, syncope, TIA, cerebrovascular events (Appropriate)

71 (32%)

2. Prior testing that is concerning for heart disease (e.g., chest x-ray, baseline scout images for

stress echocardiogram, ECG, elevation of serum BNP) (Appropriate)

20 (9%)

17. Initial evaluation of murmur in patients for whom there is a reasonable suspicion of valvular

or structural heart disease (Appropriate)

18 (8%)

10. Evaluation of known or suspected pulmonary hypertension, including evaluation of right ventricularfunction and estimated pulmonary artery pressure (Appropriate)

15 (7%)

42. Routine (yearly) reevaluation of patients with heart failure (systolic or diastolic) in whom thereis no change in clinical status (Inappropriate)

9 (4%)

AUC, Appropriate Use Criteria; TIA, transient ischemic attack; ECG, electrocardiogram; BNP, brain natriuretic peptide.

Table 3 Most common inappropriate indications for transthoracic echocardiogram

Indication as listed in the AUC for echocardiography

No. of studies (%)

(N = 23 inappropriate studies)

42. Routine (yearly) reevaluation of patients with heart failure (systolic or diastolic) in whom there

is no change in clinical status

9 (39%)

21. Routine (yearly) reevaluation of an asymptomatic patient with mild native AS or mild-moderate

native MS and no change in clinical status

5 (22%)

5. Patients who have isolated APC or PVC without other evidence of heart disease 3 (13%)

AUC, Appropriate Use Criteria; AS, aortic stenosis; MS, mitral stenosis; APC, atrial premature contractions; PVC, premature ventricular contrac-

tions.

Appropriate

Innapropriate

Unclassified77%

14%

9%

Figure 1 Appropriateness classifications for TTEs as deter-mined by the web-based Echo AUC App.

Journal of the American Society of EchocardiographyVolume 24 Number 3

Bhave et al 273

common indication for TTE was AUC indication 1, ‘‘Symptomspotentially due to suspected cardiac etiology.’’ (32% of classifiableTTEs [n = 71 TTEs]; Table 2), whereas the most common inappropri-ate indication was AUC indication 42, ‘‘Routine (yearly) reevaluationof patients with heart failure in whom there is no change in clinicalstatus’’ (5% of classifiable TTEs [n = 11]; Table 3).

Overall, a large majority of TTEs (77% [n= 199]) were found to beappropriate by the Echo AUCApp (Figure 1). The remainder of stud-ies were Inappropriate (9% [n = 23]) or Unclassified (14% [n = 36],Table 4). Appropriate studies were more likely to be ordered by car-diologists compared with non-cardiologists (84%, 119/142 vs. 69%,80/116; P = .01). Differences between cardiologist versus non-cardiologist ordering of Inappropriate (6%, 8/142 vs. 13%, 15/116;P = NS) and Unclassified studies (11%, 15/142 vs. 18%, 21/116;P = NS) did not meet statistical significance.

Appropriate studies were also more likely to be initial TTEs (90%,152/168 of TTEs with no prior TTEwere Appropriate vs. 52%, 47/90of studies with prior TTEs; P < .001). Among studies with prior TTEs(n = 90), those with prior TTEs within the last year were less fre-quently Appropriate compared with those for which the prior TTEswere more than 1 year prior (33%, 5/10 vs. 69%, 52/75; P = .04).A comparison of Inappropriate studies between those with priorstudies within the last year and more than 1 year did not reach statis-tical significance (47%, 7/15 vs. 13%, 10/75; P = NS). Overall, 74%(17/23) of Inappropriate TTEs were studies for which prior TTEshad been performed. No significant difference in Unclassified desig-nations occurred between studies with and without prior TTEs(18%, 16/90 vs. 12%, 20/168; P = NS).

Manually derived and Echo AUCApp-derived appropriateness de-terminations were found to be similar (Appropriate: 77% by EchoAUC App vs. 81% by manual, kappa statistic 0.84 for Appropriate

vs. Other; Inappropriate: 9% by Echo AUC App vs. 10% by manual,kappa statistic 0.95 for Inappropriate vs. Other; Unclassified: 14% byEchoAUCApp vs. 9% bymanual, kappa statistic 0.70 for Unclassifiedvs. Other). Overall, Echo AUC App-determined and manuallydetermined classifications of indication/category (indications 1-51or Unclassified) and overall levels of appropriateness (Appropriate,Inappropriate, or Unclassified) agreed in 82% (n = 211, kappa statistic0.79) and 94% (n = 242, kappa statistic 0.83) of studies, respectively.

Overall, in 18% (n = 47) of studies the manual rater and the EchoAUC App chose different individual indications (indications 1-51, orUnclassified). In 66% (n = 31) of these, the ultimate appropriatenessdesignation was not affected (Avs. I vs. Unclassified). Among these 31studies, the most common indication pairing (n = 12) was AUCindication 1 (‘‘Symptoms suspected due to a cardiac etiology..’’)and AUC indication 41 (Initial evaluation of known or suspected

Table 5 Echo Appropriate Use Criteria App completion by appropriateness category

Appropriate Inappropriate Not classified All studies (N = 258)

Screens required 4.3 6 1.3 4.8 6 1.2 5.8 6 1.1* 4.6 6 1.3Echo AUC App Completion time (mean 6 SD, sec) 51 6 31 64 6 28† 74 6 54‡ 55 6 38

AUC, Appropriate Use Criteria.*P < .001 compared with Appropriate TTEs; P < .01 compared with Innapropriate TTEs.

†P = .04 compared with Appropriate TTEs.

‡P < .01 compared with Appropriate TTEs.

Table 4 Most common indications for transthoracic echocardiogram that were Unclassified by the 2007 Appropriate Use Criteriafor transthoracic echocardiogram

Indication for TTE unclassified in the AUC for echocardiography

No. of studies (%)

(N = 36 unclassified studies)

Surveillance related to solid-organ transplant program 11 (31%)

Heart Failure: revaluation of patient with heart failure (systolic or diastolic), >1 year

since prior study with no change in clinical status

6 (17%)

Preoperative evaluation 6 (17%)Native valvular stenosis or regurgitation: routine reevaluation with severity of valve

disease or time interval for follow-up not specifically addressed by the AUC

6 (17%)

TTE, Transthoracic echocardiogram; AUC, Appropriate Use Criteria.

274 Bhave et al Journal of the American Society of EchocardiographyMarch 2011

heart failure [systolic or diastolic]), both of which are Appropriate. Ineach of these 12 studies, the clinical scenario involved symptoms sug-gestive of heart failure; thus, both of these indications seemedto reasonably apply. Among the 16 studies for which the EchoAUC App-derived and manually derived appropriateness determina-tions were discrepant, the Echo AUC App found 2 manually deter-mined Inappropriate studies to be Unclassified, 12 manuallydetermined Appropriate studies to be Unclassified, and 2 manuallydetermined Unclassified studies to be Appropriate. Thus, all studiesfor which there was a discrepant designation involved one ratingmethodology selecting an Unclassified designation. In no study didthe two methods find a directly conflicting appropriateness designa-tion (Appropriate vs. Inappropriate). The indications most commonlyinvolved in a discrepant appropriateness designation were thoserequiring the rater to determine a ‘‘change in clinical status.’’ For these16 studies with a discrepant designations, the Echo AUC App morefrequently was the rating methodology to determine a study to beUnclassified (14/16 of discrepant studies).

The number of screens required to complete the Echo AUC Appfor all studies was 4.6 6 1.3 (mean 6 SD), with fewer screensrequired for TTEs deemed Appropriate (4.361.3) and more screensfor TTEs deemed Inappropriate (4.8 6 1.2, P = .08 compared withAppropriate TTEs) and Unclassified (5.8 6 1.1, P < .001 comparedwith Appropriate transesophageal echocardiograms; P < .01compared with Inappropriate TTEs).

Echo AUC App completion time was collected automatically forall studies (n = 258). Echo AUC App users achieved an appropriate-ness determination in a mean 55 6 SD 38 seconds (range 25-280seconds). The median Echo AUC App completion time was 41 sec-onds, and 71% (184/258) of studies were completed in less than 1minute. Echo AUC App completion times for TTEs deemedAppropriate (n = 199), Inappropriate (n = 23), and Unclassified(n = 36) were 51 6 31, 64 6 28 (P = .04 for comparison withAppropriate TTEs) and 74 6 54 seconds (P < .01 for comparisonwith Appropriate TTEs), respectively (Table 5).

DISCUSSION

Our results suggest that use of a web-based decision-support applica-tion for determining TTE appropriateness according to the ACCF/ASE AUC for TTE at the point of service is rapid, feasible, and accu-rate. We demonstrate that such an application distills the process ofidentifying relevant TTE indications from an 18-page paper documentdown to a few simple responses as part of an electronic decision tree,consuming less than 1 minute of the user’s time in a majority of cases.To our knowledge, this study is the first to test the feasibility of an elec-tronic, practitioner-entry AUC application.

Hendel and colleagues11 previously developed a computer-basedprogram for analyzing appropriateness of myocardial perfusion imag-ing, which they found to be accurate in a multicenter study. This priorstudy differs slightly from ours in that clinical information collectedmanually at the point of service was forwarded to a central locationfor electronic entry. Our study expands on this methodology by test-ing the use of an electronic application in the clinical arena; such thatpractitioners directly enter clinical information in the electronic appli-cation on patient presentation, and thus AUC indications and appro-priateness determinations are established before performance of eachstudies. We find that with practitioner entry of data into an electronicapplication at the point of service, AUC appropriateness determina-tions can be established rapidly and accurately, with minimal disrup-tion in clinical work flow.

We chose to test the Echo AUCApp at the point of service, becausethis has direct applicability to upcoming changes in the clinical prac-tice of echocardiography. The Intersocietal Commission for theAccreditation of Echocardiography Laboratories has announcedthat AUC tracking and reporting the appropriateness of echocardio-grams will be required as part of the echocardiography laboratory ac-creditation procedures beginning in January 2012.12 An electronic orweb-based application, such as the one tested in this study, representsa viable option for echocardiography laboratories to rapidly andefficiently meet this requirement. In addition, our findings suggest

Journal of the American Society of EchocardiographyVolume 24 Number 3

Bhave et al 275

that such an application deserves testing at the point of order, poten-tially allowing appropriateness decision-support aides to be incorpo-rated as part of order-entry systems.4-9,13

In keeping with the results of prior studies at our institution and else-where, a majority of TTEs were found to be appropriate (77%), anda small but important fraction of outpatient TTEs were ordered forInappropriate indications (9%). Also consistent with prior studies, a sig-nificant number of TTEs had indications that were unclassified (14%)or not addressed by the 2007 AUC for TTE.14 The ACCF, in conjunc-tion with the ASE and other imaging societies, has recently completedan update of the 2007 AUC for echocardiography, which addressesmany of these gaps and provides additional indications and clarifica-tions. Although we designed and assessed the Echo AUC App in thisstudy using algorithms based on the 2007 AUC for TTE, the feasibilityand rapidity of electronic application of the AUC demonstrated willlikely be preserved, and perhaps enhanced, when the Echo AUC Appis updated to reflect the new more complete and efficient algorithms.

Our findings further demonstrate that the accuracy of appropriate-ness determinations does not suffer with the Echo AUCApp. In addi-tion to determining TTE indications and levels of appropriatenesswith the Echo AUC App, we performed blinded manual indicationassignments and appropriateness determinations using a processthat we have previously validated.6 We demonstrated a high levelof agreement between Echo AUC App and manual determinationsof indication numbers and appropriateness designations. The levelsof agreement between Echo AUC App and manual determinationsfor both indication numbers/categories and overall appropriatenesswere similar to comparisons between two blinded manual reviewersin our prior validation of this process,6 suggesting that the Echo AUCApp’s results are non-inferior to those of manual review.

As would be expected, there remain studies for which more thanone AUC indication reasonably applies, no matter what method ofindication and appropriateness determination is used. Fortunately,our findings in this study support prior data suggesting that despitethis, the ultimate appropriateness designation is rarely affected (94%agreement). In our study, the most common discrepancy in ultimateappropriateness designation (Appropriate, vs. Inappropriate, vs.Unclassified) involved one method determining the study to beunclassifiable. This generally occurred because of different interpreta-tion in whether a ‘‘change in clinical status’’ had occurred, with theEcho AUC App more frequently selecting Unclassified in the smallnumber of discrepant studies. Thismight be expected, because aman-ual rater can seek further clinical information and take more time toidentify subtle clinical details to justify a ‘‘change in clinical status’’that may not be immediately obvious. Although this represents a po-tential limitation to electronic algorithms to apply the AUC, it shouldbe remembered the appropriateness designation agreed in themajority of cases (94%) in our study, and it is clear that rigorous,time-consuming manual application of the AUC is not feasible inbusy routine clinical practice. Further, although discrepancies wererare even with the current version of the AUC, the soon-to-be-pub-lished updated AUCwill fill in many of the indication gaps and clarifythewording of existing indications, which is likely to further reduce thenumber of unclassifiable indications by any appropriateness determi-nation methodology.

On the basis of our results, a web-based system could provide anefficient means of tracking appropriateness for the purposes of labora-tory accreditation and quality improvement. Even in high-volume cen-ters, such a program could be used to collect comprehensive data forexternal reporting to meet the new upcoming accreditation require-ment.Moreover, an individual laboratory that uses such an application

can query its database to track ordering patterns, potentially identify-ing opportunities for targeted education to reduce inappropriateTTE use. Our study suggests that such an application warrants testingat the point of order, incorporated as an educational device to influ-ence appropriate use of echocardiography procedures or as part ofan order entry system. Either application represents an attractive alter-native to the cumbersome, time-consuming, and expensive pre-certification process that is frequently used for other cardiac imagingmodalities but is impractical for a high-volume procedure such as TTE.

LIMITATIONS

There are limitations of our study that deserve mention. First, becauseour study was performed at a single-center, academic medical institu-tion with a centralized echocardiography laboratory, we cannot besure that the Echo AUC App would function as well in other practicesettings. Of particular concern are practice settings where prior echo-cardiogram dates and findings are not easily retrievable, because themajority of Inappropriate TTEs we identified were repeat studies.Second, because we chose to study only outpatient TTEs, we cannotcomment on how this Echo AUC App would perform in an inpatientsetting. However, it should be noted that inpatient TTEs have beenshown to be appropriate in more than 90% of cases in a prior study,4

largely because most inpatients have had recent changes in clinicalstatus. The availability of up-to-date clinical information (e.g., admis-sion notes, laboratories, and other imaging studies) in the inpatientsetting and the more universal use of order-entry systems suggestthat such an application could be effectively applied. Finally, becausewe evaluated the Echo AUC App at the point of service, we cannotdraw definitive conclusions about its potential effectiveness at thepoint of order, where the time limitations of busy clinicians are thegreatest. However, prior studies have found that point-of-orderdecision-support applications15,16 can reduce inappropriate use ofradiology procedures, presumably because they educate cliniciansregarding inappropriate test indications or encourage them to thinkcritically about test utility.17

CONCLUSIONS

This study demonstrates that a web-based application of the AUC forechocardiography represents a practical and feasible instrument withwhich echocardiography laboratories can track and report the appro-priateness of procedures they perform. This is an important andtimely observation, because AUC tracking and reporting will soonbe required for laboratory accreditation.3 Such an application alsoholds great promise for use at the point of order, ideally as an educa-tional guidance tool to influence appropriate use of echocardio-graphic procedures, potentially within an electronic order entrysystem. Further study of this or similar electronic applications of theAUC in the clinical arena will be needed to determine whetherthey can lead to demonstrable improvements in appropriate use ofechocardiographic procedures and cost-effective, high-quality patientcare.

ACKNOWLEDGMENT

Since our article was submitted for publication, a new documentupdating the AUC for echocardiography has been completed and

276 Bhave et al Journal of the American Society of EchocardiographyMarch 2011

published online.18 We understand that this article is beingco-published in the current issue of the Journal of the AmericanSociety of Echocardiography. Because the listing of clinical scenarioshas been expanded from that included in the 2007 document2 onwhich our study was based, we acknowledge that the web-based ap-plication of the AUC described in our article will need to be modifiedto reflect the updated recommendations. Although further evaluationwill be needed, we believe that the same general approach describedin our article should be applicable and that this approach has promisefor implementation, at the point of order, as a means to enhance theappropriate use of echocardiography.

REFERENCES

1. PearlmanAS, Ryan T, PicardMH,Douglas PS. Evolving trends in the use ofechocardiography: a study of Medicare beneficiaries. J Am Coll Cardiol2007;49:2283-91.

2. Douglas PS, Khanderia B, Stainback RF, Weissman NJ, Brindis RG,Patel MR, et al. ACCF/ASE/ACEP/ASNC/SCAI/SCCT/SCMR 2007appropriateness criteria for transthoracic and transesophageal echocardi-ography: a report of the American College of Cardiology FoundationQuality Strategic Directions Committee Appropriateness Criteria Work-ing Group, American Society of Echocardiography, American College ofEmergency Physicians, American Society of Nuclear Cardiology, Societyfor Cardiovascular Angiography and Interventions, Society of Cardiovas-cular Computed Tomography, and the Society for CardiovascularMagnetic Resonance. Endorsed by the American College of Chest Physi-cians and the Society of Critical Care Medicine. J Am Soc Echocardiogr2007;20:787-805.

3. IAC to incorporate measurement of Appropriate Use Criteria (AUC). TheIAC Newsletter, 3(1), Spring 2010. Available at: http://www.intersocietal.org/iac/news/2010/spr2010_auc.htm. Accessed December 12, 2010.

4. Martin (Bhave) NM, Picard MH. Use and appropriateness of transthoracicechocardiography in an academic medical center: a pilot observationalstudy. J Am Soc Echocardiogr 2009;22:48-52.

5. Ward RP, Krauss D, Mansour IN, Lemieux N, Gera N, Lang RM. Compar-ison of the clinical application of the American College of CardiologyFoundation/American Society of Echocardiography appropriateness crite-ria for outpatient transthoracic echocardiography in academic and com-munity practice settings. J Am Soc Echocardiogr 2009;22:1375-81.

6. WardRP,Mansour IN, LemieuxN,GeraN,Mehta R, LangRM. Prospectiveevaluation of the clinical application of the AmericanCollege of CardiologyFoundation/American Society of Echocardiography criteria for transtho-racic echocardiography. JACC Cardiovasc Imaging 2008;1:663-71.

7. Rao G, Sajnani N, Kusnetzky LL, Main ML. Appropriate use of transtho-racic echocardiography. Am J Cardiol 2010;105:1640-2.

8. Aggarwal NR, Wuthiwaropas P, Karon BL, Miller FA, Pellikka PA. Applica-tion of the appropriateness criteria for echocardiography in an academicmedical center. J Am Soc Echocardiogr 2010;23:267-74.

9. Willens HJ, Gomez-Marin O, Heldman A, Chakko S, Postel C, Hasan T,et al. Adherence to appropriateness criteria for transthoracic echocardiog-raphy: comparisons between a regional department of Veterans Affairshealth care system and academic practice and between physicians andmid-level providers. J Am Soc Echocardiogr 2009;22:793-9.

10. Kirkpatrick JN, Ky B, Rahmouni HW, Chirinos JA, Farmer SA, Fields AV,et al. Application of appropriateness criteria in outpatient transthoracicechocardiography. J Am Soc Echocardiogr 2009;22:53-9.

11. Hendel RC, Cerqueira M, Douglas PS, Caruth KC, Allen JM, Jensen NC,et al. A multicenter assessment of the use of single-photon emission com-puted tomography myocardial perfusion imaging with appropriatenesscriteria. J Am Coll Cardiol 2010;55:156-62.

12. The complete 2010 ICAEL standards for accreditation in adult echocardiog-raphy testing: Parts I through IV. [web-based monograph]. Available at:http://www.icael.org/icael/standards/ICAEL_2010_Standards%20(Adult%20Echo).pdf. Accessed October 11, 2010.

13. Ward RP. Appropriateness criteria for echocardiography: an importantstep toward improving quality. J Am Soc Echocardiogr 2009;22:60-2.

14. Martin NM, Picard MH. Two years of appropriateness criteria for echocar-diography: what have we learned and what else do we need to do? J AmSoc Echocardiogr 2009;22:800-2.

15. Vartanians VM, Sistrom CL, Weilburg JB, Rosenthal DI, Thrall JH. Increas-ing the appropriateness of outpatient imaging: effects of a barrier to order-ing low-yield examinations. Radiology 2010;255:842-9.

16. Solberg LI, Wei F, Butler JC, Palattao KJ, Vinz CA, Marshall MA. Effects ofelectronic decision support on high-tech diagnostic imaging orders andpatients. Am J Manag Care 2010;16:102-6.

17. Hendel RC. Utilization management of cardiovascular imaging: pre-certification and appropriateness. JACC Cardiovasc Imaging 2008;1:241-8.

18. Douglas PS, Garcia MJ, Haines DE, Lai WW, Manning WJ, Patel AR, et al.ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCCM/SCCT/SCMR 2011appropriate use criteria for echocardiography. J Am Coll Cardiol 2011;24:229-67.