Evidence-based Medicine Search: a customizable federatedsearch enginePaul J. Bracke, MLS; David K. Howse, MLS; Samuel M. Keim, MD

See end of article for authors' affiliations. DOl; 10.3163/1536-5050.96.2.108

Purpose: This paper reports on the development of atool by the Arizona Health Sciences Library (AHSL)for searching clinical evidence that can becustomized for different user groups.

Brief Description: The AHSL provides services tothe University of Arizona's (UA's) health sciencesprograms and to the University Medical CenterLibrarians at AHSL collaborated with UA College ofMedicine faculty to create an innovative searchengine, Evidence-based Medicine (EBM) Search, thatprovides users with a simple search interface to EBMresources and presents results organized according

to an evidence pyramid. EBM Search was developedwith a web-based configuration component thatallows the tool to be customized for differentspecialties.

Outcomes/Conclusion: Informal and anecdotalfeedback from physicians indicates that EBM Searchis a useful tool with potential in teaching evidence-based decision making. While formal evaluation isstill being planned, a tool such as EBM Search,which can be configured for specific userpopulations, may help lower barriers to informationresources in an academic health sciences center.

Highlights

• A tool for federating search of clinicai resources wasdeveloped through a quick and iterative process in-volving librarians and physicians. The tool embedsthe search knowledge of reference librarians into asimple search interface that appeals to busy clini-cians.

• The tool is highly configurable and can be customizedfor different clinical and education environments.

Imp l ica t ions

• Useful tools for enhancing user access to library re-sources can be developed in a quick manner if userneeds and workflow are well understood. These toolscan be implemented by staff in the library with appro-priate skills.

• Iterative design processes can speed the design ofinnovative library services.

• Expert search skills can be an important part of de-signing technologies that provide more accuratesearch results to clinicians in a convenient manner.The ability of searchers to understand the changingneeds of their users, as weil as the changing natureof the information environment, can provide new op-portunities to apply their knowledge.

INTRODUCTION

Background

Libraries and other information organizations havebeen grappling with the challenge of developing moreefficient tools to allow users to discover informationrelevant to their needs, with many efforts focused oncreating tools that facilitate access to a broad range ofcontent to broad audiences. Web search engines from

Alta Vista to Google, sometimes referred to as hori-zontal search engines, exemplify these approaches.Federated search engines implemented by many li-braries provide a similar approach to enhancing accessto the content of licensed and free article databasesand other resources [1^]. Recently, however, interesthas increased in developing vertical search enginesthat provide access to targeted content to better meetthe needs of particular communities or subject do-mains [5]. On the open web, Google Custom Search isone example of this approach 16].

At the same time, there has been significant interestin clinical medicine and other health professions inbetter integrating evidence from the medical literatureand other sources into clinical practice. The complex-ities of knowledge translation [7] and the time con-straints in clinical settings [8], however, create barriersto adopting evidence-based practice (EBP). Studieshave shown that the user experience provided by com-mercial clinical information platforms can have a pos-itive impact on both clinicians' perception of a searchtool and their ability to find answers to clinical ques-tions 19].

In addition to challenges specific to EBP, the con-venience of general search engines has created a stan-dard for ease of search, demonstrated in studies of cli-nician search behavior [10]. In response, many librarieshave implemented federated search tools with single,integrated results sets 11-4]. Several projects have alsocreated sophisticated search engines that federate clin-ical evidence in a single result set [11--14]. Addition-ally, other health sciences libraries have undertakenprojects to create specialized search interfaces for spe-cific types of resources [15, 16].

Librarian and clinician collaboration

The Arizona Health Sciences Library (AHSL) felt theseapproaches could be improved by organizing the userexperience around one of the visual constructs used to

108 J Med Libr Assoc 96(2) April 2008

EBM Search

teach evidence-based medicine (EBM) at the Universityof Arizona [17]. Further, clinicians and librarians at theinstitution felt that the commercial clinical searchproducts available to them were inherently limited bynot integrating content from multiple publishersequally. This was especially true of electronic book re-sources.

To develop an approach to the problem of integrat-ing information searching and library expertise effi-ciently into clinical practice, the team chose local de-velopment of a tool because no commercial productwas available to the institution that provided federatedsearching of licensed and freely available clinical evi-dence, provided supported customized display of re-sults, and supported the embedding of power-search-ing strategies in a manner that was transparent to us-ers. In light of this context, the AHSL, as a librarydedicated to improving the integration of clinical evi-dence into clinical practice, sought to explore the fol-lowing question: Is it possible to create a seardn toolthat is both simple and convenient to use and suffi-ciently powerful to provide effecäve access to clinicalevidence?

As described elsewhere, it became apparent to theauthors that synergies between clinical departmentsand the health sciences library clearly provided an op-portunity for integrating evidence into clinical deci-sion making [18]. Librarians had been involved inEBM efforts for several years in an instructional ca-pacity, and while the team felt that teaching cliniciansto effectively search and evaluate information hadmerits, instruction also seemed to be a limited ap-proach. Given the time constraints inherent in the clin-ical workplace, it was also seen as unlikely that phy-sicians would apply this instruction in practice. Like-wise, models such as the information ist that have beenimplemented in other settings were seen as unrealisticin the University of Arizona environment, both for fi-nancial reasons and the unlikelihood that local clini-cians would relinquish control of query formulation atthe point of clinical decision making.

The goal of the project was to create a scalable mod-el for integrating librarian expertise into clinical deci-sion making, without requiring their availability inreal-time. To meet this goal, the team decided to de-velop a search tool that would allow physicians to con-trol query formulation and result evaluation, but thatwould embed the expertise of librarians in resourceselection and search construction. It would also embedclinical knowledge into the interface design to assistusers in making relevance judgments.

Evidence-based medicine search development

The search tool, EBM Search, was programmed usingCold Fusion with a Microsoft SQL server databasebackend between February and August 2006. A rapiddevelopment process was employed, which focused onprototype development followed by an iterative feed-back and modification process. The first iteration of theend-user search tool, developed in two weeks, provid-ed the initial proof-of-concept and incorporated sev-

eral design considerations. First, the search interfaceneeded to be simple. Second, for practical and philo-sophical reasons, the team decided that providing aresults screen that integrated results from all resourcesincluded in EBM Search into one single set was un-desirable, in part, because the amount of developmentthat would be required to manage a single results setwas seen as unrealistic. More importantly, both clinicaland library partners felt that an integrated results setwas not optimal for enabling users to quickly select aresource at the appropriate level of evidence for theirneeds. Accordingly, the interface was organizedaround the evidence pyramid [16], a well-understoodconcept among local clinicians. Third, the search toolneeded to incorporate librarian-designed search hedg-es to improve relevance for the target group, therebyreducing the need for expert searching skills fromphysicians. Fourth, EBM Search would use each re-source's native interfaces for vocabulary mapping andrelevance ranking, as these features were better devel-oped in the individual resources than could be devel-oped in this project. Finally, where feasible and mean-ingful, the number of results from each provider wasdisplayed to enhance usability.

The initial implementation was designed for the de-partment of emergency medicine (EM) and allowedlocal emergency department (ED) physicians to exe-cute searches on topics of their choice. Resources wereselected through consultation with the EM residencydirector and interviews with EM residents. Based onthis input, the resulting search provided access to pre-executed queries in a number of resources: PubMedClinical Queries, the Cochrane Database of SystematicReviews, the Database of Abstracts of Reviews of Ef-fects (DARE), topic reviews from UpToDate, and text-books from StatlRef. The links to PubMed ClinicalQueries were limited in several ways, including a limitto core clinical and EM journals. In this case, core jour-nals were determined by local clinicians to be thosemost useful for finding clinical answers.

Figure 1 illustrates the tool's processes. When aseardi is submitted, the search tool executes severaltasks. First, the physicians' search is post-pended withEBM search filters that have been developed by a ref-erence librarian and modified with input from an EDphysician. Filters have been customized for each da-tabase to take advantage of the native functionality.Second, extensible markup language (XML) web ser-vices are used to obtain the number of results for dis-play, where supported. Third, uniform resource loca-tors (URLs) have been constructed to link users to pre-executed searches in the native interface. For example,a search for "myocardial infarction" would, amongother things, link to a PubMed search for randomizedcontrol trials on myocardial infarction using the fol-lowing strategy:

("myocardial infarction"[mh] OR myocardial infarction[tw])AND randomized controlled triallpt] OR randomizedITiab]AND controUedlTiab! AND trial[Tiab]

Finally, the results were displayed using a ranking

J Med Libr Assoc 96(2) April 2008 109

Bracke et al.

Figure 1Evidence-based medicine (EBM) search too! process

-—ainical Evtteixe

L M Construdlon

\

Saa-tíi «nd RmultaIrilQrfsce Corivtruckofi

/

c—'^Conngivatkxis

^ ^ ^

SeardiConfigjmdon

Tod

The process involved in searching and configuring EBM Search lo provide access to abstracting and indexing (A&l} services, textbooks, and other sources ofclinical evidence.

system based or\ an EBM pyramid (Figure 2). Querysources were grouped for display according to level ofevidence, accompanied by an illustration of the levelof evidence provided for each grouping. For example,the EBM Search tool for "emergency medicine" in-cludes groupings for Systematic Reviews, PubMedClinical Studies, PubMed (All), Evidence-BasedGuidelines, and Textbooks. Tlie Systematic Reviewsgrouping includes links to searches in the CochraneDatabase of Systematized Reviews and DARE, as wellas three different searches for systematic reviews inPubMed that apply progressive limits to narrow thesearch. This method provides an interface that utilizesa two-tiered relevance ranking system to allow phy-sicians to quickly select a desired level of evidence.Sources are organized by relevance to clinical decisioncategories such as therapy, diagnosis, and treatment,then the relevance-ranking features of each source areused to sort at the article and chapter level.

INITIAL FEEDBACK AND CONTINUEDDEVELOPMENT

The prototype was well received by the EM depart-ment and generated interest from other departmentsafter presentation in a grand rounds session. Althoughformal evaluation was still needed, the system clearlyhad potential and merited further development to beconfigurable for specialties other than EM. The systemshould also be modifiable by librarians without theassistance of a programmer.

In subsequent phases of development, the user in-terface remained essenäally unchanged. Minor for-matting changes were made, and additional resourcessuch as the National Guideline Clearinghouse wereadded. Some adjustments were also made to thesearch hedges, primarily to refine the limits appliedto PubMed searches. For example, searches forPubMed Clinical Studies were disaggregated to pro-vide separate access to articles on therapy, diagnosis,and prognosis, with each of these further limited toEM journals. Most development at this point focusedon the backend tools that would allow librarians, orother managers, to create and configure search toolsfor specific audiences (Figure 3). The goal was to createa framework that would allow librarians or end usersto develop specialized search tools with little under-standing of the mechanics of the EBM Search. Themanagement features supported several basic func-tions: creating a search tool, adding resource group-ings, assigrüng a level of evidence to each resourcegrouping, adding resources to each grouping, modi-fying the labeling of each resource, modifying thesearch parameters of resources, adding resources tomultiple groups, and adding multiple instances of aresource to a single group, each with different searchparameters.

Over the course of three months, the backend toolsneeded to manage EBM Search were developed andrefined through a process in which initial prototypeswere quickly developed and then modified in an iter-

110 J Med Libr Assoc 96(2) April 2008

EBM Search

Figure 2Search and results screen

AHSL Emergency Medicine

C* rw|ulr*il}

And: [

Key Word '(MYOCARDIAL INFARCTION) AND (ASPIRIN)' Saarch Results:bvidence Rank Database Results

Systematic Reviews

PwiwB

Cochrane Systematic Reviews

DARE

PubMed SfStsmaCii: ReviewsHuman/English/last 5 yrs

EJ/Core Clinical

ResuU

Result

2Z5

49

A PubMed Clinical Studies

Therapy;Clinical

Diagnosis:EJ/Coro Clinical

Prognosis; 446EJ/Core Clinical 171

PubMed (All)

AllEl/Core Clinical

46011604

The search resuHs organize sources by relevance to clinical decision categories, and users are linked to pre-executed queries in each source.

ative fashion, as reference librarians reviewed and cri-tiqued initial functionality and suggested next steps.The result was a system that allowed nontechnical in-dividuals to create customized federated search toolswith only one point of intervention from technicalstaff: the addition and configuration of new searchableresources. This phase of development was completedin June 2006, although bug fixes and other minor en-hancements have continued in the time since.

DISCUSSION AND FUTURE DIRECTIONS

The design of the EBM Search too! is important forseveral reasons. First, clinician-initiated queries are en-riched with librarians' knowledge of resources and ex-pert searching, and the tool has the potential to pro-vide an added value for specific target groups by al-lowing convenient but powerful access to clinical evi-dence. Second, the configurability of the toolencourages collaboration between librarians and au-

diences to allow for assessing user need and creatinga tool that meets these needs. Third, adapting the ev-idence ranking system from the Oxford Centre for Ev-idence-Based Medicine [17] potentially reduces thetime needed to conduct critical appraisal. In addition,isolating Cochrane Reviews in the interface permitsusers to navigate to the highest-quality results first, ifthe source is appropriate for the clinical question. Fi-nally, the evidence pyramid layout can function as ateadiing aid, allowing users of the system who areunfamiliar with the principles of EBM to make con-nections between real search results and levels of ev-idence.

While the HBM Search Tool has demonstrated prom-ise, a number of enhancements could improve its ca-pabilities. First, the usability of the interface for eval-uating information sources could be improved. In part,this could be achieved by fetching and displaying re-sult set sizes for more resources. Displaying result setsize is a useful heuristic device allowing the selection

J Med Libr Assoc 96(2) April 2008 111

Bracke et al.

Figure 3Configuration tooi

/\|-|SLSearch Managemerkt

Welcome to tBM S^orrh Managemen

Dspartnient Usti

Back to Inlernal

t DEtall InfomaUo

1 N«ril« • ím i l j om, -fltd'.

ne ' LI _ " ' . i

Short hsmoi It on-

Funliv Madtcin

_ru!jMi[J Clinic»!

âreup Litt

>l It

Dkl/W

•1 Stj

Hb l i

dtii r

HwrtrOHngn

The configuration features allow librarians and other nontechnical stafi to develop their own search tools.

of a resource that appears to meet a desired level ofspecificity, while saving time by allowing users to skipempty sets. In other cases, displaying the number ofresults might be less useful as a point of comparison.For example, knowing the number of results from atextbook search does not allow one to compare it tothe results from an article database. In these cases, al-ternative methods for communicating what a user canexpect by following any given link need to be devel-oped. Technical issues, however, limit the number ofresources for which results set sizes can be easily re-trieved and displayed. Currently, vendor support forweb services or other XML gateways for search andretrieval is limited, and Z39.50-based solutions can becomplex to implement. In the long run, this issueshould be less significant as more vendors supportXML-based services such as search/retrieve via URL(SRU) or the National Information Standards Organi-zation Metasearch XML Gateway Standard (MXG) [19,201.

It is also essential that the tool remain adaptable toevolving clinical workflows. One of the primary rea-sons for the initial positive response to the EBM Searchtool is that its design takes constraints of the clinicalworkflow into account. More factors, however, mightbe taken into account to further improve design in fu-ture iterations. In addition to the interface enhance-ments discussed above, it is also important to considerways in which the search tool can be embedded inother online environments and workflows. For exam-ple, mobile versions of the search tool would be a log-ical future consideration, as would embedding the

search tool in portals, hospital intranets, or other in-formation technology resources that clinicians use. De-spite the possibility of further enhancements, it wouldbe important to maintain the foundational aspects ofthe current design that have led to a service that, an-ecdotally, has provided clinicians with a useful way toquickly access evidence. Continuing to provide a sim-ple search interface and results interface organizedaround known constructs for grading levels of evi-dence will be critical.

REFERENCES

Ï. Chen X. MetaLib, WebFeat, and Google: the strengths andweaknesses of federated search engines compared with Goo-gle. Online Inf Rev 2006;30(4):4]3-27.2. Crawford W. Meta, federated, distributed: search solu-tions. Am Lib 2004 Aug;35(7):80.3. Newton VW, Silberger K. Simplifying complexity througha single federated search box. Online 2007 Jul/Aug:19-21.4. Tennant R. The right solution: federated search tools. LibrJ 2003 Jun 15;128(ll):28,30.5. Notess GR. Building vertical search engines. Online 2007Jul/Aug:37-9.6. Google. Google custom search engine [web document].Mountain View, CA: Google, [rev 2007; cited 13 Oct 2007].<http://www.gaogle.com/coop/cse/>.7. Lang ES, Wyer PC, Haynes RB. Knowledge translation:closing the evidence-to-practice gap. Ann Emerg Med 2006Mar;49(3):355-63.8. Ely m Osheroff lA, Ebell MH, Chambliss ML, VinsonDC, Stevermer JJ, Fifer EA. Obstacles to answering doctors'questions about patient care with evidence: qualitative study.BMI 2002 Mar;324(7339):710.

112 J Med Libr Assoc 96(2) April 2008

EBM Search

9. Campbell R, Ash J. An evaluation of five bedside infor-mation products using a user-centered, task-oriented ap-proach. J Med Libr Assoc 2006 Oct;94(4):435-^l,e206-e207.10. Meats E, Brassey J, Heneghan C, Glasziou R Using theTurning Research Into Practice (TRIP) database: how do cli-nicians really search? J Med Libr Assoc 2007 Apr;95(2):156-63.11. Haynes RB, Holland J, Cotoi C, McKinlay RJ, WilczynskiNL, Walters LA, Jedras D, Parrish R, McKibbon KA, Garg A,Walter SD. McMaster PLUS: a cluster randomized clinical tri-al of an intervention to accelerate clinical use of evidence-based information from digital libraries. J Am Med InformAssoc 2006 Nov/Dec;13(6):593-600.12. Coiera E, Waither M, Nguyen K, Lovell NH. Architecturefor knowledge-based and federated search of online clinicalevidence. J Med Internet Res [serial online]. 2005;7{5):e52.13. Fbust JE, Bergen P, Maxeiner GL, Pawlowki PN. Improv-ing e-book access via a library-developed full-text searchtool. J Med Libr Assoc 2007 Jan;95{l):40-5.14. Mendonca EA, Cimino JJ, Johnson SB, Seol YH. Access-ing heterogeneous sources of evidence to answer clinicalquestions. J Biomed Inform 2001 Apr;34(2):85-98.15. Chattopadhyay A, Tannery NH, Silverman DAL, BergenP, Epstein BA. Design and implementation of a library-basedinformation service in molecular biology and genetics at theUniversity of Pittsburgh. J Med Libr Assoc 2006 Jul;94(3):307-13,el92.16. TRIP Database. Introduction & background Iweb docu-ment]. Cardiff, Wales: Trip Database, ¡rev. 2006; cited 25 Sep2007]. <http://www.tripdatabase.com/aboutus/>.

17. Centre for Evidence-Based Medicine. Levels of evidence[web document]. Oxford: The Centre, [rev. 14 Mar 2003; cited25 Sep 2007]. <http://www.cebm.net/levels_of_evidence.asp>.18. Howse DK, Bracke PJ, Keim SM. Technology mediator: anew role for the reference librarian? Biomed Digit Libr [serialonline!. 2006 Oct;3:10.19. Library of Congress. Search/retrieve via URL [web doc-ument]. Washington, DC: The Library, [rev. 6 Sep 2007; cited13 Oct 2007]. <http://www.loc.gov/standards/sru/>.20. National Information Standards Organization. Meta-search XML gateway ¡mplementers guide version 1.0. Balti-more, MD: The Organization, 2006. (NISO RP-2006-02}.

AUTHORS' AFFILIATIONS

Paul ] . Bracke, MLS (corresponding author), pbracke©purdue.edu. Associate Dean for Information Tech-nology and Resource Services, Purdue University Li-braries, 504 West State Street, West Lafayette, IN47907-2058; David K. Howse, MLS, dhowse®ahsl.arizona.edu. Information Services Librarian, Ari-zona Health Sciences Library, 1501 North CampbellAvenue, PO. Box 245079, Tucson, AZ 85724-5079;Samuel M. Keim, MD, sam@aemrc.arizona.edu. As-sociate Professor, Department of Emergency Medicine,University of Arizona, PQ Box 245057, Tucson, AZ85724-5057

Received August 2007; accepted November 2007

J Med Libr Assoc 96(2) April 2008 113