MYCIN II: Design and Implementation of aTherapy Reference with Complex Content-Based Indexing

David K. Kim MSMayo Medical School, Rochester, Minnesota; kim.david@mayo.edu

Lawrence M. Fagan MD, PhDKathleen T. Jones

Daniel C. Berrios MD, MPHStanford Medical Informatics, Department of Medicine, Stanford University School of

Medicine, Stanford, California; (fagan, kones, berriosl(smi.stanford.eduVictor L. Yu MD

Division of Infectious Disease, University of Pittsburgh School of Medicine, Pittsburgh,Pennsylvania; vly+@pitt.edu

ABSTRACT

We describe the construction ofMYCIN II, a prototypesystem that provides for content-based markup andsearch of a forthcoming clinical therapeutics textbook,Antimicrobial Therapy and Vaccines.Existing commercial search technology for digitalreferences utilizes generic tools such as textword-basedsearches with geographical or statistical refinements.We suggest that the drawbacks of such systemssignificantly restrict their use in everyday clinicalpractice. This is in spite ofthe fact that there is a greatneed for the information contained within these samereferences.The system we describe is intended to supplementkeyword searching so that certain important questionscan be asked easily and can be answered reliably (interms ofprecision and recall). Our method attacks thisproblem in a restricted domain of knowledge--clinicalinfectious disease. For example, we would like to beable to answer the class of questions exemplified by thefollowing query: "What antimicrobial agents can beused to treat endocarditis caused by Eikenellacorrodens?"We have compiled and analyzed a list ofsuch questionsto develop a concept-based markup scheme. Thisscheme was then applied within an HTML markup toelectronically "highlight" passages from three textbookchapters. We constructed a functioning web-basedsearch interface. Our system also provides semi-automated querying of PubMed using our conceptmarkup and the user's actions as a guide.

KEYWORDS: Information retrieval; electronic textbooks;drug therapy, computer assisted; hypermedia; abstractingand indexing; content-based markup (non-MeSH).

INTRODUCTION AND BACKGROUND

Physicians in clinical practice have an ongoing need foraccess to diverse sorts of information. They also facesignificant barriers in obtaining that information. Covellet al.' found that a significant number (70%) of questionsarising during an outpatient visit to an internist's officewent unanswered. More than half of these questionswould be potentially answerable with consultation ofreliable, quick, up-to-date tertiaryt references. Notableproblems include: Lack of adequate organization orindexing, concern that accessible textbooks were out ofdate, and lack of print sources that would answer thequestions they asked.

What kinds of questions are these? In the Covell study,requests were dominated by practical management andtherapeutic issues; approximately 70% were regardingtreatment, diagnosis, and drug therapy. Other studies ina variety of practice settings2 3 4 support this claim.

There is a need for current, well-indexed, clinicaldiagnostic and therapeutic references. Manyauthoritative textbooks are available in CD-ROM form.These use graphical user interfaces with hierarchicalnavigation as well as textword search facilities. To usesuch tools, one must use keywords to "triangulate"passages of interest (for example, "eikenella","endocarditis", and "therapy -OR- treatment"). It is ouranecdotal experience that such tools are rarely used inpractice, despite their intuitive promise. Existing CD-ROMs do not help enough to make it worth finding and

t In this context, tertiary refers to texts and edited compilationsof review material.

1091-8280/98/$5.00 0 1998 AMIA, Inc. 175

using them in a clinical setting. Possibly contributing tothis inadequacy are problems inherent in textwordsearches; even those relying on statistical weighting orgeographical strategies suffer from several difficulties.These include a trade-off between precision and recall*as well as matches lost to synonymy and pronominalreferences5. Reliability of search results also dependsupon searching skill, which undermines the benefit ofusing an authoritative reference such as a textbook. Wehave attempted to remove a fundamental contribution tothese problems by pre-assigning semantic meanings tosections of text.

Two of the authors previously helped constructMYCIN6, an expert system to assist in antimicrobialtherapy choice. Experience with this formal knowledgebase thus influenced the resulting hybrid approach,which attempts to impose some of the ontological'('definitional") structure of an expert system on a nearlyfree-text data source. Bernstein, et al., arguablypioneered the concept of computer-assisted compilationof a structured-text knowledge base in creation of theHepatitis Knowledge Base7. In our case, there is aconstraint on the source text in terms of content--Editorscollaborated with authors to produce a fixed topicaloutline for chapters to help ensure that answers to thosequestions deemed important would be presentsomewhere in the text. Authors were required to includespecific recommendations in several areas (e.g.antimicrobial therapy). Bottom-line recommendationswere stressed; in this sense the chapters are motivated bythe idea of a hospital "consult" where a specialist iscalled in to render the best opinion possible for givendata. We feel that such specialist opinion, even incanned form, will be useful as long as it is authoritative;the Covell study' noted among internists that 69% of thequestions they posed fell into a subspecialty outside theirown.

In our system, ontological structure takes the form of asemantic markup. This markup indicates occurrence notonly of certain facts or terms in the text, but also of thecontext in which they occur. The application of questionand context models to assisted searching of the primarymedical literature has been explored in depth byPurcell8 9 '1. MYCIN II builds on Purcell's work,

* Precision=(# of relevant hits)/(total # of hits) Thiscorresponds intuitively with the clinical concept of specificity.Recall=(# of relevant hits)/(# of relevant hits in overalldatabase) This corresponds intuitively with the clinicalconcept of sensitivity.I Gr. Onto- having to do with existence. An ontology, in ourrestricted sense, is a conceptualization or enumeration of terms,contexts, or properties which we wish to label and with whichwe wish to reason.

notably in its starting assumption of coverage of adomain of knowledge by a limited number of questiontypes.

The design of our system provides for relatively simplelinking to external databases. Specifically, the systemconstructs and submits searches to Medline. This hasbeen implemented previously by Cimino et al." A linkto primary literature serves as an important supplementwhen considering the problem of potentially out-of-datetextbook information. Providing abstracts in this wayserves to provide "just-in-time information"'2 at aminimal extra cost in the user's time.

DESIGN CONSIDERATIONS

Our facility is designed to efficiently steer a user torelevant passages of a text when asked one of apredetermined class of questions. We also wish toprovide facile linking from these search results toexternal resources. By first building a model of what theuser is interested in, we extend the text with a structureddescription of its contents. We use this description in thematching process in order to reduce the drawbacksmentioned above.To this end, we first require adequate coverage ofconcepts by our list of questions. These questions do not,by design, cover all topics that are implied by the set oftextwords in the document. The process by which these"canonical" classes of questions and the resultingmarkup are created must be such that a diligent pollingof experts will yield a representative basis. I.e., we mustelicit those concepts whose identification in the text willallow the handling of the large majority of eventualquestions. In this way, we tacitly generate a minimalontology for the domain at hand.

System Description

Figure 1: Block Diagram of System

176

Figure 1 contains a block diagram of the overall system.The marked text takes the form of HTML with semanticmarkup applied within comment tags. The search engineand front-end interface were implemented in Perl usingthe Common Gateway Interface.

As discussed, authors and potential users of the bookAntimicrobial Therapy and Vaccines'3 were asked whatsort of questions the system ought to answer. This listwas analyzed to group like questions together intoclasses, for example: "What antimicrobial therapy canbe usedfor ORGANISM X in INFECTION TYPE Y?"Passages that answer this question need to specify thedrug to be used (DRUG="ceftriaxone") and must refer totreatment (INTENT=TREAT). These essential termsand contexts (e.g., DRUG and INTENT=TREAT) wereextracted by inspection and used to define our semanticmarkup. We refer to these definable concepts as tagparameters below. Three chapters, two dealing withbacteria and one dealing with an antimicrobial agent,were marked by one of the editors for the initial versionof this system.

Figure 2 shows a query screen from our current systemand the parameter/value pairs it explicitly and implicitlygenerates. These will be used to search through themarkup to identify matching text passages.

What antmcrobid agedts) can be used In rent of ll~ikeneila coocdens jT'

Rl Rictiantwenaccordingto the followg In TF

N ~~~~~~~~~~OIGt~eikeneIla corrodensEfficacyof: th-r-pEFFedinbest or'infirsfBest orra- INFed l

rNot Recomended orcaidied (rdhpcfyFind conibbration onlyo.r AE EAM

* Tlefloatioa ofi....f.otion..[ !n.d..r..si* l °Underlying d ase NRESTRICTED? ......................11

......................................................

Figure 2: User's Query Screen

An example of a marked section of text is shown inFigure 3. It is taken from a section on Eikenellacorrodens endocarditis. Answering passages are flankedby begin <!-- MARK ...--> and end <!-- /MARK-->markers. The markup for both passages contains the tagvalues corresponding to the initial query, as well as thesame specific drug names (DRUG="ceftriaxone",etc.)discussed. They differ in efficacy and order ofrecommendation--This regimen is considered best(EFF="clinbest") for penicillin allergic patients(CONdition="allergy to penicillin") but is first line(EFF="'clinfirst") overall.

Figure 3: Marked Text For Example

Figure 4 shows the results screen that the user sees afterthe local textbook search is executed. It is organized in a

master-detail fashion; selecting See Excerpt for any

match ("hit") shown on the left will display that portionof the document in the right-hand frame.

*ced Ja dfr awao of Lkwp.

lustory 1pe HpI aud .iJlu, not mnspylacic-type.a veinas (orpma inwbo ovpatflh sm isbeig consid:. Ahe e-cse ofprotc vamemndocria to orgammhave boon curedwith*ediscmcAhtrepla~enet

Notice also that each match provides its own Medlinequery option. Clicking on this brings up a guided querypage (Figure 5), containing options for query termstaken from the markup surrounding the passage ofinterest

F s~~z , as__s,

se na-f.ae wmam.. .....

m~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~f

Figure 5: Guided Query Screen

177

Ah<-MclNIRr=lTEWgCr ThH3=M44N*EAwidu cac Ai dagsdk l"oct 'bnti~ii'

4M5Am*,yatd*dipmiadocMxp sumrchaofdzworraddbeAdWdhWd Ix voaric fora imraftifdny.

4-MAMGNIBNr=TREAC4r ThMi3AM#N4&dwwpmi Ickehd,"a cW- "celionl fbn6'

clpakrto iaihi' ->Itisispdilytfrbstre rapjtinvEiialisxydTypeHatdhiaknIeinotkt-)-MgadaforerinWnnalpatinrtbknuisbciuoitbvd.

4-&AMWI~>

Guided Medline QueryOptm .bnbdowbedesZrXumtynuueh Baum, -iorkdasdymb qsieyouukatd

t.imAsc s&w&*.duIstcck&btb1dR*ro'.w_Yksmqddadbdeas6

Therapy Choice:,, Gl, ._ ...............................

............. ..........................................................................

i

sss--_-

I||||_E

s

!

Options are pre-selected based upon the user's initialquery. Upon submission, the user-selected query termsare mapped to search phrases for the target system,PubMed.

The system described provides the following facilities:1. Searching based on context and meaning. The

context is specified as a tag parameter and thus isnot dependent upon document structure. Toillustrate, a particular treatment recommendationcould be mentioned in the Contraindications sectionrather than Therapy (e.g., tetracycline for apenicillin-allergic patient). The meaning of a wordor phrase may be obscured from simple matching byvariant vocabulary or pronominal reference, e.g."clavulanic acid" or "it" in a sentence referring to"clavulanate".

2. Facile linking to external, more current informationresources. This provides a "bootstrapping" functionbetween from our system to PubMed without theuser's needing to define a keyword search strategy.

DISCUSSION

Production of Markup. We began with three chaptersand a large number of questions. Our anecdotalexperience was that the markup schema required fewerchanges as more questions were added. We believe thatthis rapid convergence is suggestive both of coverageand consistency.To date, the process of applying the markup schema hasnot been prohibitive, requiring approximately 1-2 hoursper chapter for an infectious disease expert. Relatedwork is underway at Stanford to produce a tool to assistin this markup process, especially for sentence-by-sentence markup'. The current version assistsmechanically in the process, but the ultimate goal is toprovide a partially-automated process that providessuggestions from the text itself via an informationextraction system.

General Issues Regarding The Markup. In the presentsystem, markup has taken the form of inline, human-readable, HTML-like annotation. Such markup wouldbe conceptually equivalent to an external indexfile withtokenized references to vocabulary terms (e.g. "org269'instead of "Eikenella corrodens") and pointers "into" thesource text. If a standardized inline markup (say,XML/SGML) were used, the full inline form might beretained through publication to allow ad hoc informationprocessing. It would be directly usable and inspectableby the end user (perhaps in an SGML-aware browser)instead of serving solely as a vehicle for answering thosequestion-types identified in advance. Such an exposureof the underlying content-based index could more easilyallow interoperability with local systems, although this is

not an absolute requirement (token translation tablescould be used). Examples include:* Use of patient descriptors from a computer-based

patient record as query specifiers.* Use of markup to link to local resources (pharmacy

cost data, locally licensed clinical references, orderentry gateways, etc.)

In such a case, the scope of the markup might have to bemore carefully defined, thought-through, and validatedagainstpossible uses.A second issue involves use of the representation choiceas a conceptual model. Conceiving of the sets of markupparameters as statements or assertions, rather than asoverlapping regions of text attributes, has been helpful indesigning the markup schema and semantics. Potentialambiguities or implied contradictions were most easilyrecognized when reflexively asking the question, "Whatstatements are we able to generate from this markup'vector"'?'" It is important to identify such problems inadvance. For example, we noted that separately-interpretable or "alternate" tagging of the same text issometimes needed. In the inline-markup model, thiscorresponds to nested tagging without inheritance. Inthese cases the same text needs to be marked up in two(non-combinable) contexts. We do not explore here themany other issues of syntax and semantics encounteredduring this work.

Physical Density of Markup. The question naturallyarises, "Should we mark every sentence, or even everysentence that answers a given question?" Instead, onecould mark only occasional, exceptional, very important,or non-obviously placed passages. In MYCIN II, wehave chosen consistent question-driven coverage--If afact was specifically elicited from the authors, and/orfalls into our list of canonical questions, all answeringpassages are marked. Physical granularity was usually atthe sentence or paragraph level, but occasionally phraseswere separately marked.

Breadth of Conceptual Coverage. We have attemptedto construct a markup that anticipates all reasonablequestions in our givenfield, not just all specific questionswe have gathered (our minimal goal). For this reason,any mention of antimicrobial therapy (whether of anamed or unnamed organism, whether it is a vagueguideline or a specific dose recommendation) is markedINTENT=TRE and AGENT_TYPE=AMA (Anti-Microbial Agent). This anticipates extensions andrevisions of existing questions.

External Search Benefits of Markup. To pose a queryto a journal database such as Medline, a user generally

II by "vector" we mean a tuple of parameter name/value pairs

178

must enter or select search terms. Some systems (e.g.PubMed) keep a record of document descriptors, eitherfrom prior search requests or tacitly derived fromreturned documents. This allows for "bootstrapping" ofnew searches via such offerings as "see Related articles","Smart search", etc. As shown above, such internalinformation is used by MYCIN II when bridging to aPubMed search (Figure 5). This makes use of theoriginal information the user has entered as well as his orher response to subsequent prompts. These prompts arethemselves based on search results deemed to be ofinterest. In this "navigational" picture, an interactionbetween the question posed and the markup underlying aselected example record creates search parameters forthe external search. It should be mentioned that this wasnot an initially anticipated use of the markup but wasreadily adopted! We expect increased precision andrecall also in the external (PubMed) linking and that thiswill be most noticeable in the cases of quick searchesand less-experienced users.

SUMMARY

We have constructed a system with the followingfeatures: A fixed set of question types, an extensibleembedded markup which provides content-basedindexing, matching based on context and meaning, andthe ability to answer questions and link to externalinformation resources with the query structure intact.

AcknowledgmentsWe acknowledge Edward Shortliffe for developing theoriginal MYCIN (and allowing us to use the nameMYCIN II) and for helping create the laboratoryenvironment in which both projects were accomplished.David Kim thanks Stanford Medical Informatics forgraciously hosting him during this work. This work wassupported in part by contract 5N44-C061025-001 fromthe National Cancer Institute to Lexical Technology Inc.

References

1. Covell DG, Uman GC, Manning PR. InformationNeeds In Office Practice: Are They Being Met?Ann Intern Med 1985; 103:596-599.

2. Northup DE, Moore-West M, Skipper B, Teaf SR.Characteristics of Clinical Information-Searching:Investigation Using Critical Incident Technique. JMed Educ 1983; 58:873-88 1.

3. Williamson JW, German PS, Weiss R, Skinner EA,Bowes F. Health Science Information Managementand Continuing Education of Physicians. AnnIntern Med 1989; 110:151-160.

4. OsheroffJA, Forsythe DE, Buchanan BG,Bankowitz RA, Blumenfeld BH, Miller RA.Physicians' information needs: analysis of questionsposed during clinical teaching. Ann Intern Med1991; 114(7): 576-581.

5. Salton G, Buckley C. Global Text Matching ForInformation Retrieval. Science 1991 253(5023):335-343.

6. Wraith SM, Aikins JS, Buchanan BG, Clancey WJ,Davis R, Fagan LM, Hannigan JF, Scott AC,Shortliffe EH, van Melle WJ, Yu VL, Axline SG,Cohen SN. Computerized consultation system forselection of antimicrobial therapy. Am J HospPharm 1976 Dec;33(12):1304-1308.

7. Bernstein L, Siegel E, Goldstein C. The HepatitisKnowledge Base: A Prototype Information TransferSystem. Ann Intern Med, 1980; 93:169-18 1.

8. Purcell GP, Shortliffe EH. Contextual Models ofClinical Publications for Enhancing Retrieval fromFull-Text Databases. Proc Annu Symp Comput ApplMed Care. 1995;: 851-857.

9. Purcell GP, Mar DD. SCOUT: InformationRetrieval From Full-Text Medical Literature. ProcAnnu Symp Comput Appl Med Care 1992;:91-95

10. Purcell GP. Contextual Document Models forSearching the Clinical Literature. Ph.D. Thesis

11. Cimino JJ, Johnson SB, Aguirre A, Roderer N,Clayton PD. The MEDLINE Button. Proc AnnuSymp Comput Appl Med Care 1992;:81-85

12. Chueh H, Barnett GO. "Just-In-Time" ClinicalInformation. Acad Med 1997 Jun;72(6):512-517

13. Yu VL, Merigan TC, and Barriere S, Edwards G,itors. Antimicrobial Therapy and Vaccines.Williams & Wilkins. In Press.

14. Berrios DC, Kehler A, Kim DK, Yu VL, Fagan LM.Automated Text Markup for Information Retrievalfrom an Electronic Textbook of Infectious Disease.Stanford Medical Informatics, Technical Report No.SMI-98-0714, Stanford University, Stanford, CA,1998.

179