A metadata vocabulary for self- and third-party labeling of health web-sites:Health Information Disclosure, Description and Evaluation Language (HIDDEL)

G. Eysenbach1), C. KIhler'), G. Yihunel), K. Lampe2), P. Cross3), D. Brickley3) 4)

1) Dept. of Clinical Social Medicine, Unit for Cybermedicine, University of Heidelberg, Germany,email: evyvi.com

2) FinOHTA / STAKES Finnish Office for Health Technology Assessment, Finland3) ILRT Institute for Learning and Research Technology, United Kingdom

4) W3C Metadata Interest Group, MIT, Boston

We describe HIDDEL (Health Information Disclosure,Description and Evaluation Language), formerly knownas medPICS (platform for Internet content selection inmedicine), a metadata vocabulary designed to enhancetransparency, trust and quality of health information onthe web. The vocabulary may be used (1) by webmastersto self-describe their contents and policies; (2) by info-mediaries (eg. Healthfinder, NHS Direct/NeLH), e.g.third party evaluators, rating or portal services, to an-notate other websites; (3) and by users, to describe theirpreferences. As an XML application it conforms to theW3C's RDF Specification. The metadata vocabulary isprimarily intended to enable descriptions ofwhole healthwebsites or health information providers. The vocabularyis designed to provide a computer-readable electronic"label" ofa health website, telling users who is behindthe website, how the website is sponsored, what the con-tent, aim and target audience is, how the information wascompiled, what risks the service bears, or what peoplesay about the resource. Client-software can "read" thislabel automatically, compare it to the user's own set ofpreferences and needs, and alert and advise users.

Introduction

Meta-data on the World-Wide-Web is mostly used asadditional information distributed together with thedocument it describes (i.e. using the META tag inHTML), but there are also technical possibilities to dis-tribute meta-data by third parties independently from thedocument or information provider, by using the W3CPICS Standard (Platform for Internet Content Selection,http:llwww.w3.or&LRDF/). PICS was essentially designedto allow third parties to distnbute meta-data about otherresources, primarily to allow filter software to blockaccess of minors to pornographic and harmfil material.The PICS standard is currently migrating towards be-coming an application of the XML2RDF technology of

the W3C (see http://www.w3.org/RDF/) 1. The successorstandard, RDF (the Resource Description Framework),grew out of work on expanding PICS (then called PICS-NG) to provide for more flexible descriptive capabilities(e.g. textual comments). RDF is a W3C Reconmendationfor Web "resource description" which includes labeling,classification, cataloguing, rating etc. RDF in turn adoptsthe W3C XML Recommendation as a new file format forex-changing such data, replacing the PICS 1.1 format.In this paper we present a self-description and third-partyrating metadata vocabulary for networked health re-sources, expressed for example in PICS, XML or RDF.

Health information providerself-descrbes and discloses properties of the service

HDDEL[medPICS]metadatavocabulary

User Third partydesecbes individual needs (expert, rater)

and expectations descrbes and annotatesInformation and services

Figure 1. HIDDEL (formerly known as MedPICS) isdesigned to be applied by three different user groups

A number of other initiatives are working on developing,identifying and standardizing generic metadata vocabu-laries, for example the Dublin Core (DC) group 2; 3. How-ever, the DC vocabulary is primarily designed to be usedto provide descriptive metainformation supplied by we-bauthors, but not from external evaluators, and does notprovide disclosure elements required to allow consumersto make informed decisions when using a health resource.For example, there are no DC elements allowing the

1067-5027/01/45.000 2001 AMIA, Inc. 169

description of potential conflicts of interest or biases;disclosure of sponsors; purpose of the site, or descriptionof internal quality management procedures. All theseelements have been recognized as being important ele-ments of an "ethical" website.4; 5 Full disclosure is espe-cially important in health care, thus, generic models suchas Dublin Core have thus far not taken up the idea thatdisclosure could be provided as meta-data. Moreover,most of the DC elements are generally more suitable tobe applied on a document level, rather than on a websiteor health information provider level. For example, someDublin Core elements have been developed to describethe date of publishing or last update of a document, butthere are no elements allowing description of the generalpolicy for how often the content is reviewed and updated.Health consumers also should be able to check the levelof evaluation the interactive application has undergone,and to tell what the purpose of the information on thewebsite is (e.g. educational vs marketing) or whetherthere are any gross biases.

vocabulary and framework can then be used on the clientside to code user preferences, allowing automatic selec-tion of suitable information meeting personal qualitycriteria. To fill this gap, we proposed (as early as in1997) a medical core metadata vocabulary set based onthe W3C PICS (Platform for Internet Content Selection)standard, which we called medPICS 6; 7. MedPICS 0.3already contained elements that are evaluative, e.g. ele-ments typically assigned by third parties, as well as self-description which have disclosure character.The central idea behind promoting the widespread use ofMedPICS is that users may express their personal prefer-ences (e.g. "the website should not be sponsored by apharmaceutical company and should target consumers")in a computer-readable metadata format, and the client-software could then automatically and continuously com-pare these preferences with the respective metadata givenby the health information provider (and possibly by thirdparties, e.g. portal sites) whenever a user accesses a site(Fig. 3), or this meta-data can be indexed to allow directsearches for sites meeting a given users preferences.

Client (uTr)

adblt laypezuai

UrnSt

Upr mets

' efrenesS

Figure 2. HIDDEL/medPICS allows health informationproviders to disclose and describe properties oftheir siteor service. Users use the same vocabulary and the samescales to describe their needs or expectations, and thirdparties use the same vocabulary to describe, annotate,comment on, or evaluate sites or services. This enablesthe user to make informed choices and to select siteswhich meet his/her own needs. "Quality" is defined asthe gap between consumer needs/expectations and themeta-data supplied by the information provider and/orthirdparties.

In summary, there is a need to expand the meta-dataconcept and to shift hinking from using meta-data ele-ments only to enhance information retrieval, towards amodel where meta-data elements may also describe dis-closure information, site policies on the host side, andstatements of third parties about other sites. The same

Thrd p.ty Host (Inf. prods)o~~~~f',t grow"

/ a-mit Gayp.:s -

10 ruadixguljcfurcin*~ Vadlagjmwoin-.

Blsalz~suiia

oapt. -sA<oza-;-

pimaL6 goupfcabetwepne ppd

HIbbE/msdFICSencqdsd[lejs1 Adcks" biformation ofth. Iformstlqpmovi.kr+Advice dtthid'fements.

Figure 3. Widespread use ofHIDDEL/medPICS wouldallow users to maximize the perceived quality ofservicesby automatically selecting sites which comply to individ-ual user needs, or get advice ifthey don 't.

Model formulation process

An overview of the development process to date is givenin Figure 4. The vocabulary is still under discussion andpresented here in its version 0.6. The development of theoriginal medPICS vocabulary 0.3 has been describedelsewhere 6. Briefly, it was based on a systematic litera-ture review of the then available articles, reports anddocuments pertaining to quality of health information onthe web, such as evaluation guidelines of OMNI and

170

BHIA; as well as FDA hearings pertaining to cross-

border sales of drugs.

Descripve vocabularies(used by khh pro4der)

EvaluaIve vocabularis(used by "*d pe )

HON EU DESIRE_- \ HM IWhiteP Project

\jp OL id. 1~ [1996

Dublin Core m.dPICS 0.3(1997) Third pD iCore, 1999 4-

(Mai OMl. WHO

ConemnW rkshop Sytamaic rAir

Heidolbeig, 912000e. _

CISMeF imnaiae an tbeweb-o 2000 fb ubm#bdq

MdhIIfMUJ - medPCS / mrdRDF 0.6

I.wIrope wertPAlbc conwihaion P 200120i__2001 -I-_ TC2S1

HiDDEL b CEN/ISO

Compmhenslve vocabulary (002 ?)

Figure 4. Overview of the HIDDEL/medPICS develop-mentprocess thusfar

A further important step in developing the vocabularywas to systematically review the literature for studieswhich already operationalised some of the quality crite-ria. This review was conducted in May 2000An invitational workshop held in September of 2000brought together 80 medical researchers, health profes-sionals, consumer representatives, representatives ofgovernment agencies, inter-governmental and non-

governmental medical associations, librarians and com-

puter scientists participants from 20 different countries todiscuss quality issues and requirements for health infor-mation (see Workshop Proceedings athttp://www.jmir.org/2000/3/suppl2/). The focus of theworkshop was to attempt an operationalisation and im-plementation of proposed ethical guidelines for publish-ing health information on the web 8d1. Breakout groups ofthe workshop tried to establish which "quality criteria"could be operationalised. This involves for example theformulation of a clear question asked to the informationprovider or external evaluator, and the definition of thepossible values a element can take, including the defini-tion of controlled vocabularies. MedPICS is now prefera-bly called HIDDEL to account for the fact that it is a

technology independent vocabulary (not tied to the PICStechnology).

Model description

structure, see. Fig. 5). Examples for basic elements areInfoprovider_feedback email_technical (containing theemail address for technical questions) or Info-provider_feedback_email_content (for content ques-tions).Each basic element can in principle be combined withone of 9 sub-elements, for example CHECKED (What isthe result of the evaluator verifying the element?),CHECKED_REASON (Explanation for CHECKED),PRESENT (Basic element is present on the site), HOW(Description of how users can identify the basic elementon the page(s) themselves).An information provider can for example combine thebasic element Infoprovider feedback_email_content withthe subelement HOW and describe in this element "Thefeedback email is provided at the bottom of each page".A third-party evaluator could use the subelementsCHECKED to confirm this, or to express disagreement.For many (basic) elements, a controlled vocabulary existsto complement the free text value, in this case, elementsend with " CV".Elements can have additional attributes, giving additionalinformation about the respective element, for examplewho entered them and when.In XML, the health information provider could publishon his webpage the following statements indicating hisaddress and the email which should be used for givingfeedback:

<infoprovider><feedback>

<address>Bergheimer Str. 58, Heidelberg

</address><email> feedback@mysite.com </email>

</feedback></infoprovider>

Rating services (gateways) evaluating some of thesestatements could publish a statement like the following:

The vocabulary consists of about 50 so-called basic ele-ments, which may be presented as a taxonomy (tree

171

<infoprovider><feedback>

<adress>Bergheimer Str. 58, Heidelberg<checked by=medcertain team'>

Positive</checked></address><email> feedbackemysite.com<checked by=Nmedcertain teamNcreator-NGuntherN date='22.1.2001N>Negative<reason creator=NGuntherN>Email was bounced! </reason>

</checked></email>

</feedback></infoprovider>

Figure 5. First two levels ofthe tree structure ofHIDDEL/MedPICS basic elements

Pilot testing within MedCERTAIN

The vocabulary is currently going through a public con-

sultation phase and will partly be tested in the context ofthe MedCERTAIN (MedPICS Certification and Rating ofTrustworthy Health Information on the Net) project 12.

MedCERTAIN will help to implement this infrastructure,educate consumers and provide a certain level of moni-toring to assure that meta-data is not abused for market-ing purposes. One aim of the MedCERTAIN project is tomake implementation for health information providerseasy, even if they lack technical experience: They justhave to fill in a form at MedCERTAIN and the metadatawizard will create a set of metadata for them. A so-calledMedCERTAIN level 1 "transparency mark" is awarded,if certain metadata fields are submitted. In a level-2evaluation, meta-data will be verified, and a level-3evaluation means that the content has been checked,approved, recommended or certified by third parties suchas medical experts or societies (this information can alsobe harvested from trusted gateways, as described below).The MedCERTAIN project will generate further data on

the comprehensiveness, applicability and reliability of thevocabulary.

Figure 6. Harvesting options.

Harvesting opinions

HIDDEL/MedPICS descriptions may either be publishedby health information providers themselves (Fig. 6, lefttop) or may be published by portals or gateways e.g. asXML annotated lists (Fig. 6, right top). Harvesting soft-ware may aggregate this meta-information to compile an"open directory" of site descriptions, and the meta-infor-mation can be fed into search engines.

HIDDEL/MedPICS labels as benchmarking indica-Discussion tors for measuring achievements in public (e-)health

The different envirouments for which we envisage HID-DELiMedPICS to be used are (1) self-rating by informa-tion providers using XML: (2) self-rating by informationproviders for level-l certification in the MedCERTAINcontext 12; (3) publication of evaluations, descriptionsand annotations by third-party rating services asXMLIRDF; (4) evaluation of websites by experts / expertcomunities using the HIDDELIMedPICS vocabulary inthe MedCERTAIN context; (5) consumers expressingtheir information preferences using browser add-ons.

The U.S. Department of Health and Human Services(HHS) has articulated the need for website "labels" in itsreport Wired for Health and Well-Being 13, though it notnecessarily referred to meta-data. The HHS also hasmade it a national health objective to increase the numberof health Web sites that disclose certain information 4; 14With sites using electronic HIDDEL/MedPICS labels,progress in the area can be easily measured and quanti-fied. For example, automatic search engines may deter-mine how many websites disclose the infonration de-manded in the Healthy People 2010 document 14 asHIDDEL/MedPICS meta-data.

172

Client-side tools for downstream filteringAnother application scenario is the development of ap-propriate client-side tools which will enable users to settheir own preferences, enabling client and host to "nego-tiate" (possibly also integrating third-party opinions fromother sites), and to let the browser display warnings andalerts if a host does not comply to the personal prefer-ences or "quality criteria" of the user (Fig. 3).

Conclusion

In summary, we think that widespread implementation ofthis vocabulary may enhance trust among consumers intonetworked health resources, enable e-health providers topublish disclosure infortion in a standardized way,assure interoperability of rating services and gateways,and promote an infrastructure which enables consumersto make informed decisions. This "quality" managementapproach is - other than many "seal of approval" ap-proaches - perhaps the first approach which is able totake into account that different user preferences andneeds inevitably lead to different notions of what actunllyconstitutes "quality".

A link to the most recent version ofHIDDEL/MedPICSas well as additional documentation can be found athttp.//www.medcertain.org/metadata/index.htm

Acknowledgements

Partly funded by the European Union, Action Plan forSafer Use ofthe Internet, MedCERTAIN project.

References

1. Swick RR., Brickley D. PICS Rating Vocabularies inXMLJRDF. 27-3-2000. URL: http://www.w3.org/rdf-pics

2. Weibel, S., Kunze, J., Lagoze, C., and Wolf, M. Dublin CoreMetadata for Resource Discovefy. Intemet RFC 2413.RFC2413 . 2000. URL: http.//www.ietf.org/rfc/rfc2413.txt

3. Malet G, Munoz F, Appleyard R, Hersh W. A model forenhancing Internet medical document retrieval with 'medicalcore metadata.". JAm Med Infomn Assoc 1999;6:163-172.

4. Baur C, Deering MJ. Proposed Frameworks to Improve theQuality of Health Web Sites: Review. MedGenMed 2000;2:URL:http://www.medscape.com/Medscape/GeneralMedicine/joumaV/2000/v02.nO5/mgmO926.baur/pnt-nmgm926.baur.html

5. Eysenbach G, Sa ER, Kuss 0, Diepgen TL. A framework forevaluating e-health: Systematic review of empirical studies as-sessing the quality of health information and services for pa-tients on the Internet. submitted 2001;

6. Eysenbach G, Diepgen TL Labeling and filtering of medicalinformation on the Internet. Methods InfMed 1999;38:80-88.URL:http://www.yi.com/home/EysenbachGunther/publications/1999/Eysenbachl999e_Meth1nfMed.pdf

7. Eysenbach G, Diepgen TL. Towards quality management ofmedical information on the internet: evaluation, labelling, andfiltering of information. BMJ 1998;317:1496-1500.URL: http://www.bmj.com/cgi/content/fulV317/7l71/1496

8. e-Health Ethics Initiative. e-Health Code of Ethics. J MedInternet Res 2000;2:e9. URL: http://wwwjmir.org/2000/2/e9/

9. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On theNet Code of Conduct for medical and health Websites. Com-put BiolMed 1998;28:603-610.

10. Hi-Ethics Group. Health Internet Ethics: Ethical Principles ForOffering Intemet Health Services to Consumers. HIETHICS.2000. 3-12-2000. URL: http.//www.hiethics.orPrinciples/

11. Winker MA, Flanagin A, Chi-Lum B, et al. Guidelines forMedical and Health Infomution Sites on the Internet. JAMA2000;283:1600-1606.URL:http://jama.ama-assn.org/issues/v283n12/fulVjsc00054.html

12. Eysenbach G, Yihune G, Lampe K, Cross P, Brickley D.MedCERTAIN: Quality Management, Certification and Rat-ing of Health Information on the Net. Proc AMIA Symp2000;230-234.

13. Science Panel on Interactive Conmnunication and Health.Wired for health and well-being; the emergence of interactivehealth conmunication. Eng, T. R. and Gustafson, D. H. 1999.Washington, DC, US Department of Health and Human Serv-ices, US Govemment Printing Office.

14. Healthy People 2010: Understanding and Improving Health(Second Edition). 017-001-00547-9. 2000. Washington,DC,Office of Disease Prevention and Health Promotion, U.S. De-part of Health and Human Services.URL: http://www.health.gov/healthypeople/docummntI

173