www.hon.ch

Non Governmental Organization

Quality of health related information on the Web

Health On the Net Foundation: assessing the quality of health Web page

all over the world

Celia BoyerHealth On the Net Foundation

International Consultation meeting on Global Health Library development

Geneva 26-27 November 2009

www.healthonnet.org

Outline

• Health website certification

• The HONcode database

• Search engine technologies of HON

• Further developments

www.healthonnet.org

On the Internet nobody knows who you are

Uneven quality of health information online

Quantity of information

available

Healt

h O

n t

he N

et

20

09

www.healthonnet.org

Organisation Non Gouvernementale

Founders:

Professor Jean-Raoul Scherrer, Dr. Donald Lindberg (director of the NLM, Bethesda, USA),

Professor Ron D. Appel (director of the Swiss Institute of Bioinformatics), Professor Marion

Ball (John Hopkins School of Nursing et IBM Research), Professor Jan van Bemmel, Mr.

Michel Carpentier (former director of the DGXIII), Mr. Jean-Claude Peterschmitt (former

director Digital Europe), M. Guy-Olivier Segond (Conseiller d'Etat, Genève, Suisse).

President: Professor Antoine Geissbuhler

Team:

Composed of physicians, biologists, computer scientists, quality assuror,

lawyer

www.healthonnet.org

The mission of Heath On the Net Foundation

To guide Internet users by highlighting reliable, comprehensible,

relevant and trustworthy sources of online health and medical

information

Since 1995, HON has been tackling the major obstacles of

Internet usage: the overwhelming quantity of information and

the uneven quality of health information available online

www.healthonnet.org

… specific for health websites … applicable for all

sites

Various initiatives for controlling the quality

www.healthonnet.org

1995 UK OMNI* 19 themes–> 80 questions. A directory of unaudited sites.

1996 CH HONcode 8 principles – free online application with a simple questionnaire –> manual review and annual audit.

1998 UK Discern* 16 questions (unaudited self-evaluation)

1997/99 F Netscoring* 49 criteria, 312 points (French only)

1998/99 Es WebMedica Based on existing codes (Spanish only)

2000 D MedCertain* Level 1 –> 72 questions

2001 F Toucanomètre* French webmasters MD –> 17 items; 40 points

2001 US URAC 7 standards –> 52 questions. Cost $13’000.

2002 EU Guidelines* European Commission 6 directives. Created with participation of HON

2003 D AFGIS Code Membership fee (German only)

*little or no implementation

www.healthonnet.org

The HONcode

Sponsorship Clearly distinguish advertising from editorial

content

Authoritativeness

Complementarity

Privacy

Attribution

Justifiability

Transparency

Financial disclosure

Indicate the qualifications of the authors

Information should support, not replace, doctor-

patient relationship; Mission and Audience of

siteRespect the privacy and confidentiality of

personal data submitted to the site by the visitor

Cite the source(s) of published information and

dating of medical and health pages

Site must back up claims relating to benefits

and performance

Accessible presentation, identities of editor and

Webmaster, accurate email contact

Identify funding sources

http://www.HON.ch/HONcode/ - eEurope 2002: Quality Criteria for Health related Websites

8 principles based on

elementary ethical criteria

Easy to understand and to

implement

Voluntary and pedagogical

Free of charge certification and

multilingual

Unique HONcode seal attesting

of the certification by HON

www.healthonnet.org

How to gauge the reliability and accuracy

Establish standards and guidelines

Certification• General need for clear and transparent

standards for disseminating health information

• Both external justification of practices and internal quality management

• ‘Looking over the shoulder’ provides an incentive for further work

The

HONcode

Certification

www.healthonnet.org

The Web publisher submits a request to be certified and to obtain the right to display the HONcode seal

The Web publisher fills in an online questionnaire

The HONcode review team conducts a thorough evaluation of the website

According to findings, website is:certifiedcertified under condition to modify/add some statement on the websitenot certified

If certified, a unique HONcode seal is granted

Each website is re-evaluated each year

The certified website is integrated into the database of HONcode certified websites, HONcodeHunt

HONcode certification protocol

www.healthonnet.org

HONcode certification protocol

Websites are evaluated according to the conformity assessment document.

The outcome of the certification is stated in the HONcode certificate. The HONcode seal is issued when the website is found to be in compliance.

The HONcode seal is valid for one year. The website is re-assessed before the end of the period.

All violations of the Code must be remedied. Webmasters receive guidelines and instructions from HON.

The webmaster pledges to respect the HONcode during the time the website displays the HONcode seal. The webmaster must notify HON when the website changes significantly.

The assessments are confidential, except for the information that must be published in the HONcode database on HON’s website.

HON is accredited according to the adapted EN 45011/ISO65 norm

www.healthonnet.org

Monitoring

Automatic and manual monitoring of certified websites or those in

the process of certification

Automatic links tracking

Automatic identification of significant content changes at different

times

Civic sense of the surfer’s responsibility: online complaint reporting

www.healthonnet.org

The HONcode database: HONcodeHunt

+ 7’900 sites

102 countries

35 languages English: 45%French: 17%

1,400,000 web pages certified in Google.com

Web pages characterised by:compliance with the HONcode principlescorresponding excerptsindexed with MeSH terms and HON labelscontent labels (women’s health, information for patients, videos, images...)

Examples:ClinicalTrials.govHealthInsite.gov.au10000steps.org.aubetterhealth.vic.gov.auThe National Urban LeagueMEDLINEplus.govPubMedStop-tabac.chHospital 12 de OctubreSociedad Argentina de PediatríaDiabetes Netzwerk Deutschlande-sante.frchu-rouen.freurordis.org

www.healthonnet.org

HON certification

Government-level collaboration in France: HONcode is the

official international standard for quality of health websites

in France since November 2007.

Intended for ALL the health Web publishers

Voluntary initiative

For the citizen

For health care providers, pharmaceutical and the eHealth

industries

www.healthonnet.org

Collaboration with the French National Authority for Health

Article L 161-38 de la loi n° 2004-810 du 13 août 2

004 relative à l’assurance maladie :

La Haute Autorité de santé est chargée d’établir une

procédure de certification des sites informatiques dédiés à

la santé, et des logiciels d’aide à la prescription médicale

ayant respecté un ensemble de bonnes pratiques.«À

compter du 1er janvier 2006, cette certification est mise

en oeuvre et délivrée par un organisme accrédité

attestant du respect des règles de bonne pratique

édictées par la Haute Autorité de santé.

HON has been chosen by the HAS to implement the certification of French

health websites according to the French bill passed on the 13 August 2004.

This certification

Will be performed by HON through the implementation of the HONcode.

Décret n° 2004-1139 du 26 octobre 2004

relatif à la Haute Autorité de santé

Article R 161-75

La Haute Autorité de santé détermine les règles

de bonne pratique devant être respectées par les

sites informatiques dédiés à la santé et les

logiciels d’aide à la prescription médicale pour

lesquels la certification mentionnée à l’article L.

161-38 est demandée. Elle définit les modalités

de cette certification.

www.healthonnet.org

Evolution of the HONcode

Proposition and acceptance of the HONcode2.0

http://www.hon.ch/HONcode/Webmasters/Guidelines/

guidelines_web2.html

www.healthonnet.org

Results of the survey

It is mandatory to establish rules for collaborative platforms (Web

2.0)

89% agree

The rules governing collaborative platforms as proposed by HON

have a positive impact

86% of respondents believe that the rules of conduct for collaborative platforms

have a positive impact on the use of the Web.

Additional guidelines for Web 2.0 applications are:

Useful for the majority of webmasters (87% total and 91% of Web2.0)

19

www.healthonnet.org

Recognizing diversity in a globalizing world

“Just because someone is easier to reach does not mean they are

easier to understand. The opposite, in fact, is true. » Dominique

Wolton

Regional offices for the HON foundation:

French-speaking Africa: Bamako, Mali

English-speaking Africa: collaboration with the Medical Research Council Johannesburg, South Africa:

Spanish-speaking countries: Valencia, Spain

Arabic-speaking countries: collaboration with the King Saud Bin Abdulaziz University for Health Science (KSAU-HS) in Saudi Arabia

www.healthonnet.org

International recognition

In May 2004, HON won the

«eEurope Award for eHealth» presented

by the European Union and the EU

Commission

In 2002, HON Foundation was recognized

as a non-governmental organization and

granted special consultative status with

the United Nations Economic and Social

Council

www.healthonnet.org

Search engine technologies of HON

www.healthonnet.org

MARVIN the crawler

Web crawler (Baujard et al, 1998)Multi-Agent Retrieval Vagabond on Information Networks

Use of MeSH* thesaurus and other medical dictionaries

Detection of medical and health related Web pages

Semi-automatic selection of most relevant health and medical Web pages

Several languages processed

English, French, Spanish, Italian, German, Portuguese, Danish, Dutch

*MeSH: Medical Subject Headings

www.healthonnet.org

HON multilingual medical term extractor

Use of MeSH terminology: indexing and information retrievalmultilingualhierarchically structured

Extraction of medical terms synonymslexical normalisation

Weighting of medical terms co-occurrences with other conceptsweight of relations between concepts

Evaluated as the best performing medical term categoriser amongst French systems (Névéol et al, 2005)

Search system continually improved based on analysis of HONcode reviews and audits

www.healthonnet.org

www.healthonnet.org

http://www.hon.ch/Project/HONmali/

Thematic search engines: French-speaking Africa

www.healthonnet.org

www.healthonnet.org

Thematic search engines: English-speaking Africa

www.healthonnet.org

Thematic search engines: Diabetes certified websites

A collaboration between the

Division of Clinical Informatics

of the Harvard Medical School,

Joslin Dubai Diabetes Centre

and HON

www.healthonnet.org

Thematic search engines: Toxicology certified websites

www.healthonnet.org

Thematic search engines: Visual diseases

www.healthonnet.org

www.healthonnet.org

How to judge the information on the Web?

www.healthonnet.org

Specialised search engine

EU project: IST-2001-33260

www.healthonnet.org

www.healthonnet.org

Targeted search engine

www.healthonnet.org

Specialized search engine

www.healthonnet.org

Experimental Question answering system

how to treat diabetes

www.healthonnet.org

Identification of HONcode certified sites

www.healthonnet.org

www.healthonnet.org

Identification of HONcode websites in Wikipedia

www.healthonnet.org

Identification of HONcode websites

Through the HON toolbar available to everyone, one can easily find and search all sites certified by HON.

http://www.hon.ch/HONcode/Patients/Plugin/Plugins.html

Accredited

Not accredited

Accredited web sites

Search engine

www.healthonnet.org

Mobile Health: HON applications

MedRomand

An application to find a doctor in Geneva area with your Iphone (blackberry

too)

http://www.santeromande.ch/cgi-romandie/medecins_iphone2.pl

www.healthonnet.org

www.healthonnet.org

www.healthonnet.org

IPhone Application

HONPedia

An application to find trustworthiness health information on your IPhone about

medical topics

www.healthonnet.org

www.healthonnet.org

HON Web 2 approach Why?

Collaborative websites are more and more sophisticated and

users interact creating and editing content

81% [1] of the Internet users go online to find solutions to their

health problems

To allow citizens to gather its experiences

To address trust non certified sites at a large scale

To be complementary with certification (not to replace)

From laypeople to laypeople

To continue educating users to trust issues

[1] http://www.harrisinteractive.com/news/newsletters/healthnews/HI_HealthCareNews2008Vol8_Iss8.pdf,

www.healthonnet.org

Trust vision at HON

TrustTrust

www.healthonnet.org

Conclusion

Active for over 13 years

Educate citizens to efficiently use the medical information

Automatic tools developed in order to support human expertise and tackle the large number of existing Web pages

2001 eEurope Quality Criteria for Health Web sites is based on the HONcode principles

Collaboration with the French HAS to be the official accrediting body for all French health Web sites

Adherence to the HONcode enhances a website’s overall quality“More than just a mouse click: Research into work practices behind the assignment of medical trust marks on the World Wide Web”

Adams SA, de Bont AA. IJMI 2006

www.healthonnet.org

Future developments

HONcode to be a participating body in the implementation of the WHO GHL and eHealth resolution action plan

Collaboration with governing bodies worldwide to provide quality control to all national health websites

EU Recommendation of Quality Criteria for Health Related Web sites carried out by HON

Continued advancement in this area

www.healthonnet.org

Visit our specific services

http://www.SanteRomande.ch/

http://www.Provisu.ch/

Health topics: Provisu (visual conditions)

Regional service: SantéRomande (directory for the French speaking Swiss)

http://www.SCAHT.org/

http://www.HealthOnNet.org/