Health Care Stories are good for you

Sharon Manson Singer, EvidenceNetwork.caSteve Buist, Hamilton Spectator

Jennifer Verma, CHSRF

Introduce EvidenceNetwork.ca Talk about the Hierarchy of Evidence What makes a good health story? A bad

one? Top ten questions to ask of health experts

about their research Where to go for data How to assess the quality of data Last word to you the audience

Overview

EvidenceNetwork.ca links journalists with health policy experts to provide access to credible, evidence-based information.

What does EvidenceNetwork.ca do?

EvidenceNetwork.ca is a non-partisan, web-based project funded by the Canadian

Institutes of Health Research and the Manitoba Health Research Council to make the

latest evidence on controversial health policy issues available to the media.

What is EvidenceNetwork.ca?

The Canadian Health Accord is scheduled for renegotiation in 2014. Canadians will have to make decisions about many complex health policy issues, including;

• Aging population impact • Rising drug costs • Health care accessibility • Private sector financing/delivery • User fees • Sustainability of the healthcare system

EvidenceNetwork.ca is committed to working with the media to build a healthy dialogue around Canadian healthcare.

Why do we do it?

Hierarchy of Evidence – Top Tier

Type of Study Expected Results

Systematic Review and meta-analysis

Use upper tier studies in a synthesis of research findings

Strongest evidence – only as good as the underlying studies

Hierarchy of Evidence – Upper Tier

Type of Study How good is it?

Randomized experiments

Natural experiments

Well designed with sufficient sample size

High quality source of exogenous variation generating comparison group

Well designed pre and post measures

Analytical techniques are appropriate

Hierarchy of Evidence –Middle Tier

Type of Study How good is it?

Some control in the assignment of treatment

Correlational studies

Limited source of exogenous variables or some control of selection or process

Well designed pre and post measures

Appropriate data with large sample

Reasonable approach to estimating counterfactuals

Hierarchy of Evidence – Lower Tier

Type of Study How good is it?

Studies without a comparison group

Participant Satisfaction

Expert Opinions

Credible case selection with explicit causal logic model

Quality outcome measures

Collect feedback from participants on quality of intervention

Respected individuals or organizations with explicit rationale for opinion

Hierarchy of Evidence – Lower Tier

Type of Study How good is it?

Exploratory case studies

Less credible or explicit case selection criteria, theory of change or outcome measure(s)

www.EvidenceNetwork.ca

Sharon Manson Singer, PhD smansonsinger@gmail.com

Contact Us

Examples of bad science, dubious science or no

scienceSteve Buist, Investigations Editor

The Hamilton Spectator

Examples of bad science, dubious science or no

scienceLuigi Di Bella and the “miracle” cure for cancer

(Pulling for the underdog . . .)

Examples of bad science, dubious science or no

scienceAutism and the MMR vaccine

(Science is a lot easier if you just make it up . . .)

Examples of bad science, dubious science or no

scienceMultiple sclerosis and “liberation therapy”

(The underdog tale, with a modern-day social media twist . . .)

Examples of bad science, dubious science or no

scienceClimate change and the human touch

(What makes for a balanced story . . .)

10 questions to consider when writing about

science

10 questions to consider when writing about

science1. Who is conducting the science?

10 questions to consider when writing about

science1. Who is conducting the science?2. Who is paying for the research?

10 questions to consider when writing about

science1. Who is conducting the science?2. Who is paying for the research?

3. Who is paying the researcher?

10 questions to consider when writing about

science1. Who is conducting the science?2. Who is paying for the research?

3. Who is paying the researcher?4. Where are the results being published?

10 questions to consider when writing about

science1. Who is conducting the science?2. Who is paying for the research?

3. Who is paying the researcher?4. Where are the results being published?5. What was the population being tested?

10 questions to consider when writing about

science6. What was the sample size?

10 questions to consider when writing about

science6. What was the sample size?

7. How significant are the results?

10 questions to consider when writing about

science6. What was the sample size?

7. How significant are the results?8. What do other people think, and do those people have their own conflicts of interest?

10 questions to consider when writing about

science9. How do these results fit into the context of what’s already known?

10 questions to consider when writing about

science9. How do these results fit into the context

of what’s already known?10. Are there opposing viewpoints and how

much weight should those viewpoints be given?

Making measures meaningful:Finding and interpreting

healthcare dataJenn Verma, Director,

Collaboration for Innovation and Improvement

31

mythbusters USING EVIDENCE TO DEBUNK COMMON MISCONCEPTIONS IN CANADIAN HEALTHCARE

Data, data…everywhere

32

33

THE LATEST RESEARCH SHOWS THAT WE REALLY SHOULD DO SOMETHING

WITH ALL THIS RESEARCH

In a review of World Health Organization (WHO) and World Bank recommendations on five topics (contracting, healthcare financing, HHR, tuberculosis control and tobacco control): ◦ 2/8 publications cited systematic reviews;◦5/14 WHO and 2/7 World Bank recommendations

were consistent with both the direction and nature of effect claims from systematic reviews.

Hoffman SJ, Lavis JN, Bennett S. 2009. The use of research evidence in two international organizations’ recommendations about health systems. Healthcare Policy

9(1): 66-86.

34

High-quality and

performance health

systems

Effective

Accessible

Safe

Equitable

Patient-centredEfficient

Integrated

Appropriately

resourced

Focused on

Population Health

35

Adapte

d w

ith p

erm

issi

on f

rom

Healt

h Q

ualit

y O

nta

rio

(20

11

)

F/P/T Jurisdictions

P/T Ministries of Health

P/T Health Regions

Statistics Canada

Public Health Agency of Canada

Health Canada

Provincial Health Quality Councils

NL Centre for Health Information

NB Health Council

Commissaire à la santé et au bien-être du Québec

Health Quality Ontario

MB Institute of Patient Safety

Health Quality Council SK

Health Quality Council AB

BC Patient Safety and Quality Council

Subject Matter Experts

Canadian Institute for Health

Information

Institute for Clinical Evaluative Sciences

(ON)

Canadian Stroke Network

Canadian Cardiovascular

Outcomes Research Team

POWER Study (ON)

Cardiac Care Network (ON)

Collaboration for Excellence in

Healthcare Quality

Alberta Diabetes Surveillance System

Non-jurisdictional Organizations

Canadian Institute for Health

Information

Health Council of Canada

Canadian Patient Safety Institute

Canadian Partnership Against

Cancer

Accreditation Canada

Fraser Institute

Ontario Hospital Association

Heart and Stroke Foundation

Canadian Diabetes Association

Cancer Care Ontario

Select International

Reporting Initiatives

National Quality Forum (US)

Agency for Health Research and Quality (US)

NHS Indicators for Quality

Improvement (UK)

Australian Commission on

Quality and Safety in Health Care

Quality and Efficiency in

Swedish Health Care

OECD

RAND

36Adapted with permission from the Health Council of Canada (2011)

37

38

Health Indicators provide a Dashboard for Health and Healthcare

They can let you know that things are running smoothly.

They can alert you to problems that may need attention.

rising BMI (Body Mass Index) doesn’t explain the root cause of weight gain.

In 2009, Canadians received 121 CT scans per 1000 people. There were also 8 MRI units per million population (vs. 12 MRI units per million as the OECD average). OECD (2011) reports Canada is “lagging behind,” but there is no agreed-upon benchmark.

In 2009, Canada had 2.4 physicians per 1000 population (vs. 3.1 OECD avg), but…◦ We have more physicians than ever before – Is this about supply or

distribution and deployment? ◦ We also have more nurses per 1000 people (9.4 in Canada vs. 8.4

OECD avg)39

Interpreting Data…

Interpreting Data Comparing apples-to-apples?

◦ Age standardization◦ Risk adjustment

Measuring intangibles◦ e.g., quality of life◦ Composite indicators

40

41

42

Lavi

s J.

et

al.

200

9. S

UP

PO

RT

To

ols

for

evid

ence

-inf

orm

ed

hea

lth

Pol

icym

akin

g (S

TP

). H

ealth

Res

earc

h P

olic

y &

Sys

tem

s 7(

Su

ppl 1

).

http

://w

ww

.hea

lth-p

olic

ysys

tem

s.co

m/c

onte

nt/7

/S1/

I1

“Numbers can’t ‘talk’ but they can tell you as much as your human

sources can. But just like with human sources, you have to ask” (Niles, 2007).

Niles R. 2007. Statistics every writer should know: A simple guide to understanding basic statistics, for journalists and other writers who might not know math. http://nilesonline.com/stats/

43

Useful Links

44

Support Tools for Policy Making http://www.chsrf.ca/PublicationsAndResources/

ResearchReports/Support_Tools_for_Policy-Making.aspx

Mythbusters http://www.chsrf.ca/PublicationsAndResources/Mythbusters.aspx

What If? http://www.chsrf.ca/Programs/HealthcareFinancingInnovationAndTransformation/WhatIf.aspx

CHSRF’s Quality of Healthcare in Canada: A Chartbook (2010)CIHI’s Making Sense of Health Indicators (2011)HCC’s A Citizen’s Guide to Health Indicators (2011)CIHI’s Making Sense of Health Rankings (2008)OECD’s Health Data 2011: How Does Canada Compare? (2011)