The National Program for Quality

Indicators In Community

Healthcare: Methodological

Issues

Orly Manor7th meeting of the Eastern Mediterranean Region of the

International Biometric Society (EMR-IBS)

Tel-Aviv April 22-25, 2013

2

“Efforts to improve

quality require

efforts to measure it” 

(Casalino, 2000:NEJM)

Healthcare in Israel3

National Health Insurance law (1995) Universal healthcare

Standard basket of medical services

Four health plans (kupot cholim)

“Justice, equity and solidarity…medical services will be

offered based on medical considerations, with

reasonable quality.”

Healthcare in Israel4

Health Tax - progressive , paid to the National Insurance Institute

(funds are distributed to the health plans according to a capitation

formulae) + modest copayment directly to health plan for specific

services

Open enrollment, no option to reject applicant, annual option to

switch plans

Managed competition between plans (uniform benefits)

Competition is based on quality and nature of services

5

Israel National Program for Quality Indicators

in Community Healthcare (QICH)

Supervisory bodies were established “to follow and

assess the influence of the NHI law on health services in

Israel, their quality, effectiveness and cost"

QICH started as a research project (Porath & Rabinovitz,

2002) and later adopted as a national program.

Full cooperation and support by all four health plans

Mission

To provide the public and policy-makers information

on the quality of community healthcare provided in

Israel. This information covers various health

categories and is intended to promote and improve

the standard of healthcare in Israel.

6

Main Product

Annual report presenting national results

of quality indicators in community healthcare

Enables

the evaluation of developments and changes in healthcare over time

the early identification of risk factors in the Israeli population and in

sub-populations

the comparison of healthcare quality in Israel with other countries

7

8

"Not everything that

counts can be

counted,

and not everything

that can be counted

counts."

Einstein

Quality Indicators- Methods9

Measures of clinical performance (structure, process, outcome)

Based on electronic health records from the four health

plans

All indicators are rates

Some indicators are conditional on others

Covariates: age, sex, SEP (proxy)

10

Quality indicators (2013)

Example: Diabetes11

Blood glucose levels of individuals with diabetes are

directly related to the development of complications:

cardiovascular disease, blindness, kidney failure

Monitoring blood glucose by periodic hemoglobin A1c

testing and achieving adequate glycemic control

Example: Diabetes12

Prevalence measure:

Rate of individuals with diabetes mellitus from the entire population (overall

and by age and gender)

Process measure:

Rate of individuals with diabetes with documented levels of hemoglobin

A1c (HbA1c)

Outcome (intermediate) measure:

Rate of individuals with controlled levels of HbA1c from patients with

diabetes with documented levels of hemoglobin A1c (HbA1c)

Methodological Issues

1. Criteria

2. Population coverage

3. Data quality-measurement

error

4. Data sources

5. Consistency of measures

6. Reporting

13

14

Criteria

Implementation

Population Coverage

Population-based, near-complete coverage (not

sample)

Transfers between health plans

Births/deaths

Other populations: e.g., soldiers

15

Data Quality- misclassification error

Estimating the prevalence of a medical condition in the absence of neither

a gold standard nor an additional classification.

We wish to estimate - prevalence, sensitivity and specificity , yet df=1.

We can use a Bayesian approach- simultaneous inferences of the

prevalence, sensitivity and specificity and positive and negative predictive

value (Joseph et al 1995)

Selecting priors- experts’ opinion, understanding sources of data and

errors (Greenland 2009)

16

Data Quality- Consistency of Measures

Uniform definitions across health plans

Membership

Data collection period

Numerator and denominator

17

Data Sources

1. Medical records (e.g., documentation of BMI)

2. Nurse’s records (e.g., documentation of vaccination)

3. Pharmacy claim records (e.g., medication purchase)

4. Laboratory results (e.g., HbA1c levels)

5. Hospital procedure codes (e.g., CABG)

6. Other (e.g., mammography)

18

Data Quality – Sources of Data and Sources of Error

1. Medical records (e.g., documentation of BMI)

2. Nurse’s records (e.g., documentation of vaccination)

3. Pharmacy claim records (e.g., medication purchase)

4. Laboratory results (e.g., HbA1c levels)

5. Hospital procedure codes (e.g., CABG)

6. Other (e.g., mammography)

19

Automated vs manual data input (variation between and within health plans)

Pop-up options vs typing in vaccine name, historical data

(variation between health plans)ATC vs YARPA/LARGO(variation between health plans)

Standardized values for calibration(variation between laboratories)

MOH codes used for billing (too broad)

Self reported, billing-based

Data Quality – Checks and Audit

1. Internal (health plans) Data checks (BI) Feedback loops/criterion validity

2. Quasi-external (directorate) Between and within health plan data checks (outliers)

Comparison with existing national data

3. External (independent auditor) Process audit of infrastructure changes Process audit of indicator implementation

20

Reporting21

Transparency + court ruling of the public

reporting of indicators by health plan

Case mix: substantial differences between

health plans by SEP

Limited available data on SEPFriedberg et al. (2011). Rand Corporation: Methodological Issues in Public Reporting

Reporting - Adjusting for Case Mix

22

Israel is divided into statistical areas.

The Israel Central Bureau of Statistics calculates SEP

scores for each statistical area using recent census

information

Currently-using GIS each person’s address (unidentified

data) is linked to his statistical area and the respective

SEP score

Reporting -Benchmarking

23

Setting benchmarks: setting, testing, comparing??

Diabetes care24

Diabetes care25

Diabetes care includes routine monitoring and

proper control of:

• Blood glucose levels (93%)

• Cholesterol levels (90%)

• Blood pressure measurement (92%)

• BMI assessment (86%)

• Eye examination (65%)

• Influenza (55%) and pneumococcal vaccination (77%)

Diabetes care26

International comparisons – Diabetes care (2010)

27

*US data from the National Committee for Quality Assurance, HEDIS data set for 2010

Diabetes care – Israel and England (QOF) (2009)

Thank you29

Clalit

Chaim Bitterman

Orit Ya’akobson

Arnon Cohen

Margalit Goldsprecht

Tamara Koren

Meuhedet

Liora Valinsky

Yossi Zini

Alon Yaffe

External auditor: Aliza Lukach

Israel National Institute for Health Policy Research

Advisory boards

Leumit

Daniel Vardi

Eran Matz

Doron Dushnitzky

Nirit Peretz

MaccabiYair BirenboimEinat ElranNesya GordonRachel MaromGuy Levy

Directorate: Orly Manor, Arie Ben-Yehuda, Amir Shmueli, Ora Paltiel, Ronit Calderon and

Dena Jaffe

Directorate staff: Wiessam Abu Ahmad, Galit Shefer.

Importance and Relevance30

Evidence – Moving Target

HbA1c control

31

“Intensive therapy was stopped after a mean of 3.5 years due to increased mortality” The Action to Control Cardiovascular Risk in Diabetes Study Group N Engl J Med 2008; 358:2545-2559

.

Ability to Quantify – Defining Diabetes

32

Reporting33

Quasi-external Audit34

Comparison old v. new 2009 Three-year trend 2008-2010

New ReportRelative

difference

Measure nameOld

2009New 2009DiffRel Diff200820092010

2008-2009

2009-2010

Influenza vaccination0.6050.6050.000.00% 0.600.600.580.92%-4.60%

Pneumococcal vacciation0.5000.796 0.810.790.78-1.62%-1.75%

HbA1c documentation0.9310.9310.000.00% 0.920.930.930.93%-0.06%

Controlled HbA1c (0-74 yrs) 0.4770.455-0.02-4.61% 0.440.460.443.01%-2.50%

Uncontrolled HbA1c0.0960.0960.000.00% 0.100.100.09-6.62%-8.55%

 Absolute value >3%

 Absolute value 1-3%