1

The Case for EHDIA Critical Evaluation of the Evidence

Martyn Hyde, PhD

Mount Sinai Hospital, University of Toronto, Ontario, Canada

mhyde@mtsinai.on.ca

2

Why this topic, here, now?

Endorsement of UNHS/EHDI:

• NIH Consensus Statement 1993• European Consensus Statement 1998• JCIH Year 2000 Guidelines• AAP Guidelines• UK Gov. Screening Committee approval• Legislation, mandates, agency support,

widespread implementation in US and UK• Many other regions starting or evaluating

3

US Preventive Services Task Force, Oct 2001

www.ahcpr.gov/clinic/3rduspstf/newhearrr.htm…….Thompson et al JAMA. 2001;286(16):2000-10

• ‘the evidence is insufficient to recommend for or against routine screening of newborns for hearing loss during the postpartum hospitalization (I recommendation)’

• ‘good evidence that newborn hearing screening leads to earlier identification and treatment’

• evidence of clinically important improvement in speech & language skills at age 3 years or beyond is inconclusive because of study design limitations

4

So what?

• USPSTF is influential (Guide to Clinical Preventive Services)

Negative impacts:

• Recommendations open to misinterpretation• Undermines credibility of advocates• Causes public and political confusion• Facilitates opportunistic cost-cutting or denial of

new funding

5

On the other hand…..

Positive effects:

• To re-examine our clinical and scientific beliefs and presumptions is beneficial

• The critique guides further research efforts

• Identifies important areas for attention in existing & planned programs

6

Conditions for implementing ANY population screening

• Population screening is special because society imposes care on asymptomatic persons who did not seek it*

World Health Organization 1968, Publication #34,

Principles & practice of screening for disease

• Significant burden to affected individual and to society• Accurate screening test acceptable to subjects• Accurate procedure for confirmation and diagnosis• Effective intervention

7

Analytic frameworkHarris et al. Am J Prev Med 2001;20(3S):21-35

• A structured approach to identifying key questions

and links in the evidence chain

• Prevalence of target condition? High-risk group?• Screen acceptability, sensitivity, specificity, variation?• Earlier identification/intervention in real programs?• Intervention improves health outcomes?• Intervention efficacy in clinical trials, comparative

effectiveness in screened vs non-screened groups?• Harms from screening, from intervention?

• ON BALANCE DO BENEFITS OUTWEIGH HARMS?

8

Burden of Target Condition

• MUST define target condition precisely– Affects prevalence and most program aspects– If undefined cannot evaluate or meta-analyse results– EHDI program targets differ, sometimes not defined

• Key variables to consider:– Hearing loss severity, laterality, frequencies, type

• Major questions:– Lower severity limit? Unilateral? Chronic conductive?

• Issues when pushing lower limits:– Impact? Detectable? Effective intervention? High false-

positive rate? Unknown prevalence?

9

0

1

2

3

4

5

6

7

8

Me

an

Ag

e o

f D

iag

no

sis

(y

ea

rs)

Screened Referred with highrisk factors

Referred with nohigh risk factors

Referral Group

mild moderate mod-severe severe profound

Age at diagnosis, by severity and route to diagnosis, N=613 with HAs (Ontario)

Durieux-Smith & Whittingham, J Sp Lang Pathol Audiol, 2000

10

Prevalence of hearing loss

• Governs overall societal burden (case volume)• Strongly affects screening yield & PPV of ‘refer’• Depends on target conditions • Difficult to estimate accurately, minimum of 500 true

cases needed, more for subgroups

• UNHS programs can give only inaccurate lower bound (low N, imperfect coverage, compliance, sensitivity, only refers followed)

• Prevalence known accurately only for bilateral >= moderate loss at age >= 3 yrs in the UK !

11

Most accurate prevalence data (by far)

Fortnum H et al, BMJ 2001;323:536

UK health/education provider survey, all cases born 80-9517,160 children identified with bilateral PHL >40 dBHL

Prevalence ~ 1/1000 at 3 yrs, doubles by age 9!

Progressive & late onset losses underestimated!Strong surveillance program crucial in EHDI!

Other studies suggest 30 dB criterion doubles prevalence, and including unilaterals increases it by about 50%

12

Prevalence, PHI >40 dBHL ave bilateral

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7 8 9 10 11 12

age in years

ca

se

s p

er

tho

us

an

d

Fortnum et al BMJ 2001;323:536

13

Screening sensitivity

• NOT known accurately

• UNHS programs do NOT provide sensitivity estimates. Misses unknown. Programs can only yield imprecise lower bounds.

• To determine sensitivity, ALL screened infants must have definitive audiometry asap

• One good study to date, modest sample size

14

Best sensitivity/specificity data

US NIH: Norton et al 00, Ear Hear 21(5):508

2,995 at-risk screened with OAE (T & DP), AABRReliable frequency-ear-specific VRA 8-12 months!

For target PHL >= 30 dBHL ave 2,4 kHz:

Sensitivity ~ 85% all tests at specificity of 90% (10% FP)

For target PHL >= 30 dBHL ave 1,2,4 kHz:ABR better than OAE

15

90% confidence limits on sensitivity point estimate of 0.9

0.7

0.8

0.9

1

10 100 1000

number of true cases

sen

siti

vity

lo 90% CL

hi 90% CL

16

To estimate sensitivity

• MUST specify target condition• MUST define at what specificity

• Compare yield to total known cases in birth cohort - need strong local ascertainment /national case registry

• Compare yield to estimated prevalence for the SAME target condition and comparable population

• => sensitivity for > 40 dBHL average at least 80%

• sensitivity for smaller losses uncertain, lower

17

Screening specificity

• High specificity VERY important. Determines false-positive numbers (FP harms) and PPV.

• Much more accurate data than for sensitivity• UNHS pass rates approximate specificity, because most

babies are free of target condition (prevalence low)• Probably underestimated because of transient losses

• Series protocols with AABR can achieve up to 99%, BUT

• Large variation and trends observed. Tester skills and protocol details probably crucial.

18

A logical flaw ?

• The Task Forces favor high-risk screening.• Risk assessment requires eg parental interview about

familial loss, for ALL non-NICU babies.• This takes time and is notoriously inaccurate.• OAE screening takes less time and is more accurate.• OAE screening is NOT a hearing test, but a risk

assessment tool. Its PPV and that of a positive familial risk factor are comparable!

• Therefore, we should screen all non-NICU babies with OAE to determine their risk status…. that is,DO UNHS IN ORDER TO DO TARGETED SCREENING!

19

Health Outcomes

• The Task Forces are interested in effects of preventive measures on important health outcomes

• USPSTF: ‘…health outcomes are measures that a patient can feel or experience, including death, quality of life, pain and function.’

20

USPSTF evidence review

• Focussed on evidence of benefits for speech & language development vs potential harms of FP screens

• Speech/language development clearly a health outcome

• Yoshinaga-Itano’s key studies considered susceptible to sampling bias, especially re parental involvement, and also sub-optimal in statistical analysis

• There appears (to us) to be no systematic examination of other possible health outcomes

21

Speech/language development is a problematic outcome choice• It is NOT the most direct health outcome• NOT the most tractable area for UNHS evaluation• Subject to a HOST of confounding variables

– loss severity; family involvement; communication training; cognitive levels; comorbidities; cultural & socioeconomic factors

• Experimental study of UNHS effects is NOT feasible– randomization, controls unethical or illegal; even if

possible, sample size requirements formidable because of multiple confounding variables

• Cohort studies vs historical controls, and case-control studies, however large and well-done, will always be subject potential confounding by unknown covariates.

22

Some plausible EHDI Outcomes

• Benefit? Harm? Value? Generality? Measures?• Earlier family knowledge

– empowerment, anxiety

• Earlier, improved hearing– HAs, ALDs, etc

• Earlier, appropriate communication– HAs, communication strategies, ASL, AVT, etc

• Improved auditory perceptual development• Improved speech/ language development• Improved early educational performance

23

Earlier, improved hearing.Is it a (valued) health outcome?

• WHO ICF 2001 p7: ‘Examples of health domains include seeing, hearing, walking..’

• Do infants ‘feel or experience’ hearing (USPSTF)?

• Is earlier, improved hearing in infancy important in and of itself, regardless of its long-term effects???

24

The infant cannot tell us…so:

• Would a year (month, day?) of inability to hear be (a) noticed and (b) negatively valued by an older child or young adult?

• If so, earlier improved hearing is a valued health outcome in the infant

• Alternative is that it is NOT a valued health outcome BECAUSE the infant is an infant...

25

Why has earlier ability to hear not been addressed in reviews?

• USPSTF review updated a previous (1995) review with speech & language development as the outcome

• ‘Delayed detection of hearing loss in infancy compromises speech and language development…’

NOT• ‘Delayed detection of hearing loss in infancy prevents

early hearing and can compromise early, effective family communication…’

• Too obvious? But if important, why not include it…?• Another example of the ‘hidden handicap’ effect?

26

Does UNHS lead to earlier, improved hearing?

• What constitutes a substantive hearing improvement?• How do we measure it (behavioural, physiological)?• Can we rely on laws of physics, eg for detectability? • What is the effect of hearing loss severity on any

incremental improvement due to UNHS?• Can HAs improve hearing in a young infant?• Are HAs actually fitted earlier and appropriately?• How much earlier, and for how many, is significant?• Is earlier access to cochlear implants a benefit?• Etc

• Most are new ?s, not addressed by reviews focussed on speech/language development

27

Enhancing systematic reviews 1

• Improve the process to define key outcomes

• Process should be comprehensive, explicit and well-documented

• In-depth consultation with content experts, affected families, deaf and hard-of-hearing persons

• Historical choices of outcome should be evaluated critically and not be presumed valid

28

Enhancing systematic reviews 2

• DO NOT replace clinical opinion by methodologic opinion

• Increase depth & strength of rationale, for methodologic criticism of key evidence sources

• Each major criticism needs an explicit evidence base

• Avoid contentious areas of design & statistical analysis

• Increase quantitativity, eg replace ‘possible sampling bias’ by a demonstration that a potential confounding variable could explain the observed main-effect size

29

Enhancing systematic reviews 3

• Incorporate personal/societal values more fully and without bias into the weighing of benefits & harms

• Eg, ‘potential’ FP harms given strong weight, yet studies to date show no significant harms and/or strong family support for early identification (not incorporated)

• Selection of speech/language development and ignoring improved hearing involves implicit valuation

• Values ARE quantifiable, eg by health state utility analysis (standard gambles, time trade-off)

30

Concluding Remarks

• Good public health policy results from an interplay of evidence, context and human values.

• Evidence informs policy but does not govern it.

• Infants cannot complain of lack of hearing, and cannot seek interventions we would surely not deny ourselves. We can and we must do their complaining for them.

• Infants and families have a fundamental right to early and effective communication.