BIRTH DATA QUALITY VITAL RECORDS: A CULTURE OF QUALITY NAPHSIS Annual Meeting | Seattle | June 8-11,...

BIRTH DATA QUALITY

VITAL RECORDS: A CULTURE OF QUALITY

NAPHSIS Annual Meeting | Seattle | June 8-11, 2014

An Update

THE BIRTH DATA QUALITY

WORKGROUP

Karyn Backus(Cha i r ) (CT)

Col leen Fontana (Cha i r ) (R I )

Sukhjeet Ahuja(NAPHSIS)

Mary Chase (CO)

Greg Crawford (KS)

Mel issa Gambatese(NYC)

Jean Hreczan (DE)

Andrew Jessen (AK)

David Just ice

(NCHS)

Ann Madsen

Stra ight(NYC)

Joyce Mart in

(NCHS)

Judy Nagy (OH)

Sharon Pagnano

(MA)

Phyl l i s Reed (WA)

Shae Sutton (SC)

Marie Thoma

(NCHS)

Ela ine Tretter (MD)

Louise Wishart

(DE)

BIRTH DATA QUALITY WORKGROUP

HOSPITAL REPORTS/ ENGAGING

HOSPITALS SUBGROUP

CHARGE

Birth Data Quality Workgroup mission was to address identified issues with birth data quality.

Subgroup 1: Hospital ReportsCharge: Recommend a process (metrics, means of communication, actions) for vital records offices to use to provide data to hospitals to help them improve their reporting.

Goal 1: Survey all jurisdictions to learn their current activities

Subgroup 2: Engaging HospitalsCharge: To develop approaches to engaging hospitals to improve systems and procedures for data gathering.

Step 1: Develop an outline for engaging hospitals

In October 2013, subgroups 1 and 2 were combined as charges were overlapping.

CHARGE

SURVEY RESULTS

The survey is a starting point; a tool to gauge where we are and who is doing what.

• Are all jurisdictions actively engaged in Evaluating and Ensuring Data Quality (EEDQ) from the birth facilities?

• Is communication with hospitals on EEDQ issues regularly utilized as a means for improving quality?

• Does communication with hospitals make a difference?

WHY DID WE DO THE SURVEY?

The survey is a starting point; a tool to gauge where we are and who is doing what. (cont . )

• What else are jurisdictions doing to improve the data quality from hospitals? (types of metrics, types of awareness efforts)

• Do jurisdictions want to do more or are they satisfied with where they are?

• What other barriers are there to EEDQ (organizational dynamics, staffing, failure to utilize standard tools, etc.)

WHY DID WE DO THE SURVEY?

First administered in March 2013 41/52 (78.8%) – United States 2/5 (40.0%) – Territories

In October 2013, non-responding jurisdictions contacted by email with opportunity to participate 5 additional U.S. jurisdictions responded

SURVEY PARTICIPATION

Final participation rates: 46/52 (88.4%) – United States; representing 83% of

US Births 2/5 (40.0%) – Territories

SURVEY PARTICIPATION

Participated 84.2%

Did not participate

15.8%

All jurisdictions

Encourage Improved Collaboration within

the Jurisdiction

Findings:Majority maintain registration and statistics staffs in same unit

Majority indicate room for improvement to maximize EEDQ

More than half reported being understaffed for EEDQ activities

Nearly all report desire to do more extensive EEDQ

FINDINGS & RECOMMENDATIONS

JURISDICITONAL DYNAMICS

Encourage Improved Collaboration within

the Jurisdiction

Recommendations: Jurisdictions should improve collaboration between staffs

Next steps: Identify jurisdictional successes with or impediments to collaboration

Offer recommendations for improving or enhancing the relationship between the two staffs


JURISDICITONAL DYNAMICS

Increase Jurisdictional Compliance with Established Standards

Findings: (revised only)


STANDARDIZATION NOT UNIVERSAL


Recommendations:All jurisdictions should develop standardized worksheets based on NCHS standard

Jurisdictions should mandate use by birth facilities

Next steps:Clarification from NAPHSIS/NCHS regarding what constitutes compliance

Explore why jurisdictions did not adopt standards

Consider improving the standards based on identified issues




Findings (all jurisdictions):EBRS vary in their forward-facing quality control measures Missings versus unknowns Logic checks, soft edits, hard edits

Jurisdictional interpretation of completeness varied Some reported that EBRS was suffi cient to ensure

data quality

Recommendations: Jurisdictions should confirm that systems are compliant with standards




Next steps:Clarify whether missing versus unknown is a QA concern as it pertains to record completeness

Review existing standards for data collection systems and document rationale for why various standards are imposed

Share review with jurisdictions so they can better evaluate their systems



Increase Data Quality Evaluations

Finding 1: Range in jurisdictional utilization of data quality

analyses

Recommendations: Learn about the various metrics available for

assessing quality Self-evaluate to expose data validity issues

Next steps: Develop ways to educate jurisdictions about data

quality constructs Collaborate with NCHS to provide additional data

quality reports


DATA QUALITY EVALUATIONS


Finding 2:Types of QA metrics utilized variesFrequency and timing of QA reports varies

Completeness: 52% outside EBRS Logic checks: 61% outside EBRS Audits: 6% regularly, 17% rarely or as needed Other QA: 78% perform at least one, 17% perform

all three




Recommendations:Move closer to near real-time evaluations and away from year-end

Use multiple metrics to detect quality issuesLook for alternative resources for QA, given staffing limitations Hospital self-audits, collaborate with other

programs, funding for targeted QA, linkage with hospital discharges




Next steps:Determine which metrics are efficient and effective at measuring quality and recommend these as a best practice

Develop standardized analytical programs for recommended metrics (69% prefer SAS)

Investigate the development of NCHS-based hospital reports



Advocate for Data Quality

Findings 1: Jurisdictions that provided feedback realized improvement.


PROVIDE FEEDBACK

Some jurisdictions have no efforts in place to advocate for birth data quality; many have only a few targeted efforts.


Recommendations: Improve communication with hospital partners about performance.

Increase education and awareness about the merit of data quality and common issues. Provide trainings and newsletters. Educate hospital staff about the public health value

of birth certificate data. Educate non-hospital staff about data quality initiatives and data limitations.

Publish reports about performance to increase transparency.


PROVIDE FEEDBACK


Next Steps:Develop a set of best practices for communicating with hospitals about ongoing performance.

Develop educational materials (newsletters, letters, reports) for hospital staff and non-hospital stakeholders.

Evaluate expressed concerns over publishing hospital-level reports (e.g., confidentiality, misinterpretation).


PROVIDE FEEDBACK


Finding 2:The hospital administration was rarely identified as recipients of performance reports.

Recommendation:Engage upper-level clinicians and hospital administrators in the merit of providing quality birth data.


PROVIDE FEEDBACK


Next steps:Pursue effective avenues for communicating with higher-level hospital administration.

The subgroup will transition into the “Engaging Hospitals” charge.


PROVIDE FEEDBACK

The national survey provided information that was not previously available.

Serves as a spring board for initiatives at the jurisdictional and birth facility level. StandardizationEducationCommunication

Supporting the overall initiative to develop a national model for improving birth data quality.

SUMMARY

91% of jurisdictions want to do more quality review.

Our subgroup is striving to provide tools to all jurisdictions to reach that goal.

Thank you to all jurisdictions who participated in the survey.

You are helping us go from Good to

GREAT!

CONCLUSIONS

Sukhjeet Ahuja (NAPHSIS)

Karyn Backus (CT)

Bruce Cohen (MA)

Isabel le Horon (Chair) (MD)

Renata Howland (NYC)

Michel le Osterman

(NCHS)

Elaine Tretter (MD)


PRENATAL CARE ITEMS

SUBGROUP

To assess the quality of prenatal care data items collected on the U.S. Standard Certificate of Live Birth; and

Recommend changes for improvement

CHARGE

BackgroundData uses

History of collecting PNC data

Data quality

Barriers to the collection of accurate data

Survey of jurisdictions

LAST YEAR

Further review of survey findings and follow up with selected jurisdictions

Review of Guide to Completing the Certificate of Live Birth

Survey of MCH Experts

Development of Recommendations

THIS YEAR

Date of first prenatal care visit

Date of last prenatal care visit

Number of prenatal care visits

PNC DATA ITEMS

Last year—data collection

This year—telephone follow up with 11 jurisdictions Efforts in placeEffectiveness of efforts

SURVEY

Prepared by NCHS (with assistance from NAPHSIS, birth information specialists, and clinical experts)

Purpose—to assist facility birth registrars in providing complete, accurate data

Contains: Definitions Instructions Sources of information Key words/abbreviations

REVIEW OF GUIDE TO COMPLETING THE CERTIFICATE OF LIVE BIRTH

Queried MCH data collection and research experts concerning “date of last prenatal care visit”

SURVEY OF MATERNAL AND CHILD HEALTH EXPERTS

1. Include edits/edit rules in EBRS2. Require facilities to obtain data from PNC

records3. Encourage PNC data collection from

providers at 35+ weeks, with update following delivery

4. Consider preparing a comprehensive worksheet for PNC providers

5. Recommend that hospitals with EHRs store paper PNC records in central location

RECOMMENDATION 1IMPROVE DATA COLLECTION

1. Visit facilities routinely2. Provide training for new birth

registrars3. Ensure facilities have (and use)

Guidelines4. Promote use of eLearning training

RECOMMENDATION 2IMPROVE TRAINING

1. Hold annual conference for birth registrars

2. Recognize good performance3. Prepare newsletter4. Identify data users and solicit

assistance5. Communicate with PNC providers re:

need for accurate data6. Assist birth registrars to find solutions

to problems

RECOMMENDATION 3IMPROVE COMMUNICATION WITH HOSPITAL

STAFF AND PNC PROVIDERS

Assess data quality regularly and contact facilities for corrected data ASAP

Provide facilities with completeness/accuracy reportsProvide comparison data to encourage improvement

Concentrate on poorest performing facilitiesRecommend strategies for improvementFollow changes in trends

Conduct simple analyses to identify hospitals providing questionable data

RECOMMENDATION 4REVIEW DATA QUALITY AND TAKE

ACTION

Source: Connecticut Department of Public Health

Conduct routinelyReview both mother and newborn recordsMeet with hospital staff Prepare detailed analysis of findings

Point out areas where improvement is needed Provide instructions for improvement Provide positive as well as negative findings

Provide findings to high level hospital staffConduct follow up audit in 6 months if facility

is performing poorly.Encourage hospitals to do internal auditsLook for outside resources to support audit

RECOMMENDATION 5AUDIT HOSPITAL MEDICAL RECORDS

Raise awareness

Encourage use

RECOMMENDATION 6RAISE AWARENESS OF GUIDELINES

Drop “Date of Last Prenatal Care Visit” from the U.S. Standard Certificate of Live BirthNo body of literature or significant research findings describing use of this data item

Recorded inconsistently across facilities and jurisdictions

IncompleteLittle analytic, research or practical utility

RECOMMENDATION 7CERTIFICATE CHANGE

Establish clear definitionEnlist assistance from clinicians and MCH experts

Test to ensure that new definition improves data quality

If data quality cannot be improved, reevaluate whether item should remain on certificate

RECOMMENDATION 8FOCUS ON IMPROVING “NUMBER OF PRENATAL

CARE VISITS”

Sal ly Almond(Cha i r ) (MN)

Marie Aschl iman(UT )

Karyn Backus(CT )

Lucy England(CDC)

Col leen Fontana(R I )

Mel issa Gambatese(NYC)

Saeed Hamdan(CDC)

Cather ine Haralson(TN)

Kerry L ionadh(OR)

Joyce Mart in(NCHS)

TJ Mathews(NCHS)

Carol Moyer(KS )

Marie Thoma(Cha i r ) (NCHS)

Ela ine Tretter(MD)


LEARNING CREATIONSUBGROUP

e

Birth record timeliness is much improved

Slide 50

WHY CREATE LEARNING?e

PRINCIPAL SOURCE OF PAYMENT FOR DELIVERY: 38-STATE AND THE DISTRICT OF COLUMBIA

REPORTING AREA, 2012

Source: CDC/NCHS, National Vital Statistics System.

Medicaid43.9%

Private insuranc

e46.9%

Self-Pay (uninsured

)4.1%

Other5.1%

AVAILABLE, UNIFORM, USER-FRIENDLY

Some jurisdictions have created own training

Convenience for jurisdictionsConvenience for stakeholdersDisparate registration rolesReduce barriers to timely fi ling

WHAT WILL IT LOOK LIKE?

Scope, format, and audience for eLearning

PROPOSED FORMAT

Two modules:1. Motivation - two tracks

Clinical Clerical

2. Medical - detailed, specific data instructions

MODULE 1 - MOTIVATION

Objectives: Understand public health importance Recognize the significance of complete and

accurate data Identify how my role is critical Benefits me, patients, facilities, &

community

Divided into TWO tracks for clinical & clerical staff

BIRTHS WITH MEDICAID AS THE PRINCIPAL SOURCE OF PAYMENT FOR THE DELIVERY BY

AGE OF MOTHER: REVISED REPORTING AREA, 2012

Source: CDC/NCHS, National Vital Statistics System.

Under 20 20-24 25-29 30-34 35-39 40 and over

0

10

20

30

40

50

60

70

80 75

64.8

41.9

27.9 26.1 26.9

Age of mother

Perc

ent

TECHNICAL DETAILS

Professionally designedInteractiveNavigable Approximately 20-30 min per module

MODULE 2 – MEDICAL

2003 Birth data setHighlighting

specific or troublesome data

IndexedResource links

RESOURCES – THE GUIDE

Guide to Completing the Facility Worksheets

ADDITIONAL RESOURCE LINKS

Professional programs List of medical abbreviationsNCHS - worksheetsCDC

http://www.naphsis.org/

https://www.awhonn.org/awhonn/index.do

WHAT’S IN IT FOR ME?

CME, CNE, CEU, CECH availableHospital convenienceState convenience

HOW TO PREPARE?

Our next steps….and yours

WHAT’S NEXT?

Promote convenienceConsider requiring

trainingKnow your state-

specific data – supplemental training may be necessary

Plan website or link Share your ideas

Joyce Martin (Chair) (NCHS)

Sukhjeet Ahuja (Chair)

(NAPHSIS)

Karyn Backus (CT)

Mark Flotow(IL)

David Justice(NCHS)

Michel le Osterman

(NCHS)

Phyl l is Reed(WA)

Marie Thoma(NCHS)

Elaine Tretter (MD)


CU T-ITEM SUBGROUP

Recommend items to CUT from the 2003 national standard birth data file because of:

Poor data quality andLack of potential for improvement

NOT to add items to the standard birth certificateNOT to recommend changes that would require modifications to jurisdictional electronic birth systems

No changes to item wording etc. (e.g., not from the current wording “Moderate/heavy” meconium staining to “Thick” meconium staining)

CUT ITEM SUBGROUP CHARGE

Developed “short” list of items (i.e., low hanging fruit) Focus on medical health data items Items of questionable data quality Jurisdictional and NCHS experience Audits and validity studies

Missing from short list Items considered key (i.e., date of LMP) Rare events, e.g., the maternal morbidity and

congenital anomaly items (not enough information).

Field survey of jurisdictions for input on short list, other items of concern, and improvement potential

FIRST STEPS - WHICH ITEMS TO REVIEW?

Special thanks

to

Sukhjeet Ahuja

for developing the survey

and leading this eff ort

SURVEY OF

JURISDICTIONS TO

IDENTIFY BIRTH ITEMS

OF POOR DATA QUALITY

Input from the jurisdictions on “short” list of potential items to cut from the national birth file

Asked about specific data items known to have poor quality based on information from sources previously mentioned:Validity studiesInput from BDQW membersInquiries from jurisdictions to NCHS

SURVEY GOAL

Prenatal care Date of last prenatal care

visit Total # of prenatal visits

Risk Factors in This Pregnancy Other previous poor

pregnancy outcomes Previous preterm birth

Obstetric Procedures External cephalic version Tocolysis

Onset of Labor Premature rupture of

membranes Precipitous labor, <3 hours Prolonged labor, >=20

hours

Characteristics of Labor and Delivery Steroids received by

mother Antibiotics received by

mother Moderate/heavy meconium

staining Fetal intolerance of labor

Abnormal Conditions of the Newborn Assisted ventilation

required for >6 hours Assisted ventilation

required immediately following delivery

ITEMS

NAPHSIS sent out via SurveyMonkey

Sent to NAPHSIS “Primary Contacts”

March 2014Target: Revised Jurisdictions

METHODS

30 responses

2 unrevised

2 submitted 2 responses (diff erent staff )

26 revised jurisdictions(including 2 territories)

* 52% of revised 2014 births *

RESPONSE

93% have assessed data qualityEach item assessed by at least 12 jurisdictions (43%)

When is data quality assessed?At regular intervals (every 6 months, annually)

When data quality starts to decline (more missing data, out of range data)

Soon after adoption of the 2003 revision

DATA QUALITY ASSESSMENT

0

20

40

60Top 3 Assessment Methods

Percent

HOW IS DATA QUALITY ASSESSED?

0

30

60

90

Items of most concern

Perc

ent

ITEMS OF POOR DATA QUALITY

“Other” items listed more than onceDate of last menstrual periodGestational diabetesPregnancy-induced hypertension Date of first prenatal care visit Mother received WIC food during this pregnancy

ITEMS OF POOR DATA QUALITY

Impr

oved

trai

ning

Clear

er in

stru

ctio

ns

Bette

r defi

nitio

n0

1020304050607080

Average % of “yes” responses

Perc

ent

For all items:

Improved training>

Clearer instructions

>Better definition

POSSIBLE IMPROVEMENT

Has been very helpful

Used in subgroup deliberations on items

Decision-making process and Recommendations

USEFUL INFORMATION

DECISION-MAKING- PROCESS

Date of last PNC visit Total # of prenatal care visits

Previous preterm birth (Risk Factors in this Pregnancy) Other poor pregnancy outcomes (Risk Factors in this Pregnancy)

Tocolysis (Obstetric procedures) External cephalic version (Obstetric procedures)

Fetal intolerance of labor (Characteristics of L&D) Steroids received by mother (Characteristics of L&D) Moderate/heavy meconium staining (Characteristics of L&D) Antibiotics received by mother during labor (Characteristics of

L&D)

Precipitous labor (Onset of Labor) Prolonged labor (Onset of Labor) Premature rupture of the membranes (Onset of Labor)

Assisted ventilation following delivery (Abnormal conditions newborn)

Assisted ventilation > 6 hrs (Abnormal conditions newborn)

CUT ITEM SHORT LIST

DECISION MAKING PROCESS

Criteria

Rankings

High

Medium

Low

Extremely low

Clear interpretati

on

Potential for

improvement

Consistent w/ BC items

Public Health

usefulness

Data quality

CRITERIA

1/ Item title is unambiguous and can be interpreted by birth information specialist without need for specific clinical expertise or higher level training

2/ Item wording in hospital records is consistent with birth certificate item

3/ Assumes acceptable data quality.

Data quality

Public Health

usefulness3

Potential for

improvement

Clear interpretatio

n1


Improve

Watch

Cut

POSSIBLE OUTCOMES

Watch = Information currently available suggests the item is problematic, but not enough information available for final decision

Published studiesNCHS/State hospital medical records compared w/ BC

NCHS/State birth information specialist interviews

Other studies on revised birth data (very limited)

Workgroup member experienceE.g. Maryland audits

NCHS experience Jurisdictional and researcher queriesObserved consistent data issues

NAPHSIS survey on jurisdictional experience

PRIMARY SOURCES USED FOR REVIEW

Criteria

Meconium staining

Previous preterm birth

ProlongedLabor =>20 hrs

Data quality EL EL L*

Clear interpretation L H M


EL/L H M

Potential for improvement

EL M/H M/H

Potential PH usefulness

EL/L H L/M

Recommendation

MATRIX

*Limited information available on data quality.

BirthsRECOMMENDATIONS

Total number of prenatal care visits

Previous preterm birth

Precipitous labor (<3 hours)

Premature rupture of the membranes (=>12 hours)

Antibiotics received by mother during labor

IMPROVE

Prolonged labor (=>20 hours)

WATCH

Date of last prenatal visit

Other poor pregnancy outcomes

Moderate/heavy meconium straining

Fetal intolerance of labor

Recommendations approved by the

Good to Great Group!

CUT

Note: Items previously cut from the standard = are attempted forceps delivery, attempted vacuum deliveryand non-vertex presentation.

Fetal DeathsRECOMMENDATIONS

CutDate of last prenatal visitOther poor pregnancy outcomes

Improve?Previous preterm birthTotal # of prenatal care visits

NOTE: Moderate/heavy meconium staining, fetal intolerance of labor and prolonged labor are not included on U.S. Standard Report of Fetal death

RECOMMENDATIONSFETAL DEATHS

NEXT STEPS

The birth data quality workgroup will continue to review birth certificate items to ID poorly performing items

Plan to complete the short list review within next 2 months

Follow-up on items on “watch” list (additional info available?)

Re-examine certificate for additional potential items to review

List (“first wave”?) of items to cut from national fi le by November 2014

NEXT STEPS

Beginning with 2015 data year, NCHS will no longer require data be sent on the cut items

NCHS will not review items and will disable all validations and verifications from our processing so that states will not receive feedback on the cut items

Jurisdictional discretion as to whether to continue to collect these items!

NCHS will continue to accept data for the dropped items i.e., no need to modify fi les sent to NCHS

NEXT STEPS IMPLEMENTATION

The cut items will not be included in national birth and fetal death data sets beginning with the 2015 fi le

Items will be dropped from the standard worksheets and Facility Guidebook

NCHS will place a notice on the NCHS revision website explaining the change

NEXT STEPSIMPLEMENTATION

For items not cut, recommend improvements to definitions and instructions where appropriate via:

E-learning training and Facility Guidebook – improved instructions

Special approaches – e.g., more detailed instructions (e.g. gestational age and source of payment

More audits and validity studies to monitor improvement and aid decisions for next r e v i s i o n

NEXT STEPS IMPROVING ITEMS

We have learned a lot from this process and intend to use this knowledge to foster

continual improvement and meaningful change in vital statistics birth and fetal

data.

Thank you!

Co-chairs Isabelle Horon (MD), David Justice (NCHS), Joyce Martin

(NCHS)

MEMBERS

Members Sukhjeet Ahuja

(NAPHSIS) Sally Almond (MN) Marie Aschliman (UT) Karyn Backus (CT) Mary Chase (CO) Bruce Cohen (MA) Greg Crawford (KS) Victoria Daher (CA) Claudia Fabian (IL) Mark Flotow (IL) Coleen Fontana (RI) Jean Hreczan (DE) Melissa Gambatese

(NYC)

Karen Hampton (OR) Catherine Haralson

(TN) Sarah Hargand (OR) Aldona Herrndorf (CA) Cindy Hooley (VT) Renata Howland (NYC) Andrew Jessen (AK) Kerry Lionadh (OR) Terri Mack (CA) Ann Madsen Straight

(NYC) Lloyd Mueller (CT) Laura Ninneman (WI)

Michelle Osterman (NCHS)

Sharon Pagnano (MA) John Paulson (OH) Mariah Pokorny (SD) Phyllis Reed (WA) Matthew Rowe (WY) Margarita Segundo

(CO) Joann Steele (FL) Shae Sutton (SC) Marie Thoma (NCHS) Elaine Tretter (MD) Louise Wishart (DE)

SubgroupsHospital reports/Engaging hospitals

Karyn Backus (CT), Colleen Fontana (RI)Prenatal care data

Isabelle Horon (MD)New subgroups

E-learning training for hospital staff Sally Almond (MN), Marie Thoma (NCHS)

Identify items to drop from the national birth data file Sukhjeet Ahuja (NAPHSIS), Joyce Martin (NCHS)

Future subgroupsFetal death expert panel

PRIORITIES

Breakfast meetingTuesday @ 7:30amWillow Room

Grab breakfast and come join in the fun!

JOIN US!

BIRTH DATA QUALITY VITAL RECORDS: A CULTURE OF QUALITY NAPHSIS Annual Meeting | Seattle | June 8-11,...

Documents

Transcript of BIRTH DATA QUALITY VITAL RECORDS: A CULTURE OF QUALITY NAPHSIS Annual Meeting | Seattle | June 8-11,...