Post on 22-Mar-2016
description
Don’t Waste My Time: Here’s Why Our Data Look Bad and What We
Really Need to Improve the Quality of the Data
NCES MIS Conference 2012
Agenda• Why did we propose this session?• Informal poll grading data quality (DQ)• Identify current resources to support DQ• Feedback/discussion– What would it take to REALLY improve DQ?
Why this Session?• We:– believe responsible use of data is powerful– are concerned about the quality of some of the
data we steward– are concerned about how data are being used* – feel a sense of urgency to get the data right…so
there is a chance the data will be used responsibly, proactively, and often!
How We Want to Look When We Review the Data…
How We Actually Look When We Review the Data…
Photo(s) are stock photos. Release for web use of all photos on file.
Data Quality Assessment
If you did a DQ assessment today…what’s the health of your data?
Photo(s) are stock photos. Release for web use of all photos on file.
Health of IDEA Data At First Submission to ED
“C”
“C+”
Incomplete, missing elements, computational errors, impossible, improbable
Photo(s) are stock photos. Release for web use of all photos on file.
What Does Healthy Data Look Like?
“Fit for use”• EDGB DQ elements– Timeliness– Accuracy– Completeness– Validity– Usability
• Usable…Dependable
Photo(s) are stock photos. Release for web use of all photos on file.
Current Resources to Improve DQ
• Federal• State• From participants
Federal Resources for States• Technical Assistance– General, targeted, or intensive
• Pre-submission tools– Instructions! Q&As! Definitions!– Webinars – conference sessions - calls
• Submission tools– Reports of “impossible” data errors– Reports that look like legacy reports
• Post-submission tools
Resources Within ED
• EDFacts Data Governance!!!!– It ALL goes back to relationships.
• EQuIP
State/District/School Resources
• Georgia• What resources do you use to ensure data
quality?– What is data verification? – What is data validation?– What happens at the SEA and LEA levels?
Do We Agree There’s a Gap Between
Reality and Desired Data Quality?
What would it take to REALLY improve DQ?
• Systems changes?• Personnel?• Tools?• ??????????
So How Do We Take the Next Step?
• School• District• State• Federal
From the Participants During the Session
Currently Doing to Improve DQ:- Creating reports for
intended audience- Coaching- Validation checks- Coding- Providing definitions- Guidance about
improvement
Challenges to Improve DQ:- Late data submissions- Resource allocations- Format violations- Source input errors- Early system maturity
related to SLDS
From Participants During Session: What’s the Difference between Verification and Validation?
Verification- Human checks- Superintendent sign-off- Audits – monitor using
independent source- Personnel training
about data: here’s what you should know
Validation- Something automated- Built via IT- Catch outliers- Year-to-year change
reports- Business rules- Verify reliability
From Participants During Session: What Can USDOE Do to Impact
State Data Quality?• Minimize change – keep data elements stable!• 2-year advance notice for any data changes• Audit paper & electronic files• Use the data or lose the quality• Communicate with states and locals about our data use• More public relations about benefits to using Common Education Data Standards (CEDS)• Less reporting flexibility• Get LEA input about data• ALL Conferences: What’s collected and WHY…Build shared responsibility about the data• Usability testing: Talk about how usable the data are• Include the vendor community in the discussions• Quality control the EDFacts file specifications • Understandable descriptions of data elements (not just for techies)
Thank You
• Bonnie Dye: BoDye@doe.k12.ga.us • Kelly.Worthington@ed.gov• Meredith.Miceli@ed.gov• Bobbi.Stettner-Eaton@ed.gov