Broad Data Analysis
description
Transcript of Broad Data Analysis
![Page 1: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/1.jpg)
Child Outcome Data
Broad Data Analysis
![Page 2: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/2.jpg)
Broad Analysis: Child Outcomes
Does our state’s data look different than the national data?
Are our state child outcomes trends stable over time?Is the data trending upwards?Is the data trending downwards?
Is our state performing more poorly in some outcomes than others?
Are the child outcomes similar across programs?
What about data quality? Can we be confident in our data?
![Page 3: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/3.jpg)
Child Outcomes: State vs. National
![Page 4: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/4.jpg)
Child Outcomes: State vs. National
![Page 5: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/5.jpg)
National Vs. State Meaningful Differences
![Page 6: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/6.jpg)
Child Outcomes: National vs. State FFY11 and State FFY12
![Page 7: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/7.jpg)
Data Quality – Trends
![Page 8: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/8.jpg)
Virginia Trends
![Page 9: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/9.jpg)
Virginia Trends
Outcome 3:Use of Appropriate Behaviors to Meet
Needs
Outcome 2: Acquisition and
Use of Knowledge and Skills
![Page 10: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/10.jpg)
Virginia Trends
![Page 11: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/11.jpg)
![Page 12: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/12.jpg)
![Page 13: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/13.jpg)
Child Outcome Results FFY11/FFY12
Meaningful Differences Calculation
![Page 14: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/14.jpg)
Child Outcomes: National vs. State FFY11 and State FFY12
![Page 15: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/15.jpg)
Child Outcomes: Local vs. State
![Page 16: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/16.jpg)
Child Outcomes: Local vs. State
![Page 17: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/17.jpg)
Child Outcomes: Local vs. State
![Page 18: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/18.jpg)
Child Outcomes: Local vs. State
![Page 19: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/19.jpg)
Child Outcomes: Local vs. State
![Page 20: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/20.jpg)
Child Outcomes: Local vs. State
![Page 21: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/21.jpg)
Data Quality ElementsCompleteness of data
number of children reported for the outcome/number who exited
Virginia’s results: average= 65%; range for Local Systems = 17% - 100%
Expected Patterns for Progress Categories
Virginia’s state date is within these parameters for all three outcomes
Child Outcomes State Trends Over TimeAs noted on previous slides, Virginia’s results do not show
wide variations which would trigger concerns about data quality
Category a Category e
0 <5%
>10% >65%
![Page 22: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/22.jpg)
Family Outcome Data
Broad Data Analysis
![Page 23: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/23.jpg)
Broad Analysis: Family OutcomesDoes our state’s data look different than the
national data?Are our state family outcomes trends stable
over time?Is the data trending upwards?Is the data trending downwards?
Is our state performing more poorly in some outcomes than others?
Are the family outcomes similar across programs?
What about data quality? Can we be confident in our data?
![Page 24: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/24.jpg)
![Page 25: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/25.jpg)
Family Outcomes: State Trends over Time
![Page 26: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/26.jpg)
![Page 27: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/27.jpg)
Family Outcomes: Local vs. State 2012-20134A: EI has helped the family know their rights
![Page 28: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/28.jpg)
Family Outcomes: Local vs. State 2012-20134B: EI has helped the family communicate their children’s needs
![Page 29: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/29.jpg)
Family Outcomes: Local vs. State 2012-20134C: EI has helped the family help their child develop and learn
![Page 30: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/30.jpg)
Data Quality
The data analysis for Virginia’ survey is quite extensive, using rigorous data analysis standards.
Virginia’s response rate, like other states that use mailing as the means to disseminate the family survey, is lower than states who use other mechanisms.
![Page 31: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/31.jpg)
Broad Infrastructure Analysis
![Page 32: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/32.jpg)
Analysis MechanismsUse of federal monitoring tools and
proceduresCommunication with Local Systems through:
Ongoing TARegional MeetingsTrainingsSystem Manager Meetings
Monitoring ResultsRecord Reviews
QMRsIndividualized TA
Local System Contract Deliverables
![Page 33: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/33.jpg)
Infrastructure Analysis:Information Sources
Local self-reportingStakeholder inputLSM surveysTraining/meeting evaluationsMonitoring/QMRDispute ResolutionsObservations through TA and
other interactions
![Page 34: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/34.jpg)
Broad Infrastructure Analysis
Governance - MixedStrengths
Code of Virginia establishes infrastructureVICCState Interagency Agreement; interagency
partnershipsDBHDS local contract with Local Lead Agencies
specifies LS infrastructure requirementsWeaknesses
Variable job roles and skill sets of LSMsVariable strength and specificity of LLA/provider
contracts Variable support from LLA leadership
![Page 35: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/35.jpg)
Broad Infrastructure AnalysisFiscal - Mixed
StrengthsAdditional state fundingMedicaid EI Services programTrainings on fiscal issuesStrong working relationship with fiscal office at the
state lead agency (DBHDS)Weaknesses
Variable fiscal skill set of LSMsCode of Virginia does not require fiscal commitment at
local levelVariety of Local Lead Agencies Inconsistent reporting of fiscal data at local levelLack of fiscal data in state data systemPerception/reality that there is not enough money
![Page 36: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/36.jpg)
Broad Infrastructure Analysis
Quality Standards – Not sure Practice Manual articulates expected
practices, but these are not labeled quality practices
Monitoring and Accountability – StrengthLocal self-monitoring and supervision for
continuous improvement – variableProfessional Development – StrengthTechnical Assistance – StrengthData – Weakness
![Page 37: Broad Data Analysis](https://reader035.fdocuments.us/reader035/viewer/2022070405/56813fed550346895daaef5b/html5/thumbnails/37.jpg)
Closing Thoughts
Based on this broad review, did anything
strike you as an area of focus for our Systemic
Improvement Plan?