From a Culture of Guesstimates to a Culture of Evidence
22
From a Culture of From a Culture of Guesstimates to a Guesstimates to a Culture of Culture of Evidence Evidence
description
From a Culture of Guesstimates to a Culture of Evidence. About Speakers. Blaine Bennett Dick Whipple Hector Gonzales Michael Taft Michael Nguyen. About SWTJC. Large service area 11 Counties (16,812 sq. miles) HSI - 82% 50%+ need remediation 42% financial aid AtD 2005 – Round 1. - PowerPoint PPT Presentation
Transcript of From a Culture of Guesstimates to a Culture of Evidence
From a Culture of From a Culture of Guesstimates to a Guesstimates to a
Culture of Culture of EvidenceEvidence
About SpeakersAbout Speakers Blaine BennettBlaine Bennett Dick WhippleDick Whipple Hector GonzalesHector Gonzales Michael TaftMichael Taft Michael NguyenMichael Nguyen
About SWTJCAbout SWTJC Large service areaLarge service area
11 Counties (16,812 sq. 11 Counties (16,812 sq. miles)miles)
HSI - 82%HSI - 82% 50%+ need 50%+ need
remediationremediation 42% financial aid42% financial aid AtD 2005 – Round 1AtD 2005 – Round 1
About AtDAbout AtD AtD GoalsAtD Goals Increase percentage who:Increase percentage who:
Succeed in dev ed classesSucceed in dev ed classes Enroll / succeed in Enroll / succeed in
gatekeeper classesgatekeeper classes ReturnReturn GraduateGraduate
Fall 2005 – Data difficult to acquire. Fall 2005 – Data difficult to acquire. Need to know what’s going on (AtD Need to know what’s going on (AtD data driven) with students. data driven) with students.
Began development of a student data Began development of a student data warehouse - Estudiaswarehouse - Estudias
Wanted to study the dev-ed to gateway Wanted to study the dev-ed to gateway process more closelyprocess more closely (e.g. How long does it take a student to go (e.g. How long does it take a student to go
through dev-ed and complete the gateway through dev-ed and complete the gateway course?)course?)
In The Beginning…In The Beginning…
Data and Data and Questions?????Questions?????
Is data readily accessible for real Is data readily accessible for real time planning & decision making?time planning & decision making?
Can advisors readily access a Can advisors readily access a comprehensive look at a student?comprehensive look at a student?
Can cohorts of students be easily Can cohorts of students be easily identified ? identified ?
Can gaps in student progress be Can gaps in student progress be readily identified?readily identified?
Example: Example: Completion Rates?Completion Rates? Hard to get basic info from Hard to get basic info from
DatatelDatatel Solution: Estudias Data Solution: Estudias Data
WarehouseWarehouse
Developmental placement?Developmental placement? Success rates? Duration?Success rates? Duration?
By course, faculty, cohort, high school, By course, faculty, cohort, high school, placement score, modes of instructionplacement score, modes of instruction
Which are the gateway courses?Which are the gateway courses? Post-gateway success?Post-gateway success?
Probability, time-to-degree/transferProbability, time-to-degree/transfer Interventions?Interventions?
What kind? What audience? Do they What kind? What audience? Do they work?work?
Access to Data Leads Access to Data Leads to More Questionsto More Questions
Levels were tied with course enrollment to Levels were tied with course enrollment to gauge a student’s capacity for success in a gauge a student’s capacity for success in a course.course.
Prior to Estudias, a student/advisor would see Prior to Estudias, a student/advisor would see something like this in the course description:something like this in the course description:
Student must be at math level 1 to enroll; Student must be at math level 1 to enroll; i.e., Accuplacer 30 or greater.i.e., Accuplacer 30 or greater.
Before EstudiasBefore Estudias
Developmental prerequisites at this time Developmental prerequisites at this time were were
best guesstimates!best guesstimates!
Before EstudiasBefore Estudias
From guesstimates From guesstimates to evidenceto evidence
Placement ranges were not Placement ranges were not based on refined data analysisbased on refined data analysis
Cutoff scores hard to Cutoff scores hard to remember, many tests, course remember, many tests, course completionscompletions
Data were not fine-grained / Data were not fine-grained / diagnosticdiagnostic
Looking at class and test history, impossible to see where student is in dev ed sequence
Too Much Data, Not Too Much Data, Not Enough Information Enough Information
SolutionSolution Logistical regression on Logistical regression on
scoresscores Incorporate gradesIncorporate grades Higher grades = higher Higher grades = higher
scoresscores Bottom Line
Test History
Class History
ExampleExample
Susan SmithSusan Smith Accuplacer 78 (high Accuplacer 78 (high
level 2)level 2)
Aaron AbelAaron Abel Accuplacer 63 (barely Accuplacer 63 (barely
level 2)level 2)
Although these students are both in Level 2, they are very different students
Solution: new level Solution: new level based on better based on better
datadata
The score accurately reflects the fact that each of these students has completed one developmental class and more precisely illustrates the differences between them
Old Old LevelLevel
New New ScoreScore
Aaron AbelAaron AbelAccuplacer 63 (barely Accuplacer 63 (barely level 2)level 2) Passed Level 2 with a Passed Level 2 with a DD
Level Level 22
Low Low Level Level 200200
Susan SmithAccuplacer 78 (almost Accuplacer 78 (almost level 3)level 3) Passed Level 2 with a Passed Level 2 with a AA
Level Level 22
High High Level Level 275275
Data and Data and Questions?????Questions?????
Can advisors readily access a Can advisors readily access a comprehensive look at a student?comprehensive look at a student?
Can cohorts of students be easily Can cohorts of students be easily identified ? identified ?
Can gaps in student progress be Can gaps in student progress be readily identified?readily identified?
Is data readily accessible for real Is data readily accessible for real time planning & decision making?time planning & decision making?
Example: Bubble Example: Bubble StudentStudent
Find Bubble Students
Credits > 50 GPA > 2.5 Math: Level 2
(high)
Compare Cohorts
Started 2005 FAPerformance of: Cohort 1: Placed into Level 2 Cohort 2: Placed into Gateway
How many completed Gateway over time?
Examine Cohort in Detail
Cohort 1: 2005FA, placed into Level 2.
How many: Enrolled in Level 2 Completed Level 2 Enrolled Gateway Completed Gateway
Examine Cohort in Detail
How many: Enrolled in Level 2, Completed Level 2, Enrolled Gateway, Completed Gateway
QuestionsQuestions Blaine BennettBlaine Bennett
[email protected]@swtjc.cc.tx.us Dick WhippleDick Whipple [email protected]@swtjc.cc.tx.us Hector Gonzales Hector Gonzales
[email protected]@swtjc.cc.tx.us Michael Taft Michael Taft [email protected]@zogotech.com Michael Nguyen Michael Nguyen [email protected]@zogotech.com
Slides at: www.zogotech.com/atd
