Post on 29-Nov-2014
description
Proprietary - Not for duplication
American Public University System
Comprehensive Assessment of Student Retention in Online
Learning ProgramsWally Boston, Ed.D. and Phil Ice,
Ed.D.
October 19, 2010
The State of Online Learning
Approximately 4 million students taking online courses in Fall 2008 with projected 12.9% growth rate.
Outpacing face-to-face growth by 6 to 1.
US Department of Education meta-analysis – online is more effective than face-to-face (http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf)
% Growth in U.S. Online Enrollments
Retention
A concern in higher education since the late 1800’s. High School GPA, SAT and/or ACT scores, etc.
(background characteristics) most reliable predictors of retention; key reason why most selective colleges have highest graduation rates.
Tinto, Astin, Braxton and others have demonstrated the importance of academic and social integration for student persistence.
Measures of academic and social integration well defined in the face-to-face setting – not in online.
Online dropout rates much higher than face-to-face – some studies show up to 7 times higher.
APUS Study
American Public University System operates through American Military University and American Public University
Founded in 1991 Accredited by HLC and DETC 79 degree programs encompassing associates’,
bachelor’s, and master’s Approximately 75,000 students as of 6/30/10 100% online, asynchronous courses Monthly course starts
APUS student growth
Strategic Management Problem
• Student enrollment growth beginning in 2006 increased dramatically over previous three years.
• Senior management wanted to measure student retention and see if there factors that influenced persistence.
• APUS maintains comprehensive data warehouse that integrates student demographic data and academic records.
• Selected 2007 as base year.• Selected all undergrad students pursuing degree or
certificate programs.• Determined 2007 student status as of 12/31/2009 as either
active, graduated, or disenrolled (note: APUS automatically disenrolls students after 12 months of not enrolling in at least one class).
APUS Student Body Characteristics
• 95% of students were part-time in 2007.• # of first time, full time freshmen equaled 26.• 83% bachelor’s, 16% associates, 1% certificates• 87% of students transferred credits from previously attended
colleges or from ACE-evaluated military training.• Mean credit hours received in transfer request were 39 credit
hours. 90% of students received less than 60 credit hours.• Active-duty military students represented 69% of
undergraduate enrollment in 2007.• 24% of new students in 2007 dropped out after attending only
two classes.• Gender – Males = 66%, Females = 34%.• Age – Mean = 29.8 and Median = 28.
Method
• 21,521 undergraduates completed at least one courses at APUS in 2007. 20,569 records selected.
• 10,064 active (49%) at 12/31/2009.• 6,858 disenrolled (33%) at 12/31/2009.• 3,647 graduated (18%) at 12/31/2009.• First pass analysis used regression with forward entry.• Independent variables selected of Transfer Credits Received,
Age, Gender, Ethnicity, Cumulative GPA, Last Course Grade Received, Military / Civilian Status, Degree Program, Course Duration, Time Since Last Course
Categorical variables reduced to binary dummy variables and some variables collapsed into buckets
Non relevant data removed from model and re-run.
Sample Groupings
Age bucketed into IPEDs classification Plus and minus grades collapsed into single variable 15 hour transfer credit blocks defined as binary dummy
variables with no transfer credit a separate entry
Results
45 variables were found to be significant predictors of retention
32.8% of variance accounted for by the model No transfer credits – 15.8% No of Courses completed in 2007 – 4.5% Last Grade Received of F – 3.8% Last Grade Received of W (Course Withdrawal) – 2.7% Cumulative 4.00 GPA – 1.4% No other variable over 0.6% No difference in regression outcomes in segregating active
duty military students from civilian students. Race and gender were insignificant variables in this analysis.
Potential Implications
Lack of transfer credit may indicate inadequate academic preparation or part-time students may be overwhelmed with requirements for graduation (mean # of courses completed in 2007 was 3.6 which would not be sufficient to graduate in 10 years IF no transfer credits were received. 74% of no transfer credit students were disenrolled at 12/31/2009.
Registrations are likely a proxy for activity. Graduated students at 12/31/2009 averaged 6.1 courses taken in 2007. Disenrolled students averaged 2.5 courses completed in 2007.
F or W may have significant psychological impact. Department of Defense requires service members to reimburse Department for tuition if grade received is F or W. For disenrolled students, last grade of F = 31% and last grade of W = 15% of that population.
4.0 GPA may be related to students transferring to other institutions. Represents 9% of disenrolled students.
Actionable Intelligence
Students who lack transfer credit may need additional student services.
Given early dropout decision of students, special focus on retention in first three classes may improve overall persistence.
Mandating introductory skills assessment may reduce the number of F’s and W’s received by students and should be a strategic objective.
More research needed on perceived institutional value for 4.0 GPA students.
How representative is data from IPEDS regarding online institutions’ persistence rates?
Selected 10 institutions with large or totally online student populations.
8 institutions were for-profit, 2 were non-profit. Unduplicated ugrad headcount for Fall 2007 ranged
from 6,233 to 334,851 Total unduplicated headcount for Fall 2007 was
624,239. Total FTE headcount for Fall 2007 was 303,206 (high
part-time %’s). Total # of graduates within 150% of normal time for Fall
2007 was 522. Total completions for academic year 2008 was 50,869.
Measuring persistence – IPEDS alternative
Matching completions in a year against unduplicated or FTE headcount is not representative of persistence rate because of high growth rate in online attendance during this period.
Completions/graduates represent an average time to complete of 6.7 years at APUS, thus completions reported to IPEDS represent multiple years when the students started.
Some type of cohort tracking by year would be more appropriate to measure persistence.
Measuring persistence – IPEDS alternative
Averaging Method of Retention Calculation
APUS Undergraduate Students
2007 Calendar Year
Undup Count
Dropped
Graduated
Active Grad + Active
Undup Grad %
Grad + Active
%
20,569 6,858 3,647 10,064 13,711 17.7% 66.7%
APUS Undergraduate Students
2005 Calendar Year
Undup Count
Dropped
Graduated
Active Grad + Active
Undup Grad %
Grad + Active
%
8,921 3,869 2,564 2,488 5,052 28.7% 56.6%
Global Implications
Findings related to Transfer Credit and Courses taken per year are relevant for measuring part-time student participation in higher education. Mean of 6.7 years for APUS bachelor’s student to graduate exceeds current IPEDS measurements in addition to only first time, full time students are tracked.
IPEDS collects data on annual number of completions (graduations of non first time, full time students). Modifying system to measure transfer students would simply require tracking new students at institution by cohort year who were not first time, full time.
Directions for Future Research
Introduction of LMS and student survey data into the model
Model refinement using techniques such as HLM, Decision Trees and Neural Network Analysis
Qualitative inquiry into predictors to develop an explanatory model through triangulation
Multi-Institutional data aggregation and analysis with an eye toward detecting potential differences in models.
Findings on no difference between race/gender should be pursued in studies with other online programs to see if military orientation is explanation or if online/color blind instruction is a factor.