Data Driven Retention Strategies for Online Students

Monroe Community College 1

Data Driven Retention Strategies for Online Students

Dr. Jeffrey P. BartkovichMarie J. Fetzner

October 21, 2003

2003 League CIT


Purpose of Presentation

• Provide an overview of MCC’s online research agenda

• Review research that impacts online student retention– Services– Policies

• Discuss online retention strategies that are informed by research data


MCC’s Online Program

• Joined SUNY Learning Network (SLN) in 1997

• Fall 2002: 1,716 non-duplicated headcount– 44.2% FT; 59.8% PT– 37.4% online only; 62.6% OL and other– OL represents 10.7% of MCC’s enrollment– 71.3% Female; 28.7% Male

• Over 120 course sections


MCC’s Online Research Agenda

• Level 1 – Descriptive– Basic demographics– Basic production– Basic enrollment– Basic finance



• Level 2 – Evaluative– Basic performance– Student and faculty satisfaction– Integration with campus IR– Modify existing surveys “to fit”



• Level 3 – Comparative– Statewide benchmarks– National benchmarks– Complete surveys-–be a case study

• Level 4 – Theoretical– External – in the literature– Internal – on the campus


Data-Driven Retention Strategies

Part I – ServicesCharacteristics and Attitudes of

Non-Retained Online Students

Office of Educational Technology ServicesMonroe Community College

Jeff Bartkovich and Marie Fetzner


The Problem

Why are certain MCC online students (those with grades of F/W) not succeeding in their online courses?


MCC’s Online Retention Research

• Archival Data Analysis– Past three years’ grades and demographics

for students in online courses retrieved from student records database

• Student Survey– Survey created and administered to

sample of students who received an “F” or “W” in an online course for the Fall semester in 2000, 2001 and 2002


Study Limitations

• Overall response rate is low—difficulty in obtaining responses from the target population (contact information)

• Small sample size (especially for Hispanic and Asian students)

• Ability to generalize

• Good News: feedback was received from non-retained online students


Basic Student Demographics OL and Site-based, By

Percent*Spring 2002 Fall 2002

OL Site-based

OL Site-based

Female 68 55 71 55

Male 32 45 29 45

Minority 20 25 19 24

Non-Minority 80 75 81 76

Less than 20 18 27 15 30

20-24 33 33 35 32

25-29 16 11 15 11

30 and over 33 29 34 27*SOURCE: MCC Institutional Research Office


Overall Grade DistributionOL vs. Site-based, By

Percent*

*SOURCE: MCC Institutional Research Office

**Includes + and -

Spring 2002 Fall 2002

Grade ** OL Site-based OL Site-based

A 34.5 32.8 34.0 28.7

B 20.5 26.1 19.7 26.2

C 10.0 15.3 10.6 15.2

D 3.9 5.0 3.7 5.1

F 12.9 8.2 14.1 7.4

W 16.3 11.4 14.5 10.7


Grade RatesOL vs. Site-based, Fall

Semester

1999 2000 2001 2002

OL N 970 1,417 1,916 2473

C and Higher

69.7% 65.8% 65.3% 63.6%

F/W 24.8% 26.4% 29.7% 30.2%

MCC N 43,779 45,235 48,288 52,428

C and Higher

70.3% 69.7% 70.7% 70.7%

F/W 20.0% 21.3% 20.6% 20.5%


Success Rates in Comparable Online and Traditional

Courses% C or Better Percentage

Online Traditional

Point Difference

Total 65.3 69.6 -4.3

Full-timePart-time

56.374.1

70.366.7

-13.97.4

1st time FT1st time & at risk1st time & not at risk

40.744.466.3

72.266.674.0

-31.9-22.2-7.7

Under 25 years of age25 year of age or older

55.076.3

67.875.7

-12.80.7


Top 10 Reasons for MCC Non-retained OL

students

When asked to identify the one reason why they were not successful in their online course, students reported that they:

1. Couldn’t handle the balance between school and other responsibilities (15%)

2. Got behind, couldn’t catch up (13%)3. Course was too unstructured (13%)4. Lacked the needed computer skills (13%)5. Personal problems (10%)


Top 10 Reasons, con’t

6. Course was too hard (9%)7. Lacked motivation (5%)8. Course took too much time (5%)9. Space opened up in a campus-based

class (5%)10.Didn’t like the instructor’s teaching

style (3%)


Reasons for Taking an OL Course*

• Reasons for Taking Course Online:– Conflict with Personal Schedule 49%– Family Responsibilities 23%– Distance or Transportation 10%– Other 10%– Course Not Offered on Campus 5%– Interest in Technology/Internet 3%

*Retained Students


Reasons for Taking an OL Course

MCC vs. all other SUNY Students*

• PRIMARY Reason MCC* SUNY*– Personal Schedule Conflict 49% 31%– Family Responsibilities 23% 16%– Distance or Transportation 10% 20%– Other 10% 12% – Not Offered On Campus 5% 15%– Interactive Technology 3% 7%

*Retained Students


Retention Data

• Conducting further analysis to investigate the relationship between online student reasons for non-success, and satisfaction variables for successful online students

• Services that were implemented as a result of the retention study to be discussed after information is presented in Part II


Data-Driven Retention Strategies

Part II - Policies

An Analysis of SLN Students’ Performance and Differentiation

Office of Institutional ResearchMonroe Community College

Angel Andreu


The Problem

It is observed that the failure and withdrawal (F/W) grades for online students are increasing in number and percentage.


The Observation

Percentage of F/W Grades, Fall Semester

1998 1999 2000 2001 2002

Online* 19 25 26 30 30

College 17 20 21 21 21

*N in 1998 was 409; 2002 was 2,473


The Research

Previous research identified characteristics of students at risk of F/W grades in online courses. These were:

– First time students– Full-time students with less than 30

earned credits– Students under the age of 25– Minority students


The Research

The MCC “Andreu” study affirmed these characteristics and looked specifically at:– Time of registration– Basic academic skills


The Data: Time of Registration

• Only 7% of the “C or better” students registered during the first week of classes, compared to 16% of the “F/W” students

• Students who registered during the first week of classes had a 50/50 chance of earning a grade of C or better

• The odds of getting C or better decreased by 13% for each week closer to the first week of classes


The Data: Age

• 65% of the F/W grades were among students under the age of 25

• Between the ages of 21 and 22, a student had a 50/50 chance of earning a C or better


The Data: Age

Probability of Earning

An SLN GPA of C or Better

Figure 7

AGE, Years

605040302010

Pre

dic

ted

pro

ba

bili

ty

1.00

.90

.80

.70

.60

.50

.40


The Data:Age and Time of

Registration

• Age and time of registration are each significant contributors to grade performance

• Their interaction is not significant• The best chance of getting a C or

better– Register 5 weeks before the start of the

semester– Be age 25 or above


The Data: Experience(i.e. number of accumulated credit

hours)

• The rate of F/W grades between first and second year students is significant

• Experience is not significant when combined with age or time of registration


The Data: Part/Full Time Status

• Part/Full time status is not significant • The interaction of status and age was

significant


The Data:Part/Full Time Status

Estimated Marginal Means of SLN GPA

Figure 10

Age Group

30 & Over25 - 2920 - 24Less Than 20

Est

ima

ted

GP

A M

ea

ns

3.2

3.0

2.8

2.6

2.4

2.2

2.0

1.8

1.6

1.4

Status

Full-Time

Part-Time


The Data:Basic Academic Skills

• MCC’s ACCUPLACER tool– The variables used included reading

and sentence scores– The metrics are placement scores at

the college English level (78 for reading, 83 for sentences)


The Data:Basic Academic Skills

• For students to have a 50/50 chance of earning a C or better in an online course, their basic academic skills should be at the college English level


What Does the Analysis Show?

Factors important to success in online courses:– Time of registration– The age of the student– The academic preparedness of the

student



Factors in the decision model:– College is enrollment driven– The online program is supporting

enrollment growth– The demographics of enrollment

growth for the College are the same demographics of non-success in online courses



Policy Factors:– Should policy account for motivation– At what “odds of failure” do we restrict

enrollment– Should online growth be allowed to

continue to support college enrollment growth

– What is the balance between student success and online enrollment growth


Policy Development 1

• In all cases, a student with an ACCUPLACER score below 78 (reading) and/or 83 (sentence) is prohibited from taking an online course at MCC.

• This prohibition is lifted at such time that the student successfully completes the Transitional Studies coursework appropriate to address the deficiency.

• All online sections will be closed to registration one week before the start of class. Qualified students (see definition on next slide) who wish to enroll after this date will be allowed to “green slip” in until the College’s ‘Add’ deadline.



• Qualified students include mature students with a record of success in college-level coursework. Student must also have satisfactory ACCUPLACER scores to have successfully completed the Transitional Studies coursework appropriate to address any deficiency. Students who have successfully completed an online course previously are also considered qualified. Note that data indicate that students 22 years old or older have a significantly greater chance of success in online coursework.



Considerations– Forget policy, make it a course

prerequisite– Must be programmable– Cannot rely upon faculty involvement

for exceptions due to availability– Does resolution of the online problem

become a standard



Draft X– Students in remedial English courses

are prohibited from enrolling in online courses

– All registrations for online courses are closed on the first day of classes


Research to Inform Practice:

Student Advice

Question to online students who received F/W grade on their online course:

“What advice would you give a fellow student considering registering for an online course?”


Research to Inform Practice: Student Advice

• Results – Top Five Topics– Be ready for independent learning. Must

be self-motivated.– Once you get started, stay on track.– Online courses require lots of reading,

homework and research. Get ready for the demands.

– Must develop a schedule to go online, must manage your time.

– Talk to other people, other students before you take the class and during the class



Student Advice

• Practical Use of Online Student Advice– Integrate into brochures, orientation

seminars– Do mock testimonies for video displays – Share with Counseling and Advising staff– Share with Faculty


Research to Inform Practice:Managing Services

• Manage the Expectations– Additional pre-start information sharing– Additional orientations offered

• Manage the Support Services– Greater integration with support team

– Additional points of access to services

• Manage the Academics– Faculty course design issues and training– Emphasis on multiple interactions



• Manage the Expectations– Welcome Letter from Academic Vice

President has been expanded– Face-to-face student orientations offered*

– MCC Online Orientation CD distributed

– Phone Calls made to all Fall 2002 Online Students and 1st-time Spring 2003 Online Students

*additional analysis conducted



• Manage the Support Services– Student Services web page expansion

– Revisions made to the service gateway to online learning web page

– Brochures at Records and Registration and at the Counseling Center



• Manage the Academics– Retention strategies integrated into

faculty trainings

– Discipline-specific retention studies

– Discussions on other academic support—online tutoring, writing centers, etc.

– Pilot test CourseSpace (use of online template for web course enhancement)



On-campus Orientation

Question:

Is there value to an on-campus orientation course for online performance?




On-Campus Orientation Results: – Most students felt they had the necessary skills

to be successful– Skill preparation is not related to performance – Students who attended the on-campus program

were more likely to earn a C or better C/Better F/W

On-campus Orientation 78% 18%Online Orientation only 64% 30%




To Continue the Course– Practice Demonstration Course– Structure not Skills– Market the grade differential– Keep faculty involved in the program to

provide “authentic experience”




Question: How do you define retention?

It depends:– Research–two basic presentations

As an enrollment statistic As a performance indicator

– Policy Be consistent, clarify your statistics


Defining the Retention and Withdrawal Rates

All SLN Courses Fall 2002Annual 2002-03

First day enrollment 1851 4154

First week enrollment 1857 4225

Census day enrollment

1714 3847

Last day enrollment 1716 3832

Retention as an enrollment statistic


Online Retention Rates

Fall 2002 1st Week Census Day Last Day

1st Day 1.02 .94 .95

1st Week .92 .92

Census 1.00

All 2002-03 1st Week Census Day Last Day

1st Day 1.02 .93 .92

1st Week .91 .91

Census 1.00


Defining the Retention and Withdrawal Rates

Retention as a performance measure(All SLN Courses)

Grade Fall 2002Annual

2002-2003

A to D 1,253 2,755

F, W, Other 463 1,077


Withdrawal Rates:All MCC and Online

Fall 2002F&W

Grades

Spring 2003F&W

Grades

2002-2003F&W

Grades

All MCC 20.5% 19.6% 19.5%

Online Only 30.0% 29.5% 28.1%


Next Steps:Things to Consider

• Services • Policies• Definition of “Retention” for data

collection• Replicate studies at other institutions


Contact Information

Dr. Jeffrey P. [email protected]

Marie J. [email protected]

– MCC Web Page: www.monroecc.edu

– MCC Online Learning page:http://www.monroecc.edu/depts/distlearn/index.htm

Data Driven Retention Strategies for Online Students

Documents

Transcript of Data Driven Retention Strategies for Online Students