Assessing Distance Assessing Distance Learning Programs – A Learning Programs – A

ModelModel

League for Innovation in the League for Innovation in the Community CollegeCommunity College

Conference on Instructional Conference on Instructional TechnologyTechnology

20062006Mary Beth OrrangeMary Beth Orrange

Professor of MathematicsProfessor of MathematicsJason L. SteinitzJason L. Steinitz

Professor of Social ScienceProfessor of Social ScienceErie Community CollegeErie Community College

Session description in Session description in programprogram

Participants develop a national model for Participants develop a national model for distance learning program assessment, distance learning program assessment, including identifying the variables and including identifying the variables and

discussing issues related to interpreting results. discussing issues related to interpreting results. The assessment plan and the results of the two-The assessment plan and the results of the two-

year study of Erie Community College’s year study of Erie Community College’s distance learning program are presented and distance learning program are presented and

used as the starting point for the development used as the starting point for the development of the assessment model. The goal is to have of the assessment model. The goal is to have

participants develop a list of variables that participants develop a list of variables that provide meaningful measures of student provide meaningful measures of student learning in distance learning programs.learning in distance learning programs.

Outline of this sessionOutline of this session•IntroductionIntroduction•Brief outline of the assessment of Erie Brief outline of the assessment of Erie Community College’s distance learning Community College’s distance learning program presented program presented •Session participants will brainstorm a list of Session participants will brainstorm a list of variables for distance learning assessment variables for distance learning assessment using the think-pair-share collaborative using the think-pair-share collaborative learning techniquelearning technique•Participants will then prioritize variablesParticipants will then prioritize variables•Participants develop a national model for Participants develop a national model for distance learning program assessment distance learning program assessment •Next year’s round table discussion on Next year’s round table discussion on distance learning assessment will be distance learning assessment will be planned. planned.

Erie Community CollegeErie Community CollegeECCECC

• A three campus college with an A three campus college with an approximate FTE count of 10,000approximate FTE count of 10,000

• Part of SUNY (State University of New Part of SUNY (State University of New York) but not part of SLNYork) but not part of SLN

• A full complement of programs (64) A full complement of programs (64) and coursesand courses

• Strong union environmentStrong union environment

Distance Learning at Erie Distance Learning at Erie Community CollegeCommunity College

• Approximately 130 online courses each Approximately 130 online courses each semestersemester

• Approximately 30 telecourses each Approximately 30 telecourses each semestersemester

• Approved by the Commission on Higher Approved by the Commission on Higher Education, Middle States Region, to offer Education, Middle States Region, to offer programs at a distance programs at a distance

• Approved by the State Education Approved by the State Education Department of New York to offer programs Department of New York to offer programs at a distanceat a distance

View ECC’s Assessment View ECC’s Assessment Project ResultsProject Results

• Visit the website Visit the website http://south.ecc.edu/dlhttp://south.ecc.edu/dl

• Click on Assessment>Perkins Year 2Click on Assessment>Perkins Year 2

Assessment itemsAssessment items

• Passing rates of courses taught at a Passing rates of courses taught at a distancedistance

• Passing rates of seated versions of courses Passing rates of seated versions of courses taught at a distancetaught at a distance

• Determination of when the passing rates are Determination of when the passing rates are significantly different between seated and significantly different between seated and distance courses using the chi-square testdistance courses using the chi-square test

• Pivot table in Excel for passing rates for all Pivot table in Excel for passing rates for all four semestersfour semesters

Assessment itemsAssessment items

• Demographics of students enrolled in Demographics of students enrolled in distance coursesdistance courses

• Student satisfaction Survey administered Student satisfaction Survey administered to students enrolled in distance coursesto students enrolled in distance courses

• Self-assessment of skills improvement Self-assessment of skills improvement administered to students enrolled in administered to students enrolled in online, telecourses, and seated coursesonline, telecourses, and seated courses

• Characteristics of faculty members Characteristics of faculty members teaching at a distanceteaching at a distance

Assessment itemsAssessment items

• Reasons for withdrawal from courses at Reasons for withdrawal from courses at a distancea distance

• Study of when students stopped Study of when students stopped working in courses at a distance vs working in courses at a distance vs when they withdrewwhen they withdrew

• Comparison of passing and withdrawal Comparison of passing and withdrawal rates for pairs of courses taught by the rates for pairs of courses taught by the same instructor in the same semester.same instructor in the same semester.

Develop a ModelDevelop a ModelThe The thinkthink part part

Use the next five minutes to Use the next five minutes to make a list of items that you make a list of items that you

think should be included in the think should be included in the assessment model. assessment model.

Develop a ModelDevelop a ModelThe The pairpair part part

Turn to someone seated near you Turn to someone seated near you and spend the next five minutes and spend the next five minutes

discussing your two lists. discussing your two lists. Combine the two lists into one.Combine the two lists into one.

Develop a ModelDevelop a ModelThe The shareshare part part

We will develop a master list We will develop a master list of all the variables listed by of all the variables listed by

each group in the room.each group in the room.

Develop a ModelDevelop a Model

We will prioritize the list using the We will prioritize the list using the quizdome product. quizdome product.

For each item, select:For each item, select:

““a” if you feel the item a” if you feel the item mustmust be be includedincluded

““b” if you feel the item b” if you feel the item maymay be included be included

““c” if the item c” if the item doesdoes notnot belong in the belong in the model.model.

Implement the ModelImplement the Model

Return to your campus and Return to your campus and collect data regarding one of collect data regarding one of

the items we selected.the items we selected.

Refine the ModelRefine the Model

Come back next year and discuss Come back next year and discuss your results at the roundtable your results at the roundtable session on the Assessment of session on the Assessment of

Distance Learning.Distance Learning.

Contact us:Contact us:Jason L. Seinitz, PhDJason L. Seinitz, PhD

steinitz@ecc.edusteinitz@ecc.edu

Professor of Social Professor of Social ScienceScience

(716) 851 1305(716) 851 1305

Mary Beth OrrangeMary Beth Orrange

orrange@ecc.eduorrange@ecc.edu

Professor of Professor of MathematicsMathematics

(716) 851 1784(716) 851 1784

Erie Community College, Erie Community College, South CampusSouth Campus

4041 Southwestern Blvd.4041 Southwestern Blvd.

Orchard Park, NY 14127Orchard Park, NY 14127

See you next year!See you next year!

This PowerPoint show is available at http://south.ecc.edu/orrange