Post on 14-Dec-2015
Closing the Loop on Student Learning Assessment
Mohua Bose, Ph.D Mary Beth Hanner, Ph.D, RN, ANEF
Director of Outcomes Assessment Provost and Chief Academic Officer
Excelsior College, Albany, NY
Presentation at MSCHE, December 12, 2011
Purpose
Describe a framework that addresses assessment of student learning in a non-traditional degree completion institution.
Discuss improvements based on assessment results at the program and institutional level.
Describe the process of facilitating an assessment culture.
Background
Excelsior College founded in 1971 (previously Regents College).
1998 Chartered as a private, independent college.
Large student body (approximately 33,000 students).
Distance education, flexible learning format.
Philosophy:
“What you know is more important that where or how you learned it”
Background
Valuing prior learning is a central premise for adult learners (Green, 1998).
Acceptance of a variety of credit sources:Examples: Regionally accredited institutions ACE/PONSI approved courses Exams – CLEP, DANTE,EC EC portfolio assessment for prior learning Online courses at Excelsior and preferred providers
Residency requirement (presently minimal, historically none).
Background
Accreditation Middle States (all programs) NLNAC (specialized accreditation - Nursing) ABET (specialized accreditation - Technology) IACBE (specialized accreditation - Business)
Challenge:Who is a typical student?
Career oriented working adult
Average age: 39 year
College 1
College 4
College 3
College 2
College 5
Excelsior College
No first time, full time students.Credits transferred from other approved sources ranges from 25% to
75% of total credits required for the degree program.
At a distance
Accountability and Continuous Improvement
OAIR Steering Committee
OAIR Office
School ofBusiness
& Tech
School of Liberal
Arts
School of
Nursing
School of Health Sciences
Institutional Assessment Plan for Student Learning (IAPSL)
Institutional Assessment Plan for Student Learning (IAPSL)
Provides a framework for assessment of learning outcomes at course, program and institutional level.
Addresses the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011).
IAPSL link: http://www.excelsior.edu/web/excelsior-the-institution
Establish clear learning
outcomes
Use the ResultsProvide learning
opportunities
Assess Student Learning
Assessment Process at Schools
Step 1: Establish Learning Outcomes
Ultimate responsibility to determine learning outcomes for a program rests with the faculty.
Instructional/Adjunct faculty receive an online faculty training that orients them to EC. The orientation includes an introduction to learning outcomes assessment.
Development of online courses requires
completion of a structured template that involves program, course and module outcomes.
Step 1: Establish Learning Outcomes (continued)
A raised awareness of the importance of learning outcomes assessment at the College due to OAIRSC and OAIR offices. Senior level personnel significantly involved (eg. Deans).
In addition to OAIR, an assessment director within the schools work with the faculty.
Continuous and evolving process in defining learning outcomes.
Step 2: Create Learning Opportunities
Monitoring the learning opportunities provided to students:
Course revisions based on:Centralized student course evaluationsFaculty evaluation of coursesFaculty curriculum committee
Curriculum reviews
Example: Curriculum Review
Committee of 3 reviewers (some members are completely external to the college).
2-3 days including two teleconferences. Reviewers examine curricular structure, topical
areas covered, course descriptions. Provide feedback on curriculum rigor,
curriculum currency, and program outcomes attainability and currency.
Feedback from the external curriculum review is taken back to program faculty.
Step 3: Assessing Student Learning
Direct assessments of program outcome measures for all programs using:oCapstones, Portfolios, or Exams (theory or
performance)o State licensing exams (e.g. NCLEX)
Direct assessments of general education outcomes using:o ETS Proficiency Profile & VALUE Rubrics
Step 3: Assessing Student Learning
Also have indirect assessments:
Alumni surveys
Supervisor surveys
Centralized for all degree programs
Associate Degree in Nursing
NCLEX pass rates Consistently at or above national average
over a decade. Dipped below for one quarter in national
and state pass rates by approximately 1%. Conducted analysis of subscales on the
NCLEX exam. Examined the CPNE exam.
Associate Degree in Nursing
Analysis of the data revealed that students scored lower on o Knowledge of adolescent growth and
developmentoCare of the ill adolescent
After faculty review – new curriculum requirements.
Growth and Development across the Lifespan.
Associate Degree in Nursing
After approximately 2 years• Scores exceeded both state and national
levels• NCLEX scores in overall exam and
subscales
Remained up for a number of years.
Continuing to monitor.
General Education
Assess WritingOutcome: Write with grammatical
competence, clarity, and coherence
Administered standardized assessment: ETS Proficiency Profile
Reading Writing Mathematics Critical thinking
General Education
ETS Proficiency Profile administered annually 2009, 2010, and 2011.
Results pertaining to Writing show that Excelsior students at the Associate’s and Bachelor’s level have performed at or above the ETS norms for writing.
General Education
College decided to use multiple measures to assess writing.
Use of VALUE rubrics in writing Capstone papers from Bachelor’s degree
programs across the College used. Independent panel of reviewers. Data collection completed, shows
variations in student performance. Closing the loop takes time.
Key Steps in Implementing Outcomes Assessment
Institutional support and commitment. Creation of a college-wide oversight
committee to set a relevant and strategic agenda for assessment of student learning and institutional performance.
A visionary and guiding document for implementation of outcomes assessment College-wide.
Key Steps in Implementing Outcomes Assessment
Training and implementation of an assessment management tool/database.
Centralized assessment model supported by a distributed model of assessment staff.
Showcase assessment results - Quarterly Dean’s presentations at Steering Committees.
Bi-annual assessment audits. Continuous and evolving process!