National Learning Infrastructure Initiative

Transformative Assessment Focus Session

March 15, 2002

Mount Royal College Context

Four year undergraduate college Comprehensive credit program base: 11,000

credit students Large credit-free operation: 51,000 registrations

in continuing education, languages institute, international education and Conservatory of Music operations

80% of credit students from Calgary area 240 FT faculty; ~ 350 PT faculty

“Sensibilities” of Mount Royal College’s Assessment Program

Informed by contemporary views and activity that complement institutional cultures

Supported and resourced Transformative/Inclusive (Mertens, 1998) Utilization Focused: R2 (Patton, 1997) Change “aware” (Rogers, 1962)

Fundamental Principles for Developing an Assessment Framework

• A function of our Vision:

Canada’s leading undergraduate college educating and training individuals for success in the new economy

• A function of our Stakeholders and what they Value

• A function of our Mission (Core Purpose):

Student Success & Satisfaction

Assessment is:

Principles for Development: Assessment is a Function of Institutional Mission

MRC’s Core Purpose:

Student Success and Satisfaction

Principles for Development: Assessment is a Function of Institutional Vision

Vision Statement Speaks to the need for:

Canada’s leading innovation/progressiveness

undergraduate college relevance of what we offer

educating & training individuals quality of how its delivered

for success in the new economy value-added benefits

Principles for Development: Assessment is a Function of Institutional Values

MRC’s Values: Student-centered

Progressive, innovative, leading

Personalized teaching & services of highest quality

Education as an instrument for individual,

community & economic development

Open communication, honesty, integrity, respect for individuals

Value-added

What Does This Mean?

MRC’s assessment framework answers the following fundamental questions:

1. What do key stakeholders value from MRC?

2. Are MRC’s educational offerings & services relevant to the needs & values of our stakeholders?

3. Does our student-client focus foster student academic success and satisfaction?

What Does This Mean?

4. Do our educational & business practices meet or exceed nationally/internationally recognized standards for quality?

5. Do our students/clients & other stakeholders realize value-added benefits that enhance their success in the new economy?

6. Is MRC positioned as being among Canada’s most innovative and progressive undergraduate colleges?

Values of Primary Stakeholders

JUDGED ON OUTCOMES

Students, Alumni, Community, Faculty/Staff, Governments, Investors, other comparable

providers.

Who are they? What do they value? What criteria do they use to measure value?

Public Accountability

PERFORMANCE PROGRESS ON VISION

What must we do to achieve our vision? What are our implementation strategies? How will success be defined?

Creating Shared Goals for Institutional Transformation

Identifying/Creating the Context for Change Major goals in strategic documents Priorities Issues of the day Expectations of key stakeholders Perceptions of the institution Demonstrate and communicate quality and

effectiveness of what is done

How To . . .

Institutional Research Assessment Task Group Key Questions: Institution-wide dialogues Project-based work Program-based initiatives (Academic/Service) Collaborative Environmental Sensitivity & Scanning

http://www.petersons.com/ugrad/consider.html

CASE STUDY

Course Adaptation Research Project

Mount Royal College

2000-2002

Classroom Plus

…use of learning technologies to augment, enhance and extend face-to-face instruction.

Course Adaptation for Web-Supported Delivery Pilot Project

Over-riding goal: use of learning technologies to foster more flexible

teaching-learning environments for faculty and students End-result: “hybrid” courses, blending face-to-

face and online in various proportions 20 projects over a two-year period Significant research/evaluation component Funding: Provincial grant (Learning Enhancement

Envelope)

Course Adaptation for Web Supported Activity Project

GOAL #4 To study the impact of such course adaptation

on teaching-learning environments from a variety of perspectives including Impact on the student learning experience Impact on student learning Impact on faculty teaching practice Impact on faculty role, responsibilities and workload

Course Adaptation Research Project: CARP Targeted Outcomes

Reality Based & Results Oriented Instructional Technology and Pedagogical

Quality Models Cost: time is the most important currency Professional “Location” Institutional Diffusion Inventive yet rigorous methods Authentic and visible evidence of student

learning

CARP Project Overview: Research Components

Activity-based costingFlashlight, NCHEMS, Custom

Assessment of teaching,learning and technology

Flashlight, FAST

Evidence for student learningVisible Knowledge Maps

Support Faculty Research Interests

Map Coding & Analysis (QSR Nud.ist & SPSS)

Visible Knowledge Class Benchmarked Learning Map Individual Learning Maps Consistency and frequency of key conceptual ideas Accuracy and application of key conceptual ideas Correlations between time concepts are introduced and their appearance on maps Significance of change over time Relationship between measures (e.g. extent to mastery) Relationship between Flashlight results and student grades Totality of knowledge, sources of learning