Authoring Assessment-Rich Learning Environments:A Faculty Development Tool to Drive Transformation

The Advanced Educational Computing Project: A Boutique Model

EDUCAUSE NLII Conference, New Orleans, 29 January 2001

--------------------------<<>>-------------------------

Donald P. Buckley, Ph.D.Associate Professor of Biology

Director of Instructional Technology, School of Health SciencesQuinnipiac University; Hamden, CT 06518

Computerworld Smithsonian Laureate Apple Distinguished Educator

email: don.buckley@quinnipiac.eduWWW: http://faculty.quinnipiac.edu/health/biology/buckley/welcome.html

--------------------------<<>>-------------------------

In memory of

Dr. Karen L. Smith

Director, Faculty Center for Teaching and Learning

University of Central Florida

Overview

• Pedagogical Feature Set of Interactive Multimedia LearningWare

• Interactivity fosters active learning

• Sensory rich information formats enable brain-based teaching/learning

• Communication tools promote social construction of knowledge

• Computers facilitate routine assessment

• Course Management Systems

• Web-delivered content

• Routine formative assessment

• Communication tools to promote cooperative learning styles

• Faculty Development

• Technology integration …to serve learning-centered pedagogies

• Transition to the Learning Paradigm

• Faculty development should be transformational & community wide

Tra

nsf

orm

atio

nT

ran

sfo

rmat

ion

Sca

lab

ilit

y

Institutional Change Is Necessary,But Faculty Populations Have

Structure:

Institutional change means that transformational experiences must be enticing to individuals less tolerant of complexity, ambiguity or failure …

mid and late adopters (Conservatives and Skeptics).

Technology Often Heralded as a Solution

InstructionalParadigm

LearningParadigm

Faculty Practices Pedagogical Potential of the Technology

Faculty still operating within the Instructional Paradigm will not recognize the value of advanced instructional technology, which is best suited to supporting learning

centered pedagogies.

Also, most Faculty are Naive about Pedagogical Feature Set

InteractiveInteractive Multimedia: Multimedia: • • interactivity fosters active-learninginteractivity fosters active-learning

Interactive Interactive MultimediaMultimedia: : • • different information formats engage different cognitive different information formats engage different cognitive processes and therefore open different learning opportunitiesprocesses and therefore open different learning opportunities

Communication Tools:Communication Tools:• • promote cooperative learning and the social construction ofpromote cooperative learning and the social construction of knowledgeknowledge

Computing components:Computing components:• • simulations provide experience in the process of inquirysimulations provide experience in the process of inquiry• • authoring tools promote student construction of meaningauthoring tools promote student construction of meaning• • powerful formative assessment tools are multi-facetedpowerful formative assessment tools are multi-faceted

Traditional Technology Training

Limited potential (e.g., slide show authoring)

o Because “faculty don’t have the time to commit to deeper efforts”

Problem: making a better lecture is not transformational

o It doesn’t foster transition to learning-centered pedagogies

o Faculty wonder “why spend all that effort for limited gains?”

Result: limited faculty willingness to participate in more training

Solution: Up the Ante

Capture the pedagogical high ground …focus on learning & inquiry.

Focus on genuinely transformational faculty development to change faculty attitudes and

behaviors

Authoring experiences can be transformational

AEAECC

U N I V E R S I T Y U N I V E R S I T Y O F O F H A R T F O R D H A R T F O R D

Advanced Educational Computing

A Tool To Foster Student-Centered Learning

Advanced Educational Computing

A Tool To Foster Student-Centered Learning

FIPSFIPSEE

NSFNSFCulpeCulpeperper

Deep Authoring Is Transformational Experience

It was enormously stimulating for most participants

to create learning environments that would

enable them to teach things that they

could not teach well before.

It was enormously stimulating for most participants

to create learning environments that would

enable them to teach things that they

could not teach well before.

Full-fledged Interactive Multimedia Authoring Chosen for Training

Full-fledged Interactive Multimedia Authoring Chosen for Training

Because Programming Was Required

To Meet Faculty Functionality Goals

To customize interfaces that allow students to explore the content areas in intuitive ways

To build simulations that emulate the real world

To build formative assessment tools

Because Programming Was Required

To Meet Faculty Functionality Goals

To customize interfaces that allow students to explore the content areas in intuitive ways

To build simulations that emulate the real world

To build formative assessment tools

Major Features of Faculty Development ProgramMajor Features of Faculty Development Program

1. Full-fledged Interactive Multimedia Authoring …Transformational

2. But our goal was NOT to create multimedia authors

3. Emphasize pedagogy …this is not about “learning technology.’’

4. Keep the technology transparent ! Technology is the mentor’s job.

5. Heavy reliance on Faculty Mentoring Faculty and other collaboration.

6. High quality support necessary to minimize risk and maximize success.

7. Promote a culture of teaching reform and pedagogical scholarship.

8. Create a versatile faculty reward structure that values innovation.

1. Full-fledged Interactive Multimedia Authoring …Transformational

2. But our goal was NOT to create multimedia authors

3. Emphasize pedagogy …this is not about “learning technology.’’

4. Keep the technology transparent ! Technology is the mentor’s job.

5. Heavy reliance on Faculty Mentoring Faculty and other collaboration.

6. High quality support necessary to minimize risk and maximize success.

7. Promote a culture of teaching reform and pedagogical scholarship.

8. Create a versatile faculty reward structure that values innovation.

Major Features of Faculty Experience

Focus on Focus on Pedagogical Innovation Pedagogical Innovation & Student & Student LearningLearning

Train clusters of 3-4 facultyTrain clusters of 3-4 faculty from a discipline (interdisciplinary next time) from a discipline (interdisciplinary next time)

First, First, 30 hours of group introduction30 hours of group introduction (1:1 trainee:trainer ratio) (1:1 trainee:trainer ratio)

Then, committed and Then, committed and prolonged individual mentoringprolonged individual mentoring

Let each faculty member build a tool to solve a problem in Let each faculty member build a tool to solve a problem in their content areatheir content area

Author small projectsAuthor small projects to ensure success and formative development to ensure success and formative development

Don’t let faculty failDon’t let faculty fail

Build a faculty Build a faculty reward structurereward structure …development, class, grants, presentations …development, class, grants, presentations

Summary of Core Training ConceptsSummary of Core Training Concepts

Use Technology that Provides Transformational Use Technology that Provides Transformational

Pedagogical ExperiencesPedagogical Experiences

But Keep the Technology TransparentBut Keep the Technology Transparent

Focus on Pedagogical InnovationFocus on Pedagogical Innovation

Build Collaborations Build Collaborations

Seek the Eager-BeaversSeek the Eager-Beavers

Use Technology that Provides Transformational Use Technology that Provides Transformational

Pedagogical ExperiencesPedagogical Experiences

But Keep the Technology TransparentBut Keep the Technology Transparent

Focus on Pedagogical InnovationFocus on Pedagogical Innovation

Build Collaborations Build Collaborations

Seek the Eager-BeaversSeek the Eager-Beavers

Recruitment Preconceptions

Recruitment Preconceptions

Humanists would be reluctant ...wrong!

Specialists would be naturals ...wrong!

educational specialists

cognitive types

techies ...including toy-collectors

REALITY

Train the eager-beavers

Eager-beavers pop up in surprising places

Look for people with a history of service ...risk takers, “altruists”

Making eager beavers successful makes other eager beavers

Humanists would be reluctant ...wrong!

Specialists would be naturals ...wrong!

educational specialists

cognitive types

techies ...including toy-collectors

REALITY

Train the eager-beavers

Eager-beavers pop up in surprising places

Look for people with a history of service ...risk takers, “altruists”

Making eager beavers successful makes other eager beavers

Authoring Tool Feature Set

WYSIWGWYSIWGEnvironmentEnvironment

ScriptingScriptingEnvironmentEnvironment

Media Media IntegrationIntegration

for example:for example: SuperCard SuperCard or or ToolBoxToolBox

Ease of Ease of UseUse

ComprehensiveComprehensiveContent Editing Content Editing

Tool SetTool Set

WWWWWWAuthoringAuthoring

AuthoringAuthoringMetaphorMetaphor

Which Way to Institutionalization?

Media CenterMedia Center

Media CenterMedia Center

Faculty Development

Slide show presentations

Faculty Development

Slide show presentations

GOAL:GOAL:

sophisticated sophisticated practitioners practitioners

&&

pedagogical pedagogical innovatorsinnovators

Faculty DevelopmentMultimedia authoring

Faculty DevelopmentMultimedia authoring

Scale of Faculty Participation in AEC Scale of Faculty Participation in AEC

ProgramProgram

Art 54

Biology 2

Business 4

Chemistry 4

Communication 7

Computer Science 4

Education 5

Engineering/Technology 6

English/Humanities 6

Foreign Languages 2

Health Professions 5

Hillyer College/Humanities 3

Judaic Studies 1

Music 3

Physics 1

Political Science 3

Psychology 5

Sociology 4

Two Year Total (1 June 1996) 74

Number of faculty trained:74 out of a population of 320

Art History

AEC-Trained Faculty?

non-AEC AEC total

innovator 31 14 45-0.027563 0.041345

FacultySelf middle adopter 64 31 95

Images -0.218159 0.327238

late adopter 22 7 290.429189 -0.643783

total 120 52 172

Chi-square = 0.7558 ns (p=0.6853), 2 df

table contents:CountStandardized Residuals

AEC-Trained Faculty?

non-AEC AEC total

innovator 31 14 45-0.027563 0.041345

FacultySelf middle adopter 64 31 95

Images -0.218159 0.327238

late adopter 22 7 290.429189 -0.643783

total 120 52 172

Chi-square = 0.7558 ns (p=0.6853), 2 df

table contents:CountStandardized Residuals

Faculty Self-Description of Developmental Faculty Self-Description of Developmental StageStage

No significant differences were observed between AEC-trained faculty No significant differences were observed between AEC-trained faculty and other faculty. and other faculty.

This suggests that AEC outcomes are not based on preferential This suggests that AEC outcomes are not based on preferential recruitment of recruitment of

evangelists into the program.evangelists into the program.

Faculty Attitudes About Student Faculty Attitudes About Student MotivationMotivation

AEC Faculty tend to be more optimistic that educational AEC Faculty tend to be more optimistic that educational technology can improve student motivation to learn.technology can improve student motivation to learn.

Does educational technology improve student motivation to learn?

AEC-Trained Faculty?

non-AEC AEC total

Agree 72 41 113-0.812245 1.24413

FacultyAttitudes Disagree 22 4 26

0.883004 -1.35251

No Opinion 28 7 350.698406 -1.06976

total 122 52 174

Chi-square = 6.4* (p=0.0398), 2 df

table contents:CountStandardized Residuals

Does educational technology improve student motivation to learn?

AEC-Trained Faculty?

non-AEC AEC total

Agree 72 41 113-0.812245 1.24413

FacultyAttitudes Disagree 22 4 26

0.883004 -1.35251

No Opinion 28 7 350.698406 -1.06976

total 122 52 174

Chi-square = 6.4* (p=0.0398), 2 df

table contents:CountStandardized Residuals

Faculty Attitudes About Student Faculty Attitudes About Student AbilityAbility

AEC Faculty tend to be more optimistic that educational AEC Faculty tend to be more optimistic that educational technology can improve student ability to learn.technology can improve student ability to learn.

Does educational technology improve student ability to learn?

AEC-Trained Faculty?

non-AEC AEC total

Agree 70 41 111-0.780972 1.17146 0

FacultyAttitudes Disagree 19 2 21

1.17011 -1.75516 0

No Opinion 28 9 370.471159 -0.706739 0

total 117 52 169

Chi-square = 7.153* (p=0.0280), 2 df

table contents:CountStandardized Residuals

Does educational technology improve student ability to learn?

AEC-Trained Faculty?

non-AEC AEC total

Agree 70 41 111-0.780972 1.17146 0

FacultyAttitudes Disagree 19 2 21

1.17011 -1.75516 0

No Opinion 28 9 370.471159 -0.706739 0

total 117 52 169

Chi-square = 7.153* (p=0.0280), 2 df

table contents:CountStandardized Residuals

Faculty Multimedia UseFaculty Multimedia UseAEC Faculty were trained in interactive multimedia authoring.AEC Faculty were trained in interactive multimedia authoring.

These data indicate that AEC-trained faculty exhibit These data indicate that AEC-trained faculty exhibit much greater tendency to use multimedia.much greater tendency to use multimedia.

AEC-Trained Faculty?

non-AEC AEC total

Often 11 11 22-1.12428 1.71998

FacultyMultimedia Sometimes 33 21 54

Use -0.785957 1.20239

Never 66 15 811.22766 -1.87813

total 110 47 157

Chi-square = 11.32** (p=0.0035), 2 df

table contents:CountStandardized Residuals

AEC-Trained Faculty?

non-AEC AEC total

Often 11 11 22-1.12428 1.71998

FacultyMultimedia Sometimes 33 21 54

Use -0.785957 1.20239

Never 66 15 811.22766 -1.87813

total 110 47 157

Chi-square = 11.32** (p=0.0035), 2 df

table contents:CountStandardized Residuals

Overall Information Technology Use

AEC-Trained Faculty?

non-AEC AEC total

0 77 18 951.23065 -1.9004

Numberof Types of 1 21 11 32

Information -0.325482 0.502616Technology

Required 2 14 12 26In Classes -1.00893 1.55801

≥3 12 11 23-1.04448 1.61291

total 124 52 176

Chi-square = 12.62** (p=0.0055), 3 df

table contents:CountStandardized Residuals

AEC-Trained Faculty?

non-AEC AEC total

0 77 18 951.23065 -1.9004

Numberof Types of 1 21 11 32

Information -0.325482 0.502616Technology

Required 2 14 12 26In Classes -1.00893 1.55801

≥3 12 11 23-1.04448 1.61291

total 124 52 176

Chi-square = 12.62** (p=0.0055), 3 df

table contents:CountStandardized Residuals

1. Faculty report ability to teach new things

2. Early data indicate improved student learning ...e.g., class means increased by as much as 3

grade levels

3. AEC has created a forum on active-learning styles

4. Greatly increased cross-disciplinary collaboration

5. Changed faculty culture ...increased adoption of advanced educational technology tools & learning

centered pedagogies

1. Faculty report ability to teach new things

2. Early data indicate improved student learning ...e.g., class means increased by as much as 3

grade levels

3. AEC has created a forum on active-learning styles

4. Greatly increased cross-disciplinary collaboration

5. Changed faculty culture ...increased adoption of advanced educational technology tools & learning

centered pedagogies

AEC Project Outcomes

AEAECC

FIPSFIPSEE

NSFNSFCulpeCulpeperper

Edward KlonoskiEdward Klonoski ◊ ◊ UNIVERSITY UNIVERSITY OFOF HARTFORD HARTFORD ◊ ◊ Donald BuckleyDonald Buckley

Project Outcomes:

Institutional Change in

Faculty Attitudes

and Technology Use in

Teaching

Project Outcomes:

Institutional Change in

Faculty Attitudes

and Technology Use in

Teaching

Overview

• Pedagogical Feature Set of Interactive Multimedia LearningWare

• Interactivity fosters active learning

• Sensory rich information formats enable brain-based teaching/learning

• Communication tools promote social construction of knowledge

• Computers facilitate routine assessment

• Course Management Systems

• Web-delivered content

• Routine formative assessment

• Communication tools to promote cooperative learning styles

• Faculty Development

• Technology integration …to serve learning-centered pedagogies

• Transition to the Learning Paradigm

• Faculty development should be transformational & community wide

Tra

nsf

orm

atio

nT

ran

sfo

rmat

ion

Sca

lab

ilit

y

Problem with IMM Training: Scalability

Authoring IMM LearningWare is a deep experience

Problem …very effort intensive

Faculty do become sophisticated consumers of LearningWare

Another kind of authoring experience needed, to scale up and integrate pedagogies/technologies

Course Management Systems?

Institutional DB

CMS Database

Course Management Systems:The Enabling Technology

Infrastructure?

Faculty

Content Comm Tools Assessment

Student Experience on the Web

Some Emergent Goals for Utilizing CMS Technology:

Technology-assisted Facilitation of Learning-centered Teaching Styles

Content Delivery

Communication

Assessment

LectureContent delivery

ActivitiesProblem-basedProject-basedCase-based

Episodic PervasiveRoutine reflection within and among

teams

Summative Formative

Teacher-centered Learner-centered

CMS Pedagogical Tools A Continuum of Teaching Styles

A model for coupling the feature set of course management systems to learning centered principles.

Smart Tutor

Web-based Homework System: routine formative

assessment out of class time

Research Simulation

Emulating the Process of Professional Investigation

Revision of Class Timecontent delivery system

from Lecture to Discovery Activities

Mitigating the Coverage DilemmaDevelop Epistemological Skills

Students Need to Build Meaning

The Coverage Dilemma

Coverage

Learning& Inquiry

NOW

Emphasis on Delivery of Content

Emphasis on Effective Learning

A Solution to the Dilemma?

GOAL

Coverage

Emphasis on Delivery of Content

Learning& Inquiry

Emphasis on Effective Learning

Can we use technology to mitigate

the Coverage Dilemma?

Routine Online Assessment In Class

TraditionalApproach

WebAssisted

Foundational InformationInquiry-orientation and powerful pedagogies

smart tutor homework