Karen Swan, Scott Day, Leonard Bogle & Traci van Prooyen

University of Illinois at Springfield

Exploring MOOC Pedagogies

American Council on Education’s (ACE) College Credit Recommendation Service isevaluating MOOCs for content and approving them for specific college credits if students pass ACE CREDIT tests ($150).ACE CREDIT evaluates MOOCs for their content; we are reviewing MOOCs for their pedagogical approaches.

CONTEXT

We developed an instrument to characterize the pedagogical approaches taken in MOOCs --

modeled on a similar instrument proposed by Tom Reeves (1996) for describing the pedagogical approaches found in computer-based instruction --“Pedagogical dimensions are concerned with those aspects of design and implementation . . . that directly affect learning.”

AMP (Assessing MOOC Pedagogy) tool

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

AMP TOOL

METHODSBy Fall 2013, ACE evaluated and approved 5 Coursera, 1 EdX, and 7 Udacity MOOCs -- College Algebra, BioElectricity, Genetics, Pre-Calculus, and Single Variable Calculus, Circuits, Introduction to Artificial Intelligence, Introduction to Computer Science, Introduction to Physics, Introduction to Statistics, Introduction to Parallel Programming, 3-D Modeling and HTML 5 Game Development

We reviewed the same 13 MOOCs and four other Coursera MOOCs that covered non-STEM courses.

METHODSWe iteratively revised the AMP tool as we tested its efficacy on the approved MOOCs, first by reducing the scale of each dimension from ten to five, and then by establishing specific criteria for each dimension, to improve inter-rater reliability. We reviewed all the approved MOOCs using the revised instrument.

Four researchers reviewed each MOOCs. Inter-rater reliability was >80% on all the reviews before meeting to achieve consensus on the ratings.

The approved COURSERA courses followed a format that resembled the traditional lecture/text/testing routine of traditional university courses, including cohorts and weekly hard deadlines.

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupport

ed) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupport

ed) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

college algebra

bioelectricity

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupport

ed) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupport

ed) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

genetics & evolution

pre-calculuspedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupport

ed) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupport

ed) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

single variable calculus pedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

college algebra bioelectricity genetics & evolution

pre-calculus single variable calculus

circuitspedagogical

category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more structure)

approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

The Coursera courses approved for ACE credit, as well as EdX’s Circuits, tended to be objectivist, teacher-centered, convergent, highly structured, more abstract than concrete, with minimal feedback. The courses tended to fall somewhere in the middle between supporting and not supporting cooperative learning, accommodating and not accommodating individual differences, artificial and authentic activities/assessment, and passive and active user roles.

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)

cooperative learning (unsupported) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

college algebra bioelectricity genetics & evolutionpre-calculus single variable calculus circuits

UDACITY courses followed a format that resembled programmed learning, including self-pacing and embedded quizzes.

interactive 3-d graphicspedagogical

category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)

cooperative learning (unsupported) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

html 5 graphicspedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

pedagogical category ratings

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative

learning (unsupported) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupported) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

introduction to artificial intelligence

introduction to computer sciencepedagogical

category ratingsepistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more structure)

approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupported) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupported) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

introduction to parallel processingpedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

introduction to physicspedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

introduction to statisticspedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to

content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5

(frequent, constructive

)cooperative

learning(unsupport

ed) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupport

ed) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

Udacity courses tended to be neither objectivist nor constructivist, slightly less teacher-centered and convergent than Coursera courses, highly structured, halfway between abstract and concrete, with immediate, clear, and constructive feedback. They also tended not to support cooperative learning, but because of their self-directed approach were quite accommodating of individual differences. Udacity also made an effort to develop authentic activities, and perhaps because of the large number of CS courses, supported a more generative user role than the ACE-approved courses from Coursera and EdX.

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)

focus of activities (convergent) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)

cooperative learning (unsupported) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

interactive 3-d graphics HTML 5 graphics intro to artificial intelligence intro to computer science intro to parallel processing

intro to physics intro to statisics

  EDX COURSERA UDACITYepistemology 1.0 1.0 2.4role of teacher 1.0 1.4 2.0

focus of activities 1.0 1.0 1.9structure 5.0 5.0 4.9

approach to content 4.0 3.6 3.0feedback 3.0 2.0 4.3

cooperative learning 2.0 2.8 2.1accommodation of individual

differences 2.0

2.6 3.0activities/assessment 1.0 2.6 3.3

user role 2.0 3.0 3.1average ratings across platforms

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

Coursera EdX Udacity

comparisons of ACE approved courses across platforms

art & inquirypedagogical

category  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

comic books & graphic novels

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

jazz improvisation

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

the music of the Beatles

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

art & inquiry comic books & graphic novelsjazz improvisation the music of the Beatles

non-stem

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

art & inquiry comic books & graphic novelsjazz improvisation

non-stem

  STEM BEATLES NON-STEMepistemology 1.0 1.0 4.7role of teacher 1.4 1.0 3.0

focus of activities 1.0 1.0 4.3structure 5.0 4.0 3.7

approach to content 3.6 2.0 2.3feedback 2.0 2.0 3.7

cooperative learning 2.8 2.0 3.0accommodation of individual

differences 2.6 2.0 2.7activities/assessment 2.6 1.0 4.7

user role 3.0 2.0 4.3average ratings across domains

pedagogical category

  ratings  

epistemology (objectivist) 1 2 3 4 5 (constructivist)

role of teacher (teacher-centered) 1 2 3 4 5 (student-

centered)focus of activities (convergent

) 1 2 3 4 5 (divergent)

structure (less structure) 1 2 3 4 5 (more

structure) approach to content (concrete) 1 2 3 4 5 (abstract)

feedback (infrequent, unclear) 1 2 3 4 5 (frequent,

constructive)cooperative learning (unsupporte

d) 1 2 3 4 5 (integral)

accommodation of individual

differences(unsupporte

d) 1 2 3 4 5 (multifaceted)

activities/assessment (artificial) 1 2 3 4 5 (authentic)

user role (passive) 1 2 3 4 5 (generative)

STEM Beatles non-STEM

(objectivist) 1 2 3 4 5 (constructivist)

(teacher-centered) 1 2 3 4 5 (student-

centered)(convergent

) 1 2 3 4 5 (divergent)

(less structure) 1 2 3 4 5 (more

structure) (concrete) 1 2 3 4 5 (abstract)

(infrequent, unclear) 1 2 3 4 5

(frequent, constructiv

e)(unsupporte

d) 1 2 3 4 5 (integral)

(unsupported) 1 2 3 4 5 (multifacet

ed)(artificial) 1 2 3 4 5 (authentic)

(passive) 1 2 3 4 5 (generative)

two metaphors for learning (Sfard, 1998)

acquisition

participation

CONCLUSIONS• AMP tool can effectively characterize

MOOC pedagogies• At least two distinct pedagogical

approaches can be distinguished• Future research will explore these

distinctions, as well as MOOCs offered on different platforms & differently organized

• Future research will try to link pedagogical to learning outcomes

lbogl1@uis.edu sday1@uis.edu

kswan4@uis.edutvanp2@uis.edu