Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle
description
Transcript of Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle
Evolution of Institutional Capacity to support the Assessment-Change-Effectiveness Cycle
Course Content & Design
Implementation Examples
Assessment Strategiesby Dr. Greg Witteman, U. Guam
The big questions• What are students’ abilities before taking the class?
– What Knowledge, Skills, or Abilities (KSAs) should the students enter the class with?
• What are the major objectives of the class?– What KSAs should the students leave the class with?
• What is the role of assessment in course design or delivery? – Can outcomes be improved? (and how would we
know?)
Type of Course Content
• Self contained Content (no prerequisites)– Examples: Anatomy and Physiology I, and II
• Prerequisite knowledge needed– Examples: Conservation Biology and
Comparative Vertebrate Biology labs
Abilities (student) Objectives (Course) Assessment (results)
Self contained course content
• Anatomy and Physiology I & II
– Assumes no prior knowledge – Each part of the course builds on knowledge
gained earlier in the course (hierarchy)
Chem. Cytology Histology Organs Systems
Abilities (student) Objectives (Course) Assessment (results)
Course content assumes KSAs
• e.g. Without general biology, math and ecology species population size, distribution and probability of extinction can not be meaningfully presented
E.S policy Defining C.B. Demography Methods and tools
General Biology
Ecology
Envtl. Sci. Eco.
Lab Math
Cons.Bio.
Previouscourses
Abilities (student) Objectives (Course) Assessment (results)
Course Design and Delivery Strategies
Course content – (what type of subject matter and students?)
Course goals – (what do we hope to accomplish?)
• What should students be able to do to finish the course?
• What should they know after the course ends?
– Delivery and Assessment methods • (how and when do we know if we accomplished our goals ?)
The Ranges of KSA’s and goals for Science Courses
No Knowledgelimited
background
Have Knowledge from: H.S.,
College, other
Lower Division non-Sci. Majors
Lower Division Science Majors
Upper Division Science Majors
Remember Understand
Remember Understand
Analyze Apply
Create
Entry KSAs
Courses
Course goals
Abilities (student) Objectives (Course) Assessment (results)
Example 1: Anatomy and Physiology• Non-science majors (freshmen/sophomore)
– Extreme range of educational and cultural backgrounds
• Goal: “Remember and Understand”: – Content heavy (terminology for structures must be learned before
subsequent structures, function & concepts can be presented)
• Two semester course, concurrent lab. (one grade)• Large lecture hall, no blackboard (PowerPoint and
overheads only). Typical lab facilities.• No pre-requisites or assumed knowledge
Abilities (student) Objectives (Course) Assessment (results)
Original Course Delivery model
Lectures Labs
Assignments
Quizzes Tests
Student outcomes (the scores)
Abilities (student) Objectives (Course) Assessment (results)
Concepts build on those previously presented in class.
Students must leave course with knowledge and ability in every topic presented (which will be needed for subsequent courses and professional certification)
Advantages and Disadvantages(assessing this approach)
Lectures Labs
Assignments
Quizzes Tests
Student outcomes (the scores)
Pro: consistent delivery of material (Year-to-year)
Con: can not adapt to differences in student learning (can leave gaps in knowledge)
Abilities (student) Objectives (Course) Assessment (results)
Problems with this approach
Abilities (student) Objectives (Course) Assessment (results)
Changes in content delivery (availability & access): Addition of supplemental materials on web and in library, adopted new texts.
Addressing these problems (adding adaptive assessment TO THIS approach)
• Weekly adaptive quizzes– Goal = insure that content is covered and retained by
“recycling” or re-emphasizing poorly understood topics (subsequent content adapts to knowledge gaps indicated by objective measures of student performance).
• Pre-post tests (entry assessments, midterms and final)– Goal = overall course improvement by evaluating: successful
presentations of individual topics, retention of material after completion of course (i.e.:of first semester material in second semester)
Abilities (student) Objectives (Course) Assessment (results)
Adaptive Assessment
Quiz 1
30 Neurons 3040 Neuroglia 5030 Function 20
Results 1
70%80%60%
Quiz 2
40 Neurons 4060 Neuroglia 400 (N. Function) 20
Needsreview?
Review
Abilities (student) Objectives (Course) Assessment (results)
Assessment “feedback” loop
Pro: an adaptive approach can be used to close the gaps in student knowledge
Con: takes time and effort by instructor to implement.
Note: this can be used for both self contained and pre-requisite knowledge courses
Abilities (student) Objectives (Course) Assessment (results)
Course Changes: Technology classroom (NIH-RISE)
• Interactive media & Video on demand – Dissections, models, background & supplemental
information
• View PowerPoint lectures
• Print lecture notes and outlines
• Automated practice quizzes and tests – (Web-based, secure (CGI),with feedback (in TC only)
• Equal access to course materials
Example 2: Comparative Vertebrate Biology
• Upper division biology majors – Juniors/Seniors (pre-med, pre-vet, pre-graduate school)
• “Remember, Understand & Analyze”– Content and concept intensive
• Lecture & lab (20 students)
• Assumes basic understanding of lower-division general biology, basic laboratory skills, understanding of correct terminology.
Abilities (student) Objectives (Course) Assessment (results)
Course design & delivery (lab)
• Lab exercises with questions – (non-adaptive, “you must know this…” and “how is this structure similar to …”).
Assumes good study habits and skills.
• Student presentations of lecture summaries in lab– Identify difficult concepts with need for more in-depth coverage (identify gaps in
knowledge)
– Create participatory classroom dynamic
• Lab documentation projects– Students create class materials (self documenting review materials)
– “problem based learning”
Abilities (student) Objectives (Course) Assessment (results)
Lectures Labs
Assignments
Quizzes Tests
Student outcomes (the scores)
NOTE: Student self-assessment/feedback is within lab prior to tests
Course changes: NIH-Rise-TC
• Visual review (Digital Video) of in-lab procedures, dissections, and specimens – A picture is worth a thousand words, and a video is
worth a thousand pictures!
• Equal access to information resources – (“go find everything you can about…” on the
web.)
Learner-centered approaches to assessment/improvement implemented
• Human Anatomy and Physiology– Use of adaptive weekly quizzes to “fine-tune” delivery of
individual lectures (year-to-year), guarantee a sufficient knowledge foundation on which to build additional concepts (within the semester)
– Entry and Exit assessments provide the information necessary to measure success in meeting the course goals
• Comparative Vertebrate Anatomy Lab– Use of self and peer assessment to identify and fill gaps in
knowledge, skills and abilities before they are formally tested.
Application of NIH-Rise Technology Classroom capabilities
• Increases in information resources availability– Online resources (web)
– Class notes, outlines and presentations.
• Dynamic content capabilities – Self serve video (TB server capable of 600+ hours of MP1
video)
– Interactive content (practice exams, data exploration)
• Provide all the Information Technology tools needed for “Problem based learning”
• …and, let’s not forget Technology Development support and training for faculty
The bad news
• There is no free lunch– Incorporating adaptive assessment (in A&P)
requires an additional 3 hours each week to quiz and lecture preparation and delivery (50% increase after initial development of course materials)
– Adding self and peer assessment (2 hours additional planning and setup for TC use).
Still to be addressed...• Why did I do this?
– Student outcomes poor relative to objective standards
• Why should others do this?• How can others incorporate these types of
assessment features in their courses?– Drink more coffee
– Give up sleep
• Conclusion: still some implementation obstacles