Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded...
-
Upload
ava-stephens -
Category
Documents
-
view
214 -
download
1
Transcript of Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded...
Conducting an Assessment Audit of General Education
Courses for Purposes of Identifying Embedded
AssignmentsDr. Blase Scarnati
Chair, Liberal Studies CommitteeAssociate Professor, Musicology, School of Music
Dr. Cynthia ConnAssociate Director, Office of Academic Assessment
Clinical Instructor, Educational Technology Program
Dr. Bruce FoxChair, Liberal Studies Committee Assessment
SubcommitteeDirector, Honors Program
Agenda Overview Definition of Skills Pilot Study Content Analysis Group Discussion: Ideas, Projects, and
Challenges
Flagstaff, Arizona State supported Doctoral intensive Significant distance learning
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Northern Arizona University
19,000 total students Approximately 6,000 graduate
students (3,800 FTE) Approximately 5,500 students
statewide
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Northern Arizona University
Approximately 11,400 undergraduates on the Flagstaff campus
Students come to us in a variety of ways
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Northern Arizona University
Approved Spring 1998 Fully implemented Fall 1999
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Liberal Studies Program
Coherent program Essential skills necessary for
citizenship Key themes
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Citizenship
35 credit total requirement Foundation requirements Distribution courses Courses embedded within the
academic major
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Liberal Studies Program Requirements
English composition course (4 credits)
Mathematics foundation course (3 credits)
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Foundation Requirements
Aesthetic and Humanistic Inquiry (3-6 credits)
Cultural Understanding (3-6 credits)
Laboratory Science (4 credits) Science and Applied Science
(3-6 credits) Social and Political Worlds (3-6
credits)
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Distribution Requirements (25 Credits)
Environmental Consciousness Technology and Its Impact Valuing the Diversity of
Human Experience
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Thematic Foci
Effective oral communication Critical reading Effective written communication Quantitative analysis and/or spatial
reasoning Creative thinking Critical thinking Scientific inquiry Ethical reasoning Effective use of technology
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Essential Skills
Courses embedded within the academic major
Diversity requirement Required freshman colloquium
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
University Requirements
Junior level writing course (3-4 credits)
Senior capstone experience (3 credits minimum)
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
In the Academic Major
U.S. Ethnic Diversity (3 credits) Global Awareness (3 credits)
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Diversity Requirement
Discontinued effective Fall 2004 Redesigned
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Required Freshman Colloquium (3 Credits)
Faculty body Six colleges Student representative Faculty Senate representative Administrative and programmatic
representatives
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Liberal Studies Committee (LSC)
Liberal Studies Committee Assessment Subcommittee
Liberal Studies Program Assessment Plan (Fall 2004)
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Assessment
Assess the achievement of NAU students
Effective writing Effective oral communication Critical reading
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Assessment Plan Goals (1)
Create a more cohesive curriculum of study
Analytical procedures Inform adjustments to the
curriculum
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Assessment Plan Goals (2)
To what extent are courses and embedded assignments that state they teach effective writing, critical reading, and/or effective oral communication aligned with the outcomes defined during the 2003-2004 academic year by the Liberal Studies Committee?
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Research Question 1
How should issues of non-alignment between courses and defined outcomes be addressed?
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Research Question 2
What expectations should be established for course content, assessment, and the broader curriculum?
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Research Question 3
Specifically, to enhance the cohesiveness of the curriculum, what changes should be made to the existing expectations (for faculty, departments, etc.) for submission and acceptance of courses into the Liberal Studies Program?
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Research Question 3
Courses and embedded assignments
Aligned with the outcomes Pilot Project
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Assessment Audit
Defining Skills Working definitions created by LSC
subcommittee Working definitions tentatively
approved by entire LSC Definitions “vetted” through
consultation with internal constituents, primarily departments
Consensus definitions approved by LSC
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Advantages:– Create buy in from other campus
units– Liberal Studies process seen as
open and participative– Better definitions!
Key strategy: visit folks personally. Do NOT rely solely on electronic communication
Agenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussio
n
Defining SkillsAgenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Students will be able to:– Demonstrate steps in the writing
process– Tailor writing to a specific audience– Focus writing on a specific purpose– Provide logically coherent pieces of
written work– Apply general writing standards
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Effective Writing
Students will be able to: Summarize accurately and comprehensively Recognize the most significant textual and
visual elements Incorporate/use specific examples and/or
quotations Recognize patterns and valid comparatives Demonstrate an
understanding/comprehension of the material
Demonstrate evidence of an ability to analyze and evaluate texts
Demonstrate the ability to explicitly identify criteria and use these criteria accurately to evaluate a piece of written work
Critical ReadingAgenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Students will be able to:– Organize and deliver content based
on audience, location, allotted time, and purpose
– Speak with appropriate oral communications techniques
– Listen and respond thoroughly and thoughtfully to questions
– Create appropriate professional supplemental materials that reinforce the presentation
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Oral Communication
(aka “Methods of Formal Analysis”)
(aka “Quantitative Literacy”)
Suggested titles: More inclusive?
Agenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussio
n
Quantitative & Spatial Analysis
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Methods of Formal Analysis Application of numerical, visual or
symbolic perspectives for the purpose of drawing inferences, understand phenomena or making predictions in situations that involve natural phenomena or real-world decision making
Agenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussio
n
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Pilot Project Testing of Protocols ITERATIVE Foundation Principles
– Based on embedded assignments– Actively involves faculty– Supplements direct evidence with
indirect date sources
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Pilot Project Process
– Identify faculty teaching courses that include desired skill
– Recruit these faculty as sources of embedded assignments
– Use faculty consultants to develop and evaluate rubrics
– Use rubrics to assess level of skill mastery
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Pilot Project Process
– Collect assessment data for a stratified sample of courses
– Compile and analyze data– Triangulate using indirect data
sources– Evaluate protocols
Agenda
Overview
Skills
Pilot Study
Content Analysis
Group Discussion
Pilot Project Informing and Involving the Wider
Campus Community- Communications Plan Strategy to
inform entire campus community of assessment process in a deliberate and on-going manner
- Create a Steering Committee of on and off campus constituencies
Content Analysis Purpose
- Identify the alignment between Liberal Studies skills as they are now defined and syllabi of current Liberal Studies courses.
- Identify embedded assignments that could be used for assessment purposes
Agenda
Overview
Skills
Pilot Study
Content Analysis
Small Discussion
Full Group Discussion
Agenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussion
How will findings be used?- Identify potential faculty for
participation as consultants for Pilot Study
- Target courses with strongest alignment for first phase of data collection
- Recommend policy changes regarding content of syllabi and expectations related to assessment
Agenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussion
Content Analysis
Ideas: What works well at your institution with respect to assessing General Education student learning outcomes?
Challenges: What challenges concerning General Education and Assessment do you face at your institution?
Group DiscussionAgenda
Overview
Skills
Pilot Study
ContentAnalysis
GroupDiscussion
Walvoord, Barbara E. (2004). Assessment Clear and Simple : A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.
Neuendorf, Kimberly A. (2002). The Content Analysis Guidebook. Thousand Oaks: Sage Publications.
ReferencesAgenda
Overview
Skills
Pilot Study
Content Analysis
Small Discussion
Full Group Discussion
Contact Information
Cynthia Conn, Ph.D.Associate Director, Office of Academic Assessment
Clinical Instructor, Educational Technology ProgramP.O. Box 4091
Flagstaff, Arizona [email protected]
(928) 523-7624
Bruce Fox, Ph.D.Chair, Liberal Studies Committee Assessment Subcommittee
Director, Honors ProgramP.O. Box 5689
Flagstaff, Arizona [email protected]
(928) 523-6636
Blase Scarnati, Ph.D.Chair, NAU Liberal Studies Committee
Associate Professor, School of MusicP.O. Box 6040
Flagstaff, Arizona [email protected]
(928) 523-6537