Post on 09-May-2018
Developing and Implementing a General-EducationAssessment Plan
Lindsey GuinnDirector of Assessment and Institutional Research
Linda TroostProfessor of English
December 7, 2017
Learning Objectives for this Workshop
• Learning how W&J College mapped out a general-education assessment plan;
•Gaining insight into how one might create a plan to educate faculty on general-education assessment;
•Understanding the value of general-education assessment.
W&J’s Assessment Structure
• Director of Assessment and Institutional Research (DAIR), both academic and administrative
• Gen Ed DTF, a disappearing task force on general education
• Academic Affairs Assessment Committee, a standing committee
• DAIR• five faculty members appointed by the VPAA
representing a diverse group with attention to gender, race, and area of specialty
• one faculty member serving as the liaison to the Curriculum and Program Committee
What We Learned at the AAC&U* General-Education Assessment Institute (2015)
•We do not have to assess everything every year.
•We do not have to assess every student.
• It is NOT about a single course; it is about the whole curriculum.
•Assessment and grading may (and hopefully do) overlap, but they are not identical.
*Association of American Colleges & Universities
Our Goals
Developing a gen-ed assessment plan that:• is sustainable;• is flexible;•builds on the assessment we are already doing;•accomplishes our goals manageably; and •will provide the institution-wide data we need for Middle States.
Our Institutional Learning Outcomes
They reflect what we want our students to be capable of when they graduate:
“W&J graduates demonstrate
intellectual curiosity, grounded in interdisciplinary thinking;
informed analysis and decision making;
integrity and individual agency;
the ability to communicate ideas;
and a commitment to local and national communities, paired with responsible global citizenship.”
Each Element Has Descriptive Language
W&J graduates demonstrate intellectual curiosity,
grounded in interdisciplinary thinking;
lifelong learning; connecting natural and social sciences, arts, &
humanities to make meaning from experiences in the world
“we explore beyond boundaries”
Each is Reflected in our Newly Revised Curriculum
FOUNDATIONS BREADTH DEPTH(First Year Seminar; First Year Composition; Writing; Foreign Language; Cultural Diversity; Health and Wellness Education)
(Artistic; Behavioral; Historical; Literary; Logical; Scientific)
(Academic major including discipline-specific writing, communication, technology and integrative capstone)
W&J graduates demonstrate intellectual curiosity, grounded in interdisciplinary thinking;
“grounded in interdisciplinary thinking”
“connecting natural and social sciences, arts, & humanities to make meaning from experiences in the world”
informed analysis and decision making;
“finding, evaluating, using information appropriately”, “effective reasoning”, “independence of thought”, “depth of knowledge”
integrity
and individual agency; “taking responsibility for physical and emotional health”
“initiative”, “self-awareness”, “motivation”, “self-direction”
the ability to communicate ideas;
“written communication”, “study of languages”
“quantitative reasoning”, “critical and creative thinking”, “adaptability”, “textual, visual, and artistic literacies”
“oral and written communication”, “information technologies”
and a commitment to local and national communities, paired with responsible global citizenship.
“respectful of diversity”, “understanding difference andcommonality, at home and abroad”
FOUNDATIONS BREADTH DEPTH
(FYS; Composition; W; Modern Language; Diversity; Health and Wellness)
(Artistic; Behavioral; Historical; Literary; Logical; Scientific)
(Academic major including discipline-specific writing, communication, technology and integrative capstone)
W&J graduatesdemonstrate intellectual curiosity, grounded in interdisciplinary thinking;
“grounded in interdisciplinary thinking”
“connecting natural and social sciences, arts, & humanities to make meaning from experiences in the world”
AAC&U’s Essential Learning Outcomes
Knowledge of Human Cultures and the Physical and Natural Worldthrough study in the sciences and mathematics, social sciences, humanities, histories, languages, and the arts
Intellectual and Practical Skills including: inquiry and analysis; critical and creative thinking; written and oral communication; quantitative literacy; information literacy; teamwork and problem-solving
Personal and Social Responsibility including: civic knowledge and engagement—local and global; intercultural knowledge and competence; ethical reasoning and action; foundations and skills for lifelong learning
Integrative and Applied Learning including: synthesis and advanced accomplishment across general and specialized studies
Middle-States-Specified Learning Outcomes
• Communication Skills (Written)
• Communication Skills (Oral)
• Scientific Reasoning
• Quantitative Reasoning
• Critical Analysis
• Critical Reasoning
• Technological Competency
• Information Literacy
• Study of Values, Ethics, and Diverse Perspectives
Our Learning OutcomesLEARNING OUTCOMES CAPABILITIES: What we want our students
to be able to do when they graduate
W&J graduates demonstrate intellectual curiosity, grounded in interdisciplinary thinking;
Critical Analysis (Inquiry and Analysis)Critical ThinkingCreative ThinkingIntegrative LearningApplying Modes of Thinking
informed analysis and decision making; Information LiteracyProblem SolvingReadingQuantitative Ability
integrity and individual agency; DiversityTeamworkHealth and WellnessEthical Reasoning/Practice
the ability to communicate ideas; DiversityWritingCommunication (Oral)Technology Literacy
and a commitment to local and national communities, paired with responsible global citizenship.
Civic EngagementDiversity
LEARNING OUTCOMES CAPABILITIES: What we want our students to be able to do when they graduate
W&J graduates demonstrate intellectual curiosity, grounded in interdisciplinary thinking;
Critical Analysis (Inquiry and Analysis)Critical ThinkingCreative ThinkingIntegrative LearningApplying Modes of Thinking
Ours Match Well With OthersAAC&U’s Essential Learning Outcomes
intellectual and practical skills including: inquiry and analysis; critical and creative thinking; written and oral communication; quantitative literacy; information literacy; teamwork and problem solving
Middle-States-Specified Learning Outcomes
Critical Analysis
Critical Reasoning
OUTCOMES CAPABILITIESW&J graduates demonstrate intellectual curiosity, grounded in interdisciplinary thinking;
Critical Analysis (Inquiry and Analysis)Critical ThinkingCreative ThinkingIntegrative LearningApplying Modes of Thinking
Gen Ed Assessment ScheduleCAPABILITIES 15-16 16-17 17-18 18-19
MIDDLE STATES
19-20
Technology Literacy Develop Assess
Writing Develop AssessCommunication (Oral) Develop AssessCritical Analysis & Reasoning Develop Assess
Information Literacy Develop Assess
Quantitative Reasoning Develop AssessScientific Reasoning Develop AssessDiversity (Diverse Perspectives) Develop AssessIntegrity Develop Assess
Integrative Thinking across Disciplines
Develop Assess
Integrative Thinking in the Major Develop Assess
Diverse Perspectives Develop Assess
Institutional Learning Outcomes Graduates will demonstrate the ability to produce written work with the following features:
1. A controlling idea or thesis;2. Supporting evidence;3. Accurate, sound analysis or explanation;4. A sense of audience;5. Minimal mechanical errors;6. Presentation of ideas appropriate to the discipline;
7. Understanding of genre and style conventions appropriate to the discipline;
8. Correct documentation appropriate to the discipline.
RubricW&J RUBRIC
Written
Communication
EXCEEDS EXPECTATIONS
(4)
MEETS EXPECTATIONS
(3)
BELOW EXPECTATIONS
(2)
NOT ACCEPTABLE
(1)
CONTROLLING
IDEA, THESIS
(W1)
Precisely stated, clear
controlling idea,
question, or claim
(thesis).
Writing stays on task.
Clear controlling idea,
question, or claim
(thesis).
Writing stays on task.
Vague controlling idea,
question, or claim
(thesis).
Writing strays at times
from task.
No focus inferable.
Writing often wanders.
EVIDENCE
(W2)
Ample evidence (e.g.,
data, statistics, facts,
quotations, etc.)
presented that is valid
(correct, current, from
reliable sources and
authorities).
Adequate valid evidence. Valid evidence mingled
with invalid evidence.
Minimal valid evidence;
minimal evidence of any
kind.
ANALYSIS,
EXPLANATION A:
REASONING
(W3)
Statements and
evidence clearly and
correctly presented;
relevant to thesis.
Deep, sound conclusions
follow clearly from
ample evidence.
Assumptions or
viewpoint identified and
taken into account.
Statements and
evidence adequately
presented; relevant to
thesis.
Sound conclusions
follow clearly from
sufficient evidence.
Some statements or
evidence incorrect or
vaguely presented; some
irrelevancy to thesis.
Some conclusions do not
follow from evidence
presented or are based
on false assumptions or
opinion.
Many irrelevant, wrong
statements and much
invalid, irrelevant
evidence.
Many flawed conclusions
based on false
assumptions, opinion, or
invalid evidence.
Assessment: Written Communication% of students meeting expectations (score of 3 or 4)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Controlling Idea,Thesis (W1)
Evidence (W2) Analysis,Explanation A&B:
Reasoning,Breadth, Depth
(W3)
Sense of Audience(W4)
Mechanics (W5) Ideas, Concepts inDiscipline (W6)
Conventions ofStyle, Genre inDiscipline (W7)
Documentation inDiscipline (W8)
% Meeting expectation (Score of 3 or 4) Goal
Findings
•Goal: 80% of students will meet expectations.
•Results: over 80% in five areas; 72 to 79% in three areas.
Conclusions
•We fell short in:•analysis (W3), •mechanics (W5), and•documentation in the discipline (W8).
•The previous graduation requirements worked somewhat, but we hope the current ones work better.
Institutional Learning Outcomes Graduates will demonstrate the ability to engage in effective oral communication by being able to deliver a presentation or engage in a dialogue centered on discipline-specific content illustrating the following features:
1. Clear purpose;
2. Content that supports the purpose adapted to the audience/group and discipline;
3. Structure that supports the purpose: (a) in the case of a presentation, structure is an intentional design, with a clear beginning, middle, and end, (b) in the case of dialogue, structure involves balancing participation and engaging in active listening;
4. Information and knowledge used in an accurate and ethical manner;
5. When applicable, correct documentation appropriate to the discipline.
6. Delivery (appropriate to the speaker’s abilities) that supports purpose.
RubricW&J RUBRIC
Oral
Communication
EXCEEDS EXPECTATIONS
(4)
MEETS EXPECTATIONS
(3)
BELOW EXPECTATIONS
(2)
NOT ACCEPTABLE
(1)
PURPOSE
(O1)
Thesis or objective
stated clearly and
precisely at start.
Speaker stays on task.
Thesis or objective stated
at start.
Speaker stays on task.
Thesis or objective stated
vaguely or at some point
other than start.
Speaker strays from task.
No thesis or objective
stated or inferable.
Speaker off-topic half the
time.
CONTENT
(O2)
Sophisticated, complex
use of discipline-specific
evidence (e.g., data,
statistics, facts,
quotations) to support
conclusions; relevant to
purpose.
Appropriate style, tone,
vocabulary, and
complexity chosen for
audience and situation.
Straightforward use of
discipline-specific
evidence to support
conclusions; relevant to
purpose.
Largely appropriate style,
tone, vocabulary, and
complexity chosen for
audience and situation.
Simplistic use of
discipline-specific
evidence to support
conclusions; content
irrelevant or thin.
Speaker misses the mark
due to misreading of
situation or audience;
level of discourse
sometimes too high or
low.
Minimal or no use of
discipline-specific
evidence to support
conclusions; overall poor
reasoning; content does
not serve purpose.
Seriously inappropriate
choices made in style,
tone, vocabulary, and
complexity for audience
or situation.
Assessment: Oral Communication% of students meeting expectations (score of 3 or 4)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Purpose (O1) Content (O2) Structure (O3) Information (O4) Documentation (O5) Delivery (O6)
% Meeting expectation (Score of 3 or 4) Goal
Conclusions
•We did well.
•The previous graduation requirements worked very well, and we hope the current ones work as effectively.
Institutional Learning Outcomes Graduates will demonstrate the ability to select and use information resources and technology appropriate to the work of the discipline to achieve the following goals:
1. Retrieve, evaluate, manipulate, or present information in task-appropriate ways;
2. Create, format, or present text, images, or other media in an audience-aware manner;
3. Understand legal, ethical, privacy, or security issues critical to the use of information technology.
Findings
•Background: assessment was based on departmental technology SLOs and, therefore, took a variety of forms and assessed different-T skills.
•Goal: 80% of departments will meet expectations for technology skills.
•Result: 73%.
Conclusions
•We fell short.
•Relevant departments have been asked to:• review their technology SLOs,• review assessment processes and data, and•decide how best to close the loop.
Assessments Performed in 2016-17
Written communication• Student work graded in departments using a common
rubric
Oral communication• Student work graded in departments using a common
rubric
Technology skills• Student skills assessed in departments using a
department-specific rubric
Assessments Planned for 2017-18 Scientific Reasoning
• Student work graded in non-major science and first course in a departmental major sequence carrying SCIENCE designation using a common rubric
Logical Reasoning• Student work graded in non-major logic and first course in
a departmental major sequence carrying LOGIC designation using a common rubric
Critical and Analytical Reasoning• Student work collected and graded by a small group of
faculty using a common rubric
Information Technology• HEDS Survey
Educational Workshops Held for Faculty
•General-education assessment
•Program-level assessment
•How to use rubrics
•How to develop rubrics
Faculty Buy-In
•Education through workshops
•Committee participation
•Updates at department-chair meetings
•Updates at faculty meetings
• Individual meetings with chairs and departments
•Annual-report-writing help days
•An assessment manual
The W&J Assessment Manual
• Program-Level Assessment
•General-Education Assessment
• Course Evaluations
•Appendix•How-To Guides (creating mission statements, course
SLOs, program SLOs)• Curriculum-Map Templates• Rubric to Evaluate Program-Level Assessment Plans• Policy for Sharing Assessment Data•Glossary
Documentation of Assessment
Annual Report•Completed by all departments and programs
•Section 1 – Administrative Information•Section 2 – Faculty•Section 3 – Students•Section 4 – Goals and Assessment
Feedback – Rubric for Evaluating the Annual Report’s Section on Assessment
EXEMPLARY (3) ACCEPTABLE (2) EMERGING (1) COMMENTS
Mission, Goals, and Outcomes Includes all elements Includes most elements Does not include elements Note – unchanged from last year
Assessment Plan 2016-2017
Institutional SLO for Oral
Communication
Includes a complete assessment
plan including sample, material,
person(s) making the assessment,
methods of assessment, and data.
Includes most of the assessment
plan including sample, material,
person(s) making the assessment,
methods of assessment, and some
data.
Does not include a complete
assessment plan and lacks a number
of elements such as: sample,
material, person(s) making the
assessment, methods of
assessment, and no data.
Reflection: Assessment Plan 2016-
2017 Institutional SLO for Oral
Communication
Includes complete refection on how
assessing oral communication went,
any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
Includes some refection on how
assessing oral communication went,
any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
Does not include refection on how
assessing oral communication went,
any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
Assessment Plan 2016-2017
Institutional SLO for Written
Communication
Includes a complete assessment
plan including sample, material,
person(s) making the assessment,
methods of assessment, and data.
Includes most of the assessment
plan including sample, material,
person(s) making the assessment,
methods of assessment, and some
data.
Does not include a complete
assessment plan and lacks a number
of elements such as: sample,
material, person(s) making the
assessment, methods of
assessment, and no data.
Reflection: Assessment Plan 2016-
2017 Institutional SLO for Written
Communication
Includes complete refection on how
assessing written communication
went, any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
Includes some refection on how
assessing written communication
went, any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
Does not include refection on how
assessing written communication
went, any issues or challenges your
department encountered, any
difficulties your department had
with the rubric, and what your
department would change about
your process the next time you
assess this SLO
EXEMPLARY (3) ACCEPTABLE (2) EMERGING (1) COMMENTS
Mission, Goals, and
Outcomes
Includes all elements Includes most
elements
Does not include
elements
Note – unchanged
from last year
Assessment Plan
2016-2017
Institutional SLO for
Oral Communication
Includes a complete
assessment plan
including sample,
material, person(s)
making the
assessment, methods
of assessment, and
data.
Includes most of the
assessment plan
including sample,
material, person(s)
making the
assessment, methods
of assessment, and
some data.
Does not include a
complete assessment
plan and lacks a
number of elements
such as: sample,
material, person(s)
making the
assessment, methods
of assessment, and
no data.
Reflection:
Assessment Plan
2016-2017
Institutional SLO for
Oral Communication
Includes complete
refection on how
assessing oral
communication went,
any issues or
challenges your
department
encountered, any
difficulties your
department had with
the rubric, and what
your department
would change about
your process the next
Includes some
refection on how
assessing oral
communication went,
any issues or
challenges your
department
encountered, any
difficulties your
department had with
the rubric, and what
your department
would change about
your process the next
Does not include
refection on how
assessing oral
communication went,
any issues or
challenges your
department
encountered, any
difficulties your
department had with
the rubric, and what
your department
would change about
your process the next