Outcomes Assessment Using Capstone Experiences
description
Transcript of Outcomes Assessment Using Capstone Experiences
![Page 1: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/1.jpg)
Outcomes Assessment Using Capstone Experiences
Joseph A. ShaeiwitzChemical Engineering Department
West Virginia UniversityP.O. Box 6102
Morgantown, WV 26506-6102304-293-2111 ext. 2410
http://www.che.cemr.wvu.edu/ugrad/outcomes/
![Page 2: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/2.jpg)
Assessment Survey
Please complete the pre-workshop assessment survey
![Page 3: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/3.jpg)
Outline
Introduction/MotivationHow to include as many
outcomes as possible in a capstone experience
Desired attributes of capstone experiences
![Page 4: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/4.jpg)
Outline (cont’d) Development of rubrics to assess
capstone experiences Simultaneous assessment and
grading Using oral presentations for
assessment of capstone experiences
Summary
![Page 5: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/5.jpg)
Outline
Introduction/MotivationHow to include as many
outcomes as possible in a capstone experience
Desired attributes of capstone experiences
![Page 6: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/6.jpg)
Motivation Self-assessment measures such as
surveys are insufficient, assessment by the faculty is needed
All engineering programs already have capstone experiences
Students are expected to apply previously learned knowledge in capstone experiences
![Page 7: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/7.jpg)
oneclass
oneclass
education process
one course
one course
enteringcollege
graduate alumnus
Feedback Model
![Page 8: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/8.jpg)
Outline
Introduction/MotivationHow to include as many
outcomes as possible in a capstone experience
Desired attributes of capstone experiences
![Page 9: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/9.jpg)
Required Outcomes Apply math, sci,
engr Experiments Design Teams Solve engr problems Ethics
Communication Broad education Life-long learning Contemp. Issues Modern tools
![Page 10: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/10.jpg)
Exercise 1 Suggest a capstone experience
that includes as many required outcomes as is possible
Focus on what needs to be added to commonly used experiences to include additional outcomes
![Page 11: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/11.jpg)
Some Ideas Comprehensive individual/group design
project Includes all technical aspects Include material not taught in class Can include safety, life-cycle analysis,
pollution prevention, environmental impact statement
Requires use of specialized, sophisticated software
![Page 12: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/12.jpg)
Outline
Introduction/MotivationHow to include as many
outcomes as possible in a capstone experience
Desired attributes of capstone experiences
![Page 13: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/13.jpg)
Attributes of Capstone Experiences
What attributes do you expect to see in your capstone experiences?
What skills and/or knowledge do you expect to be demonstrated in capstone experiences?
![Page 14: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/14.jpg)
Exercise 2 List as many attributes as possible
for the capstone experience described in Exercise 1
Focus on what you would be evaluating when grading the final project report
![Page 15: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/15.jpg)
Some Ideas Writing – grammar, punctuation,
format, good technical explanations, logical development
Oral – use of visual aids, organization, delivery mechanics
Technical – apply math, science, engineering; design of equipment; optimization based on economics
![Page 16: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/16.jpg)
Outline Development of rubrics to assess
capstone experiences Simultaneous assessment and
grading Using oral presentations for
assessment of capstone experiences
Summary
![Page 17: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/17.jpg)
Development of RubricsWhat attributes do you expect
to see in your capstone experiences?
What skills and/or knowledge do you expect to be demonstrated in capstone experiences?
![Page 18: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/18.jpg)
Development of Rubrics (cont’d)
How would you describe excellence for each attribute?
What is not acceptable for each attribute?
Fill in intermediate descriptions
![Page 19: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/19.jpg)
Four-Point Scale for Rubrics
4 – exceeds expectations
3 – meets expectations
2 – below expectations
1 – not acceptable
![Page 20: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/20.jpg)
Use of Rubrics
Oral Reports
Written Reports
Technical Content
![Page 21: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/21.jpg)
Advantages of Method
Can involve many faculty members
Standardization of evaluation Is done while gradingCan be adapted to guideline
for grading
![Page 22: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/22.jpg)
Exercise 3
For the attributes listed in Exercise 2, develop a rubric for evaluation of these attributes
![Page 23: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/23.jpg)
Attribute 1-Not acceptable 2-Below
expectations 3-Meets expectations
4-Exceeds expectations
Score
Report Mechanics Organization inappropriate content of
most sections of report some content in inappropriate section of report
content appropriate to all section of report
unique organization enhances readability and/or understandability of report
Complete Story Told no story told, very incomplete
aspects of complete story missing
complete story told additional material enhances quality of report
Aesthetics unacceptable – e.g., tables and figures cannot be read/understood, fonts difficult to read
some portions are sloppy and difficulty to read
text, tables, figures readable and understandable
text, tables, figures so clear and understandable as to enhance report impact
Format so many format errors as to make report useless
some format errors format followed unique format aspects that enhance report impact
Spelling any spelling errors only spelling errors are different spellings for same pronunciation
no spelling errors
Grammar and Punctuation too many grammar and punctuation errors
grammar and punctuation errors
only a very few grammar or punctuation errors
no grammar or punctuation errors
Length far too long or too short
too long or too short
appropriate length
![Page 24: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/24.jpg)
Attribute 1-Not acceptable 2-Below
expectations 3-Meets expectations
4-Exceeds expectations
Score
Effective use of Visual Aids (VA)
Clarity and readability not clear or readable difficulty reading clear and readable superior clarity and readability
Use of space on VA VA unreadable because too crowded
too little or too much information of VA
appropriate amount of information on VA
VAs very well laid out
Lettering readable font unreadable font too small font readable Color, over- or under-use (if used)
colors too hard to distinguish, colors do not project well
poor choice and use of colors
primary/easily distinguishable colors
use of color enhances clarity of presentation
Wording concise slides full of text slides too wordy slides appropriate Appropriate amount of information per VA
so much information per VA or so much missing information to make VA useless
too much information per VA, missing information such as size of total pie
appropriate level of information per slide
Presentation Organization Logical order of topics totally disjointed, no
organization some items presented out of order
organization as per guidelines
superior organization enhances communication
Appropriate use of time: Not too long /short
far too long or far too short
somewhat too long or too short
appropriate length
Complete "story" told story missing, no story told
story incomplete complete story told
Introduction: Problem stated problem not stated problem poorly stated
problem clearly stated problem clearly stated, good perspective on problem shown
![Page 25: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/25.jpg)
Attribute 1-Not acceptable 2-Below expectations
3-Meets expectations
4-Exceeds expectations
Score
Design of equipment, Analysis of performance of existing equipment, Understand interrelationship between equipment in process
Design of individual equipment
major errors in individual equipment design
some errors in equipment design
equipment designed correctly
unique aspects of equipment design enhance result
Understand interrelationship between equipment on flowsheet
no understanding of equipment interrelationship
minimum understanding of equipment interrelationship
clear understanding of equipment interrelationship
exploitation of equipment interrelationship to enhance result
Constraints/limitations of individual equipment and flowsheet understood
constraints/ limitations not understood
not all constraints/ limitations understood
constraints/ limitations clearly understood
exploitation of constraints/ limitations to enhance result
Response to questions indicates understanding of ChE principles
response to questions demonstrates lack of understanding
response to questions shows gaps in understanding
response to questions shows clear understanding
response to questions shows superior understanding
Significance of conclusions understood
lack of understanding gaps in understanding clear understanding superior understanding
Apply chemistry, math, physics, engineering science
Apply engineering science inability to apply principles weak application of principles
good application of principles
superior application of principles
Response to questions indicates ability to apply these principles
response to questions demonstrates lack of ability to apply these principles
response to questions shows gaps in ability to apply these principles
response to questions shows clear ability to apply these principles
response to questions shows superior ability to apply these principles
![Page 26: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/26.jpg)
Scoring Sheet for all Design Projects
Technical Content Design of equipment, analysis of performance of existing equipment, understand interrelationship between equipment in process
Apply chemistry, math, physics, engineering science Resolve complex problem into components Apply economic, physical constraints and optimization methods to obtain solution
Computer usage Application of safety principles Other economic, global, societal, and legal considerations
Oral Presentation
Effective use of visual aids Presentation organization Effective use of software to prepare visual aids Group synergism/dynamics Presentation mechanics Response to questions
Written Report
Report mechanics Content Effective use of software to prepare written report
![Page 27: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/27.jpg)
Outline Development of rubrics to assess
capstone experiences Simultaneous assessment and
grading Using oral presentations for
assessment of capstone experiences
Summary
![Page 28: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/28.jpg)
Primary Trait Analysis Rubric-like grading guide Converts “unstated criteria” (e.g., “it
feels like a B,” to “highly explicit criteria.”
Converts from norm-referenced to criterion-referenced grading
B. E. Walvoord and V. J. Anderson, Effective Grading. A Tool for Learning and Assessment, Jossey-Bass, 1998.
![Page 29: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/29.jpg)
Our Experiences Five-point scale is difficult to develop We use four-point scale shown earlier Approximate grades
• A = several 4, all others 3• B = all 3• C = average about 2.5 or so• D = average around 2• F = average well below 2
![Page 30: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/30.jpg)
Large-Group Design Project Students work under direction of
student chief engineer Faculty members play roles – one is
client, other is student company vice president
Evaluations like in company Rubric developed for grading
![Page 31: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/31.jpg)
Participation (35%)
F D C B A
No attendance Did not
complete assignment and displayed poor attitude
No participation in group meetings
Missed 5 or more group meetings
Turned in assignments sporadically
Displayed a poor attitude during group activities
Missed 3 or more group meetings
Turned in late assignments
Was not task oriented during group meetings
Attended all group meetings
Completed all assignments
Worked well with the group during meetings
Attended all group meetings
Completes all assignments with an excellent attitude
Performed extra tasks that produced new insight within the group
Technical Quality (40%)
F D C B A
No understanding & synthesis of research
The ideas were not expressed clearly
Major errors throughout design calculations
Poor understanding & synthesis of research
The ideas were not expressed clearly
Many minor errors in calculations
Average understanding & synthesis of research
The ideas were expressed clearly
Few errors in design calculations
Good understanding & synthesis of research
The ideas were expressed clearly
No errors in design calculations
Excellent understanding & synthesis of research
Ideas exceeded expectations and were expressed clearly
No errors in design calculations
Communication (25%)
F D C B A
![Page 32: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/32.jpg)
Outline Development of rubrics to assess
capstone experiences Simultaneous assessment and
grading Using oral presentations for
assessment of capstone experiences
Summary
![Page 33: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/33.jpg)
Feedback from Q & A Session
Have presentation followed by question and answer session
Students get immediate feedback Ask questions and follow-up
questions Look for patterns – common errors
or common good ideas
![Page 34: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/34.jpg)
Types of Questions Asked Can they explain what was done
and why? Quiz students on related or
peripheral concepts Error is observed – convert
session into tutorial How has student applied what
was learned?
![Page 35: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/35.jpg)
Information Seeking
Level of understanding and ability to communicate it
Was “right” answer obtained for wrong reason or accidentally?
Is “big picture” understood? How was solution obtained?
![Page 36: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/36.jpg)
Example – weak student Student chooses reactor
temperature of 325°C Question: Why was this reactor
temperature chosen? Good: Because it was economic
optimum Bad: I just chose it
![Page 37: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/37.jpg)
Example – better student Student chooses reactor temperature
of 325°C and shows graph illustrating that it is economic optimum
Question: Why is this optimum? Why are economics less favorable above and below this temperature?
Good: Provides explanation Bad: I just tried a range of
temperatures and this is what I found.
![Page 38: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/38.jpg)
Example – very good student Student chooses reactor temperature
of 325°C, shows graph illustrating that it is economic optimum, and explains why
Question: Did you investigate other variables such as pressure, conversion?
Good: Yes, and explains result Bad: No, I just investigated
temperature.
![Page 39: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/39.jpg)
Example – excellent student
Student chooses reactor temperature of 325°C, shows graph illustrating that it is economic optimum, explains why, and does same for other decision variables
Question: It becomes more difficult to find good questions
![Page 40: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/40.jpg)
Exercise 4
Ask me questions about this presentation aimed at revealing my understanding of assessment, ABET EC 2000, rubric development, etc.
![Page 41: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/41.jpg)
Outline Development of rubrics to assess
capstone experiences Simultaneous assessment and
grading Using oral presentations for
assessment of capstone experiences
Summary
![Page 42: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/42.jpg)
Summary Assessment using capstone
experiences can complement surveys – assessment by the faculty
It is possible to include many outcomes in capstone experiences
Identify desired attributes of capstone experiences
![Page 43: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/43.jpg)
Summary (cont’d) Can develop rubrics to assess
capstone experiences Can use primary trait analysis for
simultaneous assessment and grading
Can use oral presentations as part of assessment of capstone experiences
![Page 44: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/44.jpg)
Web Site
http://www.che.cemr.wvu.edu/ugrad/outcomes/
![Page 45: Outcomes Assessment Using Capstone Experiences](https://reader030.fdocuments.us/reader030/viewer/2022013101/56814f51550346895dbcf560/html5/thumbnails/45.jpg)
Assessment Survey
Please complete the post-workshop assessment survey