2004 Second Annual Summer Assessment Seminar
-
Upload
inez-logan -
Category
Documents
-
view
30 -
download
0
description
Transcript of 2004 Second Annual Summer Assessment Seminar
2004 Second Annual Summer Assessment
Seminar
Outcomes Assessment of Student Learning: Highlighting
Assessment Approaches at Texas Tech
Outcomes Assessment of Student Learning
Presenters:Matt Baker & Chad Davis – Agricultural Education and CommunicationJon Bredeson- Electrical & Computer
EngineeringDavid Driskill & Glenn Hill – ArchitecturePhil Marshall – Political Science
Outcomes Assessment of Student Learning
Purposes of Seminar: Provide an overview of outcomes
assessment of student learning
Present approaches to outcomes assessment from academic programs at Texas Tech
Outcomes Assessment of Student Learning
Outcomes Assessment defined:“. . . The systematic collection, review,
and use of information about educational programs undertaken for the purpose of improving student learning and development.”
Palomba & Banta, Assessment Essentials (1999)
Outcomes Assessment of Student Learning
Overarching Goal: Achieving excellence in undergraduate and graduate programs Assessment: Demonstrating that programs have achieved their intended purposes
Key to effective student outcomesassessment is faculty leadership at the
departmental level
Outcomes Assessment of Student Learning
What do they already know and what skills do they already possess?
Entrance exams and incoming surveysWhat have they learned (what do they know, do and value)?
Comprehensive examsCapstone courses/experiencesCourse embedded assessmentsPerformance-based mastery testsPortfolios from internships and student teaching Certification testsSatisfaction surveys and focus groups
Post-graduation surveys of graduates & employers
Outcomes Assessment in Agricultural Education
Matt BakerChad Davis
Anticipated Outcomes
We want our program to enhance certain disciplinary competenciesWe want our program to enhance graduates’ ability to think critically and creativelyWe want our program to improve graduates’ quality of life
Self-Assessment of Knowledge
Teaching StrategiesApplicationFoundationsInternational
Self-Assessment of Ability
CommunicationIdea Generation and ReasoningAttentiveness and Quantitative AbilitiesPerceptual, Spatial, & Memory Abilities
Self-Assessment of Skills
ContentProcessSocialResource ManagementComplex Problem SolvingSystems SkillsTechnical Skills
Critical Thinking
Richard PaulCritical thinkers use a set of intellectual standardsIntellectual standards guide the thinking processThink about thinking for the purpose of improving the thought process
California Critical Thinking Disposition Inventory (CCTDI)
Result of a Delphi study of critical thinking expertsSeven constructs are measured analyticity, self-confidence,
inquisitiveness, maturity, open-mindedness, systematicity, and truth-seeking
the total score is calculated by adding the construct scores
CCTDI Constructs
Analyticity - reasoning based on facts Self-confidence - secure in intellectual judgementInquisitiveness - intellectual curiosityMaturity - awareness that problems are complex
CCTDI Constructs
Open-mindedness - tolerance of diverse viewsSystematicity - organized, diligent inquiryTruth-seeking - knowing the truth is more important than winning the argument
Typology of Creativity
ExpressiveProductiveInventiveInnovativeEmergenative
Taylor, 1959Taylor, 1959
Torrance Test For Creative Thinking (TTCT)
For this three-part timed test, subjects are asked to construct a picture, complete a series of incomplete drawings, and complete drawings from sets of parallel lines.
TTCT Constructs
Fluency - the ability to produce a large number of figural imagesOriginality - unusualness or rarity of responseElaboration - ability to develop, embroider, embellish, carry out, or otherwise elaborate ideas.
TTCT Constructs
Abstractness - the ability to produce good titles and to capture the essence of information involvedResistance to Closure - the ability to keep a figure open and delay closure long enough to make the mental leap that makes original ideas possible
Quality of Life
Quality of Life Profile (QOLP) generic measure of well-being developed and validated by multidisciplinary
research team, University of Toronto relationship between individual/environment assumes QOL is a judgement
Focuses on three fundamental areas of life common to all human beings
QOL - Domains
Being
Belonging
Becoming
Individual’s Quality of Life
QOL - Being
Psychological Spiritual
Physical
Body & Health
Thoughts & Feelings Beliefs &
Values
Being Domain(Who person is as an individual)
QOL - Belonging
Social Community
Physical
Live & Spend Time
People Around You Access to
Things
Belonging Domain(How environments and others
fit with person)
QOL - Becoming
Leisure Growth
Practical
Daily Things
Things For Enjoyment
Improve & Change
Becoming Domain(What person does to achieve
hopes, goals, aspirations)
Individual’s Quality of Life
QOL Profile
Bei
ngPhysical
Psychological
Spiritual
Belonging
Physical
Social
Community
BecomingLeisure
Growth
Practical
OUTCOME ASSEMENTSJon Bredeson, Chair
Electrical and Computer Engineering
ABET
Accreditation Board for Engineering and Technology Very crucial to be accreditedCriterion 2. Program Educational ObjectivesDetailed published educational objectives Process where objectives are determined and periodically evaluated
ABET
A curriculum and processes that prepare students for the achievement of these objectivesA system of ongoing evaluation that demonstrates achievement of these objectives and uses the results to improve the effectiveness of the program
ABET
Criterion 3. Program Outcomes and AssessmentWhat students are expected to know or be able to do at graduation from the programEngineering programs must demonstrate that their graduates have a through kEach program must have an assessment process with documented results.
ABET
Evidence must be given to the further development and improvement of the programThe assessment process must demonstrate that the outcomes of the program are being measuredInstitute of Electrical and Electronic Engineers role
ECE at Texas Tech
Senior exit interviews. Initial form followed by individual interview with ChairAll courses assessed, by faculty and all students with formsInstitutional Research data usedABET & Curriculum Committees interpret data and make recommendations
ECE at Texas Tech
First EC 2000 visit in 1999 at Texas TechReport need for all programs this coming yearNext visit Fall 2005 for Electrical Engineering and Computer Engineering
Assessment of Student Outcomes
College of ArchitectureDavid A. Driskill, AIA
Glenn E. Hill, AIA
Guide to Student Performance Criteria
1998 (revised 9/2003)
The National Architectural Accrediting Board
“The program must ensure that all its graduates possess the skills and knowledge defined by the performance criteria set out below, which constitute the minimum requirements for meeting the demands of an internship leading to registration for practice.”
Three Levels of Accomplishment
“Awareness: familiarity with specific information, including facts, definitions, concepts, rules, methods , processes or settings. Students can correctly recall information without necessarily being able to paraphrase or summarize it.”
“Understanding: assimilation and comprehension of information. Students can correctly paraphrase or summarize information without necessarily being able to relate it to other material or see its fullest implications.”
“Ability: skill in relating specific information to the accomplishment of tasks. Students can correctly select the information that is appropriate to a situation and apply it to the solution of specific problems.”
Demonstration of Accomplishment
Awareness is demonstrated with evidence that the material is covered in lectures and readings.
Understanding is demonstrated with evidence that the material is tested.
Ability is demonstrated with evidence from studio and course projects and writings.
Graphic Matrix
Required Course Numbers /Student Outcomes
1 through 37
Team Room & Criteria Folders
Team Room & Criteria Folders
Team Room & Criteria Folders
Team Room & Criteria Folders
Ongoing Assessments
Internal Reviews at the end of each semester
Learning Walls throughout the Building
Use of QuestionnairesPhil MarshallPolitical Science
(Brian Cannon – Earl Survey Research Lab)
Several steps are important to assessment projects
Project planning and questionnaire design
Data collection and management
Data analysis and reporting
Project planning and questionnaire design
Determining the population of interest and assembling the sample. For Political Science, undergraduate alumni were of interest. E-mail addresses were obtained from the Arts & Sciences Office of the Dean Development Officer.
Writing questions that address topics of interest. In most cases, someone has already done something similar to what you are doing– this can be used as a starting point.
Refining the questions to ensure that the data you get will be pertinent to your situation. Keep it simple.
Data collection and managementEarl Survey Research Laboratorywww.ttu.edu/~esrl
Deciding on a method for obtaining data. For Political Science, a web-based survey was selected: quick, efficient, minimal labor required.
Collecting the data. An e-mail was sent to the sample asking each person to participate in a brief survey online. A follow-up e-mail was sent a week later to remind those who had not yet responded.
Data were stored at the ESRL.
Data analysis and reporting
Results were compiled by the ESRL and presented in summary form.
The results are descriptive and provide a baseline for future assessment efforts.
The basic set of questions used in the Political Science survey have also been used by other departments at TTU, with additional questions added or modified for a customized survey.
On the following slides, sample survey questions are presented.
Sample Survey Question
1. To what extent is your TTU undergraduate major related to your current occupation?
Very closely relatedSomewhat relatedNot closely related
Not related at all
Sample Survey Question
2. Please indicate the extent to which you agree or disagree with the following statement: “My undergraduate major prepared me well for my career”
Agree StronglyAgree SomewhatDisagree SomewhatDisagree Strongly
Sample Survey Question
3. Have you taken additional graduate courses since finishing your undergraduate degree?
YesNo
Sample Survey Question
4. Please indicate the degree(s) that you received from TTU
BAMAMPAPhD
Sample Survey Question
5. To what extent do you feel you were prepared for graduate study?
Very Well PreparedSomewhat preparedNot at all prepared
Outcomes Assessment of Student Learning: Other
Resources
SPAR homepage: http://techdata.irs.ttu.edu/stratreport/
IRIM: http://www.irim.ttu.edu/Assessment at other universities:
Colorado:http://
www.colorado.edu/pba/perfmeas/index.htm
Oregon State: http://oregonstate.edu/ap/assess/
Outcomes Assessment of Student Learning
DISCUSSION, COMMENTS, andQUESTIONS
THANK YOU!