Course Embedded Assessment of Graduate Learning: An Innovative Approach Based on The Degree
Qualifications Profile
Presented by William M. White, Ed. D. and Indre Cuckler, M. A.
School of Graduate Studies
Who We Are
Who We Are
Who We Are
Who We Are
Outline
Levels of Assessment Definition of Assessment Theoretical Framework Program Goals and Outcomes Competency-Based Assessment Course Embedded Assessment Model Assessment Instruments Curriculum Mapping Resources Questions/Discussion
Levels of Assessment
Levels of Assessment:
Level 1. Assessing individual student learning within courses
Level 2. Assessing individual student learning across courses
Level 3. Assessing courses
Level 4. Assessing programs
Level 5. Assessing the institution(Miller & Leskes, 2005)
Definition of AssessmentDefinition of Assessment
Assessment is the systematic collection , review, and use of information about educational programs undertaken for the purpose of improving student learning and development.
(Palomba & Banta,1999)
Assessment Cycle
Establish
criteria for
success
Report
assessment results
Theoretical Framework
Theoretical Framework for MSU’s School of Graduate Studies Assessment
based on:
• Course Embedded Assessment model
• The Lumina Foundation’s Degree Qualifications Profile (DQP)
The Degree Qualifications Profile model allows our graduate programs to frame clear expectations of what students should be expected to know and be able to do at the associate, bachelor’s, and master’s level regardless of their academic discipline.
The Degree Qualifications Profile (2011)
Five Areas of Learning:1.Specialized Knowledge2.Broad, Integrative Knowledge3.Intellectual Skills Analytic Inquiry Use of Information Resources Engaging Diverse Perspectives Quantitative Fluency Communication Fluency
4. Applied Learning5. Civic Learning
Graduate Level Learning
• How do I know graduate level learning when I see it? • What are its characteristics?
The Degree Profile provides guidance to the above questions.Example:
Analytic inquiry
At the associate level, the student
Identifies, categorizes and distinguishes among elements of ideas, concepts, theories and/or practical approaches to standard problems.
At the bachelor’s level, the student
Differentiates and evaluates theories and approaches to complex standard and non-standard problems within his or her major field and at least one other academic field.
At the master’s level, the student
Disaggregates, adapts, reformulates and employs principal ideas, techniques or methods at the forefront of his or her field of study in the context of an essay or project.
Programs at MSU’s School of Graduate Studies
Mission
The School of Graduate Studies believes in the transformative power of lifelong inquiry to enhance lives. We seek to develop and maintain innovative graduate programs that fosteropportunities for personal and professional success in an increasingly multidisciplinary world.
Master of Health Science (GHSC), Master of Arts or Science in Interdisciplinary Studies (GIDS), Master of Criminal Justice Administration (GCJA), and Master of Arts in Psychology (GPSY)
Program GoalsProgram Goals Student Learning Outcomes(program level)
Student Learning Outcomes(program level)
G1: Provide an affordable and high quality graduate health science program.
G2: Foster student achievement of core competencies appropriate for health science professionals.
G3: Prepare students to serve the community and become leaders of change.
G4: Prepare graduates for careers in health administration and health services related areas.
1. Communication Skills: Students will be able to demonstrate the ability to create logical oral and written arguments, explanations and reflections based on their individual or collaborative efforts to both general and specialized audiences.
2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of the health science field in the context of an essay, a thesis or a project.
3. Knowledge, Sensitivity, and Awareness of Individual and Cultural Diversity: Students will demonstrate knowledge, sensitivity and awareness of individual and cultural diversity demonstrated through a project, paper or performance.
4. Values and Ethics: Students will assess and justify their engagement with respect to civic, social and global responsibilities.
5. Application of Knowledge and Skills in Health Science Professional Settings: Students will be able to create a project, paper, exhibit, performance or other appropriate artifacts reflecting the integration of knowledge acquired in a practicum, work community and /or research activities with knowledge and /or skills gained from at least two academic disciplines in different segments of the curriculum.
6. Qualitative and Mixed Methods Fluency: Students will demonstrate knowledge and ability to interpret, evaluate design and implement qualitative and mix-methods research appropriate to health science studies in a formal research document or by performance a project.
Note: student learning outcomes (1,2,&5) are adopted from the Lumina Foundation’s Degree Qualifications Profile
Master of Health Science (GHSC)
Program GoalsProgram Goals Student Learning Outcomes(program level)
Student Learning Outcomes(program level)
G1: Provide an affordable and high quality interdisciplinary studies program.
G2: Foster student achievement of core competencies appropriate for graduate level learners and professionals.
G3: Prepare students to serve the community and become leaders of change.
G4: Prepare graduates for careers in interdisciplinary fields according to students’ academic concentrations.
1. Communication Skills: Students will be able to demonstrate the ability to create logical oral and written arguments, explanations and reflections based on their individual or collaborative efforts to both general and specialized audiences.
2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of their field of study in the context of an essay, a thesis or a project.
3. Knowledge, Sensitivity, and Awareness of Individual and Cultural Diversity: Students will demonstrate knowledge, sensitivity and awareness of individual and cultural diversity demonstrated through a project, paper or performance.
4. Broad, Integrative Knowledge: Students will articulate how their field (academic concentration) has developed in relation to other major domains of inquiry and/or practice.
5. Specialized Knowledge in an Academic Concentration Area: Students will be able to elucidate major theories, research methods, and approaches to inquiry and/or schools of practice in their field (academic concentration).
6. Research and Evaluation in Academic Concentration Area: Graduates will be able to utilize the scientific approach to knowledge generation and apply the appropriate research methods to practice.
Note: student learning outcomes (1,2,&5) are adopted from the Lumina Foundation’s Degree Qualifications Profile
Master of Arts or Science in Interdisciplinary Studies (GIDS)
Program GoalsProgram Goals Student Learning Outcomes(program level)
Student Learning Outcomes(program level)
G1: Provide an affordable and high quality graduate criminal justice administration program.
G2: Foster student achievement of core competencies appropriate for criminal justice administration professionals.
G3: Prepare students to serve the community and become leaders of change.
G4: Prepare graduates for careers in criminal justice administration field and related areas.
1. Communication Skills: Students will be able to demonstrate the ability to create logical oral and written arguments, explanations and reflections based on their individual or collaborative efforts to both general and specialized audiences.
2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of their field of study in the context of an essay, a thesis or a project.
3. Knowledge, Sensitivity, and Awareness of Individual and Cultural Diversity: Students will demonstrate knowledge, sensitivity and awareness of individual and cultural diversity demonstrated through a project, paper or performance.
4. Broad, Integrative Knowledge: Students will articulate how criminal justice administration field has developed in relation to other major domains of inquiry and/or practice.
5. Engaging Diverse Perspectives: Students will address core issues in criminal justice administration from the perspective of either a different point in time, culture, language, political order, or technological context.
6. Application of Knowledge and Skills in Criminal Justice Administration Professional Settings: Students will be able to create a project, paper, exhibit, performance or other appropriate means reflecting the integration of knowledge acquired in a practicum, work community and /or research activities with knowledge and /or skills gained from at least two academic disciplines in different segments of the curriculum.
Note: student learning outcomes (1,2, 4,&6)are adopted from the Lumina Foundation’s Degree Qualifications Profile
Master of Criminal Justice Administration (GCJA)
Program GoalsProgram Goals Student Learning Outcomes(program level)
Student Learning Outcomes(program level)
G1: Provide an affordable and high quality graduate psychology program.
G2: Foster student achievement of core competencies appropriate for mental health and human services professionals.
G3: Prepare students to serve the community and become leaders of change.
G4: Prepare graduates for careers in mental health, human services, and related areas.
G5: Prepare students for doctoral and post graduate studies in psychology.
1. Communication Skills: Students will be able to demonstrate the ability to create logical oral and written arguments, explanations and reflections based on their individual or collaborative efforts to both general and specialized audiences.
2. Critical Thinking: Students will be able to deconstruct, adapt, reformulate and employ core disciplinary concepts, techniques or methods at the forefront of their field of study in the context of an essay, a thesis or a project.
3. Knowledge, Sensitivity, and Awareness of Individual and Cultural Diversity: Students will demonstrate knowledge, sensitivity and awareness of individual and cultural diversity demonstrated through a project, paper or performance.
4. Information Literacy: Students will be able to provide evidence (through papers, projects, notebooks, computer files or catalogues) contributing to, expanding , assessing and/or refining either a broadly recognized information resource or an information base within the field of psychology.
5. Research and Evaluation in Psychology: Graduates will be able to utilize the scientific approach to knowledge generation in psychology and apply the appropriate research methods to practice.
6. Quantitative Fluency: Students will apply mathematical, formal logic and/or statistical tools to problems appropriate to the field of psychology in a project, paper or performance.
Note: student learning outcomes (1,2,4, & 6) are adopted from the Lumina Foundation’s Degree Qualifications Profile
Master of Arts in Psychology (GPSY)
Competency-Based Assessment
Competency - based assessment measures an individual’s performance against a predetermined criterion.
There are three levels of competency achievement throughout a program of study:
competency INTRODUCED (I) competency PRACTICED (P) competency REFINED (R)
Note: the competency achievement expectations are established by faculty in each course.
Competency-Based Assessment Chart
Yes
NoYes
No
Review courses and
revise curriculum
Evaluate student work at the
beginning of a program in the
pre-selected courses
Review courses
and revise curriculum
Write summary
report
Write summary
reportWrite summary
report
Write summary
report
Write summary
report
Write summary
report
Arecompetencies
PRACTICED (P)in a course?
Review courses
and revise curriculum No
Are competencies INTRODUCED (I)
in a course?
Yes
Evaluate student work in the middle of a
program in the pre-selected
courses
Are competencies
REFINED (R) ina course?
Evaluate student work
toward the end of a program
in the pre-selected courses
Arecompetencies
achieved?(Does the program do
what is intended?)
Curricular Alignment
Figure 1. Relationship among standards/objectives, instructional activities and materials, and assessments/tests. Adopted from Anderson (2002) Curricular Alignment: A Re-Examination.
A signature assignment to measure achievement of student learning is required in courses where assessment data are being collected for program level assessment.
Course Embedded Assessment
Standards/Objectives(MSU: Core Competencies)
Assessments/Tests(MSU: Rubrics)
Instructional Activities/ Materials
(MSU: Assignments)
Learning Outcome Assessment (2011-2012)
Three student learning outcomes (SLOs) are assessed in 2011-2012 cycle in all four graduate programs:
1. Communication Skills2. Critical Thinking3. Cultural & Social Awareness(Note: other student learning outcomes will be assessed in 2012-2013)
How are these outcomes assessed? By utilizing direct measures:• Written Assignments, and• Theses or Projects
andindirect measures: • Student Reflection Journals • Course Evaluations
Assessment Instruments
Rubrics are utilized to assess learning achievement.
Written Communication Skills Rubric
Critical Thinking Rubric
Cultural and Social Awareness Rubric
*Note: rubrics must meet graduate level learning expectations.
Student Learning Outcomes
Program level student learning outcomes must be reflected in individual courses.
Review your program’s outcomes and align them with your course outcomes.
All syllabi must include the following statement—
“Upon successful completion of this course, students will have the following competencies…” [list competencies]
Student Learning Outcomes (example)
IMPORTANT: Student learning outcomes have to be measurable!
course level
Curriculum Mapping
Curriculum maps
Shows when, what, and on what level learning is delivered throughout the program.
Why curriculum mapping is important?
Provides clarity and transparency of what is going on in the program and in specific courses.
Curriculum mapping is essential to program and course level assessment.
Program level student learning outcomes should be closely aligned with course level student learning outcomes and this alignment should be depicted in a curriculum map.
Core Courses GPSY 523 Professional Ethics
GPSY 545 History & Systems
GPSY 563 Research Methods
GPSY 505 Statistics
GPSY 517 Adv.
Develop. Psychology
GPSY 527 Social
Psychology
GPSY 520 Physiological Psychology
GPSY 525 Cognitive
Psychology
GPSY 610 Adv.
Principles of Learning
GPSY 620 Psychopathol
ogy
GPSY 630 Psychologic
al Testing
GPSY 670 Portfolio
GPSY 680 Thesis or Project
Program Outcomes
1.Communication Skills I
AP P P
AP P P P R R
A2.Critical Thinking
IA
P
P P PA
P P P P R RA
3.Information Literacy I
AP P P P P
AP P P P R R
A
4..Quantitative Fluency I
APA
R RA
5.Knowledge sensitivity,
awareness of individual and
cultural diversity
IA
P PA
P R RA
6.Research and evaluation in psychology
IA
P P PA
P P P RA
Master of Arts in Psychology Program (GPSY) Curriculum Map
Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.
Core Courses GHSC 501
Focus & Design
GHSC 563 Research Methods
GHSC 616 Health Care
Systems
GHCS 621 Health Care
Financial Mngm
GHSC 622 The Health
Care Executive
GHSC 503 Behavior in
Organizations
GHSC 504 Transforming Organizational
Cultures
GHSC 580 Concentrat.
area and field
experience
GHSC 581 Concentratio
n area and field
experience
GHSC 582 Concentrat.
area and field
experience
GHSC 583 Concentratio area and
field experience
GCJA 670 Portfolio
GCJA680 Thesis or Project
Program Outcomes
1.Communication Skills I
AP P P P P
AP P P P R R
A
2.Critical Thinking
IA
P P P P P PA
R RA
3.Application of HSC I
A P A
P P R RA
4.Values&
EthicsIA
PA
P R R RA
5.Knowledge sensitivity, and awareness of individual and
cultural diversity
IA
PA
P P P P R RA
6.Qualitative and Mixed Methods
FluencyIA
PA
R RA
Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.
Master of Health Science Program (GHSC) Curriculum Map (Concentration: Health Care Executive)
Core Courses
GHSC 501 Focus & Design
GHSC 563 Research Methods
GHSC 511 Multidsp.
Aspects of Aging
GHCS 515 Psyc. Aspects
of Aging
GHSC 516 Soc.
Aspects of Aging
GHSC 572 Working Across
Disabilities
GHSC 575 Public Policy
GHSC 580 Concentrat.
area and field
experience
GHSC 581 Concentrat.
area and field
experience
GHSC 582 Concentrat
area and field
experience
GHSC 583 Concentrat
area and field
experience
GCJA 670 Portfolio
GCJA680 Thesis or Project
Program Outcomes
1.Communication Skills I
AP P P P
AP R R
A
2.Critical Thinking I
AP P P P P
AP R R
A
3.Application of HSC I
AI P
A P P P R R
A
4.Values&
EthicsIA
PA
P P P R RA
5.Knowledge sensitivity,
and awareness of
individual and cultural
diversity
IA
PA
P P P R RA
6.Qualitative and Mixed Methods Research Fluency
IA
PA
R RA
Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.
Master of Health Science Program (GHSC) Curriculum Map (Concentration: Aging Studies)
Core Courses GIDS 501 Focus & Design
GIDS 550 Professional
Writing
GIDS 557 Interdisciplinary
Context and Theory
GIDS 563 Research Methods
GIDS 567 Statistical Methods
GIDS 580Concentration
area
GIDS 581Concentration
area
GIDS 582Concentration
area
GIDS 583Concentration
area
GIDS 670 Portfolio
GIDS 680 Thesis or Project
Program Outcomes
1.Communication Skills I
AP P
AP P P R R
A
2.Critical Thinking
IA
P P P PA
P P P R R A
Specialized 3.knowledge in an
academic concentration
IA
PA
P P R R RA
4.Broad Integrative Knowledge
(interdisciplinary)
IA
PA
R RA
5. Knowledge sensitivity, and awareness of individual and
cultural diversity
IA
PA
P P P R RA
6.Research and evaluation in an
academic concentration
IA
PA
P P P R RA
Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.
MA/MS Interdisciplinary Studies Program (GIDS) Curriculum Map
Core Courses GCJA 501 Focus & Design
GCJA 531 History of Criminal Justice
GCJA 563 Research Methods
GCJA 580Concentration
area
GCJA 581Concentration
area
GCJA 582Concentration
area
GCJA 583Concentration
area
GCJA 584 Concentration area and field
research
GCJA 585 Concentration area and field
research
GCJA 670 Portfolio
GCJA680 Thesis or Project
Program Outcomes
1.Communication Skills I
AP P P
AP P P R R
A
2.Critical Thinking
IA
P PA
P P P R RA
3. EngagingDiverse
PerspectivesIA
PA
P P P R RA
4.Broad Integrative Knowledge
IA
PA
P P P R RA
5.Application of CJA I
PPA
P P R RA
6. Knowledge sensitivity, and awareness of individual and
cultural diversity
IA
PA
P P P R RA
Letters: I – students are introduced to the outcome; P- students further develop the outcome; R- students can demonstrate a refinement of the outcome; A- data are collected and analyzed for program level assessment. Colors: orange- indicates outcomes assessed in 11-12 cycle; green- indicates courses where data are collected for program level assessment.
Master of Criminal Justice Administration Program (GCJA) Curriculum Map
Summary
“ By acting on the calls for reform in assessment, graduate education can truly lead the way in fulfilling its mission of excellence in teaching, research, and service to its students and to the broader community.” (Lydell, 2008)
References
Association of American Colleges and Universities (AAC&U). Value Rubrics. Retrieved fromhttp://www.aacu.org/value/rubrics/index.cfm
Anderson, L.W. (2002). Curricular alignment: A re-examination. Theory into Practice, 41, (4). Retrieved from Academic Search Premier.
Harden, R. M. (2001). AMEE Guide No 21: Curriculum mapping: a tool for transparent and authentic teaching and learning. Med Tech, 23, (2).
Lydell, L. (2008). Assessing outcomes in graduate education. On the Horizon, 16, (2). Retrieved from Proquest data base.
Lumina Foundation. (2011). The Degree Qualifications Profile. Retrieved from http://www.luminafoundation.org/publications/The_Degree_Qualifications_Profile.pdf
Miller, R., & Leskes, A. (2005). Levels of assessment: From the student to the institution. Washington, DC: AAC&U. Retrieved from http://www.aacu.org/pdf/LevelsOfAssessment.pdf
Questions/DiscussionQuestions/Discussion
For more information please contact:
Dr. William M. White, DeanOffice: (304) 929-1438; Email: [email protected] of Graduate Studies or
Indre Cuckler, M. A.Psychology Program SpecialistOffice: (304)-929-1401; Email: [email protected] of Graduate Studies
Thank you !
Top Related