Life Sciences has been experiencing tremendous growth From 600 majors in 2000 to 1860 majors in 2012...

1
Life Sciences has been experiencing tremendous Life Sciences has been experiencing tremendous growth growth From 600 majors in 2000 to 1860 majors in 2012 From 600 majors in 2000 to 1860 majors in 2012 The School currently offers 8 undergraduate The School currently offers 8 undergraduate concentrations, however all students enroll in concentrations, however all students enroll in a common set of core courses a common set of core courses Determining Priorities Determining Priorities Where to begin? Where to begin? Analysis of retention and student achievement Analysis of retention and student achievement throughout the core courses showed that: throughout the core courses showed that: For introductory courses, about 85% of For introductory courses, about 85% of initially enrolled students completed the initially enrolled students completed the course with a grade and at least 70% course with a grade and at least 70% earned a C or better earned a C or better For upper division courses, over 90% of For upper division courses, over 90% of initially enrolled students completed the initially enrolled students completed the course with a grade and at least 75% course with a grade and at least 75% earned a C or better earned a C or better Unifying the Instructional Experience Unifying the Instructional Experience Initial program assessment indicated students Initial program assessment indicated students were encountering markedly different were encountering markedly different instructional experiences in the instructional experiences in the same same course course Program Learning Outcomes were revisited Program Learning Outcomes were revisited Instructional guides were created for Instructional guides were created for laboratory instructors and students laboratory instructors and students Lecture instructors began holding group Lecture instructors began holding group meetings to discuss the introductory biology meetings to discuss the introductory biology courses courses Where are we now? Where are we now? Students experience equivalent instruction, Students experience equivalent instruction, learning exercises, and assessment across the learning exercises, and assessment across the 32 introductory lab sections 32 introductory lab sections Professors teaching the introductory lectures Professors teaching the introductory lectures are implementing a different textbook this fall are implementing a different textbook this fall and designing common instructional and and designing common instructional and assessment plans assessment plans The new text places more emphasis on The new text places more emphasis on fundamental concepts and provides interactive fundamental concepts and provides interactive online learning resources for students online learning resources for students Student responses to content questions Student responses to content questions administered as part of the final exam have administered as part of the final exam have indicated variable mastery of program outcomes indicated variable mastery of program outcomes Student responses to essay questions Student responses to essay questions administered as part of the final exam have administered as part of the final exam have indicated variable writing proficiency indicated variable writing proficiency UNLV is a large, urban research university with 28,000 UNLV is a large, urban research university with 28,000 students, and over 700 teaching faculty students, and over 700 teaching faculty NWCCU issued UNLV a recommendation on academic NWCCU issued UNLV a recommendation on academic assessment assessment Not doing enough assessment Not doing enough assessment Not doing it well enough Not doing it well enough UNLV responded by charging the deans with assessment UNLV responded by charging the deans with assessment Ensure unit leaders submitted meaningful assessment Ensure unit leaders submitted meaningful assessment plans and reports plans and reports Review assessment data on a regular basis Review assessment data on a regular basis Faculty Assessment Workshops Faculty Assessment Workshops Focus on improving student learning rather than Focus on improving student learning rather than accreditation accreditation Designed to target all teaching faculty Designed to target all teaching faculty Mandatory (with cooperation from deans) Mandatory (with cooperation from deans) Focused on assessment at two levels with emphasis on links Focused on assessment at two levels with emphasis on links between them between them Course Course Program Program measurement, and closing the loop measurement, and closing the loop Emphasis on interdisciplinary information exchanges Emphasis on interdisciplinary information exchanges School of Life Sciences- Jenifer Utz, Ph.D. School of Architecture- Kevin Kemner, M. Arch. The Workshops – Dan Bubb, Ph.D. & Lindsay Couzens, Ph.D. Department of Dance- Louis Kavouras, Ph.D. Building a Faculty-Driven Assessment Culture: Making it Meaningful at Large Institutions Daniel K. Bubb, Ph.D. & Lindsay S. Couzens, Ph.D. University of Nevada, Las Vegas UNLV UNLV s Assessment Initiatives made us realize that assessment s Assessment Initiatives made us realize that assessment was going to be a fact of life was going to be a fact of life It gave faculty tools for how to think about assessment and It gave faculty tools for how to think about assessment and learning outcomes. learning outcomes. It should be a useful tool. It should be a useful tool. Rather than fight it, we should embrace it and find ways to Rather than fight it, we should embrace it and find ways to make the endeavor and information vital and robust. make the endeavor and information vital and robust. In discussions we realized several things In discussions we realized several things We were assessing students all the time. We were assessing students all the time. We realized that if it MOVED, we assessed it! We realized that if it MOVED, we assessed it! The big problem was how to make the information flow and put it The big problem was how to make the information flow and put it somewhere where we all could see it and use it. somewhere where we all could see it and use it. Isadora (Integrated Student Assessment Database Online Record Isadora (Integrated Student Assessment Database Online Record Advising) Advising) Multi-relational Database system Multi-relational Database system Records are linked within the department and to the Records are linked within the department and to the university’s records university’s records SOA assessment history modeled on accreditation visits SOA assessment history modeled on accreditation visits Typically six year cycle with intermediate reporting Typically six year cycle with intermediate reporting Accreditation criteria include well defined program learning Accreditation criteria include well defined program learning objectives objectives This led to a checklist mentality of assessment This led to a checklist mentality of assessment In this environment university assessment has been historically In this environment university assessment has been historically seen as a secondary concern. seen as a secondary concern. Past five years there has been a shift in the accreditation Past five years there has been a shift in the accreditation process. process. Mandate from accrediting agencies to move away from the Mandate from accrediting agencies to move away from the checklist checklist Develop issues based structure to professional degree programs. Develop issues based structure to professional degree programs. Holistic development of the student Holistic development of the student Timing of shift in accreditation process was opportunistic for the Timing of shift in accreditation process was opportunistic for the school of architecture as it coincides with UNLV’s General school of architecture as it coincides with UNLV’s General Education reform and new emphasis on assessment process. Education reform and new emphasis on assessment process. Motivated faculty of school of architecture to commit to Motivated faculty of school of architecture to commit to revisiting school-wide learning objectives. revisiting school-wide learning objectives. Reverse engineering of SOA learning objectives from top down: Reverse engineering of SOA learning objectives from top down: School wide objectives School wide objectives Program/milestone objectives Program/milestone objectives University accomplishment /assessment points University accomplishment /assessment points Learning community objectives Learning community objectives Courses that can work synthetically to bridge knowledge Courses that can work synthetically to bridge knowledge Course objectives Course objectives Generated through year long internal SOA faculty workshops: Generated through year long internal SOA faculty workshops: Develop common vocabulary for objectives Develop common vocabulary for objectives Familiarize faculty with changes in learning objective Familiarize faculty with changes in learning objective expectations. expectations. Resistance from faculty in technically oriented courses Resistance from faculty in technically oriented courses How will we assess? How will we assess? Workshop to determine assessment process and tools Workshop to determine assessment process and tools Assessment to engage entire faculty (N=16) Assessment to engage entire faculty (N=16) What will we assess? What will we assess? Milestone accomplishments Milestone accomplishments Assess the learning community not individual courses. Assess the learning community not individual courses. Emphasis on the synthesis and application of knowledge Emphasis on the synthesis and application of knowledge Student generated end of year portfolios. Student generated end of year portfolios. Guidelines developed to encourage students to demonstrate Guidelines developed to encourage students to demonstrate breadth of their achievement. breadth of their achievement. Broad spectrum demonstrating the integration of skills from Broad spectrum demonstrating the integration of skills from the learning community. the learning community. Assessment process: Assessment process: Generate electronic assessment tool. Generate electronic assessment tool. PDF based collection of student portfolios PDF based collection of student portfolios Improve collection process Improve collection process Rubric Rubric Observation / questionnaire Observation / questionnaire Digital collection of responses Digital collection of responses Closing the loop: Closing the loop: Individual faculty response to the assessment tool during the Individual faculty response to the assessment tool during the We owe many thanks to Jenifer Utz, Louis We owe many thanks to Jenifer Utz, Louis Kavouras, and Kevin Kemner for sharing their Kavouras, and Kevin Kemner for sharing their assessment journeys with us. assessment journeys with us.

Transcript of Life Sciences has been experiencing tremendous growth From 600 majors in 2000 to 1860 majors in 2012...

Page 1: Life Sciences has been experiencing tremendous growth From 600 majors in 2000 to 1860 majors in 2012 The School currently offers 8 undergraduate concentrations,

Life Sciences has been experiencing tremendous growthLife Sciences has been experiencing tremendous growthFrom 600 majors in 2000 to 1860 majors in 2012From 600 majors in 2000 to 1860 majors in 2012

The School currently offers 8 undergraduate The School currently offers 8 undergraduate concentrations, however all students enroll in a concentrations, however all students enroll in a common set of core coursescommon set of core courses

Determining PrioritiesDetermining Priorities•Where to begin?Where to begin?• Analysis of retention and student achievement Analysis of retention and student achievement

throughout the core courses showed that:throughout the core courses showed that:• For introductory courses, about 85% of initially For introductory courses, about 85% of initially

enrolled students completed the course with a enrolled students completed the course with a grade and at least 70% earned a C or bettergrade and at least 70% earned a C or better

• For upper division courses, over 90% of initially For upper division courses, over 90% of initially enrolled students completed the course with a enrolled students completed the course with a grade and at least 75% earned a C or bettergrade and at least 75% earned a C or better

Unifying the Instructional ExperienceUnifying the Instructional Experience

• Initial program assessment indicated students were Initial program assessment indicated students were encountering markedly different instructional encountering markedly different instructional experiences in the experiences in the ““samesame”” course course• Program Learning Outcomes were revisitedProgram Learning Outcomes were revisited• Instructional guides were created for laboratory Instructional guides were created for laboratory

instructors and studentsinstructors and students• Lecture instructors began holding group meetings to Lecture instructors began holding group meetings to

discuss the introductory biology coursesdiscuss the introductory biology courses

Where are we now?Where are we now?

• Students experience equivalent instruction, learning Students experience equivalent instruction, learning exercises, and assessment across the 32 introductory exercises, and assessment across the 32 introductory lab sectionslab sections• Professors teaching the introductory lectures are Professors teaching the introductory lectures are implementing a different textbook this fall and implementing a different textbook this fall and designing common instructional and assessment plansdesigning common instructional and assessment plans• The new text places more emphasis on fundamental The new text places more emphasis on fundamental

concepts and provides interactive online learning concepts and provides interactive online learning resources for studentsresources for students

• Student responses to content questions administered as Student responses to content questions administered as part of the final exam have indicated variable mastery part of the final exam have indicated variable mastery of program outcomesof program outcomes• Student responses to essay questions administered as Student responses to essay questions administered as part of the final exam have indicated variable writing part of the final exam have indicated variable writing proficiency proficiency

•UNLV is a large, urban research university with 28,000 students, UNLV is a large, urban research university with 28,000 students, and over 700 teaching facultyand over 700 teaching faculty

•NWCCU issued UNLV a recommendation on academic assessmentNWCCU issued UNLV a recommendation on academic assessment•Not doing enough assessmentNot doing enough assessment•Not doing it well enoughNot doing it well enough

•UNLV responded by charging the deans with assessmentUNLV responded by charging the deans with assessment•Ensure unit leaders submitted meaningful assessment plans and Ensure unit leaders submitted meaningful assessment plans and reportsreports•Review assessment data on a regular basisReview assessment data on a regular basis

•Faculty Assessment WorkshopsFaculty Assessment Workshops•Focus on improving student learning rather than accreditationFocus on improving student learning rather than accreditation•Designed to target all teaching facultyDesigned to target all teaching faculty•Mandatory (with cooperation from deans)Mandatory (with cooperation from deans)•Focused on assessment at two levels with emphasis on links between Focused on assessment at two levels with emphasis on links between themthem•CourseCourse•ProgramProgram

•Disseminated best practices on learning objectives, measurement, and Disseminated best practices on learning objectives, measurement, and closing the loopclosing the loop

•Emphasis on interdisciplinary information exchangesEmphasis on interdisciplinary information exchanges

School of Life Sciences- Jenifer Utz, Ph.D. School of Architecture- Kevin Kemner, M. Arch.The Workshops – Dan Bubb, Ph.D. & Lindsay Couzens, Ph.D.

Department of Dance- Louis Kavouras, Ph.D.

Building a Faculty-Driven Assessment Culture: Making it Meaningful at Large InstitutionsDaniel K. Bubb, Ph.D. & Lindsay S. Couzens, Ph.D.

University of Nevada, Las Vegas

•UNLVUNLV’’s Assessment Initiatives made us realize that assessment was going s Assessment Initiatives made us realize that assessment was going to be a fact of lifeto be a fact of life

•It gave faculty tools for how to think about assessment and learning It gave faculty tools for how to think about assessment and learning outcomes.outcomes.•It should be a useful tool. It should be a useful tool. •Rather than fight it, we should embrace it and find ways to make the Rather than fight it, we should embrace it and find ways to make the

endeavor and information vital and robust.endeavor and information vital and robust.

•In discussions we realized several thingsIn discussions we realized several things•We were assessing students all the time.We were assessing students all the time.

•We realized that if it MOVED, we assessed it!We realized that if it MOVED, we assessed it!

The big problem was how to make the information flow and put it The big problem was how to make the information flow and put it somewhere where we all could see it and use it.somewhere where we all could see it and use it.

Isadora (Integrated Student Assessment Database Online Record Isadora (Integrated Student Assessment Database Online Record Advising)Advising)

•Multi-relational Database systemMulti-relational Database system• Records are linked within the department and to the university’s recordsRecords are linked within the department and to the university’s records• All faculty have access to the database (accessible by mobile devices All faculty have access to the database (accessible by mobile devices

and on desktop computers)and on desktop computers)• Data can be input automatically to university assessment report formsData can be input automatically to university assessment report forms

SOA assessment history modeled on accreditation visitsSOA assessment history modeled on accreditation visits• Typically six year cycle with intermediate reportingTypically six year cycle with intermediate reporting• Accreditation criteria include well defined program learning objectivesAccreditation criteria include well defined program learning objectives• This led to a checklist mentality of assessmentThis led to a checklist mentality of assessment• In this environment university assessment has been historically seen as a In this environment university assessment has been historically seen as a

secondary concern.secondary concern.

Past five years there has been a shift in the accreditation process.Past five years there has been a shift in the accreditation process.• Mandate from accrediting agencies to move away from the checklistMandate from accrediting agencies to move away from the checklist• Develop issues based structure to professional degree programs.Develop issues based structure to professional degree programs.• Holistic development of the studentHolistic development of the student

Timing of shift in accreditation process was opportunistic for the school of Timing of shift in accreditation process was opportunistic for the school of architecture as it coincides with UNLV’s General Education reform and new architecture as it coincides with UNLV’s General Education reform and new emphasis on assessment process.emphasis on assessment process.• Motivated faculty of school of architecture to commit to revisiting school-Motivated faculty of school of architecture to commit to revisiting school-

wide learning objectives.wide learning objectives.

Reverse engineering of SOA learning objectives from top down:Reverse engineering of SOA learning objectives from top down:• School wide objectivesSchool wide objectives• Program/milestone objectivesProgram/milestone objectives• University accomplishment /assessment pointsUniversity accomplishment /assessment points

• Learning community objectivesLearning community objectives• Courses that can work synthetically to bridge knowledgeCourses that can work synthetically to bridge knowledge

• Course objectivesCourse objectives

Generated through year long internal SOA faculty workshops:Generated through year long internal SOA faculty workshops:• Develop common vocabulary for objectivesDevelop common vocabulary for objectives• Familiarize faculty with changes in learning objective expectations.Familiarize faculty with changes in learning objective expectations.• Resistance from faculty in technically oriented coursesResistance from faculty in technically oriented courses

How will we assess?How will we assess?• Workshop to determine assessment process and toolsWorkshop to determine assessment process and tools• Assessment to engage entire faculty (N=16)Assessment to engage entire faculty (N=16)

What will we assess?What will we assess?• Milestone accomplishmentsMilestone accomplishments• Assess the learning community not individual courses.Assess the learning community not individual courses.• Emphasis on the synthesis and application of knowledgeEmphasis on the synthesis and application of knowledge

• Student generated end of year portfolios.Student generated end of year portfolios.• Guidelines developed to encourage students to demonstrate breadth of Guidelines developed to encourage students to demonstrate breadth of

their achievement.their achievement.• Broad spectrum demonstrating the integration of skills from the learning Broad spectrum demonstrating the integration of skills from the learning

community.community.

Assessment process:Assessment process:• Generate electronic assessment tool.Generate electronic assessment tool.• PDF based collection of student portfoliosPDF based collection of student portfolios• Improve collection processImprove collection process

• RubricRubric• Observation / questionnaireObservation / questionnaire• Digital collection of responsesDigital collection of responses

Closing the loop:Closing the loop:• Individual faculty response to the assessment tool during the course of the Individual faculty response to the assessment tool during the course of the

summer.summer.• Generate two part report based on faculty responses:Generate two part report based on faculty responses:• RubricRubric• Improvement report based on faculty observation.Improvement report based on faculty observation.

• Assessment report presented to the faculty during pre-semester planning Assessment report presented to the faculty during pre-semester planning session.session.

• Improvements incorporated by faculty in upcoming academic year.Improvements incorporated by faculty in upcoming academic year.

We owe many thanks to Jenifer Utz, Louis Kavouras, and We owe many thanks to Jenifer Utz, Louis Kavouras, and Kevin Kemner for sharing their assessment journeys Kevin Kemner for sharing their assessment journeys with us. with us.