Slide 1

AAC&U General Education and Assessment February 2014 Martha VanCleave Professor of Mathematics, Interim Director of Division of Continuing Education Using ePortfolios for General Education Assessment Linfield College

Slide 2

Private, comprehensive, undergraduate institution in the Pacific Northwest 47 majors through three programs: McMinnville - traditional liberal arts and pre-professional programs (1700 students) A Portland Campus - Linfield-Good Samaritan School of Nursing (450 students) The Adult Degree Program - a pioneer in adult education primarily online (805, 410 FTE students). Linfield College

Slide 3

The Linfield Curriculum(LC) Linfields general education curriculum was revised in 2007

Slide 4

The Linfield Curriculum Structure Four major components: Inquiry Seminar(INQS) writing intensive first year course Six Modes of Inquiry Diversity Studies Major Writing-Intensive (MWI). Linfield College

Slide 5

The Six Modes of Inquiry Creative Studies (CS) Individuals, Systems, and Societies (IS) Natural World (NW) Quantitative Reasoning (QR) Ultimate Questions (UQ) Vital Past (VP) Linfield College

Slide 6

Upper-Division Mode of Inquiry An upper-division course (300 or above) In one of the six modes of inquiry CS, IS, NW, QR, UQ or VP From outside the major department Linfield College

Slide 7

Diversity Studies Global Pluralisms (GP) U.S. Pluralisms (US) Linfield College

Slide 8

Linfield Curriculum Course Requirements - LCs Students are required to complete: Inquiry Seminar (INQS) 1 course during first or second year Modes of Inquiry 6 courses,1 from each area Diversity Studies - 2 courses,1 from each area Upper-Division Mode of Inquiry - 1 course Major Writing Intensive Course - 1 course The INQS is 4 semester credits, all other courses are a minimum of 3 credits Total of 11 courses, minimum of 34 credits

Slide 9

Learning Outcomes U nique for each Mode of Inquiry and Diversity Studies area Developed by a separate group of faculty (working group) for each Mode of Inquiry and Diversity Studies area Consistent with LEAP Essential Learning Outcomes/VALUE rubrics Requests for courses to fulfill requirements reviewed and approved by working group Le arning outcomes utilized when reviewing courses Learning Outcomes must be included in syllabus Syllabus must describe how learning outcomes will be met

Slide 10

Linfield Curriculum Assessment Plan (LCAT) adopted 2010

Slide 11

LCAT Students must demonstrate meeting the learning objectives of each mode of inquiry and area of diversity studies through: Direct evidence Choosing an exemplar or exemplars Posting the work in an online repository Indirect Evidence Supporting the choice of these exemplars with a reflection on how they meet the learning outcomes Linfield College

Slide 12

LCAT Faculty Working group for each LC designation Develop rubric for assessing evidence of student learning outcomes Assess random sample of student submissions Linfield College

Slide 13

LCAT Assessment of LC submissions on a rotating schedule 2 designations per year First assessment summer 2011 - QR and VP 2012 US and NW 2013 GP and UQ 2014 CS and IS Cycle repeats Linfield College

Slide 14

First Round of Assessment QR and VP assessed

Slide 15

QR and VP assessment Exemplars submitted in LMS Limited ability to provide instructions for students No tool for collecting demographic data Submissions downloaded for evaluation Record of evaluation compiled manually Linfield College

Slide 16

Assessing QR

Slide 17

ExemplarySatisfactoryUnsatisfactoryNA 1. Pose questions involving quantitative relationships... 35 (39.8%)24 (27.3%)25 (28.4%)4 (4.5%) 2. Analyze problems by discussing models 17 (19.3%)29 (33.0%)38 (43.2%)4 (4.5%) 3. Understand uses and constraints 7 (8%)17 (19.3%)59 (67.0%)5 (5.7%) 4. Communicate and critique quantitative arguments. 10 (11.4%)20 (22.7%)52 (59.1%)6 (6.8%) Assessing QR Calibrate use of assessment rubric Random selection of submissions scored by 2 readers, then consensus Results Linfield College

Slide 18

Assessing QR Observations and recommendations Rated satisfactory or above #1 67.1%#2 52.3%#3 27.3%#4 34.1% Difficulty submitting materials from online courses Students need guidance Revision of learning outcomes Need benchmarks Linfield College

Slide 19

Lessons Learned Students need direction and structure Faculty should make connections between coursework and LC learning outcomes Linfield College

Slide 20

Implementing an ePortfolio system

Slide 21

Selecting the ePortfolio Platform Working group Faculty Dean of Student Affairs Registrar Associate Dean of Faculty Chief Technology Officer Director of Educational Technologies Linfield College

Slide 22

Selecting the ePortfolio Platform Criteria Tiered access Matrices Evaluation Areas for student reflection Ability to aggregate data Aesthetics and usability Support from provider Linfield College

Slide 23

TaskStream and Linfield Create a Directed Response Folio (DRF) Work done by team of faculty Supported by TaskStream Mentoring Services Linfield College

Slide 24

The DRF Linfield College.

Slide 25

The DRF form https://www.taskstream.com/Main/main_frame.asp https://www.taskstream.com/Main/main_frame.asp Form for each area Course Location Course Subject Code Course Number Course Name Course Delivery (option that best describes how this course was taught) Year Course Was Taken Class Standing Term Course Was Taken Linfield College

Slide 26

The DRF form LC Outcomes Please select all of the QR learning outcomes covered in this course. For courses taken at Linfield, this information can be found in your course syllabus. For transfer courses, check all that apply. Note that for courses taken at Linfield, all four learning outcomes should be selected. In addition, if this is a transfer course, please select "Transfer Course." Description of Assignment(s) for Submission Supporting Description of Quantitative Reasoning Exemplars Linfield College

Slide 27

TaskStream and Linfield Enroll faculty and students Create single sign-on using CatNET Username and Password https://www.linfield.edu/taskstream/index.php Batch process enrollment via secure FTP site Linfield College

Slide 28

Assessment Using Taskstream

Slide 29

Developing Rubrics with Taskstream

Slide 30

Beneficial Features of TaskStream DRF provides structure for students Form developed to provide explicit instructions Online evaluation using the rubric Data storage Data reporting Linfield College

Slide 31

Assessing NW

Slide 32

ExemplarySatisfactoryUnsatisfactoryAve. Score 1. Use scientific method... 13 (17%)6 (8%)56 (75%)0.43 2. Think critically 27 (36%)29 (39%)19 (25%)1.11 3. Historical development of ideas 12 (16%)5 (7%)58 (77%)0.39 4. Appreciate how knowledge affects... 12 (16%)26 (35%)37 (49%)0.67 Assessing NW Calibrate use of assessment rubric Random selection of submissions scored by readers Results Linfield College

Slide 33

Assessing NW Observations and recommendations Rated satisfactory or above #1 25%#2 75%#3 23%#4 51% Only 6 submission (8%) received at least a satisfactory for all four learning outcomes 13 submissions (17%) received no satisfactory scores Most students submitted a single document Lack of understanding of what constitutes a goo exemplar Revision of learning outcomes Linfield College

Slide 34

New NW Learning Outcomes Courses with the NW designation are intended to help students understand the scientific method. In particular, students must demonstrate 1) an understanding of the theoretical and/or experimental background of a particular topic or model, sufficient to form a hypothesis 2) an ability to critically analyze results of scientific inquiry in light of assumptions, 3) an understanding of how scientific results can be extended to more general situations in contemporary society Courses with NW designation address all of the above learning outcomes.

Slide 35

Lessons Learned More attention needs to be paid to developing studentsunderstanding of what makes a good exemplar Linfield College

Slide 36

Advantages of Using Taskstream Streamlines mechanics of evaluation process Additional reports available (summary statistics) Provides for more meaningful faculty discussions Between pairs of faculty evaluators Among members of working group (4 faculty) With faculty teaching courses in each area through sharing of assessment results

Slide 37

Challenges Student understanding of the LC assessment Individual student data is not assessed Problematic to ensure students take the task seriously Need to further develop reflective habit of mind Faculty development Use of ePortfolio Consistent articulation of relationship between LC learning outcomes and course content Linfield College

Slide 38

Thanks Linfield College Charles Dunn - Curriculum Committee Chair, director of LC Assessment, co-author of DRF Susan Agre-Kippenhan Vice President of Academic Affairs, Dean of Faculty TaskStream Ben Coulter Manager, Systems Implementation Jose Romero Mentoring Services Lauren McCarty Associate Director of Marketing Dara Wexler Education Solutions Specialist Trudy Milburn - Education Solutions Specialist Linfield College

Slide 39

Contact Questions contact Martha VanCleave mvcleave@linfield.edu Linfield College