The Role of Faculty Institutes in Improving Student Learning: Core Abilities Assessment Report

Submitted by Minerva Holk

June 2014

1 | P a g e

Table of Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Planning for the Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 The Institute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 The Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Feedback from Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Appendix A Invitation to Faculty. . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Appendix B Sample Collection Materials . . . . . . . . . . . . . . . . . . . . 11 Appendix C Permission to use Student Work . . . . . . . . . . . . . . . . . 14 Appendix D Permission to use Online Student Work . . . . . . . . . . . 15 Appendix E Sample Rating Materials for Assignments . . . . . . . . . . 16 Appendix F Sample Rating Materials for Student Samples . . . . . . . 17 Appendix G Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Appendix H Faculty Evaluation Tool . . . . . . . . . . . . . . . . . . . . . . . 19 Appendix I Institute Evaluation Responses . . . . . . . . . . . . . . . . . . . . 20

Introduction

The Summer Institute was held June 17 and 18, 2014, and focused on the Thinking Core Ability.

Twenty-nine (29) faculty attended. The faculty represented all the divisions at OC and included full-time

(16) and adjunct (13). There were seasoned Institute attendees and nine who attended for the first time.

The format was slightly different from previous Institutes in that the first day included a workshop on

Quality Matters (QM) titled “Standard 2/3 Connecting Learning Objectives and Assessments.” The

workshop was presented twice and the faculty attended one session. The other half of the day was used to

evaluate sample assignments and student work. The Quality Matters sessions were held in HSS 110 and

the rating sessions were held in HSS 112. There were 16 full-time faculty (55.2%) of the participants

were and 13 (44.8%) were adjunct faculty. Nine participants (31%) had not attended a previous Institute.

An overview of the process and the results attained will be presented, followed by recommendations for

future Institutes.

Planning for the Institute

Invitations were sent using the all faculty email option. First time participants were required to

watch an on-line video prior to the Institute. Viewing the pre-Institute video was mandatory for new

Institute participants and replaced the previous use of the first hour of the Institute to orient new faculty

attendees. The video link was also available to returning participants who wanted to refresh personal

knowledge of the goals and procedures of the Institute. The video reviewed the history of Core Abilities

at Olympic College, reviewed the five Core Abilities, and explained the expectations for participants

during the two-day workshop.

All faculty were invited to submit sample assignment prompts and five samples of student work.

Faculty who were attending the Institute were requested to submit samples but few did. One possible

reason for the decrease in samples submitted was the presentations that were to be done by the faculty

who attended the Winter 2013 Institute on the second afternoon of the Institute Specific guidelines on the

requested submissions are included in Appendix B. A coversheet was attached and to be completed for

the samples submitted including the faculty assessment of the competency level for meeting the identified

Core Ability for the assignment and each student sample. Samples were accepted in hard copy or

electronically via Canvas or the Institute Coordinators mailbox. Samples were only requested from

faculty who were attending the Institute.

Submission of samples was not a prerequisite for attendance at the Institute which resulted in

fewer submissions and a more thorough evaluation of the assignments received. The limited sample

submissions resulted in a change in the structure of future assessment Institutes and will be discussed in

the recommendations. The decision was made to print enough copies of each submission for all

participants to rate them at the Institute. In general, sample collection and preparation continues to require

significant effort both in terms of time needed to follow-up with faculty, and time involved to prepare the

samples for copying. Electronic submissions decreased the work involved preparing for copying.

An outside catering company was selected to provide the lunches for the Institute. Aside from

lunch, coffee was provided each morning and light refreshments were provided in the morning before

beginning the Institute as well as in the afternoon. Name tags for participants, and packets containing

several key documents were prepared by the Office of Instruction. Lunch orders from registered faculty

were collected, and the catering order submitted, by the Office of Instruction.

The Institute

Institute participants were given Quality Matters™ packets containing pertinent information

related to learning objectives, assessment and measurement, and a Quality Matters™ Rubric Workbook

for Higher Education. The packet also included a verb wheel based on Bloom’s Taxonomy.

Information relating to the Core Abilities was also provided including agenda (See Appendix I), the

College’s Mission Statement and Guidelines for Assessing Core Abilities, Outcomes and Rubric for the

Thinking Core Ability, a Glossary of Terms for Core Abilities Rubrics. Outcomes and rubrics for the

other Core Abilities (Communication, Global Perspective, Lifelong Learning, Information Literacy and

Technology) were available for faculty. Each participant was given a name tag. Table numbers were not

assigned at this Institute due to the last minute accommodation of several attendees’ schedules and being

in separate rooms for each half of the first day. The faculty were given the option of self-selecting their

Table collaborators and to use the same group on day two of the Institute.

The self-selected table groups were a mix of experienced and new faculty, as well as adjunct and

full-time (Table 1). Working in groups of four, with one group of five, assignments and student samples

were rated using the Thinking Rubric developed by the Core Abilities’ Taskforce. There was

representation at the Institute from each division at Olympic College (Table 2). The participants engaged

in the rating process with enthusiasm and diligence.

The initial step in the process was a group norming session in which each participant read the

same assignments and the samples of student work submitted for the specific assignments. The

participants rated both the assignment and the respective student samples according to the four levels of

proficiency utilized in each Core Ability: Emerging, Developing, Competent and Strong. Individual

evaluation was followed by a collaborative discussion during which group members at each table collated

their results and created a ‘table rating’ for the assignment and the sample. The table results were then

reported to the group at large and discussion of the results followed. An important component of the

norming session is to increase consistency in rating and identify possible biases in the evaluators. Of the

seven groups, the norming assignment was rated as providing the opportunity to meet Thinking Outcomes

one and two at the developing level. The student samples provided evidence that Thinking Outcomes one

and two were met at the emerging level. There was active participation in the discussion that followed

resulting in clarification of the purpose of the Institute which is to assess student learning using the Core

Abilities. After the norming session, the individual groups applied the rating process to assignments and

samples specific to the Thinking Core Ability. The Mission Statement for Assessing Core Abilities

states: Olympic College seeks to improve teaching and learning by focusing inquiry to ensure that

students are getting ample opportunities to develop Core Abilities; and students are performing

sufficiently on Core Abilities. There was uniform agreement about the Outcomes met by the assignments

and student samples.

The tables below give the specific information according to teams arranged for the review and

discussion of assignments and samples each day.

Table 1: Institution Level Assessment of Core Abilities Summer Institute 2014 Thinking

Group 1 (AM) Carey, Diane Land, Cynthia

Westlund, Mark Bermea, Nancy Elauria, Angela Lawrence, Amy

Group 2 (AM) Rubin, Tova

Gold, Emma Kirk, Nari Latham, Colin Tuncol, Goker

Group 3 (AM) Farley, Sandra

Group 1 (PM) Bright, Kathy

Rubin, Tova *# Kerdus, Mary

Krystal, Barbara *

Group (PM) Johnson, Dawnette # Hayward, Victoria #

Parker, Barbara Mackaben, Kandace *

Group 3 (PM) Mulligan, Anne

Baldwin, Kathleen *# Stevens, Leon * Latham, Colin *

Group 4 (PM)

Blackwell, Kevin Puskarcik, Russ *# Pellock, JD DeWire, Elinor *#

Key:

*: Part time #: new

Table 2: Institutional Level Assessment of Core Abilities Summer Institute 2014 Faculty Participant List by Division

Business and Technology

Mathematics, Engineering, Sciences and

Health

Nursing Social Sciences and Humanities

Library

Bermea, Nancy Elauria, Angela Hayward, Victoria Baldwin, Kathleen Bright, Kathy

Blackwell, Kevin Lawrence, Amy Holk, Minerva DeWire, Elinor Carey, Dianne

Farley, Sandra Pellock, JD Johnson, Dawnette Gold, Emma

MacKaben, Kandace

Rubin, Tova Kerdus, Mary Hendrickson, Kimberly

Parker, Barbara Tuncol, Goker Mulligan, Anne Kirk, Nari

Puskarcik, Russ Krystal, Barbara

Stevens, Leon Land, Cynthia Westlund, Mark Colin Latham

Executive Summary

The Core Abilities Summer Institute 2014 focused on the Thinking Core Ability as identified in the

Olympic College Master Plan for Core Ability development and assessment. The outcomes for the

Thinking Core Ability are:

Outcome 1: Graduates engage in critical analysis. Outcome 2: Graduates engage in creative problem solving. Outcome 3: Graduates engage in quantitative reasoning. The Data Each participant was given a stack of ratings sheets (See Appendix E & F). Each faculty member

was asked to rate the assignment prompt and the set of student samples that accompanied the assignment

prompt as Emerging, Developing, Competent or Strong according to the Thinking Core Ability Rubric.

After each participant had rated the sample and the accompanying assignments, the tables were asked to

discuss both the assignment and the samples and to reach a consensus rating for the assignment and for

each sample. The table groups reviewed and rated the assignments individually and then as a group and

discussed the ratings before assessing the student samples. After the student samples had been evaluated,

the groups discussed the proficiency levels for each assignment and determine the leveling from their

group for each assignment and the levels attained by the student samples. The discussion also provided an

opportunity to write comments for the Institute Chair to send back to the faculty who submitted the

sample. The limited time at this Institute for evaluation of assignments and samples worked out well as

there were three assignments and 16 student samples were reviewed. Two student samples were five to

six pages and one was two pages. The assignments ranged from one page to four.

Sample one was from a nursing faculty. The assignment was rated at competent for Outcomes

one and two and not applicable for outcome number three. The assessment by the faculty who submitted

the assignment was consistent with the assessment by the evaluators at the Institute. Four student samples

were rated as Outcome One: emerging, one; developing, two, competent, one; and strong, one. For

Outcome Two the samples were rated: emerging, one; developing, one, competent, one; and strong, one.

Sample two was from an English faculty. The faculty had rated the assignment as developing and

addressed two Core Ability Outcomes. The faculty at the Institute rated the assignment as competent for

the first Outcome and did not identify the assignment as meeting either Outcome Two or Three. The

student samples had been rated by the faculty submitting as emerging or developing. The Institute

evaluation was that Outcome One was met at the competent level by three students, developing by two

and emerging by one student. There was unanimous agreement by the Institute assessment that Outcomes

Two and Three were not addressed by the assignment or the student samples.

The third sample assignment was rated by the submitting faculty as addressing Outcomes One

and Two at the Strong level. Of the five assignments submitted, two were rated by the submitting faculty

as strong and three were at developing. The Institute assessment supported that Outcomes One and Two

were addressed by the assignment and provided opportunity for the students to gain a measure of

competency at the level of competent. The student samples received ratings between less than emerging

and strong from the Institute groups. One student sample was rated as strong, two competent, one

developing and one less than emerging. The samples the faculty rated as strong were rated by the

Institute as one strong and one competent.

1. Opportunity for Student Growth

The assignment ratings and the sample ratings by the faculty submitting and the faculty at the

Institute support that the assignments provide sufficient opportunities so that students can learn,

grow, and improve their comprehension of the material being addressed in the academic unit

being taught. The assignments submitted were rated by faculty submitting as either developing,

competent, or strong. The submission rated at the developing level was from a course offered

early in the student’s college experience, likely first quarter. The assignment was also completed

early in the quarter. There were similarities between the groups and the ratings assessed.

2. Consistency

There was consistency between the raters as to whether the assignments and student samples

addressed the Thinking Core Ability. The results indicate consistency between; a) faculty at

different tables rating the same samples and b) faculty ratings of their own assignments and the

ratings given to those assignments by their peers at the Institute. The small number of

assignments and samples presented limits the interpretation of the findings.

The faculty members who attended the Institute were energized by the process and worked very

diligently. According to the evaluation forms completed at the end of the Institute, more understanding is

needed by some faculty about the Core Abilities. The increased awareness of the Core Abilities was

particularly evident among faculty attending an Institute for the first time. This reflects positively on the

sustained campus-wide efforts to increase awareness of, and buy-in for, Core Abilities at Olympic

College. Many participants expressed continued enthusiasm for integrating Core Abilities into their

classrooms and reviewing their assignments to improve their clarity and usefulness vis-à-vis the

integration of Core Abilities. Review of the Core Abilities and evaluation from the attendees is leading to

potential consolidation of the committees at Olympic College involved with similar aspects of assessing

student learning. The connection with the Quality Matters™ may prove beneficial to the group at this

Institute according to the evaluation information and many faculty are interested in increasing their

abilities to assess student learning.

The second afternoon of the Institute was designed to have faculty share updates on the projects

they worked on at the Winter Institute 2013. The presentations were representative of all disciplines of

the College and provided an opportunity for faculty to showcase some of their excellent work as well as

discussion of some challenges related to the assessment process. The workshop on Quality Matters™ had

stimulated the need for deliberateness in the assessment process and the importance of clearly defined

outcomes for the identified learning goals. Several faculty included on the evaluation form that they

would like to have more opportunities for this type of association with their colleagues. The presentations

ranged from brief updates of syllabi revision and course development to the development of a new

program and curriculum. Two faculty presented a project designed to showcase student accomplishment

of the outcomes and the collaboration between faculty teaching different sections of the same course to

identify the Core Abilities addressed and accomplished. It was an excellent example and the faculty

present were very positive in their feedback. The sharing from the faculty provided insight into the

challenges of teaching and the similarities of many situations even though the course content may be very

different.

Feedback from Participants

An Institute Evaluation Form was distributed at the end of the second day. It was completed by

26 of the 29 participants. The evaluation tool and a complete list of the comments received are included in

Appendices H and I. Post-it Notes were placed on every table and participants were encouraged to write

suggestions and comments throughout the Institute. In previous Institutes the Post-it Notes were used by

many faculty but no comments or questions were received using Post-it Notes at the Summer 2014

Institute. The overwhelming belief that the Core Ability Institutes are worthwhile was reaffirmed by 23

faculty selecting the yes option, one chose no and two did not respond. Eighteen faculty indicated they

will be changing learning outcomes or assessments based on what they learned in the Institute; one opted

for maybe; two said no; and two did not respond. As a result of attending the Institute; 23 said they would

attend a similar event in the future, 21 chose the strongly agree, two agreed, and one selected ‘no

opinion’.

The recommendations below are based on information gleaned from the Institute Evaluation

form, observations made during the Institute by the two Coordinators, and data derived from the rating

sheets.

Recommendations

1. Make explicit the link between this work and Olympic College’s accreditation by the Northwest

Commission on College and Universities (NWCCU) and what accreditation really means to

Olympic College.

Some faculty members do not see, or understand, the link between the work being done on

Outcomes and the stipulations placed on the College by our regional accrediting commission.

2. Update tutorial available in Tegrity (no longer available at Olympic College) to continue

streamlining the work we do at the Institutes and make available online using Panopto.

The Pre-Institute Workshop has been on Tegrity for two years and will be updated to make it

more pertinent to the changes we are making related to the assessment and tracking of

competencies in the Core Abilities. One attendee at the 2014 Institute commented on the

evaluation tool that “the video could be condensed”. Having new Institute participants view the

video before the Institute has streamlined the introductory period and enabled faculty to get

started with the norming and evaluation sooner increasing efficiency at the Institute. It was also

suggested that a Tegrity (Panopto would be used as Tegrity is no longer an option.) lecture on the

basics of Course Mapping would be helpful and the LAC will consider moving this forward. Key

elements of this lecture might include: how mapping is done (a demonstration of the database),

why it is done, and some common errors and considerations.

3. Revise and implement change in sample collection process.

Collection of the sample assignments and student work requires significant time on the part of the

Core Ability Chair. Submitting the samples takes time and effort on the part of the faculty

member who are contributing samples for evaluation. Changes made to allow submission of

samples electronically has been minimally adopted. Discussions are underway in the LAC to

investigate the feasibility of collecting samples via Canvas©. Collecting via Canvas© will

increase objectivity of selection of student samples and copies may be made by the LAC, tracking

will also be facilitated. Rating assignments and student work is a necessary requirement to

measure learning outcomes at Olympic College and a broad range of samples is desired,

employing other tactics to solicit samples is necessary. Another change to the collection of

samples will be implemented before the Winter 2014 Faculty Institute to specifically choose

classes on the list of approved as meeting Core Ability graduation requirements, and being taught

during fall quarter, 2014. The method implementation will be included in the Winter 2014

Institute Summary Report.

i. Detailed instructions will be provided to faculty from whom samples have been

requested. This will make the process of sample preparation and duplication more

streamlined and efficient for the facilitators. Specifically, in addition to the assignment

and grading/assessment rubric, participants will be asked to submit 10 samples which

should be single-sided, paper-clipped rather than stapled, and all names must be removed

prior to submission.

ii. The option of submission via Canvas© is anticipated for sample collection during fall

quarter, 2014. The utilization of Dropbox© may also be an option to support faculty

submissions electronically throughout the quarter. A conversation has been initiated with

Kathy Bright, Director of Distance Learning, to determine the most appropriate

implementation process – specifically which platform is utilized by the greatest number

of faculty? Canvas©, Dropbox.com, Wiggio, Googledocs, or some other site?

4. Consider having chairperson for Institute prepare anonymous listing of faculty assessment of

assignment and student samples.

Faculty are asked to rate the assignments and the samples submitted. Since the Winter 2012

Institute several faculty have expressed interest in knowing how the faculty member who

submitted an assignment had rated the assignment. A concern of the Institute chair is the

possibility that knowing how an assignment or student sample was “graded/rated” by the

instructor before rating it oneself (at the Institute) would compromise the assessment ratings at

the Institutes. The information is available in the raw data and could be included in the Institute

Summary but the faculty would like to know sooner. Further discussion amongst the LAC

members will be done before this information is shared for specific assignments and samples.

5. Continue the conversation with the campus community why stipends are necessary for the

continuation of this work.

Faculty consistently report on the feedback survey that they would not continue participating in

the Institutes if a stipend were not offered. Yet, in these tough economic times, alternatives to

stipends must be considered. However, given the fact that the Institutes occur on non-contract

days, best practices stress that those involved evaluating student outcomes must participate

voluntarily (as opposed to, say, program evaluation by IPP which is mandatory). Additionally the

time required to conduct this work is difficult to undertake during the quarter while faculty are

teaching and grading.

6. Continue curriculum development and incorporation of the Core Abilities at the Institutes.

Feedback at the Summer 2014 Institute was positive about the cross-discipline efforts at the

Institute. Some faculty members would like examples of how other instructors have implemented

Core Abilities as a division. Sharing examples of instructors’ incorporation of Core Abilities into

assignment prompts directly was also desired.

7. Feedback to those who submit samples requires increased systemization.

Faculty requests for feedback on their samples and assignments is challenging using the current

method which is tasked to the Institute chair. The tracking of the faculty’s submission and the

table ratings must be collected and an email composed and sent to the faculty. It would be

helpful if an automated system could be implemented.

8. Sharing examples of instructors’ incorporation of Core Abilities into assignment prompts directly

was also desired.

Faculty who participated in the Winter Institute 2013 shared the progress made in their courses on

the afternoon of the second day of the Summer Institute 2014. The feedback on the evaluation

form was very positive for this section of the Institute and should continue to be a part of the

Institutes.

9. The Quality Matters™ Workshop received high ratings on the evaluation forms. The faculty

requested additional similar workshops to support their development of outcomes and appropriate

assessment methods for their courses. See the Professional Development Subcommittee’s Year

End Report for Learning Assessment Council, May 2014 for additional information about Quality

Matters™.

Appendix A Application to Participate in The Core Abilities Institute

Summer 2014

I would like to participate in Institutional Level Analysis and Curriculum Development tied to Core Abilities. As part of this process, I agree to engage with other faculty in collaborative discussion and curriculum development on Tuesday, June 17th from 09 AM-4 PM and Wednesday, June 18h from 09 AM-4 PM. The location will be sent to attendees after receipt of application. Faculty attending for the first time are required to watch the Pre-Institute Workshop on Tegrity. The link will be included in the follow-up email. I understand I will receive a $400 stipend for participating in this entire two-day process.

To apply, please complete this sheet and send it as an email attachment to: (mholk@olympic.edu) by Friday, May 30, 2014.

Faculty Name: __________________________ Discipline: ______________________ Division: ________________ Do you work full-time or part-time at OC: ________________

Please check this box if this is your first time attending a Core Abilities Institute: ___ Please check this box if you need to integrate Core Abilities into your syllabi: ___ Please check your first preference for the Quality Matters Workshop: AM_____ PM_______

There is limited funding for stipends so please reply early for your best chance to participate.

An effort will be made to include faculty (full and part-time) representing all of the various divisions as well as first time attendees.

All faculty are encouraged to apply.

Appendix B

DUE JUNE 10, 2014 Contact Person (faculty name, please print): _____________________________ (this name will be removed before samples are evaluated) Course Number: _____________ Number of Samples1: ________ (example: SOC 190, MATH& 264) (Please submit 5 random samples) Course Background: Please tell us a bit about your course:

• Are there any prerequisites on the course? If so, please list: _____________

• When is this assignment given? Early Middle End of Course ?

• Does this Assignment build on other areas of the course (previous assignments)? If so please tell us what previous assignments have been completed that would help understand the quality of the samples submitted: __________________________ ____________________________________________________

• Are there any other instructions/resources/preparation (e.g. library tutorial or in-class demonstration) given to students that would help contextualize the samples? If so, please list: ____________________________________________________

1 Please see the section “What NOT to Submit” on p. 4 for some guidelines on the kinds of samples we can and cannot use.

COVER SHEET SUMMER 2014 SAMPLES FOR ASSESSING CORE ABILITIES

• Please number each sample in the top right corner in pencil. Then please tell us the level you think each sample attains:

Sample 1 Emerging Developing Competent Strong Sample 2 Emerging Developing Competent Strong Sample 3 Emerging Developing Competent Strong Sample 4 Emerging Developing Competent Strong Sample 5 Emerging Developing Competent Strong

If you would like feedback about what we learned after using your samples at the institute please indicate below:

Yes, I would like feedback No, I do not need any feedback

Please select one Core Ability the work is most clearly demonstrating.

Thinking Outcome 1: Graduates engage in critical analysis. What general level do you feel this assignment addresses (check only one box for each outcome)2:

Emerging Developing

Competent Strong

2 Please see any of the rubrics for a definitions of the standards expected at each of these four levels.

Thinking Outcome 2: Graduates engage in creative problem solving. What general level do you feel this assignment addresses (check only one box for each outcome):

Emerging Developing

Competent Strong

Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level do you feel this assignment addresses (check only one box for each outcome):

Emerging Developing

Competent Strong Send submission via campus mail to: Minerva Holk: mholk@olymic.edu What NOT to Submit

• When selecting the 5 samples to submit please select random samples so that we can obtain a representative cross section of the student population. In short, we do not only need or want examples of the best work.

• Do not submit graded work – only clean, unmarked copies of student work should be submitted.

• Do not submit copies of multiple choice, True-False, short answer or matching tests.

• Do not submit copies of entire student course or program portfolios or journals. Components of a portfolio or journal that demonstrate a particular Core Ability may be submitted.

• Do not submit samples in a foreign language or that cannot easily be understood by faculty outside your discipline (for example containing complex chemical or mathematical equations)

• Materials will not be returned - do not submit original student work or work that you need returned for grading or other purposes.

• Do not submit samples from developmental or ABE courses.

• Do not submit samples without the corresponding assignment and cover sheet.

Thank you for your help with these efforts!

Appendix C

PERMISSION TO USE STUDENT WORK

Quarter & Year: Instructor: As indicated by my signature below, I authorize Olympic College to use my work as part of one or more Olympic College activities that are specific to campus-wide assessment for student learning outcomes. Your name and other identifying information will be removed. PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________ PRINT NAME _________________________________________ SIGN NAME_____________________________________

Appendix D

PERMISSION TO USE STUDENT WORK: ONLINE CLASS

Hello class,

As of Summer 2012, new students are required to take coursework addressing each of five Core Abilities. The Core Abilities are: Communication, Thinking, Information Literacy and Technology, Lifelong Learning, and Global Perspective. This class (Class name, Number) should satisfy the thinking and XXX (fill in which core abilities your class addresses) core abilities, while meeting the existing quantitative reasoning requirement, too.

In preparation for this requirement there will be an institute during your break in which faculty will evaluate student work from classes that might be used to meet the requirements to determine whether students are, in fact, demonstrating that the Core Abilities are addressed.

I will be submitting X Project (fill in title of assignment) to this group. Your name and any identifying information will be removed before any other faculty see the projects, and all work will be shredded once the papers have been evaluated.

If you do not want your work to be included, please let me know before Friday, and I will not submit it.

Thank you in advance,

Your Name

Table #: _______

SAMPLE LETTER: _______

TABLE SUMMARY: _______

Please indicate outcome and the level of attainment the work is demonstrating (check all that apply):

Thinking Outcome 1: Graduates engage in critical analysis. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

Thinking Outcome 2: Graduates engage in creative problem solving. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

APPENDIX E EVALUATION TOOL-THINKING ASSIGNMENT

WINTER 2011 ASSESSING CORE ABILITIES TABLE SUMMARY:____

Suggestions for faculty:

APPENDIX F EVALUATION TOOL-THINKING STUDENT SAMPLE

WINTER 2011 ASSESSING CORE ABILITIES TABLE SUMMARY

Table #: _______

SAMPLE LETTER: _______

TABLE SUMMARY: _______

Please indicate outcome and the level of attainment the work is demonstrating (check all that apply):

Thinking Outcome 1: Graduates engage in critical analysis. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

Thinking Outcome 2: Graduates engage in creative problem solving. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

Thinking Outcome 3: Graduates engage in quantitative reasoning. What general level does the team believe this assignment addresses (indicate the number of team members in box for each outcome level):

Emerging Developing Competent Strong

Suggestions for faculty:

Appendix G

Agenda for Summer Institute 2014

Tuesday, June 17, 2014: 08:00 a.m. to 5:00 p.m. Faculty Institute/Quality Matters Workshop

Lunch Provided at 1200

Faculty Institute

08:00-12:00 and 1:00 p.m.-5:00 p.m. (Alternates with Quality Matters)

a. Core Abilities, definitions and explanations of rubrics b. Explanation and discussion of the goal(s) of the Institutes in general c. Norming in preparation of assessing assignments and student work d. Assessing assignments and samples of student work

Quality Matters: 0800 a.m.-12:00 p.m. and 1 p.m.-5:00 p.m. Alternates with Faculty Institute Wednesday, June 18, 2014: 09:00- 3:30 p.m.

09:00-11:00: Assessing assignments and samples of student work

11:00-12:00: Presentations from Winter Institute

12:00-12:45: Lunch provided

12:45-1:45: Presentations from Winter Institute 2013

1:45-2:45: work on either outcomes or course mapping of Core Abilities in prof/tech for graduation requirements

2:45-3:45: Large Group Debrief: Post-Its, Progress, and Questions

Appendix H

Faculty Institute Summer 2014 Evaluation Summary Data

Please add any additional comments on the back. Thank you for participating!

1. This Institute included a professional development workshop for the first time. Do you think professional development is an important component for an Institute?

a. Yes b. No 2. Was the topic for the Quality Matters workshop appropriate/or related to the other work being

done at the Institute? Circle one: Yes No a. List other topics that would be helpful to you

3. Was the information useful? Circle one: Yes No

4. Will you be changing learning outcomes or assessments based on what you learned? Circle one: Yes No

5. Are you interested in additional training and/or information to increase your understanding of the Core Abilities and the process set up to assess them?

a. Outcome development, either course or assignment b. Rubric development c. Grading on Canvas d. Other?

________________________________________________________________________________________________________________________________________________

6. What did you like best about this Institute?

7. Would you participate in a similar event in the future? □ Strongly agree □ Agree □ Disagree □ Strongly disagree □ No opinion

8. Do you have suggestions to improve the event? Circle one: Yes 11 No If yes, please elaborate:

9. What questions remain about Core Abilities and assessment of student learning at Olympic

College? 10. Do you see a need for more workshops on course mapping with Gloria Martin? Circle one:

a. Yes b. No 11. Do you have suggestions for the timing of the Institutes? Rank in order of preference

a. One day b. Two consecutive days c. First non-contract day after end of quarter d. Second non-contract day after end of quarter

Appendix I

Faculty Institute Summer 2014 Evaluation Summary Data

Please add any additional comments on the back. Thank you for participating!

1. This Institute included a professional development workshop for the first time. Do you think professional development is an important component for an Institute?

a. Yes 25 (one added definitely) b. No 0 2. Was the topic for the Quality Matters workshop appropriate/or related to the other work being

done at the Institute? Circle one: Yes 22 No 0 Not sure (added by evaluator): 1 a. List other topics that would be helpful to you

i. viewing other online courses ii. I would do separate workshops for each standard so that we would go in more

depth, rather than offer the same workshop twice & split the group iii. Accessibility and core abilities- what does a “strong” core ability look like for

someone with a disability? iv. Student motivation v. Designing to prevent cheating

vi. Other of the QM workshops. Also, function of Canvas. Since we are changing to Panopto, that would be good now.

vii. Not sure viii. Rubic [sic]

ix. Accessibility would be good. Repeat outcomes & assessment for those who have not done them.

x. More Quality Matters training. xi. I would like to see other standards covered every month for a half day if

possible.

3. Was the information useful? Circle one: Yes 23 No 1 I really didn’t get new info; Two (2) people gave no response, one added “I will take a look at them again”

4. Will you be changing learning outcomes or assessments based on what you learned? Circle one: Yes 18 (added comment by one: “I will be assessing now”) No 2 Maybe 1 (option not on evaluation form. No response 2; added comment “Creating a blended class and need to redo all outcomes”

5. Are you interested in additional training and/or information to increase your understanding of

the Core Abilities and the process set up to assess them? a. Outcome development, either course or assignment 9 b. Rubric development 17 c. Grading on Canvas 11 d. Other?

i. No: 1; No response 1; ii. advanced Canvas;

iii. “Is this data driven? What are the statistics showing this is a benefit to students”.

iv. All of the above. v. Anything that will help us do things faster.

6. What did you like best about this Institute?

a. Working with peers and seeing their perspectives and comments b. Learning from other faculty about their projects c. Interaction with colleagues d. Group support e. No answer f. Aligning outcomes with assessment g. Opportunity to work on my courses h. Casual atmosphere i. QM session j. Collaboration with other instructors k. Working with faculty in other disciplines l. Getting to hear what people have done with this information on 2nd afternoon.

Appreciate the venue & food. m. I liked the organization of this institute. n. QM workshop o. Looking at student samples; assessing them, really enjoyed the QM session- It helped to

clarify several things for me p. Interaction and discussion to develop usable perspective q. Working with other faculty, friendly atmosphere. r. Working with fellow faculty/see what I am doing right based on the presentations. s. Interaction and working with individuals from other areas of study/department t. The presentation of winter institute was the most informative u. Working with other disciplines & with my own to level assessment v. Networking with other faculty and discussing our challenges w. The chance to work with other profs & the QM training x. Meeting other disciplines and collaborating. Seeing how other faculty apply core

abilities.

7. Would you participate in a similar event in the future? □ Strongly agree 12 □ Agree 11 □ Disagree □ Strongly disagree □ No opinion 1

8. Do you have suggestions to improve the event? Circle one: Yes 14 No 7 NoResponse 1 If yes, please elaborate:

a. More examples with rubrics b. More planning needed. QM would make more sense if I was told to bring my

syllabus/course outline c. Give instructions to participants before 1st day on what they will be working on/should

bring to class. d. One selected yes but did not elaborate

e. Show examples from all disciplines that includes professor’s evaluation of that work/example

f. NO MORE RATING STUDENT SAMPLES. g. Overall less time is needed to complete tasks. Could have accomplished all the tasks in ¼

of the given time. h. Breaks every 2 hours; other activities besides reading, especially just after submitting

grades i. I would like to have had some of the terminology and associations with the Core

Abilities evaluations explained before doing the evaluations j. More time for interacting with staff from a variety of disciplines- and, a chance for each

staff to share what is new in their field or that they have learned in the past three months.

k. 0900 start was better. More information in advance on what would be covered. It was fine, but I did not know I’d be evaluating course/student work.

l. The exemplars & evaluation are a bit long. (May not be exact words, could not make out word after eval.)

m. It seemed less organized & structured than in the past. Participants & leaders were coming & going & it felt like a little of both were not taking it as being important

n. Better organization and pace. Maybe stricter assignments. o. (u on data sheet) work with discipline on specific event p. Agenda in advance. Structured breaks (during 2 sessions in separate rooms we had to

interrupt to get coffee.)

9. What questions remain about Core Abilities and assessment of student learning at Olympic College?

a. Getting campus involved b. ? c. What happens to our evaluations of student work? d. No response e. No response f. No response g. No response h. No response i. NA j. No response k. Will our evaluations of our courses be double checked by anyone l. No response m. No response n. No response o. No response p. Need more!! q. No response r. Are we going to be required to put it in the syllabus s. Several-will do workshops etc. t. No response

u. Other core abilities & criteria v. None w. None right now x. Was not familiar with core abilities. I have a better understanding.

10. Do you see a need for more workshops on course mapping with Gloria Martin? Circle one:

a. Yes 13 (maybe a video) b. No 8 b. Comment: I don’t know what this (course mapping) is. c. No selection but suggested: maybe a good review d. Comment: I am not familiar with course mapping & I need a class

11. Do you have suggestions for the timing of the Institutes? Rank in order of preference

a. One day 4, 1, 1, 4, 2, 1, 1, 1, 1 (Is that possible? (2 days of work zipped into a day?), 2, 4, 1

b. Two consecutive days 1, 1, 1, 1, 1, 1, 1, 3, 1, 3, 3 (circled option also), 1, 1 c. First non-contract day after end of quarter 3, 1, 3, 1, 1 or 2, 2 or 3, 4, 1, 1,1 d. Second non-contract day after end of quarter 2, 1, 1, 2, 1 or 2, 2 or 3), 1, 2, 1 (this person

also selected one day) added Thanks! , 1

Additional comments for question 11 from Evaluation Forms: -Winter institute provided opportunity to do assessment project (rather than just grading student work) -Three people selected two options without prioritizing -One, no answer -Two added and e option: after grades are due; one went on to select or c. -Additional comment: nice video…could be edited down to ½ the time -Not sure what this comment was in reference to but someone wrote: “8 standards running” -The pre-video helped to review. Thanks! -One person selected b, c, & d without prioritizing and stated open to when classes are given.

0

2

4

6

8

10

12

One day Two days 1st NC day 2nd NC day

Num

ber p

refe

rrin

g Institute Timing Preference

1st

2nd

3rd

4th