ACOTE Standard A.5 - famu.edu E - ACOTE Narrati… · admitted averaged a 3.6 cumulative GPA, a...

ACOTE Standard A.5.6 Exam Pass Rate

Narrative

The Master of Science Occupational Therapy program is submitting a response regarding an area of non-compliance with ACOTE standard A.5.6, which is related to performance on the NBCOT. The standard states “The average pass rate over the 3 most recent calendar years for graduates attempting the national certification exam within 12 months of graduation from the program must be 80% or higher (regardless of the number of attempts). If a program has less than 25 test takers in the 3 most recent calendar years, the program may include test takers from additional years until it reaches 25 or until the 5 most recent calendar years are included in the total”. The average pass rate for 2014-2016 was 75.76% (i.e. 50/66).

There has been much variability in terms of students' performance on the NBCOT. Pass rate range from 92% in 2014 to 68% in 2016. The decline in performance has resulted in a comprehensive review of the program to include identifying factors that contributed to low performance. To address pass rates, the program has developed an action plan that was informed by data analysis and findings from a comprehensive review of the program. The following narrative summarizes the results of data analysis and the actions taken to date to address pass rate. Future actions are outlined on the Plan of Correction submitted June 20th, 2017.

Factors Contributing to Low Pass Rate

Student Factors

The program analyzed data from past students admitted to the program with a particular emphasis on those who passed the NBCOT examination within one year of graduation. The variables of interest included students cumulative grade point average (GPA), prerequisite courses grade point average, Graduate Record Exam (GRE) scores, and the analytical writing section of the GRE.

Data analysis revealed two admissions factors that correlated to success on the NBCOT exam. The most significant was the prerequisite GPA followed by cumulative GRE score. Students with less than a 3.0 prerequisite GPA were less likely to successfully pass the NBCOT examination within one year of graduation.

Data collected from students who did not pass the licensure examination within one year of graduation also revealed varying perceptions associated with their performance. Following, are a sample of comments collected from students who did not pass the licensure examination in 2015-2016.

1. Did not take it until after six months due to financial reasons.

2. Personal issues including having a baby, cancer in the family, and having to send money to family in another country.

3. Had to go to work.

4. Got married.

5. Had trouble paying for the exam the first time. Did not pass, could not afford to take it a second time within the first year.

6. Took the exam three days after my one year graduation date and passed it the first time. Her results did not count in our 2015 pass rate. She cited finances as a reason she had to wait so long.

Actions based on Program Review and Assessment Findings

The admission criteria were changed for the 2016 and 2017 incoming cohort. The program requires a minimum cumulative GPA of 3.25, a prerequisite GPA of 3.0, a cumulative GRE score of 280 and an analytical writing score of 3.5. The last two cohorts admitted averaged a 3.6 cumulative GPA, a 3.21 prerequisite GPA, a 298 cumulative GRE score, and a 3.75 analytical writing score. The results from these cohorts will not be available until 2018-2019.

The program implemented the use of the Occupational Therapy Centralized Application Service (OTCAS) for the admission process. This has resulted in a larger pool of qualified candidates for the program.

Students must enroll in two online courses, one each semester of Level II fieldwork. The first course (i.e. Advanced Topics) has students self-assess their competencies, develop a study plan, and take the Occupational Therapy Knowledge Examination (OTKE). Each student is sent the results of the OTKE and the instructor sends a summary of what knowledge domains the student needs to further develop. Students are also provided additional resources to build and enhance their understanding of the competencies measured on the licensure exam. Resources include information from the NBCOT website, and the AOTA website. They are also referred to key textbooks for further reference.

Second semester online course provides sample questions, a breakdown of how to dissect a question and pick the best answer, and further supplemental resources. The students are given the OTKE two additional times, with the goal of having to pass it with a score of 70% or above to pass the course. Again, the instructor sends a summary of their individual results and areas to concentrate on in their preparation for the NBCOT exam. Results revealed that 100% of the spring 2017 graduates passed the OTKE with a score of 70% or better. The program requires students to engage in an intensive three-day test preparation course at the end of the semester prior to graduation.

Once graduated, students are encouraged to contact their professors to let them know of their progress in studying for the NBCOT and when they plan to take the exam. As of the time of this report, six students have taken the exam. Their results are currently not available.

Students are advised to take the NBCOT exam within three months of graduation. They are encouraged to take practice exams leading up to the time they sit for the NBCOT. Students can contact program faculty to receive practice test results and address any areas of concerns as measured by the test.

Curriculum

NBCOT performance results in 2015 prompted the program to review the program curriculum and course content. The faculty determined that some courses needed to be modified to enhance course content and strengthen their alignment to program outcomes and ACOTE standards. Faculty have attended faculty development courses, blackboard certification training, and continuing education to increase their course development and test writing skills.

All course content and outcome measures are assessed by program faculty in order to meet all ACOTE curriculum related standards. The NBCOT 2013 Matrix Study for the OTR was used to analyze the existing curriculum. Gaps in the curriculum were identified and appropriate courses modified to meet current standards. One gap noted was in mental health. Although mental health concepts are addressed early in the curriculum including in the Biopsychosocial II: Adults and Aging, program faculty determined that students would benefit from additional content related to the emerging practice of mental health and occupational therapy. As a result, the content was introduced in a Special Topics course and is being further developed to be a stand-alone mental health course.

The sequencing of courses in the curriculum was also assessed. The first-year courses begin with anatomy, neuroanatomy, and foundation courses which introduce theory and the Occupational Therapy Practice Framework. A conditions course was added earlier in the sequence to increase students’ knowledge of conditions seen across the life span. This change in sequence allows students in their second years to concentrate on assessment and intervention as a means of improving performance on the NBCOT.

Evidence-based practice and research were areas of concern on the NBCOT. Based on these findings, students are introduced to evidence-based practice and research the second semester of the first year. By their first semester second year, students are required to complete a research project and submit it for presentation at the state and/or national occupational therapy conference. This year, one student presentation was voted the best research project at the Florida Occupational Therapy Association annual conference. All students were selected to present at that conference. Eight student research projects were accepted for presentation at the American Occupational Therapy Association annual conference. One group was chosen for the young scientist honor.

The faculty reevaluated the textbooks used in courses. Seven of the ten top textbooks used by NBCOT for item questions are required textbooks in the program. Faculty regularly receive and review occupational therapy textbooks and update textbooks

when appropriate to ensure students are reading the most recent and valid literature in occupational therapy.

Faculty

The number of faculty has recently increased from four to five full time equivalents. A new Academic Fieldwork Coordinator, who had worked as an adjunct professor, was hired. There has been a reduction in the number of classes taught by individual faculty.

The program has recently hired a new adjunct faculty who has prior experience teaching in a MSOT program. The program continues to search for qualified adjunct faculty to assist in teaching.

Summary

The program has taken the decline in the pass rate very seriously and have taken steps to analyze and rectify the problems which may be impacting a student’s ability to be successful on the NBCOT. The program and faculty are confident that with the changes in admission criteria, curriculum mapping and sequencing, course content, and additional faculty, the success of the graduating students will be realized in the next two years.

PROGRAM EVALUATION PLAN FLORIDA A&M UNIVERSITY 2016-2017

Submitted June 20, 2017

ACOTE Standard

Related Program Goal / Outcomes

Assessment Tool

Measurement Criteria Actual Results (2016-2017)

Use of Results for Continuous Improvement (Closing the Loop)

Review Period Review Process Who analyzes and reviews data

A 5.3 - Faculty Effectiveness

Faculty will employ best practices in pedagogy in support of student learning within the field. Students will demonstrate proficiency in course content. Faculty will demonstrate effectiveness in teaching.

State University System Student Assessment of Instruction (SUSSAI).

95% of faculty will be rated good or better for courses taught in the fall and spring.

100% (n=5) faculty were rated as good or better for courses taught in fall and spring. Analysis of open ended responses resulted in the following recommendations: • Addition of more

NBCOT type questions throughout courses.

• Addition of more applied or “hands on” (active learning) experiences.

In response to student feedback, TOP HAT, a program that facilitates the incorporation of NBCOT type questions in courses and lectures, was initiated summer 2017 in the functional human motion. In fall 2016, all lecture courses in the MSOT curriculum increased active learning activities.

Fall and Spring semesters.

Program Director and faculty.

Students’ course final grades.

80% of students will receive a B or better in all MSOT courses.

89% (n=25) Year 2 MSOT students received a B or better in all MSOT courses for 2016-2017.

While the goal was met, students who did not receive a B or better in any MSOT course received individualized advisement to strengthen study habits and skills. Individual course instructors are responsible for providing continuous feedback regarding student performance.

Annual faculty evaluations.

At least 90% of faculty will receive an average rating of 3.5

All faculty (n=5) received an average score of 4 out of 5 on

While the target was met, the program will implement peer evaluation of teaching beginning fall 2017 in an effort to

2

ACOTE Standard


Assessment Tool




out of 5 on the teaching effectiveness component of the faculty evaluation form.

the teaching effectiveness component of the annual faculty evaluation.

enhance teaching effectiveness. Program will continue to monitor this outcome.

A 5.3 - Student progression

Students will successfully matriculate through the program.

Progression audit.

95% graduation rate per cohort.

96% (n=27) students in the cohort graduated Spring 2017.

While the target was met, students requested more clarification regarding progression through the program. This resulted in modifications in the student handbook and fieldwork manual.

Annually The Program Director and OT faculty reviews and analyzes the data.

95% of students will maintain a 3.0 or better throughout their matriculation in the program.

99% (n=80) students achieved a 3.0 or better for academic year 2016-2017. Students must maintain a cumulative grade point average of a 3.0. They must repeat a D grade and only can do this one time. Students who are not successfully matriculating through the program are required to develop a progression plan with their academic advisor.

While the target was met, an individualized program of study was developed for students that did not maintain a 3.0 GPA. For the one student that did not meet this goal, an individualized progression and study plan was completed by the student and the Program Director.

Every semester

Student advisement tracking log.

100% of students will be advised once a semester by OT faculty.

All students (n=81) were advised at minimum once a semester by OT faculty members over the course of 2016-2017 academic year.

While the target was met, the student advisement tracking log was revised and is scheduled to be implemented in Fall 2017 to increase consistency among faculty in advisement.

Academic year (summer through fall).

Program Director and all faculty.

3

ACOTE Standard


Assessment Tool




A 5.3 - Student Retention


Student progress report.

95% of MSOT students will be retained in the program.

100% (n=81) of MSOT students were retained for the 2016-2017 academic year.

While 100% of students were retained, data analysis revealed that two students were having difficulty in didactic course work, and received advisement. Moreover, any students experiencing difficulty in courses or fieldwork are required to see their instructor or advisor.

Midterm of every semester.

The Program Director and all faculty.

A 5.3 - Fieldwork and experiential component performance evaluation

Students will successfully complete Fieldwork I and II. Students will demonstrate competency in assessment and intervention at the end of Fieldwork Level II.

The Fieldwork I Evaluation for Fieldwork Level I.

100% of students will be rated at satisfactory or better for Fieldwork Level I.

All (n=28) students met the target for Fieldwork Level I for academic year 2016-2017.

While 100% of students met the target for Level 1, the program will continue to monitor student performance in Fieldwork Level I.

Every semester students are engaged in either Fieldwork Level I or Fieldwork Level II starting in the second year of the program. Student progress is evaluated by assignments or evaluations at midterm and final evaluation as needed.

The Academic Fieldwork Coordinator. The AFWC reports fieldwork issues to faculty at faculty meetings.

Fieldwork Performance Evaluation for Occupational Therapy Student for Fieldwork Level II.

100% of students will score a 3 out of 4 or better indicating that they met the minimum standard for Fieldwork Level II.

96% (n=27) students met the target for Fieldwork Level II for academic year 2016-2017.

For the one student that did not meet the standard for Level II, the Academic Fieldwork Coordinator presented student performance results to the faculty. Consistent issues (such as lack of preparedness for the site, competency issues) triggered a plan of progression in conjunction with the fieldwork educator, academic fieldwork coordinator and student.

80% of students will score a 3 out of 4 or better on the evaluation items of the Fieldwork Performance Evaluation for the OT student.

83% (n=23) students scored a 3 out of 4 or better on the evaluation items of the Fieldwork Performance Evaluation.

While the target was met, the program will continue to utilize the evaluation to enhance didactic classes by using common and standardized evaluation tools. The program will also continue to monitor student performance on the evaluation

At the end of each Fieldwork Level II rotation.

The Academic Fieldwork Coordinator. The results are shared with faculty at faculty

4

ACOTE Standard


Assessment Tool




items on the Fieldwork Performance Evaluation.

meetings.

80% of students will score a 3 out of 4 or better on the intervention items of the Fieldwork Performance Evaluation.

87% (n=24) students scored a 3 out of 4 or better on the intervention items on the Fieldwork Performance Evaluation.

While the target was met, the program will continue to utilize the evaluation to enhance didactic classes by using evidence based interventions. The program will also continue to monitor student performance on the intervention items on the Fieldwork Performance Evaluation.

A 5.3 - Student Evaluation of Fieldwork Experience

Improve student satisfaction and overall experience in fieldwork.

Students Evaluation of Fieldwork Experience.

80% of students will rate the Educational Opportunities using “Aspects of the environment” at 3 out of 4 or better.

61% (n=17) students rated educational opportunities at 3 out of 4 or better.

Results revealed that students were seeking more educational/ clinical opportunities at existing sites. Interventions made by the program included additional collaborations with sites using the site-specific objectives tool, to identify additional opportunities that can be offered at sites. Further interventions were made to include a yearly Fieldwork Educator Certification Workshop.

Each Fieldwork II Rotation.

Academic Fieldwork Coordinator & Program Director and faculty.

100% of students will rate their FWE on qualities of fieldwork educator at 4 out of 5 (agree) or better.

100% (n=28) students rated their FWE at 4 out of 5 (agree) or better.

The program will continue to monitor students’ ratings of the FWE.

5

ACOTE Standard


Assessment Tool




Blackboard Discussion Boards – Weekly prompts.

100% of students will participate in at least 90% of Discussion boards.

100% (n=28) students participated in at least 95% of Discussion Boards. Qualitative analysis revealed themes that indicated students were progressing adequately with completing evaluations and interventions; used occupation-based interventions and developed organizational and coping skills.

The program will continue to monitor students’ participation in Discussion Board weekly prompts.

Each Fieldwork Level II Rotation.

Academic Fieldwork Coordinator and Program Director and faculty.

Electronic Journal Entries and Reflections in Blackboard.

100% of students enrolled in Fieldwork IIA&B will complete 90% of the weekly journal entries reflecting on strengths and areas of growth using specific examples, as well as the relationship with Fieldwork Coordinator.

100% (n=28) students completed 90% of the journal entries. The results revealed that students wanted earlier exposure to standardized and common assessment tools.

While the target was met, and in response to students’ feedback, the program will implement added content in standardized and common assessment tools beginning with the admitted 2017 fall cohort.

Journals reviewed weekly by the Academic Fieldwork Coordinator.

Academic Fieldwork Coordinator, Faculty and Program Director.

6

ACOTE Standard


Assessment Tool




A 5.3 - Student satisfaction with the program

Graduates will be satisfied with their overall experiences in the program.

Exit interviews from the Division of Occupational Therapy.

80% of graduates will rate the program as satisfactory or better.

82% (n=22) graduates rated the program as satisfactory or better.

While the goal was met, results from both the exit interviews and University Graduate Exit Survey were utilized to improve the curriculum. Following is a sample of recommendations from the survey:

• Students requested more exposure to clients in the first year. In response to student feedback, OTH 5206 will be revised to provide students more exposure to clients in their first semester fall 2017.

• Students requested more applied experiences in classes. In response, the curriculum will be changed to allow for experiential activities in the first year of the program. Further, more applied activities will be included in lecture classes.

Annual review of exit surveys and interviews.

The Program Director and Occupational Therapy Faculty.

University Graduate Exit Survey.

80% of MSOT graduates completing the University Graduate Exit survey will strongly agree that they obtained a sense of competence in their field of study.

82% (n=22) graduates strongly agreed they obtained a sense of competence in their study.

A 5.3 - Graduates’ performance on the NBCOT certification exam.

Graduates from the MSOT program will pass the NBCOT certification examination within one year of graduation.

OTKE Pass rate.

100% of students will pass the Occupational Therapy Knowledge Exam (or equivalent) prior to graduation.

100% (n=28) students passed the OTKE in Spring 2017.

While 100% of students passed the OTKE, students demonstrated difficulty passing the NBCOT exam, thus the program made the following changes:

• Reviewed individual student profiles and developed an individualized action plan to improve student success on the NBCOT exam.

• The results from the OTKE were utilized to revise the curriculum.

Implemented changes based on results: Revised course sequencing and began curriculum mapping.

Each fall and spring semester.

Advanced Topics and Professional Development II course instructors and program faculty.

7

ACOTE Standard


Assessment Tool




Changes to be implemented fall 2017:

• Change the measurement tool from the OTKE to the NBCOT Practice Test.

• Utilize the final four weeks in the program for intensive review of examination content.

• Hire consultant for program development to include test strategy and use of analysis for information gathered.

NBCOT pass rate results.

80% of MSOT students will pass the NBCOT certification examination within one year of graduation.

2016 pass rate: 68% Implemented changes based on results: Performed an in-depth analysis of students who took the NBCOT exam. An analysis of student profiles indicated that pre-requisite GPA and GRE scores were strong predictors of success on the NBCOT exam. Based on these findings, the program revised the admission criteria beginning fall 2016. An action plan was developed to include the following and will be implemented fall 2017:

• Continue curriculum mapping to identify overlap and gaps in addressing the accreditation standards.

• Review all syllabi for course content and exam alignment.

• Review all outcome measures for courses to determine strengths and weaknesses in course content and exam alignment.

• Compile and analyze student admission profile data for 2015,

Systematically check the NBCOT website each month.

The Program Director.

8

ACOTE Standard


Assessment Tool




2016, and 2017 • Review course outcome measures

to optimize synthesis of program content.

A.5.3 Job Placement

Graduates of the program will find gainful employment as an occupational therapist.

Email survey to graduates asking for job placements.

75% of graduates will find gainful employment as an occupational therapist within one year of graduation.

2016: 71% (n=5) graduates employed as occupational therapists. The two who are not employed worked as occupational therapists under a temporary license. They did not pass the NBCOT exam and lost their positions.

For the two students who did not pass the NBCOT exam, faculty are in regular communication with the graduates providing information on resources to facilitate improvements on the NBCOT exam. Examples include:

• Commercially available test preparatory resources.

• Mobile Apps • Peer mentorship • Faculty mentorship

Continue to monitor graduate job placement.

Continuous and ongoing as appropriate per student.


Florida Education and Training Placement Information Program (FETPIP) data.

2014-2015: 89% (n=24) of graduates were employed within one year of graduation based on the most recent FETPIP data.

A 5.3 – Employer satisfaction

Employers of graduates of the MSOT program will be satisfied with their performance.

The Program Employer Satisfaction survey.

90% of MSOT employers surveyed will rate graduate’s performance as satisfactory or better.

100% (n=10) of the employers surveyed rated FAMU MSOT graduates performance as satisfactory or better.

Continue to monitor employer satisfaction of the FAMU MSOT graduates.

Yearly distribution of employee satisfaction survey.


of 3

FLORIDA A&M UNIVERSITY

FACULTY EVALUATION FORM

FROM (College/School)

(Division)

(Department)

FACULTY

MEMBER

EVALUATION

PERIOD

PART A. Teaching Effectiveness: Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Organization and planning of courses.

0 1 2 3 4 5

2. Evidence of careful preparation documented by

current course outlines, syllabi, or other

appropriate methods distributed at the beginning of class meetings.

0 1 2 3 4 5

3. Ability to make course(s) challenging, inspire interest and thought in subject matter, and

encourage students to understand all phases of the

course(s).

0 1 2 3 4 5

4. Quality of scholarship and professionalism;

mastery and currency in principal subject area and

related disciplines.

0 1 2 3 4 5

5. Ability to maintain professional classroom

decorum, and to project an atmosphere of friendliness and concerns for students.

0 1 2 3 4 5

6. Punctuality with respect to classroom attendance,

adherence to announced schedule; grading of tests and assignment; and submitting required reports.

0 1 2 3 4 5

7. Accessibility and openness to questions; maintenance of office hours and availability for

student consultation.

0 1 2 3 4 5

8. Appropriateness, relevance and fairness of

examinations or student evaluations.

0 1 2 3 4 5

COMMENTS: Immediate Supervisor:

Faculty Member:

Revised – 5/80

of 3

PART B. Research and Creative Ability Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Evidence of presentation of papers or creative

works at professional meetings/societies or

exhibitions.

0 1 2 3 4 5

2. Evidence of papers or articles published in

professional journals and/or of books published.

0 1 2 3 4 5

3. Evidence of recognized creative work in his/her

field such as musical compositions, paintings,

product design, sculptures, dance, theatre, etc.

0 1 2 3 4 5

4. Extent of on-going research or creative endeavors.

0 1 2 3 4 5

5. Extent of development of research projects or

submission of proposals relative to research or

creative ability.

0 1 2 3 4 5


Faculty Member:

PART C. Performance in Service Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Performance on departmental college/school, and

university-wide committees, councils and task

forces.

0 1 2 3 4 5

2. Contribution to faculty meetings and regular

program activities.

0 1 2 3 4 5

3. Membership in appropriate professional

organizations and contributions to these.

0 1 2 3 4 5

4. Involvement in activities leading to professional

development, continuing education, certification,

or licensure.

0 1 2 3 4 5

5. Performance in community activities on a local,

state, national or international level.

0 1 2 3 4 5


Faculty Member:

of 3

PART D. Other University Duties Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Performance as an academic advisor.

0 1 2 3 4 5

2. Performance as a counselor/advisor to clubs or organizations, direction of interns, or other

university related groups.

0 1 2 3 4 5

3. Performance in administrative duties.

0 1 2 3 4 5


Faculty Member:

SIGNATURES AND GENERAL COMMENTS

Signature of

Department

Chairperson:

Date

*Signature of

Faculty Member:

Date

Signature of

Division Director:

Date

Signature of

Dean:

Date

*I HAVE HAD A CONFERENCE WITH MY SUPERVISOR AND HAVE BEEN DULY APPRISED OF

THIS EVALUATION.

1

MSOT Student Advisement Log

Log of Meetings: Semester/Year

# Date: Time Name SID Year Response 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

SSTTUUDDEENNTT EEVVAALLUUAATTIIOONN OOFF TTHHEE FFIIEELLDDWWOORRKK EEXXPPEERRIIEENNCCEE ((SSEEFFWWEE))

Purpose:

This evaluation serves as a tool for fieldwork sites, academic programs, and students. The main

objectives of this evaluation are to:

Enable the Level II fieldwork student who is completing a placement at the site to

evaluate and provide feedback to the supervisor and fieldwork setting;

Enable academic programs, fieldwork sites, and fieldwork educators to benefit from

student feedback in order to develop and refine their Level II fieldwork programs;

Ensure that all aspects of the fieldwork program reflect the sequence, depth, focus, and

scope of content of the curriculum design;

Provide objective information to students who are selecting sites for future Level II

fieldwork; and

Provide a means of evaluation to ensure that fieldwork is performed in settings that

provide educational experiences applicable to the academic program.

This form is designed to offer each program the opportunity to gather meaningful and useful

information. Sections outlined with thick black double borders are designed to be customized by

your program as needed. Pages involving evaluation of individual fieldwork educators have been

positioned at the end of the form to allow academic programs to easily remove these pages before

making them available for student review, if they choose to do so.

1


Instructions to the Student:

Complete this STUDENT EVALUATION OF THE FIELDWORK EXPERIENCE (SEFWE)

form before your final meeting with your fieldwork supervisor(s). It is imperative that you

review the form with your supervisor and that both parties sign on page 1. Copy the form

so that a copy remains at the site and a copy is forwarded to your Academic Fieldwork

Coordinator at your educational program. This information may be reviewed by future

students as well. The evaluation of the student (FWPE) should be reviewed first,

followed by the student’s evaluation of the fieldwork experience (SEFWE), allowing the

student to be honest and constructive.

Fieldwork Site _____________________________________ Site Code ________

Address ___________________________________________________________

Placement Dates: from _________________________ to _______________________

Order of Placement: [ ] First [ ] Second [ ] Third [ ] Fourth

Living Accommodations: (include type, cost, location, condition)

Public transportation in the area:

Please write your e-mail address here if you don’t mind future students contacting you to

ask you about your experience at this site: -

______________________________________

We have mutually shared and clarified this Student Evaluation of the Fieldwork

Experience report. _______________________________________ ________________________________________ Student's Signature FW Educator's Signature _______________________________________ ________________________________________ Student's Name (Please Print) FW Educator’s Name and credentials (Please Print) FW Educator’s years of experience ____________

2

ORIENTATION Indicate your view of the orientation by checking "Satisfactory" (S) or "Needs Improvement” (I) regarding the three factors of adequacy, organization, and timeliness.

TOPIC Adequate Organized Timely NA

S I S I S I

1. Site-specific fieldwork objectives

2. Student supervision process

3. Requirements/assignments for students

4. Student schedule (daily/weekly/monthly)

5. Staff introductions

6. Overview of physical facilities

7. Agency/Department mission

8. Overview of organizational structure

9. Services provided by the agency

10. Agency/Department policies and procedures

11. Role of other team members

12. Documentation procedures

13. Safety and emergency procedures

14. Confidentiality/HIPAA

15. OSHA—Standard precautions

16. Community resources for service recipients

17. Department model of practice

18. Role of occupational therapy services

19. Methods for evaluating OT services

20. Other

Comments or suggestions regarding your orientation to this fieldwork placement: __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

CASELOAD

List approximate number of each age List approximate number of each primary category in your caseload. condition/problem/diagnosis in your caseload

Age Number Condition/Problem Number

0–3 years old

3–5 years old

6–12 years old

13–21 years old

22–65 years old

> 65 years old

3

OCCUPATIONAL THERAPY PROCESS Indicate the approximate number of screenings/evaluations you did; also indicate their value to your learning experience by circling the appropriate number with #1 being least valuable and #5 being the most valuable.

REQUIRED Yes No

HOW MANY

EDUCATIONAL VALUE

1. Client/patient screening 1 2 3 4 5

2. Client/patient evaluations (Use specific names of evaluations)

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

3. Written treatment/care plans 1 2 3 4 5

4. Discharge summary 1 2 3 4 5

List major therapeutic interventions frequently used and indicate whether it was provided in group, individually, Co-Treatment, or consultation. List other professionals involved.

Therapeutic Interventions Individual Group Co-Tx Consultation

Occupation-based activity, i.e., play, shopping, ADL, IADL, work, school activities, etc. (within client’s own context with his or her goals)

1.

2.

3.

4.

Purposeful activity (therapeutic context leading to occupation)

1.

2.

3.

4

4.

Preparatory methods, i.e., sensory, PAMs, splinting, exercise, etc. (preparation for occupation-based activity)

1.

2.

3.

4.

THEORY—FRAMES OF REFERENCE—MODELS OF PRACTICE Indicate frequency of theory/frames of reference used

Never Rarely Occasionally Frequently

Model of Human Occupation Occupational Adaptation Ecology of Human Performance Person–Environment–Occupation Model Biomechanical Frame of Reference Rehabilitation Frame of Reference Neurodevelopmental Theory Sensory Integration Behaviorism Cognitive Theory Cognitive Disability Frame of Reference Motor Learning Frame of Reference Other (list)

FIELDWORK ASSIGNMENTS

List the types of assignments required of you at this placement (check all that apply), and indicate their educational value (1 = not valuable ------- 5 = very valuable)

Case study applying the Practice Framework 1 2 3 4 5 N/A

Evidence-based practice presentation:

Topic:

1 2 3 4 5 N/A

Revision of site-specific fieldwork objectives 1 2 3 4 5 N/A

Program development

Topic:

1 2 3 4 5 N/A

In-service/presentation

Topic:

1 2 3 4 5 N/A

5

Research

Topic:

1 2 3 4 5 N/A

Other (list) 1 2 3 4 5

ASPECTS OF THE ENVIRONMENT

1 = Rarely

2 = Occasionally

3 = Frequently

4 = Consistently

1 2 3 4

Staff and administration demonstrated cultural sensitivity

The Practice Framework was integrated into practice

Student work area/supplies/equipment were adequate

Opportunities to collaborate with and/or supervise OTs, OTAs, and/or aides

Opportunities to network with other professionals

Opportunities to interact with other OT students

Opportunities to interact with students from other disciplines

Staff used a team approach to care

Opportunities to observe role modeling of therapeutic relationships

Opportunities to expand knowledge of community resources

Opportunities to participate in research

Additional educational opportunities (specify):

How would you describe the pace of this setting? (circle one) Slow

Med Fast

Types of documentation used in this setting:

Ending student caseload expectation: _____ # of clients per week or day

Ending student productivity expectation: _____ % per day (direct care)

SUPERVISION

What was the primary model of supervision used? (check one) one supervisor : one student one supervisor : group of students two supervisors : one student one supervisor : two students distant supervision (primarily off-site) three or more supervisors : one student (count person as supervisor if supervision occurred at least

weekly)

List fieldwork educators who participated in your learning experience.

Name Credentials Frequency Individual Group

1.

2.

3.

4.

7

ACADEMIC PREPARATION Rate the relevance and adequacy of your academic coursework relative to the needs of THIS fieldwork placement, circling the appropriate number. (Note: may attach own course number)

Adequacy for Placement Relevance for Placement

Anatomy and Kinesiology

Low High 1 2 3 4 5

Low High 1 2 3 4 5

Neurodevelopment 1 2 3 4 5 1 2 3 4 5

Human development 1 2 3 4 5 1 2 3 4 5

Evaluation 1 2 3 4 5 1 2 3 4 5

Intervention planning 1 2 3 4 5 1 2 3 4 5

Interventions (individual, group, activities, methods)

1 2 3 4 5 1 2 3 4 5

Theory 1 2 3 4 5 1 2 3 4 5

Documentation skills 1 2 3 4 5 1 2 3 4 5

Leadership 1 2 3 4 5 1 2 3 4 5

Professional behavior and communication 1 2 3 4 5 1 2 3 4 5

Therapeutic use of self 1 2 3 4 5 1 2 3 4 5

Level I fieldwork 1 2 3 4 5 1 2 3 4 5

Program development 1 2 3 4 5 1 2 3 4 5

What were the strongest aspects of your academic program relevant to preparing you for THIS Level II fieldwork experience? Indicate your top 5.

Informatics Occ. as Life Org A & K Foundations Level I FW

Pathology Neuro Administration Theory Peds electives

Env. Competence Research courses Prog design/eval Consult/collab Older adult elect.

Interventions Evaluations Adapting Env Human comp. Community elect.

Social Roles History Occupational Sci Other:

What changes would you recommend in your academic program relative to the needs of THIS Level II fieldwork experience?

___________________________________________________________________________________________________________________________________________________________________________

SUMMARY

1 = Strongly disagree 2 = Disagree 3 = No Opinion 4 = Agree 5 = Strongly agree

1 2 3 4 5

Expectations of fieldwork experience were clearly defined

Expectations were challenging but not overwhelming

Experiences supported student's professional development

8

Experiences matched student's expectations

What particular qualities or personal performance skills do you feel that a student should have to function successfully on this fieldwork placement?

_________________________________________________________

__________________________________________________________________________________________________________________ What advice do you have for future students who wish to prepare for this placement?

Study the following evaluations:

___________________________________________________________________________________________________________________________________________________________________________

Study the following intervention methods:

___________________________________________________________________________________________________________________________________________________________________________

Read up on the following in advance:

___________________________________________________________________________________________________________________________________________________________________________ Overall, what changes would you recommend in this Level II fieldwork experience?

_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Please feel free to add any further comments, descriptions, or information concerning your fieldwork at this center.

_____________________________________________________________

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

9

Indicate the number that seems descriptive of each fieldwork educator. Please make a copy of this page for each individual.

FIELDWORK EDUCATOR

NAME:___________________________________

FIELDWORK EDUCATOR YEARS OF EXPERIENCE: __________

1 = Strongly Disagree

2 = Disagree

3 = No opinion

4 = Agree

5 = Strongly agree

1 2 3 4 5

Provided ongoing positive feedback in a timely manner

Provided ongoing constructive feedback in a timely manner

Reviewed written work in a timely manner

Made specific suggestions to student to improve performance

Provided clear performance expectations

Sequenced learning experiences to grade progression

Used a variety of instructional strategies

Taught knowledge and skills to facilitate learning and challenge student

Identified resources to promote student development

Presented clear explanations

Facilitated student’s clinical reasoning

Used a variety of supervisory approaches to facilitate student performance

Elicited and responded to student feedback and concerns

Adjusted responsibilities to facilitate student's growth

Supervision changed as fieldwork progressed

Provided a positive role model of professional behavior in practice

Modeled and encouraged occupation-based practice

Modeled and encouraged client-centered practice

Modeled and encouraged evidence-based practice

Frequency of meetings/types of meetings with supervisor (value/frequency):

_________________________________________________________________

_________________________________________________________________

_________________________________________________________________

_________________________________________________________________

____

General comments on supervision:

of 18 FLOTEC, Level II Fieldwork Site Specific Objectives Checklist

Adapted from the New England Occupational Therapy Education Council, Inc. (NEOTEC). 2010

If you see a tool bar stating SOME MACROS HAVE BEEN DISABLED, you will need to click on enable to complete. When saving the document pleases

change the title of the document to reflect your facilities name.

Level II Fieldwork Site Specific Objectives Checklist for use with:

The AOTA Fieldwork Performance Evaluation (for the Occupational Therapy Student)

Prepared by:

FLORIDA OCCUPATIONAL THERAPY EDUCATIONAL CONSORTIUM (FLOTEC)

Site: Date:

Fieldwork Educator/Contact Person:

Phone #: Fax #:

E-mail:

Please check one:

As a fieldwork site for students from we currently utilize the goals as stated in the program fieldwork handbook and do not have additional site specific goals.

As a fieldwork site for students from we currently utilize the goal as stated in the

program’s fieldwork handbook as well as the following site specific goals:

I. FUNDAMENTALS OF PRACTICE

FWPE item # 1: Adheres consistently to AOTA Code of Ethics & Ethics Standards (AOTA 2010, draft) & site’s policies &

procedures; including, when relevant, those related to human subject research as stated below:

Demonstrates concern for well-being & safety of recipients of services (beneficence)

Intentionally refrains from actions that cause harm (nonmaleficence)

Respects right of individual to self rule (autonomy, confidentiality)

Provides services in fair & equitable manner (social justice)

Complies with institutional rules, local, state, federal, international laws & AOTA documents applicable to profession of occupational therapy (procedural justice)

Provides comprehensive, accurate, & objective information when representing profession (veracity)

Treats colleagues & other professionals with respect, fairness, discretion, & integrity (fidelity)

Other:



FWPE item #2: Adheres consistently to safety regulations. Anticipates potentially hazardous situations & takes steps to

prevent accidents for clients & staff members throughout fieldwork related activities including:

Record review

Medication side effects

Post-surgical

Infection control

Fall prevention

Swallowing

Food allergies

Ambulation status

Behavioral system/privilege level (e.g. locked area/unit, on grounds)

1:1 for personal safety/suicide precautions

Sharps count

Environment set up (no clutter, spills, unsafe items, etc.)

OSHA/BBP

I.V./ Lines

ER codes/protocols

Restraint reduction

HIPAA

W/C locks/bedrails/call button

Vital signs (BP, O2)

Trach/Ventilator monitoring

Fire/Evacuation/Lockdown

CPR certification

Communication re: change in status

Other:

FWPE item #3: Uses sound judgment in regard to safety of self & others during all fieldwork related activities:

Adheres to facility policies & procedures

Thorough chart reviews/checks MD orders/parent agreement for IEP

Consistently analyzes space for potential hazards based on client risk factors

Addresses anticipated safety concerns

Provides safe supervision of client based on client status

Accurately identifies ambulation needs/functional mobility status

Uses safe transfer techniques/equipment according to protocols

Determines wheelchair positioning needs (e.g. footrests, cushions, trays/supports, etc.)

Correctly positions client (e.g. in chair/bed; at desk, for feeding, etc)

Provides supervision of client based on client status to ensure safety

Demonstrates proper splinting techniques such as

Correct selection of type

Correct selection of materials

Making adjustments as needed



FWPE item #3 (con’t): Uses sound judgment in regard to safety of self & others during all fieldwork related activities:

Operates equipment according to training protocols

Attends to professional boundaries in therapeutic use of self-disclosure

Effectively limit sets & redirects client (s)

Establishes safe group climate (reinforce expectations/group rules or contract)

Other:

II. BASIC TENETS

FWPE items #4 -6: Clearly, confidently, & accurately communicates values/beliefs of occupational therapy profession,

occupation as method/outcome, roles of OT/OTA as collaborative team appropriate to setting, using examples & language

consistent w/ OTPF:

Verbally

Via written material (e.g., handout, article, sample job description, etc.)

Communicates about these 3 tenets with:

Client

Families/ significant others

OTA

OT

OTA

SLP

Teacher

Aides

MDs

Regulatory bodies

General public (e.g., promotional materials, in-services)

Nursing

LISCW

Psychologist

CRTS

CRC

AT

MT

3rd Party payers

Others:

FWPE items #4-6(cont): Clearly, confidently, & accurately communicates values/beliefs of occupational therapy profession, occupation as method/outcome, roles of OT/OTA as collaborative team appropriate to setting,



using examples & language consistent w/ OTPF:

Communicates about these 3 tenets in:

Client intervention/education

Meetings

Documentation/correspondence

In-services, brochures, bulletin boards,

media announcements, etc.

Other:

Communicates re: occupation using:

Examples of occupation-based assessment tools

Citations of literature/evidence base for use of occupation relative to person/context

Terms & examples specific to person, organization, population (facility mission/level of care/services delivery)

OTPF language (revised) verbally & in written work

Current AOTA official documents/fact sheets

Other:

Communicates re: OT/OTA roles using:

Current AOTA official documents

Federal & state laws/practice acts governing evaluation/intervention

State laws/practice acts re: role of OT/OTA

Other:

FWPE item #7: Effectively collaborates with clients, family/significant others throughout occupational therapy process

(evaluation, intervention, outcome):

Seeks & responds to client feedback

Maintains client focus in sessions

Respectfully engages in discussion when conflict arises to address concerns

Provides written documentation of collaborative plan (e.g., home program)

Incorporates client/family priorities & interests

Tailors client/family education to individual need

Other :

III. EVALUATION AND SCREENING



FWPE item #8: Articulates clear & logical rationale for evaluation process:

Describes reasoning based on client, condition, context, FOR/EBP

Explains choice of occupation-based &/or client factors

Discuss psychometric properties (validity & reliability) of assessment tool

Other:

FWPE item #9: Selects relevant screening/assessment*methods (*see assessment chart):

Selects assessments according to:

Future context

Evidence

Psychometric properties/validity/reliability

Client condition

Client priorities

Current context

Selects assessment based on Theories/Frames of reference pertinent to setting such as:

PEOP

Biomechanical

Acquisitional

Psychodynamic

Cognitive behavioral

DBT

Sensory Processing

Sensory Integrative

NDT

Functional Group Model

MOHO

Developmental

Motor Learning

Coping

Clinical Reasoning

Rehabilitation

Occupational Adaptation

Ecology of Human Performance

Cognitive/Cognitive Disability

Other:

FWPE item #10: Determines occupational profile & performance through appropriate assessment methods (see assessment

chart on last page for specific tools/competency expectations)



FWPE item #11: Assesses client factors & context that support or hinder occupational performance (see assessment chart for

specific tools/competency expectations)

FWPE item #12: Obtains sufficient/necessary information from relevant resources such as client, families, significant others, service providers, & records prior to & during evaluation process via:

Thorough record/chart review

Client interview

Observation of client performance in areas of occupation (ADL/IADL, education, work, play, leisure, social participation, rest/sleep)

Assessment instruments addressing occupational performance (see assessment chart)

Observation of client person performance skills (motor & praxis, emotional regulation, cognitive, communication/social, sensory/perceptual)

Assessment instruments addressing client performance skills

Observation of client performance patterns (roles, routines, rituals, habits)

Assessment instruments addressing client performance patterns (see assessment chart)

Assess of client factors (see assessment chart)

Observation in current context(s) (personal, physical/environment, social, cultural, temporal, virtual)

Gathering information re: anticipated future context(s)

Gathering input from family/significant others/service providers (PCA, nursing, teachers, team members, referral source)

Discussion of psychosocial factors that effect performance/disposition (e.g., motivation, adjustment, anxiety, self-concept, participation, etc.,)

Assessment instruments that address psychosocial factors that affect performance/disposition (see assessment chart)

Occupational profile addresses

Who is client?

Client problems

Client values/interests/needs

Why seeking services?

Other (s):

Occupational history

Client successes/ strengths

Priorities

Influence of environment/context

FWPE item #13: Administers assessments in uniform manner to ensure valid/reliable results:



Adheres to assessment tool protocols/procedures (format, script, item use, scoring, etc.)

FWPE item #14: Adjusts/modifies assessment procedures based on client’s needs, behaviors, and cultural variables such as:

Fatigue

O2 sat/respiration rate

BP/heart rate

Frustration tolerance

Anxiety

Cultural beliefs, values, customs, expectations

Cognitive status

Pain

Language

Attention

Inability to perform task

Refusal

Vision Acuity

Hearing Acuity

Concerns: safety (please specify):

Other:

FWPE item #15: Interprets evaluation results to determine client’s occupational performance strengths & challenges by

integrating quantitative & qualitative information such as:

Standardized assessment results

Information re: client condition/dx

Subjective/objective impressions

Verbal reports of others (team, family/caretakers, etc.)

Observation of client’s performance

Client’s stated values, beliefs/motivations

Identified problems/needs

Other:

FWPE item #16: establishes accurate & appropriate plan based on evaluation results, integrating factors such as client’s

priorities, context(s), theories & evidence-based practice:



Integrates information with client priorities to create plan relative to setting/scope of practice

Incorporates client’s present and future context(s) (personal, cultural, temporal, virtual, physical, social) in clinical reasoning/intervention planning

Utilizes summarized evidence from Critically Appraised Topics/Papers (CATs/CAPs) to guide decision-making/reasoning (http://www.otcats.com.index.html )

Uses EBP approach (e.g., PICO question: Person, Intervention, Comparison, Outcome) to search for/find relevant evidence according to client priorities & frame of reference

Critically appraises findings (e.g., CAT: http://www.otcats.com/template/index.html ; or CanChild (www.canchild.ca/en/canchildresources/educationalmaterials.asp#CriticalReciew )

Uses structures method to review evidence (journals, case studies, consensus of experts)

Creates realistic plan reflective of accurate understanding of client abilities and potential

Sets goals consistent with client priorities, theory/frame of reference, evidence & setting

Other:

FWPE item #17: documents results of evaluation process in manner that demonstrates objective measurement of client’s

occupational performance:

Records observed performance in areas of occupation (ADL/IADL, education, work, play, leisure, social participation, rest/sleep) as per setting’s policies & procedures/scope of practice

Accurately reports standardized assessment data (raw scores/results) as applicable

Formulates goals that are specific, measureable, realistic, attainable, time-limited

Utilizes outcome measurement methods when available or per setting policies

Other:

IV. INTERVENTION

FWPE item #18: articulates a clear and logical rationale for intervention process:

Verbally in supervision sessions

Via written assignments (journal, case study)

Via sharing EBP article reviews

In rounds/team meetings

Verbally in client sessions

In pt education materials

In written documentation

Via in-services

Other:

FWPE item #19 (part 1): Utilizes evidence from published research & relevant resources to make informed decisions with

supervisor/client/tem/caregiver/agency (as appropriate) using:

http://www.otcats.com.index.html/

http://www.otcats.com/template/index.html

http://www.canchild.ca/en/canchildresources/educationalmaterials.asp#CriticalReciew



Critically Appraised Papers (CAPs)/Critically Appraised Topics (CATs) (www.aota.org/Educate/Research.aspx; http://www.otcats.com/index.html)

Articles from peer reviewed journals (e.g., AJOT, OTJR, etc.,)


supervisor/client/team/caregiver/agency (as appropriate) using:

Discussion/sharing of material learned via other sources (textbooks, OT Practice, coursework, association website searches, conferences, etc.,) in supervision

Other:

FWPE items #20 & 21: Chooses relevant occupations that motivate & challenge clients to facilitate meeting established

goals based on client’s:

Condition/Status

Progress

Stated interests

Beliefs & values

Psychosocial needs

Current context & resources

Future context & resources

Other:

FWPE items #22 & 23: Implements client & occupation based intervention plans considering areas of occupation/outcomes

such as:

Role competence

ADL

Play Work

IADL

Sleep/rest

Social participation

Education

Leisure

Adaptation

Health/wellness

Quality of life

Self advocacy

Occupational (social) justice

Other:

FWPE item #24: Modifies task, approach, occupations & environment to maximize client performance by:

Adapting sequence of activity & objects

used

sensory input

Visual/verbal cues

http://www.aota.org/Educate/Research.aspx

http://www.otcats.com/index.html



Amount of physical assistance provided

Social demand (1:1 vs. group, family vs. peer(s), rules/norms)

Amount of emotional/behavioral support provided

Changing length/frequency/timing of sessions

Cognitive demand

Physical requirements

Promoting safety (awareness, education/feedback, environmental

modifications, removing potential sources of

injury, etc.,)

Creating adaptive device(s)

Reviewing/revising expectations with client(s) relative to desired occupations,

role(s) & context

Other:

FWPE item #25: Updates, modifies, or terminates intervention plan based upon careful monitoring of client’s status:

Accurately represents client progress verbally & in documentation

Accurately reports change in client status (e.g., illness, affect) affecting performance

Frequently re-evaluates effectiveness of intervention based on goal achievement and/or outcome measurement

Subjective data consistent with objective data reported verbally or in documentation

FWPE item #26: Documents client’s response to services in a manner that demonstrates efficacy of interventions via:

Progress reports with quantitative data (goal attainment scaling, excel charts/graphing, re-assessment, score comparison, outcome measurement results)

Narrative summary with qualitative descriptors according to problems identified/goals achieved

Other:

V. MANAGEMENT OF OCCUPATIONAL THERPAY SERVICES

FWPE item #27: Demonstrates through practice or discussion ability to assign appropriate responsibilities to the

occupational therapy assistant & occupational therapy aide:

Describes or assigns duties commensurate to educational level, assessed competency, federal & state laws regulating use of supportive personnel

Considers number of clients, complexity of needs, type of setting, safety

Describes or provides type of supervision required (close, direct, line of sight)

Provides reference for state statues/regulations governing performance or services & definitions of supervision (www.aota.org/Practitioners/Licensure/StateRegs/Supervision/36455.aspx)

Other:

FWPE item #28: Demonstrates through practice or discussion ability to actively collaborate with occupational therapy

assistant:

http://www.aota.org/Practitioners/Licensure/StateRegs/Supervision/36455.aspx



Describes &/or engages in tasks with OTA relative to job description &/or scope of practice as defined by state guidelines (e.g., soliciting contributions to evaluation process &/or delegating implementing & adjusting

intervention plan) in accordance with AOTA Official Guidelines for Supervision, Roles & Responsibilities

(www.aota.org/Practitioners/Official.aspx)

Completes alternate assignment to meet objective (please describe or attach):

FWPE item #29: Demonstrates understanding the costs and funding related to occupational therapy services at this site:

Discusses political issues/policy decisions that affect funding

Outlines how services are regulated and funds allocated pertaining to local and/or federal laws such as IDEA, ADA, Medicare/Medicaid, etc.

Describes agency billing/payment system (grant funding, types of insurance, private pay, cost-share, state/federal funding)

Describes eligibility criteria for reimbursement and discharge

Identifies possible resources available (grants, community partnerships, sources for donations, fundraising ideas, etc.,)

Demonstrates awareness of risk management and liability as part of costs and quality care

Demonstrates awareness of budgetary implications when procuring/using supplies

Other:

FWPE item #30: Accomplishes organizational goals by establishing priorities, developing strategies, and meeting deadlines:

Articulates setting’s mission & values

Schedules meetings/sessions according to facility expectations

Begins & ends sessions on time

Attends meetings on time

Reports meetings in concise manner

Meets paper work deadlines per policy & procedures

Prioritizes workload according to policies & caseload demands

Uses time management strategies (checklists, templates, to-do list)

Other:

FWPE item #31: Produces the volume of work required in the expected time frame:

http://www.aota.org/Practitioners/Official.aspx



Is self directed in managing schedule to meet workload/caseload

Gathers necessary evaluation data within allotted amount of time – specify: per day

per week per month

Completes evaluation write-up with documentation co-signed & in chart/record within:

8 hrs 24 hrs 1 week other:

Conducts (specify number) of evaluations: per day

per week per month

Serves caseload commensurate with entry-level therapist (please specify # of clients/groups):

per day per week per month

Completes progress note documentation within expected time period of:

Completes (specify number) of progress notes: per day per week per month

Other:

VI. COMMUNICATION

FWPE item #32: Clearly & effectively communicates verbally and nonverbally with clients, families, significant others,

colleagues, service providers, and the public:

Uses language appropriate to the recipient of information, including but not limited to funding agencies & regulatory agencies

Gauges use of terminology to level of understanding of person with whom communicating

Utilizes examples to illustrate meaning/intent

Uses active listening strategies (restates/paraphrases) to ensure both parties have shared understanding of information/plan

Uses multiple modes of communication (verbal, written, nonverbal)

Makes eye contact when appropriate

Attends to physical boundaries/body space

Demonstrates professional presentation/demeanor in facial expression, posture,



grooming affect, & attire

Utilizes setting’s services for translators of translation of written materials when indicated/available

Other:

FWPE item #33 – 34: Produces clear and accurate documentation according to site requirements. All written communication

is legible, using proper spelling, punctuation, and grammar:

Completes computerized &/or hand-written documentation per setting protocols/formats

Uses approved institutional terminology/abbreviations

Uses technology when available to check work (grammar, spelling)

Uses strategies such as proof reading, reading aloud, checking against template, asking colleague/peer to proof if feasible

Other:

FWPE item #35: Uses language appropriate to the recipient of the information, including but not limited to funding agencies

& regulatory agencies:

Writes in a manner conductive to being read by recipients of services & other disciplines, free of jargon, retaining language consistent with OTPF-Revised (client profile, analysis of occupational performance, areas, skills/patterns,

influence of context(s), client factors)


Utilizes examples to illustrate meaning/intent (verbal/demonstration)

Takes into account cultural differences, providing handouts in client’s first language, when available, providing illustrations with written content

Adjusts content (verbal/nonverbal) in response to clients/family/caregivers colleagues’ response

Provides clear & concise instructions

Other:



VII. PROFESSIONAL BEHAVIORS

FWPE item #36: collaborates with supervisor(s) to maximize the learning experience:

Asks supervisor for specific feedback

Consistently checks in to clarify expectations

Shares information about learning style with supervisor and asks for help as needed to adjust

Utilizes structures in setting to support learning (e.g., student manual, reviews expectations, tracks own caseload/workload)

Asserts need to schedule supervision meetings

Discusses concerns & identify possible avenues for changes or improvements

Discusses &/or negotiates need for adjustments to supervisory relationship, performance expectations, caseload,

& learning environment to improve quality of experience (e.g., reviews AOTA Fieldwork Experience Assessment Tool)

Other:

FWPE item #37: Takes responsibility for attaining professional competence by seeking out learning opportunities &

interactions with supervisor(s) & others:

Comes to supervision w/ list of questions/concerns & possible options for how to address them

Takes initiative to meet w/ other members of team to understand their role/perspective

Reviews testing materials/manuals on own prior to observing or administering

Seeks out, reviews & shares reading materials/articles on frames of reference/EBP, client conditions, public law/policy, etc.,

Pilots new program ideas/improvements (e.g., assessment tools, outcome measures, groups, new forms or procedures, etc.,) when feasible/available

Collaborates in research design or data collection with others (per IRB approval)

Exercises good judgment when choosing to attend in-services or other continuing education opportunities (e.g., based on workload management, caseload focus, scope of practice)

Other:

FWPE item #38: Responds constructively to feedback:



Engages in mutual feedback exchange (e.g., listen, clarify, acknowledge feedback &/or redirection, provide examples, ask “How can I improve?”; discuss ways to make active changes, identify what would be helpful,

discusses options)

Demonstrates commitment to learning by identifying specific goals/actions to improve behavior/performance in collaboration with supervisor

Processes feedback & seeks support from supervisor appropriate to context of supervisory relationship & learning opportunity

Utilizes tools to reflect on own performance or variables affecting performance (e.g., self-assessment on FWPE, journaling, FEAT)

Takes initiative to contact academic program resource persons for support if needed

Other:

FWPE item #39: Demonstrates consistent work behaviors including initiative, preparedness, dependability, and work site

maintenance:

Takes initiative to address workload management

Demonstrates consistent work behaviors in both task & interpersonal interaction

Attends to site cleanliness, safety & maintenance of supplies as appropriate to role

Comes prepared for meetings/sessions

Takes responsibility to address areas of personal/professional growth

Proactively plans for & requests appropriate supports or accommodations in manner consistent with federal law & site resources (e.g., open in communication, provides appropriate documentation, requests reasonable

accommodation if indicated)

Other:

FWPE item #40: Demonstrates effective time management:

Monitors, maintains & adapts own schedule in accordance w/ site’s priorities

Organizes agenda or materials for meetings & sessions

Conducts evaluation &/or intervention sessions w/in allotted time, inclusive of set-up/clean-up

Completes documentation/paperwork in timely manner

Completes learning activities by due dates

FWPE item #41: Demonstrates positive interpersonal skills including but not limited to cooperation, flexibility, tact, and

empathy in social interactions w/ clients/patients, peers & colleagues:

Communicates concerns in 1st person manner (e.g., “I statements”)



Remains calm when conveying point of view when conflict arises

Compromises as needed when negotiating workload

Demonstrates flexibility to support own learning or department mission (e.g., extra effort, stay late if needed, etc.,)

Demonstrates ongoing awareness of impact of own behavior on others

Displays positive regard for others

Demonstrates effective use of self disclosure (e.g., moderate)/therapeutic use of self to build rapport, establish alliance(s) & motivate others (peers/colleagues/clients)

Provides timely & specific feedback

Sets limits to maintain safety & support positive behavior/performance improvement

FWPE item #42: Demonstrates respect for diversity factors of others including but not limited to socio-cultural,

socioeconomic, spiritual, and lifestyle choices:

Demonstrates awareness of own background and sensitivity to worldviews of others (clients, family, colleagues)

Refrains from imposing own beliefs & values on others

Maintains clients’ dignity

Gathers information about clients’ cultural values &/or spiritual beliefs

Incorporates clients’ values & beliefs into therapeutic interactions & interventions

Considers clients socioeconomic & community resources & lifestyle when designing intervention plans & discharge planning

Demonstrates tolerance for differences in others & willingness to work w/ all clients

Other expectations not noted above:

Print Name

Signature Date

Assessment Checklist (attach to Level II FW Site Specific Objectives)



Site:

Date:

Formal Assessment Tools

Expected to gain Proficiency

Expected to gain familiarity

Expected to gain awareness/observe

Allen Cognitive Level Screening:

Allen Diagnostic Module

Ashworth Scale (Tone)

Assessment of Motor & Perceptual Skills

Bay Area Functional Performance Evaluation

Beery Visual Motor Integration test

Box and Block test

Borgs Scale: Rate of Perceive Exertion

Bruininks-Oseretsky Test -2 Children’s Assessment of Participation & Enjoyment/Preference for Activities of Children

Children’s Occupational Self Assessment

Canadian Occupational Performance Measure

Cognitive Assessment of Minnesota

Coping Inventory

Crawford Small parts Dexterity Test

Dynamometer

Early Coping Inventory

Functional Independence Measure:

Global Deterioration Scale

Goniometry

Hawaii Early Learning Profile

Jepson Hand Function Test

Kitchen Task Assessment

Kohlman Evaluation of Living Skills

LOTCA

Manual Muscle Testing

MOCA(Montreal Cognitive Assessment) Mini Mental State:

Formal Assessment Tools Expected to gain

Proficiency

Expected to gain

familiarity

Expected to gain

awareness/observe



MOHO The Model of Human Occupation Screening Tool (MOHOST)

The Occupational Circumstances Assessment Interview & Rating Scale (OSCAIRS)

The Occupational Performance History Interview – II (OPHI – II) The Occupational Self-Assessment (OSA)

The Occupational Therapy Psychosocial Assessment of Learning (OTPAL)

Motor-Free Visual Perception Test (MVPT-3)

Moberg Pick Up Test:

Nine Hole Peg Test

Occupational Performance History Interview

Occupational Self Assessment

Peabody Developmental Motor Scale:

Pediatric Evaluation of Disability (PEDI)

Piers Harris Children’s Self Concept Scale

Pinch Meter

Purdue Pegboard

Routine Task Inventory

School Function Assessment (SFA)

Sensory Integration & Praxis Tests

Semmes-Weinstein Monofilament:

Sensory Profile

Volumeter:

WeeFIM

Others : (Please list below)

PLAN OF CORRECTION ACOTE STANDARDS (All information must be submitted in typewritten format.)

Plan of Correction Submitted to ACOTE by Florida A&M University Date: 06/19/2017 (Institution)

Program length (including prerequisites and fieldwork): [ ] less than one year [ ] at least one year in length, but less than two years [X ] at least two years in length

Signatures: I affirm that the contents of this report are true to the best of my knowledge, information and belief. , Director, Assoc. Professor , , Dean, School of Allied Health Sci. (Program Director's Signature) (Program Director's Title) (Administrator's Signature) (Administrator's Title)

AREA OF NONCOMPLIANCE

PLAN OF ACTION TO BE TAKEN TIME LINE FOR ACTION

DOCUMENTATION TO BE SUBMITTED

(Projected Submission Date)

PROGRESS ON ACTIONS (IF ANY) IMPLEMENTED TO

DATE

A. 5.6

Certification Exam

Pass Rate

The average pass

rate over the 3 most

recent calendar

years for graduates

attempting the

national certification

exam within 12

months of

graduation from the

program must be

80% or higher

(regardless of the

number of attempts).

If a program has less

than 25 test takers in

the 3 most recent

calendar years, the

program may

Review of all Courses 1. Curriculum mapping to identify overlap and

gaps in addressing the accreditation standards. Emphasis will be placed on course sequencing, programs outcomes and assignments/activities within each course.

2. Review all syllabi for course content alignment to program outcomes and accreditation requirements.

Faculty Credentialing 1. All program faculty will be credentialed and

maintain current licensure and continuing education requirements.

Fall 2017

Fall 2017

Course syllabi

Curriculum

Faculty Credentials

Program began curriculum mapping exercise summer 2016. This resulted in modifications to two courses: OTH 5206 and OTH 5243. Course content was augmented with the addition of sample questions, mental health content and active learning activities.

In process of completing faculty credentials.

include test takers

from additional

years until it reaches

25 or until the 5

most recent calendar

years are included in

the total.

Student Exam Prep 1. Develop intensive study sessions

(boot camp) after completion of Fieldwork Level II b.

2. Review of summative assessments for

each course in the curriculum. 3. Purchase comprehensive examination for

all graduating MSOT students through the national examination council.

4. Examination will be given once in fall

semester and twice in spring semester year of graduation.

5. Students must receive a score of 70% or above to graduate.

Three attempts to meet pass criteria.

Fall 2017

Spring 2018

Spring 2018

Spring 2018 Spring 2018

Study session agenda

Examples of actual outcome measures.

Results of comprehensive examination for graduating class of 2018.

Results of comprehensive examination for graduating class of 2018. Results of comprehensive examination for graduating class of 2018.

Already require a three-day workshop for students prior to graduation. Evaluating feasibility if extending the workshop and/or “boot camp”.

This process has been started.

The program currently uses the OTKE. Looking at other resources that are closely related to the NBCOT exam.

Plan of Correction – ACOTE Standards (continued) Institution Florida A&M University Page of

AREA OF NONCOMPLIANCE

PLAN OF ACTION TO BE TAKEN TIME LINE FOR ACTION

DOCUMENTATION TO BE SUBMITTED

(Projected Submission Date)

PROGRESS ON ACTIONS (IF ANY) IMPLEMENTED

TO DATE

Division of Occupational Therapy Narrative Response to ACOTE Standard A.5.3

June 20, 2017 The Master of Science in Occupational Therapy (MSOT) program at Florida A&M University (FAMU) is in compliance with ACOTE standard A.5.3. The program routinely secures and documents sufficient qualitative and quantitative information to all for meaningful analysis about the extent to which the program is meeting stated goals and objectives in the areas of faculty effectiveness, students’ progression, student retention, fieldwork performance, student evaluation of fieldwork, student satisfaction with the program, performance on NBCOT exam, job placement and performance as determined by employer satisfaction. These key performance indicators, which are aligned with the program goals/outcomes (see Program Evaluation Plan) are assessed and evaluated on an annual basis. Each component of standard A.5.3 has been addressed by a program goal/outcome, appropriate measures and targets for performance. Quantitative and qualitative data analysis techniques were employed in analyzing data for each goal where appropriate. Following, are the ACOTE performance indicators, related assessment tools, and current performance outcomes. Faculty Effectiveness: To determine faculty effectiveness in teaching and learning, the program utilizes the State University System Student Assessment of Instruction (SUSSAI), student comprehensive examinations, final grades and annual faculty evaluations. At the end of each course, students complete the SUSSAI (using a rating scale of Excellent to Poor), a university-wide tool used to assess course instructor’s performance on eight items related to course objectives and assignments, communication of ideas and information, expression of expectations for student performance in class, availability to assist students in or out of class, respect and concern for student, stimulation of interest in the course, and facilitation of learning. The Occupational Therapy (OT) program has set a goal that at least 95% of program faculty will achieve a rating of good or better on this instrument as a measure of faculty effectiveness. 100% of faculty met the goal for academic year 2016-2017. The SUSSAI is offered to students via Blackboard each semester. The program director reviews the descriptive results on all faculty within the program. Student comments were analyzed for themes. Two consistent themes emerged: (1) students desired more clinical activities to occur in the first year and (2) students desired more practice questions to help assist with passing the national examination. As a result, a new online program, Top Hat, was introduced in Functional Human motion summer 2017. This program allows instructors to embed questions throughout the lecture. This formative assessment technique allows instructors to verify whether students have identified key concepts within the lecture.

Division of Occupational Therapy

Narrative Response to ACOTE Standard A.5.3 Division of Occupational Therapy

Narrative Response to ACOTE Standard A.5.3

2

An additional measure of faculty effectiveness is the use of final grades for all courses with a related goal that at least 80% of students will receive a B or better. Every semester the program director reviews all Master of Science Occupational Therapy (MSOT) students’ grades and GPA. Results revealed that 89% of Year 2 MSOT students received a grade of B or better in all courses. In addition to the SUSSAI and course final grades, the program also uses annual program faculty evaluations to measure faculty effectiveness. For this measure, the program set a target that at least 90% of faculty will receive a rating of 3.5 out of 5 on the teaching effectiveness component of the Annual Faculty Evaluation Form. In 2016-2017, all faculty received an average rating of 4 out of 5. While the target was met, the faculty added a new assessment to determine faculty effectiveness. Beginning fall 2017, a peer review of all course instructors will be conducted. Student Progression: The MSOT program requires all students to matriculate successfully. To this end, students are required to maintain a cumulative grade point average of 3.0 or better. Additionally, the program allows for only one retake of D grades, as course repeats may delay progression. As a way to measure student progression, the program tracks students’ progress annually, using a progression audit and student advisement tracking logs. For this measure, the program has established a target goal of 95%. In spring 2017, 96% of students graduated and 99% of the students achieved a 3.0 or better for academic year 2016-17. While the target was met for both measures, students in the graduating cohort requested clarification of the program’s progression policies. Based on this feedback, the program revised its student handbook and fieldwork manual to clarify the policies on matriculation. The student handbook now clearly states that a student must maintain a cumulative grade point average of a 3.0, a D grade requires the student to retake the course, even if this results in a delay of progression through the program, and a course can only be repeated once. Any student who is not progressing successfully through the program is required to develop a progression plan with his or her academic advisor. The fieldwork manual has also been revised to clarify that, at midterm, if a student is in jeopardy of failing Level II fieldwork, a corrective plan of action is established between the Academic Fieldwork Coordinator (AFWC), the Fieldwork Educator (FE) and the student. If a student fails his/her Level II Fieldwork placement, the student may be placed at another site based on the discretion of the AFWC and faculty. A corrective plan of action must be developed and completed before a student is placed in another site. One student was required to complete a corrective plan of action and subsequent reassignment for Level II Fieldwork in 2017. Prior to beginning coursework, students are advised to schedule appointments with their advisors at least once per semester. To ensure that students are meeting with advisors on a consistent basis and progressing successfully through the program, the program




3

has set a target goal that 100% of students will obtain advising once per semester by an OT faculty member. In 2016-17, all students were advised. While the target was met for this goal, faculty recommended that the student advising log be revised to increase consistency within advising among OT faculty. The use of the revised form will begin fall 2017. Student retention: Student retention is tracked through the student progress report. The program has established a 95% retention goal. Currently, 100% of students in the MSOT program have been retained for the 2016-2017 academic year. Student progress is assessed throughout the semester to include a review of student performance as measured by midterm grades. The instructor and/or the advisor advise those students who are in jeopardy of not passing a course by mid-semester. This results in the development of a corrective action plan to assist the student in successfully matriculating through the program while at the same time aiding the program in reaching its retention goals. As stated elsewhere in this narrative, the fieldwork and student manuals have been revised to more clearly state the retention and progression policies of the program. Fieldwork and experiential component performance evaluation: The program has established a goal that all students successfully complete their Level I and Level II fieldwork assignments. The Fieldwork Educator using the Fieldwork Performance Evaluation for the OT student evaluates the students. The goal for Level II is 100% of students will score a 3 out of 4 or better on the FWPE OTS indicating that they met the minimum standard. In 2016-2017 academic year, one student did not successfully pass the Level II rotation I placement. The AFWC presented the issues to the faculty and a corrective action plan was established. The corrective plan of action has been implemented and the student is following it accordingly. Because Fieldwork is such an integral component in the development of professional occupational therapists, and based on feedback from students in 2015, two additional goals have been established for this program evaluation objective. The first goal is 80% of students will score a 3 out of 4 or better on the evaluation items of the Fieldwork Performance Evaluation for the OT student. This goal was established after receiving feedback from fieldwork educators and students stating a desire for more exposure to the most commonly used assessment tools on fieldwork. Students also scored less in this domain on the Occupational Therapy Knowledge Examination (OTKE). Changes were made in the content of courses to introduce more standardized and common assessments. At this time, 83% of students placed in Level II fieldwork for 2016-2017 met this goal. This was monitored by the AFWC and shared with faculty at faculty meetings. The second goal resulting from feedback from students and fieldwork educators is 80% of students will score a 3 out of 4 or better on the intervention items of the Fieldwork Performance Evaluation for the OT Student. In 2016-2017, 87% of the most recent class placed on Level II fieldwork met this goal. This was monitored by the AFWC and




4

shared with faculty at faculty meetings. Faculty will continue to incorporate evidence-based interventions into their course content as preliminary results revealed that students reported feeling more prepared for fieldwork. Students’ fieldwork performance scores reflect this increased performance and confidence. Student Evaluation of Fieldwork Experience: The Students Evaluation of Fieldwork Experience is used to assess the student’s impression of the curriculum and areas of course content which may need revision or change. In order to evaluate student satisfaction and overall experience in fieldwork, the program has established a goal that 80% of students will rate the educational opportunities, based on the Student Evaluation of the Fieldwork Experience form as 3 or better. 61% of the students rated this portion of the evaluation at a 3 or better. Data revealed that students were seeking more educational/clinical opportunities at existing sites. Interventions made by the program included additional collaborations with the sites, using the site-specific objective tool, to identify additional opportunities that can be offered at the sites. Further interventions were made to encourage new fieldwork educators to complete the Fieldwork Educator Certification Workshop in addition to existing fieldwork educators. This workshop is offered annually. All Fieldwork Educators are invited to an annual certification course in supervision of the fieldwork student. Fieldwork educators who are already certified are offered to take the course as a review for updated processes and regulations. This has been met with positive comments and reviews from the area Fieldwork Educators. Students’ success is also contingent on the supervision they are receiving and their relationship with their fieldwork educator. To further assess student evaluation of fieldwork experience, the program has set a target goal that 100% of students will rate their FWE on qualities of fieldwork educator at 4 out of 5 (agree) or better. In 2016-2017 academic year, all students rated their fieldwork educator a 4 or better. Ongoing monitoring of the students and their fieldwork experience will continue. Furthermore, the program has set a target goal that 100% of students will participate in at least 90% of Blackboard Discussion boards with weekly prompts. These discussions allow students to share their experiences with the AFWC and their cohort. In addition to Blackboard Discussion prompts, students are asked to reflect on their professional growth, competency, and relationship with the Fieldwork Educator through electronic journal entries and reflections through Blackboard. These journals also act as an early warning alert to the AFWC so that they can work with the student and Fieldwork Educator to increase the student’s success in the placement. In the academic year 2016-2017, 100% of students completed 90% of the journal entries. While the target goal was met and in response to students’ feedback, the program will implement added content in standardized and common assessment tools beginning with the admitted 2017 fall cohort. Student satisfaction with the program: The program has set a target that 80% of graduates will rate the program as satisfactory or better. For this measure, the program conducted exit interviews. Results revealed that 82% of graduates rated the program as




5

satisfactory or better. Another measure used to assess student satisfaction is the University exit survey. Survey results show that 82% of program graduates strongly agreed they obtained a sense of competency in their field of study. The program director analyzes this data annually and shares with faculty. While the goal was met for this target, student comments were analyzed to enhance the program course content and curriculum. Some changes resulting from these surveys included changes in course content, such as more experiential and active learning opportunities in didactic classes, and more exposure to NBCOT exam questions. Graduates’ performance on the NBCOT exam: To ensure graduates successful performance on the NBCOT exam, the program has set targets that 100% of students will pass the OTKE exam or equivalent prior to graduation and that 80% of MSOT graduates will pass the NBCOT exam within one year of graduation. The OTKE allows the faculty and the students to assess the strengths and weaknesses in the knowledge domains found on the NBCOT exam. In spring 2017, 100% of students passed the OTKE. In 2016, the program had a 68% pass rate on the NBCOT exam. So while 100% of students passed the OTKE, students demonstrated difficulty passing the NBCOT exam. Based on these results, faculty performed an in depth analysis of students who took the NBCOT exam. Student success on the NBCOT examination has been an ongoing analysis resulting in changes in admission criteria, course content, and learning opportunities for the students. An analysis of the admissions criteria has shown that there is a relationship between graduate record exam (GRE) scores, writing analysis scores, and prerequisite grades. Thus, the past two admission classes were evaluated using four quantitative scores. These are cumulative grade point average, prerequisite grade point average, GRE scores, and the analytic writing portion of the GRE. The past two admissions classes now have an average of a 3.6 cumulative grade point average, 3.2 prerequisite grade point average, an average graduation record exam score of 298, and an average analytical writing score of a 3.5. As part of this analysis, faculty reviewed student profiles which included pre-requites GPA and GRE scores, typically considered as strong predictors of success on the NBCOT exam. So after this analysis, program faculty revised its admission requirements beginning fall 2016 using the new predictors. Faculty also revised course sequencing of the curriculum and began mapping its goals and program outcomes to the curriculum. Beginning fall 2017 to further ensure graduates performance on the NBCOT exam, the program will implement these changes:

• Change the measurement tool from the OTKE to the NBCOT Practice Test. • Utilize the final four weeks in the program for intensive review of examination

content. • Hire a consultant for program development to include test strategy and use of

analysis for information gathered. • Continue curriculum mapping to identify overlap and gaps in addressing the

accreditation standards.




6

• Review all syllabi for course content and exam alignment. • Review all outcome measures for courses to determine strengths and

weaknesses in course content and exam alignment. • Continue to compile and analyze student admission profile data for 2015, 2016,

and 2017. • Review course outcome measures to optimize synthesis of program content.

Job Placement: The program has established a goal that 75% of its graduates will find gainful employment as an occupational therapist within one year of graduation. To assess this measure the program utilizes a single item email survey and the Florida Education Training Program (FETPIP) data. Results from the electronic survey revealed that 71% of MSOT graduates were employed as occupational therapists. The two graduates who are not employed worked as occupational therapists under a temporary license. They did not pass the NBCOT exam and lost their positions. Faculty are in regular communication with those graduates providing information on resources to facilitate improvements on the NBCOT exam. The 2014-2015 (most recent data) FETPIP data revealed that 89% of graduates were employed within one year of graduation. This data tracks only graduates that are employed in the state of Florida. The program will continue to monitor job placement.

Employer Satisfaction: An employer satisfaction survey is sent yearly to address the program goal that 90% of MSOT employers surveyed will rate graduate’s performance as satisfactory or better. The survey is sent electronically to employers who have hired Florida A&M University occupational therapy graduates. The most recent survey from 2016 showed that 100% of the employers surveyed rated FAMU MSOT graduates’ performance as satisfactory or better. The program will continue to monitor and respond to this goal as appropriate.

The program is in compliance with ACOTE standard A 5.3. Program goals/outcomes are assessed and evaluated on an annual basis and reported in the institutional assessment portal. Results from assessment helps to inform program improvement and the overall student experience.

PROGRAM EVALUATION PLAN FLORIDA A&M UNIVERSITY 2016-2017

Submitted June 20, 2017

ACOTE Standard


Assessment Tool




A 5.3 - Faculty Effectiveness

Faculty will employ best practices in pedagogy in support of student learning within the field. Students will demonstrate proficiency in course content. Faculty will demonstrate effectiveness in teaching.

State University System Student Assessment of Instruction (SUSSAI).

95% of faculty will be rated good or better for courses taught in the fall and spring.

100% (n=5) faculty were rated as good or better for courses taught in fall and spring. Analysis of open ended responses resulted in the following recommendations: • Addition of more

NBCOT type questions throughout courses.

• Addition of more applied or “hands on” (active learning) experiences.

In response to student feedback, TOP HAT, a program that facilitates the incorporation of NBCOT type questions in courses and lectures, was initiated summer 2017 in the functional human motion. In fall 2016, all lecture courses in the MSOT curriculum increased active learning activities.

Fall and Spring semesters.


Students’ course final grades.

80% of students will receive a B or better in all MSOT courses.

89% (n=25) Year 2 MSOT students received a B or better in all MSOT courses for 2016-2017.

While the goal was met, students who did not receive a B or better in any MSOT course received individualized advisement to strengthen study habits and skills. Individual course instructors are responsible for providing continuous feedback regarding student performance.

Annual faculty evaluations.

At least 90% of faculty will receive an average rating of 3.5

All faculty (n=5) received an average score of 4 out of 5 on

While the target was met, the program will implement peer evaluation of teaching beginning fall 2017 in an effort to

2

ACOTE Standard


Assessment Tool




out of 5 on the teaching effectiveness component of the faculty evaluation form.

the teaching effectiveness component of the annual faculty evaluation.

enhance teaching effectiveness. Program will continue to monitor this outcome.

A 5.3 - Student progression


Progression audit.

95% graduation rate per cohort.

96% (n=27) students in the cohort graduated Spring 2017.

While the target was met, students requested more clarification regarding progression through the program. This resulted in modifications in the student handbook and fieldwork manual.

Annually The Program Director and OT faculty reviews and analyzes the data.

95% of students will maintain a 3.0 or better throughout their matriculation in the program.

99% (n=80) students achieved a 3.0 or better for academic year 2016-2017. Students must maintain a cumulative grade point average of a 3.0. They must repeat a D grade and only can do this one time. Students who are not successfully matriculating through the program are required to develop a progression plan with their academic advisor.

While the target was met, an individualized program of study was developed for students that did not maintain a 3.0 GPA. For the one student that did not meet this goal, an individualized progression and study plan was completed by the student and the Program Director.

Every semester

Student advisement tracking log.

100% of students will be advised once a semester by OT faculty.

All students (n=81) were advised at minimum once a semester by OT faculty members over the course of 2016-2017 academic year.

While the target was met, the student advisement tracking log was revised and is scheduled to be implemented in Fall 2017 to increase consistency among faculty in advisement.

Academic year (summer through fall).

Program Director and all faculty.

3

ACOTE Standard


Assessment Tool




A 5.3 - Student Retention


Student progress report.

95% of MSOT students will be retained in the program.

100% (n=81) of MSOT students were retained for the 2016-2017 academic year.

While 100% of students were retained, data analysis revealed that two students were having difficulty in didactic course work, and received advisement. Moreover, any students experiencing difficulty in courses or fieldwork are required to see their instructor or advisor.

Midterm of every semester.

The Program Director and all faculty.

A 5.3 - Fieldwork and experiential component performance evaluation

Students will successfully complete Fieldwork I and II. Students will demonstrate competency in assessment and intervention at the end of Fieldwork Level II.

The Fieldwork I Evaluation for Fieldwork Level I.

100% of students will be rated at satisfactory or better for Fieldwork Level I.

All (n=28) students met the target for Fieldwork Level I for academic year 2016-2017.

While 100% of students met the target for Level 1, the program will continue to monitor student performance in Fieldwork Level I.

Every semester students are engaged in either Fieldwork Level I or Fieldwork Level II starting in the second year of the program. Student progress is evaluated by assignments or evaluations at midterm and final evaluation as needed.

The Academic Fieldwork Coordinator. The AFWC reports fieldwork issues to faculty at faculty meetings.

Fieldwork Performance Evaluation for Occupational Therapy Student for Fieldwork Level II.

100% of students will score a 3 out of 4 or better indicating that they met the minimum standard for Fieldwork Level II.

96% (n=27) students met the target for Fieldwork Level II for academic year 2016-2017.

For the one student that did not meet the standard for Level II, the Academic Fieldwork Coordinator presented student performance results to the faculty. Consistent issues (such as lack of preparedness for the site, competency issues) triggered a plan of progression in conjunction with the fieldwork educator, academic fieldwork coordinator and student.

80% of students will score a 3 out of 4 or better on the evaluation items of the Fieldwork Performance Evaluation for the OT student.

83% (n=23) students scored a 3 out of 4 or better on the evaluation items of the Fieldwork Performance Evaluation.

While the target was met, the program will continue to utilize the evaluation to enhance didactic classes by using common and standardized evaluation tools. The program will also continue to monitor student performance on the evaluation

At the end of each Fieldwork Level II rotation.

The Academic Fieldwork Coordinator. The results are shared with faculty at faculty

4

ACOTE Standard


Assessment Tool




items on the Fieldwork Performance Evaluation.

meetings.

80% of students will score a 3 out of 4 or better on the intervention items of the Fieldwork Performance Evaluation.

87% (n=24) students scored a 3 out of 4 or better on the intervention items on the Fieldwork Performance Evaluation.

While the target was met, the program will continue to utilize the evaluation to enhance didactic classes by using evidence based interventions. The program will also continue to monitor student performance on the intervention items on the Fieldwork Performance Evaluation.

A 5.3 - Student Evaluation of Fieldwork Experience

Improve student satisfaction and overall experience in fieldwork.

Students Evaluation of Fieldwork Experience.

80% of students will rate the Educational Opportunities using “Aspects of the environment” at 3 out of 4 or better.

61% (n=17) students rated educational opportunities at 3 out of 4 or better.

Results revealed that students were seeking more educational/ clinical opportunities at existing sites. Interventions made by the program included additional collaborations with sites using the site-specific objectives tool, to identify additional opportunities that can be offered at sites. Further interventions were made to include a yearly Fieldwork Educator Certification Workshop.

Each Fieldwork II Rotation.

Academic Fieldwork Coordinator & Program Director and faculty.

100% of students will rate their FWE on qualities of fieldwork educator at 4 out of 5 (agree) or better.

100% (n=28) students rated their FWE at 4 out of 5 (agree) or better.

The program will continue to monitor students’ ratings of the FWE.

5

ACOTE Standard


Assessment Tool




Blackboard Discussion Boards – Weekly prompts.

100% of students will participate in at least 90% of Discussion boards.

100% (n=28) students participated in at least 95% of Discussion Boards. Qualitative analysis revealed themes that indicated students were progressing adequately with completing evaluations and interventions; used occupation-based interventions and developed organizational and coping skills.

The program will continue to monitor students’ participation in Discussion Board weekly prompts.

Each Fieldwork Level II Rotation.

Academic Fieldwork Coordinator and Program Director and faculty.

Electronic Journal Entries and Reflections in Blackboard.

100% of students enrolled in Fieldwork IIA&B will complete 90% of the weekly journal entries reflecting on strengths and areas of growth using specific examples, as well as the relationship with Fieldwork Coordinator.

100% (n=28) students completed 90% of the journal entries. The results revealed that students wanted earlier exposure to standardized and common assessment tools.

While the target was met, and in response to students’ feedback, the program will implement added content in standardized and common assessment tools beginning with the admitted 2017 fall cohort.

Journals reviewed weekly by the Academic Fieldwork Coordinator.

Academic Fieldwork Coordinator, Faculty and Program Director.

6

ACOTE Standard


Assessment Tool




A 5.3 - Student satisfaction with the program

Graduates will be satisfied with their overall experiences in the program.

Exit interviews from the Division of Occupational Therapy.

80% of graduates will rate the program as satisfactory or better.

82% (n=22) graduates rated the program as satisfactory or better.

While the goal was met, results from both the exit interviews and University Graduate Exit Survey were utilized to improve the curriculum. Following is a sample of recommendations from the survey:

• Students requested more exposure to clients in the first year. In response to student feedback, OTH 5206 will be revised to provide students more exposure to clients in their first semester fall 2017.

• Students requested more applied experiences in classes. In response, the curriculum will be changed to allow for experiential activities in the first year of the program. Further, more applied activities will be included in lecture classes.

Annual review of exit surveys and interviews.

The Program Director and Occupational Therapy Faculty.

University Graduate Exit Survey.

80% of MSOT graduates completing the University Graduate Exit survey will strongly agree that they obtained a sense of competence in their field of study.

82% (n=22) graduates strongly agreed they obtained a sense of competence in their study.

A 5.3 - Graduates’ performance on the NBCOT certification exam.

Graduates from the MSOT program will pass the NBCOT certification examination within one year of graduation.

OTKE Pass rate.

100% of students will pass the Occupational Therapy Knowledge Exam (or equivalent) prior to graduation.

100% (n=28) students passed the OTKE in Spring 2017.

While 100% of students passed the OTKE, students demonstrated difficulty passing the NBCOT exam, thus the program made the following changes:

• Reviewed individual student profiles and developed an individualized action plan to improve student success on the NBCOT exam.

• The results from the OTKE were utilized to revise the curriculum.

Implemented changes based on results: Revised course sequencing and began curriculum mapping.

Each fall and spring semester.

Advanced Topics and Professional Development II course instructors and program faculty.

7

ACOTE Standard


Assessment Tool




Changes to be implemented fall 2017:

• Change the measurement tool from the OTKE to the NBCOT Practice Test.

• Utilize the final four weeks in the program for intensive review of examination content.

• Hire consultant for program development to include test strategy and use of analysis for information gathered.

NBCOT pass rate results.

80% of MSOT students will pass the NBCOT certification examination within one year of graduation.

2016 pass rate: 68% Implemented changes based on results: Performed an in-depth analysis of students who took the NBCOT exam. An analysis of student profiles indicated that pre-requisite GPA and GRE scores were strong predictors of success on the NBCOT exam. Based on these findings, the program revised the admission criteria beginning fall 2016. An action plan was developed to include the following and will be implemented fall 2017:

• Continue curriculum mapping to identify overlap and gaps in addressing the accreditation standards.

• Review all syllabi for course content and exam alignment.

• Review all outcome measures for courses to determine strengths and weaknesses in course content and exam alignment.

• Compile and analyze student admission profile data for 2015,

Systematically check the NBCOT website each month.


8

ACOTE Standard


Assessment Tool




2016, and 2017 • Review course outcome measures

to optimize synthesis of program content.

A.5.3 Job Placement

Graduates of the program will find gainful employment as an occupational therapist.

Email survey to graduates asking for job placements.

75% of graduates will find gainful employment as an occupational therapist within one year of graduation.

2016: 71% (n=5) graduates employed as occupational therapists. The two who are not employed worked as occupational therapists under a temporary license. They did not pass the NBCOT exam and lost their positions.

For the two students who did not pass the NBCOT exam, faculty are in regular communication with the graduates providing information on resources to facilitate improvements on the NBCOT exam. Examples include:

• Commercially available test preparatory resources.

• Mobile Apps • Peer mentorship • Faculty mentorship

Continue to monitor graduate job placement.

Continuous and ongoing as appropriate per student.


Florida Education and Training Placement Information Program (FETPIP) data.

2014-2015: 89% (n=24) of graduates were employed within one year of graduation based on the most recent FETPIP data.

A 5.3 – Employer satisfaction

Employers of graduates of the MSOT program will be satisfied with their performance.

The Program Employer Satisfaction survey.

90% of MSOT employers surveyed will rate graduate’s performance as satisfactory or better.

100% (n=10) of the employers surveyed rated FAMU MSOT graduates performance as satisfactory or better.

Continue to monitor employer satisfaction of the FAMU MSOT graduates.

Yearly distribution of employee satisfaction survey.


of 3

FLORIDA A&M UNIVERSITY

FACULTY EVALUATION FORM

FROM (College/School)

(Division)

(Department)

FACULTY

MEMBER

EVALUATION

PERIOD

PART A. Teaching Effectiveness: Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Organization and planning of courses.

0 1 2 3 4 5

2. Evidence of careful preparation documented by

current course outlines, syllabi, or other

appropriate methods distributed at the beginning of class meetings.

0 1 2 3 4 5

3. Ability to make course(s) challenging, inspire interest and thought in subject matter, and

encourage students to understand all phases of the

course(s).

0 1 2 3 4 5

4. Quality of scholarship and professionalism;

mastery and currency in principal subject area and

related disciplines.

0 1 2 3 4 5

5. Ability to maintain professional classroom

decorum, and to project an atmosphere of friendliness and concerns for students.

0 1 2 3 4 5

6. Punctuality with respect to classroom attendance,

adherence to announced schedule; grading of tests and assignment; and submitting required reports.

0 1 2 3 4 5

7. Accessibility and openness to questions; maintenance of office hours and availability for

student consultation.

0 1 2 3 4 5

8. Appropriateness, relevance and fairness of

examinations or student evaluations.

0 1 2 3 4 5


Faculty Member:

Revised – 5/80

of 3

PART B. Research and Creative Ability Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Evidence of presentation of papers or creative

works at professional meetings/societies or

exhibitions.

0 1 2 3 4 5

2. Evidence of papers or articles published in

professional journals and/or of books published.

0 1 2 3 4 5

3. Evidence of recognized creative work in his/her

field such as musical compositions, paintings,

product design, sculptures, dance, theatre, etc.

0 1 2 3 4 5

4. Extent of on-going research or creative endeavors.

0 1 2 3 4 5

5. Extent of development of research projects or

submission of proposals relative to research or

creative ability.

0 1 2 3 4 5


Faculty Member:

PART C. Performance in Service Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Performance on departmental college/school, and

university-wide committees, councils and task

forces.

0 1 2 3 4 5

2. Contribution to faculty meetings and regular

program activities.

0 1 2 3 4 5

3. Membership in appropriate professional

organizations and contributions to these.

0 1 2 3 4 5

4. Involvement in activities leading to professional

development, continuing education, certification,

or licensure.

0 1 2 3 4 5

5. Performance in community activities on a local,

state, national or international level.

0 1 2 3 4 5


Faculty Member:

of 3

PART D. Other University Duties Not

Applicable

Poor

Fair

Good

Very

Good

Excellent

1. Performance as an academic advisor.

0 1 2 3 4 5

2. Performance as a counselor/advisor to clubs or organizations, direction of interns, or other

university related groups.

0 1 2 3 4 5

3. Performance in administrative duties.

0 1 2 3 4 5


Faculty Member:

SIGNATURES AND GENERAL COMMENTS

Signature of

Department

Chairperson:

Date

*Signature of

Faculty Member:

Date

Signature of

Division Director:

Date

Signature of

Dean:

Date

*I HAVE HAD A CONFERENCE WITH MY SUPERVISOR AND HAVE BEEN DULY APPRISED OF

THIS EVALUATION.

1

MSOT Student Advisement Log

Log of Meetings: Semester/Year

# Date: Time Name SID Year Response 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16


Purpose:

This evaluation serves as a tool for fieldwork sites, academic programs, and students. The main

objectives of this evaluation are to:

Enable the Level II fieldwork student who is completing a placement at the site to

evaluate and provide feedback to the supervisor and fieldwork setting;

Enable academic programs, fieldwork sites, and fieldwork educators to benefit from

student feedback in order to develop and refine their Level II fieldwork programs;

Ensure that all aspects of the fieldwork program reflect the sequence, depth, focus, and

scope of content of the curriculum design;

Provide objective information to students who are selecting sites for future Level II

fieldwork; and

Provide a means of evaluation to ensure that fieldwork is performed in settings that

provide educational experiences applicable to the academic program.

This form is designed to offer each program the opportunity to gather meaningful and useful

information. Sections outlined with thick black double borders are designed to be customized by

your program as needed. Pages involving evaluation of individual fieldwork educators have been

positioned at the end of the form to allow academic programs to easily remove these pages before

making them available for student review, if they choose to do so.