ww2.lipscomb.edu - prog… · Web viewMission Statement and Program Goals. The mission for the...
Transcript of ww2.lipscomb.edu - prog… · Web viewMission Statement and Program Goals. The mission for the...
Graduate Studies in Counseling
Program Evaluation Report
2014-2015 Academic Year
Graduate Department of Psychology, Counseling, and Family Science
Lipscomb University
One University Park DriveNashville, TN 37024-3951
1-800-333-4358 x5906615-966-5906
1
Table of ContentsDescription of Program, Faculty and Curriculum 6
MISSION STATEMENT AND PROGRAM GOALS 6
Program Goals 6
Program Objectives 6
Program Description 7
PROGRAM FACULTY 9
CURRICULUM 9
Courses 10
Systematic Plan for Program Evaluation 10
ACCREDITATION STANDARDS 10
PHILOSOPHY OF PROGRAM EVALUATION 11
EIGHT-POINT STUDENT, FACULTY AND PROGRAM EVALUATION PLAN 15
Application and Interview 15
New Student Orientation 15
First Semester 15
Pre-Practicum Semesters 16
Application to Candidacy 16
Clinical Semesters 16
Final Semester 16
Alumni 16
MEETING PROGRAM GOALS AND OBJECTIVES THROUGH THE EIGHT POINT ASSESSMENT PLAN 16
Program Goals 17
Program Objectives 17
PROGRAM EVALUATION CALENDAR 19
PROGRAM EVALUATION RESPONSIBILITY ASSIGNMENTS 20
Internal Evaluation of Students 22
DESCRIPTION OF DATA COLLECTION PROCEDURE PRE-ADMISSION 22
Review of Applicants 22
DESCRIPTION OF DATA COLLECTION PROCEDURE 24
Skills Course Evaluation 24
Progress Review Committee (PRC) Evaluation 24
Grades for Each Course 25
2
Student Candidacy Review 25
Clinical End of Semester Reviews 26
Final Student Review 27
Remediation and Gatekeeping for the Profession 27
INTERNAL PROGRAM EVALUATION OF STUDENTS REVIEW 28
Strengths 28
Areas to Improve 29
Internal Program Evaluation of Faculty and Program 31
DESCRIPTION OF DATA COLLECTION PROCEDURE 31
New Student Orientation Feedback 31
Course Evaluations 32
Mid-Program Evaluation 33
Practicum Orientation Evaluation 34
Group Supervisor Evaluation 35
Student Site and Site Supervisor Evaluations 37
Exit Interview 38
Final Program Evaluation 38
INTERNAL PROGRAM EVALUATION OF FACULTY AND PROGRAM REVIEW 39
Strengths 39
Areas to Improve Error! Bookmark not defined.
External Evaluation of Program and Students 42
DESCRIPTION OF DATA COLLECTION PROCEDURE 42
Site Supervisor Mid-Semester and End of Semester Evaluations 42
Site Supervisor Program Evaluations 43
CPCE Exam 44
Alumni Survey 45
Employer Survey 46
EXTERNAL EVALUATION OF PROGRAM AND STUDENTS REVIEW 46
Strengths 46
Areas to Improve 47
Program Modifications based on Review Data 48
OVERALL PROGRAM STRENGTHS 48
Internal Evaluation of Students 48
Internal Evaluation of Program and Faculty 48
3
External Evaluation of the Program and Students 49
Strengths Indicated from All Internal and External Evaluations 49
OVERALL PROGRAM AREAS TO IMPROVE 50
Internal Evaluation of Students 50
Internal Evaluation of Program and Faculty 50
External Evaluation of Program and Students 51
Areas to Improve Indicated by Internal and External Evaluations 51
Questions to Consider Error! Bookmark not defined.
PLAN TO ADDRESS CHALLENGES 51
CONCLUSION 53
4
Table of FiguresFigure 1: Graduate Studies in Counseling within the College of Arts and Sciences 8
Figure 2: Diagram of Program Philosophy 12
Figure 3: Eight-Point Student, Faculty, and Program Evaluation 14
Figure 4: Internal and External Program Evaluation Assessments 15
Figure 5: Matching Assessments with Program Goals and Objectives 17
Figure 6: Program Evaluation Calendar 19
Figure 7: Assessment of Students and Philosophy of Program Evaluation 24
Figure 8: External Evaluations and Philosophy of Program Evaluation 42
Figure 9: Philosophy of Program Assessment with Strengths and Areas to Improve 52
5
Description of Program, Faculty and Curriculum
Mission Statement and Program Goals The mission for the Lipscomb University Graduate Studies in Counseling program is to provide an educational environment characterized by academic excellence and Christian faith, where counseling students are prepared to become competent, confidant, committed and caring professional mental health counselors. See Figure 1.
Figure 1: The Four C's of our Mission
Program GoalsThe Graduate Studies in Counseling program promotes the following goals:
1. To attract and develop a high quality, diverse student body. 2. To increase student knowledge in the field of professional counseling.3. To facilitate competence in the practice of professional counseling.4. To serve and improve the quality of life in the local, state, national, and international community.5. To encourage an understanding and commitment to the scientist practitioner model.6. To facilitate an understanding of the individual characteristics that each student brings to the counseling
process.
Program ObjectivesThe Graduate Studies in Counseling program promotes the following objectives:
1. To gain knowledge of the counseling profession and other related helping professionals.2. To gain knowledge of and become accountable to the ACA Professional Code of Ethics.3. To gain knowledge and competence in serving and advocating for culturally diverse populations, including
assessment, treatment planning, treatment, and outcome evaluation.4. To gain knowledge of eight core knowledge areas: professional orientation and ethical practice, social and
cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation.
5. To gain knowledge of a wellness model of mental health.6. To gain knowledge in the foundations of research and inquiry.
6
Through academic excellence and Christian
faith, counseling students are prepared to become:
CaringCommitted to the Profession
ConfidentCompetent
Program DescriptionThe Graduate Studies in Counseling program is housed in the College of Arts and Sciences. The department of Psychology, Counseling, and Family Science is part of the School of Communications and Social Sciences within that college. Our department includes programs in Undergraduate Psychology and Family Science, Graduate Studies in Psychology, Graduate Studies in Counseling and Marriage and Family. See Figure 2.
7
Figure 2: Graduate Studies in Counseling within the College of Arts and Sciences
8
LIFE ProgramSchool of Natural
Sciences and Mathematics
Lipscomb Family Therapy
Center
Master of Science in
Marriage and Family Therapy
University Counseling
Center
Master of Science in
Clinical Mental Health
Master of Science in
Psychology
Graduate Programs
Family Science
Psychology
Undergraduate
Department of Psychology and
Counseling
School of Communicatio
n and Social Sciences
School of Humanities
School of Fine and Performing
Arts
COLLEGE OF ARTS AND SCIENCES
Program Faculty
FacultyOur departmental faculty consists of full-time and part-time professors representing the areas of professional counseling, marriage and family therapy, and the fields of clinical, experimental, neuroscience, developmental and cognitive psychology. We supplement this group with excellent part-time faculty members who are experts in their fields. For further information about each faculty member, visit our website.
Core FacultyJake Morris, B.A., M.Ed., Ph.D. (University of Mississippi), LPC, Director of Graduate Studies in Psychology and
Counseling, Professor of Psychology and Counseling
Melanie H. Morris. B.S. (Lipscomb University), M.S., Ph.D. (University of Mississippi), Licensed Psychologist, HSP, Clinical Director of Graduate Studies in Professional Counseling, Assistant Professor of Psychology and Counseling
Shanna D. Ray, B.S. (Lipscomb University), M.S. (Middle Tennessee State University), M.S., Ph.D. (Vanderbilt University), Department Chair of Graduate Studies in Psychology and Counseling, Thesis Coordinator, Associate Professor of Psychology and Counseling
Paul E. Turner, A.A. (Faulkner University), B.S. (Auburn University), M.A. (Southern Christian University), M.S. (University of Western Kentucky), M.A. (University of Southern Mississippi), Ph.D. (University of Southern Mississippi), Professor of Psychology and Counseling
J. Dale Alden III, B.S. (Freed Hardeman University), M.S. (Abilene Christian University), Ph.D. (Virginia Polytechnic University), Internship/Residency, (Vanderbilt University School of Medicine), Licensed Psychologist, HSP, Assistant Professor of Psychology and Counseling
Douglas Ribeiro, B.S., M.S. (Freed Hardeman University), Ph.D. (Georgia State University), Assistant Professor of Psychology and Counseling
Denis’ Thomas, B.A. (Abilene Christian University), M.Ed. (Middle Tennessee State University), Ph.D. (University of Tennessee),NCC, Coordinator of Play Therapy Specialization, Assistant Professor of Psychology and Counseling
Scott, Frank,
Curriculum The Master of Science in Clinical Mental Health Counseling offers a 61-hour curriculum designed to meet the academic and graduate clinical experiences required for licensure as a Professional Counselor with Mental Health Service Provider designation (LPC-MHSP) in the State of Tennessee (TN). This credential allows students to provide counseling services in a variety of settings as outlined by the State Board of Certification for Professional Counselors and Marital and Family Therapists and Clinical Pastoral Therapists. For detailed information about this licensure, see the rules of the State Board of Certification for Professional Counselors and Marital and Family Therapists and Clinical Pastoral Therapists. Our program’s emphasis on faith integration is especially helpful for those who wish to practice in faith-related settings (although students are not limited to only these settings). Graduation from the Counseling Program requires the following completed steps:
● Cumulative GPA of 3.0 or higher
● Successful completion of Plan of Study (POS)
● Successful completion of the Progress Review process
9
● Approval for Candidacy
● Successful completion of Clinical Experiences
● Successful completion of Coursework including Capstone and comprehensive exam
● Completion of Application for Graduation and completing all required paperwork
● Recommendation of the Graduate Studies in Counseling faculty
Courses
Core Courses (46 Hours)
COUN 5103 Theories of Counseling and Personality
COUN 5113 Introduction to Clinical Mental Health
COUN 5203 Introduction to Psychopathology and Adaptive Behavior
COUN 5303 Lifespan Development
COUN 5603 Research Methods and Statistics
COUN 5703 Ethics and Professional Issues in Counseling
COUN 5803 Integration of Psychology and Christianity Seminar
COUN 5903 Introduction to Counseling Skills
COUN 6053 Multicultural Issues in Counseling
COUN 6103 Group Dynamics and Group Counseling
COUN 6253 Career Counseling
COUN 6303 Substance Abuse Counseling
COUN 6603 Advanced Psychopathology
COUN 6653 Assessment Techniques
COUN 6703 Effective Counseling and Treatment Planning
COUN 6951 Capstone Seminar in Counseling
10
Clinical Experience Courses (9 Hours)
COUN 6803 Practicum
COUN 6853 Internship I
COUN 6903 Internship II
COUN 6910 Internship Continuation (if needed)
Elective Courses (select 6 Hours)
COUN 5003 Survey of Psychology (for students who did not major in psychology)
COUN 6003 Grief and Crisis Counseling
COUN 6153 Marriage and Family Counseling
COUN 6203 Child and Adolescent Counseling
COUN 6353 Addictive Disorders
COUN 6403 Psychopharmacology
COUN 6453 Gerontology
COUN 6503 Human Sexuality
PSG 6553 Psychology of Religion and Spirituality
COUN 675V Psychological/Counseling Topics
COUN 675V Introduction to Play Therapy
COUN 675V Advanced Play Therapy and Expressive Arts
PSG 6933 Thesis I
PSG 6943 Thesis II
Systematic Plan for Program EvaluationAccreditation Standards The primary accrediting body for the counseling programs is the Council for Accreditation of Counseling and Related Educational Programs (CACREP). The 2009 CACREP standards and policies are guidelines for the plan for program evaluation for the Graduate Studies in Counseling program at Lipscomb University. The following sections are excerpted from these standards, which can be found in their entirety here, and guided our comprehensive program evaluation plan, including internal evaluation of students, internal evaluation of the program and faculty, and external evaluation of students and program.
Section I: THE LEARNING ENVIRONMENT : STRUCTURE AND EVALUATION
THE ACADEMIC UNIT (CACREP, 2009, p. 5).
P. The program faculty conducts a systematic developmental assessment of each student’s progress throughout the program, including consideration of the student’s academic performance, professional development, and personal development. Consistent with established institutional due process policy and the American Counseling Association’s (ACA) code of ethics and other relevant codes of ethics and standards of practice, if evaluations indicate that a student is not appropriate for the program, faculty members help facilitate the student’s transition out of the program and, if possible, into a more appropriate area of study.
EVALUATION (CACREP, 2009, p. 8)
11
AA. Program faculty members engage in continuous systematic program evaluation indicating how the mission, objectives, and student learning outcomes are measured and met. The plan includes the following:
1. A review by program faculty of programs, curricular offerings, and characteristics of program applicants.2. Formal follow-up studies of program graduates to assess graduate perceptions and evaluations of major
aspects of the program.3. Formal studies of site supervisors and program graduate employers that assess their perceptions and
evaluations of major aspects of the program.4. Assessment of student learning and performance on professional identity, professional practice, and
program area standards.5. Evidence of the use of findings to inform program modifications.6. Distribution of an official report that documents outcomes of the systematic program evaluation, with
descriptions of any program modifications, to students currently in the program, program faculty, institutional administrators, personnel in cooperating agencies (e.g., employers, site supervisors), and the public.
BB. Students have regular and systematic opportunities to formally evaluate faculty who provide curricular experiences and supervisors of clinical experiences.
CC. Annual results of student course evaluations are provided to faculty.
DD. Written faculty evaluation procedures are presented to program faculty and supervisors at the beginning of each evaluation period and whenever changes are made in the procedures.
Philosophy of Program Evaluation The philosophical approach that guided our plan for program evaluation was to integrate three areas to create a comprehensive plan to uphold 1) the standards of the profession, 2) Lipscomb University’s ideals, and 3) student growth. The mission for the Lipscomb University Graduate Studies in Counseling is to provide an educational environment characterized by academic excellence and Christian faith. Counseling students are prepared to become competent, confident, committed and caring (the 4 C’s) professional mental health counselors in eight core areas of knowledge: professional orientation and ethical practice, social and cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation. This takes place both inside and outside of the classroom and is characterized by academic, professional, and personal growth. See Figure 3.
12
Figure 3: Diagram of Program Philosophy
Student Learning Objectives The Clinical Mental Health Counseling Program at Lipscomb University used our program philosophy to develop 15 student learning outcomes.
Educational Environment of Academic Excellence and Christian Faith
A.1.Students are prepared to become competent professional mental health counselors.
A.2. Students are prepared to become confidant professional mental health counselors.
A.3.Students are prepared to become committed professional mental health counselors.
A.4.Students are prepared to become caring professional mental health counselors.
Core Areas of Knowledge
B.1.Students gain academic and practitioner knowledge in professional orientation and ethical practice.
B.2. Students gain academic and practitioner knowledge in social and cultural diversity.
B.3.Students gain academic and practitioner knowledge in human growth and development.
B.4.Students gain academic and practitioner knowledge in career development.
13
Academic Excellence & Christian Faithprofessional mental health counselors who
are:competentconfidentcommittedcaring
Growthacademic
professionalpersonal
8 Core Areas of Knowledgeprofessional orientation and
ethical practicesocial and cultural diversityhuman growth and developmentcareer developmenthelping relationshipsgroup workassessment and testingresearch and program evaluation
B.5.Students gain academic and practitioner knowledge in helping relationships.
B.6.Students gain academic and practitioner knowledge in group work.
B.7.Students gain academic and practitioner knowledge in assessment.
B.8.Students gain academic and practitioner knowledge in research and program evaluation.
Student Growth
C.1.Students grow in academic knowledge of the counseling profession.
C.2.Students develop a professional counseling identity through involvement in local, state and national professional
organizations.
C.3.Students experience personal growth including awareness of personal issues, potential biases, identification of a
personal counseling theory, and strengths.
In order to monitor our effectiveness as a training program, we have implemented a comprehensive plan to provide continuous and systematic data that we review, consider, and evaluate to determine our areas of strength and address areas that could be improved. We believe this on-going process is critical in improving our program to meet the needs of a changing culture and profession.
The program is based on data collected at eight key points, structured around a student’s progress through the program: the application stage, orientation to the program, first semester, each pre-practicum semester, application to candidacy, each clinical semester, the last semester, and as alumni. Our system for program evaluation contains 26 different assessments, around 90 assessments during a typical student’s program cycle, and thousands of data points. We sought data from faculty, students, clinical site supervisors, alumni, employers of graduates, and an outside standardized test. See Figure 4.
14
Systematic Student, Faculty and Program Evaluation
Figure 4: Eight-Point Student, Faculty, and Program Evaluation
15
Internal Eval of Faculty/ Program
Course EvaluationsInternational Service
Trip Culture and Values Review
Internal Eval of Students
GradesProgress Review
Pre-Practicum Semesters
Internal Eval of Faculty/ Program
Course Evaluations
Internal Eval of Students
Skills EvaluationGradesProgress Review
Semester 1
Internal Eval of Faculty/ Program
Orientation Evaluation
Orientation
Internal Eval of Faculty/ Program
Stakeholder Program Evaluation
Annual Faculty Evaluations
Internal Eval of Students
Evaluation of Application to Determine Admission
Applicant Interview
Application/ Interview
External Eval of Program
Alumni SurveyEmployer Survey
Alumni
External Eval of Program
National CPCE Exam
Internal Eval of Faculty/ Program
Exit InterviewCourse EvalFinal Program Evaluation
Internal Eval of Students
Final Student ReviewGrades
Final Semester
External Site Supervisor Eval of Program
Mid-term Eval of Students
End of Semester Eval of Students
Internal Eval of Faculty/ Program
Group Supervisor EvalSite Supervisor EvalSite EvalCourse Eval
Internal Eval of Students
End of Semester EvalGrades
Clinical Semesters
Internal Eval of Faculty/ Program
Mid-Program EvaluationPracticum Orientation
Evaluation
Internal Evalof Students
Admission to Candidacy Review
Application to
Candidacy
Eight-Point Student, Faculty and Program Evaluation Plan The program evaluation plan for the Graduate Studies in Counseling Program revolved around three areas: eight evaluations of students within the program, twelve evaluations of the program within the program, and six evaluations of students and the program from external sources. See Figure 5. Some assessments were used multiple times during the program cycle. See Figure 4.
Figure 5: Internal and External Program Evaluation Assessments
Application and InterviewThe first point of data collection for evaluating the program comes from information provided by potential students from applications to the program as well as assessment data from the interview process. This includes self-reported information on ethnicity, faith and gender, undergraduate GPA, and GRE scores. Also, during the summer semester we conduct the annual stakeholder review of the program and each faculty member is evaluated. Since this conceptually gives us a starting point for the next academic year, we have placed those here.
New Student OrientationFor each new cohort admitted to the program, we hold an orientation early in the first semester. During the orientation, we distribute the counseling handbook and other materials, give an overview of the curriculum, and outline the clinical experiences and expectations. Students are asked to provide feedback on the orientation to help us make it more beneficial as the final agenda item for the orientation meeting.
First SemesterDuring the first semester, we conduct three internal evaluations of students and one student evaluation of faculty and facilities. The first semester in the program provides an important time for faculty to begin evaluating student skills, character, and grades and address any areas of deficiency that are identified. Since all first semester students are required to take the Introduction to Counseling Skills course, we conduct a skills evaluation for students in that course. The Progress Review Committee meets at the conclusion of each semester to review student dispositions, grades and
16
Clinical Site Supervisor Mid-Term Evaluations
Clinical Site Supervisor End of Semester
Evaluations
Site Supervisor Program Evaluations
Alumni Surveys
Employer Surveys
CPCE Exam
External Evaluation
of Program and Students
Stakeholder Program Review
Annual Faculty Evaluations
New Student Orientation Evaluation
Course Evaluations
Service Trip Culture and Values Review
Mid-Program Evaluation
Practicum Orientation Evaluation
Group Supervisor Evaluation
Student Site Supervisor Evaluation
Site Evaluation
Exit Interview
Final Program Evaluation
Internal Program Evaluation
of Program
Application Review
Applicant Interview
Skills Course Evaluation
Progress Review Committee Evaluation
Grades for each course
Student Candidacy Review
Clinical End of Semester Reviews
Final Student Review
Internal Evaluation
of Students
any faculty concerns for each pre-candidacy student. Students provide feedback about faculty and facilities through course evaluations.
Pre-Practicum SemestersFor each semester between the first semester and Practicum, students are evaluated through grades. As with every semester, any concerns with student behavior, grades, or dispositions are discussed by the Progress Review Committee. Many students opt to go on an international service trip before clinical work begins, so we have added that opportunity for feedback here although students may opt to go during their clinical semesters as well. Students provide feedback through course evaluations.
Application to CandidacyAll students formally apply for candidacy. The candidacy application form verifies that students have: rectified any undergraduate deficiencies; completed at least 12 hours of graduate work, including a minimum of four core classes; maintained a 3.0 GPA; received successful reviews by the departmental Performance Review Committee; and earned successful evaluations on the Professional Performance Review forms during the first two semesters. Students also complete a mid-program evaluation at the practicum orientation meeting, which falls at about this time in the program cycle.
Clinical SemestersStudents spend at least three semesters (one calendar year) registered for a clinical experience (Practicum, Internship 1, or Internship 2). These semesters provide opportunities to evaluate real-world student counseling skills, as well as academic, professional and personal growth. During these semesters, we require site supervisor feedback on the students and our program. To ensure high quality site placements, students evaluate and give feedback on placement sites and site supervisors.
Final SemesterBefore students graduate from the Graduate Studies in Counseling program at Lipscomb University, we conduct a Final Student Review to ensure that all students meet graduation requirements. Students complete the CPCE exam, participate in an Exit Interview, and complete the Final Program Evaluation form to give summative feedback on the program and their experience.
Alumni Finally, we assess data from those who have graduated from our program and those who employ graduates from our program. Specifically, we want to assess graduates professionally in the areas of competence, confidence, commitment to the profession and caring in their work as professional mental health counselors. We contact employers of our alumni to gather feedback on our graduates.
Meeting Program Goals and Objectives Through the Eight Point Assessment Plan Using the eight-point program evaluation plan and the three categories of evaluations (internal student, internal program and faculty, and external student and program) as a template, we aligned our program goals and objectives to the 26 different assessments we conduct during the program cycle. See Figure 6.
17
Meeting Program Goals and Objectives Program Goals Program Objectives
1 2 3 4 5 6 1 2 3 4 5 6
Internal Evaluation of Students
Application Review X X
Applicant Interview X X
Skills Course Evaluation X X X X X X
Progress Review Committee Evaluation X X X X X X
Grades (every semester) X X X X X X X X X X
Student Candidacy Review X X X X X X
Clinical End of Semester Reviews X X X X X X X X X X
Final Student Review X X X
Internal Program Evaluation
Stakeholder Program Evaluation X X X X X X X X X X X X
Annual Faculty Evaluation X X X X
New Student Orientation Evaluation X X X
Course Evaluations X X X
Service Trip Feedback X X X X X X X
Mid-Program Evaluation X X X X X X X
Practicum Orientation Evaluation X X
Group Supervisor Evaluation X X X X X
Student Site Supervisor Evaluation
Site Evaluation
Exit Interview X X X X X
Final Program Evaluation X X X X X X X
External Program and Student Evaluations
Clinical Site Supervisor Mid-Term Evals X X X X X X X X X X
Clinical Site Supervisor End Semester Evals X X X X X X X X X X
Site Supervisor Program Evaluations X
18
Alumni Surveys X X X
Employer Surveys X X X
CPCE Exam X X X X X X
Figure 6: Matching Assessments with Program Goals and Objectives
Program GoalsThe Graduate Studies in Counseling program promotes the following goals:
1. To attract and develop a high quality, diverse student body. 2. To increase student knowledge in the field of professional counseling.3. To facilitate competence in the practice of professional counseling.4. To serve and improve the quality of life in the local, state, national, and international community.5. To encourage an understanding and commitment to the scientist practitioner model.6. To facilitate an understanding of the individual characteristics that each student brings to the counseling process.
Program ObjectivesThe Graduate Studies in Counseling program promotes the following objectives:
1. To gain knowledge of the counseling profession and other related helping professionals.2. To gain knowledge of and become accountable to the ACA Professional Code of Ethics.3. To gain knowledge and competence in serving and advocating for culturally diverse populations, including assessment,
treatment planning, treatment, and outcome evaluation.4. To gain knowledge of eight core knowledge areas: professional orientation and ethical practice, social and cultural
diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation.
5. To gain knowledge of a wellness model of mental health.6. To gain knowledge in the foundations of research and inquiry.
To aid in our implementation of the program, we have created a calendar of tasks for each semester and a chart to delineate responsibilities. See Figure 7 and the following chart.
19
Program Evaluation Calendar
Figure 7: Program Evaluation Calendar
20
EndEVAL OF STUDENTSGradesProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview
INTERNAL PROGRAM EVALSAnnual Faculty EvaluationFaculty Supervisor Course
EvaluationsGroup Supervisor Course
EvaluationsStudent Mid-Program
EvaluationStudent Evaluation of Site
SupervisorStudent Evaluation of Site
EXTERNAL EVALSSite Supervisor EvaluationsEmployer SurveyCPCE Exam
MidEVAL OF STUDENTSReview of ApplicationsAdmission to Candidacy Review
INTERNAL PROGRAM EVALSAnnual Program Evaluation
Report meetingService Trip Feedback
EXTERNAL EVALSSite Supervisor Mid-Semester
Student Evaluations
BeginningINTERNAL PROGRAM EVALSPracticum Orientation
Evaluation
Summer Semester
EndEVAL OF STUDENTSGradesSkills Course EvalProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview
INTERNAL PROGRAM EVALSFaculty Supervisor Course
EvaluationsGroup Supervisor Course
EvaluationsStudent Mid-Program
EvaluationStudent Evaluation of Site
SupervisorStudent Evaluation of Site
EXTERNAL EVALSSite Supervisor EvaluationsAlumni SurveyCPCE Exam
MidEVAL OF STUDENTSAdmission to Candidacy Review
INTERNAL PROGRAM EVALSService Trip Feedback
EXTERNAL EVALSSite Supervisor Mid-Semester
Student EvaluationsSite Supervisor Program
Evaluation
BeginningINTERNAL PROGRAM EVALSNew Student Orientation
EvaluationPracticum Orientation
Evaluation
Spring Semester
EndEVAL OF STUDENTSGradesSkills Course EvalProgress Review CommitteeClinical EvaluationsFinal Semester Student ReviewFinal Semester Exit Interview
INTERNAL PROGRAM EVALSFaculty Supervisor Course
EvaluationsGroup Supervisor Course
EvaluationsStudent Mid-Program
EvaluationStudent Evaluation of Site
SupervisorStudent Evaluation of Site
EXTERNAL EVALSSite Supervisor EvaluationsCPCE Exam
MidEVAL OF STUDENTSReview of ApplicationsAdmission to Candidacy Review
EXTERNAL EVALSSite Supervisor Mid-Semester
Student Evaluations
BeginningINTERNAL PROGRAM EVALSNew Student Orientation
EvaluationPracticum Orientation
Evaluation
Fall Semester
Program Evaluation Responsibility Assignments Beginning of Semesters
WITHIN PROGRAM EVALUATIONNew Student Orientation Evaluation Clinical DirectorPracticum Orientation Evaluation Clinical DirectorStudent Mid-Program Evaluations Director of Program Evaluation
Mid-SemestersEVALUATION OF STUDENTSReview of Applications Department ChairAdmission to Candidacy Review Progress Review Committee
WITHIN PROGRAM EVALUATIONStakeholder Program Evaluation Report Meeting Director of Program EvaluationAnnual Faculty Evaluation Department ChairService Trip Feedback Service Trip Faculty
EXTERNAL EVALUATIONSSite Supervisor Mid- Semester Student Evaluations Clinical FacultySite Supervisor Program Evaluation Clinical Director
End of SemestersEVALUATION OF STUDENTSGrades All facultySkills Course Evaluation Skills Course FacultyProgress Review Committee Assessment Progress Review Committee Clinical Evaluation Practicum, Internship 1, and Internship 2 FacultyFinal Semester Student Review Department ChairFinal Semester Exit Interview Capstone Course Faculty
WITHIN PROGRAM EVALUATIONFaculty Course Evaluations Director of Institutional ResearchGroup Supervisor Evaluations Director of Institutional ResearchStudent Final Program Evaluation Director of Program EvaluationStudent Evaluation of Site Supervisor Clinical FacultyStudent Evaluation of Site Clinical FacultyFinal Exit Interview Capstone Faculty
EXTERNAL EVALUATIONSite Supervisor Final Evaluations Clinical FacultyEmployer Program Evaluation Clinical DirectorAlumni Program Evaluation Director of Program EvaluationCPCE Exam Capstone Faculty
21
Internal Evaluation of StudentsDescription of Data Collection Procedure Pre-Admission Our first review of students begins before admission to the program, during the application process. Applicants to the Graduate Studies in Counseling program submit the following application materials to be considered:
1. Application Form2. Application Fee3. Goals Statement4. Graduate Record Examination (GRE) Standardized exam score5. GPA6. Test of English as a Foreign Language (TOEFL) for international students7. Three letters of reference8. Official Transcript(s) from all schools attended
After submitting all application materials, admission materials are reviewed by members of the Graduate Studies in Counseling Admissions Committee, which consists of the Director of the Graduate Program, the Clinical Director of the Graduate Program, and two additional counseling faculty appointed by the Director.
Review of ApplicantsOur first program goal is, “To attract and develop a high quality, diverse student body.” To improve upon the quality of students, in the spring of 2014 we implemented an evaluation rubric to help in reviewing admission materials. This provided a baseline assessment of the quality and diversity of the students that applied to the program. We examined the variables of age, undergraduate GPA, GRE scores, race and ethnicity, faith or religious preference, and gender to assess quality and diversity. Thirty-six students submitted all materials necessary to apply for the Graduate Studies in Counseling Program for the fall 2015 semester. The mean age of applicant was X years with an average undergraduate GPA of 3.44 and an average GRE score of 151 in the verbal section and 146 in the quantitative section. Applicants self-identified data related to race/ethnicity and gender, but responses to both were optional. For the fall 2015 cohort, X% of applicants were female and X% male. Of the thirty-six applicants, X% chose not to disclose their ethnicity. Another X% identify as African American or black and X% identified themselves as Caucasian. Rubrics were also used to provide a fair and consistent evaluation of the applicants. Although scores were very consistent across reviewers and differences in scores were small, below are the top three categories (1-3) and the bottom three categories (14-16) across interviewees:
Caring
Cultural
Sensitivity
Competence
Commit
ment
Ability to be Reflective
Professio
nal
Presentation
Confi
dence
Ability to think Critically
Dependability
Relates to Others
Attending Skills
Ability to Listen
Verbal
Expression
Group B
ehaviors
Enthusiasm
Un
derstanding the Profession
4.71 4.68 4.56 4.56 4.48 4.48 4.46 4.44 4.43 4.43 4.41 4.38 4.37 4.32 4.32 4.27
23
To help us capture a snapshot of our typical student, we examined the variables of age, undergraduate GPA, GRE scores, race and ethnicity, faith or religious preference, and gender to assess quality and diversity. Twenty-five new students enrolled in classes during the fall and spring semesters of the 2014-2015 academic year. The mean age of accepted students was 27 years with an average undergraduate GPA of 3.54 and an average GRE score of 151 Verbal and 148.2 Quantitative. Applicants self-identified data related to race and ethnicity, faith or religious preference, and gender, but it is noteworthy that not all chose to provide that information on their application, especially in the category of religious preference (22 out of 25 chose not to provide, equaling 88%).
Figure 9: Gender Distribution of New Students
24
Figure : Ethnic Distribution of New Students
Description of Data Collection Procedure As part of our internal evaluation of counseling students admitted to the program, faculty reviewed students’ academic, professional and personal growth throughout the program. Specifically, we identified six separate types of faculty-conducted evaluations, and a minimum of one evaluation per student every semester they were enrolled in the program. See Figure 11. This systematic evaluation helped us identify problematic behaviors and concerns earlier in the program and more accurately evaluate student learning outcomes.
8 Core Knowledge Areas Mission Statement Growth
Ethics
Diversity
Development
Career
Helping
Group
Assessment
Research
Competent
Confident
Commitment
Caring
Academic
Professional
Personal
Skills Course Evaluation
X X X X X X X X X X
PRC Evaluation
X X X X X X X X
GradesX X X X X X X X X X
Candidacy Review
X X X X X X X
Clinical Review
X X X X X X X X X X X X X X
Final Student Review
X X X X X X X X X X
Figure 11: Assessments of Students and Student Learning Outcomes
Skills Course EvaluationThe Introduction to Counseling Skills course has been identified as an important course to begin evaluating counseling behaviors. It meets two of our program goals: to facilitate competence in the practice of professional counseling, and to facilitate an understanding of the individual characteristics that each student brings to the counseling process. Because the Skills course is required at the beginning of the program, it gives us an early evaluation of personal and professional
25
Figure : Age Distribution of New Students
behaviors in the first semester. This assessment improves gatekeeping for the profession at the beginning of the program. The faculty members of the course complete assessments rating counseling students from 1 (low) to 5 (high) on 28 questions. Questions are grouped in the following domains with the following average scores: classroom conduct (5.0), caring (4.83), professional responsibility (4.87), competence (4.78), and confidence and maturity (4.8). These scores represent improvements across all domains. Unfortunately, reporting for the skills course evaluation is somewhat incomplete. Three sections were provided in the 2014-15’ school year, but only one instructor (n=13) reported complete results. Another instructor was only able to report final course grades, which, it should be noted, were all A’s. Detailed scores were collected for the third course, but the report for these was apparently lost. Moving forward, skills course instructors should be encouraged to follow a uniform process for data collection and storage.
Progress Review Committee (PRC) EvaluationThe Progress Review Committee meets each semester to discuss student progress. Faculty for each course that a student enrolled until accepted for candidacy are required to complete the Professional Performance Review form assessing the following areas: open to new ideas, flexible, cooperates with others, accepts and uses feedback, aware of impact on others, effectively deals with conflict, accepts personal responsibility, expresses feelings effectively and appropriately, attends to ethical and legal considerations, and takes initiative, and is motivated. Faculty consider not only academic performance, but also professional, intrapersonal, and interpersonal skills and behavior. Informally, students are evaluated on the appropriateness of skills through interactions with faculty, staff, and other students.
After both the first and second semesters (plus additional semesters until students are accepted for candidacy), the professional performance review committee meets to review the Progress Review forms for each student. If a faculty member has concerns about the appropriateness of a student’s progress, course performance, or behavior, then the student receives a Notice of Concern letter. If students do not receive written notification, they are considered in good standing.
Grades for Each CourseOur second program goal is, “To increase student knowledge in the field of professional counseling,” and one way we address that goal is through academic knowledge gained through courses. All coursework in the university is assigned a letter grade (no plus or minus grades) as an assessment of student knowledge of course material. Students earn grades on an “A” (excellent), “B” (good), “C” (marginal) and “F” (failing) scale. An “I” (incomplete) may be given under special circumstances, but is computed as an “F” until the course is completed. Each letter is assigned a quality point used to determine Grade Point Average (GPA). At the beginning of each course, students receive a syllabus detailing expectations for the course and how grades will be assigned.
The curriculum for the program requires students to complete 46 hours in the following core courses: Theories of Counseling and Personality,Introduction to Clinical Mental Health,Introduction to Psychopathology and Adaptive Behavior, Lifespan Development,Research Methods and Statistics,Ethics and Professional Issues in Counseling,Integration of Psychology and Christianity Seminar, Introduction to Counseling Skills,Multicultural Issues in Counseling,Group Dynamics and Group Counseling,Career Counseling,Substance Abuse Counseling,Advanced PsychopathologyAssessment Techniques,Effective Counseling and Treatment Planning, andCapstone Seminar in Counseling. The program also requires students to complete a minimum of 9 hours clinical experience (Practicum, Internship 1, and Internship 2), and 6 hours of electives.
Grades are systematically reviewed through the Progress Review Committee Evaluations, Student Candidacy Review, and Final Student Review. Advisors review student grades each semester as needed, especially for each student with a performance improvement plan.
26
Student Candidacy ReviewAfter the student receives approval from the Progress Review Committee to proceed into the second semester of coursework and completes at least 12 hours of coursework, the student may apply for “Candidacy” which allows the student to continue progress in the program and pursue the clinical experience. For admission to Candidacy, the student satisfy the following requirements:
1. Complete all required undergraduate deficiencies if admitted on condition, including successful completion of a PIP leading to “Graduate Student” status.
2. Complete at least twelve hours of graduate work (in no less than two semesters). The twelve hours must include four of the following: COUN 5903 Introduction to Counseling Skills, COUN 5113 Introduction to Clinical Mental Health Counseling, COUN 5203 Introduction to Psychopathology and Adaptive Behavior, COUN 5103 Theories of Counseling and Personality, COUN 5703 Ethics and Professional Issues in Counseling, and COUN 6603 Advanced Psychopathology.
3. Maintain a 3.00 GPA on all courses taken toward the requirements for the degree with no incomplete grades.
4. Receive successful reviews by PRC and successful evaluations on Professional Performance Review forms during the first and second semesters.
5. File an “Application for Candidacy” form with Graduate Studies in Counseling which will initiate a formal review by the PRC. Applications are due each semester on the Monday after graduation by 4:00 p.m. to the departmental administrative assistant.
Clinical End of Semester ReviewsThe Lipscomb University Faculty Group Supervisor assesses each student enrolled in a Practicum or Internship clinical experience. At the end of the course, students are evaluated in the following domains: assessment and diagnostics, therapeutic intervention, supervision, ethical and professional behavior, strengths and opportunities for improvement. The evaluation form also documents that students meet the clinical hourly and direct service requirements.
Hourly Requirements. Practicum students are required to complete a minimum of 150 hours, with at least 40 hours in direct service. Internship students are required to complete a total (over 2 or more semesters) of 600 hours, with at least 240 in direct service. Our students exceeded the minimum hourly requirements (by 29.38% in Practicum), accumulated experience with groups each semester, and received an average of 43.02 hours of direct supervision each semester.
Clinical Hours Required
Hours
Average
Total
Required
Direct Hours
Average
Direct
Group Site Supervision
Faculty Supervision
Practicum
n=14
150 194.07 60
62.44 11.81 12.57 26.96
Last year 213 73.66 21.43 18.73 25.58
Difference -18.93 -11.22 -9.62 -6.16 1.38
Internship 1 & 2
n=26
300 386.1 120
149.25 49.21 15.21 26.93
Last year 322.9 133 33 23.7 25.59
Difference 63.2 16.25 16.21 -8.49 1.34
27
Evaluation Categories. The evaluation form for the clinical experiences rated students in four domains: assessment and diagnostics, therapeutic intervention, supervision, ethical and professional behavior. The site supervisors completed a nearly identical evaluation for the students, which will be reviewed in that section. The scale ranked students as excellent (4), competent (3), needs work (2) or very poor (1). Although, as expected, beginning Practicum students scored lower in all categories except supervision, the rank order of the categories was same. This means that Lipscomb students demonstrated strong characteristics in supervision, along with high ethical and professional behavior, but scored the weakest in assessment and diagnostics, regardless of which clinical experience they were enrolled.
Evaluation Categories for Practicum, Internship 1, and Internship 2 (n=40) All Clinical
Students
Practicum Internship
Supervision 3.76 3.86 3.65
Ethical and Professional Behavior 3.54 3.53 3.55
Therapeutic Interventions 3.34 3.19 3.48
Assessment and Diagnostics 3.17 3.05 3.29
Evaluation Questions. When reviewing students, it helped to examine the specific questions that students, overall, scored highest and lowest on to determine strengths and areas to improve. Students averaged a score of 3.75 or higher on 9 questions. The five lowest scoring questions had an average score below 3.15.
Questions with mean scores of 3.8 or higher (n=40) Mean Score
Ethical and Professional Behavior
Is tactful 3.80
Is cooperative with supervisor and peers 3.83
Shows maturity as expected of a student at the end of graduate training 3.84
Supervision
Openly receives suggestions and criticisms without defensiveness 3.90
Willing to carry a sufficient case load 3.84
Participates constructively in group supervision 3.90
Provides useful feedback to peers in group supervision 3.98
Therapeutic Interventions
Demonstrates appropriate interpersonal skills with clients 3.76
Recognizes and encourages clients’ strengths 3.77
Assessment and Diagnostics
28
none
Questions with lowest mean scores (n=40) Mean Score
Ethical and Professional Behavior
none
Supervision
none
Therapeutic Interventions
Demonstrates competence with marriage & family therapy skills 3.13
Implementation of crisis protocol when appropriate 3.12
Assessment and Diagnostics
Demonstrates competency in diagnosis and use of DSM 3.14
Shows skill in developing and implementing treatment plans 3.08
Demonstrates thorough clinical interviewing skills 3.11
The data indicates that students 1) demonstrate strong professional and ethical behaviors, especially with tact, cooperation, and maturity; 2) students participate well and give and receive feedback in supervision; and 3) they have good interpersonal skills with clients and recognize their client’s strengths. The areas that need work, though still rated competent, were 1) marriage and family interventions; 2) implementation of crisis protocol when appropriate; 3) competency using DSM for diagnosis; 4) developing and implementing treatment plans; and 5) clinical interviewing skills.
Final Student ReviewAs a final review to ensure that all academic requirements are met in order for the degree to be conferred, students complete the Petition to Walk in the semester in which all coursework will be completed for graduation. The Graduate Program Director reviewed and signed the form as verification. This final review was a checklist of requirements.
Remediation and Gatekeeping for the ProfessionIn student discipline and gatekeeping for the profession, we strive to come alongside students and use remediation to facilitate growth – academically, personally, and professionally. We believe many deficiencies can be successfully addressed by the student and faculty member within the context of a course or other experience (e.g., clinical experiences) without recourse to more formal procedures. Whenever it is not appropriate to only address concerns about a student within the context of a course or other experience—either because they are too serious or extend beyond a single course or experience—those concerns are brought to the attention of the Performance Review Committee (PRC) by completing a Professional Performance Review (PRR) form. This form outlines deficiencies in the student’s professional, intrapersonal, and interpersonal functioning needing to be addressed, and they are handled in the same manner as progress or annual reviews.
29
In the event that a student fails to meet program expectations, the student may be placed on probation and provided with a Performance Improvement Plan (PIP) to aid the student in overcoming performance deficiencies and return to good standing in the program. The plan is normally developed by the Progress Review Committee (except for PIPs related to program admission decisions, which are handled by the Graduate Studies in Counseling Admissions Committee) and outlines required activities and deadlines that must be achieved by the student, as well as consequences for failure to complete the PIP. Failure to successfully complete directives prescribed in the PIP could result in temporary or permanent dismissal from the program. If the PRC determines that a student’s performance deficiencies are of such a nature and severity that neither the student nor the program will benefit from a PIP, then a student may be immediately dismissed temporarily or permanently from the program.
A Performance Improvement Plan included 1) a detailed description of the performance/behavioral deficiencies, 2) a detailed explanation of the level of progress or performance required to remediate each deficiency—with reference to supporting documentation from written standards in the program (e.g., syllabi, practicum evaluation criteria), 3) the required actions to be taken and the documentation necessary to substantiate successful completion of the action steps, 4) the specific timeframe for completion of the plan, and 5) the consequences for failure to complete the plan. The plan is signed and dated by the student, the advisor, and Graduate Studies in Counseling Director, with a copy given to the student and another placed in the student’s file. The advisor and student discuss and establish mechanisms to aid the student in completing the PIP in the allotted time, and the advisor monitors the student’s plan progress and reports this progress to the PRC.
Based on a review of the student’s progress, the Progress Review Committee determines whether the student meets the terms of the Performance Improvement Plan. If the student demonstrates compliance with the terms of the PIP, the student is considered in good standing and no longer on probation. If the student fails to complete the PIP in accordance with its terms, the student was is subject to sanctions up to and including temporary or permanent dismissal from the program. The PRC communicates its determinations in writing to the student, with a copy placed in the student’s file. Final decisions regarding dismissal due to performance/behavioral deficiencies, when necessary, are made by the Graduate Studies in Counseling Director, upon recommendation by the Performance Review Committee.
Internal Program Evaluation of Students Review Through six evaluation opportunities (once admitted to the program) and a minimum of 33 faculty evaluations per student (for a typical 7 semester program cycle), faculty assessed students early and consistently throughout the program. This enabled us to identify developing patterns in students systematically and from a variety of faculty viewpoints. We were able to 1) evaluate students’ academic, professional and personal growth 2) assess the eight core knowledge areas, and 3) determine how well our students demonstrated caring, competence, confidence, and commitment to the profession.
Strengths
● Quality Students – We accept a diverse group of students related to ethnicity and age. We have many
indications that Lipscomb counseling students are achieving academic excellence, dedicated to personal growth, and are developing well professionally. Most of our students receive excellent grades, recommendations to continue progressing, and good to excellent faculty clinical evaluations. Most students progress through the program without any need for a program improvement plan or other remediation. Our students are open to criticism and constructively work with others to improve.
● Ethical and Professional Behavior – Students consistently demonstrate strong professional and ethical behaviors,
especially with tact, cooperation, and, this year, maturity. They also score high on valuing cross-cultural
30
sensitivity and respecting alternative worldviews, along with effective communication skills.
● Therapeutic Interventions – Students have good interpersonal skills with clients and recognize their client’s
strengths. They also score high at maintaining expected caseloads at their sites.
● Feedback in Supervision – Students continue to participate well in supervision, and they averaged high scores in
appropriately giving and receiving feedback.
● Experience Conducting Groups – Students are spending more hours facilitating group work in internship.
● Above and Beyond Minimum Requirements – Students are dedicated to learning both in and out of the
classroom. Practicum students on average exceed the number of required hours by 29.38%.
Improvements from Last Year
● Faculty Supervision – For the second straight year, students, on average, received more than the accreditation
standard minimum of 24 hours per semester of faculty supervision.
● Experience Conducting Groups – Despite decreases in the practicum group, internship students are spending
more than 49 hours, on average, conducting groups.
● Assessment, Diagnosis and Clinical Interventions – Although students continue to score lower overall in these
categories, the data indicates that our students continue to score competently in these domains.
Areas to Improve
● Implementation of Crisis Protocol when Appropriate –students scored the third lowest in this category.
● Reviewing Research – Although students read and consult current research for classes, the data indicates that
skill may not be fully used in their practitioner work.
● Defending Clinical Treatment Procedures – students create and follow treatment plans, but may lack the ability
to articulate the rationale and purpose.
31
Internal Program Evaluation of Faculty and ProgramDescription of Data Collection Procedure We believe that our students are our best sources to determine how well we are doing at providing an educational environment characterized by academic excellence and Christian faith where counseling students are prepared to become competent, confident, committed and caring professional mental health counselors in eight core areas of knowledge: professional orientation and ethical practice, social and cultural diversity, human growth and development, career development, helping relationships, group work, assessment and testing, and research and program evaluation. They also are most familiar with their own academic, professional, and personal growth. Therefore we designed 10 assessments during the program to solicit feedback from students. In addition, we also conducted two additional internal evaluations: the annual program evaluation review, which is presented to faculty during the summer and presents the results of this report, and annual faculty evaluations.
Annual Program EvaluationDuring the summer, the director of program assessment presents the comprehensive results of program evaluation to the faculty and other stakeholders. This presentation outlines the results and conclusions that are summarized in this report. Faculty discusses the strengths, areas to improve, and strategies to address these, which are summarized in the section, “Program Modifications Based on Review Data” below.
Annual Faculty EvaluationsEach faculty member receives a formal evaluation during August from the department chair. During this meeting, faculty goals in teaching, intellectual pursuits, and service are reviewed for the past year and upcoming year. For tenure and promotion, teaching is evaluated in the areas of: course design and management, collegiality and teamwork, and continued academician. Intellectual pursuits are evaluated by scholarly and intellectual activities. Service evaluation criteria include: academic support service, institutional service, church related service, community engagement, professional and discipline-specific service.
New Student Orientation FeedbackBefore the fall and spring semesters, students attend a mandatory program orientation meeting for new students. This meeting introduces students to faculty, presents an overview of the program, and gives students an informational packet, including a copy of the program handbook. At the conclusion of the orientation, students evaluate the orientation. The survey asks the following questions, which students rated from Strongly Disagree to Strongly Agree (a scale of 1 to 5):
Please rate your agreement with the following statements (n=48)
The orientation provided valuable information regarding the program and my professional options.
4.46
The orientation was well organized. 4.46
The orientation helped me feel informed about my choice of academic studies. 4.30
The individual providing the orientation was warm and enthusiastic. 4.63
The orientation provided me many opportunities to ask questions and get concerns resolved.
4.31
33
The orientation provided useful reference materials. 4.52
Overall, the orientation met my expectations. 4.42
We also asked the students for comments (to explain any of their answers above), strengths of the orientation, and areas to improve.
Comments:
“This was a fun night! The food was great, the atmosphere was welcoming and fun, and the organization of information into 4 C's was helpful and engaging. It was a great night!”
“I really appreciated the folders with so much information I will need throughout my education, all in one place. And I felt relieved to meet with my advisor so early on and know where to land when I have a concern.”
“I didn't really feel there was much of a question portion, mainly because it was later at night and people were tired.”
“The information provided will greatly assist in my planning while in the program and as I move into a career.”
Strengths:
● Formal and informal face time with students (e.g. the panel, the tours).
● Plenty of time for Q & A with faculty and staff.
● The interview time.
Areas to Improve:
● “Shorten the schedule by reducing waiting times for interviews and/or cutting the campus tour.”
● “More structure in the student panel discussion; it dragged on in many places.”
The new student orientation, according to student feedback, provided valuable information and included ample face time between prospects, current students, and faculty. Last year’s proposal to implement more Q&A was well received. However, for the second year in a row, attendees report that condensing it and adding more structure would be welcome improvements.
Course EvaluationsFor every course that students enroll, they have the opportunity to complete a course evaluation. This evaluation process is a university-wide process and is conducted at the university, not departmental, level. Email reminders encourage students to complete course evaluations and offer incentives. The evaluation asks students questions in the following domains: course as a whole, teacher effectiveness, interpersonal relationships, evaluation methods, classroom facilities, and additional course questions. Students rate on a 5-point Likert scale from 1 (poor) to 5 (excellent) with 3 as average. Data is tabulated for the department and individual faculty data is sent to the department chair for review.
In responding to course evaluations for all COUN courses (except clinical courses, which are described in the Group Supervisor section), students rated courses, facilities and faculty on a Likert scale from 1 to 5. Course evaluations had
34
about a 60% return rate. Students reported that they put a high level of effort into making their courses a meaningful experience (4.16) and spent about 3.46 hours a week on homework per course, increases of .16 and .15 respectively over the previous year. They rated the work required as above average (3.59), an increase of .08 over last year. Students indicated that, on average, before courses began, their interest level was a 3.8, but after taking the course, their interest level rose to a 4.52 Therefore, we know our students are motivated, spending about 10.38 hours a week outside of classes on homework if enrolled in three courses, and are becoming more interested in the subject matter as they learn more.
The charts below outline how students rated courses and faculty.
Course Evaluations (n=301) COURSE Scale 1-5
Requirements 4.45
How good was the course as a whole 4.5
How well was it organized 4.42
Tests & other assignments cover the material from the class 4.37
Beneficial choices of texts, other materials, and assignments 4.36
How challenging was it intellectually 4.28
Evaluations require critical thinking 4.43
Effective testing & assignments 4.31
Conducive university facilities to learning 4.26
Computer equipment to learning 4.3
Course Evaluations (n=301) PROFESSOR Scale 1-5
Patient and respectful 4.63
Relate this subject to the broader issues of life 4.67
Enthusiastic about the subject 4.66
Keeping appointments and designated office hours 4.62
Approachable 4.46
Encouraging 4.43
Effective interpersonal relationships 4.53
35
Direct and encourage open classroom discussion 4.64
Effectiveness of teacher 4.56
Fairly deal with students 4.71
Clearly communicate information and expectations 4.59
Stimulate critical thinking in writing, discussions, and testing 4.27
Returning graded material within a reasonable time 4.35
Stimulate interest and hold attention 4.31
Providing clear comments for improvement 4.27
Students, overall, scored course requirements, the course as a whole, and the critical thinking stimulated by the course well. Though still well above average, the lowest scoring areas were the university facilities and the intellectual challenge of the courses. In evaluating the professor, students believe their instructors to be patient and respectful, good at relating the subject to the broader issue of life, and enthusiastic about the subject. They are keeping appointments and designated office hours and are approachable, which all scored 4.45 or higher. For the second year in a row, the lowest scoring area was on providing clear comments for improvement (4.27). Other lower scoring areas were stimulating interest and holding attention, returning graded material within a reasonable time, and stimulating critical thinking. Clearly communicating information and expectations was one area of major improvement over last year, with ratings increasing by .25.
International Service Trip EvaluationStudents have the opportunity, but are not required, to travel internationally for academic and service learning trips. During the spring semester, students travel to Kenya for two weeks while enrolled in Multicultural Issues in Counseling and Integration of Psychology and Christianity Seminar (or a Special Topics course). Some years, students travel to Moldova. And during the summer, students may travel to Ghana for courses in Loss, Trauma and Resilience and Research Practicum.
Students taking courses that were part of an international service trip completed the university-wide course evaluation that all courses include. In addition, as part of a larger research project, students traveling to Kenya for study also had the opportunity to voluntarily participate in quantitative and qualitative research assessing reflective thinking, spirituality, and multicultural practices. Students completed three pre- and post-test questionnaires (Level of Reflective Thinking Survey, Daily Spiritual Experiences Scale, and the Multicultural Counseling Inventory) and submitted five photographs with a narrative explanation on the integration of faith and counseling. The results of this research are being submitted for publication and conference presentations, and thus are not included here.
Mid-Program EvaluationStudents evaluate their experience with the program at the mid-point in the program. They rate how well they think the program has prepared them academically in the eight core areas, how the program facilitates their growth academically, professionally, and personally, how they believe professional activities enhance their counselor effectiveness, and how they rate themselves on empathy, comfort with strong emotional expression, and their counseling skills. They also complete five open-ended questions:
36
1. At this point in the program, in what professional organization(s) and activities are you involved?2. What has been the most important element of this program (e.g., courses, assignments, activities, personal
work, other) 3. What has detracted from your growth and learning in the program?4. What would make the program better for upcoming students?5. What are you wiling to contribute to make the program better?
14 students completed the Mid-Program Evaluation. Although the sample size was small, it was interesting to note several results.
● On a scale from 1 (very poor) to 5 (exceptional) the three content areas that students felt the least prepared for
academically were: Career Development (3), Counseling Skills (3), and Research (3.4).
● On a scale from 1 (very poor) to 5 (exceptional) the three content areas that students felt the most prepared for
academically were: Emotional Expression (4.67), Professional Orientation and Ethical Practice (4.5), and Social and Cultural Diversity (4.33).
● Students report personal growth, (4.33), academic growth (4.2), and professional growth (4.33) and they believe
that professional activities enhance their effectiveness as a counselor (4.2).
● Some of the most important elements in the program that students identified included courses, practice/ role
play/ skills videos, assignments and activities, and the multicultural emphasis.
● The two biggest distractions to learning were 1) balancing school, work, and personal life, and 2) lack of
intellectual and academic rigor/expectations.
● Although there were no clear themes in what would improve the program, a few of the responses included:
more attention to burnout, inviting professional guest speakers to discuss the profession, and raising the academic standards.
Practicum Orientation EvaluationStudents are required to attend an orientation meeting before beginning Practicum. The orientation provides information for how to dress, behave and compile final notebooks. Students receive clinical handbooks, and requirements are reviewed.
Students enroll to begin Practicum in all three semesters – fall, spring, and summer. A practicum orientation is conducted before each semester, and students are asked to provide feedback. This is done both to assess the orientation and to model giving and receiving feedback, which students are expected to do in all clinical experiences. Eighteen respondents answered four Likert-scale (ranging from 0 to 100) questions asking:
1) How helpful was the information you received?2) How was the content and the amount of information (verbally and in your packets)?3) How would you rate the orientation overall?4) How well did the orientation meet your expectations?
Overall, students evaluated the fall and spring practicum orientations as helpful with an average score of 81.44 out of 100, with all answers falling between 64 and 100. Students also found the orientation to be helpful as it pertained to the information given (average score 76.28) and the verbal and written contents of the orientation (average score 78.61). Students reported that the orientation exceeded their expectations (average score 76.22).
Next, students are asked to complete the following open-ended questions and add any additional comments:
37
5) Please explain your answer on Question #4.6) Please list a few things that were especially helpful. 7) Please list a few things that would have improved the orientation.8) What suggestions do you have for changes to improve the orientation for future students? 9) What was the most memorable thing from the orientation?
Open-ended questions elicited the following comments. When asked about what was especially helpful, students mentioned the panel of students five times. Other items found to be helpful were the information regarding privacy/security/encryption and review of paperwork and documentation procedures. Five students mentioned the training videos as helpful tools, and students found that having time to meet with the faculty supervisors and supervision group members as helpful as well.
Students indicated that the orientation could be improved by being longer, supplemented with an additional meeting earlier in the semester, and/or being held at a time other than during finals week (4 mentions). Other improvements include more Q&A time (2 mentions), more information for setting up the iPods and using Time2Track (4), and providing more information leading up to the orientation (1 mention).
Finally, students found the Peer Panel and videos to be the most memorable portion of the orientation with 2 mentions each. Unfortunately, eight students did not respond to this question, and one student commented s/he would have left it blank but felt that it was important to emphasize how doing the orientation at the end of the semester was inconvenient and distracted from the helpfulness of the meeting.
Group Supervisor EvaluationTo more effectively evaluate faculty group supervision, we modified the course evaluation given by the university to more accurately reflect a practicum/ internship course. The new evaluation was implemented in the fall of 2013. Students rate on a 5-point Likert scale from 1 (poor) to 5 (excellent) with 3 as average. Over the fall, spring, and summer terms 72 students responded to the course evaluation, about a 50% return rate. Students indicated that they put a high level of effort into making it a meaningful experience (4.44), and reported spending an average of 3.73 hours per week on homework.
Course Evaluations (n=72) COURSE Scale 1-5
How good was the course as a whole 4.01
How challenging was it intellectually 3.95
Requirements 3.73
Work required 4.16
How well was it organized 3.47
Beneficial choices of texts, other materials, and assignments 3.76
Choice of locations for the course 3.36
38
Course Evaluations (n=72) FACULTY GROUP SUPERVISOR Scale 1-5
Stimulate critical thinking in writing, discussions, and testing 4.23
Direct and encourage open classroom discussion 4.21
Effectiveness: content application to professional practice 4.5
Relate this subject to the broader issues of life 4.29
Effectiveness of teacher 4.23
Enthusiastic about the subject 4.17
Effectiveness: open discussion 4.46
Effective interpersonal relationships 4.29
Patient and respectful 4.26
Effectiveness of faculty supervisor 4.3
Effectiveness: group facilitation 4.37
Encouraging of students 4.31
Stimulate interest and hold attention 3.93
Approachability 4.23
Constructive feedback 4.16
Encouraging 4.4
Patient-respectful 4.26
Approachable 4.26
Handling of inappropriate student behavior 4.45
Effectiveness of interpersonal relationships 4.26
Keeping appointments and designated office hours 4.33
Effectiveness of evaluation methods 3.96
Fairly deal with students 4.46
Clearly communicate information and expectations 3.51
Receptive to feedback 4.21
39
Time management 3.83
Providing clear comments for improvement 4.22
Effectiveness of interpersonal relationships with site supervisor 4.16
Returning graded material within a reasonable time 3.97
In addition, we began requiring clinical students to complete Arcinue’s Group Supervision Scale and Efstation, Patton, & Kardash’s Supervisory Working Alliance Supervisee Form at the conclusion of each clinical course. The data included (X number of) respondents in groups led by 4 faculty group supervisors, reporting at the end of two semesters. A more in depth analysis will be conducted as part of a larger research project, but as a preliminary understanding, the overall scores, averaging all faculty led groups together is included below:
Group Supervision Scale n=67 Scale 1-5
Group Safety 1-5 4.61
Skill Development 1-5 4.54
Group Management 4-20 17.85
Supervisory Working Alliance
Client Focus 1-7 6.28
Rapport 1-7 6.26
Based on the responses across the course evaluation, Group Supervision Scale and Supervisory Working Alliance students have a strong sense of group safety, agree their supervisors improve their skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at applying course content to professional practice, facilitating open discussions, encouraging students, dealing fairly with students, and handling inappropriate behavior.
However, almost every area scored lower than the previous year, some by a significant margin. First, students reported much lower clarity of communication of information and expectations, a 1.1-point decrease from last year. The second glaring area of need is stimulating interest and holding attention, which decreased from 4.81 to 3.93 this year.
Student Site EvaluationsEach semester that students enroll in clinical courses, they are asked to evaluate their practicum or internship site. They respond to three questions about the site to assess their difficulty acquiring hours, direct hours, and benefits of site activities. Rating their sites on a scale of 1 (strongly disagree) to 5 (strongly agree). According to the RedCap report on student site evaluations, only six new responses have been recorded since the Spring 2014 semester began. This may suggest a failure in the current data collection strategy, thus meriting a proposal for new data collection methods. Nonetheless, the average recorded responses for 2014-2015 were:
40
Evaluation of the Site n=75 Mean Last Year Mean This Year
I had no difficulty achieving my required total site hours. 4.25 3.67
There was no difficulty getting clients for my face-to-face minimum. 4 4.67
The activities completing my site hours were beneficial. 4.57 4.0
Overall, students had positive experiences with their sites, and they agreed that the activities completing their site hours were beneficial. For the second year in a row, students increasingly struggled to achieve their total hours, though they reported more success achieving the face-to-face minimum requirement.
Student Site Supervisor EvaluationsIn addition to evaluating clinical sites, students also provide feedback about their experiences with their site supervisors. Students answer six questions about their site supervisor to assess the following: goal-achievement, case supervision, weekly supervision, regularly scheduled supervision, supervisor availability, and in-vivo supervision. Questions are scored on a 5-point Likert scale.
Unfortunately, a failure in collecting data for this category seems to have occurred for 2014-15’ reporting. RedCap has only recorded one student response for the year, and these responses are recorded below. For future reporting, the 2013-14’ results should be retained for comparison with next year’s data. A new strategy for data collection is recommended to prevent future failures.
Evaluation of the Site Supervisor n=75 Mean
Last Year
Mean
This Year
The agreed-upon initial goals were achieved during my time there. 4.39 5
I received meaningful case supervision from my site supervisor. 4.45 5
Supervision occurred at least weekly. 4.48 5
Supervision occurred at a regularly scheduled time. 4.33 5
My site supervisor was available when needed. 4.53 4
My site supervisor provided in-vivo supervision. 36% 100%
In addition, we ask three open-ended questions to allow students to provide additional feedback. Two of the questions are: Please elaborate on your overall rating, and what special preparation is needed for students at this site? The student response recorded this year mentioned that the site’s culture pushed students toward excellence while also creating a nurturing environment for students; the student cited time management as the primary skill students should be prepared to use at this site.
The third open-ended question was: What type of students would not benefit well from this experience? Like the last two years, the student responded that a student who required a lot of direction, lacked initiative and motivation, and
41
was not a self-starter would not benefit from this experience.
Exit InterviewBy the time a student completes the program, multiple evaluations a semester have occurred. The Final Student Review is a short, informal assessment conducted by the department chair (who is also the Capstone professor) in the form of a Closing Conversation. The questions to the students generally include:
1. How would you rate the quality of instruction that you received in this program?
2. How would you rate the availability and helpfulness of the faculty in this program?
3. How would you rate the availability and helpfulness of the support staff?
4. How well do you believe you are prepared by the program to function as a licensed professional counselor?
5. Would you recommend this program to a friend, family member or colleague?
Final Program EvaluationThe Final Program Evaluation is identical to the mid-program evaluation. Students rate how well they think the program has prepared them academically in the eight core areas, how the program facilitates their growth academically, professionally, and personally, how they believe professional activities enhance their counselor effectiveness, and how they rate themselves on empathy, comfort with strong emotional expression, and their counseling skills. They also complete five open-ended questions.
11 students completed the Final Program Evaluation. Each was completing the 61-hour program with an average of X hours (range from Y-Z) in an average of X semesters (ranging from Y-Z semesters). While last year, all respondents completed the program in 7 to 9 semesters, this year, X completed the program in Y semesters and about X% completed the program in X semesters or less. Although the sample size was small, it was interesting to note several results:
● On a scale from 1 (low) to 10 (high) the two content areas that students felt the least prepared for academically
were: Career Development (4.5) and Assessment (6.8). These were the same bottom two last year and the year before that. Both Career and Assessment dropped for the second consecutive year.
● On a scale from 1 (low) to 10 (high) the two content areas that students felt the most prepared for academically
were: Professional Orientation and Ethical Practice (10), and Helping Relationships (10).
● The program highly promoted student growth, with personal, academic, and professional growth each receiving
“exceptional” ratings.
● 10 of the 11 participating graduating students declined to rate their counseling skills at this final point in the
program.
● The Practicum and Internship experiences and other courses were the most often listed elements most
important in the program. Multiple respondents also emphasized experiential components in class, such as mock sessions.
● The most suggested learning detractor was online class. The degree plan was also mentioned, with one student
complaining that had s/he taken Research Methods before practicum and internship, s/he would have been better prepared.
42
● Things that would make the program better included more emphasis and preparation for the NCE, less busy
work for the online classes, better job placement assistance, and more opportunities for scholarships and grants, i.e. helping students minimize debt.
Internal Program Evaluation of Faculty and Program Review Students have opportunities to let us know what they think throughout the program. They provide feedback after the New Student Orientation, through in course evaluations, from international service trips, at Mid-Program, after the Practicum Orientation, for each Lipscomb clinical group supervisor, about their Practicum and Internship sites, about site supervisors, during the Exit Interview, and at the end of the program. In addition to the ten internal program evaluations from students, faculty receives feedback on teaching, scholarly activity, and service during an annual review. We summarize all the evaluation information in the Stakeholder Program Review (discussed at the end of the report).
The following is a summary of these internal reviews of our program strengths and challenges.
Strengths
● Timely Completion – Most students (65%) complete the program in seven or eight semesters.
● Courses – Students, overall, scored course requirements, the course as a whole, and the organization of the
course highly.
● Professors – In evaluating the course professor, students rated their instructors patient and respectful, good at
relating the subject to the broader issue of life, and enthusiastic about the subject. They kept appointments and designated office hours and were approachable.
● Faculty Group Supervisors –Students have a strong sense of group safety, agree their supervisors improve their
skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at stimulating critical thinking, encouraging open discussions, applying the content to professional practice and relating the subject to broader issues of life. The supervisors are effective teachers, enthusiastic about their supervision and effective at open discussion.
● Orientations – The orientations for new students and Practicum continue to provide valuable information and
are helpful to students. The new student orientation, according to student feedback, provided valuable information and was well organized. It was perceived to be fun, friendly, warm and welcoming. At the Practicum orientation, students found the student panel, privacy and security information, and review of paperwork and documentation to be especially helpful.
● Knowledge – The content areas that students report feeling the most prepared for academically include:
Professional Orientation and Ethical Practice, Group, and Helping Relationships.
● Student Growth – Students perceive growth that continues throughout the program. Students reported high
levels of personal growth, academic growth, and professional growth at mid-program and even higher levels at the end of the program.
● Clinical Experiences – The Practicum and Internship experiences were the most often listed elements most
important in the program. Overall, students had positive experiences with their sites and site supervisors, especially when it came to supervisor availability and meeting weekly for supervision.
43
Improvements from Last Year
● Mentoring – We incorporated students further along in the program at the orientations to provide their
experience and answer questions. Students responded very positively.
● Orientations – While both the Program Orientation and the Practicum Orientations are perceived as too dense
last year, this year, students reported that they were informative, better organized, and more fun.
● Program Organization – The program is perceived as more organized now.
● Professional Involvement – Last year, students who were least involved in professional organizations saw less
value of professional activities in enhancing their effectiveness as a counselor. We now require membership in the American Counseling Association and encourage state and local professional involvement, and students report a higher value of professional activities.
Areas to Improve
● Knowledge – Career and Assessment continue to be the lowest scoring content areas, and both had lower scores
than last year.
● Facilities and Locations – Students report a strong dislike for the classrooms and furniture in Ward. They also
reported mixed responses to the location of clinical supervision groups (inconsistency in scoring, which may reflect the different locations). Students also repeatedly mentioned parking challenges.
● Library – Students want more library resources, better electronic journal access through databases, and
individual and group graduate study places.
● Grading and Feedback – Returning graded material within a reasonable time and providing clear comments for
improvement consistently has lower scores. Students also indicated that professors are not always clearly communicating information and expectations.
● Integrating Technology – Another low-scoring question was about the importance of computer equipment to
learning. Professors may need to address integrating more relevant technology.
● Advising – Students mentioned that they would like better advising. While not a theme across assessments, this
was mentioned a few times.
● Program Benefits –Students indicated disapproval of the discrepancy in benefits between the mental health
counseling and marriage and family therapy programs.
● Student Challenges – Program faculty should be aware that students report struggling to balance school, work,
and personal life, and they feel the financial pressure of graduate school.
44
External Evaluation of Program and Students Description of Data Collection Procedure
8 Core Knowledge Areas Mission Statement Growth
Ethics
Diversity
Development
Career
Helping
Group
Assessment
Research
Competent
Confident
Commitment
Caring
Academic
Professional
Personal
Site Supervisor Mid-Semester Eval
X X X X X X X X X X X X X
Site Supervisor End Semester Eval
X X X X X X X X X X X X X
Site Supervisor Program Eval
X X X X
CPCE Exam X X X X X X X X
Alumni Survey X X X X
Employer Survey X X X X
Evaluation Categories for Practicum, Internship 1, and Internship 2 (n=78)
All Clinical
Students
Last Year All Clinical Students
Practicum Internship
Supervision 3.66 3.86 3.50 3.74
Ethical and Professional Behavior 3.55 3.79 3.43 3.60
Therapeutic Interventions 3.41 3.65 3.28 3.47
Assessment and Diagnostics 3.38 3.66 3.29 3.42
Evaluation Questions. When reviewing students, it helped to examine the specific questions that site supervisors ranked students, overall, highest and lowest on to determine strengths and areas to improve. Students averaged a score of 3.68 or higher on 10 questions, all within the categories of supervision and ethical and profession behavior. The eight lowest
46
scoring questions had an average score below 3.3, and were mostly in the categories of assessment and therapeutic interventions.
Questions with mean scores of 3.68 or higher (n=78) Mean Score
Supervision
Participates constructively in group supervision 3.71
Openly receives suggestions and criticisms without defensiveness 3.72
Prompt and prepared for supervision meetings 3.70
Uses feedback to modify subsequent counseling behavior 3.69
Self-examines 3.72
Ethical and Professional Behavior
Is enthusiastic 3.74
Is able to work independently 3.69
Prompt and dressed appropriately for client appointments 3.73
Is dependable 3.68
Is tactful 3.76
Therapeutic Interventions
none
Assessment and Diagnostics
none
Questions with lowest mean scores (n=78) Mean Score
Supervision
none
Ethical and Professional Behavior
Consults and integrates research to develop a foundation for clinical work 3.26
Therapeutic Interventions
Willing to take appropriate risks in assessment & treatment 3.22
Demonstrates competence with child interventions 3.14
47
Demonstrates competence with group interventions 3.26
Implements crisis protocol appropriately when needed 3.14
Demonstrates understanding of site crisis protocol 3.29
Assessment and Diagnostics
Makes accurate diagnoses within 2-3 sessions 3.26
Competent use of instruments 3.25
The data indicates that students 1) demonstrate strong supervisee skills, especially with participation in supervision, receiving suggestions and criticism, and being prompt and prepared; and 2) have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently. Areas that need work, though still rated competent, included 1) implementation of crisis protocol; 2) demonstrating competence with child interventions; 3) and taking appropriate risks in assessment and treatment.
Site Supervisor Program EvaluationsLicensed practitioners working in the counseling profession who supervise our students have a unique and valuable perspective on the program. They witness the academic knowledge and skills that students have developed, as well as the growth of students during clinical experiences. Therefore, we sought their evaluation of our program as well.
We asked site supervisors to complete a web-based survey. From that survey we learned that respondents had supervised an average of X Lipscomb counseling students since the counseling program inception in 2007, with a range from Y-Z students. The data revealed that X (Y%) have been supervising Lipscomb students for one to two years, and X (Y%) have been supervising Lipscomb students for three or more years.
Preparedness of Students. We asked site supervisors on a scale from 0 to 100 (0 = not prepared, 50 = adequate, and 100 = extremely prepared) how prepared Lipscomb students seemed to be compared to other university counseling students. They rated us above adequate at 66. Last year was a nearly identical rating at 65.5 (scores ranged from 22-96). Next, we asked what suggestions the respondents might have to help our students become more prepared. While there were not clear themes, respondents suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Other suggestions included more practical/ experiential/ role play experience and students demonstrating more flexible availability at their site.
Cooperation and Communication Between Lipscomb and Site. The program was rated much higher, however, when we asked about the cooperation between the Lipscomb program faculty and site supervisors (0 = poor, 50 = okay, 100 = excellent), which resulted in an average of 83.9. The average rating of respondents for communication between the Lipscomb program faculty and site supervisors was rated at 80.6. We asked for suggestions that could enhance communication. The largest category of those answers, 46% of responses, suggested nothing or keeping things the same. Interestingly, one said, “Sometime all the emails are overwhelming,” while another said, “Continue to send out the newsletter with opportunities for training for supervisors.”
Employability of Students. Finally, we wanted to know how confident site supervisors would be about hiring a Lipscomb graduate (0 = no, 50 = somewhat, 100 = absolutely). The average score indicated confidence in hiring Lipscomb students, at 75.5. Next, we asked why or why not. A few of the comments included:
48
● The students we have had have been great. We have carefully screened them for our program. While I think of
Lipscomb as a conservative university, the students we have had have been open to a variety of views and have worked with clients with a variety of backgrounds with openness and care.
● Students are professional and eager to learn. They appear to have a solid understanding of ethics and the
importance of building their therapeutic orientation.
● They seem to have the academic background and are more than willing to get the experience needed to work in
the field.
● We have hired some of your students.
Survey Question n=13 Average Score Last Year
Average Score
Compared to other counseling students from other universities at the start of practicum, how prepared do
Lipscomb Counseling students seem to you?
65.5 66
How would you rate the cooperation between the Lipscomb program faculty and you?
83.9 83.3
How would you rate the communication between the Lipscomb program faculty and you?
80.6 81.5
Given your impression of the Lipscomb University counselor training program, how confident would you feel hiring
Lipscomb graduates for your agency?
75.5 81.6
CPCE ExamEvery student takes the Counselor Preparation Comprehensive Examination (CPCE) during the Capstone class in the last semester of the program. The CPCE is designed to assess counseling students’ knowledge in eight core knowledge areas. Although the categories have slightly different titles, they closely correspond to the eight Core Areas of Knowledge identified by our accrediting body. The CPCE categories are: professional orientation and ethics, social and cultural foundations, human growth and development, career and life development, helping relationships, group work, appraisal, and research and program evaluation.
Our sample sizes were small (X in the fall and Y in the spring) and thus subject to extreme scores significantly influencing the mean. However, when comparing each group’s mean for each content area to the national mean (n=602), we were able to determine the trends which students demonstrated strength and areas needing improvement. Students, on average, scored above the national mean in the areas of group work, social and cultural foundations, helping relationships, research and program evaluation, and professional orientation and ethics. Students scored, on average below the national mean in appraisal, human growth and development, and career and life development.
Core Area LU Mean National Mean Difference
Group work 13.04 10.58 2.46
Social and cultural foundations 10.835 8.57 2.265
49
Helping relationships 12.54 10.32 2.22
Research and program evaluation 10.365 9.6 0.765
Professional orientation and ethics 12.46 11.97 0.49
Appraisal 8.71 10.1 -1.39
Human growth and development 10.885 12.37 -1.485
Career and life development 7.565 10.57 -3.005
Alumni SurveyIn order to gauge how effective the counseling program is at equipping current professionals who graduate from the program, as well as to collect data regarding licensure status and further education, we solicit alumni feedback. A link for an online survey is sent to all alumni of the counseling program for whom the university has contact email addresses. We seek to determine how alumni rate themselves on the qualities of caring, competence, commitment to the profession, and confidence in professional abilities. Further, we want to know what alumni consider the most beneficial aspect of the program. In addition, we ask alumni to list all licenses and credentials and if they have taken and passed the three exams required for licensure in Tennessee: the National Counselor’s Exam, the National Clinical Mental Health Counseling Exam, and the Tennessee Jurisprudence exam.
Respondents to the survey included 66 graduates of the graduate counseling program, both the professional counseling (the old program) and the clinical mental health counseling (the new program). From those respondents, 13 took the NCE, 5 took the NCMHCE, and 8 took the Tennessee Jurisprudence Exam, the three exams required for licensure in the state of Tennessee. All passed the NCE and NCMHCE, and five of eight passed the Tennessee Jurisprudence Exam.
We asked alumni to rate how instrumental the counseling program at Lipscomb was in developing the characteristics of caring, professional competence, commitment to the profession, and confidence in professional abilities. They evaluated each question from 1 (no influence) to 10 (very influential). In descending order, alumni rated competence (7.37) first, then caring (7.27), commitment to the profession (7.07), and confidence (6.27).
Alumni reported to us what they perceived as the most beneficial aspects of the program. These answers were grouped into three themes in order of significance: caring faculty and staff; courses, especially practicum and internship; and preparation for the National Counselor’s Exam and licensure. Some examples of the responses included:
Caring Faculty and Staff:
“The faculty's commitment and concern in my growth and development as a counselor.”
“Some of the professors who were so great at teaching and were passionate about the field.”
“…Teachers & staff members who were genuinely interested in a student's personal achievement & therefore made themselves readily available to help students succeed in assignments & classes when the student was struggling.”
“The availability of teachers and their willingness to connect personally with students and help however they could.”
50
Courses:
“The internship and practicum were by far the most beneficial, assuming you have great supervisors.”
Preparation for NCE Exam and Licensure:
“Good preparation for NCE and NCMHCE. Diagnosis covered well, ethics covered well…”
Employer SurveyOne of the best sources of feedback for how Lipscomb counseling graduates are equipped comes from those who employ them. We polled employers to learn how they view our graduates. The sample size was extremely small (n=5), but we did gather some valuable data.
The employers had hired from 0 to 6 graduates, with an average of 3.4. We asked them, “Compared to other counseling graduates from other universities, how prepared for the profession do Lipscomb Counseling graduates seem to you?” On a scale from 1 to 100, with 50 being equally prepared, the mean was 79 (range from 60 to 90). All respondents were moderately or extremely confident about hiring a Lipscomb counseling graduate.
Employers rated their Lipscomb counseling graduate employees on the four C’s from our mission statement Again, the sample size was small this year and last year, but employers rated us higher in all four categories:
This Year Last Year
● Caring 94.2 (90.33)
● Competence 85.4 (80.66)
● Commitment to the Profession 83.75 (78.22)
● Confidence 85.4 (74.55)
To summarize, employers rated this program’s graduates as well above average in preparedness, and all expressed confidence in hiring a Lipscomb graduate. They scored alumni very high in caring and high in competence, professional commitment and confidence.
Some of the comments included:
● At the beginning of the program, provide students with a clear picture of the licensing process and post-master's
job market including income potential, etc.
● Additional exposure to psychological assessment would be beneficial, including personality assessment (PAI,
MMPI) and cognitive assessment (WAIS, WASI).
● Students can benefit by knowing various intervention strategies for dealing with Anxiety, Depression, Addiction
etc.
External Evaluation of Program and Students Review
Strengths
● Knowledge Areas – Students scored well on CPCE in the areas of Group Work, Social and Cultural Foundations,
and Helping Relationships.
51
● Students as Supervisees – students demonstrate strong supervisee skills, especially with participation in
supervision, receiving suggestions and criticism, and being prompt and prepared; and students have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently.
● Preparation for the Profession – Alumni indicated that practicum and internship were helpful and that they felt
prepared for the National Counselor’s Exam and licensure; in fact most who attempted, passed the three exams required for licensure in Tennessee. Alumni and employers rated the competence of Lipscomb graduates just below their caring, and employers indicated that Lipscomb students are more prepared than students from other universities.
● Employability – Site supervisors report that they are highly confident about hiring Lipscomb counseling
graduates, and employers report they are moderately to extremely confident about hiring a Lipscomb counseling graduate.
● Cooperation and Communication – Lipscomb faculty supervisors cooperate and communicate well with clinical
site supervisors.
Areas of Improvement
● Professional Identity – Employers indicated an increase in Lipscomb alumni involvement in professional
activities. By requiring students to maintain membership in the American Counseling Association and encouraging involvement in other professional activities, the community is noticing more commitment to the profession.
● Assessment and Diagnosis – While still rating these categories lowest, site supervisors did not score Lipscomb
students as weak as last year overall in the assessment and diagnostics category.
● Student Preparedness – We did receive the level of feedback this year about students need for more experience
through role-plays and personal counseling experience. We now incorporate mock counseling sessions in the Introduction to Skills course and highly encourage students to seek their own personal counseling.
Areas to Improve
● Integrating Research – Site supervisors are not seeing strong evidence that our students are grounded in the
current research.
● Assessments and Interventions – Site supervisors reported students lower in taking appropriate risks in
treatment and making accurate diagnoses within 2-3 sessions. They also suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Employers also noted that assessment and intervention strategies could be improved.
● Core Knowledge Areas – CPCE scores indicate that students score lower than the national average in career and
life development, human growth and development, and appraisal.
● Student Confidence – Alumni and employers rate confidence lower or lowest of the 4 C’s.
52
Program Modifications Based on Review DataDuring a typical program cycle for each student (2 years, 7 semesters, and 3 semesters of clinical experience), we have implemented nearly 90 assessments of students, the program, and the faculty. The Eight-Point Student, Faculty, and Program Evaluation Plan provides a broad approach for assessment, including internal assessments of students, internal assessments of the program and faculty, and external assessments of the students and program. We have a strong vision combining our mission statement, eight core areas of knowledge, and student growth. The curriculum more than adequately meets accreditation standards, reflects our uniquely Lipscomb University mission, and offers a range of electives to help students gain specialized knowledge and experience, including international service trips and a play therapy specialization. By proactively seeking data from students, faculty, site supervisors, employers, alumni and others, we have a solid sense of the strengths of this counseling program and areas to improve.
Overall Program Strengths
Internal Evaluation of Students
● Quality Students – We accept a diverse group of students related to ethnicity and age. We have many
indications that Lipscomb counseling students are achieving academic excellence, dedicated to personal growth, and are developing well professionally. Most of our students receive excellent grades, recommendations to continue progressing, and good to excellent faculty clinical evaluations. Most students progress through the program without any need for a program improvement plan or other remediation. Our applicants to the program present themselves in alignment with our program philosophy: high in caring, competence, and commitment to the profession. The evaluation from the Introduction to Counseling Skills class echoes this, since students scored highest in caring and professional responsibility.
● Ethical and Professional Behavior – Students consistently demonstrate strong professional and ethical behaviors,
especially with tact, cooperation, and dependability. They also score high on valuing cross-cultural sensitivity and respecting alternative worldviews, along with effective communication skills.
● Therapeutic Interventions – Students have good interpersonal skills with clients and recognize their client’s
strengths. They also score high at maintaining expected caseloads at their sites.
● Feedback in Supervision – Students continue to participate well in supervision, and they averaged high scores in
appropriately giving and receiving feedback.
● Experience Conducting Groups – Students are spending more hours facilitating group work across all clinical
experiences.
● Above and Beyond Minimum Requirements – Students are dedicated to learning both in and out of the
classroom. Practicum students on average exceed the number of required hours by 142%.
Internal Evaluation of Program and Faculty
● Timely Completion – Most students (65%) complete the program in seven or eight semesters.
● Courses – Students, overall, scored course requirements, the course as a whole, and the organization of the
course highly.
53
● Professors – In evaluating the course professor, students rated their instructors patient and respectful, good at
relating the subject to the broader issue of life, and enthusiastic about the subject. They kept appointments and designated office hours and were approachable.
● Faculty Group Supervisors –Students have a strong sense of group safety, agree their supervisors improve their
skills, report that supervisors almost always have a client focus, and have a strong rapport with their supervisors. They also indicate good group management. They highly rate the course as a whole, as well as the intellectual challenge. The group supervisors are exceptionally good at stimulating critical thinking, encouraging open discussions, applying the content to professional practice and relating the subject to broader issues of life. The supervisors are effective teachers, enthusiastic about their supervision and effective at open discussion.
● Orientations – The orientations for new students and Practicum continue to provide valuable information and
are helpful to students. The new student orientation, according to student feedback, provided valuable information and was well organized. It was perceived to be fun, friendly, warm and welcoming. At the Practicum orientation, students found the student panel, privacy and security information, and review of paperwork and documentation to be especially helpful.
● Knowledge – The content areas that students report feeling the most prepared for academically include:
Professional Orientation and Ethical Practice, Group, and Helping Relationships.
● Student Growth – Students perceive growth that continues throughout the program. Students reported high
levels of personal growth, academic growth, and professional growth at mid-program and even higher levels at the end of the program.
● Clinical Experiences – The Practicum and Internship experiences were the most often listed elements most
important in the program. Overall, students had positive experiences with their sites and site supervisors, especially when it came to supervisor availability and meeting weekly for supervision.
External Evaluation of the Program and Students
● Knowledge Areas – Students scored well on CPCE in the areas of Group Work, Social and Cultural Foundations,
and Helping Relationships.
● Students as Supervisees – students demonstrate strong supervisee skills, especially with participation in
supervision, receiving suggestions and criticism, and being prompt and prepared; and students have high ethical and professional behaviors, such as enthusiasm, maintaining appropriate working relationships, and an ability to work independently.
● Preparation for the Profession – Alumni indicated that practicum and internship were helpful and that they felt
prepared for the National Counselor’s Exam and licensure; in fact most who attempted, passed the three exams required for licensure in Tennessee. Alumni and employers rated the competence of Lipscomb graduates just below their caring, and employers indicated that Lipscomb students are more prepared than students from other universities.
● Employability – Site supervisors report that they are highly confident about hiring Lipscomb counseling
graduates, and employers report they are moderately to extremely confident about hiring a Lipscomb counseling graduate.
54
● Cooperation and Communication – Lipscomb faculty supervisors cooperate and communicate well with clinical
site supervisors.
Strengths Indicated from All Internal and External Evaluations1. Students – The students are clearly a strength of the program. We are recruiting and retaining diverse
and quality students who demonstrate strong professional and ethical behavior, are motivated to learn and do more than minimum requirements, and both give and receive feedback in supervision. They strongly rate personal, academic and professional growth at mid-program, and growth continues throughout the program. Most receive excellent grades and graduate with a minimum of 61 hours in 8 semesters or less. Most importantly, alumni pursue licensure, pass the required exams for licensure, and are viewed as employable and better prepared than other universities.
2. Cultural Diversity – The program values and admits diverse students, using a broad definition including (among other areas) ethnicity, age, and gender. Students score high on valuing cross-cultural sensitivity and respecting alternative worldviews at their clinical sites and indicated that Social and Cultural Diversity was on of the areas in which they felt most prepared. They also scored well in this area of the CPCE exam.
3. Clinical Experience – According to employers, alumni, current students, and site supervisors, students gain valuable experience in Practicum and Internship, including individual, group, and professional training experience.
4. Courses – Students are prepared academically for the profession. They rate courses highly as good overall and well-organized, and they pass national exams for state licensure at very high rates.
5. Faculty and staff – Relationships with program employees (full-time faculty, adjunct faculty, and office staff —many listed by name) are repeatedly listed as important and helpful. Professors are patient and respectful, good at relating the subject to the broader issue of life, and enthusiastic about the subject. Faculty group supervisors create group safety, improve supervisors’ skills, have a client focus, and maintain a strong rapport with their supervisees, and they cultivate cooperation and communication with site supervisors.
6. Core Knowledge Areas – Students continue to feel most prepared in the content areas of Professional Orientation and Ethical Practice, Group, and Helping Relationships.
7. Caring and Competence – Students and alumni of the program are, generally, viewed as demonstrating increasingly high levels of caring and counseling competence.
Overall Program Areas to Improve
Internal Evaluation of Students
● Confidence and Maturity – first semester students continue to score lower in this category.
● Reviewing Research – Although students read and consult current research for classes, the data indicates that
skill may not be fully used in their practitioner work.
● Defending Clinical Treatment Procedures – students create and follow treatment plans, but may lack the ability
to articulate the rationale and purpose.
55
Internal Evaluation of Program and Faculty
● Knowledge – Career and Assessment continue to be the lowest scoring content areas, and both had lower scores
than last year.
● Facilities and Locations – Students report a strong dislike for the classrooms and furniture in Ward. They also
reported mixed responses to the location of clinical supervision groups (inconsistency in scoring, which may reflect the different locations). Students also repeatedly mentioned parking challenges.
● Library – Students want more library resources, better electronic journal access through databases, and
individual and group graduate study places.
● Grading and Feedback – Returning graded material within a reasonable time and providing clear comments for
improvement consistently has lower scores. Students also indicated that professors are not always clearly communicating information and expectations.
● Integrating Technology – Another low-scoring question was about the importance of computer equipment to
learning. Professors may need to address integrating more relevant technology.
● Advising – Students mentioned that they would like better advising. While not a theme across assessments, this
was mentioned a few times.
● Program Benefits –Students indicated disapproval of the discrepancy in benefits between the mental health
counseling and marriage and family therapy programs.
● Student Challenges – Program faculty should be aware that students report struggling to balance school, work,
and personal life, and they feel the financial pressure of graduate school.
External Evaluation of Program and Students
● Integrating Research – Site supervisors are not seeing strong evidence that our students are grounded in the
current research.
● Assessments and Interventions – Site supervisors reported students lower in taking appropriate risks in
treatment and making accurate diagnoses within 2-3 sessions. They also suggested more preparation with treatment planning, diagnosis, progress notes and knowing the DSM. Employers also noted that assessment and intervention strategies could be improved.
● Core Knowledge Areas – CPCE scores indicate that students score lower than the national average in career and
life development, human growth and development, and appraisal.
● Student Confidence – Alumni and employers rate confidence lower or lowest of the 4 C’s.
Areas to Improve Indicated by All Evaluations1. Incorporating Research – Students do not demonstrate high skill in incorporating research into their
clinical work and supervision.
2. Assessment, Diagnosis and Treatment Interventions – Students score lowest in these areas. They also score themselves lower in Assessment. Faculty and site supervisors reported that assessment and diagnosis was the area that needed the most improvement.
56
3. Student Confidence – Of the 4 C’s this was consistently lower. Beginning with the skills evaluation and continuing through alumni and employer evaluations, confidence, though adequate, is lowest ranking.
4. Core Knowledge Areas – Career was consistently indicated as the weakest knowledge area. Assessment and Testing also a low scoring knowledge area according to students and supervisors. Both areas slipped in student ratings compared to last year.
5. Facilities – Students reported on course evaluations and program evaluations that the classrooms in Ward were uncomfortable and unsatisfactory. They also indicated that conducting research for classes was difficult with current library resources, specifically access to online journals. Finally, parking was considered challenging.
6. Timely Grading and Feedback – Professors and group supervisors are not always timely in grading or clear in providing helpful feedback for improvement.
Plan to Address Challenges Taking what we have learned from all 26 different types of assessments and superimposing that information on the visual representation of the philosophy of assessment, we can easily our strengths and areas to improve. See Figure 9.
Figure 13: Philosophy of Program Assessment with Strengths and Areas to Improve
The Director of Program Assessment presented a synopsis of the results of all program assessments at the annual Stakeholder Program Review meeting comprised of core faculty, staff, students, site supervisors and stakeholders. (X
57
Academic Excellence & Christian Faithprofessional mental health counselors who
are:caring competent committedconfident
Growthacademic
professionalpersonal
8 Core Areas of Knowledgeprofessional orientation and
ethical practicegroup work helping relationshipssocial and cultural diversityhuman growth and developmentresearch and program evaluationassessment and testingcareer development
Purple indicates program strength
Red indicates area to improve
number of people) people attended. After presenting the results, the group discussed ways to address the challenges and suggestions for improvement. Based on the discussion, the following recommendations were suggested.
1. Incorporating Research – Every core course syllabi will be revised to add current research as required reading.
2. Assessment, Diagnosis and Treatment Interventions –Improve confidence through practice diagnosing; improve clinical interviewing skills; and practice intake interviews and reviewing paperwork.
3. Student Confidence – Faculty will strive to incorporate more classroom demonstrations. The program will subscribe to Alexander Street Press Videos so students have access to examples.
4. Core Knowledge Areas – The Career course should be revised to add a “This Is Your Career” component, exploring students’ own career concerns and options as a relevant way to learn the content. Business skills for private practice should also be included.
5. Facilities – The building is scheduled for remodeling this year. We will add Alexander Street Press video library to enhance current library resources.
6. Timely Grading and Feedback – Faculty will work on reducing turnaround time for assignments.7. Data Input Improvements – Continue to work with computer students to create a database that will manage
assessments, reduce paperwork, and improve data collection.
8. Advising – All students are required to meet with their advisor the first semester. Students will be encouraged to meet any other time they have registration or career questions.
9. Discouragement about Student Loans and Finances – Intentionally build confidence about skills and diversity of job options; offer specialized training and workshops to improve marketability.
Conclusion Lipscomb University’s Graduate Studies in Counseling Program is proud of the highly competent and caring students graduating from our program and seeking licensure. They are perceived as qualified, above average when compared to other university counseling students, and employable. Our faculty and staff are an important part of the success of our program. The academic component is strong, as evidenced by students’ performance on external standardized tests. Students who complete the requirements for a degree from this program are well prepared as professional counselors.
This report has been a description of how we conceptualize our program, gather and analyze data, and learn from the information gathered. We value feedback from students, site supervisors, employers, alumni, and stakeholders, and we have made and will continue to make adaptations to improve our program. Our program not only prepares students for a career in the counseling profession, but also to change lives, beginning with their own.
58
Some study data were collected and managed using REDCap electronic data capture tools hosted at Lipscomb University.1 REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources.
1Paul A. Harris, Robert Taylor, Robert Thielke, Jonathon Payne, Nathaniel Gonzalez, Jose G. Conde, Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform. 2009 Apr;42(2):377-81.
59