Still Married To Assessment Although we are just as rich and...
20
Starring: Crystalyn Hottenstein,& Wesly Maekawa Jodie Kaderli, & Russell Kinningham Still Married To Assessment
Transcript of Still Married To Assessment Although we are just as rich and...
Starring:
Crystalyn Hottenstein,& Wesly Maekawa
Jodie Kaderli, & Russell Kinningham
Still Married To Assessment
Presenter
Although we are just as rich and famous, please don’t confuse us with our cousins, the Kardashians. Introduce each other.
Health Clusterians + Assessment = True Love
Health Cluster
Learning Goals Activities
Learning Goal 1: Self Appraisal
Learning Goal 2: Exploration &
Assessment, Decision Making
Learning Goal 3: Personal Integration &
Engagement
Learning Goal 4: Adapting to Change
Exploring Health Careers Info Session
X
Health Career Characteristic Inventory
X
Personal Counseling X X X X New Student
Orientation (NSO) X
Program Information Sessions
X
Health Cluster Blog (story book)
X
Wellness Fair X Intro to Health
Professions Course (HLTH 120)
X
KCC’s Ohana Day X Program Orientation X
Presenter
Like most young couples in love, we were glassy-eyed and breathless when we first met our lover “assessment.” This shows an early list of our Counseling Activities and how they line up with the counseling learning goals.
Health Cluster Assessment Matrix
2009 – 2010 SY Fall 2009 Spring 2010 Summer 2010
Exploring Health Careers Info Session
3
Health Career Characteristic Inventory
Personal Counseling New Student Orientation (NSO) 3 Program Information Sessions Health Cluster Blog Wellness Fair (Well Check Up Stations)
Intro to Health Professions Course (HLTH 120)
KCC’s Ohana Day Program Orientations Summary Frequency 2 Mean 3 Standard Deviation 0
Presenter
This is our first attempt at organizing our assessment data. You can see we still had a ways to go!
SDOs Students will continue to grow in self-‐awareness.
Students will be responsible for their choices and make informed decisions.
Learning Goal 1: Self-‐Appraisal Students can identify and evaluate personal qualities, values, attitudes, strengths, interests, skills and acquired learning and use this self-‐appraisal to develop and respond creatively to their unique career, educational and life planning processes.
Learning Goal 2: Exploration, Assessment, Decision Making
Students can explore, assess, re-‐examine, and act upon their progress toward academic, career, and life goals to make informed decisions.
Learning Goal 3: Personal Integration and Engagement
Students can engage in and reflect upon their participation in college and community activities to grow in self-‐understanding and acceptance, confidence, spiritual awareness, and appreciation and respect for diverse cultures.
Learning Goal 4: Adapting to Change Students can adjust, adapt, respond to, and be resourceful in new situations and in rapidly changing conditions.
Learning Interventions / Activities
Exploring Health Careers Information Session
Health Career Characteristic Inventory
Personal Counseling
Who HC N HS HC
When JUN JUL Sp/Su + DEC Sp/Su Ongoing
Learning Interventions / Activities
New Student Orientation (NSO)
Program Info Sessions
Personal Counseling
Health Cluster Blog (Story book)
Who HC N HS HC HS
When APR - AUG Monthly Monthly Ongoing Coming Soon
Learning Interventions / Activities
Wellness Fair
Intro to Health Professions Course (HLTH 120)
Personal Counseling
Who HS HS HC
When OCT & MAR Fall Ongoing
Learning Interventions / Activities
KCC’s Ohana Day
Program Orientation
Personal Counseling
Who HC HS N HC
When AUG Sp/Su Sp/Su + DEC Ongoing
Competencies: SWiBAT + Bloom word + Learning Outcome
SWiBAT evaluate themselves in light of a career pathway. SWiBAT integrate knowledge learned in the counseling session.
Competencies: SWiBAT + Bloom word + Learning Outcome
SWiBAT synthesize information and make an informed decision towards a specific career pathway. SWiBAT integrate knowledge learned in the counseling session.
Competencies: SWiBAT + Bloom word + Learning Outcome
SWiBAT integrate campus, class, & community activities to promote a balanced & well-‐rounded sense of interdependence. SWiBAT integrate knowledge learned in the counseling session.
Competencies: SWiBAT + Bloom word + Learning Outcome
SWiBAT formulate a plan using resources to respond to foreseeable changes. SWiBAT identify resources that will help them respond successfully to unforeseeable change. SWiBAT integrate knowledge learned in the counseling session.
Assessment: Newsprints, Notes, Rubrics, Counselor Notes, etc.
Assessment: Newsprints, Notes, Rubrics, Counselor Notes, etc.
Assessment: Newsprints, Notes, Rubrics, Counselor Notes, etc.
Assessment: Newsprints, Notes, Rubrics, Counselor Notes, etc.
HC = Health Cluster Counselors HS = Health Science Counselors N = Nursing Counselors
Presenter
Like any marriage, after our first assessment proposal, we went on a honeymoon! Oh the fun we had back in the days writing SWiBATs together and dreaming up lovely plans for assessment. This chart shows the Counseling SDOs, the Learning Interventions, and the SWiBATS we created to go along with those interventions. SWiBATS, you’ll soon understand, are the “children” we bore out of our marriage to assessment.
SWiBATs Students Will Be Able To + (Bloom’s Taxonomy Word) + (Learning Outcome)
Presenter
But it’s when the children come that’s when you realize that the Honeymoon is over! Now we’re in it and there’s no getting out of it. SWiBATS… remember those? “Student will be able to + Bloom Word + Learning Outcome…. “ Let me introduce you to our first born SWiBAT … New Student Orientation….
Health Careers New Student Orientation Goals
• Understand the prerequisite requirements for
your major
• Register for classes, starting with Math & English
• Make a plan for your next semester
• Know who to contact for help from now on*
Presenter
Although we are not the masterminds behind the New Student Orientations, we had an important role to play. We had to define how we wanted to help the students and what we wanted them to get out of our time together. We started simply by defining goals for students at NSO…
NSO Rubric
Students evaluated themselves in light of the NSO Goals and the rubric. On small cards, the indicated whether they were at level 4, 3, 2, or 1. The next slide summarizes the tallies and trends over the years.
4 Advanced
I registered for the upcoming semester, I made a plan for the next semester, and I know the program requirements
and my department counselor’s names & contact information.
3 Competent
I registered for the upcoming semester but I don’t have a
plan for the next semester. I know where to find the course requirements for the program I am interested in. I
know my department counselor’s names & contact information.
2 Basic
I did not register for the upcoming semester; however, I know how to access the schedule of classes and how to use the My UH Portal to register later on my own. I know
my department counselor’s names & contact information.
1 Just Getting Started
I did not register for the upcoming semester and I’m not
sure how to register on my own. I will schedule an appointment with my counselor at a later date. I know
my department counselor’s names & contact information.
Students Will Be Able To synthesize information and make an informed decision toward a specific career pathway.
Rate Yourself. Did you accomplish the goals of this New Student Orientation?
Presenter
The NSO goals transformed and become our first NSO rubric. Eventually, we had the students evaluate themselves in light of the NSO Goals and the rubric. We passed out cards, all they had to do was report whether they were at level 4, 3, 2, or 1 and drop it off in the collection box. The next slide summarizes the tallies and trends over the years.
NSO Tallies & Trends TRENDS Can’t tell. Not enough data.
For FYE Students Starting in Spring 2009
Dates n m n m n m n m n m
TOTALS
GRAND TOTAL n = 0 = 0 m = 3.00
out of 4.0
TALLIES
SPRING 2009 – still crawling
Presenter
Taking a step back for a moment… When we first started collecting data, we evaluated the students rather than allowing them to evaluate themselves. That’s why it looks a little blank. We discussed the NSO session and came up with an overall group score rather than individual scores. Doing it this way, we couldn’t discern any trends. What can we say, we were ROOKIE PARENTS! The next slide shows how our growth as parents.
NSO Tallies and Trends
TRENDS:
For FYE Students Starting in Fall 2009
April May June May June
Dates n m n m n m n m n m
Fri, April 16 11 40 3.64
Fri, April 20 14 45 3.21
Thurs, May 20 14 44 3.14
Thurs, June 3 16 50 3.13
Thurs, June 17 20 60 3.00
Thurs, June 24 20 54 2.70
Thurs, July 1 17 60 3.53
Thurs, July 15 23 55 2.39
Thurs, July 39 21 59 2.81
Thurs, August 12 9 24 2.67
TOTALS 25 85 3.40 14 44 3.14 56 164 2.93 61 174 2.85 9 24 2.67
GRAND TOTAL n = 165 = 491 m = 2.98
out of 4.0
Fall 2009 – Walking!!
TALLIES
This was our first semester allowing students to evaluate themselves. There was a decreasing trend due to class availability
Presenter
This was our first semester allowing the students to evaluate themselves. Doing it this way allowed us to discern trends. For example: You can see from the data that the students in the early NSOs scored higher than the students in the later NSOs. We determined that as we got closer to the beginning of the semester, less classes were available. Makes sense, right? Were we the only ones who noticed this trend? Probably not. But we were happy that we had the data to back up with others counselors also thought was happening. RESULTS: We collaborated with our “marriage counselor” LaVache... She in turn took action and eventually new classes were being added closer to the start of the semester to accommodate the new students. TODAY, as you all know, Freshmen have RESERVED seating in many sections to ensure their registration.
NSO Tallies and Trends
TRENDS: Student were consistently rating themselves between 3 and 4… exactly where we wanted them. But now, we had a new, bigger problem to tackle…
Spring 2011 –
TALLIES
For FYE Students Starting in Spring 2011 November Decemb er January X x
Dates n m n m n m n m n m Fri, Nov 12 (pm) 6 21
Fri, Nov 19 (am) 5 18 Fri, Dec 3 (am) 3 12 4.00 Fri, Dec 10 (am) 7 27 3.86 Fri, Dec 10 (pm) 7 28 4.00 Fri, Dec 17 (am) 2 8 4.00
Fri, Dec 17 (pm) Mon, Jan 3 (am) Mon, Jan 3 (pm) Tues, Jan 4 (am) Tues, Jan 4 (pm) Wed, Jan 5 (am) Wed, Jan 5 (pm) Thur, Jan 6 (am) Thur, Jan 6 (pm)
Fri, Jan 7 (pm)
TOTALS 11 39 3.55 19 75 3.95 GRAND TOTAL n = 30 = 114 m = 3.80
out of 4.0
Presenter
Jump to Spring 2011… So we continued what we had started back in 2009…collecting tallies, recording them, analyzing the trends, making changes, etc. But at the end of the 2011 Academic Year, we knew we needed to make some changes. We stopped collecting data because we no longer had Health Career specific NSOs. We needed to find a way to connect with the Pre-Health Students now that they are scattered among all the NSO sessions. We wanted to address this issue. In addition, Over the previous two years, as long as classes were available, students were consistently rating themselves between Levels 3 and 4 which means they were consistently meeting the goals of NSO DURING the NSO section… the problem we began to notice is After their first semester, many students were returning to the Kahikoluamea Center for counseling and forgetting to come to Health Sciences or Nursing Dept. We wanted to address this issue, as well.
NSO
GOALS: • To increase student
awareness of Health Science counselors
• To encourage coming to visit Health Science counselor
• To increase student satisfaction with counseling services in the Health Sciences
• To gauge whether or not they know the admission & graduation requirements of their program of interest early on (within one semester of choosing a pathway)
New Evaluation:
Growth Spurt 2011-‐2o12
Q1. How satisfied are you with advising/counseling interactions at the Health Sciences Department?
5 4 3 2 1
Extremely Satisfied Satisfied Neither Satisfied Nor Dissatisfied
Dissatisfied Extremely Dissatisfied
Q2. How important to you are the advising/counseling resources at the Health Sciences Department?
5 4 3 2 1
Extremely Important Important Neither Important Nor Unimportant
Unimportant Extremely
Unimportant
Q3. How satisfied are you with the advising/counseling resources at the Health Sciences Department?
5 4 3 2 1
Extremely Satisfied Satisfied Neither Satisfied Nor Dissatisfied
Dissatisfied Extremely Dissatisfied
Q4. How important to you are advising/counseling interactions at the Health Sciences Department?
5 4 3 2 1
Extremely Important Important Neither Important Nor Unimportant
Unimportant Extremely
Unimportant
Q5. How clearly do you understand the admission requirements for your program of interest?
5 4 3 2 1
Extremely Clear Clear Neither Clear Nor Unclear
Unclear Extremely Unclear
Q6. How clearly do you understand the graduation requirements for your program of interest?
5 4 3 2 1
Extremely Clear Clear Neither Clear Nor Unclear
Unclear Extremely Unclear
Presenter
Our new mission for NSO was to start a campaign to encourage a culture of “intentional advising” with the new pre-health students and to purposefully reach out to establish connections with them. We started by connecting via e-mail: (1) immediately after the NSO we sent a reminder of who we are, where we are located and how to contact us, and (2) Mid semester of their first semester to remind them to intentionally seek counseling in preparation for the next semester, and finally (3) a survey to compare the student satisfaction and knowledge with those Freshman who purposefully pursued counseling in the Health Sciences and those who did not. We also put an emphasis on e-advising by strengthening our e-mail templates, encouraging the use of STAR Planner, and encouraging the use of e-mail as a way to ask advising questions. We even experimented allowing students to make their own appointment by way of G-mail and Google Calendar. Our New Goals Were: To increase student awareness of Health Science counselors To encourage coming to visit Health Science counselor To increase student satisfaction with counseling services in the Health Sciences Department To gauge whether or not they know the admission and graduation requirements of their program of interest early on (within one semester of choosing a pathway)
NSO
Results: Statistically significant difference between the two groups!
Have Not Visited
Visited
Question Asked m sd m sd t d How satisfied are you with advising interactions with counselors in the Health Sciences Department?
3.6 .98 4.07 1.05 2.64*
.48
How important to you are advising resources at the Health Sciences Department?
4.21 .96 4.37 .84 1.01 .17
How satisfied are you with the advising resources at the Health Sciences Department?
3.71 .98 4.12 .81 2.75* .42
How important to you are counselor interactions at the Health Sciences Department?
3.69 .81 3.98 .88 1.87 .36
How clearly do you understand the admissions and graduation requirements of the program of your interest?
3.92 1.08 4.01 1.02 .48 .08
Ratings for new students who visited with a Health Science Advisor after one semester of coming to KCC compared to those who did not. *p<.01
Fall 2011 Data
Presenter
Results: The results show a statistically significant difference in “satisfaction” with counselor interaction and with the resources within the HS Dept between those that visited with a counselor within one semester and those who did not. This is no surprise, really. You can’t be too satisfied with something you’ve never seen or tried… but now we’ve got data to back it up. Next Steps: Continue to increase awareness of Health Science Counselors and promote a culture of “intentional Advising/Counseling” here on campus with Pre-Health majors… but how are we going to do that??
NSO …heading toward puberty
NEXT STEPS: Continue to increase awareness of Health Science Counselors and promote a culture of “intentional Advising/Counseling” with Pre-‐Health majors.
HOW? Health Careers Counseling Center
WHERE ARE WE NOW? Renovating two offices to create a center in Kauila Building.
Presenter
After much discussion, we decided that we need to have a central location for all pre-health students to go for counseling and information sessions to make us more accessible, available, easy to find, predictable, and user-friendly. With the support of our department chairs we are renovating two office spaces in Kauila building to make this dream a reality. We hope to start piloting the center by Fall but it will depend on the timeliness of the renovation. We will keep you posted.
Health Cluster Action Plan for Completion Agenda (for Health Sciences Department Counselors)
Student Services Completion
Toolkit
Degrees & Certificates Earned
Native Hawaiian Student Success
STEM Degrees Awarded
PELL Recipients Transfers to Four Year Programs
Counseling Student
Development Outcomes & Learning Goals 1-‐ 4
Department Tactical Plan Program Goals 1 -‐6
College Strategic
Plan Outcomes
A -‐ F
Activities E L A E L A E L A E L A E L A
Individual Counseling Methods 1, 2, 3, 4 1, 2, 6 A, B,D
Walk-‐In
Appointment
E-‐Mail
Phone
Imiloa
Presenter
This year we got serious about bringing our family into this community. This chart is one of many pages that shows how our learning interventions line up within our campus community. We took each category of counseling and identified every activity, every learning intervention, every tool that we use and identified how each one met the Engagement, Learning, or Achievement for the groups of “priority” students as defined in the Chancellors presentation. We also identified how each activity met the Counseling SDO’s, each Health Science, EMS, and Nursing Program Accreditation Standards, Each Department’s Tactical Plan Goals, And the College’s Strategic Plan Outcomes. If the activity didn’t fit any of them, we divorced it or fixed it. We took a good look at ourselves and our relationship with assessment and made a new commitment to “keep at it”… we renewed our vows.
Counseling Student Development Outcomes
& Learning Goals
Learning Intervention Categories (SDOs Met)
Specific Learning
Intervention Evaluated
Expected Level of Achievement
Assessment Strategy/ Instrument
Results of Program
Assessment Next Steps
#1 – Self-‐Appraisal: Students can identify and evaluate personal qualities, values, attitudes, strengths, interests, skills and acquired learning and use this self-‐appraisal to develop and respond creatively to their unique career, educational and life planning processes. #2 – Exploration, Assessment, and Decision-‐Making: Students can explore, assess, re-‐examine, and act upon their progress toward academic, career, and life goals to make informed decisions. #3 – Personal Integration and Engagement: Students can engage in and reflect upon their participation in college & community activities to grow in self-‐understanding and acceptance, confidence, spiritual awareness, and appreciation and respect for diverse cultures. #4 – Adapting to Change: Students can adjust, adapt, respond to, and be resourceful in new situations and in rapidly changing conditions.
Individual Counseling Methods (1,2,3,4)
Personal Counseling: a) In person b) e-‐advising
a) Levels 1 – 4 b) Levels 1 – 5
a) Rubric A, Evaluations AH, AN b) Evaluation B
a) Charts AH, AN b) Chart B
a) Counseling TRIAGE b) Increase Awareness, Continual Evaluation
Group Counseling Methods (1,2,3,4)
c) NSO -‐ During d) NSO -‐ Post e) Program Information Sessions
c) Levels 1 – 4 d) Levels 1 – 5 e) Levels 1 – 4
c) Rubric C, Evaluation C d) Evaluation D e) Rubric E, Evaluations EH, EN
c) Chart C d) Chart D e) Charts EH, EN
c) continue to post goals, random evaluation d) campaign to create a culture of “intentional advising” e) Summer 2012 use new Evaluation for all HS programs
Outreach Events, Programs, &
Partnerships (2,3,4) f) FHS Tour f) Levels 1 -‐ 4 f) Rubric F, Evaluation F f) Chart F f) Create new rubric,
staring assessing
Counseling Services & Topics* (1,2,3,4)
Our Students: g) Where from h) Why they seek counseling services
None. Information is extracted from Personal Counseling Evaluations AH, AN and E-‐Advising Evaluation B.
g) Chart G h) Graph H
g) Modify SARS, Record all contacts h) Collect with every Eval
Referrals (1,2,3,4) Not currently tracking. N/A N/A N/A N/A
Scholarships & Grants (1,2,3) i) Scholarships N/A N/A i) Charts IH, IN
i) Convert to Giving Tree; establish common application deadline for Health Cluster
Tools Developed (1,2,3,4)
j) Accessibility and Usefulness Evaluation
j) Levels 1 – 5 Information is extracted from Personal Counseling Evaluations AH, AN and E-‐Advising Evaluation B.
j) Chart J j) Meet with other faculty to evalute accuracy and effectiveness of new tools
Targeted Activities (1,2,3,4)
Not currently tracking. N/A N/A N/A N/A
Presenter
Using the chart from Mona, we organized our assessment data. This chart doesn’t show you the data (THERE’s TOO MUCH!!) but rather it’s an organizational map that tells you where to find the evaluation tools, data analysis, interpretation of the data, and it summarizes the next steps for each area of evaluation. We then keep it organized in an “assessment bin”. We’d now like to briefly summarize our personal reflections on three other areas of assessment…
Our personal reflections.
Presenter
Russ: Scholarships Crystalyn: Group Counseling - Online Information Sessions Wes: Individual Counseling – College Student Inventory
The Ultimate “Honey Do” List.
SDOs 1, 2, 3, 4
In-‐Person Counseling
SDOs 1, 2, 3, 4 E-‐Advising
SDOs 1, 2, 3, 4 Group
Counseling
Promote & Encourage E-‐Advising; improve templates; continuous evaluation of effectiveness
Health Cluster Website; QR Code; IM; make own appointment; when is it necessary to come in
person
Focus Group Pilot: track progress of 2 -‐ 4 NSO groups students; "intentional
advising"; re-‐create Exploratory Information Session
Pilot Health Careers Counseling Center: Continuous Evaluation of
personal counseling; reconfigure SARS for Cluster, TRIAGE
Combine SARS reporting for the Counseling
Center; log every contact (1-‐4)
AY 12-‐13 AY 13-‐14 AY 14-‐15
Presenter
Still a work in progress!
