Development Report Mental Health Counselors and Social … … · Web viewThe third column...

District-Determined Measure Example

Mental Health Counselors and Social Workers, K-12

Content Area and Grade Range: Social/Emotional Skills, grades K-12DDM Summary: This DDM assesses growth in grade K-12 students’ ability to identify social/emotional challenges and their feelings related to these challenges, and to identify, implement, and process appropriate coping strategies. Developed by: Lawrie Donovan, Social Worker, Secondary (Rockland Public Schools); Deni Howley, Social Worker, Elementary (North River Collaborative); Freea Leahy, Social Worker, Secondary (Rockland Public Schools); Matt Morse, School Adjustment Counselor, Secondary (North River Collaborative)Reviewed by: Sonya Meiran (ESE), Matt Hollaway (ESE), Craig Waterman (ESE)Pilot Districts: Rockland Public Schools and North River Collaborative (Rockland, Whitman-Hanson, Hanover and Abington)Date updated: June 2015

Table of ContentsIntroduction 2Instrument 5Administration Protocol 7Scoring Guide 10Measuring Growth and Setting Parameters 11Piloting 12Assessment Blueprint 14

North River Collaborative – DDM – Counselors and Social Workers 1

Introduction

Description of the Measure This DDM is a direct measure of students’ growth in identifying and describing social/emotional challenges and identifying, implementing and processing appropriate coping strategies. The measure is both an assessment and a teaching tool. It is based on students’ observable changes in the counseling or therapeutic session regarding their ability to identify and implement successful strategies to manage social/emotional challenges.

Target AudienceMany school counselors have a large caseload of over 50 students. The sample size for this DDM is a minimum of 5-10 students. It is recommended that a sample size of 10-20 be selected in order to improve validity of assessment. The sample will include students who have been referred for individual or small group counseling services related to social/emotional challenges and who work with the school counselor for a minimum of three months. Students should be scored at three months only, and not over the course of a full year. If a counselor has an established caseload at the beginning of the school year, he or she creates an alphabetical listing of the last names of his or her students who meet the stated criteria:

Referred by school personnel (rather than by parents); alternatively, high school students may identify themselves as being in need of services.

Receive individual or small group counseling (rather than in-class support). Receive service for social/emotional challenges (rather than other needs). Work with the school counselor for at least three months to ensure sufficient time for the

counselor’s instruction and support to impact the student’s ability to identify challenges and implement coping strategies.

From this list, the school counselor selects the first 10 students from the list, starting with those names at the beginning of the alphabet. If the counselor has fewer than 10 students on this list, all are selected, with no fewer than five. If the counselor works with fewer than five students who meet the selection criteria, this DDM will not be appropriate without modification.

Typically, when assessing particularly small numbers of students, the amount of data collected needs to be greater to support valid conclusions about the students’ growth in the core content being measured. Therefore, this measure may be modified to increase the number of times the student’s progress is assessed. Alternatively, the user can introduce additional measures, such as an observation rubric, e.g., one that describes the extent to which the student applies social/emotional coping strategies in the classroom or broader school context, rather than solely in the school counselor’s therapeutic setting. The user is encouraged to review pages 6-7 of ESE’s Special Education Implementation Brief for further information about designing measures for small numbers of students.


http://www.doe.mass.edu/edeval/ddm/SpecialEduEducators.pdf

Description of the DDM Development Process This DDM was developed during October 2014 – June 2015 under a DDM Leadership Grant (FC-217) awarded to the North River Collaborative by the Massachusetts Department of Elementary and Secondary Education (ESE). In partnership with the Learning Innovations Program at WestEd (Woburn, MA), the Collaborative convened 12 Specialized Instructional Support Personnel (SISP), including PK-12 school guidance counselors, adjustment counselors, school psychologists, school social workers, and speech, language and hearing specialists who serve students in the North River Collaborative member districts. Participants worked in small teams to strengthen and apply their assessment literacy toward the development of four direct and indirect measures of student growth related to students’ social/emotional development.

Participants grew their expertise over seven sessions by engaging in a guided DDM development process framed by a series of questions, including:

1. What is most important to measure? 2. How shall we measure what’s most important? 3. How can we strengthen and refine our measure? 4. How can we prepare our measure for broader use? 5. What do we want to gain from the pilot?

6. What did we learn from the pilot?

Throughout, participants engaged in large group discussion and critique, as well as team collaboration and problem solving.

In addition to refinements made during these sessions, each measure was strengthened based on feedback from an ESE review team. Measures were then piloted from April-May 2015. Finally, the group analyzed the data collected during the pilot phase, which informed final revisions, as described in the closing pages of this document.

Next Steps Districts in and beyond the Collaborative now have the opportunity to decide if they would like to implement or modify the attached assessment for use as a District-Determined Measure for school adjustment counselors, school psychologists, guidance counselors, and school social workers. Because this is a newly developed measure, it is important that districts engage school counselors and other SISP in examining results from the first year of implementation. Over time, staff should identify revisions or refinements that may strengthen the quality of the assessment, scoring tools, administration protocol, and/or student growth parameters to suit the circumstances and realities of each district’s local context.

Content Alignment


This measure is aligned to the following Core Course Objective (CCO): Students will identify and describe social/emotional challenges that they encounter and identify, implement, and process/reflect on relevant coping strategies.

A CCO is a statement that describes core, essential, or high priority content (knowledge, skills, or abilities), identified by those who designed the assessment. It is drawn, synthesized, or composed from a larger set of curriculum or professional standards. A primary focus of school counselors is to support the development of student attitudes, behaviors, knowledge, and skills that promote time on learning. Helping students learn the skills to manage social/emotional challenges will further that goal. In particular, school counselors often work with students who have been referred to services or are in crisis in part because they have not mastered these skills for coping with emotional stress. As a result, this DDM is aligned with health standards that represent a range of grade and developmental levels.

Most centrally, this CCO draws from Standard 5, Mental Health, from the MA Comprehensive Health Curriculum Framework, which states that students acquire knowledge about emotions, management of emotions, personality and character development, and social awareness; and will learn skills to promote self-acceptance, make decisions, and cope with stress. The relevant sub-standards are noted in the table below.

This standard is also aligned with the Massachusetts Guidelines on Implementing Social Emotional Learning Curricula, the MA Model for Comprehensive School Counseling Programs, and The National Association of Social Workers. In addition, this DDM aligns well with the professional responsibilities of school guidance counselors, as described by Specialized Instructional Support Personnel indicators I-A-1 and II-B-2, school social workers as described in NASW Standard 3, and school psychologists as described in MSPA standard II-D-2. This final professional standard reflects the essence of this DDM, which centers on facilitating the design and delivery of interventions that help students develop skills necessary to become self-regulated, self-motivated, and active learners.

Content (Standard) Weight

Students identify social/emotional challenges.

5.1 (By the end of grade 5) Identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intense positive and negative feelings

5.7 (By the end of grade 8) Identify and describe the experience of different feelings (such as elation, joy, grief, and rage) and how feelings affect daily functioning (MA Comprehensive Health Curriculum Framework)

25% of the measure

Students identify a range of coping strategies for a given social/emotional challenge.

5.16 (By the end of grade 8) Describe the signs of destructive behavior, and identify intervention strategies and kinds of professional intervention

25% of the measure


5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress

5.19 (By the end of grade 12) Explain positive techniques for handling difficult decisions (MA Comprehensive Health Curriculum Framework)

Students implement a range of coping strategies.

5.2 (By the end of grade 5) Apply methods to accommodate a variety of feelings in a constructive manner in order to promote well being.

5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress. (MA Comprehensive Health Curriculum Framework)

25% of the measure

Students will process/reflect on coping strategies

5.6 (By the end of grade 5) Explain how coping skills (such as perceiving situations as opportunities and taking action/exerting control where possible) positively influence self-concept

5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress (MA Comprehensive Health Curriculum Framework)

25% of the measure

100%

InstrumentThere are two instruments for use in this assessment. The first – the Assessment Question Sheet – is the tool the counselor uses to pose questions to the student in a one-to-one therapeutic setting. In the first column, a series of 10 questions are listed for the counselor to pose to the student. In the second column, simplified versions of these questions are provided for use with English language learners or students with language-based disabilities. The third column provides examples of student responses to each of the 10 questions. The fourth and final column notes the social/emotional skill category for each of the 10 questions; it also notes acceptable expanded response options that the counselor is expected to provide to all students to ensure non-language-based ways in which students may process and express responses to the counselor’s questions. (See Administration Protocol for further information.)

The second instrument – the Mental Health Rubric – is the tool the counselor uses to evaluate students’ responses to the 10 assessment prompts. The first column of the rubric describes the social/emotional skill and the subsequent columns describe a continuum of performance levels with associated scores.

Levels of response include:

NO EVIDENCE OF USE, meaning the student provides no evidence of the knowledge or skill requested.

INCONSISTENT EVIDENCE OF USE, meaning the student provides some initial evidence of the knowledge or skill requested, but it appears emergent or partial and does not remain stable throughout the assessment.


SOMEWHAT CONSISTENT EVIDENCE OF USE, meaning the student provides more predictable, reliable, or stable evidence of the knowledge or skill requested. It is not yet fluent or complete enough, however, to suggest the knowledge and skills are really secure and learned.

CONSISTENT EVIDENCE OF USE, meaning the student provides a readily available and complete response that demonstrates predictable, stable, and reliable evidence of the knowledge or skill such that it appears that the student has solid command of the knowledge or skill.

To clarify, the developers provide the following example: If the student is unable to describe or identify a situation occurring during the school day that

is upsetting or makes it difficult for the student to stay focused on their work, then the school counselor indicates NO EVIDENCE OF USE.

If the student is able describe or identify only one (we are asking for three examples) situation occurring during the school day that is upsetting or makes it difficult for the student to stay focused on their work, then the school counselor will indicate INCONSISTENT EVIDENCE OF USE.

If a student is able to describe or identify only two (we are asking for three examples) situations occurring during the school day that are upsetting or make it difficult for student to stay focused on their work, then the school counselor indicates SOMEWHAT CONSISTENT EVIDENCE OF USE.

If the student is able to describe or identify three or more situations occurring during the school day that is upsetting or makes it difficult for student to stay focused on their work, then the school counselor indicates CONSISTENT EVIDENCE OF USE.


Administration ProtocolThe Administration Protocol addresses how the measure is intended to be implemented to best support a common conversation about student growth across classrooms.

When is the measure administered? This is a repeated measure; the same assessment will be used to measure students’ growth over several administrations during the three-month period. Baseline data will be collected at the student’s first counseling session and then once during the final week of each month for a minimum of three months of counseling services. If, for any reason, the counselor does not meet with the student during the final week of a month, the assessment should be administered during the first week of the subsequent month. Students who do not meet with the school counselor for a minimum of three months should not be included in the final scoring of this DDM. Growth parameters are based upon a three-month period of time. Students who are seen for longer than three months may not be able to have growth assessed using the parameters as they are currently configured.

How is the measure administered? The school counselor prepares copies of the Assessment Question Sheet and the Mental Health Rubric prior to meeting with the student. The counselor may also provide additional response option supports due to student age and development or recommended through consultation with ELL or SPED providers for students who may need accommodations

During the first counseling session, the school counselor meets with the student who has been referred for services by school personnel due to difficulties exhibited in school around the areas of social/emotional challenges. For example, these areas may include emotional and behavioral outbursts in class, depressed demeanor, isolating behaviors, difficulty focusing in class, difficulty following classroom routines or teacher expectations, or feelings of overall dissatisfaction with school in general. Where they exist, teachers, senior administrators, or Student Support Teams may refer younger students. Older students may have the option to self-identify for services and support if not recommended by the above personnel or systems.

The counselor tells each student why he or she has been referred for services. The counselor then uses the assessment tool in this initial session as a means to evaluate the student’s current competencies in identifying social/emotional challenges and coping strategies. The first assessment helps the counselor determine the student’s readiness to identify, implement, and process/reflect on relevant coping strategies in relation to these social/emotional challenges.

Subsequent counseling centers on teaching, coaching, and practice sessions where the student learns about different coping strategies and practices using these strategies for different scenarios. Throughout these sessions, the school counselor coaches and prompts the student for more information to examine the student’s knowledge and build capacity for applying healthy coping strategies in the classroom or school. How the student performs when the assessment is administered during the final week of each month contributes to the counselor’s decision as to whether the student continues to receive services.


During the assessment sessions, the counselor asks the student the series of questions shown in Assessment Question Sheet that correspond with the four sections of the Mental Health Rubric. The counselor should use language and/or pictures appropriate to the student’s age and abilities, as shown in the final column in the Assessment Question Sheet and described in the accommodations section below.

The counselor may say to the student, “Tell me more,” if initial responses seem brief or truncated. The counselor may not, however, provide additional prompts, coaching, or follow up questions to draw out additional or more specific responses or to lead students toward expected responses. Teachers must adhere to this restriction in order to ensure that students are demonstrating their knowledge and skills at an independent level, without the influence or assistance of the school counselor or another adult. Consistent administration practices across multiple school counselors in a district also support comparability, i.e., the ability for counselors to compare their results with some degree of confidence that students in different settings were asked to demonstrate their understandings in the same way.

As the assessment session proceeds, the school counselor circles the number that best corresponds to the student’s demonstrated level of response. The counselor records the students’ responses in the designated spaces, along with any notes, as justification for the given scores.

What modifications are allowed? Questions may be modified for the youngest grade levels or for English language learners, as long as the questions retain the core meaning of the question and do not lead students to any particular responses. For example, instead of asking, “Can you describe some different strategies to deal with this situation?” (Question c), a school counselor working with a first grader might ask, “What can you do when this happens?” Instead of asking an English Language Learner, “When you use this strategy, what is the result?” (Question g, above), the counselor might ask, “What happens when you [name the strategy the student provided in question e, above]?” These modifications are left to the individual school counselor with the caution to always retain the clear, intended focus of the question.

The purpose of the grade-based modifications (shown in the far right column of the table above) is to strengthen the likelihood that the DDM measures the student’s emotion management, rather than the student’s ability to use expressive language. The intent is to ensure that language-based barriers do not prevent students from understanding every assessment prompt or undermine their ability to demonstrate their knowledge and skills. The grade-based modifications provide varied response options and should be used with the full range of participating students – general education students, students with disabilities, and English language learners.

Example: After the counselor asks Question a, “Can you describe or identify a situation....”, a first grade student chooses from a series of pictures that the counselor presents.


Example: After the counselor asks Question a, “Can you describe or identify a situation that....”, a third grade student draws a picture of a difficult situation. The counselor can ask further questions if it appears unclear what the picture describes.

School counselors are encouraged to work together to review the modifications they make to the questions above. They are also encouraged to revise the questions collectively, based on their experience, or develop several versions of the questions for different grade spans or abilities.

How should counselors strengthen their scoring? This DDM is based on multiple one-to-one measures that occur within the therapeutic setting. It is essential that school counselors calibrate their scoring with same-role colleagues in their district—e.g., school psychologists or school guidance counselors who work in sustained fashion with students on social/emotional skill development—or with their supervisors or department chairs.

Calibration strengthens the likelihood that observations and scoring will be conducted in a consistent and reliable manner over time and across sites. A sample calibration protocol can be found at the Rhode Island Department of Education website: http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdf .

To calibrate scoring with this DDM, the school counselor can bring videotaped samples of several student observation interviews to a collaborative session with colleagues, ensuring adherence to district policy regarding videotaping of students. Alternatively, if videotaping is problematic, the counselor may invite a same-role colleague to score actual responses in the therapeutic setting.

During the collaborative session, each counselor should score the responses independently, then compare and discuss scoring decisions to identify ways in which participants are interpreting descriptors on the rubric and/or student behaviors differently. Scorers must be sure to only score based upon the observed performance and not to allow prior knowledge of the student or other impressions of the student to bias scoring decisions.

Participants should then discuss and reconcile these differences until reaching a stronger, shared understanding of the scoring criteria. This calibration work contributes to fair and reliable scoring across a district. In some cases, such calibration leads to important revisions to the DDM scoring materials or processes. Of course, the district must approve revisions so that all counselors using the DDM continue to use the same set of tools and protocols.


http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdf

http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excellent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdf

How will accommodations be documented? Students with identified language disabilities and those in the first tier of English language development should be provided with accommodations based on age/development and identified need. These accommodations may include visual supports, such as picture cards, video clips, situation cards, smiley face charts, word charts, word cards, story cards, or art prints. They are intended to assist students in understanding the assessment questions or provide non-language-based cues for describing or identifying a social/emotional situation or coping strategy. Districts using this DDM are encouraged to provide all school counselors with the same sets of visual support tools. They may, as a district, decide to incorporate these into the assessment for all students, rather than only for those with identified learning needs.

ELL accommodations should also include first language translation for students who are in the first tier of language development. It is also important to consider any language or cultural issues that may affect the results of this assessment. For example, students may make social/emotional decisions or draw on strategies based on their own or their family’s native cultural norms and expectations. School counselors need to be aware of these potential reference points and be prepared to note such factors if they impact the results of this assessment. School counselors are strongly encouraged to consult ELL personnel to understand and anticipate these potential cultural influences. Counselors can also engage these specialists in collaborative scoring and calibration activities to strengthen school counselors’ consideration of these potentially influential factors for English language learners.

No further special accommodations are required for students with disabilities unless specified in their Individualized Education or 504 Plans.

The school counselor in the bottom segment of the Mental Health Rubric must record all provided accommodations.

Scoring GuideWho should score the assessment? The tool is designed as a teaching tool, as well as an assessment of progress. The school counselor will benefit the most from scoring students’ performance and analyzing results. In most schools there is only one school counselor, which is also the reason for the self-scoring.

Where possible, it is strongly recommended that a role-alike colleague or supervisor score a sub-sample of students at two or three data collection points during the year. This can be done by inviting the professional into the student-counselor session to observe and score, if this seems appropriate, or by scoring several videotaped segments, while adhering to district policy on videotaping students, and then comparing scores.

These professionals can follow the calibration process described earlier, but the point of this work is to provide a check for the school counselor against an independent observer, with the aim of reducing the occurrence of personal bias or leniency in the scoring process. In this case it is important that this independent observer is comfortable and has experience with the instrument,


such as a school counselor in the district but assigned to a different school.

How should counselors prepare for scoring? The best preparation for administering and scoring this assessment is to use the assessment to guide ongoing instruction and support for each student. Note that the Mental Health Rubric is intended to be used as a guide for coaching and teaching. During most of each month, the student will become familiar with the steps for identifying upsetting situations and will learn and practice different strategies to manage their feelings. The student will practice applying strategies to different scenarios within the counseling session and evaluating their effectiveness. These instructional interactions are based on the core content of the assessment. When the assessment and scoring occur during the final week of each month, the counselor gains information about the extent to which the student is able to demonstrate competence independently in each of the four areas.

How should assessment scores be calculated? After the school counselor conducts the student interview and observation and circles a number that corresponds with the student’s level of performance for each row in the rubric, totals are calculated by adding scores and recording in the bottom row of each column in the rubric. The grand total of points earned during the observation is recorded in the bottom, far right cell in the rubric. Totals can also be recorded at the end of each row in the rubric, providing additional detail to the counselor about the student’s areas of greatest success and need. The sum of these scores should equal the grand total at the bottom, far right cell.

How should gain or growth scores be calculated? The grand total score for each student should be tracked across all administrations of the assessment. The final gain score at year’s end (May) is the difference between the first and last grand total scores of the year. For example, if the student earned a grand total of 34 points on the first administration in September and a 74 on the final administration in May, the student’s gain score would 74 minus 34, or 40 points.

Measuring Growth and Settings ParametersThe assessment activities in this DDM closely match the types of activities that SISP professionals conduct in the therapeutic setting to help students strengthen and reflect on their social/emotional skills. As a repeated measure that gauges students’ monthly progress from September (or start of services) through May, this assessment also has the advantage of being relatively brief and integrated into the curriculum. (See ESE’s Technical Guide B, page 12.) It also provides counselors with an ongoing cycle of feedback so they can modify their instructional supports in response to students’ performance.

Our students come with a variety of abilities and social/emotional challenges, combined with different developmental levels and multiple mental health issues, as well as co-morbid issues. As a result, we have chosen to set student growth parameters that involve differentiating what constitutes low, moderate, and high growth according to students’ initial baseline. This approach is fair for counselors who have quite different types of student challenges represented in their caseloads.


Differentiated Student Growth ParametersInitial Student Performance (100 total pts)

Low Growth Gain Score

Moderate Growth Gain Score

High Growth Gain Score

Low Initial Performance 0-20 points < 20 point gain 20-40 point gain > 40 point gain

Moderate Initial Performance

21-35 points< 20 points gain 20-35 point gain > 35 point gain

High Initial Performance 36+ points < 20 point gain 20-30 point gain > 30 point gain

These Student Growth Parameters describe estimated growth bands based on students’ different levels of baseline or entering skills and knowledge. Analysis of the initial pilot data did not lead to changes in these parameters. Districts should revisit and refine these estimates based on careful review of data collected across classes during the first years of DDM administration, i.e., until historical trends can be established and used to inform and refine these initial parameters.

PilotingThe developers piloted this measure from April 2015 to the end of May 2015 with a representative sample of student referrals who have participated in individual or small group counseling for at least three months. A team of seven school counselors participated in the pilot, representing four elementary schools, two middle schools, and two high schools from four different school districts (Rockland, Whitman-Hanson, Hanover and Abington).

The purpose of the pilot was to determine if the design effectively measures student improvement in social/emotional growth due to the school counselor’s individual and/or small group counseling sessions. During the pilot, school counselors examined the following questions:

Question 1: Are the tools user-friendly for administrator at each school level? Feasible? Easy, clear, straight forward, not overwhelming, and aligned with daily work and purpose?

Question 2: Is this DDM applicable across the K-12 grade span? Question 3: Does the rubric measure what we want for each grade level?

Pilot Results


From Question 1 we learned that the instruments and protocols for administration were generally user-friendly and clear, with just a few needed or suggested changes:

No revisions were needed on the Mental Health Rubric or Assessment Question Sheet. School counselors piloting the DDM found the tool to be feasible, easy, clear, straightforward, and aligned with daily work and purpose. The language used in the DDM description, however, was reportedly “overwhelming, confusing and time-consuming.” Further review by the DDM development team found that the language used was necessary in order to accurately describe the DDM as a whole.

Piloting feedback specifically indicated the assessment blueprint appeared to add confusion. This is the major modification made to address the concerns of the pilot being overwhelming, confusing, and time-consuming. We recommend that the assessment blueprint not be included in the packet given to school counselors.

The pilot demonstrated a need for clearer scoring descriptions in regards to differentiated student growth parameters. We modified the scoring chart on page eleven of the DDM to include the word “initial” in the first row title. This provides a baseline for student performance.

The pilot did not indicate any need for a revision of rubric scores or an additional clarification of scoring procedures.

The pilot did not indicate any need that tools be differentiated by grade level that were not otherwise reflected in the DDM.

The pilot indicated that changes needed to be made in the section Target Audience. These changes included modification to the sample size and to the sample selection based on differing circumstances across districts. Specifications were made in the Target Audience section on page one. A minimum sample size of five and a maximum of twenty were outlined where sample size was previously not specified.

From Question 2 we learned that the DDM is applicable across the K-12 grade span. Accommodations as written in the DDM (feeling chart, drawings, lists), were used at the K-5 level only. Counselors felt that these accommodations were unnecessary at the higher level.

From Question 3 we learned that, given the short time frame of the pilot, the accuracy of student growth parameters could not be determined. These findings reinforce the need for students to be selected who will be served for at least three months and not one month as in this pilot. This criterion is noted in the section regarding selecting the sample. The growth parameters are designed for a three-month period of time and would need to be adjusted should a district wish to look at growth over a full year.

Results: Mental Health DDMStudent IDCounselor-

Student(C#-S#)

Grade Range

# of Written Assessments Completed

# of Counseling Sessions Between

Administrations

Pre-Score: April

(points)

Post-Score:

May(points)

Pre-Post Difference

(points)

Growth Gain Score

C1-S1 K-2 2 14 40 72 32 High

C1-S2 K-2 2 11 40 83 43 High

C2-S1 K-2 2 5 8 41 33 Moderate

C2-S2 3-5 2 7 19 53 34 ModerateNorth River Collaborative – DDM – Counselors and Social Workers 13

Student IDCounselor-

Student(C#-S#)

Grade Range

# of Written Assessments Completed

# of Counseling Sessions Between

Administrations

Pre-Score: April

(points)

Post-Score:

May(points)

Pre-Post Difference

(points)

Growth Gain Score

C3-S1 3-5 2 2 15 16 1 Low

C4-S1 6-8 2 3 44 51 7 Low

C4-S2 6-8 2 4 66 71 5 Low

C5-S1 6-8 2 2 58 58 0 Low

C5-S2 6-8 2 2 59 59 0 Low

C6-S1 9-12 2 6 66 66 0 Low

C6-S2 9-12 2 4 83 83 0 Low

C7-S1 9-12 2 5 35 30 -5 Low

C7-S2 9-12 2 4 68 100 32 High

Most variations in the growth gain scores were found in the K-5 grades sample. Little growth was evident in the 6-8 grades sample. Similar low growth results were found in the 9-12 grades sample. Overall, K-5 students were seen more frequently, which may have contributed to the variation in growth results. These growth gain scores reflect one month between pre and post-test. The Student Growth Parameters identified High Initial Performance as 36-50 points, however, our pilot indicates that this performance should be increased to 36 plus. The loss in points by student C7-S1 is noted by Counselor 7 to be due to student hospitalization during piloting. Counselor 7 noted the high growth indicator by student C7-S2, based on the student’s ability to resolve a major conflict during piloting.

As part of this pilot, one observer participated in the scoring process. Upon discussion, the difference of scoring between the school counselor and observer was found to be based on the observer’s familiarity of the student. As a result, specific language was inserted into the Calibration Section on page 8 to address this concern. “Scorers must be sure to only score based upon the observed performance and not to allow prior knowledge of the student or other impressions of the student to bias scoring decisions.” The difference in score was four points (school counselor scored 45 points and observer 49 points). These findings indicate a high level of calibration among scorers.


Assessment Question Sheet

Question for StudentSimplified Question

Use with grades PK-2, English Language Learners, or students with

disabilities

Examples of Student Responses

Observation Category with suggested

grade-based modifications

a. Can you describe or identify for me a situation that happens during the school day that upsets you, or makes it difficult for you to stay focused on your work? (Ask for 3 examples.)

b. How does this make you feel?

a. Tell me about something that upsets you at school or makes it hard for you to do your work. (Ask for at least three examples.)

b. How do you feel when this happens?

a. Classmate annoys them; teacher cold calls them.

b. Angry, nervous, anxious, fearful; sweating, rapid heartbeat, shortness of breath

Social/Emotional Identification

K-1: pictures of situations2-4: verbal or drawing 5+ : verbal

c. Can you describe some different strategies to deal with this situation?

d. Which strategy do you think would be the best for the situation you described? Why would it be the best?

c. What are some things you can do when this happens?

d. Which of these do you think is the best thing to do? Why?

c. Deep breathing, stretching, physical exercise, change of activity

d. I think if I go do something else I will stop being mad at her (verbal).

Strategy Identification

K-1: picture identification then verbal explanation.2-4: pictures and verbal5+: verbal

e. Which strategy is the one you actually use when this happens?

e. What do you really do when this happens?

e. I play with someone else. Strategy Implementation

K-1: pictures2-4: pictures or verbal5+: verbal

f. You said that [response to d, above] would be the best strategy for this situation. What are the important steps to apply when you use this best strategy?

g. When you use this strategy, what is the result?

h. Is this strategy successful for you? Why or why not?

i. Do you think you can use this strategy when these situations occur for you in the future?

j. Why or why not?

f. You said that [strategy named in d, above] would be the best thing to do. What are the important steps to remember when you do this?

g. What happens when you do this?

h. Does it make your bad feeling go away? Why or why not?

i. Do you think you can do this the next time this problem happens for you? Why or why not?

f. Look around the class and see if I can move to another table. Stay calm, keep breathing deeply.

g. I don't get in trouble.

h. Yes, because I think about something else.

i. Yes, I think I can. Because I know it helps me.

Strategy Evaluation

K-1: drawing result2-4: drawing or verbal5+: verbal


STUDENT ID: ________________ GRADE: _______ COUNSELOR NAME: ____________________________________ DATE: ________________________

Mental Health RubricIn a counseling setting, students identify and/or describe social/emotional challenges that they encounter and

identify, implement, and process/reflect on relevant coping strategies.

No

Evidence of Use Inconsistent

Evidence of Use Somewhat Consistent

Evidence of Use Consistent

Evidence of Use Total Score

Social/Emotional Identification

a. Student describes or identifies social/emotional challenge he or she is encountering.

0 4 7 12

Student Response/Notes

b. Student describes or identifies feelings or physical responses related to social/emotional challenge.

0 4 8 13


Strategy Identification:

c. Student identifies appropriate coping strategies. 0 4 7 12


d. Student matches a particular strategy with a challenge. 0 4 8 13


Strategy Implementation

e. Student applies appropriate coping strategies to challenge identified. 0 8 15 25


Strategy Evaluation

f. Student recalls steps of applied coping strategies. 0 2 3 5



g. Student describes and/or identifies outcome of applied coping strategies 0 2 3 5


h. Student evaluates success of applied coping strategy 0 2 3 5


i. Student decides if coping strategy can or cannot be applied in future situations. 0 2 3 5


j. Student provides a reason for decision as to whether a coping strategy can or cannot be applied in future situations. 0 2 3 5


Total Score (100 total possible points)

Describe any accommodations provided and the reasons these were provided:

Record the number of counseling sessions since the last assessment: INDIVIDUAL sessions _______ GROUP sessions_______


Development Report Mental Health Counselors and Social … … · Web viewThe third column...

Documents

Transcript of Development Report Mental Health Counselors and Social … … · Web viewThe third column...