Outcome-Based Management Program Training Presentation September 8-12, 2008.

Outcome-Based Management Program

Training Presentation

September 8-12, 2008

- 2 -- 2 -

Introductions and Icebreaker

Please say the following about yourself:

Name

Club

Position

One positive thing you’ve seen with a member or the club while working at BGCC

- 3 -- 3 -

Agenda

Introductions 10 min

Part 1: Outcome-Based Management Program Overview 15 min

Part 2: Data Collection Requirements 20 min

Part 3: Immediate Action Items and Resources 10 min

Break 10 min

Part 4: KidTrax Training and Refresher 30 min

Q&A 10 min

- 4 -- 4 -

Part 1: Outcome-Based Management Program Overview

- 5 -- 5 -

What Are Outcomes?

Outcomes are the desired results, both short term and long term, that our members will experience by attending our clubs and programs

Short term program results that are small contributions towards long term outcomes

Examples

Academics: Increased likelihood of being promoted grade levels

Character: Greater knowledge of positive vs. negative behaviors and attitudes

Health: Higher participation in sports activities outside of the club

Intermediate Outcomes Long Term Outcomes

Program end results, often difficult to measure, that target core assumptions and behavior, and are built from short term outcomes

Examples

Academics: Academic achievement and attitudes that lasts a lifetime

Character: Avoidance of detrimental and dangerous behavior

Health: Maintaining a healthy lifestyle through smart nutrition and behavior

These outcomes, while somewhat abstract, all contribute to the core mission of the Boys & Girls Clubs of Chicago

- 6 -- 6 -

How Are Outcomes Achieved?

Outcomes are achieved by using various inputs and processes, and can be visualized by using an outcome logic model

Outcome Logic Model

` Outcomes

Input Process Output Intermediate Term Long Term

Resources required to create the outcome

How the inputs are used to deliver the

programs and activities

What the club produces by running the

program, expressed using measurable

figures

Short term program results that are small contributions towards long term outcomes

Program end results, often difficult to

measure, that target core assumptions and behavior, and are built

from short term outcomes

Two Program Instructors

Two volunteers

Daily Power Hour homework help

60 unduplicated youth participate weekly

15 youth participate daily

Increased likelihood to complete assignments on time

Increased likelihood to understand the value of homework

Increased likelihood for higher academic performance and school promotion

Improved ability to identify and utilize available resources

One Program Instructor

Three volunteers Four baseball bats Four baseballs

Weekly softball team games

20 unduplicated youth participate weekly

50% of Juniors attend each week

Increased awareness of healthy exercise habits

Increased likelihood of maintaining balanced diet, weight, and overall health

- 7 -- 7 -

What Is The Outcome-Based Management Program?

The Outcome-Based Management (OBM) program will allow us to collect information to see how club attendance impacts member outcomes in terms of academics, health, and character

KidTrax Database CPS IMPACT Data

We will use data from KidTrax… …and match it with data provided by CPS…

Survey results

Length of

membership

Pre / post test results

ADA by club /

program

Time

spen

t by

club

/ pro

gram School Promotion

ISAT Scores Attendance

Grades

Suspe

nsion

Recor

ds

Outcome-Based Management Reports

CPS grades by club ADA / length of membership

CPS attendance by club ADA / length of membership

Character development by club ADA / program attendance

Leadership development by club ADA / program attendance

Health attitudes by club ADA / program attendance

…to create OBM reports that can be communicated to stakeholders and help improve programs!

- 8 -- 8 -

What Are The Benefits Of Creating The Outcome-Based Management Program?

OBM program will have wide ranging benefits at both the club level and the executive level

Remember, this is a long term initiative and benefits won’t be recognized overnight

Improve Programs and Operations

Earn More Money Improve Our Members

Outcome reports lead to…

…program assessments and benchmarks…

…and allow us to share best practices and improve programs!

Outcome reports lead to…

…quicker, easier prep for grant proposals…

…and allow us to get more money!

Member-level data and tracking lead to…

…more information to work individually with members…

…and improve their academics, character, and health!

- 9 -- 9 -

How Can We Ensure This Will Be Successful?

Directors and staff first need to understand why

we’re developing this program to make sure it’s

done right!

Understand the benefits of OBM Ensure accurate and timely data entry

Directors and staff members need to understand the following factors that are key to making this program a success

The outcomes are only as good as the data that’s

entered!

To take full advantage of the OBM program, directors and staff need to know how to input the KidTrax data and run new reports!

Understand KidTrax and its functionality

Regularly use the metrics and KidTrax

Reviewing data on a regular basis will help you understand the power of OBM!

This program is NOT intended to make your job more difficult. Instead, it will be designed to be simple, and result in more accurate data and better results

- 10 -- 10 -

Test Your Knowledge – OBM Program

1. Which of the following is NOT an example of an outcome?

A. Members improve academic performance

B. Members attend Power Hour twice per week

C.Members have greater self esteem and self confidence

D.Members have a more active lifestyle

Answer: B

- 11 -- 11 -


2. The Outcome-Based Management (OMB) program

A. Utilizes data from a number of sources, including KidTrax and CPS

B. Will generate reports that will help improve our programs

C.Is the responsibility of everyone to be successful

D.All of the above

Answer: D

- 12 -- 12 -


3. Which of the following is NOT a benefit of the OBM program?

A. Improve programs and operations

B. Earn more money

C.Increase staff support

D.Improve our members

Answer: C

- 13 -- 13 -


4. For the OBM program to be successful, everyone will be responsible for

A. Making sure the members are attending all programs

B. Relying on their Club Directors to perform the majority of activities

C.Understanding the benefits and ensuring good use of KidTrax

D.Holding more programming than in previous years

Answer: C

- 14 -- 14 -

Part 2: Data Collection Requirements

- 15 -- 15 -

Data Collection Processes

The data collection requirements will cover four processes related to attendance tracking and survey / test administration

Tracking member attendance at clubs, including entry and exit times

Club Attendance Tracking

Tracking member attendance at selected programs

Selected Program

Attendance Tracking

Administering surveys / tests developed by BGCA / clubs on a regular basis

Outcome Measurement

Toolkit Administration

Administering survey tool developed by BGCA on a regular basis

Selected Survey and Test

Administration

Used with other metrics to see how club attendance impacts members

Used with other metrics to see how program attendance impacts members

Used with program attendance to analyze changes in attitudes, behaviors, and knowledge

Used with club and program attendance to analyze changes in attitudes and behaviors

Attendance Surveys / Tests

Description

Usage

- 16 -- 16 -


Selected Program

Attendance Tracking

Outcome Measurement



AdministrationClub Attendance Tracking Requirements and Details

Key Requirements

1. Every member must check in and out at front desk to participate in club activities

2. Club staff must scan members into KidTrax on a daily basis

3. Club staff must ensure that member entry and exit times are properly inputted into KidTrax

Other Details

Responsibility: Front Door Volunteer / Assigned Club Member / Staff Member

– Please identify a person who is trained / familiar w/ KidTrax and able to assist others entering data

Frequency: Daily

Batch Scanning: Only permitted for school-based clubs by using sign-in sheet, unless there are emergency or special circumstances

– Members who enter 30 minutes after the club opens or leave 30 minutes before the club closes must have their times recorded on the sign-in sheet and be entered into the system separately

– Front door resource at stand-alone clubs should use ID scanning or manual scanning in Trax Attendance Manager

Best Practices: Many clubs have created binders containing backup IDs, printed out member bar codes, or created new sign-in sheets with Time In / Out to use as backups

- 17 -- 17 -


Selected Program

Attendance Tracking

Outcome Measurement



AdministrationChoose Your Own Adventure – Club Attendance Tracking

Front Desk, Stand-Alone Club

You’re sitting at the front desk, texting w/ your friends, when a member walks in. You ask for his ID card, but the member forgot it! You panic and start breathing heavily. Do you…

Shoo him into the club without checking in?

Stop, breathe, then perform a manual KidTrax

scan?

Uh oh, guess we won’t be able to track that member!

Good choice! Back to texting, until…

…you’re mysteriously transported to a school-based club a couple

hours later. All the members have left and they’ve put their exit times on the sign-in sheet. One member

had to leave 45 mins early, but other than that, everyone left when

the club closed. Do you…

Decide to wait a couple days to enter this data – this isn’t my club, right?

Take the initiative and batch scan everyone in w/ the same entry / exit time?

Front Desk, School-Based Club

Wait until tomorrow morning to batch everyone except the member that left

early?

We’re one team – you should be helping out!

You’re acting quick, but you need to include exit times too!

Even though it’s not done the same night, you’re doing the right thing!

- 18 -- 18 -


Selected Program

Attendance Tracking

Outcome Measurement



AdministrationProgram Attendance Tracking Requirements and Details

Key Requirements

1. Every child must check in and out with the Program Instructor to participate in selected programs and activities

2. Club staff must enter program attendance within two days of program session

3. Batch scanning is acceptable for data entry

Other Details

Responsibility: Program Instructor (work w/ Directors to get appropriate KidTrax training / assistance)

Frequency: Up to 2 days after program session

Attendance Sheets: Instructors should utilize attendance sheets provided in program planning kits

Participating Programs:

On-Going Programs

– SMART Programs

– Power Hour

– College Bound

– Keystone Club

– Torch Club

Quarterly Programs

– SMART Moves

– SMART Girls

– Street SMART

– Passport to Manhood

– Money Matters

– Career Launch

– Youth for Unity

– Triple Play

– Skill Tech / Digital Suite

- 19 -- 19 -


Selected Program

Attendance Tracking

Outcome Measurement



Administration

Choose Your Own Adventure – Program Attendance Tracking

As the Program Instructor, you’ve started your lesson for the day and have all the members pumped! Suddenly, 10 minutes into it, one of the participants runs out of the room to take a phone call and doesn’t come back. Do you…

Make note of it on the attendance sheet and not

to give her credit for attending?

Shrug it off and check her in normally– she got the gist of the lesson, right?

She’s not here for long, so she doesn’t get credit

If her phone call’s more important, she shouldn’t be included in attendance

Program’s over and you’re wiped out. You know you have two days to put the attendance into KidTrax, but you never learned how to use the system – spent too much time

playing Guitar Hero w/ the members. Do you…

Ignore it until your Program Director catches

you?

Talk with your Club Director about learning to

use KidTrax?

Batch scan it at the end of the program with the Membership Clerk?

Ignoring it only makes it worse!

The quick, sensible answer – good choice! If he knows how to enter data, he’ll be happy to help you out up to 2 days after the session

- 20 -- 20 -


Selected Program

Attendance Tracking

Outcome Measurement



Administration

Outcome Measurement Toolkit Administration Requirements and Details

Key Requirements

1. Toolkit must be completed by the same members 3x per year during time period in October, January, and May

2. A set number of members must use toolkit over this period

3. All answers must be inputted into KidTrax

Other Details

Responsibility: Club Director / Program Director / Club Manager / TEEN Reach Coordinator

Frequency / Timeline : 3 times per year (allot ~2 week; must be completed by October 24, January 18, and May 16)

– Clubs may allot extra time to input the toolkits into KidTrax, but must be uploaded by dates above

– Suggested administration dates: October 13-24, January 7-18, and May 5-16

Administration: Members must complete the appropriate survey toolkit for their grade level and should be tested at least twice per year

TEEN Reach: Participants are required to use the toolkit only once per year (suggested month: January) and must complete the separate Responsible Youth section

Expectations: Clubs are expected to survey 70% of their projected ADA goals for that year

- 21 -- 21 -


Selected Program

Attendance Tracking

Outcome Measurement



AdministrationChoose Your Own Adventure – OMTK Administration

As the Club Manager, it’s the first week in January and you’ve been busy planning for the new year. You look at your to-do list and *BAM* it hits you that you haven’t done anything to plan for the OMTK. Uh oh. Do you…

Schedule computer time over 3 weeks (2 won’t be

enough) and schedule testing by age groups?

Ignore it and feign ignorance – it’s not like

anybody’s checking

The earlier you can plan to administer the toolkit, the easier it will be!

Ignoring it will only get you comments on your annual review

You’ve scheduled the OMTK administration, but know there are

also TEEN Reach requirements that you’ll need to meet. Do you…

Work with your TEEN Reach coordinator to

ensure all participants have taken the OMTK?

Pray that everyone takes it and the Illinois Alliance

doesn’t notice?

Your TEEN Reach coordinator will appreciate it!

You don’t want to lose funding, do you?

- 22 -- 22 -


Selected Program

Attendance Tracking

Outcome Measurement



Administration

Selected Survey and Test Administration Requirements and Details

Key Requirements

Other Details

Responsibility: Program Instructor (work w/ Directors to get appropriate KidTrax training / assistance)

Frequency: 2 times per program (during first and last day of instruction)

Administration: Staff may administer tests either electronically or by paper, but they are responsible for putting the information into KidTrax

Participating Programs*:

– SMART Moves

– SMART Girls

– Street SMART

1. Selected pre / post surveys and tests must be administered to all members 2x per program at the beginning and end of program

2. Surveys / tests must be administered each time the program is run

3. All answers must be inputted into KidTrax

*These requirements do not change any guidelines for other programs (e.g., 21st Century)

- 23 -- 23 -


Selected Program

Attendance Tracking

Outcome Measurement



Administration

Choose Your Own Adventure – Selected Survey and Test Administration

As a Program Instructor, you’re preparing materials for your first class and you realize that you completely forgot about the pre-test. You don’t have access to computers, so you have to do it by paper. Do you…

Call up the service center to have them send it to

you?

Access the BGCC intranet and print it out from the

OBM section?

You don’t want to bother them with that, do you?

The quick, easy, sensible solution!

You’ve collected all of the tests and it looks like you have some work

ahead of you (average score: 4 out of 10). Frustrated, you go home for

the night and…

Sulk for a day, then spend the next two days

curriculum planning, trying to put the test out of mind

Make sure that you enter the test data in KidTrax 1-2 days after it’s given, then work on the lesson plans

Don’t despair – your participants will do great! But first get those tests into KidTrax…

There’s still plenty of time for planning – entering data in KidTrax only takes a few minutes!

- 24 -- 24 -

Enforcement and Staff Impact

To make sure we meet these new requirements, the service center will monitor clubs’ data on a regular basis and take appropriate actions to ensure data is entered accurately and timely

Reviewed by service center on an on-going basis to ensure data entered properly

Reviewed by service center after toolkit administration period to ensure clubs are capturing appropriate % of members

Selected Program Attendance Tracking

Outcome Measurement Toolkit Administration

Club Attendance TrackingReviewed by service center on an on-going basis to ensure data entered properly

• These requirements are expected to be met by all staff members and will be a part of the performance review process

• Clubs that are not in compliance will have a letter sent to the Club Director and Board President

• Staff disciplinary action will be taken for non-compliance in accordance to existing policies and practices

Selected Survey and Test Administration

Reviewed by service center on a quarterly basis to ensure that program survey being administered to all participants

Enforcement Staff ImpactData Collection Process

- 25 -- 25 -

Requirements Summary

1. The key to this program is data – make sure it’s captured accurately and timely!

2. Everything must be entered into KidTrax – no exceptions!

3. The Data Collection Requirement documents provide additional details, process flows, and best practices – review them for more information!

Many of these requirements and processes should be familiar, helping to make this a smooth transition

Make data collection a priority and try to get into a routine (i.e., enter data right as you get in or right as you leave

Club Directors / Managers, review requirements with the staff and put it on the weekly agenda

– Plan to schedule a staff meeting within the next week to review these requirements

Key Takeaways

Suggested Practices

- 26 -- 26 -

Test Your Knowledge – OBM Requirements

1. Batch attendance entry is permitted under all of the following normal scenarios EXCEPT

A. Entering club attendance at a school-based club

B. Entering club attendance at a stand-alone club

C.Entering program attendance at a school-based club

D.Entering program attendance at a stand-alone club

Answer: B

- 27 -- 27 -


2. Which of the following statements about program attendance is true?

A. Program Instructors may enter attendance information up to a week after the session

B. Batch scanning of program attendance is NOT permitted

C.Program attendance must be entered into KidTrax for all club programs

D.Instructors should utilize attendance sheets that are part of the program planning kit

Answer: D

- 28 -- 28 -


3. It is required that the Outcome Measurement Toolkit be administered to

A. All members

B. 50% of projected ADA

C.70% of projected ADA

D.As many members as possible

Answer: C

- 29 -- 29 -


4. Compliance with these requirements may result in all of the following EXCEPT

A. Increase in the number of club members

B. Exemption from disciplinary actions related to OBM

C.Annual incentives for staff members and clubs

D.Positive marks on performance reviews

Answer: A

- 30 -- 30 -

Part 3: Immediate Action Items and Resources

- 31 -- 31 -

CPS Member and Parent Consent Forms

Each member who attends CPS will be required to fill out consent forms based on their age for us to obtain their CPS information

Age <12 Ages 12 - 17 Age >18

Parent Consent Form

Parent Consent Form

Member Consent Form

Member Assent Form

Please encourage members to complete and submit forms to the club within ten (10) days of their distribution

Once completed, send forms to the service center with membership applications

- 32 -- 32 -

Student ID Number Collection Process

Clubs must use school rosters and report cards from local CPS schools to allow staff to enter student ID numbers on the consent forms if they are not provided

Note that this will be a one-time process and the service center will follow-up later in the fall to obtain ID numbers that are still missing

CPS student ID numbers can be found in three places:

– Student Rosters

– Student ID Cards

– Report Cards

Club Directors should work with school principals and administrators to collect student rosters

– A letter will be provided by the service center to help explain the importance of the program and how it can assist the school in improving its students’ performance

If available, clubs may also utilize report cards that were collected last year

Student ID Number Collection

Staff will need to

assist members by

entering their CPS student

ID number on the consent form if they

are not able to provide it

Student ID Number Entry

- 33 -- 33 -

Survey Best Practices

There are a number of best practices you can use for administering surveys and tests to members

Prepare all materials prior to the scheduled survey. If using hard copies, keep in mind kids often respond better to surveys printed on colored paper

Leave enough time to set up and arrange the room appropriately prior to the members’ arrival

Introduce yourself to the members as they arrive and give a brief description of your purpose for being there and administering the survey. Sample script:

– I am here to ask you to help make our clubs better. The survey your are about to take is being used to get your ideas about our club. It is not a test and there are no right or wrong answers — we want to know what your experiences and opinions are. Take your time and be sure to answer each question based on what you really think. If you don’t understand a question, please raise your hand or come find me and I can help you. The survey should take you only 8 to 10 minutes. (May differ depending on survey length).”

Request that members attempt to answer every question so their opinions and experiences can be included

Let members know you are here to help them if they have any questions

While members are completing the survey, quietly walk around the room (do not hover), answer questions and make sure things are running smoothly

When youth finish the survey, thank them for their time and provide incentives, if appropriate

Source: The Colorado Trust, ” Toolkit for Evaluating Positive Youth Development”

- 34 -- 34 -

BGCC Intranet Resources

The BGCC Intranet will have many resources available, including

BGCC Intranet Pathway: Downloadable Documents > Outcome-Based Management Program

Resources Available

Program Surveys / Tests (should be used if administering via paper copy)

Outcome Management Toolkit

Parental / Student Consent Forms

Requirement Documents

KidTrax Instruction Guides

Presentations

Other Forms and Guides

This will be your one-stop shop for all things OBM!

- 35 -- 35 -

Conference Call and Meeting Schedule

To support the training you have today, we will hold a conference call and be present at the club manager meeting next week to answer any additional questions and concerns

We will also distribute a Frequently Asked Questions (FAQ) document based on your feedback!

Tuesday, September 16

11:30 am – 12:30 pm

Dial In: 888-998-2663

Passcode: 6235253

Conference Call Club Manager Meeting

Wednesday, September 17

12:00 pm – 2:00 pm

Location TBD

- 36 -- 36 -

Contact Information

Please contact the following people if you have any comments or questions regarding the program

Primary Contact (Until September 19)

Louis Levine (Deloitte), (312) 513-8166, [email protected]

Additional Trainers / Contacts

George Colone (McCormick): (773) 271-8400 x 4307, [email protected]

Khari Humphries (Holy Angels): (773) 268-4819, [email protected]

Dawn Jimenez (Park Manor): (312) 520-6731, [email protected]

Sidney Johnson (King): (773) 638-5464, [email protected]

Jim Pottinger (Cotter): (773) 348-1866, [email protected]

Erin Rochford (Valentine): (773) 927-7373 x4713, [email protected]

Juan Saenz (Logan Square): (773) 342-8800 x4210, [email protected]

LaTrice Smith (GSC): (312) 235-8007, [email protected]

Katie Huckaby (GSC): (312) 235-8008, [email protected]

Ezohn Smith (GSC): (312) 235-8006, [email protected]

- 37 -- 37 -

Test Your Knowledge – Action Items

1. Members between the ages of 12 and 17 should receive which of the following forms?

A. Student Consent Only

B. Parental Consent Only

C.Student Assent Only

D.Parental Consent and Student Assent

Answer: D

- 38 -- 38 -


2. Staff should administer surveys and tests related to the OBM program

A. Printed out from the KidTrax Survey Manager

B. Found on the BGCC intranet

C.Provided by TEEN Reach coordinators

D.Part of the Program Planning Kit

Answer: B

- 39 -- 39 -


3. Which of the following is NOT a best practice for administering a survey?

A. While participants are completing the survey, quietly walk around the room to make sure things are running smoothly

B. Provide incentives for members to complete the survey

C.Do not require members to complete all questions in the survey

D.Prepare all materials prior to the scheduled survey

Answer: C

- 40 -- 40 -

Break

- 41 -- 41 -

Part 4: KidTrax Training and Refresher

- 42 -- 42 -

Manual Scanning (Trax Attendance Manager)

Purpose: Track club attendance as soon as member enters / exits stand-alone club

Situation: When member forgets his or her card and there is immediate access to Trax Attendance Manager at the front desk

Responsibility: Front Door Volunteer / Assigned Club Member / Staff Member

Refer to Manual Scanning Instruction Guide

- 43 -- 43 -

Batch Attendance Tracking (KidTrax)

Purpose: Track club attendance at school-based sites and program attendance after session completed by Program Instructor

Situation: After Program Instructor runs session, will need to record attendance for the following programs:

On-Going Programs

– SMART Moves

– Power Hour

– College Bound

– Keystone Club

– Torch Club

Responsibility: Program Instructor

Refer to Batch Attendance Instruction Guide

Quarterly Programs

– SMART Girls

– Street SMARTS

– Passport to Manhood

– Money Matters

– Career Launch

– Youth for Unity

– Triple Play

– Skill Tech / Digital Suite

- 44 -- 44 -

Survey Setup (KidTrax Survey Manager)

Purpose: Setup Outcome Measurement Toolkit, survey, or test to collect results from members in KidTrax

Situation: Staff needs to setup survey / test for member to either take directly on the computer or be entered into KidTrax by a staff member

Responsibility: Club Director / Program Director / Club Manager / TEEN Reach Coordinator / Program Instructor

Refer to Survey Setup Instruction Guide

- 45 -- 45 -

Survey Administration (KidTrax Survey Kiosk)

Purpose: Administer Outcome Measurement Toolkit, survey, or test in KidTrax to collect results from members

Situation: Staff needs to administer survey / test to members directly through the computer or access surveys and enter data on behalf of the members


Refer to Survey Administration Instruction Guide

- 46 -- 46 -

Survey Results (KidTrax Survey Manager)

Purpose: Track which members have completed a survey and review results of the assessment

Situation: Staff needs to review this information during the testing period to ensure all members have completed the assessment and review results after testing completed


Refer to Survey Results Instruction Guide

- 47 -- 47 -

Q&A

Outcome-Based Management Program Training Presentation September 8-12, 2008.

Documents

Transcript of Outcome-Based Management Program Training Presentation September 8-12, 2008.