11

New CTE Follow-up SurveyNew CTE Follow-up Survey

22

Training TopicsTraining TopicsDiscuss PurposeDiscuss PurposeExplain Need for Good Response RateExplain Need for Good Response RateExplain Interviewer ResponsibilitiesExplain Interviewer ResponsibilitiesIntroduce the SurveyIntroduce the SurveyMock InterviewMock InterviewPractice (2 w & w/o stumbling blocks) Practice (2 w & w/o stumbling blocks) Discuss Problems Discuss Problems Consistency & ConfidentialityConsistency & ConfidentialityLessons on making calculationsLessons on making calculationsClosely MonitorClosely Monitor

33

Purpose & UsePurpose & UseProvide data to meet reporting needs.Provide data to meet reporting needs.Used forUsed for

p1p1

Perkins III Core Performance Indicators.Perkins III Core Performance Indicators.

Ranking programs for State Added Cost fundingRanking programs for State Added Cost funding

LEAs to use for program improvement & aiding students not placedLEAs to use for program improvement & aiding students not placed

Program reviewsProgram reviews

Reporting CTE program impact to publicReporting CTE program impact to public

44

The Interview Process The Interview Process Qualities of Good ImplementationQualities of Good Implementation

ConsistencyConsistency

Response RateResponse Rate

Interviewer ResponsibilitiesInterviewer Responsibilities

p6-8p6-8

55

ConsistencyConsistency

Goal =Goal = to collect data from all to collect data from all students students statewidestatewide

p6p6

in exactly the same wayin exactly the same way

at exactly the same timeat exactly the same time

Timing = 9 mo after June of senior yearTiming = 9 mo after June of senior year

66

Response RateResponse Rate

Acceptable RateAcceptable Rate

0102030405060708090

100

Followup

Good

Low

P6-8P6-8

90% - 100%90% - 100%

< 50%< 50%

Relationship to Placement RateRelationship to Placement Rate

Unacceptable RateUnacceptable Rate

77

Tips for Improving RRTips for Improving RR

Use a phone survey methodUse a phone survey method

A-CA-C

Call from the school phoneCall from the school phone

Leave messages on answering machinesLeave messages on answering machines

Make at least 6 attemptsMake at least 6 attemptsOther suggestions in Attachment COther suggestions in Attachment C

Call in evenings & on weekendsCall in evenings & on weekends

88

Interviewer ResponsibilitiesInterviewer Responsibilities

Be consistentBe consistent

BEFORE Hanging UpBEFORE Hanging Up

P9-10P9-10

Follow question wording exactlyFollow question wording exactlyBe objectiveBe objective

Follow the scriptFollow the script

Check your workCheck your work

Enjoy your workEnjoy your work

99

Follow Wording ExactlyFollow Wording Exactly

Would you climb a tree?Would you climb a tree?

p9p9

Should you climb a tree?Should you climb a tree?

Could you climb a tree?Could you climb a tree?

1010

Be ObjectiveBe Objective

WHY do you say that?WHY do you say that?

p10p10

Why do YOU say that?Why do YOU say that?

Why do you say THAT?Why do you say THAT?

1111

New CTE Follow-upNew CTE Follow-up Phone Survey Phone Survey

UpfrontUpfront OpeningOpening Student contactStudent contact Proxy contactProxy contact

Part A: Current StatusPart A: Current StatusPart B: School/TrainingPart B: School/TrainingPart C: EmploymentPart C: EmploymentPart D: Only if Not WorkingPart D: Only if Not WorkingClosingClosing

P11-22P11-22

1212

Opening – Student ContactOpening – Student Contact

Hello, this is _____ from _____. Is ____ there?Hello, this is _____ from _____. Is ____ there?

StudentStudent:: Hi _____. How are you?Hi _____. How are you?

Single Single program program completecompletedd

______, in _____ grade you ______, in _____ grade you completed a program in completed a program in ________. Do you have a few ________. Do you have a few minutes to answer some minutes to answer some questions about what you are questions about what you are doing now? All responses will doing now? All responses will be kept confidential.be kept confidential.

p11p11

1313

Student contactStudent contactMultiple Multiple programs programs completedcompleted

We are talking with students who completed We are talking with students who completed programs in high school. Do you have a few programs in high school. Do you have a few minutes to answer some questions about minutes to answer some questions about what you are doing now? All responses will what you are doing now? All responses will be kept confidential.be kept confidential.

Our records show that you completed the _____, Our records show that you completed the _____, _____, and _____ programs. Are you pursuing _____, and _____ programs. Are you pursuing one of these more than the other?one of these more than the other?

Which one? ______________________Which one? ______________________

This is the class I would like you to think about This is the class I would like you to think about during this interview.during this interview.

p11p11

If no: We can only follow up on one program. : We can only follow up on one program. Which would you like to think about during this Which would you like to think about during this interview? ____________________interview? ____________________

1414

Proxy contactProxy contact

ProxyProxy:: We are talking with students who We are talking with students who completed high school programs completed high school programs to see how they are doing. to see how they are doing. [Student name] was in the _____ [Student name] was in the _____ class. How could I reach class. How could I reach him/her?him/her?

If unreachableIf unreachable

You could probably answer some of You could probably answer some of the questions. It would only take a the questions. It would only take a couple minutes and all responses couple minutes and all responses will be kept confidential.will be kept confidential.

p11p11

1515

Part A Current StatusPart A Current Status

1. Lets start what you’re doing now. Are you going to school? 1. Lets start what you’re doing now. Are you going to school? Working?Working?

p12p12

Verify applicable part of answer with statement Verify applicable part of answer with statement before coding.before coding. YesYes No No

a. in a training program or attending school or college?a. in a training program or attending school or college?

b. working as an apprentice?b. working as an apprentice? ((If YesIf Yes: Q1c = Yes*) : Q1c = Yes*)

c. working?c. working? d. on full-time, active duty in the military?d. on full-time, active duty in the military?

((If YesIf Yes: Q1c = Yes & Q7 = 40 hrs/wk): Q1c = Yes & Q7 = 40 hrs/wk)

e. on part-time duty in the military? e. on part-time duty in the military?

((If YesIf Yes: Q1c = Yes & Q7 = 40 hrs/wk): Q1c = Yes & Q7 = 40 hrs/wk)f. Other: (specify code from list below)f. Other: (specify code from list below)

________1. unpaid leave of absence (disability, Family leave)1. unpaid leave of absence (disability, Family leave)2. on seasonal layoff2. on seasonal layoff

1616

Q1 - Skip OptionsQ1 - Skip Options

If working, but not in school or training, If working, but not in school or training, Skip to Part CSkip to Part C..

If ‘Other’ only,If ‘Other’ only,

Skip to Part DSkip to Part D..

p13p13

1717

Part B: School/TrainingPart B: School/Training

I have some questions about your schooling or training.I have some questions about your schooling or training.

2. In your major area of study or 2. In your major area of study or training, how much do you use the training, how much do you use the skills you learned in this program?skills you learned in this program? Would you say you use themWould you say you use them a lot, a lot, some, hardly ever, or not at all?some, hardly ever, or not at all?

If a 3 (Hardly ever) or 4 (Not at all): If a 3 (Hardly ever) or 4 (Not at all):

Hardly Hardly NotNot

A lotA lot SomeSome EverEver at at allall

p14p14

2a. Right now you don’t use your 2a. Right now you don’t use your training, but how much do you training, but how much do you anticipate using it in the futureanticipate using it in the future . . . . . . a lot, some, hardly ever, or not at a lot, some, hardly ever, or not at all?all?

1818

p15p15

3. Where are you going to school?3. Where are you going to school?

(Note: If enrolled in more than one, identify the (Note: If enrolled in more than one, identify the primaryprimary one.) one.)

business or trade school, or career business or trade school, or career centercenter community collegecommunity college

college/universitycollege/university

military military (National Guard, West Point, Annapolis, (National Guard, West Point, Annapolis, DLI, Merchant Marines, etc.)DLI, Merchant Marines, etc.)

other ___________________________other ___________________________

1919

p16p16

4. What type of program are you in?4. What type of program are you in?

[Read list][Read list]

certificatecertificate

associate degreeassociate degree

bachelor’s degreebachelor’s degree

other (specify) ___________________other (specify) ___________________

2020

Part B – Skip OptionsPart B – Skip Options

If No Current job, If No Current job, Skip to Part DSkip to Part D..

p16p16

2121

Part C: EmploymentPart C: Employment

Now, I’d like to talk about your job as it Now, I’d like to talk about your job as it relates to your relates to your (selected) (selected) program. program.

p17p17

5.5. On your job, how much would you say On your job, how much would you say you’re using the skills you were taught?you’re using the skills you were taught?

Would you say you use them . . . ?Would you say you use them . . . ?

A lot A lot Some Some Hardly ever Hardly ever Not at Not at allall

2222

p18p18

6. How strongly do you agree with the 6. How strongly do you agree with the statement, I am satisfied with my statement, I am satisfied with my present job? Do you . . . ?present job? Do you . . . ?

Strongly agreeStrongly agree

AgreeAgree

DisagreeDisagree

Strongly disagreeStrongly disagree

2323

p18p18

7.7. How many hours a week do you How many hours a week do you work?work?

______________ hrs/week ______________ hrs/week

7a.7a. If not specificIf not specific::

Would you say it’s 35 hrs a week or Would you say it’s 35 hrs a week or more?more?

Yes Yes NoNo

2424

p19p19

8.8. Including tips and commissions, how Including tips and commissions, how much do you make an hour?much do you make an hour? $ $ _________ / hr_________ / hr

Skip to CommentsSkip to Comments

2525

Part D: Only if Not WorkingPart D: Only if Not Working

9. Are you currently looking for a 9. Are you currently looking for a job? job?

YesYes No No

p22p22

2626

CommentsComments

Thank you. That’s all I needed.Thank you. That’s all I needed.

p22p22

Now, are there any comments Now, are there any comments you’d like to make to help us you’d like to make to help us improve our programs.improve our programs.

2727

BreakBreak

Back in 10 minBack in 10 min

2828

Proxy SurveyProxy Survey

Part A: Current StatusPart A: Current Status1.1. Let’s start with what (she/he) is Let’s start with what (she/he) is

doing now.doing now.

pp

Part B: School/Training Part B: School/Training 3. Where is (he/she) going to school?3. Where is (he/she) going to school?

4.4. What type of program is (he/she) What type of program is (he/she) in?in?

2929

Part C: EmploymentPart C: Employment7. How many hours a week does

(he/she) work?

7a. 7a. If not specificIf not specific: Would you say it’s : Would you say it’s 35 35 hours a week or more?hours a week or more?

8.8. Including tips and commissions, Including tips and commissions, how how much does (he/she) make an much does (he/she) make an hour?hour?

Skip to CommentsSkip to Comments..

Part D: Only if Not Working Part D: Only if Not Working 9. Is (he/she) currently looking for a job?

pp

3030

ConsistencyConsistency

Goal =Goal = to collect data from all to collect data from all students students statewidestatewide

in exactly the same way, in exactly the same way,

at exactly the same time.at exactly the same time.

p6p6

3131

Prefaces to Avoid UsingPrefaces to Avoid Using

Apologetic – If you don’t mind my asking…Apologetic – If you don’t mind my asking…

Blameless – This is confidential, but the Blameless – This is confidential, but the state needs it.state needs it.

Estimates – Approximately how much … Estimates – Approximately how much …

On guard! – I have to ask this question.On guard! – I have to ask this question.

Proxy – You may not be able to answer…Proxy – You may not be able to answer…

Opt out – You don’t have to answer … Opt out – You don’t have to answer …

p20p20

3232

Role Play #1Role Play #1Practice Conducting the SurveyPractice Conducting the Survey

Choose a partner.Choose a partner.

Go to a room Go to a room

Practice both roles – the interviewer and Practice both roles – the interviewer and the student.the student.

Practice Practice at least three timesat least three times..

Switch roles with your partner.Switch roles with your partner.

Be KIND. Do not throw in stumbling Be KIND. Do not throw in stumbling blocks.blocks. Back in 15 minBack in 15 min

pp

3333

Potential Problem & Solution #1Potential Problem & Solution #1

Where are you going to school?Where are you going to school?

I take a class at the career center.I take a class at the career center.

?? Are you registered at the career center?Are you registered at the career center?

I’m registered at the community college, I’m registered at the community college, but they hold the class at the career but they hold the class at the career center.center.

3434

Potential Problem & Solution #2Potential Problem & Solution #2On your job, how much would you say On your job, how much would you say you’re using the skills you were taught?you’re using the skills you were taught?

Student from automechanics program Student from automechanics program working at a dealership says “not at all.”working at a dealership says “not at all.”

? ? What should the interviewer do?What should the interviewer do?

a. Mark the answer and go on.a. Mark the answer and go on.

b. Review skills they learned & ask again.b. Review skills they learned & ask again.

c. Explain why that’s not possible & ask again.c. Explain why that’s not possible & ask again.

3535

Potential Problem & Solution #3Potential Problem & Solution #3How strongly do you agree with the statement, “I How strongly do you agree with the statement, “I am satisfied with my present job”?am satisfied with my present job”? Student working at McDonald’sStudent working at McDonald’s

““Strongly Agrees”.Strongly Agrees”.

?? What should the interviewer do?What should the interviewer do?

a.a. Review potential for the program they took.Review potential for the program they took.

b.b. Talk about benefits & ask again.Talk about benefits & ask again.

c.c. Mark the answer and go on.Mark the answer and go on.

3636

Potential Problem & Solution #4Potential Problem & Solution #4

Including tips and commissions, how much Including tips and commissions, how much do you make an hour?do you make an hour?

The completer has more than 1 job.The completer has more than 1 job.

?? What should the interviewer do?What should the interviewer do?

Combine and estimate the overall Combine and estimate the overall hourly wage.hourly wage.

3737

Potential Problem & Solution #5Potential Problem & Solution #5

Including tips and commissions, how much do Including tips and commissions, how much do you make an hour?you make an hour?

The interviewer is uncomfortable about The interviewer is uncomfortable about asking the question.asking the question.

?? What should the interviewer do?What should the interviewer do? a.a. Apologize, say “I know this is sensitive, Apologize, say “I know this is sensitive, butbut

b.b. Ask for an estimate, “About how much Ask for an estimate, “About how much … …

c.c. Explain it’s optional.Explain it’s optional. Just ask the question.Just ask the question.

3838

Role Play #2Role Play #2Practice Conducting the SurveyPractice Conducting the Survey

Choose a partner/new partnerChoose a partner/new partner

Go to a roomGo to a room

Practice both roles (interviewer & Practice both roles (interviewer & responder)responder)

Practice both surveys (phone & proxy)Practice both surveys (phone & proxy)

Do the survey Do the survey twicetwice (2 each) (2 each)

Go ahead -- throw in Go ahead -- throw in stumbling blocksstumbling blocks

Back in 15 minBack in 15 min

3939

Handling Difficult CallsHandling Difficult Calls

TalkativeTalkative

A-CA-C

GrouchyGrouchy

IrateIrate

4040

Best WeaponBest Weapon

4141

ConfidentialityConfidentialityPart of the 2 Big CsPart of the 2 Big Cs

What was the first C? What was the first C?

MeaningMeaningA promise by interviewers who know A promise by interviewers who know the response of specific completers the response of specific completers not to reveal those responses to not to reveal those responses to anyoneanyone..

Keeping responses a secretKeeping responses a secret

ConsistencyConsistency

pp

4242

CalculationsCalculations

Weekly rate based on 40 hr/wkWeekly rate based on 40 hr/wk$400/wk $400/wk ÷ 40hr/wk = $10.00/hr÷ 40hr/wk = $10.00/hr

p19p19

Monthly rate based on 2080 hr/yrMonthly rate based on 2080 hr/yr$1200/mo x 12mo = $14,400/yr$1200/mo x 12mo = $14,400/yr$14,000/yr ÷ 2080 hr/yr = $6.92/hr$14,000/yr ÷ 2080 hr/yr = $6.92/hr

Annual rate based on 2080 hr/yrAnnual rate based on 2080 hr/yr$18,000/yr ÷ 2080 hr/yr = $8.65/hr$18,000/yr ÷ 2080 hr/yr = $8.65/hr

4343

Strategies for Making ContactStrategies for Making ContactChallengesChallenges

Updated recordsUpdated recordsMobility of studentsMobility of studentsWhen to callWhen to call

The Telephone Consumer Protection Act The Telephone Consumer Protection Act states that telephone solicitation cannot states that telephone solicitation cannot occur before 8:00 a.m. or after 9:00 p.moccur before 8:00 a.m. or after 9:00 p.m

Schools technically are not solicitorsSchools technically are not solicitors

State still recommends keeping within the State still recommends keeping within the 8 a.m. - 9 p.m. time frame8 a.m. - 9 p.m. time frame

p8p8

4444

Challenges (cont’d)Challenges (cont’d)

Answering MachinesAnswering Machines

Caller IDCaller ID

Reaching StudentsReaching Students

DisconnectsDisconnects

Callbacks Can’t Get ThroughCallbacks Can’t Get Through

p9p9

4545

Solving ProblemsSolving Problems

No finite list of problemsNo finite list of problemsSome may be area specificSome may be area specificSSC will help with solutionsSSC will help with solutions

Careful about making snap decisionsCareful about making snap decisionsAll solutions are not easyAll solutions are not easyRecord enough info – no need to recallRecord enough info – no need to recall

Keep own listKeep own listExperience in handlingExperience in handling

4646

Process ProblemsProcess Problems

ContactContact

Dr. Jill KrollDr. Jill Kroll

Office of Career and Office of Career and Technical PreparationTechnical Preparation

(517) 241- 4354(517) 241- 4354

4747

LunchLunch

Start at 1:00 pmStart at 1:00 pm