An Administrators Guide to Student Follow-Up and Data Quality Presented by: Dean Smith Lennox...
-
Upload
hope-charlesworth -
Category
Documents
-
view
218 -
download
0
Transcript of An Administrators Guide to Student Follow-Up and Data Quality Presented by: Dean Smith Lennox...
An Administrator’s Guide to Student Follow-Up and Data Quality
Presented by:Dean SmithLennox McLendonKathi Polis
McLendon and Polis 2
Training Objectives
To review NRS guidelines for determining what participant populations need to participate in follow-up surveys
To outline the seven-step process for conducting NRS-approved participant surveys
To examine strategies for training telephone interviewers
McLendon and Polis 3
Training Objectives
To develop a local plan for conducting NRS-compliant follow up
To review and provide input into the development of data quality standards for local programs
To discuss and clarify MAERS-related data collection and reporting questions and issues
McLendon and Polis 4
Introduction to Student Follow-Up
Requirements established by National Reporting System
Defines: Outcomes you must collectMethodologies for collecting themState reporting requirements
McLendon and Polis 5
Core Measures
Educational Gains – completion of EFLsFollow-Up Measures
Entered employmentRetained/improved employmentEarned a secondary school diploma or GEDEntered postsecondary education or job
training
McLendon and Polis 6
Two Follow-Up Methodologies
Data Match (recommended)Most cost and labor effective method
Student SurveyMichigan
2005 - 2006 = student survey2006 – 2007 = exploring data match for
employment and GED measures
McLendon and Polis 7
What Participants to Survey
Only participants with follow-up goalsEnter, retain, improve employmentEnroll in postsecondaryEarn a GED/diploma
High school diploma completion can be verified through program records.
GED can be verified through local GED Examiner (with participant’s permission granted on Follow-Up Notice)
McLendon and Polis 8
When to Survey for Program Year 2005 – 2006
Postsecondary, HS Diploma, GED Anytime prior to October 25, 2006 (for 2005- 2006 year)
Tips Postsecondary/job training – know the college’s
enrollment schedule to determine most appropriate time (e.g., mid-September for fall semester college enrollments)
GED – allow as much time as possible for participants to complete test and receive their test scores (e.g., mid-September for previous program year)
McLendon and Polis 9
When to Survey – The Plot Thickens!
Participants with entered employment follow-up goals Timing is the challenge!
Must determine TWO outcomes
1. If they got a job by the 1st quarter after exit quarter AND
2. If they were still employed by the 3rd quarter after exit
McLendon and Polis 10
When to Survey for 2005 - 2006
Exit Quarter Conduct Entered Employment Follow-Up
by end of:
Collect Retained/Improved
Employment by end of:
First QuarterJuly 1 – Sept. 30, 2005
Second Quarter Fourth Quarter
Second QuarterOct.1 – Dec. 31, 2005
Third Quarter First Quarter, Next Program Year
(July 1 – Sept. 2006)
Third QuarterJan. 1 – March 31, 2006
Fourth Quarter Not reported for 2005 – 2006
Fourth QuarterApril 1,2006 – June 30, 2006
First Quarter, Next Program Year
(July 1 – Sept. 2006)
Not reported for 2005 - 2006
McLendon and Polis 11
Activity 1 – Match Game
Let’s see how well you do at matching the right information together.
Refer to Activity 1 Handout.
McLendon and Polis 12
The Survey Process
Getting Ready:1. Select and train telephone interviewers.2. Determine your telephone survey schedule.Participant Intake:3. Inform participants of the survey.4. Ensure accurate contact information.Data Collection:5. Identify survey respondents.6. Conduct the telephone survey.7. Record the results.
McLendon and Polis 13
Getting Ready: Step 1: Select and train telephone interviewers.
Selecting telephone interviewersMay be you, your teachers, volunteers,
other staffYou need folks who:
Speak clearlyUnderstand the program and reasons for follow-
upUnderstand the adult learner populationAre polite and courteousAre patient yet persistent ANDWho WANT to be telephone interviewers!
McLendon and Polis 14
Getting Ready
Training telephone interviewersTraining should include
How to use the script, including what to say to participants to introduce the survey and get their cooperation
Ways to avoid refusalsHow to ask the survey questionsHow to record responsesHow to answer participant questions about the
survey.
McLendon and Polis 15
Getting Ready
Training telephone interviewersGo over every question to ensure
interviewers understand: the purpose, what is being asked, and what responses are desired.
Include mock interviews and other practice.Manual – Appendix C: Training Strategies
for Telephone Interviewers
McLendon and Polis 16
Program Survey
Appendices D1 – D6Standardized script for each follow-up
measure/goalMust follow script preciselyThree sections
1 – includes general participant information2 – relates to follow-up goal3 – requests participant feedback and
recommendations
McLendon and Polis 17
Activity 2: Can We Talk?
Divide into pairs. One person will be the student; the other will play the role of interviewer.
For your designated follow-up goal, conduct the telephone interview. What was the result? Did you see any problems you might encounter? If so, what were they and how might you handle
them?
McLendon and Polis 18
Step 2: Determine your telephone schedule.
Employment goals – quarterly Don’t forget
Participants with “enter employment” goals will need to be contacted twice (first for employment, second for job retention)
Postsecondary and GED goals Depends on college schedules, GED test dates,
etc. – You be the judge. All follow-up must be entered into MAERS no
later than October 25!
McLendon and Polis 19
Step 3: Inform participants of the survey.
At intakeFollow-Up Notice – Appendix A
Explains follow-up processRequests their participationProvides the participant’s permission to be
contacted for follow-upProvides permission to verify GED completion
with state or local GED testing examinerGiven to all participants with follow-up goals
(or potential to convert to follow-up goals)
McLendon and Polis 20
Step 4: Ensure accurate contact information.
Important to collect alternative contact information Addresses and phone numbers of friends, relatives,
etc. Use the Follow-Up Contact Information Form –
Appendix B Can input into MAERS’ Update Alternative
Contact screen (Appendix E) Encourage participants to keep their contact
information up-to-date
McLendon and Polis 21
Step 5: Identify survey respondents.
CRITICAL – Input participant data into MAERS at least quarterly!Enables MAERS to generate a Follow-Up
Contact List Lists all participants that need to be
contacted and when.
McLendon and Polis 22
Activity 3: Beginning Your Follow-Up Plan
Refer to the Student Follow Up PlanReview the questions for Steps 1 – 5.Complete the “Response Plan” column for
any items that you can answer right now.
McLendon and Polis 23
Step 6: Conduct the telephone survey.
Ask for the individual by first name or nickname.
If teacher is not making the call, have follow-up staff state they are calling on behalf of the participant’s teacher.
Must use the Contact Log (Appendix F). Must attempt minimum of four contacts. No answer after four attempts – mark as a
non-respondent
McLendon and Polis 24
Step 7: Record the results.
Record follow-up results in MAER’s Follow-Up Information screen (at least quarterly).
File program survey in participant’s paper file.Non-respondents – file blank survey
Adapt the Participant Follow-Up Checklist (Appendix G) for your use.
McLendon and Polis 25
Activity 4: Completing Your Follow Up Plan
Refer to the Student Follow Up Plan.Complete the “Response Plan” column
for Steps 6 and 7.
McLendon and Polis 26
Data Quality Standards
Improving the accuracy of our data U.S. Department of Education developed data
quality standards that states must address annually A guide to the policies, processes, and materials
that need to be in place for quality data Data foundation and structure Data collection and verification Data analysis and reporting Staff development
Levels of performance range from acceptable to exemplary
McLendon and Polis 27
Data Quality Standards
Michigan’s Draft Data Quality StandardsAdapted the national data quality standards
to fit local programsDraft form only – want your inputRecommendations will be forwarded to the
taskforces for further review and submission to State Office
McLendon and Polis 28
Data Quality Standards
Michigan’s Draft Data Quality Standards “Acceptable” level standards will go into
effect July 1, 2006. “Superior” level – July 1, 2007 “Exemplary” level – July 1, 2008
Training and technical assistance will be made available.
McLendon and Polis 29
Data Quality Standards
FormatThree columns1st column – states standard or process2nd column – provides examples of sample
verification3rd column – requires “yes” or “no” responseA few questions require brief narrative
response.
McLendon and Polis 30
Data Quality Standards
Process Program director submits completed checklist and
signed certification page to State Office by September 30, 2007 (for previous program year).
Written improvement plan for unmet standards Describe new policies or procedures you will put in place
to meet the standards, Identify barriers to moving to a higher quality level, and Describe the technical assistance needed to implement
the plan. DLEG will provide customized technical assistance
to help you.
McLendon and Polis 31
Activity 5: Getting Your Input
Group 1 tables will review A and C.Group 2 tables will review B and D.Make a list of any concerns, questions,
or recommendations for the standards or verification samples.
McLendon and Polis 32
Dean’s Corner
Let the games begin!!!!
McLendon and Polis 33
Always willing to help…
Dean Smith [email protected]
Lennox McLendon [email protected]
Kathi Polis [email protected]
McLendon and Polis 34
This project was developed by National Human This project was developed by National Human Resources Development, Inc. (NHRD) and the Resources Development, Inc. (NHRD) and the
National Adult Education Professional National Adult Education Professional Development Consortium in cooperation with the Development Consortium in cooperation with the
Michigan Department of Labor and Economic Michigan Department of Labor and Economic Growth and funded through a grant under Section Growth and funded through a grant under Section 222(a)(2) State Leadership Activities of the Adult 222(a)(2) State Leadership Activities of the Adult Education and Family Literacy Act, Title II of the Education and Family Literacy Act, Title II of the
Workforce Investment Act of 1998, amended. Workforce Investment Act of 1998, amended.
For more information visit:For more information visit:http:www.maepd.orghttp:www.maepd.org