The Charter Oak Group, LLC 1

Challenges To Managing To Wage-Record Based Outcomes

• Time lag– To find out that an outcome actually occurred,

you have to wait until it happens! – If you are using wage records, there is

typically a 6-9 month lag before wage records are available

– By the time managers find out what happened, it is often too late to take action for that program year

The Charter Oak Group, LLC 2

Four Approaches To Managing To Wage-Record Based Outcomes

• Use of administrative records• Use of survey based approaches,

particularly those that leverage existing survey mandates

• Use of intermediate outcome and process indicators as “leading” (or at least “less lagging”) indicators

• Use of previous outcomes data in time series to forecast future outcomes

The Charter Oak Group, LLC 3

Use of Administrative Records

• Administrative records can be used to “get a sense” of what the wage record based outcome levels will be

• Most local areas collect information on entry into employment and wage at placement from the customer at exit, and try to update this information if they contact the client for follow-up

• But, this is very labor intensive, and the information collected may still be too late to take action before the end of a program year

The Charter Oak Group, LLC 4

Leveraging Customer Surveys

• Both WIA and ES mandate customer satisfaction survey questions

• Some states and local areas add additional questions on employment, earnings gain to these surveys

• While the information is gathered is still “after exit” it can provide a strong indication, during the program year of what direction outcome levels are heading

The Charter Oak Group, LLC 5

Use of Intermediate Process and Outcome Indicators

• Use of intermediate process and outcome indicators, such as attendance rate, number and type of services received, and training completion rate, can be used to predict outcome levels. Customer characteristics can also be utilized in this approach.

• Statistical analysis can be used to determine the extent to which these indicators are related to the wage record based outcomes

• These are actually leading indicators, as they can be examined BEFORE customers actually exit

The Charter Oak Group, LLC 6

Time-Series Analysis of Previous Outcome Information

• While wage-record based outcomes are lagging, they can provide a “time-series” of outcome data points which can then be used to forecast future outcomes

• Forecasting techniques can range from simple moving averages to complex econometric approaches

• There are now user-friendly software programs available that the informed layperson can use to conduct these analyses