Talent Development Reporting Principles (TDRp): Your Guide to Measurement and Reporting David Vance...

Talent Development Reporting Principles (TDRp): Your Guide to Measurement and Reporting

David Vance

May 20, 2013

M216

Today’s Discussion 2

20-May-13Center For Talent Reporting www.centerfortalentreporting.org

Common Practitioner Questions

Running L&D like a Business

Talent Development Reporting principles (TDRp)» What is it?

» How can it help?

Q&A

3

Common Practitioner Questions


4Common Practitioner Questions

• About measurement» What should I measure?

» Which measures should I use?

» How are these measures defined?

• About reporting» How do I report these measures?

» What do senior leaders (CLO, SVPHR, CEO) want to see and in what format?

» What do we do with these reports once we have them?

• About demonstrating value» How do I show alignment to important company goals?

» How do I show the value or impact of our HR initiatives?


5

Running L&D (Talent) Like a Business


6First: Agree on Our Role

• Help our organizations achieve their goals» More quickly

» At lower cost

» More effectively

• Best accomplished by running L&D like a business focusing on:» Outcomes (Are we doing the right things?)

» Effectiveness (How well?)

» Efficiency (How much and at what cost?)


1. Identify the business goals

2. Align L&D initiatives to business goals and plan the initiatives carefully

3. Execute and report with discipline

4. Measure and evaluate


Four Steps to Run HR Like a Business 7

8

TDRp: How It Answers the Questions

andSupports Running L&D Like a Business


9Talent Development Reporting Principles (TDRp

• Brings principles, standards, definitions and recommended statements and reports to talent development

• Addresses all the questions on earlier slide

• Began in fall 2010 by Kent Barnett (CEO, Knowledge Advisors) and Tamar Elkeles (VP of Learning and Organization Development, Qualcomm)

• Engaged industry thought leaders like Fitz-enz, Bassi, Phillips, Brinkerhoff, Bersin and CLOs/ Senior Talent Leaders of major organizations


Center For Talent Reporting www.centerfortalentreporting.org

10Talent Development Reporting Principles (TDRp

• TDRp for L&D completed in 2011

• Extended to all HR processes in 2012»Learning &Development»Talent Acquisition»Leadership Development »Performance Management»Capability Management»Total Rewards (C&B)

20-May-13

11A Brief History of the Movement

2010 2011 2012 2013+


• Launched in Oct by Kent Barnett, CEO of Knowledge Advisors and Tamar Elkeles, VP of Learning & Communication at Qualcomm

• Industry thought leaders engaged

• Leading industry practitioners form Executive Council

• Knowledge Advisors provides initial funding

• Development of TDRp for L&D

• First whitepaper approved by Council

• Presentation at Learning Analytics Symposium

• TDRp extended to all key talent processes

• Presented to over 1500 people

• ASTD ICE and Singapore

• Creation of Center for Talent Reporting (October)

• New website goes live

• Membership• Credentialing,

Accreditation available

• Workshops• Webinars• User group

conference • SHRM/ANSI

standard

12The TDRp Process (And Answers to the Questions on Earlier Slide)

• Report three types of measures- Outcome, effectiveness, and efficiency

• Create three standard statements with these measures- Outcome, Effectiveness, and Efficiency

• Create three types of customized reports from these statements- Summary, Program (Initiative), and Operations


13The TDRp Reports Designed to Answer Leader’s Questions


• Are the L&D initiatives aligned to our key goals?

• What is the goal or plan?» Organization goal (e.g., 10%

increase in sales)

» Impact of initiative on it (e.g., 20% contribution or 2% increase in sales)

• What are the key programs planned to achieve it?

• How are we doing?» Year-to-date results versus

plan

• Are we going to make plan?

• What is the forecast?

14TDRp Designed to Facilitate the Management of Human Capital


• Alignment of human capital initiatives to organization goals

• Upfront agreement on the expected impact and other measures of success

• Monthly reports for management of initiatives and operations

• Quarterly reports for the Department Heads (like the CLO), SVPHR, CEO to assess progress and manage to planned results

15The Target Audience for TDRp

• Talent leaders and managers» All those responsible for programs, people, and budgets

• Senior talent leaders» SVP of HR, CLO, Head of Talent Acquisition,

heads of other talent processes (leadership development, capability management)

• Senior organizational leaders» CEO, CFO, EVPs, SVPs, governing boards

• Different reports required for different audiences


16

TDRp Framework


17Executive Reporting Focus: Standard Measures

Data Sources and Systems

Systems and processes for organizing data, calculating

measures

Specific methodology (e.g. Phillips, Brinkerhoff)

Statements

Standard Measures

Data

Reports

System Wide Analytics

Program/ Initiative Analytics

Standard Measures: • Efficiency measures: How much? How many? At what cost? • Effectiveness measures: How well?• Outcome measures: What is the impact on the business?


18Executive Reporting Focus: Three Statements



measures


Statements

Standard Measures

Data

Reports



Three Statements:• Outcome Statement• Effectiveness Statement• Efficiency Statement


19Executive Reporting Focus



measures


Statements

Standard Measures

Data

Reports



Three Reports:• Summary Report• Program Report• Operations Report


20Executive Reporting FocusSummary Conclusions, Actionable

Recommendations, Issues for Further Analysis



measures


Scope of TDRp

Statements

Standard Measures

Data

Reports

System Wide AnalyticsProgram/ Initiative

Analytics


Executive Reporting Process 21Senior Executives Learning Executives

Executive Reports

Statements

Data Sets

Extract, convert and calculate Standard Measures

Data Sources

Executive Reporting ProcessG

uid

ing

Prin

ciples

Financial Data Evaluation System

Learning Management System

Other Sources: HRIS, CRM, ERP

OutcomeStatement

Learning Effectiveness

Statement

Learning EfficiencyStatement

Outcomes Effectiveness Efficiency

L&D Summary Report

Quarterly

L&D Program Report

Monthly

L&D Operational Report

Monthly


22Executive Reporting ProcessSenior Executives Talent Development Executives

Executive Reports

Statements

Note: Data sets can be organized by processes

and/or efficiency,

effectiveness & outcomes

Data Sources

TDRp FrameworkG

uid

ing

Prin

ciples

Financial Systems

Evaluation, EOS Systems

Non-Financial, non-TDR Systems

Others: HRIS, LMS, CRM, ERP

Outcomes Effectiveness Efficiency

Talent Development Summary Report

Quarterly

Talent Development Program Report (s)

Monthly

Talent Development Operations Report (s)

Monthly

EffectivenessStatement (s)

OutcomeStatement

EfficiencyStatement (s)

Talent Development Processes

Talent Acquisition

Leadership Development

Learning & Development

Capability Management

Performance Management

Total Rewards

Extract, convert and calculate Standard Measures

Data Sets

20-May-13

23

TDRp StatementsFocus just on L&D


24The Statements

• Three standard statements» Outcome

» Effectiveness

» Efficiency

• Standard measures are used but choice of measures depends on organization

• Summary statements show» Last year’s actual

» Plan (or goal) for this year

» Year-to-date results

• Detail statements show» Monthly, quarterly, trend data

» Granularity

» Without plan


25Outcome StatementAlignment and Impact

• Shows alignment of L&D initiative to goals

• Expected impact on those goals

• Isolated impact is the ideal» Sponsor’s estimate (Quantitative or qualitative)

» L&D’s estimate reviewed and approved by senior leadership

• May use proxies for impact

• Key is some agreed-upon measure of success


26Considerations in Creating the Outcome Statement


• Requirements» Summary statement

» Lists organization’s top goals in priority order

» Identifies impact of talent initiative on goal

» Four columns- Last year’s actual (LYA)

- Current-Year Plan

- YTD Actual

- YTD Actual as % of Plan

• Options» Quantitative versus qualitative

» Isolated impact versus proxy

» Detailed statements

Statements

27Outcome Statement for L&DQuantitative & Qualitative with Proxies


2011Priority Business Outcomes and Impact of Learning and Development Initiatives Actual Plan Jun YTD % of Plan

1 Revenue: Increase Sales by 20% Corporate Goal or Actual % 10% 20% 17% 85% Impact of L&D Initiatives: 25% contribution to goal % 1% 5% 4% 80%

2 Engagement: Increase Engagement Score by 3 Points to 69.4% (1) Corporate Goal or Actual Points 1 pt 3 pts 1.9 pts 63% Impact of L&D Initiatives High/Medium/Low Low Low Low

3 Safety: Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 15% 75% Impact of L&D Initiatives: High High/Medium/Low Medium High High

4 Comply with New Regulations Corporate Goal or Actual % in compliance 100% 100% 100% 100% Impact of L&D Initiatives: Essential Essential Essential Essential

5 Retention: Improve Retention of Top Performers by 5 Points to 90% Corporate Goal or Actual Points -3 pts 5 pts 2 pts 67% Impact of L&D Initiatives None planned

Learning and DevelopmentSample High-level Business Outcome Statement for the Private Sector with Mixed Impacts

Results through June

2012

28Considerations in Creating the Effectiveness Statement


• Start with effectiveness measures directly related to the initiatives in support of organization goals

• Add effectiveness measures which relate to other organizational and department goals (if any)

• Requirements» Must be some effectiveness

measures

» Columns: LYA, Plan, YTD, YTD as % of Plan

• Options» Selected measures

» Grouping and labeling

29L&D Effectiveness StatementEffectiveness Measures

• Kirkpatrick/Phillips five levels» Level 1: Satisfaction, Forecasts of levels 3-5- Participant

- Sponsor

» Level 2: Amount learned

» Level 3: Application rate

» Level 4: Impact

» Level 5: Value (net benefit or ROI)

• Some will have only L1, L2


30Effectiveness Statement for L&D


Home

` 2011Actual Plan Jun YTD % Plan

Level 1: Employee (All programs and initiatives) Quality % top two boxes 80.0% 85.0% 85.6% 100.6% Amount learned % top two boxes 81.9% 85.0% 90.4% 106.4% Intent to apply % top two boxes 80.9% 85.0% 86.5% 101.8% Impact estimate (select programs) % top two boxes 82.8% 85.0% 85.4% 100.5% Value estimate (select programs) % top two boxes 77.8% 85.0% 85.3% 100.4% Total for Level 1 Average of measures 80.7% 85.0% 86.6% 101.9%

Level 1: Sponsor (Select programs) % top two boxes 75.0% 80.0% 77.0% 96.3%

Level 2 (Select programs) Score 78.0% 85.0% 83.0% 97.6%

Level 3 (Select programs) % who applied it 61.0% 75.0% 78.0% 104.0%

Sample High-level Effectiveness Statement Results through June

For 2012

Learning & Development

31Considerations in Creating the Efficiency Statement


• Start with efficiency measures directly related to the initiatives in support of organization goals» Number of participants,

» Number of employees

• Add efficiency measures which relate to other organizational and department goals

• Requirements» Must be some efficiency

measures

» Columns: LYA, Plan, YTD, YTD as % of Plan

• Options» Selected measures

» Their grouping and labeling

32L&D Efficiency StatementEfficiency Measures*

• Volume» Participants

» Courses, classes, hours

- Offered, taken

» By ILT, vILT, WBT, etc.

• Costs» Development, delivery,

management

» Tuition

» Opportunity

• Utilization» Courses, classes, instructors

» By ILT, vILT

• Effort to develop and maintain

• Reach

• Cycle times» Performance consulting,

development, delivery

*See a complete list in the Definition of Terms and Measurements document


Efficiency Statement for L&D


33

2011Actual Plan Jun YTD % Plan

Total Participants Number 109,618 147,500 67,357 46%Total Unique Participants Number 40,729 45,313 36,998 82%

Courses Taken by Type of Learning ILT only % of total 56% 43% 46% 107% vILT only % of total 3% 9% 8% 87% E-learning only % of total 35% 40% 40% 98% Blended % of total 6% 8% 7% 91% Total courses % of total 100% 100% 100% 100%

Utilization E-learning Courses Available Number 60 74 70 95% Taken by more than 20 Number 50 72 64 89% % taken by more than 20 % 83% 97% 91% 94%Reach % of employee reached by L&D % 85% 88% 72% 88%

Investment L&D Expenditures Million $ 15.8 20.2 10 49% Cost Reduction Thousand $ 63 295 168 57%

Sample High-level Efficiency StatementResults through June

For 2012

Learning and Development

34

TDR ReportsFocus just on L&D


35


The Reports

20-May-13

• Three levels of reports» Summary Report (for CEO)

» Program Report (for CLO)

» Operations Report (for CLO)

• Highly customized, pulling the most important measures from the statements

• Executive-level reports show» Last year’s actual

» Plan (or goal) for this year

» Year-to-date results

» Forecast for this year

• Detailed reports for managers may show» Monthly, quarterly, trend data

» Granularity

» Without plan or forecast

36Bringing it All Together: The Summary Report


• Audience: senior business and HR executives

• Most significant measures from the effectiveness, efficiency and outcome statements aggregated at the highest level

• Measures will vary by organization and year to year based on organizational priorities

• Generated at least quarterly

• Must be read holistically to obtain an accurate picture of progress against goals

Senior Business Executives

OutcomeStatement


Statement

Talent Development Summary Report

Quarterly


OutcomeStatement


37Considerations in Creating the Summary Report


• Requirements» Summary report

» Top goals for organization impact by talent initiatives

» Most important effectiveness and efficiency measures

» Columns: - LYA

- Plan

- YTD

- YTD as % of Plan

- Forecast

• Options» Quantitative versus qualitative

» Isolated impact versus proxy

» Detailed report

» Names of measures, how they are grouped, group names

New!


38L&D Summary Report

20-May-13

39The Program Report


• Purpose: Manage key programs to deliver planned results

• Audience: Senior Talent Executives

• Generated for each major Talent program» Includes measures necessary for

senior talent leaders and managers to manage program, initiative, or process results on a monthly basis.

» Typically includes outcome measures relevant to the program and the key effectiveness and efficiency measures.

Senior Talent Executives

OutcomeStatement


Statement

Talent Development Program Report

Monthly


OutcomeStatement


40Considerations in Creating the Program Report


• Requirements» Organizational goal the

initiative supports

» Impact of initiative on goal

» Sponsor’s name

» Key effectiveness measures

» Key efficiency measures

» In either rows or columns- Plan, YTD actual, YTD as % of

Plan, Forecast

- For all listed outcome, effectiveness, and efficiency measures

• Options» Level of detail (simple,

medium, complex or multi year)

Simple Program Report for L&D (page 1)


41

Enterprise Goal: Reduce Injuries by 30% in 2012 Sponsor: Swilthe, VP of Manufacturing Expected Impact of Learning: High

Program Goals Level 1

Metric Value Employee Sponsor Level 2 Level 3Deliver Phase 1 Courses for Factory A Unique Participants 3,000 80% 90% 90% 65%

Total Participants 6,000Develop Phase 2 Courses for Factory B Complete by 5/30 3 90%Deliver Phase 2 Courses Unique Participants 1,000 80% 90% 90% 70%

Total Participants 3,000 ==== ==== ==== ==== ====

Total 80% 90% 90% 68% Courses Developed Number 3 Unique Participants Number 4,000 Total Participants Number 9,000Cost (not including opportunity cost) $250Thousand $

Sample Simple Executive Program ReportFor a Safety Initiative


Simple Program Report for L&D(page 2)


42

June Year-To-Date Results and Forecast

Year-to-Date Results Current ForecastEnterprise Goal: Injuries Reduced by 20% from First Six Months of Last Year 30% Reduction

Impact of Learning: High (Based on Level 1 and 3 results and anecdotal feedback) High

Current Forecast% of Level 1 % of

Metric Value Plan Employee Sponsor Level 2 Level 3 Value PlanDeliver Phase 1 Courses Unique Participants 2,800 93% 85% 88% 95% 62% 3,200 107%

Total Participants 5,542 92% n=1872 n=3845 n=270 6,300 105%Develop Phase 2 Courses Complete by 5/30 3 100% 90% 3 100%Deliver Phase 2 Courses Unique Participants 100 10% 80% 90% 92% 61% 1,100 110%

Total Participants 284 9% n=39 n=98 n=7 3,200 107% ==== ==== ==== ==== ==== ==== ==== ====

Total 83% 89% 94% 62% Courses Developed Number 3 100% 3 100% Unique Participants Number 2,900 73% 4,300 108% Total Participants Number 5,826 65% 9,500 106%Cost (not including opportunity cost) Thousand $ $178 71% $255 102%

Sample Simple Executive Program ReportFor a Safety Initiative


43The Operations Report


• Purpose: Manage Talent operations to deliver planned results

• Audience: Talent Executives and managers

• Pulls key measures from the Efficiency Statement

• Adds a column for forecast

Senior Talent Executives

OutcomeStatement

Talent Development Operations Report (s)

Monthly


44Considerations when Creating the Operations Report


• Select the most important efficiency measures from the efficiency statement

• Requirements» Key efficiency measures that

will be actively managed on a monthly basis

» Columns for LYA, Plan, YTD, YTD as % of Plan, Forecast

• Options» Names of the measures

» How the measures are grouped

» Names of the groups

45Operations Report for L&D


2011 For 2012 Actual Plan Jun YTD % Plan Forecast

Total Participants Number 40,121 50,000 22,013 44% 48,000

Unique Participants Number 20,263 23,000 18,407 80% 24,000

Courses Taken by Type of Learning ILT only % of total 56% 43% 46% 107% 44% vILT only % of total 3% 9% 8% 87% 9% E-learning only % of total 35% 40% 40% 98% 40% Blended % of total 6% 8% 7% 91% 7% Total courses % of total 100% 100% 100% 100% 100%

Utilization E-learning Courses Available Number 60 74 70 95% 74 Taken by more than 20 Number 50 72 64 89% 72 % taken by more than 20 % 83% 97% 91% 94% 97%

Reach % of employee reached by L&D % 85% 88% 72% 88% 88%

Program Management Courses Total Developed Number 22 36 24 67% 35 Number Meeting Deadline Number 16 33 21 64% 33 % Meeting Deadline % 73% 92% 88% 95% 94% Total Delivered Number 143 178 167 94% 178 Number Meeting Deadline Number 89 160 139 87% 155 % Meeting Deadline % 62% 90% 83% 85% 87% Cycle Time Performance consulting Days 39 30 33 67% 32 Design and Development for ILT Days 56 45 44 109% 45 Design and Development for WBT Days 48 40 39 113% 39

Sample Executive Operations ReportResults through June


46

Conclusion



47TDR Designed to Meet The Reporting Need for Human Capital

• Simple, consistent approach

• Use standard definitions to extract and convert data into three types of measures- Outcome, effectiveness, and efficiency

- Over 500 measures in the online library

• Create three standard statements with these measures- Outcome, Effectiveness, and Efficiency

• Create three types of customized reports from these statements- Summary, Program (Initiative), and Operations

20-May-13


48The Center for Talent Reporting:The Home of TDRp

• Established October 2012» Not-for-profit, 501c(6) organization (used for trade associations)

• Funded by sponsors, members, revenue from workshops

• Governed by nine-member Board of Directors» Including a Standards Committee to provide continued guidance on

standards and reporting and an Advisory Council

• Mission» Develop and implement reporting standards for human capital

» Business model: Others to provide consulting and software

20-May-13

Plans for 2013 49


• Two-day workshops in 2013» Mar 26-27 Irvine

» Jun 6-7 Alexandria, VA

» Oct 15-16 Atlanta

• Webinars offered monthly» Intro to TDRp

» Implementation CSFs

» Implementation Guidance (members only)

• First TDRp conference planned for October 17

• Certification» Individuals

» Vendors providing software products employing TDRp

• Accreditation» Organizations implementing TDRp

» Consultancies providing services

• SHRM will establish a taskforce in 2013 to consider recommending TDRp for L&D as a SHRM/ANSI standard


50Learn More about TDRP

• Learn more and get implementation guidance at www.CenterforTalentReporting.org» Introduction to TDRp whitepaper

» Over 500 measures

» More than 60 sample statements and reports

» Guidance on implementation

» Workshop and webinar registration

• Contact Dave Vance for more information: [email protected]

20-May-13

http://www.centerfortalentreporting.org/

mailto:[email protected]

Your Feedback Counts! Your feedback helps ASTD continue to provide top-notch educational programs that help you stay on top of a changing profession.

Evaluation forms for this session are available NOW via the mobile app and at the following link: www.astdconference.org.

http://www.astdconference.org/

Talent Development Reporting Principles (TDRp): Your Guide to Measurement and Reporting David Vance...

Documents

Transcript of Talent Development Reporting Principles (TDRp): Your Guide to Measurement and Reporting David Vance...