PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD? Chicago, Illinois April 26, 2004.

PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD?

Chicago, Illinois

April 26, 2004

Accounting Principles Council Strategic Business Reporting Project

Project Team

Jennifer Taylor, NMSU, Chair

Tammy Anthony, NMSU

Teresa Gordon, University of Idaho

Roger Patterson, University of North Carolina-Chapel Hill

Carolann Sagesse, Cornell University

Sue Menditto, NACUBO

Kim Dight, NACUBO

Why is NACUBO’s Accounting Principles Council concerned with Performance Measurement?

Both GASB and FASB have implemented long-term projects to address the issue.



Performance Measurement Background

– GASB research effort to enhance service efforts and accomplishments reporting (SEA)

– SEA is non-financial performance information for governmental organizations

– SEA is rooted in GASB Concept Statement No.1, “Objectives of Financial Reporting”

• Assist in fulfilling government’s duty to be publicly accountable

• Enable users to assess accountability


– GASB Concepts statement No. 2, “Service Efforts and Accomplishments Reporting”

– Conceptually links accountability and external financial reporting



– GASB received Sloan Foundation research grants in 1997 and 2000 • To expand performance measurement

research• To understand external reporting of

performance information– The grants have six research phases


Six Phases of GASB Research Project

1. Performance measurement website

2. Case studies based on interviews and surveys on uses and effects of using performance measures

3. Citizen discussion groups that evaluate users responses to performance measures

4. Develop suggested criteria to enable performance information reporting

5. Experimentation with suggested criteria (future)

6. Evaluate effectiveness of suggested criteria (future)


NACUBO Performance Measurement Project Goals

• Address GASB performance reporting suggested criteria and related issues (GASB phase 4 research results)

• Inform GASB of and proactively represent higher education performance measurement practices (NACUBO advocacy role)

• Develop a simple, visual template for reporting meaningful performance information both internally and externally (GASB phase 5 – experimentation with criteria and external performance reporting)

• Develop a model useful to both public and private institutions (NACUBO Industry Guidance)


Project Approach

• Worked with GASB for guidance • Developed a survey

– To determine if performance reporting currently exists– To determine measures considered important and

relevant for higher education– To assess opinions and relevance of the GASB 16

criteria



Project Timeline

• Conduct Performance Measurement Survey – January 2004

• Analyze Survey – February 2004• Review Sample of Existing PM Reports and Prior

Research Results – April 2004 • Review Draft Template Content and Format with

Focus Groups – Summer 2004

• Develop Template – Winter 2004


Survey Goals

• Identify prevalence and type of PM reporting currently in place within higher education (industry-wide and by constituent group)

• Identify commonly valued KPI at both public and private institutions

• Assess importance of non-financial indicators in higher education

• Provide GASB with input:– Opinions about 16 reporting criteria– Use of outcome-driven and efficiency-based

measures

What is a KPI?

KPI = Key Performance IndicatorTypes of Performance Measures:• Input – selectivity or enrollment stats• Output – graduation stats, national exam• Outcome – perceptions surveys, employment

stats• Efficiency – cost per student• Effectiveness (Productivity) – difference in

entrance and exit scores


GASB Performance Reporting Criteria

1. Purpose/scope is stated clearly

2. Institutional goals/objectives stated clearly

3. Citizen involvement in goal setting

4. Multiple levels of detail presented

5. Qualitative analysis of results/challenges

6. Concise yet comprehensive, focus on key issues

7. Assessment of information reliability

8. Measures are relevant, link to goals


GASB Performance Reporting Criteria

9. Measures should relate to cost/resources/efficiency

10. Report citizen perceptions of quality/results

11. Present comparative information

12. Discuss factors affecting results

13. Show aggregation/disaggregation by user group

14. Measures should be consistent across periods, or explain changes

15. Easy to find, access, understand

16. Available annually and in timely manner

• http://www.seagov.org/sea_gasb_project/criteria_summary.pdf



Survey Highlights

• Surveyed all NACUBO Primary Rep’s• Response rate: 12%• Survey supports continuing template development

– Majority of respondents do some type of reporting, large percentage of these include internal reporting

– Majority do not have mandated reporting format– Majority do not have external guidance for voluntary

reporting

• External reports are primarily extracts of internal reports


Independents Publics

Report KPIs Externally

38% 62%

Use KPIs Internally

46% 54%

KPIs Reported and Used


Survey Highlights

• Roughly half of respondents relate reports to audited financial statements

• Top priority KPI’s identified• Certain KPI’s indicated as not being appropriate to

report• A wider variety of measures are used internally than

externally• A different approach to internal reporting• What is “important” versus what is “recommended”

to be included are not always the same KPI’s!

Survey Highlights

Independents vs. Publics• Independents report/use selectivity measures

(externally/internally) more often than publics• Publics report persistence toward graduation

statistics externally more often as compared to independents

• But both USE persistence statistics internally at comparable levels


Included in External Reports

Number of respondents: 159

Number Percentage

Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)

146 91.8%

Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years)

120 75.5%

Efficiency or comparative financial data/ratios 110 69.2%

Graduation statistics (e.g., degree granted by level or field of study)

101 63.5%

Diversity measures (e.g., statistics on race or gender for students, faculty or staff)

100 62.9%

Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)

94 59.1%

Selectivity measures (e.g., acceptance rate, high school GPA, SAT or ACT scores)

90 56.6%

Used Internally by Management

Number of respondents: 156

Number Percentage

Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)

156 100.0%


137 87.8%

Graduation statistics (e.g., degree granted by level or field of study)

135 86.5%

Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)

130 83.3%

Efficiency or comparative financial data/ratios 126 80.8%

Diversity measures (e.g., statistics on race or gender for students, faculty or staff)

121 77.6%

Student satisfaction or graduating senior survey results

118 75.6%


What SHOULD Be Reported

• Externally – top five were selected by over 67% of respondents

• Internally – top six were selected by 95% or more of respondents– Most of the indicators were identified as useful

for internal management purposes

Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) 88%Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) 83%Graduation statistics (e.g., degree granted by level or field of study) 81%Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) 80%Diversity measures (e.g., statistics on race or gender for students, faculty or staff) 67%

Top 5 Measures that SHOULD be Reported Externally


Persistence and graduation outcomes (e.g., freshmen retention rate, percentage graduating in four years) 98%

Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours) 97%

Efficiency or comparative financial data/ratios 97%Graduation statistics (e.g., degree granted by level or field of study) 95%Student satisfaction or graduating senior survey results 95%Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility) 95%

Top 5 Measures that SHOULD be Used Internally




Enrollment statistics (e.g., full-time equivalent students, head count, semester credit hours)Graduation statistics (e.g., degree granted by level or field of study)Quality of educational experience indicators (e.g., faculty to student ratios, average class size, accessibility)

Four Measures Common to External and Internal Reporting

Enrollment and Persistence Measures

• Top the list on what is being reported externally and what should be reported externally

• Top the list on what is being used internally and what should be reported internally


Measures which Should NOT Be Reported Externally or Internally

Faculty or staff morale, comparative salary studies 15%

Selectivity measures (e.g., acceptance rate, high school GPA, SAT or ACT scores) 13%

Performance on nationally recognized exams 13%Alumni or employer survey results 8%


GASB Reporting Criteria – rating (1-7)


All Respondents

Respondents Reporting Externally

Consistency from Year to Year 5.18 5.13

Regular and Timely Reporting 5.06 5.01

Relevant Measures of Results 5.02 4.95

Reliable Information 4.89 4.82

Focus on Key Measures 4.82 4.79

Mission Statement and Major Goals 4.76 4.78

Survey results tabulated.

What’s next

External research and development of a reporting template.



Objectives of Reporting Template• Verify external reporting versus survey responses• Identify additional categories of indicators

• Find examples of indicators Approach to Compiling Reporting Template• Approximately 15% of survey respondents

provided links to external reporting• Researched institutions not responding to survey

who provide external reporting on their website• Tallied indicators

Research Highlights

• 45 institutional reports reviewed • 23 categories of KPI’s identified

– 50% not on 2004 survey• 272 individual KPI’s identified

– 17% in Common Data Set or IPEDS– 13% recommended from prior surveys/studies



Research Highlights

Most Common Measurement CategoriesGraduation StatisticsPersistence and Graduation OutcomesResearch MeasuresEnrollment StatisticsSelectivity MeasuresDiversity Measures

Research Highlights

Most Common Measures• Freshman retention rate• Percent graduating in 4-6 years• Degrees granted by level or field of study• External dollars received or expended through

research/creative activity• Undergraduate transfer data• Annual giving• FTE and Headcount• Race/gender/nationality of students/faculty


Research Highlights

Most Common Weighted Measures• Freshman retention rate• Percent graduating in 4-6 years• Degrees granted by level or field of study• Entering test scores (ACT, SAT) • FTE and Headcount• Number of minority enrollments/ratio• Student/faculty ratio


Research Highlights

Most Common Unweighted Measures• External dollars received or expended through

research• Average class size• Comparative exam results• Total number of new students enrolled each Fall • Perceived quality of degree• Functional expense as % of total expense• Number of sponsored awards/national ranking



Reporting template complete.

What’s next

Focus Groups

Objectives of Focus Groups

• Develop relevant new questions not addressed by survey

• Consider next steps – Should the project aim for assistance by

conducting experimentation as set forth in the GASB Service Efforts and Accomplishments Reporting Program?

– Reaction to draft template


Focus Groups Planned

• Tuesday, April 27 at 9:30 am• NACUBO Integrated Planning and Budgeting

Workshop - May• Higher Education Association Community (ACE,

AIR, SHEEO, SHEFO, etc.) Summer• NACUBO Annual Meeting - July


PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD? Chicago, Illinois April 26, 2004.

Documents

Transcript of PERFORMANCE MEASUREMENT: WHAT MIGHT BE AHEAD? Chicago, Illinois April 26, 2004.