Australian National Audit Office - Management of Learning and … · 2016-03-08 · and HRD. 12...

1

A u s t r a l i a n N a t i o n a l A u d i t O f f i c e

T h e A u d i t o r - G e n e r a lAudit Report No.64 2001–02

Performance Audit

Management of Learning and Developmentin the Australian Public Service

2 Management of Learning and Development in the Australian Public Service

© Commonwealthof Australia 2002

ISSN 1036–7632

ISBN 0 642 80652 7

COPYRIGHT INFORMATIONThis work is copyright. Apart fromany use as permitted under theCopyright Act 1968, no part may bereproduced by any process withoutprior written permission from theCommonwealth, available fromAusInfo. Requests and inquiriesconcerning reproduction and rightsshould be addressed to:

The Manager,Legislative Services,AusInfoGPO Box 1920Canberra ACT 2601

or by email:[email protected]

3

Canberra ACT28 June 2002

Dear Madam PresidentDear Mr Speaker

The Australian National Audit Office has undertaken a performance audit inaccordance with the authority contained in the Auditor-General Act 1997. Ipresent this report of this audit, and the accompanying brochure, to theParliament. The report is titled Management of Learning and Developmentin the Australian Public Service.

Following its tabling in Parliament, the report will be placed on the AustralianNational Audit Office’s Homepage—http://www.anao.gov.au.

Yours sincerely

P. J. BarrettAuditor-General

The Honourable the President of the SenateThe Honourable the Speaker of the House of RepresentativesParliament HouseCanberra ACT


AUDITING FOR AUSTRALIA

The Auditor-General is head of theAustralian National Audit Office. TheANAO assists the Auditor-General tocarry out his duties under the Auditor-General Act 1997 to undertakeperformance audits and financialstatement audits of Commonwealthpublic sector bodies and to provideindependent reports and advice forthe Parliament, the Government andthe community. The aim is to improveCommonwealth public sectoradministration and accountability.

Auditor-General reports are availablefrom Government Info Shops. Recenttitles are shown at the back of thisreport.

For further information contact:The Publications ManagerAustralian National Audit OfficeGPO Box 707Canberra ACT 2601

Telephone: (02) 6203 7505Fax: (02) 6203 7519Email: [email protected]

ANAO audit reports and informationabout the ANAO are available at ourinternet address:

http://www.anao.gov.au

Audit Team

Alison WyseAndrew HueyMichael Lewis

5

ContentsAbbreviations/Glossary 7

Summary and Recommendations 9

Summary 11Background 11Learning and development 11Audit objective and scope 12Audit methodology 12Overall conclusion 13Recommendations 14Agencies’ responses 14

Key Findings 18Alignment of Learning and Development with Corporate Goals 18Integration of Learning and Development 19Delivering Learning and Development 20Evaluation of Learning and Development 20Role of the PSMPC 22Better Practice Principles 22

Recommendations 24

Audit Findings and Conclusions 25

1. Introduction 27Background 27Learning and development 28Legislative environment 29Previous audit coverage and other reviews 30Audit objective and scope 31Audit methodology 32Report structure 33

2. Alignment of Learning and Development with Corporate Goals 35Background 35Learning and development and corporate planning 36Previous learning and development experience informing learning and

development planning 38Identifying skills required by the organisation 39Learning and development performance information frameworks 40Conclusion 41Better Practice Principles 43

3. Integration of Learning and Development with other Human ResourceManagement Functions 44

Background 44Learning and development strategies and plans as part of an overall

human resource management strategy 45Learning and development strategies and plans are linked to other

human resource management functions 48Bridging the skills gap of staff 52


Conclusion 56Better Practice Principles 57

4. Delivering Learning and Development 58Background 58Learning and development is delivered cost-effectively, using methods

that suit the nature of the learning and development to be delivered 59Learning and development costs are regularly monitored 62Staff uptake is regularly monitored 65SES learning and development 69Outsourcing of human resource management 70Contract management 71Conclusion 74Better Practice Principles 76

5. Evaluation of Learning and Development 77Background 77Aspects of learning and development evaluated 78Evaluation planning and activity 80Reliable evaluation data 82Evaluation of return on investment and value for money 84Evaluation outcomes and future learning and development planning 85Annual reporting requirements 86Conclusion 87Better Practice Principles 89

6. Role of the PSMPC 90Background 90PSMPC approach 91PSMPC’s facilitating role 91PSMPC’s advisory role 94PSMPC’s reporting and evaluation role 95PSMPC’s service delivery role 97Conclusion 100

Appendices 103

Appendix 1: Agencies completing the survey 105Appendix 2: PSMPC learning and development products 107

Series Titles 111Better Practice Guides 116

7

Glossary/Abbreviations

Abbreviations/GlossaryABS Australian Bureau of Statistics

Alignment Vertical agreement of strategies and structures withcorporate goals, cascading to lower level plans andstrategies

APS Australian Public Service

Business unit Division or branch within an agencies’ organisationalstructure

Cost-effectiveness Best outcome at lowest price

CPG Commonwealth Procurement Guidelines

DEST Department of Education, Science and Training(formerly Department of Employment, Training andYouth Affairs)

DEWR Department of Employment and Workplace Relations(formerly Department of Employment, WorkplaceRelations and Small Business)

DFAT Department of Foreign Affairs and Trade

DVA Department of Veterans’ Affairs

Efficiency Ratio of outputs to inputs

FaCS Department of Family and Community Services

HDA Higher Duties Allowance

Health Department of Health and Ageing (formerlyDepartment of Health and Aged Care)

HRD Human resource development, variously known astraining, learning, staff development, education

HRM Human resource management, collective term forintegration of all personnel activities (e.g.: recruitment,selection, training, industrial relations, OH&S, payroll,rewards and recognition)

HRMIS Human resource management information system

IiP Investors in People

Integration Horizontal compatibility of strategies and processes,sharing of common data sets etc to achieve commonoutcomes from related functions

PSC Public Service Commission


PSMPC Public Service and Merit Protection Commission

ROI Return on investment

SES Senior Executive Service

SoSR State of the Service Report

TNA Training needs analysis

VFM Value for money

9

Summary and

Recommendations

11

Summary

Background1. Learning and development is a key people management function that isundertaken in all Australian Public Service (APS) agencies. The efficiency andcost-effectiveness with which agencies are able to deliver their outputs, andultimately outcomes, depend in large part on the skills of their workforce.Agencies can improve their overall performance by making a sound strategicinvestment in learning and development.

2. The total cost of learning and development in the APS is not known.However, in 1999–2000, agencies estimated that they spent $109 million on formallearning and development. This translates to about $1000 per APS employee.This is a conservative estimate of the direct cost of learning and development inthe APS. The inclusion of expenditure on indirect learning and development,such as on-the-job training, would substantially increase this estimate. On-the-job training, however, this data is not available.

3. Based on the data collected from an APS-wide survey of learning anddevelopment undertaken for this audit, the ANAO has estimated totalexpenditure on formal learning and development in the APS in 2000–01 is around$160 million, or an average of $1616 per full-time ongoing employee1, excludingsalary and wage costs of participant attendance. This translates to about1.1 per cent of the cost of wages and salaries. This percentage compares withthat of expenditure on learning and development in 1996 by both the privateand public sectors in Australia of some 2.5 per cent of salary and 3.5 per cent byleading United States companies. On average, each full-time ongoing APSemployee spent approximately six days on learning and development activitiesin 2000–01. This represents approximately 2.7 per cent of available staff days inany year.

Learning and development4. Learning and development2 assist organisations to effectively achieve theircorporate and people management goals. Both learning and developmenttogether are necessary to achieve this. Learning encompasses the processes bywhich job-related skills are acquired, as demonstrated by changed behaviour

1 However, the lack of data on the cost of learning and development and the varying definitions oflearning and development used by agencies limit the accuracy of these estimates.

2 This function has previously been known as training and development, staff development and training,and HRD.


on the job. Development focuses on cultural change and career preparation. Ithas a relatively long time for payback and a higher risk of achieving that paybackthan does training.3

Audit objective and scope5. The objective of the audit was to examine and report on the keymanagement factors that assist in ensuring value for money in learning anddevelopment. Consistent with this objective, the audit focused on identifyinghow agencies:

• align their learning and development strategies and plans with theirbusiness mission and corporate goals;

• integrate their learning and development with other aspects of humanresource management;

• ensure that learning and development is delivered in an efficient and costeffective manner; and

• evaluate their learning and development strategies and activities todetermine effect, including business impact.

6. The ANAO limited its examination of learning and development in theAPS to formal measurable learning and development such as training programs,conferences, seminars and Studybank. It is recognised that on-the-job training,coaching, mentoring, development opportunities and the like are keycomponents of any learning and development strategy. However, these aspectswere not included in the audit partly because of their subjective nature and theabsence of any meaningful data in agencies. The role of the Public Service andMerit Protection Commission (PSMPC) in relation to learning and developmentin the APS was also included.

Audit methodology7. The audit was undertaken in two parts. First, an APS-wide survey wasconducted of all agencies with 100 or more full-time equivalent staff engagedunder the Public Service Act 1999. Second, a detailed examination of learningand development was undertaken in the following five agencies:

• Australian Bureau of Statistics (ABS);

• Department of Family and Community Services (FaCS);

3 J Phillips, Return on investment in training and performance improvement programs, Gulf Publishing,Houston, 1997, pp. 10-11.

13

Summary

• Department of Foreign Affairs and Trade (DFAT);

• Department of Health and Ageing (Health), formerly the Department ofHealth and Aged Care; and

• Department of Veterans’ Affairs (DVA).

Overall conclusion8. Learning and Development needs to be firmly and clearly aligned withcorporate goals, and supported by meaningful performance indicators formanagement and reporting purposes. While agencies had made significantefforts to ensure that learning and development strategies are properly alignedwith the business needs identified in their corporate plans, they were unable todemonstrate the efficiency and effectiveness of their investment in learning anddevelopment and its contribution to organisational effectiveness. Generally,learning and development strategies are not being evaluated by agencies. As aresult, it is not possible to determine how successful learning and developmentoutcomes have been in assisting agencies to achieve business outcomes.

9. The lack of supporting management information and performancemeasures, such as learning and development expenditure and staff participationrates, means that agencies are not well placed to demonstrate the efficiency orcost-effectiveness of their investment in learning and development.

10. Evaluation of learning and development has been mainly limited toparticipant satisfaction with courses rather than with cost-effectivenessconsiderations. Evaluation at the current level does not support the developmentof comprehensive learning and development strategies that target priority needsto ensure the right people, have the right skills, at the right time. However, itmust be recognised that agencies are constrained in their ability to conductcomprehensive evaluations by the general lack of appropriate performancetargets and data.

11. An integrated and coherent approach to people management givesagencies a greater likelihood of achieving their organisational outcomesefficiently and effectively. The majority of agencies indicated that their learningand development strategies were part of an integrated and coherent approachto people management. Indeed, the ANAO found that 80 per cent of all agencieshad a human resource management strategic plan that specifically includedlearning and development objectives. In addition, there were particularly stronglinks between learning and development and agencies’ certified agreements andperformance management arrangements. Although career or successionmanagement systems or workforce planning arrangements were much less


common, where they did exist, they had strong links to learning anddevelopment.

12. The PSMPC’s current role in APS-wide learning and development islargely directed at supporting the delivery of the training programs it offers.Although the PSMPC facilitates the sharing of information about learning anddevelopment across agencies, there is scope for the PSMPC to undertake a morecatalytic role, particularly by better targeting its facilitation efforts; and to enhanceits advisory and reporting roles, including reporting to Parliament, by thecollection and analysis of APS-wide data on learning and development. In adevolved environment, this would require the cooperation of all APS agencieswhich might require Ministerial direction to ensure full coverage andcooperation. At least there should be a reasonable assessment of the costs andbenefits involved at both an individual agency and whole of government levelwith the aggregate picture derived from, and complementary with, agencyaccountability and performance information. This would provide a perspectivewhich could enhance efficiency and effectiveness in this area.

Recommendations13. Three recommendations were developed to help improve the role andimpact of the PSMPC. They were agreed to by the PSMPC.

14. Although the report does not include any recommendations aimed at allAPS agencies, the better practice principles outlined at the end of four of thechapters are intended to provide agencies with guidance on current betterpractice in the management of learning and development.

15. The ANAO, in conjunction with the PSMPC and other APS agencies, willfurther expand on the better practice principles outlined in this report to developa Better Practice Guide on learning and development, in the next few months,to support agencies in the management of learning and development across theAPS.

Agencies’ responses

PSMPC

16. The PSMPC believes the audit will be valuable to the APS. However, it isnot at this point persuaded, that the costs of establishing a central collectionpoint for core data on costs and activity levels of learning and developmentacross the APS would be justified. There are also difficulties associated withmandating the collection of such data in a devolved environment.

15

Summary

17. Agencies are accountable for the efficiency and effectiveness of theirexpenditure on learning and development in the devolved APS environment.Agencies also have a prime responsibility for identifying and putting strategiesin place to meet their current and future capability needs.

18. Under the Public Service Act 1999, the role of the Commission is to facilitatecontinuous improvement in people management throughout the APS, tocoordinate and support APS-wide training and career developmentopportunities, and to contribute to and foster leadership in the APS. TheCommission has taken the view that, bearing in mind the responsibilities ofagencies, it should look primarily to complement and augment their activities,particularly in relation to leadership and people management. The Commissionis accountable for the effectiveness and efficiency of tis own programs, but notthose of other agencies.

19. The Commission undertakes cross-Service evaluations, particularly underthe auspices of the State of the Service Report (SoSR). Each year the SoSR includescommentary on learning and development activities across the Service. Underits new corporate plan, the Commission will place greater emphasis on cross-Service evaluation and look to provide increased support to agencies in areasrelated to learning and development.

Australian Bureau of Statistics

20. The ABS found the report to be most informative about matters such asalignment and integration with business and human resource strategy,evaluation, etc. Some good case studies were included and the better practiceprinciples will be very useful.

21. The emphasis of the report on the need to measure performance bettercannot be argued. The report will help agencies to focus better on what is possible.The recommendation that the PSMPC might play a coordinating role in providingsupport and establishing some APS-wide learning and developmentperformance information is supported.

Department of Family and Community Services

22. FaCS agrees with the overall findings of the report. The report outlinessome very good instances of best practice and identifies areas, particularly inrelation to evaluation, cost effectiveness and return on investment, where allagencies could improve. FaCS also believes that, with some of the peoplemanagement initiatives it is currently undertaking, it will be well placed toimplement many of the report’s best practice suggestions.


23. Although the report attempts to define what constitutes ‘learning anddevelopment’ it does not take sufficient account, of the move away from formallearning and development activities to those of a much more on the job nature,such as coaching. These less formal activities are far harder to quantify, both interms of time and cost involved.

24. The report discusses the lack of rigorous evaluation of learning anddevelopment activities against business outcomes. FaCS agrees that this is anarea of deficiency. However, it believes that the report does not give sufficientconsideration to the fact that learning and development activities are often aimedas much at effecting cultural change as they are at assisting to achieve specificbusiness outcomes. Measurement of this aspect would be difficult.

25. The differing learning and development requirements of agencies, andthe measurement difficulties associated with the on the job activities such ascoaching, mean that the suggested comparative data collection recommendedis not very useful for agencies such as FaCS. The fact that training activity isoften seen as the major data set for evaluation, plus the common practice ofaveraging out time and expenditure per employee, would no doubt be of valueto some agencies but of less value to FaCS.

Department of Foreign Affairs and Trade

26. While DFAT agrees in general with the report’s three recommendationsaimed at improving the role and impact of the PSMPC’s approach to learningand development across the APS, it considers that the agencies such as DFAT,which have a substantial training and development strategy and investment,should retain independent authority to manage their training and developmentprograms to meet their own requirements. In regard to the role of the PSMPC incoordinating and supporting APS-wide training and career developmentopportunities, the department submits that the devolution of authority toagencies under the Public Service Act 1999 has placed a premium on an effectiveworking relationship on human resource management issues between agenciesand the PSMPC. It is important that agencies such as DFAT have sufficientindependence to allow them to implement their devolved authority in waysthey consider make the best use of the available resources.

Department of Health and Ageing

27. Health supports the recommendations directed to the PSMPC andconsiders that the identification and promulgation of better practice principlesrelevant to the APS would assist greatly in supporting continuous learning. Theseprinciples should recognise that different organizations may have specificlearning and development needs depending on their specific business goals.

17

Summary

Department of Veterans’ Affairs

28. DVA considered the report findings a reasonable reflection of the learningand development activity in the Department. DVA broadly supports therecommendations on the role of the PSMPC, but cautions against a reportingregime for the State of the Service Report being developed that is more onerousto collect than its return value to the APS.

29. The department has taken steps to develop a National Learning Strategy(NLS) to ensure alignment and integration of learning and development efforts.Of particular importance is the requirement under the NLS for evaluation to beundertaken at four levels—satisfaction, behavioural or skills change in theworkplace, improvements in productivity and return on investment to DVA.The department anticipates the returns from a more structured approach tolearning and development will be worthwhile.


Key Findings

Alignment of Learning and Development withCorporate Goals30. The ANAO found that agencies had made significant efforts to ensurethat their learning and development strategies and plans are driven by thebusiness needs identified in their corporate plans. The majority (88%) of agenciesresponding to the survey indicated that their organisations included developingthe capacity of the organisation’s workforce as an objective in their corporateplan or equivalent document. Similarly, 84% of agencies indicated that they hada human resource management strategic plan; 95% specifically include learningand development objectives; and 60% stated that they had a separatelydocumented learning and development strategy.

31. To ensure that learning and development strategies and plans remainaligned with corporate goals and address priority needs, the strategies and plansshould be informed by regular reviews of learning and development initiatives.Reviews have varied from the assessment of the impact of agencies’ learningand development strategies to consultation processes regarding the programsoffered through their learning and development calendars. Of the 39 agencieswith a learning and development strategy, some 85% had reviewed their learningand development strategies, at sometime, with 28 agencies having reviewedtheir strategy in the previous 12 months.

32. The ANAO considers that, for learning and development strategies andplans to be fully aligned with agencies’ corporate goals, they should addressboth their current and future skills requirements. Both the agencies surveyedand those examined in detail, used various methods to determine futureworkforce skill requirements. The most common methods included training needanalyses, future scenario planning and consultation with senior executives. Themajority of agencies (70%) advised that, generally, high-priority skills, asidentified by a training needs analysis, are incorporated into their learning anddevelopment strategies and plans. However, some agencies advised the ANAOthat learning and development is intuitively based and, in some cases, reflectsmore short-term considerations rather than longer term strategic needs.

33. Better practice APS agencies, as part of their learning and developmentstrategies, should have developed output (activity) and outcome (effectiveness)performance indicators that inform and support the planning for, delivery andassessment of, learning and development. The ANAO found that the majorityof APS agencies (66%) have learning and development performance indicators,

19

Key Findings

as part of their performance information frameworks. However, these aregenerally measures of activity, rather than measures of learning and developmenteffectiveness. The ANAO recognises that the latter measures are difficult toarticulate clearly let alone qualify. Indeed, qualitative assessments will also berequired.

Integration of Learning and Development34. Without the integration of related human resources functions, agenciesare at a risk of duplication of effort, funding programs with competing purposes,and a loss of synergy with adverse consequences for their efficiency andeffectiveness. An integrated and coherent approach to people management givesagencies a greater likelihood of achieving their outcomes efficiently andeffectively. The majority of agencies (83%) indicated that their organisation’slearning and development strategy was part of an integrated and coherentapproach to human resource management and 80% of all agencies had a humanresource management strategic plan that specifically included learning anddevelopment objectives. These findings were reflected in the five agenciesexamined in detail.

35. The ANAO found that there were particularly strong links betweenlearning and development and agencies’ certified agreements and performancemanagement arrangements. Although, career or succession management systemsor workforce planning arrangements were in the minority (24% of agencies),where they did exist, they had strong links to learning and development.

36. Learning and development that is linked to other human resourcefunctions helps to ensure that those staff with skills gaps, not just the skills gapsthemselves, are identified. Although the percentage of agencies that conductskills audits of staff is small (34%), 56% use the information for learning anddevelopment planning purposes. More commonly, training needs analyses wasused as a basis for identifying high priority skills requirements in learning anddevelopment strategies and plans.

37. The ANAO noted that managers played an important role in identifyinglearning and development needs and ensuring that staff were released fortraining, but were less involved in evaluating the results of the training back inthe workplace.

38. The use of human resource management information systems to recordlearning and development data was not widespread (25% of agencies). Morecommon was the use of stand-alone databases (33%) for such purposes.


Delivering Learning and Development39. Without a clear and consistent basis for accurately identifying theexpenditure on learning and development, the extent to which groups of staffavailed themselves of opportunities, or the cost-effectiveness of the variousdelivery methods used, agencies are unable to assess the extent to which theirlearning and development is delivered efficiently and is achieving the resultsrequired.

40. Although the majority of agencies (80%) held the view that their learningand development was delivered in an efficient and cost-effective manner, only63% of agencies were able to provide even basic data to the ANAO on totalexpenditure on learning and development. Expenditure on some componentsof learning and development, especially the cost of external providers and coursecontent material, was well known. However, few agencies were able to providedetails of the costs of different training methods. Data on staff training dayswas similarly patchy with only 34% of agencies able to provide the necessaryinformation.

41. On the other hand, while the majority of agencies advised that theyproduce reports on course attendances and cost and budget reports, only a smallminority could produce reports which compare staff participation and skillingneeds against corporate goals.

42. Agency responses indicated that little data is collected by them on SeniorExecutive Service (SES) learning and development, which makes it difficult forthem to assess the efficiency and cost-effectiveness of SES learning anddevelopment.

43. Without a solid information base for assessing efficient and cost-effectivedelivery of learning and development, agencies are also not well placed to makeinformed decisions on the outsourcing of the learning and development function.

Evaluation of Learning and Development44. A major goal of learning and development evaluation is to improve currentlearning and development processes in order to achieve maximum businessimpact. Through evaluation, agencies can also assess the efficiency and cost-effectiveness of their learning and development strategies and activities. Theseresults then form an important part of the information base for managementdecision-making and future learning and development planning and delivery.

45. Of the 67 agencies surveyed, 27 (40%) either agreed or strongly agreedthat their organisations evaluate the effectiveness of their learning anddevelopment strategies, including the impact of learning and development on

21

Key Findings

the organisational goals. However, 45% of agencies neither agreed nor disagreedand a further 15% disagreed.

46. The ANAO found that, of the 34 agencies that have evaluated at least oneaspect of their learning and development activities, 94% evaluate participantsatisfaction and 80% evaluate both their major learning and developmentactivities and the effectiveness of individual learning and development activitiesin meeting their workplace skills needs. This experience is reflected in the fiveagencies examined in detail.

47. However, the ANAO considers that evaluation just at an activity levellimits its potential usefulness. Evaluation at this level does not support thedevelopment of comprehensive learning and development strategies that targetpriority needs to ensure the right people, have the right skills, at the right time.

48. To assess the impact and value of their investment in learning anddevelopment, organisations should evaluate their learning strategies andactivities on a regular basis in a planned and structured manner. Of the34 agencies that have evaluated their learning and development activities,25 (74%) had an overall learning and development evaluation plan. Of these25 agencies, 14 also advised that they had evaluated their learning anddevelopment strategies.

49. The ANAO found that the timing of evaluation activity varied betweenagencies. As noted earlier, agencies most frequently evaluate participantsatisfaction. This is undertaken on a routine basis. However, the review ofagencies’ learning and development strategies is less frequent.

50. For learning and development evaluation to be able to achieve requiredoutcomes, the ANAO considers that access to reliable data is required. The ANAOfound that the scope of learning and development data available from agencieswas limited.

51. The ANAO found that most agencies either have not undertaken studiesof, or are generally unable to assess, the return on investment (ROI) or value formoney of their outlays on learning and development. This is largely due to thelimited availability of learning and development data. For this reason, the ANAOconsiders the assessment of learning and development ROI in the APS is oflimited use in the current environment. The ANAO found that some agencieshave evaluated the value for money of their learning and development activities,but this has largely been for contract purposes. The ANAO considers thecollection of reliable learning and development data would significantly enhanceagencies abilities to both plan for, and evaluate, the impact of their learning anddevelopment strategies and activities.


52. The ANAO observed that 84% of agencies that have recently evaluatedtheir learning and development strategies or activities found the outcomes usefulin both informing their learning and development planning and for managementdecision-making.

Role of the PSMPC53. The ANAO found that the PSMPC facilitates the sharing of informationabout learning and development across agencies by coordinating an extensiveseries of human resource management networks and working collaborativelywith agencies. However, there is scope for the PSMPC to undertake a morecatalytic role. In particular, better targeting of its facilitation efforts to ensurecontinuous improvement in learning and development.

54. The PSMPC advised that it provides advice to Government and agencyheads and managers on strategic people management and organisationalperformance issues, including learning and development. However, the ANAOwas unable to identify regular, documented specific advice to agencymanagement on such issues. The ANAO also considers that the quality of adviceto Government and agency heads would be enhanced by the collection andanalysis of APS-wide data on learning and development.

55. The PSMPC’s ability to adequately report to Parliament on learning anddevelopment, as part of its role of reporting on the performance of the APS inthe State of the Service report, is dependent on it undertaking systematicevaluations and routinely collecting supporting data. The ANAO found thatthere is a high priority need for the systematic collection and analysis of basiclearning and development performance information to underpin reporting andaccountability to Parliament. The PSMPC is in a good position to coordinate thecollection of consistent data and facilitate whole of government reporting on all(not just learning and development) significant people issues.

56. The ANAO found that the PSMPC devoted considerable effort to itsservice-wide development activities and provided a comprehensive range ofrelevant learning and development opportunities for its several client groups.However, the efficiency and cost-effectiveness of its development activities couldbe better measured and/or assessed for improved results.

Better Practice Principles57. At the end of each of four chapters of this report, the ANAO has includeda section on better practice principles. The principles have been drawn fromobservations made during the audit and from consideration of industry trends,

23

Key Findings

both within Australia and overseas. The aim of documenting these principles isto assist agencies to improve the management of their learning and development.The principles are not exhaustive. The aim is to provide agencies with adequateguidance on current better practice in the management of learning anddevelopment. The suggested performance indicators that support the betterpractice principles are summarised in Figure 1.

Figure 1Performance information and strategic learning and development

•L&D addressing business needs.

•L&D closing the gap between current and

future skills needs.

•Links between human resource

management and L&D.

•Common/shared human resource

management and L&D data.

•L&D KRA’s monitored and reported

on.

•Critical jobs and roles identified and

addressed.

•Staff days allocated to learning.

•Expenditure on learning and development.

•Uptake of learning and development.

•Learning and development targets.

•Ratio of human resource management staff

to total employees.

• Organisational impact of L&D.

•Continuous improvement of L&D.

•Evaluation results inform L&D

planning.

Learning and

development

(L&D)

Alignment

Evaluation

Integration

Delivery

Source: ANAO


Recommendations

Set out below are the ANAO’s recommendations aimed at improving the role andimpact of the PSMPC’s approach to learning and development across the APS. Allrecommendations are considered equally important.

Recommendation The ANAO recommends that, in order to supportNo. 1 continuous improvement in learning and developmentPara. 6.20 across the APS, the PSMPC further explore with

stakeholders, the scope for a more targeted facilitationrole, including the identification and promulgation ofbetter practice principles relevant to the APSenvironment.

PSMPC Response: Agreed.

Recommendation The ANAO recommends that, in order to assist agenciesNo. 2 in evaluating the impact of their learning andPara. 6.34 development and to assist Parliament in monitoring

APS-wide learning and development, the PSMPC:

• include learning and development as part of anevaluation strategy and plan undertaken under theumbrella of the State of the Service Report;

• provide guidance and support to agencies onevaluating the impact of learning and development;

• identify key APS-wide learning and developmentperformance indicators; and

• include an analysis of trends in learning anddevelopment in the State of the Service Report.


Recommendation The ANAO recommends that the PSMPC, in theNo. 3: interests of efficient and cost-effective delivery ofPara. 6.48 learning and development across the APS, regularly

assess the value for money provided by their servicesand implement an evaluation strategy for the programsoffered by the PSMPC.


25

Audit Findings

and Conclusions

27

1. Introduction

Background1.1 Learning and development is a key people management function that isundertaken in all APS agencies. The efficiency and cost-effectiveness with whichagencies are able to deliver their outputs, and ultimately outcomes, depend inlarge part on the skills of their workforce. Agencies can improve their overallperformance by making a sound strategic investment in learning anddevelopment.

1.2 The total cost of learning and development in the APS is not known.However, in 1999–2000, agencies estimated direct spending of $109 million onformal learning and development. This translates to about $1000 per APSemployee. This is a conservative estimate of the direct cost of learning anddevelopment in the APS. The inclusion of expenditure on indirect learning anddevelopment, such as on-the-job training, would substantially increase thisestimate.

1.3 Based on the data collected from an APS-wide survey of learning anddevelopment undertaken for this audit, the ANAO has estimated totalexpenditure on formal learning and development in the APS in 2000–01 at around$160 million, or an average of $1616 per full-time ongoing employee, excludingsalary and wage costs of participant attendance. This translates to about 1.1% ofthe cost of wages and salaries. This percentage compares with that of expenditureon learning and development by both the private and public sectors in Australiaof some 2.5% in 1996, the latest figures available. On average, each full-timeongoing APS employee spent approximately six days on learning anddevelopment activities in 2000–01. This represents approximately 2.7% ofavailable staff days4 in any year. However, the lack of data on the cost of learningand development and the varying definitions of learning and development usedby agencies, limits the accuracy of these estimates.

1.4 By way of comparison, in 1999 leading companies in the United States(US) spent an average of 3.5% of payroll costs on training.5 The study fromwhich these figures are drawn also identified that training expenditure for allUS companies decreased from 2.0% of payroll in 1998 to 1.8% in 1999.

4 Variously estimated at 210–220 days per year.5 This data was reported in an extract from The 2001 ASTD State of the Industry Report.


1.5 Although no recent national Australian data were publicly available atthe time of the audit, the ABS is currently undertaking a Training ExpenditureSurvey, under the sponsorship of the Australian National Training Authority.The survey will target 6000 organisations across Australia from both the publicand private sectors. The results of the survey are expected to be available inMarch 2003.

Learning and development1.6 There is no generally accepted industry and/or academic definition oflearning and development. Accordingly, the following definition of learningand development has been drawn together from several sources.

1.7 Learning and development6 assist organisations to effectively achieve theircorporate and people management goals. Both learning and developmenttogether are necessary to achieve this outcome. Learning encompasses theprocesses by which job-related skills are acquired, as demonstrated by changedbehaviour on the job. Whereas development focuses on cultural change andcareer preparation. It has a relatively long time for payback and a higher risk ofever achieving that payback than does training.7 Learning and developmentalso brings knowledge management and learning organisation principles andprocesses into alignment with people management, and training anddevelopment.

1.8 The major learning and development activities in the APS are:

• external training programs;

• external conferences and seminars;

• agency supported formal study;

• in-house training programs;

• courses provided by government for government; and

• external development programs.

1.9 Other activities that are considered to be learning and development includeon-the-job training, computer tutorials and targeted reading programs.

6 This function has previously been known as training and development, staff development and training,and HRD.

7 J Phillips, Return on investment in training and performance improvement programs, Gulf Publishing,Houston, pp. 10–11.

29

Introduction

Legislative environment1.10 The Public Service Act 1999 was introduced in December 1999. Thelegislation is principles-based and focuses on promoting the APS Values andthe APS Code of Conduct. One of the main objectives of the Public Service Act1999 is:

to establish an apolitical public service that is efficient and effective in serving theGovernment, the Parliament and the Australian public.8

1.11 The Public Service Act 1999 defines the role of the Public Sector and MeritProtection Commission. Section 41 of the Public Service Act 1999 outlines theCommissioner’s functions including his/her role in relation to learning anddevelopment in the APS. Sub-section 41(1) of the Public Service Act 1999 statesthat the Commissioner’s functions include:

(g) to develop, promote, review and evaluate APS employment policies andpractices;

(h) to facilitate the continuous improvement in people managementthroughout the APS; and

(i) to coordinate and support APS-wide training and career developmentopportunities in the APS.

1.12 The Public Service Act 1999 devolved responsibility for departmentalmanagement to agency heads. Section 57 of the Public Service Act 1999 outlinesthe responsibilities of Departmental Secretaries. Although the Public Service Act1999 does not prescribe a specific role for agency heads in relation to learningand development, agency heads are accountable for the achievement ofdepartmental outputs and outcomes. Learning and development is a tool tosupport the enhancement of organisational capability, which ultimately assistswith the achievement of departmental outputs and outcomes.

Annual reporting requirements

1.13 Under the provisions of the Public Service Act 1999, at the end of eachfinancial year the Commissioner is required to report to the Agency Minister forpresentation to the Parliament, on the activities of the PSMPC during the year.The report must also include a report on the state of the APS during the year9,and be prepared in accordance with the Annual Reporting Requirements asapproved by the Joint Committee of Public Accounts and Audit (JCPAA). The

8 Public Service Act 1999, Section 3 (a).9 Public Service Act 1999, Section 44 (2).


Commissioner, under the provisions of the Public Service Act 1999, can requestthat agency heads provide whatever information is required to facilitatepreparation of the report.

1.14 Agencies are also required to prepare an annual report and provide acopy to their Minister for tabling in the Parliament. The Annual ReportingRequirements issued by the Department of the Prime Minister and Cabinet onbehalf of the JCPAA, specifically requires agencies to report on human resourcemanagement capability and suggests that agencies report on training anddevelopment (sic). The Requirements state that:

(a) The annual report must include an assessment of the department’seffectiveness in managing and developing its staff to achieve its objectives.The focus is on the human resource capability at year end. This may includeconsideration of:

• … the department’s key training and development strategies, the outcomesof training and development, and evaluation of effectiveness;10

Previous audit coverage and other reviews1.15 Learning and development has been the subject of previous audit coverageand other reviews both within Australia and overseas. The findings of severalprevious audits were considered in the planning for this audit. These reportsinclude:

• ANAO Report No.37, 1991–92: The Management of Training in SelectedCentral Office Departments;

• ANAO Report No.27, 1994–95: Management of Study Bank;

• ANAO Report No.40, 1996–97: Human Resource Management in DFAT;

• ANAO Report No.9, 2001–02: Learning for Skills and Knowledge—CustomerService Officers;

• ANAO Report No.21, 2001–2002: Developing Policy Advice;

• Victorian Audit Office Special Report No. 32, 1994: A Competent Workforce:Professional Development;

• Irish Audit Office Report No.38, December 2000: Training and Developmentin the Civil Service;

• GAO/OGC-00-14G, September 2000: Human Capital: A self assessmentchecklist for agency leaders;

10 Prime Minister and Cabinet, JCPAA—Requirements for Annual Reports, 2001, p. 8.

31

Introduction

• OPM, Office of Merit Systems Oversight and Effectiveness, December 1998:HRM Accountability System Development Guide;

• OPM, Office of Merit Systems Oversight and Effectiveness, September1999: Strategic Human Resource Management: Aligning with the Mission; and

• State Service Report, Te Komihana O Nga Tari Kawantanga, State ServicesCommission, New Zealand, August 2001, Working Paper No. 12:A Framework for Measuring Training and Development in the State Sector.

1.16 The 1992 ANAO audit report Management of Central Office Training inSelected Departments reported on the findings of an efficiency review of themanagement of central office staff training in four departments and the role ofthe then Public Service Commissioner. The report suggested a framework fortraining in departments, which would aid in improving the efficiency andeffectiveness of training delivery. It provided guidance to departments on threekey issues: the relationship of training to departmental objectives; systematicappreciation of training needs; and an evaluation framework for training. TheANAO found that several of the issues raised in the report are still current today.These include the:

• need for a more systematic analysis of training needs;

• difficulty of establishing performance measures for training;

• limited assessment of the workplace impact of training; and

• need to clarify the role of the then Public Service Commission in relationto training.

Audit objective and scope1.17 The objective of the audit was to examine and report on the keymanagement factors that assist in ensuring value for money in targeting anddelivering learning and development. Consistent with this objective, the auditfocused on identifying how agencies:

• align their learning and development strategies and plans with theirbusiness mission and corporate goals;

• integrate their learning and development with other aspects of humanresource management;

• ensure that learning and development is delivered in an efficient and costeffective manner; and

• evaluate their learning and development strategies and activities todetermine effect, including business impact.


1.18 The ANAO limited its examination of learning and development in theAPS to formal, measurable learning and development. The role of the PSMPCin relation to learning and development in the APS was also examined.

Audit methodology1.19 The audit was undertaken in two parts. First, an APS-wide survey wasconducted of all agencies with 100 or more full-time equivalent staff engagedunder the Public Service Act 1999. Second, a detailed examination of learningand development was undertaken in the following five agencies:

• Australian Bureau of Statistics (ABS);

• Department of Family and Community Services (FaCS);

• Department of Foreign Affairs and Trade (DFAT);

• Department of Health and Ageing (Health), formerly the Department ofHealth and Aged Care; and

• Department of Veterans’ Affairs (DVA).

1.20 Audit fieldwork was conducted between August 2001 and January 2002.The audit was conducted in accordance with ANAO auditing standards at acost of $410 000.

APS-wide survey methodology

1.21 The APS-wide survey explored a range of issues from the alignment ofagencies’ learning and development strategies and plans with their corporategoals to the evaluation of learning and development and agencies’ use of thePSMPC’s HR Capability Model and Senior Executive Leadership Capability Framework.

1.22 A 100% response rate to the APS-wide survey was achieved with67 agencies completing the questionnaire. A list of these agencies is included asAppendix 1. To support the analysis of the APS-wide survey findings, theagencies were grouped according to their predominant role and size. Agencieswith fewer than 1000 full-time equivalent staff were classified as smallorganisations and represented 73% of respondents. Of the 67 agencies, 16% wereclassified as policy agencies. Eighty-four per cent were grouped as servicedelivery agencies. The agencies were also grouped accordingly to their learningand development and human resource management structures, as follows:

• 40% of agencies have implemented devolved models of both humanresource management and learning and development;

33

Introduction

Integration of learning and development with other human resource management functions

Delivering learning and development

Alignment of learning and

development with corporate

goals

learning and development Evaluation of Learning and

development (L&D)

• 37% of agencies have centralised their human resource management andlearning and development functions;

• 19% of the agencies surveyed have a centralised human resourcemanagement function and a decentralised learning and developmentmodel; and

• 4% of agencies have a decentralised human resource managementstructure and centralised learning and development function.

Report structure1.23 This report is structured around a four-part strategic model of learningand development, which aligns with the objective of the audit. Figure 2 outlinesthe four components of this model and the relationships between them.

Figure 2Strategic model of learning and development

Source: ANAO

1.24 The report addresses each of the four key aspects of learning anddevelopment examined during the audit and is structured as follows:

• Chapter 1—Introduction

• Chapter 2—Alignment of Learning and Development with CorporateGoals

• Chapter 3—Integration of Learning and Development with Other HumanResource Management Functions

• Chapter 4—Delivering Learning and Development

• Chapter 5—Evaluation of Learning and Development


• Chapter 6—Role of the PSMPC

1.25 This report aims to assist agencies to achieve maximum strategic businessimpact from their investment in learning and development, by identifying andpromoting, better practice in the management of learning and developmentacross the APS.

1.26 At the end of each of four chapters of this report, the ANAO has includeda section on better practice principles. The principles have been drawn fromobservations made during the audit and from consideration of industry trends,both within Australia and overseas. The aim of documenting these principles isto assist agencies to improve the management of their learning and development.The principles are not exhaustive. The aim is to provide agencies with adequateguidance on current better practice in the management of learning anddevelopment.

35

2. Alignment of Learning andDevelopment with Corporate Goals

This chapter examines the alignment of agencies’ learning and development strategiesand plans with their corporate goals. It discusses the extent to which agencies haveachieved this and its effect on the planning for learning and development.

Background2.1 For learning and development activities to achieve maximum business impact,agencies’ learning and development strategies and plans should be consistent withtheir corporate goals at all organisational levels (alignment). The ANAO considersthat, within such a framework, agencies’ learning and development strategies shouldbe high level documents, focused on developing the capability of their workforce.On the other hand, agencies’ learning and development plans are operationaldocuments that translate these learning and development strategies into practiceand support the delivery of learning and development. Accordingly, agencies’learning and development strategies may have a life of several years, while theirlearning and development plans are annual planning documents that are influencedby emerging needs and available resources.

2.2 As part of the APS-wide survey agencies were asked to rate themselvesagainst the question: To what extent would you agree with the following statement:our organisation’s learning and development plans and strategies are aligned with ourorganisational mission and goals. Agencies’ self-rated assessment of alignment waspositive. Of the surveyed agencies, 38% strongly agreed; 59% agreed; and 3%neither agreed nor disagreed. Given these results, the ANAO aimed to identifyif factors, such as agencies’ structures or functions, influenced their responses.

2.3 The ANAO found that the size of an agency does not have a significant impacton the achievement of alignment. However, the predominant role or function ofagencies influences alignment. For example, 64% of policy agencies rated themselvesas having achieved alignment, while only 32% of service delivery agencies ratedthemselves in this category. In their responses to the APS-wide survey questions onalignment, agencies with a decentralised human resource management functionalso indicated that they have achieved a greater level of alignment.

2.4 To be able to demonstrate effective alignment, the ANAO considers thatagencies’ learning and development strategies and plans should be:

• driven by business needs as identified in their corporate plans;

• informed by agencies’ previous learning and development experience;


• identifying the organisation’s current and future skill requirements; and

• supported by a performance information framework that informs futurelearning and development planning and measures the effectiveness oflearning and development.

2.5 This chapter discusses each of these issues and presents examples ofcurrent better practice identified during the audit.

Learning and development and corporate planning2.6 The ANAO observed that a strong senior management commitment tolearning and development influences the necessary alignment discussed earlier.In several of the agencies examined, senior management has made a clear andongoing commitment to learning and development. In these agencies learningand development is viewed as a strategic business partner as opposed to anancillary support function. This commitment to learning and development isgenerally reflected in agencies’ corporate plans and other planning documents.

2.7 In their responses to the APS-wide survey, 88% of agencies indicated thatthey have included an objective pertaining to the development of their workforcein their corporate plan or equivalent document.

2.8 All five agencies examined during the audit identified the need to developthe capability of their workforce in their corporate plans. For example, one ofthe objectives included in the ABS Corporate Plan is:

an organisation that encourages learning and innovation, performance andexcellence in all it does.11

2.9 The ANAO observed that 84% of the surveyed agencies have a humanresource management strategic plan and that 95% of these specifically includelearning and development. This demonstrates agencies’ high-level commitmentto learning and development.

2.10 At the operational level, 39 of the 65 agencies (60%) that responded to thisgroup of questions in the APS-wide survey, had a separately documentedlearning and development strategy. With 33 of the agencies advising that theirlearning and development strategy covered all staff. The ANAO found that,where agencies have a separately documented learning and developmentstrategy, the nature and content of these varied considerably. However, thelearning and development strategies examined in the five agencies during theaudit, aimed to align learning and development with corporate goals.

11 Australian Bureau of Statistics, Corporate Plan, ABS, Canberra, 2000, p. 12.

37

Alignment of Learning and Development with Corporate Goals

2.11 The ANAO found that several of the agencies examined during the audithave adopted a ‘line of sight’ or similar integrated planning framework. Thesemodels of planning highlight the linkages and/or relationships between thedifferent levels of planning across the organisations. For example, Case Study 1briefly outlines relevant aspects of DFAT’s integrated planning model.

Case Study 1Department of Foreign Affairs and Trade—

Integrated Planning FrameworkDFAT has implemented a corporate planning framework, which incorporateshuman resource management and learning and development outcomes. TheCorporate Plan 2000–2002 identifies key elements of the professionalism of itspeople. These include:

• commitment to the acquisition and maintenance of relevant foreignlanguage skills;

• flexibility and adaptability necessary to work effectively on the manydifferent issues faced and in the various environments in which theDepartment operates; and

• commitment to providing the best possible value for money throughcontinuous improvement in efficiency and productivity.

As part of the 1998–2000 Certified Agreement, the Department made acommitment to develop a strategic plan for people management. This StrategicPeople Plan aims to attract and retain skilled staff, offer professionallyrewarding career opportunities, incentives for high performance, attractiveconditions of service, and fair and transparent staffing processes.

Within the context of these strategic objectives, the Department’s Trainingand Development Strategy has been framed. Its underlying premise andphilosophy is that the Department will take a strategic and coordinatedapproach to the management of its training and development needs, andensure that this is linked to performance management. The Departmentadvanced the argument that training for training’s sake cannot be afforded.

Learning and development opportunities offered by the Staff Developmentand Post Issues Branch cover all critical capabilities, as well as developmentopportunities and re-orientation for staff returning from overseas postings.The associated planning and monitoring systems forecast training needs. Theachievement of training and development outcomes is systematicallymonitored and evaluated through the Department’s evaluation processes forposts, divisions and State offices and supports future training anddevelopment planning.


Devolved responsibilities

2.12 Collectively, 59% of the agencies surveyed have implemented a devolvedmodel of learning and development as part of their learning and developmentstrategy. This includes four of the five agencies examined during the audit. TheANAO identified that agencies have adopted various approaches in thedevolution of learning and development. Some agencies have devolved boththe responsibility for, and funding of, learning and development to businessunits. Other agencies have maintained a corporately funded centralised trainingand development capability, but devolved responsibility for learning anddevelopment across the organisation. The ANAO considers that several of theagencies examined in detail would benefit from further coordination andimproved integration of their learning and development. It is anticipated thatthis would result in:

• the development and implementation of more targeted learning anddevelopment programs, which address business needs, at both thestrategic and operational levels; and

• improved economies of scale in the development of learning anddevelopment programs.

Previous learning and development experienceinforming learning and development planning2.13 Agencies’ learning and development strategies and plans should beinformed by their ongoing reviews of learning and development and experiencein the planning for and delivery of, learning and development. This approachshould assist agencies to ensure that their learning and development strategiesand plans remain aligned with their corporate goals; address priority needs;and focus on learning and development delivery methods, which achievemaximum business impact in an effective manner.

2.14 To some extent, agencies’ experiences with learning and development haveinformed their future learning and development planning. The scope of agencies’reviews has varied from consultation processes regarding the programs offeredthrough their learning and development calendars, to extensive reviews of theimpact of agencies’ learning and development strategies. In response to the APS-wide survey, 33 (49%) agencies indicated that they have reviewed their learningand development strategies. This is a positive result, and represents 85% of the39 agencies that indicated they had a learning and development strategy orequivalent. Of the 33 agencies that had reviewed their learning and developmentstrategies, 28 had reviewed their strategy in the last 12 months. With 19 agencies

39


indicating that they are planning to review their learning and developmentstrategies in the next 12 months.

Identifying skills required by the organisation2.15 The ANAO considers that, in order for agencies’ learning and developmentstrategies and plans to be fully aligned with their corporate goals, they shouldaddress both the organisation’s current and future skills requirements.

2.16 The ANAO found agencies use various methods to identify futureworkforce skills requirements. In the APS-wide survey, agencies were asked to:indicate how future workforce skill requirements are identified by your organisation?Agencies’ responses included:

• consultation with their senior executive (84%);

• external information such as overseas trends (24%);

• other methods, including internal training committees and performanceevaluation processes (22%); and

• engagement of external expertise (21%).12

2.17 The ANAO also examined how agencies use training needs analyses toinform learning and development planning. The majority of agencies (70%)advised that, generally, high-priority skills as identified by a training needsanalysis, are incorporated into their learning and development strategies andplans. However, some agencies advised the ANAO that learning anddevelopment is intuitively based and, in some cases, reactive. Agencies top fivelearning and development priorities as identified in the APS-wide surveyincluded: leadership (59%), project management (33%), other ‘not-specified’(30%), contract management (27%) and accrual accounting or other financial(15%).

2.18 The majority of the five agencies examined during the audit had identifiedand documented their current and future organisational skills requirements.These requirements had been identified through a consolidation of data fromseveral sources including training needs analyses, future scenario planning andconsultation with senior executives. The level of detail outlined in the documentsvaried between agencies. In some agencies, these are strategic documents focusedon broad generic and or corporate skills. Other agencies have detailed theirskills requirements at the branch and/or business unit level. Health hasdeveloped a capability framework, which covers both of these areas; this isoutlined in Case Study 2.

12 Agencies may use a combination of these methods, as this was a multiple response question, allpercentages have been rounded to the nearest whole number.


Case Study 2Department of Health and Ageing’s Capability Framework

Health has identified and documented its current and future skillsrequirements. These cover both its core and business unit specific knowledgeand abilities and are outlined in the document Knowledge and Abilities and inindividual Business Unit Statements of Knowledge and Abilities.

The document Knowledge and Abilities outlines the broad departmentalknowledge and abilities that Health needs to meet its corporate objectivesand also its business unit specific knowledge and abilities required to achieveits business outcomes. The document also outlines a number of corporatelearning and development priorities for the Department.

The development of Health’s workforce against these capabilities is achievedthrough integration with its performance management system. As part ofthis process, staff’s skills levels against the capabilities are identified and theirfurther development planned. This focus on capability development is anintegral part of Health’s integrated approach to people management andlearning and development.

Learning and development performance informationframeworks2.19 The ANAO considers that better practice APS agencies, as part of theirlearning and development strategies, will have developed performanceindicators that inform and support the planning for, delivery and assessmentof, learning and development. These indicators should include a combinationof quantitative output (activity) and outcome (effectiveness) measures, with theoutcome measures focused on assessing the business impact or effectiveness ofagencies’ learning and development efforts.

2.20 The majority of APS agencies surveyed (66%) indicated that they havelearning and development performance indicators, as part of their performanceinformation frameworks. However, an analysis of the learning and developmentperformance indicators of the five agencies examined during the audit identifiedthat these are generally measures of activity, such as training days by APS-levelor organisational unit. The ANAO found that the agencies’ performanceindicators do not generally include measures of learning and developmenteffectiveness. The ANAO observed that this is a common issue for organisations.Kaplan and Norton noted from their research on organisational capability that:

… companies have devoted virtually no effort for measuring either the outcomesor the drivers of these capabilities [learning and growth] … it reflects the limited

41


progress that most organisations have made in linking employees, informationsystems and organisational alignment with their strategic objectives.13

2.21 Ultimately, changed behaviour and performance improvement in theworkplace are the true indicators of the effectiveness of learning anddevelopment. Agencies should have regard for this in developing theirperformance measures and establish a range of effectiveness measures, whichalign with their learning and development strategies and corporate goals.

2.22 The ANAO noted that the majority of agencies surveyed reported usingtheir learning and development performance indicators to also inform futurelearning and development planning. Of the 67 agencies surveyed, 28 (65%) ofthe responding agencies advised that their learning and developmentperformance indicators are actively used as a tool to inform learning anddevelopment planning. However, 28% of agencies responded that performanceagainst the indicators is not reported. On the other hand, the ANAO noted thatthree of the agencies examined report progress against the indicators to theiragency executive through their human resource management committees orsimilar bodies.

2.23 In addition to the use of performance indicators to monitor learning anddevelopment and inform future learning and development planning, the ANAOfound that four of the agencies examined have undertaken general staff surveys.These included questions about the effectiveness of learning and developmentand priority topics for learning and development. Further, 37% of the agenciessurveyed had undertaken a staff survey in 2000–01. Of these 25 agencies, 19agreed or strongly agreed that their staff survey informed learning anddevelopment planning.

Conclusion2.24 The ANAO found that agencies had made significant efforts to ensurethat their learning and development strategies and plans are driven by thebusiness needs identified in their corporate plans. The majority (88%) of agenciesresponding to the survey indicated that their organisations included developingthe capacity of the organisation’s workforce as an objective in their corporateplan or equivalent document. Similarly, 84% of agencies indicated that they hada human resource management strategic plan; 95% specifically include learningand development objectives; and 60% stated that they had a separatelydocumented learning and development strategy.

13 R Kaplan and D Norton, ‘Using the Balanced Scorecard as a Strategic Management System’, HarvardBusiness Review, January–February, 1996, pp. 75-85.


2.25 To ensure that learning and development strategies and plans remainaligned with corporate goals and address priority needs, the strategies and plansshould be informed by regular reviews of learning and development initiatives.The ANAO concluded that agencies’ experience with learning and developmenthas, to some extent, informed their future learning and development planning.Past reviews have varied from the assessment of the impact of agencies’ learningand development strategies to consultation processes regarding the programsoffered through their learning and development calendars. Of the 39 agencieswith a learning and development strategy, some 85% had reviewed their learningand development strategies, at some time, with 28 agencies having reviewedtheir strategy in the previous 12 months.

2.26 The ANAO considers that, for learning and development strategies andplans to be fully aligned with agencies’ corporate goals, they should addressboth their current and future skills requirements. Both the agencies surveyed,and those examined in detail, used various methods to determine futureworkforce skill requirements. The most common methods included training needanalyses, future scenario planning and consultation with senior executives. Themajority of agencies (70%) advised that, generally, high-priority skills, asidentified by a training needs analysis, are incorporated into their learning anddevelopment strategies and plans. However, some agencies advised the ANAOthat learning and development is intuitively based and, in some cases, reflectsmore short-term considerations rather than longer term strategic needs.

2.27 Better practice APS agencies, as part of their learning and developmentstrategies, should have developed output (activity) and outcome (effectiveness)performance indicators that inform and support the planning for, delivery andassessment of, learning and development. The ANAO found that the majorityof APS agencies (66%) have learning and development performance indicators,as part of their performance information frameworks. However, an analysis ofthe learning and development performance indicators of the five agenciesexamined during the audit identified that these are generally measures of activity,such as training days by APS level or organisational unit. Agencies’ performanceindicators do not generally include measures of learning and developmenteffectiveness. The ANAO recognises that the latter measures are difficult toarticulate clearly let alone quantify. Instead, qualitative assessments will also berequired.

2.28 The ANAO noted that the majority of APS agencies (65%) use their learningand development performance indicators to also inform future learning anddevelopment planning. However, 28% of agencies responded that performanceagainst the indicators is not reported. On the other hand, the ANAO also notedthat three of the agencies examined report progress against the indicators to

43


their agency executive through their human resource management committeesor similar bodies.

2.29 Overall, the ANAO considers that, although the majority of agencies haveachieved an alignment in their planning documents between their learning anddevelopment strategies and corporate goals, the impact of this alignment onbusiness outcomes was not as evident in practice. Without learning anddevelopment being firmly and clearly aligned with corporate goals, andsupported by meaningful performance indicators, agencies are unable to assessadequately the efficiency and effectiveness of their investment in learning anddevelopment and its contribution to organisational effectiveness.

Better Practice PrinciplesAlignment of learning and development strategies

and plans with corporate goals.So that learning and development addresses organisational capability andperformance improvement, the ANAO considers that better practice agency’slearning and development strategies and plans are:

• driven by business needs (as identified in corporate and other plans);

• informed by the outcomes of previous learning and developmentactivities and reviews;

• identifying and addressing current and future skill requirements; and

• supported by a performance information framework (with performanceindicators designed into learning and development strategies).

General performance information to support agencies in monitoring andreporting on their alignment with corporate goals could include the:

• extent to which learning and development specifically addresses businessneeds, is part of the agencies’ strategic planning and performancemanagement systems, and addresses current and future skills; and

• extent to which learning and development can be demonstrated tocontribute to closing the gap between current and future needs(supported by skills stocktake or other assessment).


3. Integration of Learning andDevelopment with other HumanResource Management Functions

This chapter discusses the extent to which agencies integrate their learning anddevelopment efforts with other human resource functions, such as performancemanagement, career and succession management etc., to ensure that staff needs arecomprehensively assessed and prioritised.

Background3.1 Historically, learning and development has functionally stood alone inorganisations. Most organisations had a separate ‘training branch’ that providedin-house training as an independent corporate function, with loose connectionsto the personnel function. However, recent strategic human resourcemanagement initiatives have aimed to draw together all human resourcefunctions through recognising interdependencies, dependencies, and the needfor common data sets, to ensure the effective achievement of organisationaloutcomes with minimal duplication of effort or waste of resources.

3.2 The integration of learning and development with the organisation’sapproaches to human resource management is examined through three aspects:

• the part played by learning and development strategies and plans in theoverall human resource management strategy;

• specific links between learning and development and other humanresource functions such as performance management, certifiedagreements, career and succession management, and workforce planning;and

• how learning and development strategies and plans identify staff withskill gaps and prioritise meeting their needs.

3.3 Each of these aspects is considered in turn, drawing on the analysis of theAPS-wide survey of learning and development and highlighting current betterpractice identified during the audit.

45

Integration of Learning and Development with other Human Resource Management Functions

Learning and development strategies and plans aspart of an overall human resource managementstrategy3.4 An integrated and coherent approach to human resource managementprovides agencies with greater assurance of achieving their outcomes efficientlyand effectively. The functions that traditionally make up human resourcemanagement (e.g.: learning and development, performance management,succession management, career planning, and workforce planning) all havecommon elements (e.g.: information requirements) that link them together. Anintegrated approach, which acknowledges and articulates these links, isnecessary for good management as well as reduced duplication, increased impactthrough alignment of objectives, structure and culture; and thereby ensuringgreater efficiency.

3.5 Agencies were asked to rate their learning and development strategies aspart of an integrated and coherent approach to human resource management.The majority of agencies (83%) either agreed or strongly agreed that theirorganisation’s learning and development strategies were part of an integratedand coherent approach to human resource management.14 Results for the66 agencies who responded are summarised in Table 1.

Table 1Agencies’ agreement with the statement: ‘Our organisation’s learningand development strategies are part of an integrated and coherentapproach to human resource management’

Response Percent of agencies Cumulative %

Strongly agree 35 35

Agree 48 83

Neither 12 95

Disagree 5 100

Source: ANAO survey

3.6 Analysis of agency responses to the supporting questions was undertakento identify support for, and correlation with, their self-assessment; as well asany better practices in achieving integration.

14 This self-assessment by respondents is in contrast with the findings of ANAO Report No.61 2001–02Managing People for Business Outcomes, where the integration of human resource managementwas found to be the weakest of nine people management ‘practice’ areas examined.


3.7 Two factors were examined in identifying the degree to which learningand development was part of agencies’ overall human resource managementstrategies. These are:

• learning and development specifically identified in the human resourcemanagement strategy as a key human resource function; and

• the contribution and importance of learning and development to theoverall human resource management strategy is sufficiently recognised.

Learning and development specifically identified as a keyhuman resource function

3.8 Most (80%) agencies indicated that learning and development was partof their overall human resource management strategy. The various ways in whichlearning and development was integrated with human resource managementincluded demonstrable links between learning and development planning andthe agency human resource management strategy supported by performanceindicators; evidence of executive endorsement of the learning and developmentstrategy; regular review of the learning and development strategy; broadapplication of this strategy; and aspects of funding.

Contribution and importance of learning and developmentsufficiently recognised

3.9 As noted earlier, the APS-wide survey revealed that the majority ofagencies (84%) had a human resource management strategic plan or equivalent.Of these agencies, 95% specifically included learning and development objectivesin their human resource management strategic plan (i.e. 80% of all agencies).Table 2 summarises the various ways in which agencies indicated that thecontribution and importance of learning and development is sufficientlyrecognised.

47


Table 2Agency indications that the contribution and importance of learning anddevelopment is sufficiently recognised

% of all

agencies

Agency executive or senior management participates in development of the 95

L&D strategy

L&D strategy endorsed by senior executive committee 90

Reviewed strategy in last 15 months 49

But: no review of strategy in 30 month period >20

Covers all agency staff 83

Funding from multiple sources:

line area budget 80

corporate budget 71

But: internal user-pays funding 14

Measurable, specific L&D performance indicators 66

But: no reporting against existing L&D performance indicators 28

Report L&D performance at organisation-wide level 57

Source: ANAO survey

3.10 Fieldwork in the five agencies largely supported the survey findings.Although, none had a specific organisation-wide learning and developmentcommittee, all of the agencies examined had a human resource managementcommittee or similar body. In addition, two of the agencies’ committees had notmet for some time or had recently been reformed following an extended periodof inactivity.

3.11 The Australian Greenhouse Office (AGO) has put in place an integratedlearning and development and human resource management strategy by usingthe Investors in People (IiP) Standard as a benchmark. A summary of theirachievements to date is presented as Case Study 3.


Case Study 3Australian Greenhouse Office—Integration of Learning and

Development and Human Resource ManagementThe Australian Greenhouse Office (AGO) is a small policy agency ofapproximately 170 people. The AGO considers it has a fully integrated humanresource management strategy that demonstrably contributes to tangibleimprovements in organisational performance. AGO has adopted the Investorsin People (IiP) Standard as a benchmark for its people management anddevelopment. The AGO achieved accreditation against the Standard inMarch 2001. The discipline of the annual IiP assessments ensures that thelearning and development needs of the staff are linked to the delivery of agencyobjectives. Executives are engaged and involved in all aspects of thedevelopment and review of the human resource management strategy andplanning processes. Corporate commitment to the benefits of a totallyintegrated human resource management approach is demonstrated in all levelsof corporate documentation. The primary goal of learning and development,as stated in the human resource management plan is:

Grow the intellectual capital of the AGO through recruiting, developing and retaininghighly skilled and committed people.

The primary goal of the learning and development strategy is:

To assist the AGO achieve its business through the continued development of its people.

The annual IiP assessment process ensures that measures are identified anddata are recorded and analysed to substantiate business performanceimprovement.

Learning and development strategies and plans arelinked to other human resource managementfunctions3.12 Learning and development is a core component of human resourcemanagement. The functions collectively known as human resource managementare related to each other through common data requirements (e.g. employeeentitlements and payroll), dependencies (e.g. performance management systemsidentify learning and development requirements, which are inputs to learningand development planning) or interdependencies (e.g. succession managementand performance assessment both identify high performers). The identificationof key commonalities and a broadly-based learning and development strategyensures minimal duplication of effort, increased efficiency, and greater alignmentof the human resource function in delivering corporate goals.

49


3.13 Agencies responding to the APS-wide survey indicated that somemechanisms were in place to deliver the benefits of linking learning anddevelopment with other human resource management functions. These includedthe existence of organisation–wide learning and development committees,development of learning and development plans, and other specific linksincluding links to the performance management scheme.

3.14 Table 3 summarises agency efforts in linking learning and developmentwith other human resource functions. Agencies were asked about links withspecific human resource functions. These included their current CertifiedAgreements, performance management systems, career or successionmanagement processes and workforce planning processes. Agencies were alsoinvited to identify ‘other’ human resource functions specifically linked tolearning and development, which four agencies did. ‘Other’ functions identifiedby agencies included higher duties, future skills and knowledge needs, and‘People Development Program’. In agencies where these functions were in placea significant proportion of them were overtly linked with learning anddevelopment, with especially strong links between certified agreements,performance management arrangements and learning and development.Although career and succession management systems or workforce planningarrangements were in the minority, when they did exist they also had stronglinks to learning and development.

Table 3Agency efforts to support linking learning and development with otherhuman resource functions

% of all

agencies

Certified agreement in place 97

L&D provisions specifically included in current Certified Agreement 94

But: L&D provisions in Certified Agreement only monitored through 7

staff survey, or not at all

Performance management system in place 99

Percentage of these agencies where the performance management system 98

is linked to learning and development

Career or succession management system in place 24

Percentage of these agencies where career or succession management 100

systems are linked to learning and development

Workforce planning processes in place 45

Percentage of these agencies where workforce planning linked to learning 73

and development

Source: ANAO survey


3.15 The Department of Education, Science and Training (formerly theDepartment of Eduction, Training and Youth Affairs) has developed a suite offrameworks that integrates the various aspects of human resource managementwith corporate business outcomes. Their learning and development frameworkis presented as Case Study 4.

Case Study 4Department of Education, Science and Training—learning and

development frameworkThe learning and development framework of the Department of Education,Science and Training has been specifically designed to ensure that learningand development is an integral part of the department’s business, and toprovide access and information for all staff on career development options.This is achieved through the detailed linking of learning and developmentfor individual performance with business outcomes, organisationalcapabilities and DEST values and behaviours. This is presented in Figure 3.

51


Figure 3DEST’s learning and development framework

Developing Organisational and Individual Capability

Learning & Development

for Individual

Performance in DEST

Future Learning and

Development Options

Career Development and Performance Agreements

Understand Anticipate Create Influence Implement

Planning for Effective Development for the Individual

Induction

Entry Level Training

o Graduate Administrative

Assistants

o Traineeships/Cadetships

� Yarrangi

� Core Skills Development

o Corporate Learning

Priorities

o Options for Individual

Development

� Mobility

� Coaching

� Mentoring

� Studies Assistance

� On the Job Learning

� Succession Planning

o Manager One Removed

o Mobility

o Higher Duties

� Extension Programmes

o Special Project Work

o Specialist Roles/Placements

o Cross Functional Activities

o Scholarships/Fellowships

� Leadership Development

o Career Development

Assessment Centres

o Senior Women In

Management

Values and Behaviours

SES Development

“B

us

ine

ss

in

flu

en

ce

s"

Collaborate

This Framework links learning and development to DEST’s business and

provides access and information on career development options

Co

rpo

rate

Pri

ori

ties

�

�

Work Level Standards

“Organisational Influences”

Bu

sin

ess P

lan

Source: Department of Education, Science and Training


Bridging the skills gap of staff3.16 Learning and development that is linked to other human resourcefunctions will help to ensure that those staff with skills gaps, not just the skillsgaps themselves, are identified. The needs of these staff to have their skillrequirements addressed and prioritised in line with corporate goals can then bemet. Identifying staff with skills gaps is dependent on the processes of:

• knowing the skills required for today and the future;

• evaluating the current stock of skills;

• assessing the gap;

• agreeing criteria for identifying staff requiring skills;

• establishing ‘greatest need’ for skills in line with corporate goals; and

• setting priorities and timing for acquiring skills.

3.17 As part of the APS-wide survey, agencies were asked about their currentskills audit activities; how they identified future skill requirements, through forexample, training needs analyses; staff needs and priorities; the roles andresponsibilities that managers have; and how they recorded staff skillingrequirements and participation in learning and development activities.

Skills audits

3.18 A skills audit is a process to systematically establish the current skill levelsof employees either individually and/or collectively, and by inference, futureskill requirements. It provides a comparative basis for scoping the gap betweencurrent skills held and the desired skills and level of capability.

3.19 The ANAO found that 34% of agencies have undertaken some skills auditactivity over the past three years. In addition, where skills audits have beenundertaken, these have usually covered less than 50% of the agencies’ workforce.In their responses to the APS-wide survey, 56% of agencies reported that skillsaudits were mostly used to support learning and development planning.However, 38% of agencies did not use the information gained from undertakingskills audits. Information gathered from skills audits is also used by agenciesfor workforce planning (32%) and recruitment (24%). This information issummarised in Table 4.

53


Table 4Agency use of skills audits

% of agencies

More than half the organisation covered in 2000–01 10

More than half the organisation covered in 1998–99, 1999–2000 14

Information from most recent skills audit used for learning and 56

development planning

But:

No skills audit undertaken in 2000–01 63

No skills audit undertaken in 1998–99, 1999–2000 52

No skills audit undertaken in 3 year period 48

Information from most recent skills audit NOT used 38

Source: ANAO survey

3.20 The ANAO identified that, of the five agencies examined, Health hadundertaken the identification of skills needs across their organisation throughtheir Performance Development Scheme (PDS). Staff members and team leadersassess skills needed and develop individual development plans in response toany resultant identified skills gaps. This process has been in place for all staffsince the introduction of the PDS in 1998.

Training needs analyses

3.21 Training needs analysis is a systematic process of identifying theknowledge and skilling needs of staff, which can be met by training. Trainingneeds analyses complement the information gained from skills audits.

3.22 Most agencies (70%) agreed or strongly agreed that high priority skills, asidentified by a training needs analysis, were incorporated in current learningand development plans and strategies. Survey results indicated that 55% ofagencies providing information on their training needs analysis activities hadcovered more than half of their organisation in 2000–01, with 46% covering morethan half of their organisation in the preceding two years. This information issummarised in Table 5. Further analysis indicated that 30% of all agencies hadundertaken training needs analysis activities in all three years. However, 18%of agencies had not undertaken any training needs analysis activities in theprevious three years.


Table 5Agency use of Training Needs Analyses

% of agencies

More than half the organisation covered in 2000–01 55

More than half the organisation covered in 1998–99, 1999–2000 46

Agree or strongly agree that high priority skills, as identified by training 70

needs analysis, are incorporated in current learning and development

plans and strategies

But:

No training needs analysis undertaken in 2000–01 19

No training needs analysis undertaken in 1998–99, 1999–2000 24

No training needs analysis undertaken in 3 year period 18

Disagree that high priority skills, as identified by training needs analysis, 12

incorporated in current L&D plans and strategies

Source: ANAO survey

3.23 Of the five agencies examined, DFAT had undertaken training needsanalyses of the entire organisation for all three years (1998–99, 1999–2000 and2000–01). DVA had undertaken a training needs analysis of the total organisationin 2000–01.

Identifying staff with skill gaps

3.24 The ANAO found that most of the agencies examined in detail hadstructured processes in place for identifying staff with skills gaps through theuse of training needs analyses and their performance management systems.Surveyed agencies used three main methods in identifying staff learning anddevelopment needs: consultations with managers or executives (88%), througha performance management system (75%), and through individual learning plans(67%).

3.25 Using various methods to identifying staff learning and developmentneeds gives a balanced picture of their total needs. However, this data needs tobe recorded and analysed to produce such a picture, and as a basis for monitoringprogress. While agencies’ performance management systems were linked to theidentification of their learning and development needs, the ability to access andinterpret learning and development needs data in a way that informs learningand development planning was limited.

3.26 Of the agencies examined in detail, four of the five had manualperformance management systems. Accordingly, learning and developmentneeds data had to be manually consolidated to identify the highest priority needs

55


to better inform learning and development planning. The one agency reviewedwith an automated on-line performance management and development systemhad a similar limitation due to restrictions on its ability to consolidate theavailable data.

3.27 The ANAO considers that improvements in agencies’ abilities to identifyand access data on current workforce skills and data on future organisationalskill requirements would greatly assist agencies in focusing learning anddevelopment activities on both current needs and future organisational capabilityrequirements.

Staff priorities

3.28 Priorities for staff to access learning and development opportunities shouldreflect organisational goals and priorities. For example, the ANAO found thethree main methods of identifying key staff to access learning and developmentwere corporate priorities (82%), recommendation of the manager (74%), andimportance of learning and development to the work area (73%). Other criteriaused included cost (33%), and criticality of staff to the work area (21%).

Role of managers

3.29 Managers have a range of responsibilities in relation to staff generally.However, managers are also responsible for ensuring that money is spent wiselyand contributes to organisational performance. This extends to their learningand development responsibilities. The formal roles and responsibilities ofmanagers in relation to learning and development covered: identifying needs(97%), ensuring staff availability for training (96%), and ensuring skills acquiredthrough learning and development are then used in the workplace (75%). TheANAO found evaluating the effectiveness of learning and development back inthe workplace to be less prevalent (54%).

Recording learning and development data

3.30 Human resource management information systems (HRMIS) provideagencies with the ability to record and use a comprehensive range of informationabout their workforce. Using HRMIS for recording staff learning anddevelopment participation and skilling requirements was not widespread, beingused in only 25% of agencies responding. More prevalent was the use of stand-alone databases (33% of agencies responding) and other local systems (37% ofagencies responding). Some agencies (9%) indicated the use of multiple recordingsystems, and 13% of agencies responding indicated that they did not recorddata on staff participation in learning and development or skill needs at all. All


five agencies examined in detail use their HRMIS or similar systems to recordstaff learning and development participation; however, only some record staffskilling requirements.

Conclusion3.31 Without the integration of related human resources functions, agenciesare at a risk of duplication of effort, funding programs with competing purposes,and a loss of synergy with adverse consequences for their efficiency andeffectiveness. An integrated and coherent approach to people management givesagencies a greater likelihood of achieving their outcomes efficiently andeffectively. The majority of agencies (83%) indicated that their organisation’slearning and development strategy was part of an integrated and coherentapproach to human resource management and 80% of all agencies had a humanresource management strategic plan that specifically included learning anddevelopment objectives. These findings were reflected in the five agenciesexamined in detail.

3.32 The ANAO found that there were particularly strong links betweenlearning and development and agencies’ certified agreements and performancemanagement arrangements. Although, career or succession management systemsor workforce planning arrangements were in the minority (24% of agencies),where they did exist, they had strong links to learning and development.

3.33 Learning and development that is linked to other human resourcefunctions helps to ensure that those staff with skills gaps, not just the skills gapsthemselves, are identified. Although the percentage of agencies that conductskills audits of staff is small (34%), 56% use the information for learning anddevelopment planning purposes. More commonly training needs analyses wasused as a basis for identifying high priority skills requirements in learning anddevelopment strategies and plans.

3.34 The ANAO found that the learning and development needs of staff wereusually identified through a combination of methods, including consultationwith managers and supervisors (88%), agencies’ performance managementsystem (75%) and individual learning plans (67%).

3.35 Access to learning and development was most commonly based oncorporate priorities (82% of agencies). The ANAO noted that managers playedan important role in identifying learning and development needs and ensuringthat staff were released for training, but were less involved in evaluating theresults of the training back in the workplace. All five of the agencies examinedin detail had a performance management system with links to learning anddevelopment. However, the ability to access and interpret learning and

57


development needs data in a way that informs learning and developmentplanning was limited.

3.36 The use of human resource management information systems to recordlearning and development data was not widespread (25% of agenciesresponding). More common was the use of stand-alone databases (33% ofagencies responding) for such purposes. However, all of the agencies examinedin detail had a HRMIS or similar system that was capable of capturing learningand development data, but the systems capability was not fully utilised by allfive of the agencies examined and the ability to extract consolidated managementinformation was limited by the way the systems are configured.

Better Practice PrinciplesIntegration of learning and development strategies and plans with theorganisation’s approach to human resource management.

So that priorities for cost-effective delivery are clearly articulated, agencylearning and development plans should be:

• clearly part of the overall human resource management strategy;

• overtly linked to other human resource functions (e.g.: performancemanagement, workforce planning, career and succession management);and

• specifically targeting staff with identified skills gaps.

General performance information to support agencies in monitoring andreporting on the integration of learning and development with other humanresource functions could include:

• the extent and nature of links between learning and development andkey human resource management functions (e.g.: performancemanagement system, career and succession management);

• the extent to which common/shared data is consistently collected andused (e.g.: performance management system, workforce planning andlearning and development);

• inclusion of, and regular reporting of, learning and development keyresult areas as part of a human resource management strategy; and

• identification of critical jobs and roles, articulation of appropriate learningand development and succession management strategies, and monitoringof achievements.


4. Delivering Learning andDevelopment

This chapter discusses the extent to which agencies ensure that learning and developmentis delivered in an efficient and cost-effective manner. It also addresses issues associatedwith Senior Executive Service (SES) learning and development, outsourcing and trainingfor contract management.

Background4.1 Cost-effective and efficient learning and development is contingent onidentifying the underlying purpose of the learning and development proposed,the target group, and the skills and knowledge to be conveyed. Delivery oflearning and development can take many forms: in-house classroom stylecourses, tertiary study, long-term development programs, and self-directedlearning programs etc. Ideally, the medium for delivery will match the needs ofthe target group and the nature of the skills and knowledge to be delivered.

4.2 The ANAO recognises the importance of informal learning anddevelopment activities such as on-the-job training, coaching, mentoring anddevelopment opportunities. However, such activities tend to be unstructuredand ad hoc and, as noted earlier, very little meaningful data is collected in theserespects by agencies. Agencies themselves are best placed to assess the benefitsof these activities from their normal performance management practices, suchas staff surveys and discussions between supervisors and staff.

4.3 The cost-effective and efficient delivery of learning and development hasbeen examined from three main aspects:

• cost-effective delivery methods that suit the nature of the learning anddevelopment to be delivered;

• regular monitoring of costs; and

• regular monitoring of staff uptake and achievement of targets.

4.4 Each of these aspects is considered in turn, drawing on the analysis of theAPS-wide survey of agencies and an examination of the policies and practicesin the five agencies reviewed. Current agency better practice is also highlighted.

4.5 Surveyed agencies were asked to provide learning and development datathat they collected and used as a management tool, or to indicate if the datawere gathered but not easily available. Agencies were not asked to specificallycollect data for the purpose of the survey that they did not already collect. Data

59

Delivering Learning and Development

requested included learning and development expenditure, administrativesupport costs, and training days. Generally agencies did not collect the datasought on a consistent basis, with more that half the agencies surveyed unableto provide the requested information. Due to the low availability of data, andthe different methods of collecting data between agencies, it was not possible tomake direct comparisons of learning and development expenditure acrossagencies. The ANAO has identified a minimum data set that would enable theconsistent collection of data across agencies for analysis by the Parliament, thePSMPC or other interested parties. This is included in the ‘better practice’summary at the end of this chapter.

Learning and development is delivered cost-effectively, using methods that suit the nature of thelearning and development to be delivered4.6 Agencies were asked to assess the extent to which their organisationensured that learning and development was delivered in an efficient andcost-effective manner. All 67 agencies surveyed responded, and the majority(80%) of agencies agreed or strongly agreed with this proposition, as depictedin Table 6.

Table 6Agencies’ agreement with the statement: ‘Our organisation ensures thatlearning and development is delivered in an efficient and cost-effectivemanner’Response Percent of agencies Cumulative %

Strongly agree 16 16

Agree 64 80

Neither 18 98

Disagree 2 100

Source: ANAO survey

4.7 Agency responses to the supporting questions were analysed to identifysupport for, and correlation with, their self-assessment; as well as good practicesin delivery of learning and development.

Learning and development program development

4.8 Approximately half (49%) of the agencies responding to the APS-widesurvey indicated that they had an organisation-wide learning and developmentcalendar to inform staff of the learning and development opportunities available


to them. Agencies were asked to identify the top three factors that influencedthe development of their organisation-wide calendars of learning anddevelopment activities. The ANAO found consideration of agencies’ learningand development strategies, training needs analyses, skills audits or equivalentand workload priorities were less influential than executive priorities, budget,and demands from staff. This finding raises the issue of the significance oflearning and development strategies in driving the delivery of learning anddevelopment programs. The efficiency and effectiveness of learning anddevelopment would be improved by giving greater weight, in the developmentof learning and development programs, to the learning and developmentstrategies endorsed by the organisation and the skills gaps identified throughneeds analysis and the like. A more methodological or planned consideration ofthese factors could contribute to greater efficiency and cost-effectiveness indelivery through ensuring that learning and development strategies asdeveloped and endorsed are actually implemented.

Delivery methods

4.9 Agencies were asked how they delivered learning and development totheir staff and how they determined the most appropriate delivery methods.Most agencies (70%) indicated that they matched delivery methods to the natureof the content being delivered. However, for most agencies, delivery was limitedto in-house training programs (79%) or externally provided training programs(75%) or conferences and seminars (58%). Few agencies used e-learningtechniques, such as computer-based training packages or courses (2%).Determinants of delivery methods, as identified by agencies, are presented inTable 7.

61


Table 7Agency determinants of delivery methods

% of agencies

Three main delivery methods

- inhouse training programs 79

- external training program 75

- external conference and seminars 58

But also other methods:

- job rotations 3

- HDA for development 2

- special projects 2

- mentoring 2

- computer based training 2

Main factor in choosing delivery method

- nature of content 70

But also effected by:

- cost or efficiency considerations 8

Greater than 50% external sourcing of learning and development

- delivery 73

- content development 55

Main reason for external sourcing of learning and development

- inadequate internal resources 31

But:

- cost savings/efficiency considerations 14

- outsourcing/ no internal function 8

Source: ANAO survey

4.10 All five agencies examined in detail indicated that the nature of the coursecontent was the major determinant of the delivery method. The agenciesexamined delivered learning and development in-house and using externalproviders or external courses. Health and DFAT used agency supported formalstudy as a major delivery method. Only DFAT placed importance on job rotations,and ABS indicated that on-the-job training is a significant delivery option. Allfive agencies used external resources, albeit for different reasons. FaCS usesexternal resources for cost savings. DVA uses external providers where there isa lack of internal expertise. ABS uses external resources to achieve the best matchbetween their requirements and the expertise available.


Learning and development costs are regularlymonitored4.11 The APS-wide survey sought information from agencies on the total amountspent on learning and development in 2000–01, and the composition of thisamount. Although 80% of agencies claimed to deliver learning and developmentin a cost-efficient way, the information provided by agencies on expenditure onlearning and development for their organisations indicated a paucity of data. Onlylimited data was available on the total cost of learning and development, thevarious activities on which it is spent, and the staff groups on whom it is spent. Ofthe 67 agencies surveyed, only 63% were able to provide data on organisation-wide expenditure on learning and development. On this basis, it was not possibleto accurately calculate a Service-wide total expenditure figure or make meaningfulcomparisons across agencies. However, as discussed in Chapter 1, based on theavailable data, the ANAO estimated the cost of formal learning and developmentin the APS in 2000–01 to be approximately $160 million.

4.12 On the basis of the expenditure and staffing information provided by theagencies that did collect expenditure data, agency expenditure per full-timeongoing staff member ranged from $245 to $3563, with an average of $1616. Theinclusion of part-time ongoing staff, or non-ongoing staff would reduce thisfigure. However, not all agencies provided data on part-time or non-ongoingstaff. Some agencies indicated that, where learning and developmentresponsibilities had been devolved to divisions or managers, so had datacollection responsibilities. Other agencies indicated that reporting systems werecurrently being developed or refined that would be able to record and reportthe data sought in the future. This information is summarised in Table 8.

Table 8Agency expenditure on learning and development, 2000–01

% of agencies

Calculated and provided overall figure for organisational expenditure on 63

learning and development in 2000–01

Average expenditure for full-time ongoing staff = $1616

Range from $245 to $3563

But:

Not able to calculate/provide total organisational expenditure on learning 37

and development in 2000–01 (or did not respond to question)

Able to specifically interrogate finance system to produce an estimate

of total expenditure (with varying degrees of accuracy) 4

Source: ANAO survey

63


4.13 Agencies were also asked to provide information on the components oflearning and development expenditure. All agencies were able to provideinformation on the use of external providers, as this information is required forannual reporting purposes. However, few agencies were able to provideinformation on the cost of travel or attendees’ salaries associated with learningand development. Further findings are provided in Table 9.

Table 9Agency expenditure by component cost

% of agencies

Able to provide expenditure on:

- external consultants or providers 100

- course content material 95

- venue hire 71

- travel costs 56

- cost of course attendee salaries 20

Source: ANAO survey

4.14 Few agencies were able to provide information on expenditure on differentdelivery methods. Agencies were more likely to be able to provide informationon expenditure on formal study (34%) than any other delivery method. Greateraccess to such data would allow agencies to compare relative efficiencies andcost-effectiveness of alternate delivery methods. Details of data availability forselected delivery methods is presented in Table 10.

Table 10Agency expenditure on selected delivery methods

% of agencies

Information provided on expenditure on:

- formal study supported by organisation 34

- in house programs 28

- external training programs 28

- external development programs 28

- use of learning centre 27

But:

Do not collect information about expenditure by specific delivery 20*

methods

Source: ANAO survey

*Significant numbers of agencies did not respond to these questions—actual percentagesmay be higher than indicated.


Agency administrative costs

4.15 Most agencies indicated that they were able to provide information onfull-time (76%) and corporate staff (77%) devoted to learning and development.However, fewer agencies were able to identify contractors (21%) or divisionalstaff (43%) devoted to learning and development. At least 6% of agenciesindicated that they did not collect any information on staff devoted to learningand development. Without information on the numbers and location of staffdevoted to learning and development, agencies are unable to quantify theadministrative costs of learning and development and so accurately cost theirtotal investment in learning and development, or identify if their learning anddevelopment is being delivered efficiently and cost-effectively. On average, somesix full time staff are devoted to learning and development in each agency, mainlyin the corporate area. Summary information on staffing is presented in Table 11.

Table 11Staff devoted to learning and development

% of agencies

Provided information on number of staff devoted to learning and

development delivery as at 30 June 2001:

- full-time ongoing 76

- contractors 21

- corporate staff 78

- divisional staff 43

Number of staff per agency devoted to learning and development

delivery as at 30 June 2001:

- average = 6.0 full-time ongoing staff

- average = 0.7 contractors

Percentage of staff devoted to learning and development located in

either corporate or divisions:

- corporate staff 77

- divisional staff 23

Percentage of agencies where all staff involved in learning and 40

development are located in Corporate:

But:

Information not collected on full-time ongoing staff, contractors, 6*

corporate or divisional staff devoted to learning and development

delivery and management

Source: ANAO survey

*Significant numbers of agencies did not respond to these questions—actual percentagesmay be higher than indicated

65


4.16 Two of the five agencies audited had calculated an overall figure foradministrative expenditure on learning and development in 2000–01. DFATregularly reported planned and actual learning and development expenditureas part of their departmental-wide performance reporting process covering allposts, divisions and offices.

Staff uptake is regularly monitored4.17 Agencies were asked to provide details of staff training days undertakenand the reporting on learning and development routinely undertaken.

Staff training days

4.18 Agencies participating in the APS-wide survey were asked to provideinformation on total training days, training days provided for staff by classificationand by employment category, and for delivery methods used. Most agencies donot collect data on staff days devoted to learning and development and thereforecould not provide the data. Agencies reported that devolution of the learning anddevelopment function to regional offices or other organisational levels has led toinconsistent data gathering and the inability to aggregate this at an organisation-wide level. Due to the low coverage of this data, further analysis has not beenundertaken. The data available are summarised in Table 12.


Table 12Staff training days

% of agencies

Data available on number of staff training days undertaken organisation- 34

wide in 2000–01.

Full-time ongoing staff:

- APS-wide average = 5.5 days p.a.

- agency averages range: 1.4 days to 16.7 days

All staff:

- APS-wide average = 4.2 days p.a.

- agency averages range: 0.8 days to 15.2 days

Data available on staff undertaking no training during 2000–01 22

- agencies where some staff did not undertake any training 7

But:

Data not gathered on number of staff learning and development days 19*

undertaken organisation-wide in 2000–01

Average more than 10 staff days learning and development in 2000–01 1.5*

Data not gathered on staff undertaking no learning and development 30*

during 2000–01

Source: ANAO survey

*Many agencies did not answer these questions—actual percentages may be higher.

4.19 Agencies were asked about the number of days staff of each employmentcategory—ongoing versus non-ongoing, full-time versus part-time—devotedto learning and development activities in 2000–01. Most agencies (between 64%and 69%) did not answer these questions on the survey. Of those agenciesresponding, a large proportion indicated that they either did not gather thisinformation or could not provide it within the timeframe of the survey. Withoutthis information, agencies are unable to confirm that staff with prioritydevelopment needs are receiving training, nor are agencies able to establish theefficiency or cost-effectiveness of the delivery of learning and development tostaff. The responses received are summarised in Table 13.

67


Table 13Staff training days by employment category

% of agencies

Data available on number of staff training days undertaken by each

employment category

- full time, ongoing staff 5

- full-time, non-ongoing 3

- part-time, ongoing 3

- part-time, non-ongoing 2

- contractors 2

But:

Data not gathered on staff undertaking L&D by employment category

- full-time, ongoing 28*

- full-time, non-ongoing 27*

- part-time, ongoing 27*

- part-time, non-ongoing 27*

- contractors 27*

Source: ANAO survey

*Many agencies did not answer these questions—actual percentages may be much higher.

4.20 Few agencies were able to provide information on the number of days oflearning and development that staff of different classifications received during2000–01. Most information (18% of agencies) was available for SES staff.However, over 50% of agencies did not answer these questions. Some agencies(25%) indicated that alternate data was available, such as total days by staffname, in broader groupings due to broad banding, by region, or only by manualextraction.

4.21 In the APS-wide survey, agencies were also asked about the number ofstaff days devoted to different delivery methods during 2000–01. Of thoseagencies that did respond (less than 50%), 15% provided data on staffparticipation in formal training programs and 9% provided data on participationin external conferences and seminars and/or development programs. However,22% of agencies advised that they do not collect this data. Without data on theutilisation of different delivery methods, agencies are unable to make informedassessments of the efficiency and cost-effectiveness of the various deliverymethods used by them.

4.22 Very few agencies (less than 5%) kept data on places available and placestaken up for different delivery methods. Most agencies (in excess of 70%) didnot respond to these questions. Without data on places available and places


taken up agencies are unable to identify which delivery methods are moreefficient and cost-effective.

Reporting on learning and development

4.23 Without some regular reporting against budget and achievement of goals,agencies are unable to monitor the efficiency and cost-effectiveness of theirlearning and development. Agencies were asked about the standard reportsgenerated from their records of staff participation and skilling requirements.The majority of agencies (72%) advised that they produce reports on courseattendances, with a slightly smaller percentage (67%) advising they are able toproduce cost and budget reports, but only a small minority (16%) can producereports which compare staff participation and skilling needs against corporategoals. Small agencies tended to produce a wider range of reports than largeagencies. The data available is summarised in Table 14.

Table 14Reports produced by agencies on staff participation and skilling needs

% of agencies

Reports generated on staff participation and skilling needs:

- course attendance 72

- cost and budget reports 67

- activity reports for L&D management 45

- against corporate goals 16

But:

Reports on Certified Agreement provisions 5

Only reporting ad hoc 3

No reports generated on staff participation and skilling needs 14

Source: ANAO survey

4.24 The ANAO found various methods of monitoring and recording staffuptake of learning and development in the five agencies examined. DFAT havedeveloped a comprehensive database covering all staff in all locations, whichprovides the ability to monitor staff completion of training and their evaluationof training. Details of the DFAT database are presented in Case Study 5.

69


Case Study 5Department of Foreign Affairs and Trade’s On-Line Training

and Development DatabaseDFAT has developed and implemented an integrated HRMIS, which includesa training and development module. The module incorporates all stages oflearning and development, from course nomination to evaluation. DFAT hasa strong senior management commitment to learning and development, whichis reflected in its approach.

As part of the posting of staff overseas, the Department requires its staff tohave successfully completed a series of mandatory training programs. Staffare not able to take up their positions if these requirements have not beenfulfilled.

The ANAO observed that for a course to be recognised as completed, staffare required to complete an on-line course evaluation form. This ensures thatstaff participation in training and development is recorded and that staff arecontinuously providing feedback to the Department on its training programs.Accordingly, this feedback informs future learning and development planningand assists the Department in the continuous improvement of its trainingand development programs.

SES learning and development4.25 The PSMPC has a significant role in SES learning and development underthe authority of the Public Service Act (1999). The APS-wide survey sought variousinformation from agencies in relation to their management of SES learning anddevelopment needs. In the main, agency responses indicated little data collectionon SES learning and development, with only 12% of agencies surveyed beingable to provide data on expenditure on SES learning and development and 18%providing data on number of days devoted to learning and development bySES staff. On the other hand, the PSMPC collects extensive data on participationin courses designed for the SES.

4.26 Limited analysis of the data provided by agencies has been undertaken,due to the low response rate to these questions. The ANAO considers thatagencies are unable to make informed assessments of the efficiency and cost-effectiveness of SES learning and development without data on expenditureand uptake. Table 15 summarised the data available from agencies on themanagement of SES learning and development.


Table 15Management of SES L&D by agencies

% of agencies

Total expenditure* on learning and development for SES:

• data provided 12

• data not gathered 27

Average expenditure on learning and development for SES:

• $1593/SES staff member

Days* devoted to learning and development by SES staff:

• data provided 18

• data not gathered 24

Average number of days devoted to learning and development by

SES staff:

• 1.2 days/SES staff member

Senior Executive Leadership Capability Framework (SELCF) used:

• specifically for SES selection and development 79

• as an evaluation framework 23

But:

Agency indicated SELCF not used 21

SES have a separate L&D strategy 3

Data on number of SES staff employed at 30 June 2001

• supplied 95

• not supplied 5

Source: ANAO survey

* Not all agencies answered these questions—actual figures may be higher.

4.27 As noted earlier, not all agencies gather data on learning and developmentexpenditure, so it is not a surprise that very few agencies were able to providedata on expenditure on SES staff (12%) versus non-SES staff (10%). A significantnumber of agencies (at least 27%) indicated that they did not collect expendituredata on a classification basis. Without a broad disaggregation of expenditureinto SES/non-SES it is not possible, in this report, to compare or contrastexpenditure trends and patterns for differing staff groups.

Outsourcing of human resource management4.28 Agencies were not specifically asked about decisions to ‘outsource’learning and development in the APS-wide survey. However, data wereincidentally collected on the impact of ‘outsourcing’ on the delivery of learning

71


and development, and a preliminary indication is provided. Where agenciesused external resources in the provision of learning and development, they wereasked to identify the main reasons. Most frequently cited reasons included costssavings, ease of administration, inadequate internal resources, lack of internalexpertise or ‘government requirements’.

4.29 The general lack of learning and development data collected by agenciesalso raises issues for the decision to outsource the learning and developmentfunction. Without a basis for assessing efficient and effective delivery of learningand development, the ANAO considers that agencies are not well placed tomake informed decisions on the outsourcing of this function.

Contract management4.30 Given that contract management is one of the major challenges facing theAPS, agencies were asked to provide information on their management of learningand development for contract management. Context to their responses wasprovided by the value of contracts managed and corporate learning anddevelopment priorities. While contract management was one of the top twopriorities for only 27% of agencies, most (97%) agencies did provide contractmanagement learning and development. The type of learning and developmentprovided and the area of the organisation with responsibility for delivery varied.Most frequently, human resource management (58% of agencies) had responsibilityfor delivering contract management learning and development. For the most part,this took the form of courses developed by government for government (65%).Information provided by agencies is summarised in Table 16.


Table 16Agency management of Contract Management learning anddevelopment

% of agencies

Agencies managing contracts in excess of $10 million in 2000–01 33

But:

Agencies not able to provide total contract management figure 13

Top two L&D priorities

- leadership 59

- project management 33

But:

- contract management 27

Agencies providing contract management L&D 97

Responsibility for delivering contract management L&D

- Human resource management 58

- Other corporate 31

- Business areas 14

- Shared between 2 or more areas 16

Delivery of contract management L&D

- courses provided by government for government 65

- other external courses 60

- in-house course developed externally 52

- in house 39

But:

- developed internally delivered by external 15

Staff attending contract management L&D activities during 2000–01

- more than 10% 19

But:

- No staff attending contract management L&D 9*

Average number of days training on contract management

- 2 days 34

- more than 2 days 10

But:

- Less than 1 day training on contract management 5**

Source: ANAO survey

*21% of agencies did not answer this question—the actual number may be higher.

**28% of agencies did not answer this question—the actual number may be higher.

73


4.31 The ANAO found that most agencies provide learning and developmenton contract management that is specifically developed and of significant duration(44% offering two or more days). However, most agencies indicated thatleadership development (59%) and project management (33%) were of higherpriority than contract management (27%), with few agencies (19%) providingcontract management learning and development activities for more than 10%of their staff. However, one agency, DEWR (formerly DEWRSB), has developedan accredited program in response to their contract management learning anddevelopment needs. Details of this program are outlined in Case Study 6.

Case Study 6Department of Employment and Workplace Relations: E-learning

Contract ManagementContract management is a significant capability requirement for DEWR. Anon-line DEWR Certificate in Contract Management has been developed tobuild capability by providing students with:

- on-demand access to training;

- on the job study time;

- structured development in all stages of contract management; and

- nationally recognised qualifications.

The contract management program is based on competencies in the PSETA(Public Sector Education and Training Australia, Inc) package to CertificateIV in Government accreditation standards, and comprises 11 modules.

The course is conducted in a virtual classroom using WebCT software, housedexternally. A range of online services are available in the web classroom toassist students e.g.; online tutorials, discussion forums and online assessments.The course also takes advantage of the latest in-house technology, videostreaming, to allow individuals to watch videos at their desktops. Videos areavailable to participants on demand, and are interspersed through the programmodules.

The Certificate in Contract Management represents a new direction in deliveryof public sector learning and development, providing a durable and livingresource for developing staff capability. The web classroom has already beenexpanded to include financial management modules, which will extend staffexposure to e-learning. DEWR anticipates that over 300 staff will undertakethe Certificate initially.

Continued next page


The on-line Certificate allows DEWR staff Australia-wide to all access highquality leaning materials, particularly in areas where staff may not normallyhave access to training. The flexible delivery of the Certificate will allow staffto balance their work and personal commitments with essential study. Theskills developed by undertaking the Certificate will ensure that DEWR staffare able to deliver high performing contract management services to itsoutsourced providers.

4.32 All five agencies examined in detail provided contract managementlearning and development to their staff, mostly under the responsibility of humanresource management. One agency, DVA, have a Contract Advisory Unit thathas developed and provided contract management training in addition to theiradvisory role. Details of DVA’s approach are included below.

Case Study 7Department of Veterans’ Affairs—Contract Management

DVA manages contracts to the value of approximately $3 billion. To assistwith this the Department has established a Contract Advisory Unit. TheContract Advisory Unit provides several functions including a nationalcontract management training program, legal advice on contract managementmatters and internal contract management consultancy services.

The provision of contract management training and the ongoing support forparticipants is an important part of the Contract Advisory Unit’s role. It hasdeveloped a comprehensive training program that covers all relevant aspectsof contract management and procurement. To further support DVA staff withcontract management responsibilities, the Contract Advisory Unit has alsodeveloped a series of DVA-specific better practice guides on procurementand contract management. Both the better practice guides and the presentationmaterials used during the contract management training are available to staffon the DVA Intranet.

Conclusion4.33 Without a clear and consistent basis for accurately identifying theexpenditure on learning and development, the extent to which groups of staffavailed themselves of opportunities, or the cost-effectiveness of the variousdelivery methods used, agencies are unable to assess the extent to which theirlearning and development is delivered efficiently and is achieving the resultsrequired.

4.34 The majority of agencies (70%) claimed that they had matched learningand development delivery methods to the nature of the content to be delivered.

75


Most commonly, agencies delivered learning and development via in-housetraining programs (79%), external training programs (75%), and externalconferences and seminars (58%). Common reasons for using external resourceswere inadequate internal resources (31%) and lack of internal expertise.

4.35 Although the majority of agencies (80%) held the view that their learningand development was delivered in an efficient and cost-effective manner, only63% of agencies were able to provide even basic data to the ANAO on totalexpenditure on learning and development. Expenditure on some components oflearning and development, especially the cost of external providers and coursecontent material, was well known. However, few agencies were able to providedetails of the costs of different training methods. Data on staff training days wassimilarly patchy with only 34% of agencies able to provide the necessaryinformation. Those agencies that could provide details of staff numbers employedin learning and development (76%) had an average of six full time staff.

4.36 On the other hand, while the majority of agencies (72%) advised that theyproduce reports on course attendances with a slightly smaller percentage (67%)advising they are able to produce cost and budget reports, only a small minority(16%) could produce reports which compare staff participation and skilling needsagainst corporate goals.

4.37 Agency responses indicated that little data is collected by them on SESlearning and development. However, the PSMPC collects extensive data onparticipation in courses designed for the SES. Nevertheless, the level of dataheld by agencies makes it difficult for them to assess the efficiency and cost-effectiveness of SES learning and development.

4.38 Without a solid information base for assessing efficient and cost-effectivedelivery of learning and development, agencies are also not well placed to makeinformed decisions on the outsourcing of the learning and development function.

4.39 Most (97%) agencies provide training in contract management even thoughcontract management was one of the top two priorities for only 27% of agencies.


Better Practice PrinciplesDelivery of learning and development is efficient and cost-effective

So that agencies are assured that their delivery is efficient and effective,learning and development delivery is underpinned by:

• cost-effective delivery methods that suit the nature of the learning anddevelopment to be delivered;

• regular monitoring of costs; and

• regular monitoring of staff uptake and the achievement of targets.

General performance information to support agencies in monitoring andreporting on their efficient and cost-effective delivery should include as aminimum:

• total number of staff days allocated to learning and development, andthe number of days per staff member (and by classification, division,full-time versus part-time etc.) ;

• total amount spent on learning and development, amount spent peremployee and percent of salaries and wages costs spent on learning anddevelopment (and by classification, division, full-time versus part-timeetc.);

• uptake of learning and development places available;

• extent to which specific targets are met (e.g. professional requirements);and

• ratio of human resource management staff engaged in learning anddevelopment to total employees.

77

5. Evaluation of Learning andDevelopment

This chapter discusses how agencies evaluate their learning and development strategiesand activities to determine their effectiveness, including the business impact.

Background5.1 Agencies are aiming to achieve performance improvement at theindividual and/or team level and ultimately, improved overall organisationalperformance through their financial commitment to learning and development.A major goal of learning and development evaluation is to improve currentlearning and development processes in order to achieve maximum businessimpact. Accordingly, evaluation is an important part of learning anddevelopment. Through evaluation agencies can assess the efficiency and cost-effectiveness of their learning and development strategies and activities. Theseresults should then inform management decision-making and future learningand development planning and delivery.

5.2 The ANAO examined how agencies evaluate their learning anddevelopment strategies and activities, and how this information informs futurelearning and development planning and delivery. This focused on severalthemes, each of which is discussed in this chapter. These include:

• evaluating learning and development for business impact asforeshadowed in agencies’ learning and development strategies;

• regular and systematic evaluation of agencies’ learning and developmentstrategies and activities;

• evaluating agencies’ learning and development strategies and activitiesusing reliable data;

• assessing the value for money of, and the return on investment from,agencies’ investments in learning and development; and

• using the results of agencies’ learning and development evaluations inmanagement decision making and in informing future planning anddelivery.


Aspects of learning and development evaluated5.3 Agencies were asked the extent to which they agreed with the statement:‘Our organisation evaluates its learning and development strategies to determineeffect, including business impact’. Of the 67 agencies surveyed, 27 (40%) eitheragreed or strongly agreed with the statement, 30 (45%) neither agreed nordisagreed and 10 (15%) disagreed with this statement.

5.4 The ANAO found that, of those 34 agencies which have evaluated aspectsof their learning and development activities, most focus on reviewing individuallearning and development activities, such as training courses. Of these agencies,94% have evaluated participant satisfaction and 80% evaluated both their majorlearning and development activities and the effectiveness of individual activitiesin meeting their workplace skills needs. Figure 4 presents the results of the APS-wide survey in relation to those aspects of learning and development whichagencies evaluate.

Figure 4Aspects of learning and development evaluated by agencies

Source: ANAO survey

Note: Course effectiveness pertains to effectiveness in meeting workplace skills needs.

5.5 The ANAO found that these results are broadly consistent with thefieldwork undertaken in the five agencies. Two of the five agencies examinedhave evaluated and/or are evaluating the impact of their learning anddevelopment strategies. The ANAO observed that Health has recently completedan internal review of its investment in learning and development. The Reviewof Investment in Learning and Development was initiated to assess whether

0% 20% 40% 60% 80% 100%

Major activities

Delivery options

* Course effectiveness

Participant satisfaction

Value for money

Fitness for purpose

Return on investment

Other

Asp

ects

of l

earn

ing

and

deve

lopm

ent

eval

uate

d

Percentage of respondents

79

Evaluation of Learning and Development

Health’s investment in learning and development is appropriate in meeting itsbusiness needs. In particular, the review looked at:

• funding and the total cost of formal learning and development;

• alignment between learning and development and business needs;

• prioritisation of learning and development activities in the context ofcapability building;

• learning and development delivery mechanisms; and

• evaluation of learning and development in the department.

5.6 Health advised that data from this review will be used as a baseline forfuture evaluation activity. The findings of the Review provided substantialinformation to inform future learning and development planning in theDepartment, better targeting of priorities, and a better return on investment.

5.7 The ANAO considers that the review demonstrates the Department’scommitment to learning and development. The findings of the review shouldprovide important benefits for the Department.

5.8 Of the five agencies examined during the audit, four had undertakencomprehensive reviews of selected learning and development programs. Theseevaluations have involved the completion of surveys, focus group discussionsand interviews with agency management. These evaluations have also focusedon the various stages of learning and development, including pre-delivery,delivery and post-delivery.

5.9 Large policy departments are more likely to evaluate their learning anddevelopment strategies. In response to the APS-wide survey, 50% of largeagencies advised that they have evaluated their learning and developmentstrategies, while only 37% of small agencies have evaluated their learning anddevelopment strategies. Policy departments in general, also place a greateremphasis on learning and development evaluation. Less than half of the agenciessurveyed advised that they evaluate their learning and development strategiesto determine business impact.

5.10 The ANAO agreed with an observation by Health that evaluation oforganisational impact should provide for different perspectives to beincorporated—that of the participant (personal value), the participant’s managers(training to develop competent, capable and motivated employees) and theexecutive (for strong strategic linkages and a clear business return on the learningand development investment. Health also noted the importance of the evaluationmethodology taking into account that many factors apart from learning anddevelopment contribute to individual job competency (e.g.: goal clarity,


organisational systems, processes and procedures), and that training is thereforenot the only (or necessarily the most appropriate) mechanism to employ.

5.11 The ANAO acknowledges a difficulty for agencies in evaluating theeffectiveness of learning and development activities is in assessing their longer-term impact. To compensate, agencies have focused on evaluating what istangible and measurable. The ANAO observed that agencies are striving toachieve improved organisational performance from their investment in learningand development, but are generally not actively assessing this through theircurrent evaluation methodologies.

5.12 The ANAO considers that agencies can strengthen their evaluation oflearning and development through stronger links to their performancemanagement systems. The routine assessment of the impact of learning anddevelopment as part of agencies’ performance management systems couldprovide valuable information for future learning and development planning.

Evaluation planning and activity5.13 The ANAO considers that agencies should evaluate their learningstrategies and activities on a regular basis in a planned and structured manner.This assists agencies in assessing the impact and value of their investment inlearning and development. An evaluation strategy that outlines an evaluationmethodology and timing of evaluations should support this.

5.14 Of the 34 agencies that evaluate their learning and development activities,25 (74%) had an overall learning and development evaluation plan. Of these 25agencies, 14 advised that they had also evaluated their learning and developmentstrategies.

5.15 The ANAO found that the timing of evaluation activity also variedbetween agencies. As noted earlier, agencies most frequently evaluate participantsatisfaction. This is undertaken on a routine basis. However, the review ofagencies’ learning and development strategies is less frequent.

5.16 The ANAO also found that there is considerable variation in agencies’evaluation plans and strategies. However, where agencies evaluate the outcomesof learning and development activities, they most frequently:

• evaluate participant satisfaction;

• assess the achievement of the learning outcomes; and

• measure participant self assessment of changes in the workplace.

81


5.17 In some agencies focus group research and post-delivery follow upassessments have further supported this level of evaluation. ABS in particularhas reviewed several of their learning and development programs.Case Study 8 outlines ABS’s review of two of its leadership and managementdevelopment programs.

Case Study 8Evaluation of the Australian Bureau of Statistic’s Leadership and

Management Development ProgramsABS has undertaken the comprehensive evaluation of two of its leadershipand management development programs. ABS offers to staff a range of in-house leadership and management programs. In 1999–2000, 145 staffparticipated in the two programs reviewed. Participation in these programsaccounted for 780 training days in 1999–2000.

The evaluations focussed on how well the programs met the organisation’sand individuals’ development needs. Participants were selected at randomfor participation in either focus groups discussions or one-to-one interviews.The senior management sponsors for each of the programs were alsointerviewed.

The evaluation report made several recommendations, which have resultedin the further development of the programs. The programs are delivered in-house, but are facilitated by external providers. As part of the evaluationprocess, consultation was undertaken with the providers and their commentsincluded in the final evaluation report.

5.18 To streamline the evaluation of learning and development activities, twoof the agencies examined during the audit have developed automated on-lineevaluation tools. These comprehensive evaluation tools provide the departmentswith useful information for future learning and development planning. CaseStudy 9 outlines FaCS’s approach.


Case Study 9Department of Family and Community Service: On-line Learning and

Development Evaluation ModuleFaCS has implemented a customised learning and development database(TAMS), which supports the ongoing administration of learning anddevelopment in the Department. The database covers a range of processesfrom participant nomination through to on-line course evaluation.

Where learning and development participation has been booked and approvedin the database, the participant is prompted by e-mail after the activitycompletion date, to advise of their attendance. If the person has participatedin a learning and development activity they are then forwarded an electronicevaluation form. If a staff member fails to complete the evaluation form, thesystem forwards, further e-mail notifications.

Following completion of the form by the participant, the completed form issubmitted electronically to the participant’s supervisor. The supervisor canthen add additional comments as necessary and mark the evaluation ascompleted.

This automated process streamlines the evaluation of learning anddevelopment activities and provides evaluation data in an electronic form,which can then be further interrogated through the database and used toinform future learning and development planning.

Reliable evaluation data5.19 The ANAO considers that agencies’ learning and development evaluationsshould be supported by access to complete and consistent learning anddevelopment data. An important aspect of learning and development datacollection is that agencies establish a minimum data set and consistently collectthis data. Accordingly, this data will enable agencies to evaluate all aspects oftheir learning and development.

5.20 As noted earlier, the ANAO found the scope of learning and developmentand human resource management data available from agencies to be limited. Inresponse to the APS-wide survey, 63% of agencies provided data on the aggregatecost of formal learning and development for 2000–01. However, only 34% ofagencies supplied data on the number of staff training days for their organisationin 2000–01. Further, only 22% of agencies were able to provide data on the numberof staff not participating in learning and development in 2000–01. Requests forlearning and development data at a more detailed level resulted in lowerresponse rates.

83


5.21 The ANAO reviewed how agencies collect learning and development data,and supporting human resource management data. Agencies have adoptedvarious methods of recording staff participation in learning and developmentand staff skilling requirements, these findings are presented in Table 17.

Table 17How agencies record staff participation in learning and developmentand staff skilling requirements*

% of respondents

Human resource management information system 25

Stand-alone database 33

Proprietary software 8

Other 37

Not recorded 13

Source: ANAO survey

* Multiple response question on APS-wide survey

5.22 The ANAO observed that several factors affect agencies’ abilities to accesslearning and development data, with two of the most significant factors beingthe inability of agencies to interrogate their HRMIS to access meaningful dataand agencies policies’ regarding the provision of data to populate the database.

5.23 The ANAO found that all five of the agencies examined have an HRMISor learning and development management information systems in place.However, several of the agencies are not able to extract meaningful learningand development data to inform their learning and development planning,because of limitations in the way the systems are configured.

5.24 To develop an understanding of the extent of the financial commitmentmade to learning and development and to inform future learning anddevelopment planning, some agencies require staff to book participation inlearning and development activities through their HRMIS or learning anddevelopment system. The ANAO considers this is a good initiative in facilitatingthe capture of learning and development data. However, depending upon systemconfiguration and agencies’ policies regarding use of their learning anddevelopment management information systems, data on participation in alllearning and development activities may not be captured. For example,participation in external learning and development programs and Studybankmay not be recorded.


5.25 The ANAO found the completion of participant evaluation forms is themost frequent method of learning and development evaluation undertaken byagencies. Several of the agencies examined consolidate this data as an input tofuture learning and development planning. However, this approach has severalweaknesses, including the scope of the evaluation tool and that not allparticipants complete the forms. This further limits the value of the data collected.This level of evaluation is important to support the continuous improvement ofagencies learning and development activities. However, it should also becomplemented by learning and development evaluations at other levels. Forexample, evaluation of changes on the job and improvements in productivity,as well as an assessment of the contribution of learning and development to theachievement of corporate goals.

5.26 ABS indicated that low level evaluation from participants and supervisorsis relatively easy and cheap to obtain and useful for immediate fine-tuning ofprograms. ABS also found that evaluating and/or reviewing the content, deliveryand relevance of individual programs is also relatively straight forward anduseful for medium term strategy.

5.27 Obtaining more meaningful information about the impact of training onbusiness is more difficult. ABS indicated that, even with the support ofmanagement and human resource practitioners to obtain this type of information,is difficult and resource intensive to develop and implement performancemeasures which provide reliable, useful data. Even where these are developedin a rudimentary sense, rarely can they be sustained over time, as the focusreverts to program planning and delivery.

Evaluation of return on investment and value formoney5.28 Although the evaluation of learning and development return oninvestment (ROI) is considered better practice, its measurement is particularlyproblematic. ROI can be defined as

… the rate at which training returns what was invested (its costs). It is an indicatorof a particular course or programme’s value, worth and merit.15

5.29 One of the main limitations of evaluating learning and development ROIis that it depends on the ability to quantify the value of learning and developmentoutcomes. Accordingly, the use of ROI as a measure of learning and developmentin the APS is limited. The ANAO found that only 63% of agencies were able toprovide aggregate data on the cost of learning and development. The ANAO

15 C A Campbell primer on determining the cost-effectiveness of training part 2, Industrial and CommercialTraining, Volume 27, Number 1, 1995, pp. 18.

85


considers even fewer agencies would be able to quantify the value of theirlearning and development outcomes. However, eight of the surveyed agenciesadvised that they evaluate the ROI of their learning and development.

5.30 Value for money (VFM) was another measure considered during the audit.For the purposes of the APS-wide survey, value for money was explained as ajudgement of the worth of funds expended in the light of the benefits received. Of the67 agencies surveyed, 15% advised that they have undertaken a VFM or ROIevaluation of their learning and development in the last three financial years.16

Agencies’ VFM evaluations have taken various forms including:

• cost comparisons between internal programs and those availablecommercially;

• benchmarking against other APS agencies; and

• assessment of value for money based on participant and supervisorfeedback.

5.31 Of the five agencies examined during the audit, none had undertaken aVFM or ROI assessment of their learning or development strategies, but allagencies recognised the role of these evaluation tools. However, several of theagencies advised that they complete VFM assessments of individual learningand development programs for contract purposes. The agencies also advisedthat the selection of external learning and development providers is undertakenin accordance with the Commonwealth Procurement Guidelines (CPG), whichincludes consideration of VFM. However, an examination of compliance withthe CPG was outside of the scope of the audit.

Evaluation outcomes and future learning anddevelopment planning5.32 Learning and development evaluation primarily aims to gatherinformation to inform and support future learning and development planning,and to assess the value for money and effectiveness of learning and development.As noted earlier, the scope of learning and development evaluations variesbetween agencies. Some agencies are focusing on assessing the impact of theirlearning and development, while other agencies’ evaluation activity is targetedat participant reaction and satisfaction.

5.33 Of the agencies that have recently evaluated their learning anddevelopment strategies or activities, 84% found the outcomes useful in informingtheir learning and development planning.

16 However, in another survey question about the aspects of learning and development evaluated,26 agencies advised that they had considered some aspect of VFM.


5.34 The ANAO considers that, for learning and development evaluations tobe successful, evaluation recommendations should be implemented and followedup. The ANAO noted that of the 34 agencies that responded to the APS-widesurvey question about the implementation of evaluation outcomes, 28 (82%)advised that their learning and development evaluation recommendations areusually implemented and followed up.

5.35 The ANAO examined agencies’ evaluations of several learning anddevelopment programs during the audit. These evaluations were comprehensiveand provided a range of data that informed learning and development planning.The ANAO found that agencies largely use evaluation findings in the furtherdevelopment of their learning and development activities. The ANAO considersthe evaluation of learning and development activities is an important function.However, focusing learning and development evaluation at this operational levellimits its strategic value.

Staff surveys

5.36 As noted earlier, the ANAO observed that some agencies also use staffsurveys to inform future learning and development planning. Thiscomplementary source of information provides agencies with useful data aboutlearning and development priorities and the perceived value of learning anddevelopment across the organisation. Of the 25 agencies that undertook a staffsurvey in 2000–01, 79% advised that the feedback from their staff surveys hasinformed their learning and development planning.

5.37 The ANAO found that four of the agencies reviewed during the audit hadundertaken staff surveys. The survey methodologies varied significantly betweenthe agencies. Some agencies had conducted staff surveys within an individualbusiness unit, while other agencies had undertaken organisation-wide surveys.In particular, FaCS has conducted two recent staff surveys and focus groupresearch. The findings of this research has significantly influenced their learningand development planning and resulted in the development and implementationof a leadership and management development program for its senior staff. TheANAO considers that, in the absence of other data to support formal learning anddevelopment evaluation, the conduct of staff surveys is one of several tools thatcan be used to inform learning and development planning.

Annual reporting requirements5.38 Agencies are required to prepare annual reports that comply with theAnnual Reporting Requirements, issued by the Department of the Prime Ministerand Cabinet on behalf of the JCPAA. In their annual reports, agencies are required

87


to specifically report on human resource management capability. The ReportingRequirements suggest that agencies also report on their key training anddevelopment (learning and development) strategies, the outcomes of trainingand development, and evaluation of the effectiveness of their training anddevelopment.

5.39 The ANAO reviewed the 2001–02 annual reports of the five agenciesexamined during the audit. The level of reporting about learning anddevelopment varied significantly. Only two of the five agencies reported thecost of and/or staff participation in learning and development in their 2001–02annual report. The ANAO observed that agencies generally provide high leveldescriptive details about their learning and development strategies. DFATprovided the most comprehensive details, with descriptions of their keystrategies and data on the number of staff days allocated to learning anddevelopment and the cost of learning and development.

5.40 The ANAO considers that agencies could provide more complete detailsabout learning and development in their annual reports if data collection wasroutine. Agencies make a significant financial commitment to the developmentof their staff and accordingly organisational capability, but discussion of this intheir annual reports is minimal. The annual reports reviewed ranged in lengthfrom 215 to 556 pages, but on average, agencies allocated about one and a halfpages to staff learning and development. Such limited information makes itdifficult for Parliament to review the learning and development performance ofagencies. Another option is to include learning and development informationon agencies’ web sites.

Conclusion5.41 A major goal of learning and development evaluation is to improve currentlearning and development processes in order to achieve maximum businessimpact. Through evaluation, agencies can also assess the efficiency and cost-effectiveness of their learning and development strategies and activities. Theseresults then form an important part of the information base for managementdecision-making and future learning and development planning and delivery.

5.42 Of the 67 agencies surveyed, 27 (40%) either agreed or strongly agreedthat their organisations evaluate the effectiveness of their learning anddevelopment strategies, including the impact of learning and development onthe organisational goals. However, 45% of agencies neither agreed nor disagreedand a further 15% disagreed.

5.43 The ANAO found that, of the 34 agencies that have evaluated at least oneaspect of their learning and development activities, 94% evaluate participant


satisfaction and 80% evaluate both their major learning and developmentactivities and the effectiveness of individual learning and development activitiesin meeting their workplace skills needs. This experience is reflected in the fiveagencies examined in detail.

5.44 However, the ANAO considers that evaluation only at an activity levellimits its potential usefulness. Evaluation at this level does not support thedevelopment of comprehensive learning and development strategies that targetpriority needs and align learning and development with agencies’ corporategoals. Accordingly, the ANAO considers that, without the regular and systematicevaluation of their learning and development strategies, agencies are unable tovalidate claims of business impact, efficiency and cost-effectiveness of learningand development.

5.45 To assess the impact and value of their investment in learning anddevelopment, organisations should evaluate their learning and developmentstrategies and activities on a regular basis in a planned and structured manner.Of the 34 agencies that evaluate their learning and development activities, 25(74%) had an overall learning and development evaluation plan. Of these25 agencies, 14 also advised that they had evaluated their learning anddevelopment strategies.

5.46 The ANAO found that the timing of evaluation activity varied betweenagencies. As noted earlier, agencies most frequently evaluate participantsatisfaction. This is undertaken on a routine basis. However, the review ofagencies learning and development strategies is less frequent.

5.47 For learning and development evaluation to be able to achieve requiredoutcomes, the ANAO considers that access to complete and consistent learningand development data is required. The ANAO found that the scope of learningand development data available from agencies was limited. For example, only63% of agencies provided data on the aggregate cost of formal learning anddevelopment for 2000–01 and only 34% of agencies supplied data on the numberof staff training days for their organisation in 2000–01. Requests for learningand development data at a more detailed level resulted in lower response rates.

5.48 The ANAO found that agencies either have not undertaken studies of, orare generally unable to assess, the ROI and VFM of their outlays on learningand development. This is largely due to the limited availability of learning anddevelopment data. For this reason, the ANAO considers the assessment oflearning and development ROI in the APS is problematic and of limited use inthe current environment. The ANAO found that some agencies have evaluatedthe VFM of their learning and development activities, but this has largely beenfor contract purposes. The ANAO considers the collection of reliable learning

89


and development data would significantly enhance agencies’ abilities to bothplan for, and evaluate, the impact of their learning and development strategiesand activities.

5.49 The ANAO observed that 84% of agencies that have recently evaluatedtheir learning and development strategies or activities found the outcomes usefulin both informing their learning and development planning and for managementdecision-making. However, the level of evaluation undertaken is relatively low.This impacts on management’s abilities to make informed strategic decisionsabout agencies’ investments in learning and development.

Better Practice PrinciplesEvaluation of learning and development strategies and activities to

determine effect (including business impact)So that continuous improvement and increased organisational effectivenessare assured, agency evaluation of learning and development shouldencompass:

• assessment of impacts (as foreshadowed in human resource managementand/or learning and development strategies and plans);

• regular and systematic assessment and review

• assessment of value for money (VFM);

• use of reliable learning and development data; and

• use of the results in decision making and future planning.

General performance information to support agencies in monitoring andreporting on evaluation for impact could include the:

• extent to which original objectives have been achieved;

• extent to which intended organisational impact of learning anddevelopment is achieved;

• extent to which learning and development activities are continuallyimproved and ineffectual activities ceased; and

• extent to which evaluation results inform future learning anddevelopment planning.


6. Role of the PSMPC

This chapter discusses the role and responsibilities of the PSMPC in supportinglearning and development across the APS. This has primarily been undertaken withinthe context of the Public Service Commissioner’s functions and statutoryresponsibilities as conferred by the Public Service Act 1999.

Background6.1 The PSMPC is an agency of 157 staff with a Budget appropriation in2001–02 of $14.8 million whose mission is:

to foster the achievement of a high performing, ethical and client focused publicservice, promoting quality management of people and work.17

6.2 The PSMPC seeks to deliver three outputs to government to support itsgoal of:

contributing to the desired outcome of encouraging the development of afuture-focused, innovative and professional APS.18

6.3 These outputs are:

• Output 1—APS policy, legislation and information;

• Output 2—leadership, performance and development in the APS; and

• Output 3—employment-related services and functions.

6.4 An examination of key documentation19 indicated four distinct but relatedroles for the PSMPC with respect to learning and development:

(a) facilitating—assistance to agencies in their planning and management oflearning and development;

(b) advising—policy advice to Government on strategic learning anddevelopment issues and advice to agency heads to support theimplementation of relevant policy and legislation;

(c) reporting and evaluation—APS-wide evaluation and reporting toGovernment and Parliament on learning and development issues to assistin policy development and accountability; and

17 The Role and Services of the PSMPC (2000), p. 3.18 Public Service Commissioner’s Annual Report 2000–01, p. 5.19 Public Service Act 1999, The Role and Services of the PSMPC (2000), Portfolio Budget Statements

2001–02: Prime Minister and Cabinet Portfolio, Budget Related paper No 1.14, part C: Agency BudgetStatements—PSMPC, Public Service Commissioner Annual Report 2000–01, and The role of thePSMPC in relation to learning and development, 13 February 2002, PSMPC internal working paper.

91

Role of the PSMPC

(d) service delivery—the delivery of learning and development programs andactivities to other APS agencies.

6.5 The ANAO examined the role of the PSMPC at two levels. First, at a whole-of-service level including the extent of the guidance offered to APS agencies toassist with their strategic learning and development planning; the extent ofadvice offered to Government and agency heads and management; and theevaluation and reporting of learning and development issues across the APS.Second, in terms of the administration of training and development programsconducted by the PSMPC for other APS agencies.

PSMPC approach6.6 The PSMPC advised that all the Public Service Commissioner’s functionsand responsibilities are undertaken in the context of a highly devolved APSemployment environment where agency heads are responsible for theemployment arrangements and performance of their organisations. The PSMPCadvised that, as a consequence, agency heads have both the authority andresponsibility to meet the distinctive needs of their own organisations, with thelegislative framework based on the premise that there is no single APS labourmarket.

6.7 In keeping with the devolution of responsibility for the management ofdepartments to Secretaries under section 57 of the Public Service Act 1999, thePSMPC provides advice and guidance to agencies, rather than prescriptive rules.The PSMPC advised the ANAO that:

… [its] approach is generally one of ‘facilitating’, ‘supporting’, ‘contributing to’and ‘fostering’. [It] also, however, has certain evaluation responsibilities and, to adegree, a coordinating, developing and promoting role.20

6.8 The PSMPC further advised that its role, supported by the legislativeframework, is based on the premise that the prime responsibility for the deliveryand effectiveness of learning and development in the APS rests with agencyheads.

PSMPC’s facilitating role6.9 The ANAO observed that the PSMPC provides a range of support servicesaimed at meeting the broad requirements of agencies across the APS. As statedearlier, the PSMPC considers that, while agencies have prime responsibility forlearning and development, it has an important APS-wide role in assisting

20 The role of the PSMPC in relation to learning and development, 13 February 2002, PSMPC internalworking paper.


agencies with their role, and in promoting good practice. The PSMPC aims toachieve this through supporting agencies in increasing the strategic focus oftheir human resource activities, including learning and development.

Support for alignment of learning and development withcorporate goals

6.10 The PSMPC provides agencies with support for the alignment of learningand development with corporate goals through collaborative approaches withagencies at various levels. These included the various human resourcemanagement networks, practitioner workshops on workforce planning and aseries of focus groups conducted with agencies in 2001 to identify future issuesand the support the PSMPC might provide. For example:

• sponsorship of the Human Resource Management Strategic Reform Groupand the Human Resource Directors’ Forum;

• delivering the Driving Change program for human resource practitioners;

• facilitating the Investors in People (IiP) program;

• regular focus groups with client agencies and agency visits by PSMPCstaff to identify future requirements and support open communicationbetween agencies and the PSMPC on learning and development needsand assess the success of current efforts; and

• specific courses offered through the APS Development Program onworkforce planning and strategic thinking.

6.11 Recent PSMPC State of the Service Reports have emphasised the need forthe APS to link workforce planning, future capability requirements and learningand development strategies. However, the PSMPC does not provide specificpolicies, frameworks or other planning mechanisms to assist agencies in thealignment of their learning and development with their corporate goals.

6.12 The PSMPC supports the integration of learning and development withother human resource functions in agencies through the range of fora and focusgroups they sponsor. They also share international and other relevant researchthrough these avenues.

Future skills requirements and capability frameworks

6.13 The PSMPC advised that future skill requirements of the APS workforcewill be agency specific due to the devolved management framework under thePublic Service Act 1999, and the many labour markets operating within the APS.They reported that many agencies had already developed agency-specific

93

Role of the PSMPC

capability frameworks to reflect their individual current and future skillrequirements. The ANAO was able to substantiate this during audit fieldwork,with all five agencies examined in detail having developed their own capabilityframeworks.

6.14 The SES Leadership Capability Framework and the HR CapabilityFramework also facilitate and encourage the integration of learning anddevelopment with other human resource functions through the scope of thecapabilities identified and the implementation strategies recommended. Forexample, both frameworks can be used to provide a consistent and integratedbasis for recruitment and selection, learning and development, career planningand succession management.

6.15 Two specific questions from the APS-wide survey were directed atidentifying agency adoption of these PSMPC initiatives. These were:

How does your organisation use the Senior Executive Leadership CapabilityFramework? and

How does your organisation use the PSMPC HR Capability Model (2001)?

6.16 Slightly more than half (58%, i.e. 40) of the agencies surveyed indicatedthat they used the PSMPC HR Capability Model. The most frequent use (by33 agencies) was as a general reference document, with 13 agencies indicatingthat they have used the Model as a general human resource planning framework.Only five agencies indicated that they used the Model as an evaluationframework.

6.17 In response to the APS-wide survey of learning and development most(78%, i.e. 52) agencies indicated that they used the PSMPC Senior ExecutiveLeadership Capability Framework. Of these agencies most (i.e. 42) indicatedthat they used the Framework specifically for selection and learning anddevelopment of either SES or other staff. Half of these agencies also indicatedthat they used the Framework as an input to organisational capabilities. Only12 agencies identified that they used the Framework for evaluation purposes.

6.18 By comparison, the PSMPC found21 that 38% of agencies specificallyreported using the Senior Executive Capability Framework in various aspectsof their workforce planning or learning and development strategies. The ANAOconsiders that, given all agencies have SES staff and executive level staff, thereis scope for agencies to increase their use and application of this Framework toprovide greater integration.

21 Public Service and Merit Protection Commission State of the Service Report 2000–01, p. 138.


6.19 In summary, the ANAO found that the extensive range of activitiescoordinated by the PSMPC demonstrates a strong commitment to its facilitationrole by encouraging the sharing of information across agencies and through theprovision of capability frameworks. However, there is scope for the PSMPC tobetter target its facilitation efforts to ensure continuous improvement in learningand development.

Recommendation No. 16.20 The ANAO recommends that, in order to support continuousimprovement in learning and development across the APS, the PSMPC furtherexplore with stakeholders, the scope for a more targeted facilitation role,including the identification and promulgation of better practice principlesrelevant to the APS environment.

PSMPC Response:

6.21 Agree.

PSMPC’s advisory role6.22 The PSMPC’s output structure for 2001–02 outlined in the Portfolio BudgetStatements 2001–02 includes:

Provide advice and support to government on APS policy … Provide advice toAgency Heads to support the implementation of government APS policy…Evaluate and provide information on, the performance of the APS through theState of the Service Report …22

6.23 The PSMPC advised the ANAO that it provides policy advice to theMinister on a range of APS employment matters, including, from time to time,advice on learning and development in the APS. The PSMPC indicated thatsuch advice focuses on key capability issues and related strategic directions ofMinisterial interest. The design of the Senior Executive Leadership CapabilityFramework was offered as an example of such work.

6.24 The PSMPC advised that it also provides advice to agency heads andmanagers on strategic people management and organisational performancethrough:

• networks for specific interest groups (e.g.: Strategic Human ResourceReform Group, Investors in People Coordinators Network, and LeadershipDevelopment Network);

22 Portfolio Budget Statements 2001–02: Prime Minister and Cabinet Portfolio, Budget Related paperNo 1.14, part C: Agency Budget Statements—PSMPC, p. 169: Revised Output structure for 2000–01,referring to changes as a result of the 2000–01 budget.

95

Role of the PSMPC

• SES seminar series and Executive Level development programs;

• access by agencies to a panel of human resource management consultants(including learning and development); and

• responding to specific ad hoc requests.

6.25 However, the ANAO was unable to identify regular, documented specificadvice to agency heads or managers on strategic people management ororganisational performance. The ANAO also considers that the quality of adviceto Government and agency management would be enhanced by the collectionand analysis of APS-wide data on learning and development.

PSMPC’s reporting and evaluation role

Learning and development performance information

6.26 The PSMPC Portfolio Budget Statements 2001–02 also includes acommitment to:

… [E]valuate, and provide information on, the performance of the APS through theState of the Service Report, Workplace Diversity Report and Statistical Bulletin. 23

6.27 Such a commitment could enable Parliament to monitor the effectivenessof learning and development across the APS.

6.28 However, there is no whole-of-APS performance information relevant tolearning and development reported in either the Public Service Commissioner’s2000–01 Annual Report or State of the Service Report.24 However, the currentlack of data on learning and development in agencies means that very fewagencies are in a position to make an informed assessment of the efficiency andcost-effectiveness of their learning and development programs. Consequently,it is difficult for the PSMPC, or any other agency, to make an APS-wideassessment at the present time.

6.29 In addition, there are no common learning and development performanceindicators across the APS; which also makes it difficult to report on a whole ofgovernment basis. The adoption of a consistent set of performance indicatorsacross agencies, together with the collection and analysis of appropriate data,will enable regular APS-wide assessments to be made in the future.

23 Portfolio Budget Statements 2001–02: Prime Minister and Cabinet Portfolio, Budget Related paperNo 1.14, part C: Agency Budget Statements—PSMPC, p. 169.

24 In preparing the 2000–01 Report, agencies were only asked to describe progress made on thedevelopment and application of workforce planning frameworks. No information was sought on learningand development issues.


Evaluation of learning and development

6.30 The PSMPC provides a range of support to agencies in undertakingevaluations of their learning and development efforts. These primarily include:

• facilitation of the IiP program;

• provision of a specific ‘evaluation’ category in the panel of consultants;and

• evaluation of emerging trends, such as ROI, and promulgation to agenciesthrough networks and fora, and support for adoption and implementation.

However, the support available to agencies from the PSMPC to assist them withevaluation and reporting is not extensive.

6.31 Reliable data is fundamental for evaluation. The PSMPC maintains theAPS Employment Database (APSED), which contains APS-wide informationon staff employed under the Public Service Act 1999, including employment basis(ongoing, non-ongoing etc), education qualifications, EEO data etc. Whileagencies are able to request specific data analyses (e.g. retention rates)25 fromthe PSMPC to support agency research and evaluation efforts, the PSMPC doesnot record any learning and development data in APSED. However, the PSMPCdoes record participation statistics for the courses they offer, which can be madeavailable to agencies on request.

6.32 The PSMPC conducts ad-hoc reviews of its own courses and client groupfeedback. The results of these reviews can also inform the agencies’ own planningand evaluation processes as well as the PSMPC’s planning processes. However,the ANAO could not identify any recent APS-wide evaluation of learning anddevelopment, nor is one planned. The absence of an evaluation strategy or planalso prevents an efficient and targeted approach to evaluation and reporting.

6.33 The PSMPC’s ability to adequately report to Parliament on the efficiencyand effectiveness of learning and development across the APS is dependent onit undertaking systematic evaluations and routinely collecting supporting data.The ANAO found that there is a high priority need for the systematic collectionand analysis of basic learning and development performance information tounderpin reporting and accountability to Parliament. The PSMPC is in a goodposition to coordinate the collection of consistent data and facilitate whole ofgovernment reporting on all (not just learning and development) people issues.

25 Public Service Commissioner Annual Report 2000–01, p. 6.

97

Role of the PSMPC

Recommendation No. 26.34 The ANAO recommends that, in order to assist agencies in evaluating theimpact of their learning and development and to assist Parliament in monitoringAPS-wide learning and development, the PSMPC:

• include learning and development as part of an evaluation strategy andplan undertaken under the umbrella of the State of the Service Report;

• provide guidance and support to agencies on evaluating the impact oflearning and development;

• identify key APS-wide learning and development performance indicators;and

• include an analysis of trends in learning and development in the State ofthe Service Report.

PSMPC response:

6.35 Agree. One of the PSMPC’s strategic priorities is to strengthen its focuson evaluation through the SoSR and to undertake more specific issues-basedprojects. The SoSR this year will include an emphasis on aspects of learning anddevelopment. Learning and development may also be an appropriate issue topursue in a specific issues-based project in the short to medium term.

6.36 The PSMPC agrees that it is important to encourage agencies to take actionto gain an understanding of what they are spending on learning anddevelopment and what value they are gaining from their investment.

6.37 The PSMPC looks forward to working with the ANAO in the second halfof 2002 to develop a Better practice Guide that will provide guidance and supportto agencies in their ongoing role of managing the delivery of an effective learningand development strategy. It is envisaged that this Better practice Guide willinclude APS-wide performance indicators and provide good practice andguidance on performance evaluation and monitoring techniques.

PSMPC’s service delivery role6.38 Output 2 of the PSMPC’s 2001–02 Portfolio Budget Statements includesthe provision of service-wide development activities. The PSMPC thereforeprovides learning and development programs on a fee-for-service basis to otherAPS agencies.

6.39 The People and Organisation Development (POD) Team of the PSMPChas specific responsibility for the development and delivery of management


and skills development programs. In 2001–02, 960 development activities areexpected to be delivered and revenue of $10.1 million raised.

6.40 Specifically, the PSMPC offers leadership extension programs for the SESand development programs for executive level and APS staff. It also convenesthe Leadership Development Network, and provides the Senior ExecutiveLeadership Capability Framework and 360-degree feedback questionnaire onSenior Executive leadership capabilities. In addition to these structured eventsand fora, the PSMPC also provides ad-hoc tailored in-house services to agencies,again on a fee-for-service basis, to explore and address specific issues.

6.41 The PSMPC also facilitates the provision of APS-wide development andtraining by developing and making available to agencies a panel of trainingconsultants.

Assessment of cost-effectiveness

6.42 The PSMPC coordinates the selection of a panel of learning anddevelopment providers through an open tender process to provide agencieswith a panel of consultants and contractors. The PSMPC indicated that this panelprovides itself and agencies with an effective benchmark of value for moneyagainst the rest of the market, given the number of tenders considered and therange of services evaluated for inclusion in the panel. Apart from the market-testing processes involved in establishing the panel of contractors and selectingproviders for the courses offered, the PSMPC has not conducted any value formoney analysis of their services.

6.43 The PSMPC states that it aims to achieve efficient and cost-effectivedelivery primarily through the range of learning and development productsthat they offer. The PSMPC target these products at meeting high priority APSneeds. A summary of the PSMPC learning and development products is atAppendix 2. The PSMPC claims that these offerings, particularly those on theAPS Development Programs calendar, represent best value for money. Whileacknowledging that it may not always be the cheapest, the PSMPC claims thatthe courses it offers are specifically tailored to APS issues and needs.

6.44 The PSMPC does not have a documented evaluation strategy or plan fortheir learning and development products and services. However, as noted earlier,the PSMPC conducts ad-hoc reviews of its own courses and reviews client groupfeedback. Recent examples include an evaluation conducted by a consultant onthe views of a sample of Band 2 Senior Executives on learning and developmentneeds; and a comprehensive internal review of the first 12 Career DevelopmentAssessment Centres (CDAC). The CDAC evaluation led to changes in the designof the program and the nature of feedback and follow up provided to participants

99

Role of the PSMPC

and their sponsoring agency. The PSMPC have also commenced a longitudinalevaluation of the SWIM program that is expected to be completed in 2002. Oneway of evaluating the effectiveness of the services offered by the PSMPC inmeeting the needs of each APS group, is to monitor the take-up of their coursesagainst the potential take up from the relevant target group to ascertain marketpenetration. However, it is recognised that not all staff within a particular groupwill share common development needs. In addition, larger agencies are betterplaced to meet many of their learning and development needs from internalresources, with less need to turn to external providers such as the PSMPC.

Contract management

6.45 An assessment of learning and development associated with contractmanagement was included in this audit as it was seen as being widely applicableacross agencies and generally indicative of how agencies respond to emerginglearning and development requirements. The Public Service Commissionerreported that

…there is general consensus across the service that the acquisition anddevelopment of contract management skills in the new environment is essentialto agency operations. 26

6.46 The 2000–01 State of the Service Report specifically examined contractmanagement27 in relation to IT outsourcing, and identified that agencies mustdevelop proficient contract management skills. The PSMPC currently offersseparate SES and non-SES workshops on the management and negotiation ofcontracts to complement the specific in-house initiatives developed by agencies.

6.47 The ANAO found that the PSMPC devoted considerable effort to itsservice-wide development activities and provided a comprehensive range ofrelevant learning and development opportunities for its several client groups.However, the efficiency and cost-effectiveness of its development activities couldbe better measured and assessed.

26 Public Service and Merit Protection Commission State of the Service Report 2000–01, p. 13.27 Public Service and Merit Protection Commission State of the Service Report 2000-01, pp. 127–9.


Recommendation No. 36.48 The ANAO recommends that the PSMPC, in the interests of efficient andcost-effective delivery of learning and development across the APS, regularlyassess the value for money provided by their services and implement anevaluation strategy for the programs offered by the PSMPC.

PSMPC response:

6.49 Agree. The PSMPC agrees with the need to assess the value for moneyprovided by its services. Mechanisms such as regular annual reviews of programshave been put in place as part of the PSMPC’s ongoing focus on ensuring thecredibility, relevance and high quality of PSMPC programs. Agency willingness(or otherwise) to fund their employees’ participation in PSMPC programs alsoprovides some discipline on the PSMPC and a guide to the value of programs.

6.50 Where possible, the PSMPC works closely with agencies to developprograms that include follow up evaluations of learning and developmentoutcomes. Recent examples of this include:

• The Career Development Assessment Centres (CDAC) where the PSMPChas undertaken an evaluation focusing on changes in behaviour andperformance of the early cohort of participants; and

• The Senior Women in Management Program (SWIM) where the PSMPCis currently conducting a review to assess the program’s current relevanceand utility, and will establish agencies’ current perceptions and futurerequirements of the programs.

6.51 The PSMPC has made a commitment in its business planning for2002–2003 to develop an evaluation strategy for the learning and developmentprograms it offers to APS agencies. This strategy will include piloting ofevaluation methodologies for higher value programs.

Conclusion6.52 The ANAO found that PSMPC facilitates the sharing of information aboutlearning and development across agencies by coordinating an extensive seriesof HRM networks and working collaboratively with agencies. However, thereis scope for the PSMPC to undertake a more catalytic role. In particular, bettertargeting of its facilitation efforts to ensure continuous improvement in learningand development.

6.53 The PSMPC advised that it provides advice to Government and agencyheads and managers on strategic people management and organisationalperformance issues, including learning and development.

101

Role of the PSMPC

6.54 However, the ANAO was unable to identify regular, documented, specificadvice to agency management on such issues. The ANAO also considers thatthe quality of advice to Government and agency heads would be enhanced bythe collection and analysis of APS-wide data on learning and development.

6.55 The PSMPC’s ability to adequately report to Parliament on the efficiencyand effectiveness of learning and development across the APS is dependent onit undertaking systematic evaluations and routinely collecting supporting data.The ANAO found that there is a high priority need for the systematic collectionand analysis of basic learning and development performance information tounderpin reporting and accountability to Parliament. The PSMPC is in a goodposition to coordinate the collection of consistent data and facilitate whole ofgovernment reporting on all (not just learning and development) significantpeople issues.

6.56 The ANAO found that the PSMPC devoted considerable effort to itsservice-wide development activities and provided a comprehensive range ofrelevant learning and development opportunities for its several client groups.However, the efficiency and cost-effectiveness of its development activities couldbe better measured and/or assessed for improved results.

Canberra ACT P. J. Barrett28 June 2002 Auditor-General

103

Appendices

105

Appendices

Appendix 1

Agencies completing the survey

All 67 agencies approached to participate in the survey returned completedresponses. These agencies were:

Aboriginal Hostels LimitedAustralian Competition and Consumer CommissionAdministrative Appeals TribunalAboriginal and Torres Strait Islander CommissionAttorney-General’s DepartmentAusAIDAustralian Broadcasting AuthorityAustralian Bureau of StatisticsAustralian Communications AuthorityAustralian Customs ServiceAustralian Electoral CommissionAustralian Geological Survey OrganisationAustralian Government Analytical LaboratoryAustralian Greenhouse OfficeAustralian Industrial RegistryAustralian Institute of Health and WelfareAustralian National Audit OfficeAustralian National Maritime MuseumAustralian New Zealand Food AuthorityAustralian Protective ServiceAustralian Radiation Protection & Nuclear Safety AgencyAustralian Securities and Investment CommissionAustralian Taxation OfficeAustralian War MemorialBureau of MeteorologyBureau of Rural ResourcesCentrelinkChild Support AgencyComcareComSuperCRS AustraliaDepartment of Agriculture, Fisheries and Forestry—AustraliaDepartment of Communications, Information Technology and the ArtsDepartment of Defence


Department of Education, Training and Youth AffairsDepartment of Employment, Workplace Relations and Small BusinessDepartment of Family & Community ServicesDepartment of Finance & AdministrationDepartment of Foreign Affairs and TradeDepartment of Health and Aged CareDepartment of Immigration and Multicultural AffairsDepartment of Industry, Science and ResourcesDepartment of the Prime Minister & CabinetDepartment of Transport and Regional ServicesDepartment of the TreasuryDepartment of Veterans’ AffairsEnvironment AustraliaFamily Court of AustraliaFederal Court of AustraliaGreat Barrier Reef Marine Park AuthorityInsolvency and Trustee Service AustraliaIP AustraliaMigration Review TribunalNational Archives of AustraliaNation Crime AuthorityNation Office for the Information EconomyNational Library of AustraliaNational Museum of AustraliaNational Native Title TribunalNational Occupational Health & Safety CommissionOffice of the Director of Public ProsecutionsProductivity CommissionPublic Service & Merit Protection CommissionQUESTACONRefugee Review TribunalRoyal Australian MintScreensound Australia

107

Appendices

Appendix 2

PSMPC learning and development productsFor SES staff:

Career Point Programs:Induction into the Australian Public ServiceOrientation to the Senior Executive ServiceExecutive Leadership Development Program (ELDP)Sustaining the Leadership ContributionExpertise Development Programs

Achieving Results Through Strategic ChangeEffective Personal LeadershipExecutive Coach and MentorLeading Complex, High Impact ProjectsManagement and Negotiation of ContractsPreparing to Appear Before Parliamentary CommitteesPublic Servants’ Accountability, Rights and ResponsibilitiesStrategies for Better GovernanceStrategic Thinking, Strategic Planning and Needs Assessment

Futures WorkshopsSeminar Series:

Band 3 ForumBreakfast BriefingsFutures ForumLunchtime SeminarsSenior Executive Updates

Leadership Extension Programs:Leadership for a Democratic SocietyLeading Australia’s Future in Asia (LAFIA)Senior Executive FellowshipsStrategic Public Sector Leaders’ Summits

Individual Development Services:Executive Coaching ServicesLearning SetsPersonal Efficiency Program (PEP)

For Executive Level (EL) staff:

Career Development Assessment Centres (CDAC)Senior Women In Management (SWIM)Executive Level development programs:

Coaching and Mentoring


Develop a Persuasive Business CaseEffective Governance in APS OrganisationsEL UpdateFinancial Management: a Strategic PerspectiveInfluencing SkillsLeadership through Policy ChangeLeading your TeamManaging Underperformance in the WorkplaceMarketing yourself into a Senior Executive Service PositionOrganisational Communication: Strategies, Tactics and ManagementPersuade Me! Writing for a Competitive EdgePlanning and Conducting a TenderPreventing and Resolving DisputesStrategic ThinkingLeading Complex, High Impact ProjectsPublic Servants Accountability, Rights and Responsibilities (PSARR)

For APS 5–6 and EL staff:

Public Sector Management Course (comprehensive tertiary level managementprogram—4 weeks over 12 months)

For APS staff:

Driving Change Program:Seminar seriesWorkshops

APS Development Programs:Getting that Contract RightProject Management—delivering critically acclaimed projectsConflict ResolutionHarassment Contact Officer TrainingHarassment Contact Officer Training—RefresherInfluencing SkillsLeadership Skills for WomenBetter LettersAdvanced Writing SkillsSmart Reading SkillsDeveloping Client SatisfactionUnderstanding the APS ValuesFinancial Management in the APS: a Management Perspective—Introduction (APS 5–6)Financial Management in the APS: a Management Perspective—Beyondthe Basics (APS 5–6)

109

Appendices

Financial Management in the APS: an Operational Perspective—Introduction (APS 1–4)Financial Management in the APS: an Operation Perspective—Beyondthe Basics (APS 1–4)Policy Formulation and Advice—IntroductionPolicy Formulation and Advice—AdvancedTracking your RightsCareer Directions WorkshopInterview Tips and TechniquesMarketing yourself into an Executive Level PositionSurviving and Thriving in the APSWinning that Job

For graduates:

Introduction to Government and the APS Lunchtime SeminarGraduate Networking SessionsAPS Values, Ethics and AccountabilityCorporate GovernanceWorkforce PlanningLeadershipProgressive Round TableE GovernmentManaging Risk in OutsourcingMaking a ContributionGraduate Training WorkshopsCareer Directions WorkshopDeveloping Client SatisfactionFinancial Management in the APSInfluencing SkillsInterview Tips and TechniquesLegislation and APS Decision-MakingPolicy Formulation and AdviceProject ManagementSmart Reading SkillsStrategic ThinkingSurviving and Thriving in the APSWriting for a Competitive Edge

Networks for specific interest groups:

Futures Forum (joint venture with other agencies)Investors in People Coordinators NetworkLeadership Development Network


Strategic HR Reform GroupHR Directors Network

Tools:

Senior Executive Leadership Capability Frameworkand the 360 Degree Feedback Questionnaire on Senior Executive LeadershipCapabilitiesHR Capability Modeland the 360 Degree Feedback Questionnaire on the HR Capability Model

Others:

Panel of consultants—offered through PSMPC’s ‘In house services’

111

Series TitlesAudit Report No.63 Performance AuditManagement of the DASFLEET Tied Contract

Audit Report No.62 Information Support ServicesBenchmarking the Finance Function Follow-up ReportBenchmarking Study

Audit Report No.61 Information Support ServicesManaging People for Business Outcomes Benchmarking Study

Audit Report No.60 Performance AuditCosting of Operational Activities and Services Follow-up AuditCentrelink

Audit Report No.59 Performance AuditAusAID Contract ManagementAustralian Agency for International Development

Audit Report No.58 Performance AuditDefence Property ManagementDepartment of Defence

Audit Report No.57 Performance AuditManagement Framework for Preventing Unlawful Entry into Australian TerritoryDepartment of Immigration and Multicultural and Indigenous Affairs

Audit Report No.56 Performance AuditWorkforce Planning in the Department ofImmigration and Multicultural and Indigenous Affairs

Audit Report No.55 Performance AuditAdministration of Tobacco ExciseAustralian Taxation Office

Audit Report No.54 Performance AuditDrug Detection in Air and Containerised Sea Cargo and Small CraftAustralian Customs Service

Audit Report No.53 Assurance and Control Assessment AuditGoods and Services Tax Administration by Commonwealth Organisations

Audit Report No.52 Financial Control and Administration AuditInternal Budgeting

Audit Report No.51 Performance AuditResearch Project ManagementCommonwealth Scientific and Industrial Research Organisation (CSIRO)


Audit Report No.50A Preliminary Examination into the Allocation of Grant Funding for the Co-Location ofNational General Practice Organisations

Audit Report No.49 Performance AuditThe Management of Commonwealth National Parks and Reserves‘Conserving our Country’Department of the Environment and Heritage

Audit Report No.48 Performance AuditRegional Assistance ProgrammeDepartment of Transport and Regional Services

Audit Report No.47 Performance AuditAdministration of the 30 Per Cent Private Health Insurance RebateDepartment of Health and Ageing, Health Insurance Commission, Australian TaxationOffice, Department of Finance and Administration, Department of the Treasury

Audit Report No.46 Performance AuditManagement of an IT Outsourcing ContractDepartment of Veterans’ Affairs

Audit Report No.45 Assurance and Control Assessment AuditRecordkeeping

Audit Report No.44 Performance AuditAustralian Defence Force Fuel ManagementDepartment of Defence

Audit Report No.43 Performance AuditIndigenous Education StrategiesDepartment of Education, Science and Training

Audit Report No.42 Performance AuditIntegrity of the Electoral RollAustralian Electoral Commission

Audit Report No.41 Performance AuditTransactional Banking Practices in Selected Agencies

Audit Report No.40 Performance AuditCorporate Governance in the Australian Broadcasting CorporationAustralian Broadcasting Corporation

Audit Report No.39 Performance AuditManagement of the Provision of Information to Job SeekersDepartment of Employment and Workplace Relations

Audit Report No.38 Performance AuditManagement of Australian Defence Force Deployments to East TimorDepartment of Defence

113

Audit Report No.37 Performance AuditPurchase of Hospital Services from State Governments—Follow Up AuditDepartment of Veterans’ Affairs

Audit Report No.36 Benchmarking StudyBenchmarking Implementation and Production Costs of Financial ManagementInformation Systems

Audit Report No.35 Performance AuditATO Progress in Addressing the Cash EconomyAustralian Taxation Office

Audit Report No.34 Assurance and Control Assessment AuditManagement of Travel—Use of Taxis

Audit Report No.33 Assurance and Control Assessment AuditSenate Order of 20 June 2001 (February 2002)

Audit Report No.32 Performance AuditHome and Community Care Follow-up AuditDepartment of Health and Ageing

Audit Report No.31 Performance AuditAudit Activity Report: July to December 2001Summary of Outcomes

Audit Report No. 30 Performance AuditTest and Evaluation of Major Defence Equipment AcquisitionsDepartment of Defence

Audit Report No.29 Financial Statement AuditAudits of the Financial Statements of Commonwealth Entities for the Period Ended30 June 2001

Audit Report No.28 Information Support ServicesAn Analysis of the Chief Financial Officer Function in Commonwealth OrganisationsBenchmark Study

Audit Report No.27 Assurance and Control Assessment AuditAgency Management of Software Licensing

Audit Report No.26 Performance AuditManagement of Fraud and Incorrect Payment in Centrelink

Audit Report No.25 Assurance and Control Assessment AuditAccounts Receivable

Audit Report No.24 Performance AuditStatus Reporting of Major Defence Acquisition ProjectsDepartment of Defence


Audit Report No.23 Performance AuditBroadcasting Planning and LicensingThe Australian Broadcasting Authority

Audit Report No.22 Protective Security AuditPersonnel Security—Management of Security Clearances

Audit Report No.21 Performance AuditDeveloping Policy AdviceDepartment of Education, Training and Youth Affairs, Department of Employment,Workplace Relations and Small Business, Department of Family and Community Services

Audit Report No.20 Performance AuditFraud Control Arrangements in the Department of Agriculture, Fisheries and Forestry—Australia (AFFA)Department of Agriculture, Fisheries and Forestry—Australia

Audit Report No.19 Assurance and Control Assessment AuditPayroll Management

Audit Report No.18 Performance AuditPerformance Information in Portfolio Budget Statements

Audit Report No.17 Performance AuditAdministration of Petroleum Excise CollectionsAustralian Taxation Office

Audit Report No.16 Performance AuditDefence Reform Program Management and OutcomesDepartment of Defence

Audit Report No.15 Performance AuditAgencies’ Oversight of Works Australia Client Advances

Audit Report No.14 Performance AuditClient Service Initiatives Follow-up AuditAustralian Trade Commission (Austrade)

Audit Report No.13 Performance AuditInternet Security within Commonwealth Government Agencies

Audit Report No.12 Financial Control and Administration AuditSelection, Implementation and Management of Financial Management InformationSystems in Commonwealth Agencies

Audit Report No.11 Performance AuditAdministration of the Federation Fund Programme

Audit Report No.10 Assurance and Control Assessment AuditManagement of Bank Accounts by Agencies

115

Audit Report No.9 Performance AuditLearning for Skills and Knowledge—Customer Service OfficersCentrelink

Audit Report No.8 Assurance and Control Assessment AuditDisposal of Infrastructure, Plant and Equipment

Audit Report No.7 Audit Activity ReportAudit Activity Report: January to June 2001Summary of Outcomes

Audit Report No.6 Performance AuditCommonwealth Fisheries Management: Follow-up AuditAustralian Fisheries Management Authority

Audit Report No.5 Performance AuditParliamentarians’ Entitlements: 1999–2000

Audit Report No.4 Performance AuditCommonwealth Estate Property SalesDepartment of Finance and Administration

Audit Report No.3 Performance AuditThe Australian Taxation Office’s Administration of Taxation RulingsAustralian Taxation Office

Audit Report No.2 Performance AuditExamination of Allegations Relating to Sales Tax FraudAustralian Taxation Office

Audit Report No.1 Financial Statement AuditControl Structures as part of the Audits of the Financial Statements of MajorCommonwealth Entities for the Year Ended 30 June 2001


Better Practice GuidesAdministration of Grants May 2002

Performance Information in Portfolio Budget Statements May 2002

Life-Cycle Costing Dec 2001

Some Better Practice Principles for DevelopingPolicy Advice Nov 2001

Rehabilitation: Managing Return to Work Jun 2001

Internet Delivery Decisions Apr 2001

Planning for the Workforce of the Future Mar 2001

Contract Management Feb 2001

AMODEL Illustrative Financial Statements 2001 May 2001

Business Continuity Management Jan 2000

Building a Better Financial Management Framework Nov 1999

Building Better Financial Management Support Nov 1999

Managing APS Staff Reductions(in Audit Report No.49 1998–99) Jun 1999

Commonwealth Agency Energy Management Jun 1999

Corporate Governance in Commonwealth Authoritiesand Companies–Principles and Better Practices Jun 1999

Managing Parliamentary Workflow Jun 1999

Cash Management Mar 1999

Management of Occupational Stress inCommonwealth Agencies Dec 1998

Security and Control for SAP R/3 Oct 1998

Selecting Suppliers: Managing the Risk Oct 1998

New Directions in Internal Audit Jul 1998

Controlling Performance and Outcomes Dec 1997

Management of Accounts Receivable Dec 1997

Protective Security Principles(in Audit Report No.21 1997–98) Dec 1997

Public Sector Travel Dec 1997

117

Audit Committees Jul 1997

Core Public Sector Corporate Governance(includes Applying Principles and Practice of CorporateGovernance in Budget Funded Agencies) Jun 1997

Management of Corporate Sponsorship Apr 1997

Telephone Call Centres Dec 1996

Telephone Call Centres Handbook Dec 1996

Paying Accounts Nov 1996

Performance Information Principles Nov 1996

Asset Management Jun 1996

Asset Management Handbook Jun 1996

Managing APS Staff Reductions Jun 1996

Australian National Audit Office - Management of Learning and … · 2016-03-08 · and HRD. 12...

Documents

Transcript of Australian National Audit Office - Management of Learning and … · 2016-03-08 · and HRD. 12...