Audit Scotland
-
Upload
tunggal-kris -
Category
Documents
-
view
220 -
download
0
Transcript of Audit Scotland
-
8/8/2019 Audit Scotland
1/13
Management Case Study
Building a Strategy Map to Drive
Performance Accountability and
Improvement: The Case of Audit Scotland
For more information please visit: www.ap-institute.com
http://www.ap-institute.com/http://www.ap-institute.com/ -
8/8/2019 Audit Scotland
2/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 2
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
Building a Strategy Map to Drive Performance Accountability and
Improvement: The Case of Audit Scotland
By
Bernard Marr* and James Creelman
Abstract: This case study describes the best-in-class implementation of a
performance management framework in form of a strategy map with supporting Key
Performance Questions and Key Performance Indicators. It shows how AuditScotland, the nations independent auditor of public bodies, was able to distill the
long stakeholder wish-list down to a simple and focused strategy map. Outlining the
strategic priorities on just one page using simple language has completely
transformed the way Audit Scotland is managing its performance to ensure external
accountability and internal performance improvements.
Version: 18th May 2010 *corresponding author
Bernard Marris the Chief Executive and Director of Research at the Advanced Performance Institute. E-mail:
James Creelman is a Fellow of the Advanced Performance Institute
The Advanced Performance Institute (API) is a world-leading independent research and advisory
organisation specialising in organisational performance. It provides expert knowledge, research, consulting
and training to performance orientated companies, governments and not-for-profit organisations across the
globe. For more reading material or information on how the API might be able to help your organisation please
visit:www.ap-institute.com
How to reference this case study:
Marr, B. and Creelman, J. (2009) Building a Strategy Map to Drive Performance Accountability and
Improvement: The Case of Audit Scotland, Management Case Study, The Advanced Performance Institute
(www.ap-institute.com).
mailto:[email protected]://www.ap-institute.com/http://www.ap-institute.com/http://www.ap-institute.com/mailto:[email protected] -
8/8/2019 Audit Scotland
3/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 3
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
Building a Strategy Map to Drive Performance Accountability and
Improvement: The Case of Audit Scotland
Introduction
Having amassed a wealth of useful data and
information regarding stakeholder priorities over the
years, Audit Scotland faced the not uncommon
challenge of distilling this down into something that
was focused and manageable.
To achieve this, Audit Scotland crafted a strategy
map that comprises just two core activities and nine
supporting performance enablers. Each of these,
which are collectively aligned to the organizations
vision, is supported by Key Performance Questions
(KPQs) and Key Performance Indicators (KPIs)
which together provide rich insights into how the
organization is progressing toward its strategic
goals.
The strategy map is designed to help Audit Scotland
make step-change improvements in performance
and to provide a more streamlined and focused
corporate plan.
An Impact framework is also being used to assess
and report the longer term benefits of Audit
Scotlands work, which will serve to provide a more
complete view of the organizations performance.
About Audit Scotland
Headquartered in Edinburgh, Audit Scotland was
created in 2000 to provide the Auditor General and
the Accounts Commission with the services thatthey need to check that public money is spent
properly, efficiently and effectively. With about 300
full-time equivalent employees and an operating
budget of 27.5 million, the organization arranges
the audits of about 200 public bodies, such as the
Scottish Government, 32 councils and 44 joint
boards and committees (including police and fire,
health boards and rescue services). Importantly,
Audit Scotland is independent of the bodies that it
audits.
Three types of audit
Audit Scotland conducts three types of audits:
1. An annual audit provides an independent
opinion of the public bodies financialarrangements and governance practices.
2. Best value audits provide an independent
opinion of an organizations outcomes focus as
well as the effectiveness of its corporate
performance management and use of
resources.
3. Performance audits provide an independent
assessment of key areas of parliamentary or
public interest, often spanning traditional sector
boundaries. Performance audit reports include
Scotland Public Finances,The role of Boards,
Efficient Justice and Public Sector Pension
Schemes.
Together, these three audit types provide a detailed
and systematic evaluation of the efficiency and
-
8/8/2019 Audit Scotland
4/13
-
8/8/2019 Audit Scotland
5/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 5
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
Stakeholder priorities
The priorities identified by the stakeholders groups,which have therefore become the key priorities of
Audit Scotland, are:
Deliver more streamlined audits in partnership
with other scrutiny bodies.
Maximize our contribution to the improvement of
public services.
Increase the impact of our work.
Become a centre of excellence for public audit.
Improve the transparency of our costs and
governance arrangements.
The priorities are based on the following issues that
stakeholders are grappling with, that emerged
through the consultation process:
increasing pressure on public spending
Increasing emphasis on more efficient working
Figure 1:Audit Scotlands Corporate Strategy Map (source: Audit Scotland Corporate Plan 2009 -12)
-
8/8/2019 Audit Scotland
6/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 6
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
The continuing need for high standards of
corporate governance, financial management
and control
The need to deliver more efficient and effectivescrutiny and accountability.
Creating the Corporate Strategy Map
The first step in creating the strategy map that
would deliver to the vision and stakeholder priorities
involved API conducting focused and individual
interviews with each member of the executive
management team and the assistant directors as
well as the organizations chair John Baillie and
another board member.
A core benefit of one-to-one interviews is that it
enabled the external facilitator to gain a clear
picture as to how individual senior managers viewed
the drivers of organizational strategic success. Any
differences were debated within the workshops
conducted to create the strategy map, with the
ultimate goal of reaching consensual agreement as
to the critical few drivers of success.
Strategy mapping workshops
Audit Scotlands management group created a draft
strategy map in two dedicated workshops. A key
role of the external facilitator was to ensure the
selection of truly strategic objectives, rather thangoals that are essentially operational.
A further role was to facilitate the creation of cross-
organizational objectives instead of ones that could
be siloed that are of relevance and interest to just
one part of the organization. Audit Scotlands
management team wanted to ensure that the
strategy map would galvanize the whole of the
organization in unison toward step change
performance improvement.
In-house consultationWide-ranging staff engagement was seen as crucial
for securing buy-in to the strategy map concept.
Through focus groups and discussions with small
groups of staff we gained feedback on what they
thought of the map, what they thought were its
strengths and weaknesses and where they saw
areas for improvement, recalls McGiffen. This
showed us how closely the management teams
views were aligned with those of their colleagues
and the staff feedback was used to inform the final
map.
According to McGiffen, the clarity and simplicity of
the map helped most staff to understand and talk
about what it means to them Within the group
discussions it became evident that colleagues could
see how it all fits together, what Audit Scotland is
trying to achieve, how it will be achieved and what
they must do in their own jobs to make it happen.
This, she adds, was largely due to the maps
simplicity and accessibility.
Ensuring the map is simple and focused
As an illustration of simplicity, the map comprises
just eleven elements: two core activities (which Audit Scotland calls objectives) and nine
performance enablers (which Audit Scotland calls
supporting activities). A prime driver of formulating
the strategy map was to distil and make
manageable a mass of data and information, Audit
Scotland rightly realized that there would be little
benefit in creating an elaborate and complex map.
Indeed the experience of other companies shows
-
8/8/2019 Audit Scotland
7/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 7
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
that complicated maps do not provide sufficient
focus onto the key drivers of success and rarely
survive over the longer term due to being too
unwieldy and unmanageable a fate Audit Scotlandwas determined to avoid.
Although simple, the map is far from simplistic.
Rather, each objective and supporting activity
serves as a doorway to a host of performance
management and measurement focal points
primarily captured through Key Performance
Questions (KPQs) and Key Performance Indicators
(KPIs) - that work together to provide an overall
picture of how well Audit Scotland is performing
and, ultimately, to steer the organization toward the
delivery of its vision and priorities (For more
information on KPQs and KPIs see the API white
paper What are KPQs and the API management
overview Key Performance Indicators1).
Although KPIs are commonly used withinorganizations, KPQs are substantially less so. So
as an explanation, a KPQ focuses on and highlights
what the organization needs to know in terms of
executing existing strategic objectives. KPQs
enable a full and focused discussion on how well
the organization is delivering to these objectives
and serve as an important bridge between
organizational goals and KPIs. Indeed KPQs are
used to provide a performance context to KPIs and
help to more effectively prioritize the indicators
chosen, and this was certainly true of Audit
Scotland.
1http://www.ap-institute.com/resources.htm
People in most organizations, and we are no
exception, have quite rightly developed strong
disciplines in KPIs as a way to monitor and improve
performance, says McGiffen. KPQs do not replaceKPIs but are an enhancement. KPQs help us to
capture a much richer set of information than cannot
be gained solely from performance data.
Today, KPQs are used to discuss performance at
many organizational levels within Audit Scotland
and Audit Scotland is refining and developing its
collection and reporting of data to address the
KPQs. A key initiative was a Performance
Management Festival held over two days to bring
focus and attention from across the business to the
performance reports.
We will now explore how the objectives/activities on
the map, KPQs and KPIs work together with Audit
Scotlands strategy management framework.
Two strategic objectives
As cited, Audit Scotlands strategy map includes just
two core activities (called objectives in their case),
these being: we will conduct excellent risk based
audits of the public sector and report them in
public, and we will systematically identify and
promote good practice to help public bodies to
improve.
Although both objectives are equally important, a
key message that emerged from the extensive
stakeholder consultation was that they were looking
to Audit Scotland to do more to help them to
improve performance. The KPQs to support the
improve objective (and note that KPQs and indeed
KPIs are still work in progress within Audit Scotland,
while they test which are the most powerful
http://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htm -
8/8/2019 Audit Scotland
8/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 8
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
questions and indicators to focus on) presently
include: to what extent has our work led to
improvement and to what extent are we identifying
and actively promoting good practice (or betterways of doing things) to help public bodies improve.
Note that the KPQs focus on both lagging (that is
how we have performed) and leading (how we will
improve performance) dimensions, enabling a fuller
overview of past, present and likely future
performance.
KPIs for these KPQs include impact reports (see
later in this case study for details of Audit Scotlands
impact framework); client feedback; number of
presentation at meetings, events and conferences
which include best practice guidance, and
performance audit reports with good practice
recommendations (see Figure 2 for examples).
Supporting Activities
How Audit Scotland achieves its objectives is
largely through the delivery of enabling objectives
(called supporting activities in their case). In
creating the map, the senior team strived to ensure
that there was a balance of focus on what matters
to stakeholders, what matters to staff and improving
some of the key business support processes to
support that work.
Stakeholder-facing performance improvement
For a stakeholder-facing improvement, consider the
activity: We will closely engage and communicate
with our key stakeholders, clients and partners, and
other scrutiny bodies.
Audit Scotland recognizes that it is only through
close engagement and communication that it can
effectively identify appropriate topics for its audit
work. Closely engaging and partnering with other
scrutiny bodies and improvement agencies also
helps to avoid duplication of effort and maximizesAudit Scotlands contribution to the improvement of
public services.
To support the improvement objective it is also
recognized that it must clearly communicate its
findings and recommendations to ensure that its
messages are heard and its recommendations are
followed up.
KPQs used for this activity are to what extent are
we keeping our stakeholders informed about our
programmes of work and their role/implications and
how well do we engage the people we work with.
KPIs include positive stakeholder feedback
(engagement levels, clarity about and acceptance of
roles), website statistics (downloads, click
throughs) and the number of shared riskassessment frameworks in place.
Internal process improvement
Step-change internal business process
improvements are largely captured by the
supporting activity: We will deliver our work and
manage our resources efficiently and effectively and
will be clear where we need to improve. We willhave excellent governance procedures and will
monitor and report on the impact of our work.
This activity sets out to ensure that Audit Scotland
has strong internal governance processes in place
and that it is deploying best practice public
management principles and procedures. To this
end, Audit Scotland carries out a programme of
-
8/8/2019 Audit Scotland
9/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 9
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
internal best value reviews to improve its efficiency
and effectiveness and manage its risks.
The KPQs for this activity include to what extenthave we effectively and efficiently managed our
resources to deliver our outputs and how effective
are our internal governance arrangements for
delivering our objectives. KPIs include: percentage
of audit reports delivered on-time and to budget,
reducing our carbon footprint by one per cent,
percentage of invoices paid within 30 days and
internal best value reports published.
Staff performance improvementMoving onto improving staff performance this is
captured by several supporting activities, including
We will maintain and develop our professional skills
and competencies.
As with any high-performing organization, Audit
Figure 2: Sample KPIs and Evidence of Performance (source: Audit Scotland Corporate Plan 2009-12)
-
8/8/2019 Audit Scotland
10/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 10
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
Scotland recognizes that it can only deliver high-
quality work if its staff have the right skills and
competencies to do their job well. To achieve this,
the organization strives to embrace an ethos ofcontinuous improvement and continues to invest in
the development its people. The organization has
recently revised its competency framework and has
embedded this into its Performance Development
System. Regular individual performance reviews are
used to assess and follow-up on training and
development needs.
The KPQs for this activity include to what extent are
our employees engaged and motivated and to
what extent do we have strong and effective
leadership in the organization. KPIs include staff
survey results, staff focus group feedback and the
development of 360 degree feedback (through
which employees are appraised by a range of
people, including their managers, colleagues and
direct reports).
Knowledge management
A particularly interesting supporting activity, in that it
clearly demonstrates how Audit Scotland is working
to improve its own internal capabilities in order to
help its stakeholders improve is We will gather,
organize and share knowledge and intelligence.
The organization recognizes that the basis of its
work is having the right knowledge and intelligence.
Therefore it is working diligently to ensure that the
right information is systematically gathered, that it is
organized in the most appropriate way, and shared
among staff and externally with stakeholders.
KPQs for this activity are to what extent do we hold
the information that we need ' to what extent are we
sharing our key findings internally, and to what
extent are we able to find and share the information
that we find. KPIs include focus group feedback,
intelligence reports, publication of technicalbulletins, number of briefing reports produced and
shared internally.
Creating Strategy Maps for Specific Activities
Audit Scotland is presently reviewing and
developing how it gathers, organizes and shares
knowledge and intelligence. This is leading to some
interesting applications of the strategy map concept,
as McGiffen explains. Our strategy map identifies
the key support activities where we need to focus
attention and commit resources, she says. What
we are doing now is creating strategy maps for
specific areas where we want to prioritize major
performance improvements, such as information
and knowledge management and shared learning.
She continues: Our stakeholders are increasingly
looking to Audit Scotland to share knowledge and
information quickly from the full range of our work
and advanced knowledge sharing and collaborative
capabilities. However, our technologies and
methodologies dont enable us to do what we need
to do to meet stakeholder requirements. We have
started to invest in this area and a strategy map
should prove a powerful mechanism for prioritizing
our investments against identified stakeholderneeds. Environmental performance is another area
for which a dedicated strategy map will be created.
Performance Reporting
With the strategy map now the core tool for driving
performance improvement, Audit Scotland ensures
that it is closely managed on a day-to-day basis,
and this is the responsibility of McGiffen and a small
-
8/8/2019 Audit Scotland
11/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 11
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
team from corporate reporting that is skilled in data
analysis. The team reports performance to the
management team and board on a quarterly basis,
who particularly look to see how objectives andactivities are improving and to focus on areas that
are proving challenging and where corrective action
might be required.
From a reporting standpoint, the strategy map and
related KPIs have also enabled McGiffen and her
team to cut the corporate plan down from a lengthy
document to something concise and focused and
that is more accessible to external stakeholders and
internal staff respectively.
Performance Management Festival
One innovative approach to engage people in
performance reporting was to stage a Performance
Management Festival. In late 2009 Audit Scotland
invited key staff for a two-day festival of
performance. The aim was to bring together KPI
owners, performance analysts, and directors and
board members to agree performance reporting
formats. Using an off-site location with music, guest
speakers and performances this innovative event,
facilitated by the API, was designed to make the
topic of measurement and performance reporting
interesting, engaging and fun.
During the two-day event staff re-designed
performance reports using the latest techniques of
information communication and data visualization.
Within these reports, headlines and narratives
(essentially a performance story) were used to
capture the key messages, using ideas and
graphic/textual formatting from magazines and
newspapers. These new performance reports were
then trialed and improved during a staged mock
performance review meeting with the directors.
Impact FrameworkPerformance improvement is also being galvanized
through the deployment of an impact framework to
assess and report on the longer term impact of
Audit Scotlands work - in particular its performance
audits - and to capture evidence of that impact. The
organization has identified four areas where it
expects its work to have an impact and contribute to
improvements:
Assurance and accountability
Planning and management
Economy and efficiency
Effectiveness and quality.
The impact framework describes the types of
measurable change expected to result from Audit
Scotlands work within the different categories ofimpact and includes examples of how individual
reports may contribute to these changes.
The impact framework is expected to dovetail well
with the strategy map in providing a complete view
of how Audit Scotland is performing now and is
likely to perform in the future.
Conclusion
In conclusion, McGiffen summarizes the role that
the strategy map has played in improving the
performance of Audit Scotland. From the beginning
we knew that we wanted our Corporate Plan to be
simpler and more relevant and meaningful for
colleagues. she says, From an external
perspective we also knew that as an organization
-
8/8/2019 Audit Scotland
12/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 12
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
we wanted a framework that would help our
stakeholders understand what we do and what we
have to be best at.
The strategy map is helping us to meet our external
and internal goals, she continues. But in the final
analysis, it is perhaps most important that the map
has better positioned Audit Scotland to be able to
demonstrate to stakeholders and clients that we are
doing what we promised that we would do.
Endnotes, References & Further Reading
Marr, Bernard (2010), The Intelligent Company: Five Steps to Success with Evidence-Based
Management, Wiley, Oxford.
Marr, Bernard (2009), Managing and Delivering Performance: How Government, Public Sector
and Not-for-profit Organizations can Measure and Manage what Really Matters, Butterworth-
Heinemann, Oxford.
Marr, Bernard (2006), Strategic Performance Management, Butterworth-Heinemann, Oxford
www.ap-institute.comFor more case studies, reports andarticles visit www.ap-institute.com
Working with the Advanced Performance Institute to develop a performance framework in form of astrategic map, KPQs and KPIs has exceeded my expectations. It has helped us clarify our strategic
objectives quickly and easilyin a way we couldnt have done on our own. The strategy map has
completely transformed the way we now communicate about our goals and deliver our strategy and it
has been an extremely powerful tool to engage everybody in managing and delivering performance.
Diane McGiffen, Director of Corporate Services, Audit Scotland
-
8/8/2019 Audit Scotland
13/13
2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 13
Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study
Highlights from the API resources library:
Our Resource and Research Library offers a selection of relevant downloads and links to books, articles and
case studies. These have been selected as useful information sources for further reading and to illustrate best
practice and leading thinking.
Some key ideas explained in simple terms:
What isPerformance Management?
What is theBalanced Scorecard?
What is aKey Performance Indicator?
To read more just click:http://www.ap-institute.com/resources.htm
http://www.ap-institute.com/Performance%20Management.htmlhttp://www.ap-institute.com/Performance%20Management.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Performance%20Management.html