Guidelines on Risk Management practices among statistical ... · Guidelines on Risk Management...

Fabrizio Rotundi

WORKSHOP ON RISK MANAGEMENT

SYSTEMS AND PRACTICES

Genève, 25-26 April 2016

Guidelines on Risk Management practices

among statistical organisations

Fabrizio Rotundi

Nov ‘14

Dec ‘14

Mar ‘15

May ‘15

Sep ‘15

Template

Framework

Surveyoutput

Benchmark analysis

Best practice

Guidelines

Nov ‘15

Road map proposal for developing Risk Management among statistical organisations

Fabrizio Rotundi

Template’s Reading-Key Criteria

The Reading-Key Criteria are the lens to analyze the experiences through

2. The “Uncertainty experts” dimension refers to the actors, roles, organizational units or

structures to which the organization assigns the responsibility to conceptualize and

control uncertainty and, consequently, the responsibility for Risk Management.

3. The “Technologies” dimension reveals the extent to which the Risk Management

System becomes embedded or decupled in the organization and refers to the practices,

procedures and tools adopted by an organization to implement Risk Management.

1. The “Risk rationalities” dimension

reflects the main purpose which any

organization grounds its own risk

strategy on (i.e. compliance,

performance, company value, etc.).

Fabrizio Rotundi

Reading-key Criteria

Project’s subdivisions and links between output’s variables

3 READING-KEY CRITERIA

To analyse the experiences

2 PARAMETERS

To elaborate the questionnaire

SEVERAL FACTORS describing each parameter

To set the questions

B. Reapplication

B. Reapplication

Fabrizio Rotundi

Method: the procedure

The methodological approach envisaged for data analysis is involving a multi-phase procedure:

1

• Carrying out the surveys to detect how many Countries can be profitably analyzed and which of them:

Question n. 4 has been used as a filter (“Is there a strategy to effectively manage uncertainties and related threats and

opportunities in your organizations?”)

2

• Leading the Items (representing consistent sets of significant features for analysis, i.e. “Training & Communication”) back

to the three Reading-keys (Risk rationalities, Uncertainty experts, Technologies) used in the Survey design phase.

3

• Defining Item parameters and making up descriptors that allow the former to be allocated among the three levels Low-

Medium-High.

4

• Allocating descriptors and Countries within a conceptual chart crossing the dimension which shows the different levels

of DEVELOPMENT (Low-Medium-High).

5

• Detecting the practices that can actually be implemented through evaluating their Reapplication, based on the

«Adaptability» criteria, that is the practice ability to be transferred to other organizations without needing any specific

actions or tools.

6

• Identifying the Best Practices (both Country and process ones) through: i. their Reapplication level; ii. their development

level; iii. their actual use, that is, the recurrence of isolated strategic behaviors throughout the sample.

7

• Analyzing the practice internal consistency by bridging back the answers provided by the Countries.

First Survey (May 2015) - Statistical analysis

Involved Institutes and Organizations 64

Respondents 34

Overall Redemption 53,1%

Anonymous 5

Double responses 2

Total of valid responses 29

Redemption of valid responses 45,3%

Countries recognizable 27

Redemption of countries recognizable 42,2%

1. Objective: The survey aims at collecting information on

the RM approaches that can be useful to establish a

suitable reference for the NSIs interested in

implementing RM in the future

2. Structure: The survey envisages four sections composed

by a total of 53 questions

3. Target-audience: NSOs members of UNECE and other

statistical organizations

Fabrizio Rotundi

Survey on RM Practices (May 2015): Roles & Accountabilities

Respondent 1 (EU): NETHERLANDS“Yes”

Respondent 2 (EU): POLAND“Yes, by a draft Regulation of the President”

Respondent 3 (Non-EU): AUSTRALIA“Yes. Operation risks: project managers are owners. Corporate risk profile: senior managers are owners”

Fabrizio Rotundi

Respondent 7 (EU) Romania:

“The risk management process is mainly influenced by

the national norms and regulations. In our approach we

look to integrate it with the specificity of statistical

production process. One direction for more efficient risk

management is to integrate it in GAMSO”.

Respondent 9 (EU) Ireland:

“Organizational, Financial, Reputational,

Compliance, Legal and Regulatory, Interagency

Dependance, Loss of Personnel, Morale and

Change Management

Survey on RM Practices (May 2015): Risk Management and Modernization

Fabrizio Rotundi

Survey (phase 2): Defining Item parameters and Descriptors

Items represent consistent sets of significant features for analysis (i.e. “Risk Framework”) complying with the

3 Reading-keys (Risk rationalities, Uncertainty experts, Technologies) identified in the Survey design phase

Parameters and Descriptors allow allocation of the countries among the levels Low-Medium-High.

* Internal/external context, Risk Framework and Process

** Actors, roles, structures

*** Practices, Procedures & Tools

Low Medium High

Attitude towards

uncertainties

Either preventative or ex-post

control system

Both preventative and ex-post

control system


control system involving a specific

audit unit

Approach to RM Previous organizational practiceInternational standards (ISO, COSO,

ecc)Customized model

UN

CE

RT

AIN

TY

EX

PE

RT

S

Organizatio

nal chart

RM function in the

organization chart

Neither a RM Unit nor a board entity

deciding on RM exists

Either a RM Unit (included in the

Organization chart) or a board

entity deciding on RM exists

Both a RM Unit (included in the

Organization chart) and a board

entity deciding on RM exist

TE

CH

NO

LOG

IES

*

Human

Resources

Human resource

adequacy

HR are either not suitable or not yet

evaluatedHR are quite suitable HR are suitable

Risk

FrameworkRIS

K

RA

TIO

NA

LIT

IES

*

ITEMSREADIN

G KEYS

DEVELOPMENTITEM PARAMETERS

Fabrizio Rotundi

Survey (phase 2): Conceptual Chart – Examples

All countries’ practices have

been allocated along the

Conceptual chart, based on

the descriptors which show

the different level of Risk

Management Development

among the statistical

organizations (Low – Medium

– High)

Low Medium High

Either preventative or ex-post

control system


control system


control system involving a specific

audit unit

NETHERLANDS FINLAND IRELAND

AUSTRALIA ITALY LITHUANIA

SLOVENIA NORWAY ESTONIA

UK CROATIA NEW ZEALAND

ROMANIA AUSTRIA

MEXICO CANADA

SOUTH AFRICA SWEDEN

REPUBLIC OF ARMENIA POLAND

SLOVAKIA

ICELAND

Either management or non-

management staff

Both management and non-

management staffEven external stakeholders

ICELAND ROMANIA SOUTH AFRICA

NETHERLANDS MEXICO CANADA

IRELAND NEW ZEALAND

LITHUANIA SLOVAKIA

AUSTRALIA FINLAND

ESTONIA NORWAY

POLAND ITALY

UK

SLOVENIA

SWEDEN

AUSTRIA

CROATIA

Specific but not structured trainingSpecific training program for

management (at any level)

Specific training program for all

personnel running RM matters

NETHERLANDS ICELAND ROMANIA

SWEDEN REPUBLIC OF ARMENIA MEXICO

NEW ZEALAND

SLOVAKIA

IRELAND

LITHUANIA

AUSTRALIA

CANADA

ESTONIA

UK

POLAND

AUSTRIA

ITALY

CROATIA

READING

KEYSITEMS

ITEM

PARAMETERS

DEVELOPMENT

Risk

Framework

Attitude

towards

uncertainties

RIS

K R

AT

ION

ALI

TIE

S

Internal and

external

stakeholders

Stakeholders

mostly involved

in RM processU

NC

ER

TA

INT

Y E

XP

ER

TS

Training &

communicationTraining system

TE

CH

NO

LOG

IES

Fabrizio Rotundi

In-Depth Survey (September 2015): Roles & Accountabilities

Respondent 2 (EU) ROMANIA:

“According to the procedure’s steps the roles,

accountabilities and tasks are the following….

- Any staff member : to fulfill the Risk Alert Form, etc…

- For Risk Officer: to collect the Risk Alert Form, etc…

- For Rm Team: to validate/invalidate the closing

solutions, etc …..

Respondent 3 (non-EU) AUSTRALIA:

“The Risk Management Framework outlines that the

head of statistical division will be the single point of

accountability for managing statistical risk but below

that managing statistical risk will be a shared

responsibility in recognition that there are many

sections that can contribute to managing this key risk

to the ABS.”

Respondent 1 (EU) THE

NETHERLANDS:

“A set of high level objectives is

identified on strategic, finance,

operational and compliance level.

Actions are identified to meet the

objectives and assigned to the

heads of divisions…”

Fabrizio Rotundi

In-Depth Surveys on Risk Management, Change management and

Modernization practices

Respondent 8 (Non-EU) Australia: “There are qualitative measures for assuring

quality for most statistical output. The NSI has

expanded its focus on managing statistical risk to

include a more holistic assessment of risk in

statistical areas that can affect data quality as

well as managing stakeholder relationships, the

impact of change programs and workforce

capability.”

Respondent 7 (EU) Romania: “Many projects related to GSBPM, GAMSO,

QAF and risk management are in progress or

finalized either under UNECE or Eurostat

initiatives. Their value is undisputable, but,

some additional actions would be required,

mainly in assisting Statistical offices to

implement results.”

Respondent 9 (Non-EU) Canada:“A management tool for all current projects is used

by project managers to manage their changes,

issues, and risks throughout the life cycle of their

project. The implementation of the corporate

tracking tool provides a consistent approach to

management for all projects and establish a

centralized service.”

Fabrizio Rotundi

The points highlighted by the general trend are:

• corporate risks* (strategic, cross-cutting, most common, ... )

are lower than operational ones;

• the absolute number of corporate risks varies depending on

the risk policy (top-down vs bottom-up approach)

Second Survey - Quantitative Results

In terms of percentage of total, statistical risks are the majority,

followed by organizational risks.

Other risks arisen are: financial, ITC, reputational, security ones.

Approximately one third of respondent countries shows

a not-negligible pervasiveness of the Risk Management

process within the organizational structures.

The high percentage of respondent countries with trained

specialists underlines an organizational culture that, as

regards Risk Management, is under a significant and

ongoing development.

Fabrizio Rotundi

Towards the Guidelines

� Selection of best practicesidentified by in-depth analysis

� Consistency analysis byresponses

SECTION 2. RISK MANAGEMENT PROCESS

1. Internal/External Communication & consultation

1.1 Internal Communication & Consultation

1.2 External Communication & Consultation

2. Context analysis

2.1 Establishing the Internal/External context

2.2 Process Mapping

3. Risk Identification

3.1 Top-down vs Bottom-up approach

3.2 Risk hierarchy

3.3 Risk Identification techniques

4. Risk Assessment

4.1 Risk Analysis & Measurement

4.2 Risk Weighting

4.3 Roles & accountabilities related to the assessment phase

5. Risk treatment

5.1 Risk treatment priorities

5.2 Roles & accountabilities related to the treatment phase

6. Monitoring & Reporting

6.1 Monitoring of treatment actions

6.2 Establishing internal/external reporting mechanisms

I. Internal reporting

II. External reporting

Fabrizio Rotundi

The Guidelines

The draft consists of two sections, whose index complies with Risk

Management standard ISO31000/2009:

� Section 1 investigates the Risk Management system;

� Section 2 focuses on the Risk management process.

The Sections 1 and 2 include Question Mark boxes that consistently

report some answers to the questions contained in the first and the

second survey.

� The Annex which include Focus and Case studies to show a practical approach to the

different elements of the Risk Management system described in the Guidelines;

� The References, concerning the main sources of the Guidelines;

� The Glossary, with the definition of the main relevant terms of the Guidelines.

The Guidelines also comprise:

Fabrizio Rotundi

The Guidelines: The Risk Management Framework

Fabrizio Rotundi

A. The Chief Statistician is responsible for ensuring an effective RM.

B. The Risk Committee is responsible for: …..

C. The Risk Manager is responsible for: …

GUIDELINES: Risk philosophy (as feature of risk strategy) and risk appetite (as feature of risk

policy) should be always kept aligned. Risk management design should be mostly contributed by

Top management with the assistance of middle/low management and technical staff.

PRACTICE: “The risk appetite (level of exposure which is deemed tolerable and justifiable) will only tolerate High

or Extreme risks when treatment measures are unable to reduce the level of inherent risk to an acceptable level.”

Establishing risk management policy and defining Accountabilities

The organization should ensure that there is accountability,

authority and appropriate competence for managing risk.

“The leadership of governance system is provided by the Executive Management Board […]. Directors, Assistant

Directors, Chiefs and Unit Heads (Divisions) are owners of Operational risk and Project risk registers. All Other Staff

are responsible for identification, documentation and management of operational and project risks.“

ISO 31000:09: The risk management policy should clearly state the

organization's objectives for, and commitment to, risk management.

Fabrizio Rotundi

Risk Management Framework – Roles and Accountabilities

1) All staff are responsible for an effective management of risks including identification of any potential risks;

2) Risk management is driven by the organizational units;

3) An Office is dedicated to the coordination of the

management process and risk analysis,

"impartial" with respect to other structures,

supporting the highest level of decision making;

4) The Risk Manager is responsible for:

collaborating with Top Management both in

identifying high risk areas related to strategic

and business processes and in planning

treatments to mitigate corporate risks;

5) The Risk Committee defines the Risk

Management policy; it is coordinated by the

Risk Manager and composed by the top

managers operating in the areas most risky;

6) Chief Statisticians and Governing body define

the strategies based on the information coming

from the RM System;

7) The Internal Auditing is responsible for reporting to the Governance on the adequacy of the RM process

and the compliance of the mitigating actions.

Fabrizio Rotundi

PRACTICE: “Better quality management practices has been endeavored through the

development and use of the risk mitigation strategy known as quality gates.”

Integration into organizational processes

ISO 31000:09: Risk Management should be embedded in all the

organization’s practices and processes in a way that is relevant,

effective and efficient

GUIDELINES: Given that statistical risks (i.e. the possibility that one or more of the

production process components fail to meet the quality standard) are unavoidably

managed at all levels (strategic, operational and project ones), it is worth noting that

even when they are managed separately they should eventually be integrated into an

organizational risk framework.

“Object Oriented Quality and Risk Management (OQRM) model is a quality framework developed

in the field of official statistics in order to improve compliance with the European Code of Practice

and deal with quality standards of statistical output.”

Fabrizio Rotundi

Risk Management Framework Integrated with Quality of Statistics

Statistical risks are events that potentially could impact on production

processes and/or integrity and quality of statistical data. They concern

statistics that are not considered by users as fit for purpose which

includes, but is not limited to, time series that are not coherent.

At operational level, statistical risks can be treated by quality management because Quality and Risk

Management are strictly connected:

� Quality management assesses if the original requirements (ISO 9001:2015) are met (review, audit,

etc.) or not. If not, corrective actions are implemented.

� Risk management identifies threats that can effect objectives. If the risk level is too high, mitigating

measures are implemented.

They can occur due to: Planned changes to systems, processes or methods; Changes in the resources

availability; Changes in the data source availability or quality; Changes in organizational issues.

Statistical risks can be identified separately, but then they should be

integrated into the organizational risk framework.

Fabrizio Rotundi

PRACTICE: “Yearly training on Risk Management and Internal Control System with an external expert is organized. A

presentation of the Risk Management system is provided to all new staff members within Statistics Austria's general training

programme (half-yearly)”

GUIDELINES: It is advisable to start training with a program devoted to

managers and employees assigned to run risk management matters at different

levels; it would be best if kick-off training activity focuses first on higher-risk

areas. It is also important to carry out training initiatives regularly, in

accordance with risk management system development, as well as concurrently

with significant organizational changes.

Human Resources and Training

ISO 31000:09: The organization should allocate appropriate

resources for risk management. Consideration should be given also

to training programmes.

“The Risk Management Training program involves General control system training, quality management issues, internal

auditing of QMS; up to 10% of staff have been trained on Risk Management so far.”

“A specific training program on risk management issues has been envisaged and addressed to all employees”.

Fabrizio Rotundi

The Guidelines: The Risk Management Process

Fabrizio Rotundi

PRACTICE: “Risks are identified by accountable managers and then gathered in strategic categories (corporate risks), in

order to be assessed, treated and monitored, based on: Monitoring risk treatments through specific indicators;

Organizational sustainability; Cross-cutting treatments; priority areas”.

GUIDELINES:

The coordination of Risk Management process phases is centralized […].

Three kinds of approach can be followed in identifying risks: Top-Down-approach;

Bottom-Up approach; Mixed approach.

The Risk Management framework includes a hierarchy of risks: Enterprise Risks,;

Operational Risks and Project Risks.

The organization should identify sources of risks, areas of

impacts, events (including changes in circumstances), their causes

and their potential impacts

ISO 31000:09: The organization should define criteria to be used

to evaluate the significance of risk

“The process starting by engaging all Directors to respond to a risk questionnaire to identify the top three/five risks from a

divisional program perspective. The risk registers are reviewed and approved to ensure [..] importance of the risks identified at

the divisional or program level”.

Defining risk criteria and Risk Identification approach

“Risk identification, analysis and management are practices aim at anticipating and removing the obstacles that may prevent

the achievement of strategic objectives. 3 levels of risks [..] have been identified: 1. Risks associated to the ESS Vision 2020

[..]; 2. Portfolio management risks; 3. Project related risks”.

Fabrizio Rotundi

Risk Management approaches and Risk Hierarchy

1. Enterprise Risks, strategic and significantly impacting on the organisation,

assessed and treated by the Executive Managers: Regulatory and compliance

risks, global financial shocks, aging consumers and workforce, emerging markets.

The hierarchy of risks reflects the selected RM approach and is related to the different levels:

3. Project Risks, impacting on the project objectives and outcomes, managed by the project risk manager:

Scope poorly defined; Resources not available; Quality requirements not clearly specified.

2. Operational Risks, impacting on a program's objectives and/or outcomes, assessed and managed by the

line managers: Inappropriate skills mix; budget cuts; poor quality outputs.

Three different approaches can be followed in managing risks:

A. Top-Down-approach: the decision making process is centralized at a

government body-level. a) Full top-down: the business units’ risks are listed at

department level; b) Prevailing top-down: the corporate risk register comes

from a detailed operational risk register.

B. Bottom-Up approach: the decision making process is located at management level and risks are

identified by any staff member performing daily work.

C. Mixed approach: the board entity states the criteria (top-down) by which the heads of unit identify

and manage risks (bottom-up).

Fabrizio Rotundi

GUIDELINES: The purpose of risk weighting is to ensure that use of resources will be focused on

the most important risks. A common approach to prioritize risks is to divide them into three bands:

• Upper, where the level of risk is regarded as intolerable whatever benefits the activity may

bring, and risk treatment is essential whatever its costs;

• Middle, where costs and benefits are taken into account and opportunities balanced against

potential consequences;

• Lower, where the level of risk is regarded as negligible that no risk treatment measures are needed.

Risk evaluation and weighting

ISO 31000:09: The purpose of risk evaluation is to assist in making

decisions, based on the outcomes of risk analysis, about which risks

need treatment and the priority for treatment implementation

PRACTICE: “The risk management matrix is a tool developed in for identifying, analyzing, evaluating and treating risks. This

tool allow to incorporate process data, participants in this activity and shows preloaded content to facilitate their operation”.

“The risk assessments by managers are also based on the risk tolerance model, […] applied sequentially to identify risks

that were deemed appropriate for potential corporate consideration (the top 6).”

“Risk appetite will only tolerate High or Extreme risks when treatment measures are unable to reduce the level of inherent

risk to an acceptable level. Low or Moderate risks will be managed within the specific area and/or routine procedures. All

Treatment measures are selected by considering the cost vs benefits”.

Fabrizio Rotundi

Risk treatment and monitoring

ISO 31000:09: The information provided in risk treatment plans should

include also those who are accountable for approving the plan and those

responsible for implementing the plan.

Both monitoring and review should be a planned part of the risk

management process and involve regular checking or surveillance.

Responsibilities for monitoring and review should be clearly defined.

“Directors/division chiefs (Risk owners) propose response actions validated by the Risk Manager. Governance select the actions

after defining their significance on a priority basis (risk strategic area, risk value, feasibility) and then entrust them to the executives.

GUIDELINES:

Responsibilities related to the treatment phase should be clearly assigned specifying who is

accountable for the management of particular risks or categories of risk, for implementing

treatment strategies and for the maintenance of risk controls.

PRACTICE: “Risk treatment [..] is assigned to managers and followed up (annually or bi-annually by the board of

directors). [..]. The treatment is assigned to person responsible for implementing the treatment as a part of normal

operations or if that is not possible a separate implementation plan is to be prepared”

“Risks and treatments are included in the regular follow up of operations after each 4 month period with focus on

effectiveness and deviations from plan”.

The overall responsibility for monitoring and review activities relies on the board and top

management. Operational risks are monitored at business unit level, project risks are monitored

within the Project Management system, and corporate risks are monitored by Senior managers.

Fabrizio Rotundi

Annex, References and Glossary of the Guidelines

� The Annex aims at: highlighting the massive information obtained and providing a

more practical approach to the different domains of Risk Management. It consists of

two sections, Risk Framework and Risk Process, showing two categories of examples:

� The References reports the main sources of the Guidelines, i.e. Research

Investigation, Ad hoc Analysis, documentation provided by the Countries

involved, National and International Standards, Models and Guidelines,

ISO, Academic Sources, papers and handbooks.

� The Glossary includes the definition of the main relevant terms of the

Guidelines, arising from the countries’ practices and the international

standards, i.e. “The ISO Guide 73:2009. Risk management Vocabulary”.

- Focus points on Risk Management core topics, in order to share practices, coming

from the NSOs, able to substantiate "theoretical" information;

- Case-studies, shortly reporting some NSOs' significant experiences on particular features of the

Risk Management systems in order to share the know-how gained from implementing Risk

Management within the different organizational contexts and highlight any element in common

among the different experiences.

Fabrizio Rotundi

Sharing, publishing and disseminating the Guidelines

Workshop on Risk

Management - 04/16

Workshop

HRMT - 09/16

Workshop MCs

11/16

Publication and

Dissemination

Sharing for comments,

suggestions, …

Eventual proposals for cooperation or network

projects under the coordination of UNECE/HLG

Sharing for comments,

suggestions, …

Fabrizio Rotundi

GAMSO 1.0

Integrated Common Framework

GSBPM 5.0

Manage business & performance describes how

the organization run its business, including

agreed changes, in order to achieve planned

outputs and outcomes.

It encompasses:

• Manage business performance

• Manage change and risk

• Manage legislation & compliance

The GSBPM recognises several over-arching

processes that apply throughout the production

phases and across statistical business processes,

including Quality and Risk management, to

investigate on financial and social justifications,

initial and following risks, cost-benefit,

information and the selected solutions planned for

proceeding in statistical production processes.

Fabrizio Rotundi

Fabrizio ROTUNDI

[email protected]

[email protected]

Guidelines on Risk Management practices among statistical ... · Guidelines on Risk Management...

Documents

Transcript of Guidelines on Risk Management practices among statistical ... · Guidelines on Risk Management...