QA/QC Plan and Guidance on Uncertainty Management for ... · involved in conceptualizing,...

QA/QC Plan and Guidance on Uncertainty Management for National Greenhouse Gas Inventory

September, 2015

of 48

Ghana’s GHG QC/QC Plan and guidance on uncertainty management

of 48

Table of Content

1.1 What is this plan about? 5

1.2 How to use this QA/QC Plan 5

1.3 Structure of the QA/QC Plan 5

1.4 Who are the target users 6

1.5 What this plan does not cover 6

1.6 How the QA/QC plan was prepared (approaches) 6

2.1 National system for GHG Inventory 7

2.2 Reforms in the national system 8

2.3 National inventory cycle 8

2.4 Current QA/QC practices and gaps in them 9

3.1 What is the QA/QC plan set to do? 11

3.2 What can the QA/QC plan do? 11

3.3 What are the key component of Ghana’s QA/QC plan? 12

3.4 Major QA/QC Principles that will anchor the plan? 12

3.5 Key components of the QA/QC Plan 14

4.1 QA/QC ROAD MAP 36

5.1 A Review of current approach for Uncertainty assessment in the GHG Inventory 39

5.2 Practical steps to assess data quality 41

5.3 How to report on progressive improvements on uncertainty of AD and EF 44

of 48

Acronym

AD - Activity Data

AFOLU - Agriculture, Forestry, and Other Land Uses

BUR - Biennial Update Report

DSUG - Department of Statistics, University of Ghana

DVLA - Driver Vehicle and Licensing Authority

EF - Emission Factors

EPA - Environmental Protection Agency

EU - European Union

FEoV - Facilitative Exchange of Views

GGHG - Ghana’s Greenhouse Gas Inventory

GHG - Greenhouse Gas

GHGIG - GHG Inventory Guidelines

GNGHGI - Guidelines for National Greenhouse Gas Inventories

GPG - Good Practice Guidance

GPGUM - Good Practice Guidance and Uncertainty Management

IAS - International Accounting Standards

ICA - International Consultation and Analysis

IPCC - Intergovernmental Panel on Climate Change

ISO - International Organization for Standardization

LECBP - Low Emissions Capacity Building Project

MESTI - Ministry of Environment, Science, Technology, and Innovation

MMDAs - Ministries, Department and Agencies

MOU - Memorandum of Understanding

NC - National Communication

NGHGIR - National Greenhouse Gas Inventory Report

NSGHGI - National System for GHG Inventory

PDCA - Plan-DO-Check and Act

QA - Quality Assurance

QA/QC - Quality assurance and quality control

QAQCP - Quality Assurance and Quality Control Plan

QC - Quality Control

SBI - Sustainable Business Initiative

SOP - Standard Operating Procedure

SR - Summary Report

TCCCA - Transparency, Consistency, Comparability, Completeness, and

Accuracy

TOR - Terms Of Reference

TTE - Technical Team of Expert

UNDP - United Nations Development Program

UNFCCC - United Nations Framework Convention on Climate Change

VALCO - Volta Aluminum Company

of 48

Acknowledgement

The UNDP Low Emission Capacity Building (LECB) Programme is a country-driven

initiative that promotes essential cooperation between relevant institutions, engaging

the public sector and industry in a concerted effort to design and implement approaches

to low emission development that are consistent with national development priorities.

National counterparts are supported to strengthen technical and institutional capacities

to identify and formulate Nationally Appropriate Mitigation Actions (NAMAs) and Low

Emission Development Strategies (LEDS) in the public and private sectors, and to

strengthen the underlying greenhouse gas inventory management and Measurement,

Reporting and Verification (MRV) systems.

The LECB Programme runs through 2016 and is active in 25 countries: Argentina,

Bhutan, Chile, China, Colombia, Costa Rica, the Democratic Republic of Congo (DRC),

Ecuador, Egypt, Ghana, Indonesia, Kenya, Lebanon, Malaysia, Mexico, Moldova,

Morocco, Peru, Philippines, Tanzania, Thailand, Trinidad and Tobago, Uganda, Vietnam

and Zambia.

The programme is supported through generous contributions from the European

Commission, the German Federal Ministry for the Environment, Nature Conservation,

Building and Nuclear Safety (BMUB), and the Government of Australia.

More information can be found at www.lowemissiondevelopment.org

This product was developed under the LECB Programme, with generous funding from

the European Commission (EC), the German Federal Ministry for the Environment,

Nature Conservation, Building and Nuclear Safety (BMUB), and the Australian

Government.

http://www.lowemissiondevelopment.org/

of 48

1. Things to know this plan

1.1 What is this plan about? The demand for high quality national GHG inventories is increasingly getting higher. For Ghana to vouch the credibility of its inventories, the integrity of the methodologies used, the completeness of reporting, and the procedures for compilation of data must meet the highest standard of transparency. The preparation of this QA/QC plan is one of the major steps Ghana is taking to reform the way the GHG inventory is done. This plan is designed to provide a step-by-step instructions to guide the inventory team on the planning and implementing QA/QC activities. The plan is simply about structuring the QA/QC activities in the inventory and communicate who will be responsible for which task at a specific time in the inventory cycle. In addition, this plan will seek to provide guidance on how to improve to report on uncertainties and steps that the inventory compilers have to commit in the process of shifting to a higher tier uncertainty assessment.

1.2 How to use this QA/QC Plan Users of this material must note that this plan is not meant to replace the guidance the IPCC 2000 Good Practice Guidance and 2006 guidelines provide on QA/QC and uncertainty management. It is rather meant to translate the principles and good guidance in practical instructions for the inventory team in Ghana. In so doing, the guide has been written in less technical language and much as possible use real life practical examples to illustrate the current situation in the inventory. As much as possible, recommendations that have been made in the various sections are as a result of the analysis of gaps in the current QA/QC practices. There are portions of the plan where tables have been used demonstrate how to go by a particular QA/QC.

1.3 Structure of the QA/QC Plan The QA/QC plan is divided into three main sections. Section 1 provides background information on Ghana’s national system for GHG inventory. The background information covers the description of the structure of the national system, reforms that have been introduced during the last inventory, and the gaps in the current QA/QC practices. Based on the understanding of the gaps, a more structured QA/QC activities with responsibilities and timelines is provided in section 2. Sub-section 1 is focused on general and sector/category specific QC procedures whereas the sub-section 2 is dedicated to the guidance on QA. Final sub-section 3 is on road map of implementation of the QA/QC plan. Section 3 of the plan gives the key highlights on the weakness on the current uncertainty assessment practices in the inventory. And on the weakness found, suggested practical ways to streamline the current uncertainty assessment practices as well as steps that need to be taken before a full-tier 1 uncertainty assessment can be adopted for implementation.

of 48

1.4 Who are the target users The target users of this QAQC Plan are people who are involved in the inventory development, planning, organization and compilation process as well as anyone else seeking the background on how Ghana’s greenhouse gas Inventory program functions. This document will also be of assistance to other countries and interested practitioners involved in conceptualizing, designing, or building a comprehensive inventory system.

1.5 What this plan does not cover Although the plan has been built on the broad principle of GHG accounting provided in the IPCC guidance, it cannot be substituted for any national or sub-national GHG accounting QA/QC standards. It does not specifically provide QA/QC guidance on sub-national GHG accounting; corporate or city level or project GHG since the boundaries at this level may be unique, localized and high heterogeneous. Therefore, it is not recommended for sub-national accounting practitioners to solely depend on this guide but use it as reference materials to complement other standards.

1.6 How the QA/QC plan was prepared (approaches) In line with the objectives outlined in the TOR, the consultants approached the assignment in three main stages. The first stage was a comprehensive review of national and international literature on GHG inventory guidelines on quality assurance and quality control plan (QA/QC) and a tier 1 uncertainty assessment. IPCC GHG Inventory guidelines, US EPA national inventory template and other relevant guidelines/standards in other countries were extensively reviewed to understand the basic requirements and procedures for developing QA/QC and steps for undertaking uncertainty assessment. In the second phase, the consultants reviewed the Ghana’s 2014 National Greenhouse Gas Inventory Report to be submitted to the UNFCCC. The aim was to understand the QA/QC procedures in place in the national inventory, the methodologies used, the challenges and gaps in the current QA/QC practices. The third stage which is the development of this draft QA/QC Plan and Tier 1 uncertainty assessment checklist was the outcome of literature review of IPCC and other international guidelines and the review of Ghana’s National Greenhouse Gas Inventory Report 2014. The final stage of this process will be a stakeholder consultative workshop to solicit the inputs of stakeholders for integration into the final QA/QC Plan.

of 48

2. Brief description of Ghana’s national system for GHG Inventories

2.1 National system for GHG Inventory For every country to produce good quality national GHG inventory, a functional national system must be put in place. The system must have functional components that work together to efficiently produce a national inventory on timely basis that is of high quality. According to NIR, 2014, Ghana’s national system for GHG inventory include the following components: institutional arrangement and their collaborative mechanism, data management system, the inventory cycle and the methodology used in the national accounting. All these components work together within legal framework (Figure 1). The system Ghana has in place has supported the preparation of the national inventory for three cycles in 2000, 2006 and 2014. Nonetheless there are still more improvements that are needed to be introduced to make it work better than the way it is operating now.

FIGURE 1: ELEMENTS OF THE NATIONAL SYSTEM FOR GHG INVENTORY (SOURCE: NIR, 2015)

of 48

The apparent lack of QA/QC plan and clear operational guidance on how to treat uncertainties present one of the areas that require improvements. The benefits of having a plan will not only serve as a standard operating procedure (SOP) for the inventory team but will also make the inventory estimates more credible (build confidence in the inventory). It will in addition ensure that the process is transparent and defensible. The idea to improve the robustness of the way inventory is conducted couldn’t have been timely at this point when developing countries including Ghana are migrating to a new reporting regime. Shifting to the new reporting regime will not only mean Ghana starts to have sustainable system for producing the inventory, it also demand a high quality inventory every 2 years under Biennial Update Report (BUR). It is even more important for Ghana to have a credible system to produce the inventory because at this stage that all inventories shall be subjected International Consultation and Analysis (ICA), it is crucial it is done in a transparent manner. It is also important that, Ghana begins to strengthen the national system to respond to the high frequency regular reporting and review that comes with it. Hence, the need for continuous improvements in the way the national system functions should be considered a priority for Ghana.

2.2 Reforms in the national system The review of the national inventory report pointed out to a number of reforms that were introduced in national GHG inventory system during the last inventory preparation. These were as follows:

harmonization of the existing institutional arrangements and its underpinning collaborative mechanisms;

decentralization of the inventory task to key ministries.

creation of awareness and capacity improvement of key institutions that are involved in the preparation of the inventory;

support to structures for the continuous data collection on GHG inventories;

Introduction of an online data archiving system.

Although these reforms have been introduced, a lot more needs to be done before it becomes fully operational. One of the reason why reforms have not fully taken off is that the national cycle is still in a transition phase and thus a lot of time is needed to concretize the reforms. The reforms also did to cover QA/QC plan and guidance on uncertainty. Therefore the preparation of this plan is expected to fulfill one the inventory reform objectives.

2.3 National inventory cycle Ghana GHG activities are categorized into 4 main stages based on the 2006 IPCC guidelines for national GHG preparation. It is a continuous cycle which starts from review, planning, preparation, management and compilation as illustrated in Figure 2. Each phase focuses on delivering specific results upon which subsequent activities are built on. At the review stage, all feedback from the preceding inventory are collated

of 48

and evaluated as input into the planning of the new inventory. The planning involves team formation, organizing training programme, drafting and signing MOUs, identification of data sources and request of data. In the preparation phase, data are processed, methods are evaluated and sector estimates are produced. The activities in the management phase is cross-cutting. They involve activities that are relevant and applicable to data gathering, data processing, GHG estimation, data archiving, reporting and reviewing. QA/QC procedures is the major activity implemented in the management phase of the inventory.

2.4 Current QA/QC practices and gaps in them The current QA/QC practices in the inventory is broadly derived from the recommendations in the IPCC guidelines. Although there is some form of QA/QC procedures in place in the national inventory they are however not streamlined because of the absence a plan to guide the implementation of QA/QC practices in a systematic way. In the 3 previous inventories, QA/QC activities in the inventory had been largely led by the lead sector with the backstopping from EPA. The current QA/QC practices to a large extent cover the following operations

Selection of methodologies;

Underlying assumptions of method of GHG estimation;

Selection of activity data and emission factors;

Means of data acquisition and management and

Documentation However, there are some inherent gaps in the current QA/QC practices in the inventory that makes it difficult to achieve the level of confidence desired. The main gaps in the QA/QC practices are summarized below:

1.

Planning

2. Preparation

3.

Management

4.

Compilation

0.

Reviewing

FIGURE 2: GHANA’S INVENTORY CYCLE

of 48

Data acquisition issues - multiple sources of same data, non-compatibility of data formats, lack of metadata from most national sources, missing data, secondary data not specifically gathered for the inventory purposes, data errors and inaccuracy, data restriction and treatment of confidential data, cost of data and data sharing.

Data updates – who, when, what and how is not clear as at now, non-functioning data sharing network, who is to bear the cost of data updates, lack of priority to updates.

Methodological issues – no standardized QA/QC protocols, application of generic approaches to address sector-specific problems, use of expert judgment in filling data gaps, use of tier 1 or default emission factors for all categories.

Process related issues – unclear assignment of QA/QC responsibilities including who plays supervisory role, roles of sector leads, roles of sector experts and the roles of data providers.

of 48

3. The QA/QC Plan

3.1 What is the QA/QC plan set to do? Preparation of the QA/QC plan is part of the on-going reforms in the national GHG inventory system. The QA/QC plan will be the fundamental document upon which the inventory is anchored. It establishes all the activities, procedures, techniques to be observed and implemented in the fulfillment of quality objectives that will be set in the inventory. The purpose of the plan is to streamline and formalize existing QA/QC procedures and communicate with clear set of objectives to the inventory team. This will ensure that the inventory becomes more transparent, credible and defensible. The plan will also give tasks to institutions and individuals who are involved in the inventory with set targets and timeframe.

3.2 What can the QA/QC plan do? This plan has the following immediate objectives:

Provide guidance to help develop Ghana’s own customized QA/QC procedures based on the IPPC guidelines and taking into account our unique capabilities.

Identify past and current QA/QC procedures in order to streamline what is being and build on to become a functional and reliable system to safeguarding the quality of Ghana’s inventory.

Define roles and responsibilities, i.e., who is responsible for reviewing the estimates, documentation, who has the final authority to approval what data goes into the accounting etc.

Establish QC procedures, first by listing the minimal QC procedures that Ghana needs to undertake. Then providing recommendations for more rigorous QC or more in-depth examination of specific sources.

Identify or establish external review or QA, establish process and schedule for inventory review (e.g. third party review).

Establish the overall schedule for the QC plan, which should occur throughout the inventory process.

Detect errors/ blunders in the calculation of emissions and reduce uncertainty as early as possible before the inventory is made public or subjected to official reviews.

of 48

Provide recommendations on how to report on the QA/QC procedures in the national communications and BURs.

3.3 What are the key component of Ghana’s QA/QC plan? The plan will have the following components

1. Roles and responsibilities - Inventory agency, coordinator, sector leads and sector experts who will together be responsible for implementation of the QA/QC

2. General QC procedures 3. Sector or category-specific QC procedures 4. Coordinating QA procedures 5. Documentation and archiving procedures

3.4 Major QA/QC Principles that will anchor the plan? The overall goal of this QA/QC plan is to strive attain the high level of quality in the inventory in abiding by the principle of Transparency Consistency Comparability Completeness and Accuracy (TCCCA) at all times for different users. Therefore the QA/QC plan will seek to translate TCCCA principles into practical actions at every stage of the inventory process. The TCCCA principles will be inter-related and applied at various levels in the data gathering, estimation and reporting. These principles as defined below will be operationalized to the extent that Ghana’s capability and

financial resources will permit.

TABLE 1: DEFINITION OF QA/QC PRINCIPLES

Definition of TCCCA principles from the point of view inventory compilers

What is Transparency? - documentation and reporting

What does transparency mean in the national inventory

Processes involved in the preparation of the inventory should be clearly documented and reported to enable independent parties to understand and replicate and the process. Transparency of reporting is when enough information on the inventory processes are provided to enable any reviewer understand the inventory and make necessary recommendation for improvement.

How can transparency be achieved through documentation

Document all remarks including assumption made within the sector datasheets, IPCC software database, and the spreadsheet containing the raw and processed data, calculations and results.

Document all third party review comments and consolidate them into improvement plan.

The documentation or remark must be written in clear language (explicit and unambiguous).

of 48

Any third party should be able to make meaning out of the remarks so that any further action can be taken. The remarks must be consistently recorded and referenced so that it can be retrieved easily.

How can the inventory team achieve transparency in reporting Make sure the following are reported in the national inventory report

Explanations of the method used in each category-specific sector,

How the emissions by source emissions are aggregated to the dataset behind the inventory trend.

Progress made addressing QC recommendations made in the QA exercise.

What does consistency mean in the inventory - documentation and checking In order to achieve consistency, as much as possible, same methods and data sources must be used in the compilation of the inventory. This will ensure that any future changes in the emission will be as a result of real difference in the emissions rather than methodological and data variations.

How is consistency attained through documentation

Document all changes in previous methods use,

provide remarks on reasons for choosing a new method,

Document changes in datasets and sources,

Document differences in AD and changes EF and

Document impacts of recalculations.

What to consider in reporting time-series consistency

Report recalculation and filling of time series gaps under each category where possible;

Report on recalculations for each category under a specific sector, reasons for recalculation and its impacts and;

Describe type of time series gap, type of method use to fill the gaps, justification for choice of method and provide a description of how the gap was filled.

What does comparability mean to the inventory team – use of decision tree This ensures that emission estimates are comparable to other nations of similar circumstances. To ensure comparability, the IPCC guidelines will be adhered to as much as possible.

How to do document to achieve comparability

Document detail information on how estimates were arrived at and information on the methodology/tiers adopted for the inventory.

How to report on information to facilitate comparability

Report on emissions using summary tables and common reporting format. Report on coverage of gases and their global warming potential

Completeness – checking, documentation and reporting

of 48

Ensure that the estimates covers all major economic activities occurring within territorial boundary of the country and their respective gases.

How to check and document for completeness

Check and document emission activities occurring in the ten region of Ghana and exclusive economic zones are covered in the inventory. Inventory to cover all gases that are associated with the activities.

Report for completeness

Report using appropriate notation keys with any additional remarks on the changes in the scope of emissions activities and sources.

Accuracy – checking Errors and omissions do not occur that may result in overestimation or underestimation of the emissions. To achieve this, expert judgment or the use of guesstimate must be reduce to the barest minimum or in the event it is used, it must be properly documented.

Check for error and uncertainties

Check uncertainty for key category every two year cycle. For the other activities, check for uncertainty every four years to coincide with every cycle. Peer review of methodology to ensure that the most appropriate ones are used.

3.5 Key components of the QA/QC Plan

3.5.1 QA/QC System The proposed GHG QA/QC system must be fully embedded in the national system for GHG inventory to ensure efficiency. There is no need to run a parallel system for QA/QC. It will rather increase transaction and time cost. Therefore the QA/QC system will have special focus on giving practical guidance on how to design a simple-to-implement structures to underpin the operationalization of the QA/QC procedures. Although the immediate objective of the QA/QC plan is to consolidate the existing general practices in the inventory, the proposed system will build on the foundation that has been laid down in order to put in place a functional system in the long run. It will be designed around the Plan – DO- Check and Act (PDCA) principles and integrated in the national inventory cycle (Figure 3). The adoption of the PDCA model is consistent with the guidance provided in Chapter 8 of the IPCC (2006) Good Practice Guidance (GPG) as explained below.

P (Plan) - Inventory planning (timelines are set based on 2 year inventory cycle with clear roles)

D (Do) - sequence of inventory activities (data collection to reporting)

C (checking) – inventory is review, checking, documentation and archiving

A (Follow up and improvement) – evaluation begin with new inventory cycle

Continual

Improvement Cycle

QA/QC Inventory Improvement

1. Check if the inventory process conforms to internationally acceptable standards Check if emissions

were measured using internationally recognized methods

Assessing the effectiveness

of the inventory system

Conclusions for future

actions

QA/QC Inventory Planning

1. Determine QA/QC plan for

the inventory process:

Document the QA/QC plan

Identify and designate responsibilities for implementing QA/QC procedures

2. Determine levels of reporting:

Designate a QA/QC Lead who will be responsible for ensuring that the objectives of the QA/QC programme are implemented

Revisit of QA/QC

process

Setting quality

objectives

Specifying necessary

processes

and resources

Selecting methods

and emission

factors

QA/QC Inventory preparation 1. Check that parameters and emissions units are

calculated accurately:

Check that units are properly labeled in calculation sheets

Check that units are correctly carried

through from beginning to end of calculations

Check that mass conversion factors are correct

Check that temporal and spatial adjustment

factors are used correctly 2. Check the integrity of database files:

Confirm that path of data is correctly represented in database and that all necessary processing steps are accounted for

Confirm that data relationships are

correctly represented in database

Ensure that data fields are properly labeled and have correct design specifications

3. Check for consistency in data sources where such consistency is expected:

Identify parameters (e.g.; activity

levels, constants) that should be common to multiple source categories and confirm that there is consistency in the values used for these parameters in emissions calculations

4. Check that the movement of inventory data among processing steps is correct:

Check that emissions data are correctly aggregated from lower reporting levels to higher reporting

levels when preparing summaries Check that emissions data are correctly transcribed between different intermediate products

Collecting activity

data

Estimating GHG

emissions and

removals

Implementing

uncertainty

assessment

Recalculations

Implementing QC

checks

Reporting

Documenting and

archiving inventory

materials

QA/QC Inventory Evaluation 1. Check the accuracy of data input

from the original reference source:

Confirm that correct references were used

Cross-check sample of input data

2. Check that emissions are calculated accurately:

Use manual recalculation of a representative sample of emissions calculations

Recalculations should cover complete set of calculations from beginning to end

3. Internal documentation review:

Check that there is detailed internal documentation to support the estimates and enable full reconstruction of the estimates

Check that inventory data are archived and stored to facilitate detailed review

Implementing QA

activities: internal

audits, expert peer

reviews

Reviews of

international review

teams

Continual

Improvement Cycle

FIGURE 3: RELATIONSHIP BETWEEN INVENTORY CYCLE AND THE QA/QC PROCEDURES

of 48

3.5.2. Planning QA/QC in the inventory (Plan) In planning QA/QC activity for a particular inventory cycle, the inventory compilers together with the inventory compilers must be informed on the following issues and any changes that have occurred and how it will affect QA/QC procedures.

UNFCCC reporting requirement and the relevant section on QA/QC as pertain to non-Annex 1 Parties.

UNFCCC decisions relevant to GHG inventory for non-Annex 1 Parties

Feedback from technical review and record of facilitate exchange of views under international consultation and analysis (ICA).

Key highlight of relevant sections of the IPCC guidelines relevant to QA/QC Based on discussions that will take place on the issues mentioned above, set QA/QC objectives for that particular inventory year. The objective that will be set is expected to drive the QA/QC activities for the inventory year. Some of the most important guidelines and information for the inventory planning are:

IPCC Good Practice Guidance and Uncertainty Management in National Greenhouse Gas Inventories.

2006 IPCC Guidelines for National Greenhouse Gas Inventories

UNFCCC Guidelines on reporting for national communication and BUR for Parties not included in Annex 1 to the convention.

Plan improvement area identified in the latest inventory report submitted to the UNFCCC.

Technical review report from the ICA process.

3.6. Preparation of QC/QA Tasks (DO)

3.6.1Roles and responsibilities

Coordination of implementation of the QA/QC Plan A multi-level institutional arrangement model will be used for the implementation of the QA/QC plan at three levels. The activities in the QA/QC will be implemented in a two year period to coincide with the inventory cycle for the categories that contribute most to emission and contribute most to total inventory uncertainty. Activities in all other categories contributing less emission will be subjected to QA on a four year basis. There will be a three-layered structure to implement the QA/QC plan. At the top of the

of 48

structure is the EPA who will play a facilitating role to make sure the second and third layers function effectively. The second layer will be in charge of the general QA/QC procedures for all sectors and will be led by the Department of Statistics1, University of Ghana. The third layer will be headed by sector leads and will be responsible for sector-specific QA procedures. The sector leads are also expected to liaise with data providers to make sure that the minimum QA procedures are followed.

Inventory Compiler

QA/QC Lead

Sector lead

Functional team

Planning team

Inventory Coordinator (EPA)

Data provider

Energy sector

Data provider

IPPU sector

Data provider

AFOLU Sector

Data provider

Waste sector

Fee

db

ack team

FIGURE 4: QA/QC ORGANIZATIONAL STRUCTURE

The QA/QC plan is anchored on the 2006 IPCC guidelines owing to the fact that there are no established in-country QA/QC standards at the national level. Inasmuch as we recognized other international accounting standards such as ISO 14064 and others, the use of 2006 IPCC guidelines will ensure comparability with countries of similar circumstances who have adopted the same guidelines.

3.6.2 Awareness, Education and Training As the demand for transparency and high quality of inventory estimates continue to increase, the work load in the inventory especially with the introduction of stringent QC procedures, more hands will be needed to get the job done both within the inventory sectors and at the level where the QA/QC Lead will be engaged. It is therefore important to create awareness on the quality requirements that have been adopted in

1 This department has the technical capacity and had indicated its willingness to play the role as QA/QC coordinator previously.

of 48

the inventory to enable all those working in specific areas to comply with the procedures. Where there is a specific special capacity gap, tailor made training topics will be designed for existing sector experts as refresher programmes. It will also serve as and fresh training for those who will be joining the team for the first time.

3.7 Evaluation (Check)

3.7.1 General QC Procedures

Key questions to decide where/when to do QC: Generally, in national inventory management, where tier 1 QC procedures are used, attention is often given to those categories that are critical to the inventory because of resource and time constraints. Therefore it is good practice to prioritize the implementation of the activities of the QA/QC plan based on the following guiding questions (table 2):

TABLE 2: FACTORS TO CONSIDER WHEN SETTING QC PRIORTIES

Focus first on these

1. Key Categories

2. Emission and removal categories with recent data changes

3. Emission and removal categories with recent methodological changes

4. Categories that contribute most to uncertainty in inventory.

Key categories - Is a source/sink a key category? How significant is this source to

total inventory estimate? What is the source’s contribution to trends in emissions over time?

Emission and removal categories with “recent data changes” - has there been a change in the data or characteristics of the source category? Has there been a change in the data or characteristics of the source category? Are emission factors different from IPCC default factors or commonly used factors? Did significant changes occur in how data are processed and managed?

Emission and removal categories with “recent methodological changes - Was a category updated a long time ago? Is there a risk of double counting or incomplete estimates?

Levels of contribution to uncertainty in the inventory.

of 48

In addition, it is important that the scheduling of the implementation of the prioritized QC activities be included in the plan. Ghana has to determine when certain procedures will be applied taking into consideration the inventory cycle The following guiding question will help to determine the specific times to apply QC procedures. Which procedures will be conducted annually or bi-annually? Which inventory cycle is considered? What level of QC is needed annually? How often do you update data? Should QC be performed prior to expert review?

3.7.2 QC Procedures

Tier 1 QC Procedures Tier 1 General Inventory Level QC procedures are checks, documentation and archiving practices that the inventory compilers use routinely throughout the inventory cycle. The focus of general QC techniques is on the processing, handling, documenting, archiving and reporting procedures that are common to all the inventory source/removal categories. In Ghana, certain level of QC procedures were followed during the last inventory although more improvements are needed to streamline the process. This plan seeks to build on the tier 1 QC procedures used in the previous inventory and suggest additional important procedures and clear roles/responsibilities toward attaining a higher tier. Table 3 contains the tier 1 QC procedures

Tier 2 QC Procedures Sector/category-specific QC procedures are directed at specific types of data used in the methods for individual source/sink categories and require knowledge of the emissions source/sink category, the types of data available and the parameters associated with emissions/removal. The sector/category specific QC measures focus on key categories and on categories where significant methodological, data revisions have taken place and categories that contribute significantly to uncertainties in the inventory. Tier 2 QC activities are in addition to the general QC conducted as part of Tier 1. Currently, Ghana’s QC procedure is largely based on tier 1. Therefore the intention to move to higher tier by strengthening QC protocols will not only improve the methodological and data quality but will also reduce uncertainties. Table 4 contains sector specific QC procedures

of 48

TABLE 3: DESCRIPTION OF GENERAL QC PROCEDURES

QC Activity Procedures Task Completed Corrective Measure Taken

Name/

Initials

Date Supporting Documents (List Document Name)

Name/

Initials

Date

Data gathering, input, and handling checks

Check that assumptions and criteria for the selection of activity data and emission factors are documented.

Cross-check descriptions of activity data and emission factors with information on categories and ensure that these are properly recorded and archived.

Check for transcription errors in data input and reference.

Check that all citations in spreadsheets and inventory are complete.

Confirm that bibliographical data references are properly cited in all internal records.

Randomly check that the originals of citations contain the material & content referenced

Cross-check a sample of input data from each category (either measurements or parameters used in calculations) for transcription errors.

Use electronic data where possible to

minimize transcription errors.

Check that spreadsheet features are used to minimize user/entry error: (a) use cell protection so fixed data cannot accidentally be changed; (b) build in automated checks, such as computational checks for calculations, or range checks for input data.

Check that emissions/removals are calculated correctly.

Reproduce a representative sample of emissions/removals calculations.

of 48

If models are used, test run the model calculations at minimal level to judge its relative accuracy.

Check that parameters and emission/removal units are correctly recorded and that appropriate conversion factors are used.

Check that units are properly labeled in calculation sheets.

Check that units are correctly carried through from beginning to end of calculations.

Check that conversion factors are correct.

Check that temporal and spatial adjustment factors are used correctly.

Check the integrity of database files.

Confirm that the appropriate data processing steps are correctly represented in the database.

Confirm that data relationships are correctly represented in the database.

Ensure that data fields are properly labeled and have the correct design specifications.

Ensure that adequate documentation of database and model structure and operation are archived.

Check for consistency in data between categories.

Identify parameters (e.g., activity data, constants) that are common to multiple categories and confirm that there is consistency in the values used for these parameters in the emissions/removals calculations.

Check that the movement of inventory data among processing steps is correct.

Check that emissions/removals data are correctly aggregated from lower reporting levels to higher reporting levels when preparing summaries.

Check that emissions/removals data are correctly transcribed between different intermediate products.

Data Documentation

of 48

Review of internal documentation and archiving.

Check that there is detailed internal documentation to support the estimates and enable duplication of calculations.

Check that every primary data element has a reference for the source of the data (via cell comments or another system of notation).

Check that inventory data, supporting data, and inventory records are archived and stored to facilitate detailed review.

Check that the archive is closed and retained in secure place (preferably on the online server) following completion of the inventory.

Check integrity of any data archiving arrangements of outside organizations involved in inventory preparation.

Check that archiving has been done both the sector and inventory level.

Calculation Checks

Check methodological and data changes resulting in recalculations.

Check for temporal consistency in time series input data for each category.

Check for consistency in the method used for calculations throughout the time series.

Reproduce a representative sample of emission calculations to ensure mathematical correctness.

Check time series consistency

Check for temporal consistency in time series input data for each category.

Check for consistency in the method used for calculations throughout the time series.

Check methodological and data changes resulting in recalculations.

of 48

Check that the effects of mitigation activities have been appropriately reflected in time series calculations.

Check completeness Confirm that estimates are reported for all categories and for all years from the appropriate base year over the period of the current inventory.

For sub-categories, confirm that the entire category is being covered.

Proved clear definition of ‘Other’ type categories.

Check that known data gaps that result in incomplete category emissions/removals estimates are documented, including qualitative evaluation of the importance of the estimate in relation to total net emissions (e.g. subcategories classified as ‘not estimated’).

Trend checks For each category, compare current inventory estimates to previous estimates, if available. If there are significant changes or departures from expected trends, re-check estimates and explain any difference. Significant changes in emissions or removals from previous years may indicate possible input or calculation errors.

Check value of implied emission factors (aggregate emissions/removals divided by activity data) across time series. Are changes in emissions or removals being captured?

Check if there any unusual or unexplained trends noticed for activity data or other parameters across the time series.

Internal consistency checks

Check that the total GHG emissions equaled the sum of the individual emission from the sectors and categories.

of 48

Check that the total GHG emissions equaled the sum of the emissions by gas.

Check that data in tables are the same as the calculation in spreadsheets and in the text.

Check that parameters used in multiple categories (e.g., animal population) are consistent across categories.

Source: This list has been adapted from IPCC Good Practice Guidance and the 2006 IPCC Guidelines for National GHG Inventories.

TABLE 4 SECTOR-SPECIFIC QC PROCEDURES

QC Activity Task Completed Corrective Measure Taken

Name/

Initials

Date Supporting Documents (List

Document Name)

Name/

Initials

Date

Data gathering and selection (emission data quality checks)

Compare emissions time series using historical data for source/sinks, significant sub-categories

Check all reference calculations including the rough work used to arrive at the emission estimates

Check completeness of methodology, coverage of gases and data throughout the time series

of 48

Assess representativeness of emission factors and all analogous emissions data.

Compare to alternative factors (e.g., IPCC default, cross-country, literature)

Search for options for more representative data from research and studies conducted in the sub-region.

Activity data quality checks: Activity data

Check methodology for filling in time series for missing data

Check consistency of historical trends

Compare multiple (national and international) data reference sources Check applicability of data from international sources in cases where local data do not exist

Check for inconsistencies across sites2

Compare site specific activity data (aggregate) with national data

Secondary data and direct measurement

Are QC activities conducted during the original preparation of the data (either as reported in published literature or as indicated by personal communications) consistent with

2 Where facility-specific data are collected, for example, activity data from variety of thermal plants operated by IPPs at different locations in the country, it will be good practice to compare the datasets to check for consistency before it is used to generate representative emission estimate for the country.

of 48

and adequate when compared against (as a minimum), general QC activities?

Does the data provider have a QA/QC protocol that covers the preparation of the data?

For surveys, what sampling techniques were used and how recently were they reviewed?

Have uncertainties in the data been estimated and documented?

Have any limitations of the secondary data been identified and documented, such as biases or incomplete estimates? Have errors been found?

Have the secondary data undergone peer review and, if so, of what nature?

Additional specific QC procedures for Energy and AFOLU sector

Check consistency in fuel balance in deference to sectoral approach

Check consistency in data reported in the national energy balance and activity data used in the inventory. Define similar level of disaggregation for sector fuel allocation.

Cross-check allocation of non-energy/feedstock fuels (fuel balance approach).

Check for uncounted fuel use on annually.

Check accuracy in methodology used for annual and bi-decade land representations.

of 48

3.7.3 General guidance on documentation Proper documentation is the cornerstone to ensure transparency in the inventory. Documentation of the national greenhouse gas inventory process is paramount for any future work. Efficient documentation guarantees greater transparency and repeatability. There are several benefits that documentation add to the quality of the inventory. For instance, it helps others who had been part of the inventory understand how the inventory was done; in case the inventory will be reviewed by third party, documentation provides a good reference sources for the reviewers especially when they want to understand which method was used for the estimation, how it was selected, assumption behind activity data and finding out the possibility of repeating the steps used for the inventory. It is therefore important to streamline the documentation done by the sectors and those who are responsible for the inventory compilation. Although some amount of documentation was done during the preparation the latest national GHG inventory, going forward a systematic way for referencing, documentation and storage must be adopted. This approach can start from a simple set of steps in the next inventory cycle with clear roles/responsibilities and timelines embedded in the inventory cycle. Then over a period of time improvements are gradually introduced into them. Generally documentation and archiving should be the responsibility of the inventory compilers. The inventory compiler must work closely with the sector leads and the sector expert to do a simple documentation checklist (Table 5) which will be used by the sector experts and the inventory compiler for documentation and sign-off (. This will also help build the paper trail in consistent manner. It is also extremely important to note that the beginning of each inventory, all the inventory experts and data providers are taken through the documentation steps for them to be abreast with what is expected of them. The documentation task should be clearly stated for every activity the inventory compilers undertake.

[A]- Documenting information on data collection Data collection must be thoroughly documented since it forms the foundation of the inventory estimates. All the sector leads and the sector experts are to keep records of the following basic information in the documentation template and sign-off. When all the documentation template are completed, the person who is responsible must keep both digital and hard copies of the template, before onward submission to the sector lead and then finally to the inventory compiler. The file naming of the completed documentation template must be consistent with the file naming convention. The basic information that must be recorded as follows:

of 48

TABLE 3: DATA COLLECTION DOCUMENTATION TEMPLATE

What to record Y N

Has data request be done

Name of institution that data is being requested from?

List of data or Information requested

Date that data/information was requested

Who did the data request?

How was data request done? Is it through a formal letter or verbal request?

What data has been provided by which institution?

What is the status of the data?

Are there missing data in what was submitted by the data provider?

What outstanding data must be collected

Signing off person

Approved by

[B]- Electronic file naming convention It is recommended that GHG inventory adopt a simple standard electronic file naming format to help enable proper record keeping and storage. As much as possible, thorough documentation of inventory (a) processes, (b) activity data, emission factors, assumptions and (c) any information that is considered relevant for will be done. In order to ensure consistency in the documentation of activities of the inventory sectors. In this respect, each sector has to do thorough documentation of any (a) vital information, (b) report, (c) data and (d) any assumption or expert judgment made in the course of the inventory. The electronic filing system will complement documentation in hard file by creating file system that will serve dual purposes.

The filing system will be at two levels will be defined by, namely, “folder” unique characters and “file” unique characters. The folders will contain number of files. Each file will describe specific metadata on items contained in them. These unique characters will define the serial numbers of electronic file. The unique character descriptors will form the file and folder shells and will be stored in both the front and deep shells. The front shell files or folder will be used by the inventory experts on daily basis to upload and record all items. At the end of the day, these files will be stored on the hard drive of the local computer host. The serialization of the files or folder will be such that, it is unique to the sector of the inventory, activity, type of record, version,

of 48

date and source. Because the front shell items are working folders/files, inventory experts will easy access through their desktop window. They will have the opportunity to edit, update, upload and retrieve all documentation without access limitation. They will be required to follow strict file/folder numbering after working on the files before uploading them on the front shell storage. In uploading data into the front shell, the user would have to be mindful of the filing procedures and make sure all the files/folders are properly saved in the directory that can be retrieved easily. The destination of the file/folders on the front shell on “my document” which will be automatically saved on the hard drive of the local host. Because the front shell items are working files/folders, two back up files will be created in the deep shell.

The deep shell version of the file/folder is a back-copy of the front shell. The deep shell will be used only for archiving and occasional retrieval. It will be hosted in two directory destinations. The first –back-up catalogue will be stored on the central server hosted at the EPA. The level of access to the files/folders that will be stored on the central database will be limited to the administrator. In the event that the administrator is not available, he/she will permit full access to the file/folders through remote access. Another option is that, the administrator could, depending on the circumstances, give password access to the person who is requesting access to the file. The next destination of the back-up files/folder would be on external drive. The external hard drive version of the files/folders is not meant for retrieval without permission for the administrator. File/Folder retrieval from the hard drive will be granted only after access is unable in the front shell and the server due to technical problems. The filing on both the front and deep shells will be with consistent with each other. The two will be separated by the plus symbol (+) which will denote hierarchy of file structure. The file numbering format for folder/files in a front or deep shell will read as follows:

Deep shell – Plus_sector_activity_type_source_version_date_by

Front Shell – sector_activity_type_source_version_date_by

The unique numbering characters are explain below

Sector – this contains unique character of the component of the project. In all there 4 components and these will describe the uniqueness;

1. EN – Energy 2. IPPU – Industrial Processes 3. AFOLU – Agriculture, forestry, and other land uses 4. WA – Waste

Activity – describes the type of category under specific sector. For example, in the Energy sector (En), mobile, or stationery combustion reports are classic

of 48

examples categories. If it happens to be mobile, then the numbering will read: ENm or stationery would read, ENs.

Type – is the type of record the file contains. It could be reports, data, scanned doc, email etc. It could be data on vehicles, electricity plants, oil refinery etc. Eg. For vehicles, ENmrd or electricity plant, ENsep

Name - contains the name that describes the content of the file. For example, if the file contains state of the economy report, then the name will be “state of the economy” or energy statistics, data from DVLA. Eg. “ENsepEnergyStatistics”

Version – is the descriptor of the current version of the document. If it is 2013 version. “ENsepEnergyStatisitics13”

Date – date of last modified. Day/month/year. “ENsepEnergyStatisitics1306”

Source – source of publication or data.” “ENsepEnergyStatisitics1306EC”

By – name of person who was the last to upload or modify the document. “ENsepEnergyStatisitics1306EC_Mr. X”

[C]. Documentation of assumption, methodological and data choices and changes in them

For data and methodological choices, document all of the following either in the IPCC software or on the methodology and data documentation checklist similar to the one presented in table 5:

Method used to generate AD, EF or GHG emissions

Any change in the methods used in the previous inventory and the reasons for

the modifications?

Who did the changes or modification?

Which aspects of the methodology has changed?

What specific improvement are needed in the methodology? And who should do

it?

What timelines have been set for the improvements?

Are all working assumptions in the calculation AD, data filling etc applied?.

of 48

3.7.4 Key QA Steps Already in Ghana’s inventory at two levels of QA exist in a form informal technical and policy review. This is in addition to the internal review check conducted by the inventory compiler on each sector spreadsheet. A new layer of formal international review through international consultation and analysis (ICA) has been introduced by the UNFCCC. This will mean that, when Ghana submits it inventory report to the UNFCCC secretariat it will be subjected to two levels of third party reviews; technical review and a form of peer multilateral review which is captured as facilitative exchange of views.

Internal review by inventory compiler The inventory compiler conducts internal consistency review on all the sector spreadsheets before they put together for further analysis. Although the focus of this review is to pick up time series and internal inconsistencies in the inventory, it could also serve as a useful way to identify errors that were omitted. In order to improve internal checks of the sector spreadsheets, Ghana could consider using referencing approach or sample portions of data in the spreadsheet to repeat it. It will also be important that the internal review process and the outcomes are recorded transparently.

Consider to use the template for the internal review of the sector spreadsheet

TABLE 4: TEMPLATE FOR KEEPING RECORDS ON INTERNAL REVIEWS

Name of internal reviewer

Date of review

Sector Main QC/QA Identified Review method used Remarks for future improvement

of 48

Independent “informal” technical review In the current inventory, Ghana has adopted informal technical review of the inventory before final submission to the UNFCCC. The purpose of this informal review is to use the same technical review guidelines the UNFCCC will apply in the formal review to assess Ghana GHG inventory. The review focuses on identifying errors and issues that will have to be addressed before submission to the UNFCCC. This is an important review and Ghana must continue with it insofar as available resources permit.

Policy review and reality check of the inventory estimates The current practice that the sector inventory results are subjected to “policy review and reality check” by the relevant Ministries, Department and Agencies (MMDAs) is an important QA step that must be maintained because it is a useful way to get feedback from intended users of the inventory results. However, the inventory compilers could consider expanding the current scope of the policy review to include transparency and accuracy checks. Although this exercise is voluntary, it must be sustained as part of the QA procedures in the inventory if financial resources are available.

UNFCCC Technical Team of Expert Review under International Consultation and Analysis (ICA) As part of the enhanced reporting regime by the non-Annex 1 Parties to the convention, the inventory report which will accompany the Biennial Update Report (BUR) will be subjected to ICA. The steps of the ICA is guided by decision 2/CP.17 (paras 56-57) and 20/CP.19 and it is two parts. The first part is the technical analysis of the BUR by the Technical Team of Expert (TTE). The TTE that will be put together by the UNFCCC secretariat will be made up of members from developed and developing country parties. The output of the technical analysis of the BUR by the TTE will be a summary report (SR). During the technical analysis, the EPA will act as the contact point. As a contact point, the EPA will be on standby to provide responses to questions and clarifications that may be raised by the TTE during the technical analysis. The second part of the ICA is the multilateral facilitative exchange of views (FEoV) which will be held as workshop under SBI for one- to three-hour session. During the workshop, Ghana will make a brief presentation on their biennial update report, followed by oral questions and answers among Parties. The outcome of the ICA will be a summary report and a record of the facilitative sharing of views. Although ICA process is seen as mandatory requirement that non-Annex Parties to the convention must comply, it could also be a good mechanism for valuable feedback on the estimate and the methodology associated with it.

3.7.5 Follow-up and improvement (ACT) At the beginning of every inventory cycle, key issues that were identified in the past inventory must be evaluated. The evaluation must focus on assessing whether or not a specific recommendations is a priority that must be corrected in the inventory cycle in question, or if certain aspects of the recommendation can be implemented, how it will be sequenced, if the inventory compilers even have the capacity at all to implement or

of 48

whether or not it will be included in the long term improvement plan of the inventory. For those issues that cannot be corrected, it is important a clear plan is prepared to give indication as who and how it will be addressed. In order to track the progress of addressing QC/QA issues in the inventory, the inventory compilers and the QA/QC Lead must adopt a standard way to record and track how the outstanding issues are being addressed. Consider using the template (table 7) below to track how key QA/QC issues are being addressed:

TABLE 5: TEMPLATE FOR TRACKING QC ISSUES

No Key

QC/QA

Issues

Sector Key

category

Gas Description of

key issues

Action

taken

Status Outstanding

task

of 48

There are 4 main sources in the inventory process where QC/QA issues are likely to emerge. At least three of them relate to QA activities which cover different types of third party review. Because of the third Party reviews have specific focus in scope and process, the key QA/QC issues that may come out it will be useful to get them addressed in a reasonable time, if the resources and capacity are made available to the inventory team. The QA activities in the inventory that are likely to generate QA/QC issues are as follows; (a) feedback from ICA process either through the technical review report or the record of facilitative exchange of views; (b) Informal independent technical national or international review which is usually conducted by Ghana before the inventory report and the associated tables are submitted to the UNFCCC and (c) comments that will be received from the policy review and reality check meetings that is organized for different ministries. Another important source that QA/QC issues could emerge is records on planned improvement areas that usually are produced during the inventory. In Ghana’s inventory, each sector produces a list of areas in the sector that needs further improvement. Some of the issues that would be identified might include QA/QC issues. At each evaluation session, special attention must be given to QA/QC issues that have been identified on specific activities in key categories and part of the inventory that contribute most to the uncertainties.

of 48

4. The roadmap for QA/QC improvement

4.1 QA/QC ROAD MAP The road map is intended to provide steps that must be followed by Ghana to have a fully functional QA/QC procedure in the inventory process. In this regard, the plan will facilitate the long term improvement of the planning and implementation of the QA/QC from the current QA/QC tier 1 to the most desirable higher tier (tier 2). The implementation of the road map, must be prioritized and occur within a reasonable time frame, with adequate capacity, commitment and should be adequately financed. The proposed steps to ensure the move from the current QA/QC practices to a more improved one must be linked to the continuous improvement of the inventory (Table 8). The sequencing of the steps in the road map into short, medium and long term must take cognizance of the importance attached to the specific areas of the inventory that requires immediate attention such as the key categories, areas where methodological and data changes are taking place and the areas with the categories that contribute most to total inventory uncertainty.

of 48

TABLE 6: ACTIVITY PLAN FOR QA/QC ROAD MAP

Overall Goal To move from the current QA/QC practice to desirable a higher tier by 2021 and beyond, taken account capability available in the country

Short –term objective Have functional tier 1 in the next inventory cycle (2016-2018) that ensures both general and sector specific QC practices are implemented.

Medium –term objective Have a functional tier 1 for the general QA/QC and tier 2 for sector specific QC in the inventory cycle (2019-2021)

Long-term objective Put a functional tier 2 in place for the general QC and a higher tier for sector specific (beyond 2021)

Tasks Priority Time frame Resources Action by Assumption(s) QC Outcome

Assess capacity and resource gaps for QA/QC

Sta

ge 1

(2016-2

018)

High Q1, 2016 review team,

funds

EPA/MESTI/ Academic Institution

Funds secured

Understand strength/weakness and prioritize improvement efforts.

Appoint QA/QC Lead

High Q1, 2016 Personnel EPA/MESTI Give clear terms of reference or MOU

QA/QC procedures streamlined.

Regular hands-on training on key QA/QC topics for inventory compilers and data owners

Medium Q2-Q3, 2016 Trainers, Software, Training manuals, Funds

QA/QC Lead/EPA

Funds secured Software and training manuals available

Improve technical capacity to roll out QC procedures.

Gather meta data on key categories and activities where the changes in data and methodology are taking place.

High Q1-Q4 2016 Funds, skilled

personnel,

Data providers and QA/QC Lead

Meta data exist but not published, data providers are willing to gather meta data where it does not exist

Improved transparency, accuracy of inventory

Provision of materials for QA/QC activities

High Q1-Q4

2016

QC templates, referencing guide, QC timelines, Funds

QA/QC Lead Sector leads

The general and sector specific QC procedures are available, funds are secured

Improvement in documentation and archiving

Storage/Archiving of all data, reports and references

High Q1-Q8, 2018 Files Servers External drive

QA/QC Lead Sector leads/inventory compilers

Storage devices are made available

Improve data retrieval and easy referencing

Ensure rigorous external review of data and

Medium Q1, 2019 Personal, task sheets

QA/QC Lead

QA task sheet prepared, funds

Improve transparency and accuracy. Detect error at early stage.

of 48

3 Assign clear QA tasks at each specific stage in a sequence that must be performed by the sector compilers, sector leads, QA/QC Leads Inventory coordinator and the EPA.

methodological choices and estimates3.

Continuous methodological refinement for estimating emissions in key categories and support for research

Sta

ge 2

(2019-2

021)

High 2018, 2021 Funds, Equipment, training

EPA Funds are made available, access research result are publicly available, competent experts are accessible

Ensured accuracy and consistency

Begin implementation of sector specific QC procedures starting with major sectors or categories

High Q1, 2019 Training, funds QA/QC Lead/ Sector leads/ data owners

Data is available at sector level. Sector/category specific QC working template

Improved transparency in the key sector/categories

Promote adoption of ISO based QC standards for Industry sector

Low Q2, 2019 Training, funds Regulatory framework

EPA/ QA/QC Lead/ Sector leads/ data owners

Industry willing to adopt ISO QC standard

Enhanced transparency and accuracy

Organize policy review and reality check QA for key Ministries

Medium Q7, 2017 Logistics MESTI Logistics are secured. Willingness of MMDAs to participate

Enhanced confidence and acceptability of the inventory results.

Organize regular meetings with Universities to identify areas in the inventory that need further research.

Medium Q1-Q4, 2018 Logistics EPA Logistics are secured. Willingness of Universities to engage with the process.

Improved methodology and data for the inventory

Develop and implement QC protocols for “primary data generation” (emission factors activity data)

Sta

ge 3

(beyond

2021)

High Q1-Q8, 2018 Funds, researchers, equipment, training, QC protocol templates

Data owners/ EPA/QA/QC Lead

Funds secured, Inventory sectors have capacity to implement, QC protocol template is available

Strengthened sector/category QC procedures.

Create a functional central climate data sharing port which could be hosted Ghana statistical service

Medium Q8, 2018 Funds, IT infrastructure, Training

MESTI/EPA/ GSS/ Sector leads

Funds are provided, IT infrastructure is in place, Willingness of institutions to participate.

Improved timely access and retrieval of data

of 48

5. Uncertainty Management

5.1 A Review of current approach for Uncertainty assessment in the GHG Inventory In the current inventory, uncertainty assessment and management practice is based on tier 1 approach. The uncertainty assessment of activity data and emission factors are derived from expert judgment using default IPCC uncertainty ranges provided in the emission factor database and 2006 guidelines (Table 9). For example, the choice of uncertainty range for a particular AD will depend on data source and how it was collected (Table 9). The overall uncertainty is automatically estimated with the tier 1 formulae in the IPCC software once the necessary uncertainty ranges are appropriately assigned to the AD and EF. This approach was adopted because most of the activity data were mainly from secondary sources that hardly reported uncertainty ranges in their metadata. Qualitative approach backed by experts’ judgment were used to assign the uncertainty ranges based on the sources of data in a consistent and transparent manner. In addition, the uncertainty ranges associated with the IPCC emission factors were also used. Using the IPCC recommended minimum uncertainty range of ±5% for facility level activity data, the uncertainty ranges were assigned to each activity based on the source. The spread of uncertainty ranges was assumed to increase according to the level of verifiability and reliability of the source of data.

TABLE 9: RANGE OF UNCERTAINITY INPUT FOR ACTIVITY DATA Activity data source Uncertainty

Range Comments

Plus Minus

Facility level measurement 5% 5% Applied to Volta Aluminum Plant

Peer reviewed literature 5% 5%

Research results 5% 5%

Enumeration 4% 2% Driver Vehicle and Licensing Authority data type

Industry archive 6.5% 6% (Ghana railway company type)

International sources 6% 5.50% (FAO, IEA, WB, etc)

National reports Including strategies, action plans etc.

Annual reports 5% 5.5%

Project reports 5% 5.5%

Energy Statistics 6% 5.5%

National Census 5% 5%

Ghana Living Standard Survey 5% 5.5%

Expert judgment 15% 12%

Personal Communication 10% 10%

of 48

Even though the way the uncertainty assessment is conducted in the current inventory is not the recommended quantitative approach for estimating uncertainties, the adopted approach helped Ghana to report transparently. The challenges that are associated with the way the uncertainty assessment can be summarized into the following issues:

Non-existence of metadata associated with AD supplied by data providers – many of the AD owners do not published metadata so it is difficult to determine uncertainties with AD. Because the inventory compilers rely on secondary data, owners who do not provide metadata, no additional information on primary data collection is made available to provide ample understand of the sampling design. Apart from the fact that the secondary data that are provided without the metadata, in most cases where they embark on primary data collection, the information on sampling protocol are not available except on request. For instance, the Ghana statistical service’s decade national population census or the 5 yearly, Ghana living standard survey, may have metadata or sampling design or estimate of sampling errors, which hardly could be given out to data users.

Use of default emission factors with IPCC uncertainty range – except VALCO which supplied facility-level emission factors, all the EFs that were used in the latest inventory were default factors from the IPCC emission factor database. The default uncertainty ranges that are associated with the default IPCC EFs were used in the uncertainty assessment. For the EF collected from VALCO, no additional information on uncertainty were provided, hence, the IPCC default uncertainty were used instead.

Filling of missing time series activity data gaps – the use of statistical extrapolation and interpolation method to fill activity data gaps may have helped to get time series data complete, it is also present a weak link in the uncertainty assessment.

Subjectivity in the determination of uncertainty range threshold - the inventory compilers used expert judgment to determine the uncertainty range threshold based on the data source. The basis for assigning the certainty range were subjective and applied to all the sectors without any variation.

Establishment of categories that contribute most to the uncertainty in the inventory may not reflect the true picture since it is based on the expert judgment and subjectivity.

of 48

5.2 Practical steps to assess data quality Generating activity data and emission factors, either through physical measurements or modeling carries certain levels of uncertainty. Datasets that are produced through such processes introduce inherent uncertainties into the inventory. In addition, the reliance on secondary activity data and default emission factors for the inventory estimation further contribute to uncertainty levels. This is because in many cases the suppliers of secondary data hardly provide information on metadata or sampling framework. Such information are crucial to understand the sources of errors, level of errors and how the errors have been managed in the generation of AD and EF. Another source of uncertainty in the use of default emission factors from the IPCC emission factor database. The use of the default AD and EF, particularly, for key category activities is a major source of uncertainty in the inventory. According the current inventory report, many of the key categories are in the transport, electricity generation, solid waste management and land management activities, therefore, the expectation is that since they contribute most emissions, if the errors in their AD and EF are high, it pushes up overall uncertainty in the inventory. In order to improve on the uncertainty assessment in the inventory having in mind the gaps in the current practice, it is important that attention is given to the practical steps that need to be taken by the inventory compilers to improve selection of AD and EF to reduce uncertainties.

Guiding principles for the assessment of activity data quality assessment

1. Use expert information to validate activity data reported by the district

assemblies through research in selected sectors.

2. Select data that have clear source and can be referenced (table 10). Dated and attributed to a source4.

3. Select or use nationally approved sources when there are variations in the same data from international sources.

4. Use data from mandated national institution in the case where there are variation in the same data from multiple agencies unless the authenticity is not in doubt using any available documentary evidence.

5. As much as possible official data which report legal economic activities in the country, however, in the event, the data would have to expanded to include “informal activities” (for instance, quantities of smuggled fuel, illegal timber etc), use peer reviewed data.

6. In cases where it is not possible to access disaggregated data from industry, security agencies and academic sources, for reasons of confidentiality, use the aggregated data supplied to you.

4 Use the table to guide you to prioritize the credibility and the data to select.

of 48

7. In using expert judgment the inventory compilers must take the following into

account:

consistency in the application of the expert judgment to similar situations in the inventory;

documentation of assumptions and how the expert judgment has been applied across the inventory;

determination of the conditions that warrant the use of expert judgment and

Reasonableness of underlying factors of the expert judgment in relation to the use of empirical data.

Exercise caution in throwing out data which appear to be an outlier unless you have reasons to do so.

8. Seek a second or third expert opinion in using data from expert judgment and document.

9. In some cases in Ghana, activity data are generated for reasons other than the inventory and in different format (eg. Vehicle data from DVLA). The original state of the data may not permit its use in the inventory. In such situations, process the data to useable format where the technical condition permits without compromising its quality.

of 48

TABLE 10: CRITERIA FOR DATA PREFERENCE

AD/EF Sources Direct measurement Estimation Census

Surveys/interviews Expert judgment

Field research

Industry

Interpolation/

extrapolation

Enumeration

Activity Receipt5

Data from government

institutions

Grey literature

Peer Review

Laboratory

Key: Data choice and preference in descending order – green means most preferred to red mean least preferred

5 Records of purchase of productivity units such as fuel consumption, etc

of 48

5.3 How to report on progressive improvements on uncertainty of AD and EF The current practice Ghana has adopted to conduct the uncertainty assessment, although under the circumstances could be considered practical option, a lot can be done to improve on that practice and also set out a plan to adopt a more quantitative approach as recommended by the IPCC. However, before Ghana can confidently switch from using qualitative approach to determine uncertainty to quantitative approach, several steps will have to be taken to streamline the current practice. In addition, similar steps must be established to guide the way the tier 1 IPCC uncertainty assessment is implemented in the future. It is therefore important that, in the subsequent inventory reports, Ghana adopts a consistent format in order to report on (a) how uncertainty assessment was conducted, who did it, the underlying assumptions and (b) steps being taken to adopt the IPCC recommended tier 1 uncertainty which is based on quantitative approach and the progress made so far.

5.3.1 Ways to improve the current practice of uncertainty assessment The apparent lack of adequate background information on AD and EF used in the inventory is the reasons for adopting the qualitative approach for estimating the uncertainties in the inventory. This is because the inventory compilers largely rely on secondary data that are either published by state institutions or supplied by industries, which hardly have accompanying metadata. From our assessment of the situation, the practice of using qualitative approach to assess uncertainty, although it is not exactly what has been recommended in the tier 1, will have to be continued until such a time that Ghana is capable of adopting the quantitative approach. We estimate that it will take a minimum of 4 years (two inventory cycles) to be able to at least adopt the tier 1 quantitative assessment of uncertainty. In the meantime, the next two inventory cycles must focus on how to improve consistency in the use of qualitative approach in the uncertainty assessment. We suggest the following for the consideration of the inventory compilers:

Document all assumptions and justifications behind the selection of percentage thresholds of uncertainty range assign to activity data and emission factors.

As much as possible, do not apply the percentage threshold of uncertainty range on the basis of the data sources, but consider a probe further if the data provider has any additional information that helps to better understand how the data was generated and error associated with it.

of 48

In assigning uncertainties range, emission factors and activity data must be treated separately for each sector. This is because the two dataset must have

been generated using different set of conditions and assumptions.

If default IPCC emission factors are used, as much as possible use the uncertainty range that are associated with the factors. Should you get to use plant or site specific emission factors probe further to see if it is possible to get additional information from the data provider to help you understand the possible sources of errors. The additional information that could be relevant in understanding the circumstances the data was generated and how possible errors have been addressed include the following; measurement or sampling protocols, descriptive statistical analysis (standard deviation, sampling error or standard error; criteria or method of identification and exclusion of data/ outliers from the results etc. If the data providers have documentary proof of how errors associated with the data they have provided give priority to that data.

For activity data, because there is high likelihood that wide variety of methods may have been used to generate them by the data providers, it is important that, extra steps are taken to at least, identify the source, and if possible, discuss with the data providers to understand how the activity data was generated and if a special technique has been used already to process the data. .

In addition, the discussion could further identify the weakness in the data being provided and how it must be addressed. What was discussed and agreed on, the next actions to take, the weaknesses in the data and what need to be done must be documented transparently. Although the information that data owners will provide may not enable the inventory compilers to estimate tier 1 uncertainty assessment, at least, it will help to inform the inventory compilers in taking a decision on the uncertainty range in a transparent manner.

Key categories deserve special attention. As much as possible, have a clear plan on how to deal with activities key categories in all the sectors. When dealing with the activity data for the categories bear in mind that they are the ones that contribute most to the inventory and therefore what uncertainties range assign to them will greatly affect the overall inventory uncertainty.

of 48

5.3.2 Steps envisaged towards adopting tier 1 uncertainty assessment Before the inventory compilers can use the IPCC tier 1 uncertainty assessment to estimate the overall inventory uncertainty, it is important that clear steps and timelines are identified and followed. Although the tier 1 approach is the simplified method of estimating uncertainties it is far an improvement of the use of qualitative approach. The tier 1 approach is a simplified error propagation method to assess the confidence range of GHG emissions. It is usually a blend statistical calculation aided by expert judgment. It is an excellent tool for understanding the main contributors to errors; it also enable targeting large contributing sources for more intense data collection and helps in devising strategies to improve GHG inventories. This information should help prioritize efforts to improve the precision of inventories in future, and can show how the overall and trend uncertainties respond as the uncertainties in individual source categories are reduced. The tier 2 approach is based on the Monte Carlo Analysis which uses complex statistical packages such as “R software” or “Stata software” tools to implement it. For the tier 1 approach an excel spreadsheet tool can be used to estimate the uncertainties. The inventory compilers also use the IPCC software to automatically generate the uncertainties once the basic error propagation range for each activity and emission factors are inserted into the software. GHG emission is the sum of products of AD and EF. The uncertainty of emissions from a particular source is obtained by calculating and applying the uncertainty associated with the source’s emission factor, and the uncertainty of activity data. The Good Practice Guidance (2000) requires uncertainty of emissions from a source to be calculated using the method given below:

U = √𝑈EF2 + 𝑈A2 U: Uncertainty of the emissions of the source (%) UEF: Uncertainty of the emissions factor (%) UA : Uncertainty of the activity data (%)

Based on the basic parameters needed to quantify uncertainties of the inventory, we recommend the following steps:

Work with data providers, publish metadata for activity data where that exist but are not made available to the public. Where metadata do not exist, work with data providers to design simple sampling protocol for the collecting data.

of 48

As much as possible, collect all available background information especially on how the data was collected, who collected the data and for what purpose, sampling frame, any statistical analysis, application of any special pre-processing technique etc.

For EF, once the IPCC factors are used in the inventory, use the uncertainty range that is accompanying the data. In case plant specific emission factors are used, request for background information including protocols direct measurement both on the field and in the laboratory.

Appoint uncertainty management lead person in the inventory, preferably from mathematics or statistics department in any public university, over the design and implementation of the steps to adopt the tier 1 approach. The uncertainty lead must be resourced to follow through the reforms that need to take place before the tier is adopted. In addition, he/she must work with the inventory compilers and the sector leads, to ensure that the data collection is conducted having in mind the important data points that are needed for the tier. He/she should also lead in all the training programmes.

of 48

References

1. Intergovernmental Panel on Climate Change. (2006).Intergovernmental Panel on Climate Change national greenhouse gas inventories programme; 2006 IPCC guidelines for national greenhouse gas inventories. Available at http://www.ipcc.ch/pdf/activity/2006gls-brochure.pdf. Accessed on 15/06/2015.

2. Thompson, V. (2011). Setting up a Sustainable National GHG Inventory System. Available at http://www.cdredd.org/sites/default/files/users/common/2011_workshop_mauritius/SNIS%20Thompson_03-7-11.pdf. Accessed on 02/06/2015.

3. Environmental Protection Agency. (2011). National Greenhouse Gas Inventory Report for 1990-2006. Available at http://unfccc.int/resource/docs/2007/arr/nir_1990-2006.pdf. Accessed on 02/07/2015.

4. Intergovernmental Panel on Climate Change. (2001). Good Practice Guidance and Uncertainty Management in National Greenhouse Gas Inventories. Available at http://www.ipcc-nggip.iges.or.jp/public/gp/english/. Accessed on 04/06/2015.

5. United Nations Framework Convention on Climate Change. (2000). Review of the implementation of commitments and of other provisions of the convention; UNFCCC guidelines on reporting and review Available at http://unfccc.int/resource/docs/cop5/07.pdf. Accessed on 02/06/2015.

6. United Nations Framework Convention on Climate Change. (2013). Introduction to the UNFCCC biennial update reporting guidelines for parties not included in annex I to the convention. Available at http://www-gio.nies.go.jp/wgia/wg11/pdf/2_2%20UNFCCC%20BUR.pdf. Accessed on 02/06/2015.

7. United Nations Framework Convention on Climate Change. (2014). Modalities and guidelines for International Consultation and Analysis (ICA). Available at http://unfccc.int/national_reports/non-annex_i_natcom/cge/items/8621.php. Accessed on 02/06/2015.

8. United Nations Framework Convention on Climate Change. (2014). Guidelines and manuals for the preparation of non-Annex I national reports and International Consultation and Analysis. Available at http://unfccc.int/national_reports/non-annex_i_natcom/guidelines_and_user_manual/items/2607.php. Accessed on 02/06/2015.

http://www.ipcc.ch/pdf/activity/2006gls-brochure.pdf

http://www.cdredd.org/sites/default/files/users/common/2011_workshop_mauritius/SNIS%20Thompson_03-7-11.pdf

http://www.cdredd.org/sites/default/files/users/common/2011_workshop_mauritius/SNIS%20Thompson_03-7-11.pdf

http://unfccc.int/resource/docs/2007/arr/nir_1990-2006.pdf.%20Accessed%20on%2002/07/2015

http://unfccc.int/resource/docs/2007/arr/nir_1990-2006.pdf.%20Accessed%20on%2002/07/2015

http://unfccc.int/resource/docs/cop5/07.pdf.%20Accessed%20on%2002/06/2015

http://unfccc.int/national_reports/non-annex_i_natcom/cge/items/8621.php

QA/QC Plan and Guidance on Uncertainty Management for ... · involved in conceptualizing,...

Documents

Transcript of QA/QC Plan and Guidance on Uncertainty Management for ... · involved in conceptualizing,...