Report on the APLAC Workshop on Reference Material ...

APLAC RMP / PTP workshop Nov. 2016 of 52

Report

on the

APLAC Workshop on

Reference Material Producer and Proficiency Testing Provider Accreditation

Held in Hong Kong, China

7-10 November 2016


Contents Section Page Background and objectives 3

Day One - summary 4

Day Two - summary 6

Day Three – summary 7

Day Four – summary 8

Annex 1 – List of participants 9

Annex 2 – Programme for the workshop 11

Annex 3 – Summary of outcomes and resolutions 17

Annex 4 – Discussion papers 19


Background and Objectives Since the last workshop undertaken in 2013, held in Tokyo, there have been changes with regard to the standards used by Accreditation Bodies for accrediting Reference Material Producers (RMPs) and ILAC Resolution (October 2016) to establish and ILAC MRA for RMPs. ILAC has adopted ISO 17034: General requirements for the competence of reference material producers and have an implementation period of 3 years (1st November 2016) after which accredited RMPs need to have demonstrated compliance with ISO 17034:2016. ILAC has previously resolved to have an MRA for Proficiency Testing Providers (PTPs) which APLAC has already applied to be a signatory of, EA having already been evaluated. The objectives of the workshop were to:

Provide training and a forum for discussion in relation to the new ISO 17034 standard and to look at the important issues of the application of ISO 17034

Review the inter-relationship between ISO 17034 and the other ISO REMCO documents ISO Guide 30: Reference materials – Selected terms and definitions, ISO Guide 31: Reference materials – Contents of certificates, labels and accompanying documentation and ISO Guide 33: Reference materials – Good practice in using reference materials

Review the revision of ISO Guide 35: Reference materials - General and statistical principles for certification, currently in draft with publication pending

Provide training and a forum for discussion on measurement uncertainty of Certified Reference Materials (CRMs) and the metrological traceability statement in CRM certificates

Discuss whether APLAC TC 008 APLAC Requirements for and guidance on the accreditation of a Reference Material Producer and the resulting scope of accreditation is still required with regard to the draft ISO 17011: Conformity assessment - General requirements for accreditation bodies accrediting conformity assessment bodies changes and ISO 17034 and what recommended changes (if any) should be considered

Formatting of scopes for accredited RMPs Discussion and training in the application of ISO/IEC 17043: Conformity assessment – General

requirements for proficiency testing Discussion and application of ISO 13528:2015: Statistical methods for use in proficiency testing by

interlaboratory comparison Formatting of scopes for accredited PTPs

The workshop was hosted by HKAS of Hong Kong, China. Altogether 32 participants from 17 economies took part in the workshop. Participants included representatives from 20 member accreditation bodies of APLAC, European Cooperation for Accreditation (EA) and Inter American Accreditation Cooperation (IAAC) as well as accredited organisations. Six speakers were invited from China, Germany, Hong Kong China, Japan and the US. A list of participants is provided in Annex 1. Ms Anne Hofstra of IANZ was appointed as the rapporteur of the workshop. APLAC sponsored the lunch and refreshments of the event while attendees were responsible for their own travel and accommodation. The full programme of the workshop is given in Annex 2. Resolutions and outputs are summarised in Annex 3. To assist discussion, the workshop coordinator requested participants to submitted discussion papers on issues they would like to raise at the Workshop. A total of 24 discussion papers were received. Copies of the discussion papers submitted by the participants of the workshop with the resolutions / actions arising are included in Annex 4 of this report. Training materials used in the workshop and participants’ presentations during the experience sharing session are also provided as attachments.


DAY ONE (7th November 2016) Session One – Welcome and Introductions Mr WW Wong from HKAS (Executive Administrator) welcomed the participants of the workshop to Hong Kong and the following guest speakers and facilitators of the sessions for the next four days:

Dr Wolfram Bremser, from BAM Dr Takeshi Saito, from NMIJ Professor Xiaohua Lu, from NIM China / APMP Mr Paul McMullen, from NATA Dr Dan Tholen, from Dan Tholen Statistical Consulting Mr Robert Knake, from A2LA Mr Dennis Lee, from Standard and Calibration Laboratory of HKSAR, China Dr Samuel Lo, from Government Laboratory of HKSAR, China

Dr Daria Wong from HKAS outlined the objectives of the workshop, the material to be covered and the deliverables, in terms of resolutions coming out of the four day workshop. Participants each introduced themselves, outlining their role in RMP / PTP and accreditation activities.

Session Two – Accreditation Criteria for RMP Important issues in the application of ISO 17034 – Part One Speaker: Dr Wolfram Bremser Dr Bremser spoke about the changes between the ISO Guide 34 and new requirements in ISO 17034; a hard copy of the PowerPoint presentation was provided to the participants. The following topics generated further discussion / clarification of the requirements and their application to the RMP: 4.1 Contractual matters Note 2 4.2 Impartiality 5.0 Structural requirements 8.0 Management system requirements The following topics were also covered:

Control of documents and records Management review Internal audits Corrective actions improvement Customer feedback

Important issues in the application of ISO 17034 – Part Two and Part Three Speakers: Dr Takeshi Saito and Dr Wolfram Bremser This session covered sections 6.0 Resource requirements and 7.0 Technical and productions requirements. A hard copy of the PowerPoint presentation was provided to the participants. The session was based on the FDIS of ISO 17034 as the standard how been published only days before the workshop got underway. One of the more significant changes between ISO Guide 34 and ISO 17034 is the requirement to use ISO Guide 35 has been removed and alternatives can now be implemented by RMPs. RMPs that opt to use alternative statistics will of course need to demonstrate that these are appropriate. It was also highlighted that unless specifically stated otherwise all the requirements of ISO 17034 apply to the production of Reference Materials (RMs) and CRMs.


The following topics generated further discussion / clarification of the requirements and their application to the RMP: 6.0 Resource requirements 6.2 Subcontracting 7.1 General requirements 7.4 Material handling and storage 7.6 Measurement procedures 7.9 Metrological traceability of certified values 7.12 Characterisation 7.17 Management of non-conforming work The following topics were also covered:

Provision of equipment, services and supplies Facilities and environmental conditions Production control Material processing Measuring equipment Data integrity and evaluation Assessment of homogeneity Assessment and monitoring of stability Assignment of property values Reference material documents and labels Distribution service

The tabled discussion papers for this session were held over until Day 2. Inter-relationship between ISO 17034 and other ISO REMCO documents

ISO Guide 30: Reference materials – Selected terms and definitions ISO Guide 31: Reference materials – Contents of certificates, labels and accompanying

documentation ISO Guide 33: Reference materials – Good practice in using reference materials

Speaker: Dr Takeshi Saito This session covered the relationship between the ISO Guides listed above and the new ISO 17034 standard. A hard copy of the PowerPoint presentation was provided to the participants. Dr Saito informed the workshop that the contents of ISO Guide 32: Calibration in analytical chemistry and use of certified reference materials is now covered by ISO Guide 33. The following topics generated further discussion / clarification of the requirements and their application to the RMP:

ISO Guide 30 ISO Guide 31

ISO Guide 35 Revision Speaker: Dr Wolfram Bremser This session covered the content of the draft Guide 35 and identified what the primary changes are in the updated document. A hard copy of the PowerPoint presentation was provided to the participants.


DAY TWO (8th November 2016)

Session Three – Issues related to RMP accreditation Speaker: Professor Xiaohua Lu The presentation from Prof. Xiaohua Lu focussed on the metrological traceability and measurement uncertainty of CRMs. Material covered included the different approaches for that could be applied to calculate measurement uncertainty including measurement uncertainty from homogeneity, characterisation and stability. A hard copy of the PowerPoint presentation was provided to participants. Revision of APLAC TC 008 Facilitator: Mr Paul McMullen Mr McMullan briefly covered the background of RMP accreditation in relation to the ILAC resolutions and the APLAC documentation used for accreditation, which was all set out in the handouts provided. With the DIS of ISO 17011 now including RMP activities, it’s time to review APLAC TC 008 and decide whether it’s required and if so what needs to be removed / amended / updated. The working group decided to retain TC008 but review the content. See the outcomes in Annex 3 for a summary of the recommended changes. A copy of the notes taken with regard to the areas requiring review and update was provided to the facilitator to help ensure the comments from the working group with regard to the desired changes in TC 008 are captured. Formulating Scopes of Accreditation for RMP Facilitator: Mr Paul McMullen The content of scopes for accredited RMP includes of detail much of which originated from ISO Guide 34 itself. There are now requirements for scopes of RMPs included in the DIS of ISO 17011. The current system for classification was deemed to be restrictive with many Accreditation Bodies using the system in the now withdrawn ILAC G12. Many Accreditation Bodies are also keen to see the requirements set out in APLAC TC 008 and IAAC MD28 reduced and allow more flexible scopes, bearing in mind that a more flexible scope will require more assessment effort to confirm the competency of the RMP to produce conforming CRMs/RMs. The ILAC document G18: Guideline for the Formulation of Scopes for Accreditation for Laboratories is under review and believed to be expanded to include scopes for both RMPs and PTPs. The working group after discussion about the use of APLAC TC 012 recommended it be withdrawn. See outcomes from the meeting in Annex 3. The conclusion, after much discussion, on serial subcontracting is set out Annex 3. The session finished up with a review of the Discussion Papers provided for RMP. The outcomes from the discussions are recorded on the papers in Annex 4.


DAY THREE (9th November 2016)

Session Four – Accreditation criteria for PTP Speaker: Mr Dan Tholen A hard copy of the PowerPoint was provided to the working group participants. Mr Tholen’s presentation covered the application of ISO 17043 and ILAC P9: ILAC Policy for the Participation in Proficiency Testing Activities. The latter document generated much discussion as to whether an accredited laboratory (to ISO/IEC 17025) within a PTP was required to participate in an independent PT programme. Also included for discussion was the application of ILAC P13: Application of ISO/IEC 17011 for the Accreditation of Proficiency testing Providers, APLAC TC 006: Proficiency Testing Frequency Benchmarks and EA-4/18: Guidance of the level and frequency of proficiency testing participation. Other topics arising from the presentation included the more frequently occurring ‘on-demand’ proficiency programmes where the PTP has a stable artefact which can be sent to the participants on request. This session also covered Discussion papers #7, 12 and 14. See Annex 4. Session Five – Statistical Methods for Proficiency Testing Speaker: Mr Dan Tholen A hard copy of the PowerPoint was provided to the working group participants. Mr Tholen provided an update on the changes to ISO 13528: 2015 Statistical methods for use in proficiency testing by interlaboratory comparison and ISO TC 69: Application of statistical methods.

Session Six – Proficiency Testing for Calibration Speaker: Mr Dennis Lee Mr Lee spoke to the working group of his organisations experiences of becoming an accredited proficiency testing provider offering PT for calibration laboratories and types of programmes his organisation offers. The presentation included not only the accreditation process from the PTPs perspective but also some of the considerations required for circulating the artefacts. Session Seven – Other Issues related to Proficiency testing the PTP accreditation Speaker: Prof. Xiaohua Lu Prof Xiaohua Lu addressed the working group on the Metrological traceability of the assigned values of PT. The presentation covered such topics as the application of ISO 13528, using metrologically traceable reference values and summaries of some of the APLAC-APMP joint PT programmes that have been run. A hard copy of the PowerPoint was provided to the working group participants. Formatting scope of accreditation for PTP Facilitator: Mr Robert Knake Mr Knake facilitated general discussion on the scope of accreditation for PTP with regard to the proposed amendments to ISO 17011, APLAC PT documents and Discussion papers #8, 10 and 17. The question was raised, do we need an APLAC TC document for the accreditation of PTP. The working group did not see a need to develop a document at this point in time.


DAY FOUR (10th November 2016) Facilitator: Mr Robert Knake The remainder of the discussion papers for PTP were covered with actions / outcomes recorded on the papers – see Annex 4.

Session Eight – Experience sharing Facilitators: Mr Dan Tholen and Dr Daria Wong This session was for each of the participating Accreditation Bodies and invited speakers to give a presentation on the types / numbers of RMPs and PTPs they accredit and types of deficiencies they are identifying during the accreditation / assessment process. A presentation was given by Dr Samuel Lo on becoming an accredited RMP / PTP and the types of programmes they offer including some of the issues they encountered. A presentation was also given by Ms Tsevelmaa Ganbat from CGL Mongolia on their experiences as an accredited RMP/PTP. Session Nine – Conclusion Facilitators: Mr Paul McMullen and Mr Robert Knake Mr McMullen and Mr Knake provided an overview of the outcomes of the four days for RMP and PTP respectively. A resolution was made that another workshop be held in two years’ time (2018). Acknowledgement and thank you was made to Dr Daria Wong and her team for organising the workshop and hosting the dinner. Acknowledgement and thank you the presenters for their time and contribution to the workshop, and thank you to the participants.


Annex 1 - List of Participants

Name of participants (Surname in bold) Organisation

Mr Ibrahim ALHASHAF GSO

Mr Victor Hugo ANGELES AGUILAR EMA

Dr Takashi ARAI IAJapan

Mr Douglas BERG PJLA Inc

Dr Fiona CHAN Wan-yin HKAS

Dr Alex CHAN Yau-chi HKAS

Ms CHANG Shu-Fen TAF

Ms Cherry CHENG Chia-Fen TAF

Dr Ursula ELLERBECK DAkkS

Ms Tsevelmaa GANBAT CGL

Mr Sergio Nicola GUZZI ACCREDIA

Dr Bill HIRT ANAB

Dr HO Chun-wah HKAS

Ms Anne HOFSTRA IANZ

Mr Morio HOSAKA JAB

Ms KAO Pao-chu TAF

Mr Robert Lee KNAKE A2LA

Mr Mahmut KÖSEOĞLU TURKAK

Ms LEE Jia-juan SAC

Mr Juan Manuel LÓPEZ NAVA EMA

Mr Marlon MACADAMIA PAB

Mr Paul MCMULLEN NATA

Dr Hiroaki OHTAKA IAJapan

Ms Bhumi Suresh RAJYAGURU NABL

Ms Anita RANI NABL

Ms Bussaya RATTANASUPA BLA-DSS

Ms Vera SHEVELEVA AAC Analitica

Ms Chanya SUTTHAJINDA NSC ONSC

Mr Edmund TSE Siu-chuen HKAS


Dr Daria WONG Ching-hang HKAS

Mr Natarajan VENKATESWARAN NABL

Mrs Chantarat VORASAPAVIT BLA-DSS

Invited speakers (Surname in bold) Organisation

Dr Wolfram BREMSER BAM

Mr Dennis LEE Standard and Calibration Laboratory of HKSAR China

Dr Samuel LO Government Laboratory of HKSAR China

Prof. Xiaohua LU NIM China/APMP

Dr Takeshi SAITO NMIJ, AIST

Mr Dan THOLEN Dan Tholen Statistical Consulting


Annex 2

APLAC Workshop on RMP and PTP accreditation

PROGRAMME

Falcon Room, Gloucester Luk Kwok Hotel, Hong Kong, China

DAY ONE (7 November 2016)

Session One - Welcome Address and Introduction

Workshop Co-ordinator : HKAS

09:00-09:30

Welcome remarks by Mr W W Wong, Executive Administrator, HKAS

Introduction by Dr Daria Wong, HKAS

Objectives

Content and format

Expected deliverables

Session Two – Accreditation criteria for RMP

Speaker : Dr Wolfram Bremser, BAM

09:30-10:30

Important issues of application of ISO 17034 – Part One

New Structure

Terms and definitions

General (Clause 4) and Structural (Clause 5) requirements

Management (Clause 8) requirements

Coffee Break (10:30-10:45)

Speaker : Dr Takeshi Saito, NMIJ and Dr Wolfram Bremser

10:45-12:30

Important Issues of the application of ISO 17034 – Part Two

Resource requirements (Clause 6)

Technical and Production requirements (Clause 7)


Lunch (12:30-13:30)

Speaker: Dr Wolfram Bremser

13:30-14:45

Important Issues of the application of ISO 17034 – Part Three

Technical and Production requirements (Clause 7) (cont’d)

Discussion paper # 1&2 (ANAB), #11 (TAF)

Speaker: Dr Takeshi Saito

14:45-15:30

Inter-relationship between ISO 17034 and other ISO REMCO documents

ISO Guide 30

ISO Guide 31

ISO Guide 33

Q&A

Discussion paper #16 (ACCREDIA)


Speaker: Dr Wolfram Bremser

15:45-17:00

ISO Guide 35 revision

Q&A

Discussion paper (if any)

End of Day One


DAY TWO (8 November 2016)

Session Three – Issues related to RMP accreditation

Speaker: Prof. Xiaohua Lu, APMP

09:00-10:00

Measurement uncertainty of CRMs

Metrological traceability statement in CRM certificates

- ISO/TR 16476 Reference Materials - Establishing and expressing

metrological traceability of quantity values assigned to reference

materials

Discussion paper (if any)

Facilitator: Mr Paul McMullen, NATA

10:00-10:45 Revision of APLAC TC 008


11:00-12:30

Revision of APLAC TC 008 (cont’d)

Discussion paper #3 (ANAB), #15 (JAB)

Policy on displaying accreditation symbol on labels of RMs

Discussion paper #4 (NATA)

Lunch (12:30-13:30)

Facilitator: Mr Paul McMullen

13:30-15:15

Formulating scope of accreditation for RMP

Discussion paper #5 & 6 (NATA), #13 (A2LA)

Application of ILAC P14 and APLAC 008 to the scope of accreditation

Revision of ISO/IEC 17011 – Potential impact on RMPs’ scope and

accreditation


Facilitator: Mr Paul McMullen

15:30-17:00

Formulating scope of accreditation for RMP (Cont’d)

APLAC TC012 on chemical reference materials for calibration – is it still

needed?

‘Serial subcontracting’ of RMP

Other issues related to the accreditation of RMPs

Discussion papers

Summary and Wrap up

End of Day Two


DAY THREE (9 November 2016)

Session Four – Accreditation criteria for PTP

Speaker: Mr Dan Tholen, Dan Tholen Statistical Consulting

09:00-09:45

Application of ISO/IEC 17043

ILAC P9 & P13

APLAC PT006

Discussion paper #7 (BLA-DSS) and related paper #24, #12 (TAF), #14

(A2LA), #18-23 (BELAC)

Session Five – Statistical Methods for PT

Speaker: Mr Dan Tholen

09:45-10:30

ISO 13528:2015

- Highlight of changes

- Application and Examples


Speaker : Mr Dan Tholen

10:45-12:30

ISO 13528:2015 (Cont’d)

- Highlight of changes

- Application and Examples

Q&A

Discussion papers (if any)

Lunch (12:30–13:30)

Session Six – PT for calibration

Speaker : Mr Dennis Lee, Standard and Calibration Laboratory, Hong Kong, China

13:30-14:15

Approach and challenges of calibration PTs

Q&A


Session Seven – Other issues related to PT and PTP accreditation

Speaker: Prof. Xiaohua Lu

14:15-15:00

Traceability of the Assigned Values of PT

Q&A




Facilitator: Mr Robert Knake, A2LA

15:15-17:00

Formatting scope of accreditation for PTP

Discussion paper #8 (NATA)

Updates on revision of ISO/IEC 17011 – potential impact on PTP’s scope

and accreditation, and APLAC PTs

Discussion paper #17 (ACCRDEDIA)

Discussion paper #10 (NABL)

Need for Inspection PTs

Need for an APLAC TC document on PTP accreditation?

End of Day 3

APLAC RMP/PTP Workshop Nov 2016 of 52

DAY FOUR (10 November 2016)

Session Eight – Experience sharing and issues for discussion

Facilitators: Mr Dan Tholen and Dr Daria Wong

Presenters: Representatives from ABs, other regional representatives, accredited RMPs and

PTPs, other interested participants

09:00-12:30 (Coffee break in-between)

Sharing of experience in assessing and accrediting RMPs and PTPs,

including top 5 deficiencies cited for RMP and/or top 5 deficiencies cited for

PTP assessments (about 15 min for each AB)

- Presenters from: A2LA, AAC Analitica, ACCREDIA, ANAB, BLA DSS,

DAkkS, ema, GAC, HKAS, IA Japan, IANZ, JAB

Lunch (12:30-13:30)

13:30-16:30 (Coffee break in-between)

Sharing of experience in assessing and accrediting RMPs and PTPs,

including top 5 deficiencies cited for RMP and/or top 5 deficiencies cited for

PTP assessments (about 15 min each) (cont’d)

- Presenters from: NABL, NATA (Discussion paper #9), NSC ONSC, PAB,

PJLA Inc, SAC, TAF, TURKAK

Sharing of experience from accredited RMPs and PTPs (about 15 min each)

- Presenters from: CGL and the Government Laboratory of Hong Kong,

China (Representative: Dr Samuel Lo)

Group Photo (16:30)

Session Nine – Conclusion

Facilitators: Mr Paul McMullen and Mr Robert Knake

16:30-17:00

Wrap up and conclusion

End of Workshop


Annex 3

Summary of outcomes / resolutions from the workshop RMP Outcomes APLAC TC008 Recommendations arising from the review of APLAC TC008:

To include a scope in the document

To expand its coverage to include guidance and reference documents concerning the use of RMs e.g. references to ISO Guide 33 and ISO/TR 16476.

The working group recommends this document becomes more of a guidance only document removing many of the ‘shall’ statements where possible. APLAC may consider the review of MR-001 as a consequence, as TC-008 is currently a requirement document for the accreditation of RMP

To review section 2 taking into account that much of what is currently covered in section 2 is now covered by ISO 17034. Highlight the distinction between the conformity assessment types i.e. testing and RMP to avoid market confusion

To review section 3 with regard to referenced clauses in ISO 17025 contained in ISO 17034 that may be relevant for testing and measurement (calibrations), for example, retain the requirement for the accreditation body to determine which clauses of ISO 17025 apply to the RMP under assessment and remove the repeated requirements

To review section 4 and 5 updating this with regard to the new ISO 17034 and remove the repeated criteria as appropriate. Volunteers for this task are Dr Bill Hirt (ANAB), Mr Rob Knake (A2LA), Mr Doug Berg (PJLA), Mr Paul McMullen (NATA), Dr Daria Wong (HKAS), Ms Kao Pao-chu (TAF) and Ms Bhumi Suresh Rajyaguru (NABL)

To remove Annex A

To amend section 6 and make reference to ISO (DIS) 17011 and ILAC G18 with regard to the formulation of scopes. ILAC G18 is under review and it is understood to be expanded to include scopes for PTP and RMP. The working group acknowledges the scope requirements in ISO (DIS) 17011 may change in the final version

To remove the requirements as currently documented for subcontracting as this is adequately covered by ISO 17034. No additional commentary is required.

APLAC TC012 After group discussion and subsequent vote it is recommended to withdraw TC012:

It appears that only a small number of accreditation bodies referencing this document The content of ISO Guide 33 covers much of this document There is some overlap with ILAC P10

General Serial subcontracting It was decided this was not allowed under 6.2.3 of ISO 17034 i.e. the subcontractor cannot subcontract the work they have been contracted to carry out without the approval of the RMP. This was in ISO Guide 34 previously. Section 6.2.3 “RMP shall not subcontract the following processes: the selection of subcontractors”.


PTP Outcomes Resolution #1 There was discussion around of measurement audits as part of the assessment process and whether this was considered PT and therefore an activity of a PTP and not appropriate for an Accreditation Body. EA have discussed whether this type of activity (i.e. taking samples or artefacts’ to a CAB for analysis during the assessment process) is witnessing or ILC activity. Some Accreditation Bodies however consider witnessing as a normal part of the assessment process to review testing and other refer to this as an assessment tool. The working group would like to recommend to the ISO 17011 working group that clear guidelines should be given in ISO 17011 (maybe as a Note) as to whether measurement audit is acceptable as part of assessment activity and if measurement audit is considered a PT activity conducted by PTP. Resolution #2 There was discussion on the need to develop a TC type document for the accreditation of PTP. The working group did not see a need to develop a document at this point in time. Any Accreditation Body seeing a need for this type of guidance could contact APLAC directly with a proposal scoping out the document. Resolution #3 For the outcome of the discussion on scopes – it was decided to refer to ILAC G18 (in particular once expanded to include PTP and RMP), with each Accreditation Body able to define categories as they see fit. The requirements are now also defined in the DIS of ISO 17011 so no further action is required by the group at this time. Future training / workshops Resolution #4 A resolution was made that another workshop be held in two years’ time (2018).


Annex 4

Discussion papers

Discussion Paper #1

APLAC Workshop on RMP and PTP Accreditation

7 – 10 November 2016 Hong Kong, China

Discussion Paper

Name of party submitting the paper: Bill Hirt / ANAB – USA - 1

Discussion topic related to: RMP accreditation Related standard/APLAC document and clause number (where applicable):

ISO Guide 34, section 5.17

Question/Discussion item: We have found that many Certificates of Analysis for chemical RMs in many ISO 17025 laboratories list uncertainties as “expanded uncertainties” and not Ucrm uncertainties. Unless the Certified Value uncertainties specify Ucrm, we in the public are unaware whether. Expanded uncertainty is conforming to ISO 17025 or to ISO Guide 34 and ISO Guide 35. How do we improve this uncertainty about uncertainties? Outcome: There was general discussion within the working group about the understanding of the uncertainty values on certificates for chemical RM’s and that further education of the users will be / is required. No decision or further action was required.


Discussion paper #2



Discussion Paper

Name of party submitting the paper: Bill Hirt / ANAB – USA

Discussion topic related to: RMP accreditation Related standard/APLAC document and clause number (where applicable):

ISO Guide 34, section 5.17

Question/Discussion item: RMP accreditations represent both RMs and CRMs. We understand that the certificates for RMs may represent Product Information Sheets or other designation of their documentation. It is entirely possible however that many RMs may be issued accompanied with Certificates of Analysis, very similar to CRM certificates. They simply may not have any Certified Values with verified traceability. This may present confusion in the marketplace on distinguishing RMs from CRMs since each of them may have CoA reports and certificates that can legitimately use the AB symbol for RMP provider. Could we discuss whether we suggest any caveats or clarifying designation we want on CoA’s to distinguish RMs from CRMs … beyond looking for Certified Values? Outcome: Following discussion in the working group it was established that as ISO Guide 31 covers the requirements to differentiating between CRMs and RMs on certificates, no further documentation is considered necessary at this time.


Discussion paper #3



Discussion Paper

Name of party submitting the paper: Bill Hirt / ANAB – USA - 3

Discussion topic related to: RMP accreditation Related standard/APLAC document and clause number (where applicable): TC-

008 2015 particularly section 6

Question/Discussion item: APLAC TC-008 has morphed over the past few revisions from a Guidance document to a Requirementsdocument with over 85 “shalls” and over 60 “shoulds” plus over a dozen phrases like “the RMP will needto …”. In addition, since most of us look forward very anxiously to have these accreditations allow the use of the ILAC mark on their RMP certificates, it is highly unlikely that ILAC will approve this use oftheir mark if regions such as APLAC have added requirements that are not in the ISO / REMCOstandard, and other regions do not accredit their RMPs to the same requirements. Shouldn’t we removeALL additional requirements in our APLAC document and return it to a Guidance Document, especiallysince we are just having the new revision of ISO 17034 to implement? As a secondary discussion point, should we decide to maintain the requirements in TC-008, we would like to more thoroughly discuss two specific areas: first, the significance and implications and reporting requirements for RVC (reference value capabilities) of RMPs … and second, if CRM uncertainties were to be reported on scopes of RMP accreditation, how is this value-added, since the few examples we see on AB websites and most proposals by RMPs from our AB have suggested only a generic maximum cap of uncertainty, like less than 5%? Outcome: The working group decided to revise, not remove, APLAC TC 008. See Annex 3 for the recommended changes.


Discussion paper #4



Discussion Paper 5 NATA

Name of party submitting the paper: NATA Discussion topic related to: Can the Accreditation Body logo (Symbol) be used on Reference Material Labels. (Session three) Related standard/APLAC document and clause number (where applicable): ISO Guide 34 (17034), ISO Guide 31 Clause 6, ‘Labels’ ILAC-IAF Resolution 2003-18 Conformity Assessment Services The Annual Meeting, acting on the recommendation of the Technical Committee, in support of the principle that IAF, its members and their constituents, should avoid actions that create confusion or conflict of interest between the respective roles of accreditation bodies and certification/registration bodies, resolved that IAF will not endorse

1. the provision of any conformity assessment services by Accreditation Body members of IAF (other than insofar as the accreditation of Conformity Assessment Bodies may be termed a conformity assessment service)

2. the provision of any conformity assessment services by accredited Conformity Assessment Bodies to denote conformity with any of the standards that are used as the basis for accreditation.

Note: Currently, services identified as conformity assessment services are:

- Calibration (ISO/IEC 17025)

- Testing (ISO/IEC 17025 and ISO 15189)

- Inspection (ISO/IEC 17020)

- Management System Certification (future ISO/IEC 17021)

- Personnel Certification (ISO/IEC 17024)

- Product Certification (ISO/IEC Guide 65) Question/Discussion item: The point of origin of this discussion paper is a question coming from EA. Is it acceptable for a RMP to include the accreditation logo on a (C) RM, including on a label? While this question may seem simple at first, it raises many issues of which ILAC and IAF have overall control (for example see IAF resolution 2003-18). Issues to consider as part of the discussion; While it is permissible to attach a logo on Calibration and Inspection labels, doing so on RMs

could get confused with product certification. Some (at EA) feel that it is permissible (obviously) for the logo to go on the certificate which can

be affixed to the CRM (or its packaging) but not simply to put the logo on the item without linkage to the property value information. i.e. when the logo is affixed to RM packaging, it shall


be accompanied by a statement that affiliates the logo to the reported specified property only. Does 17034 cover both production of the material as well as assignment of property value? Should we acknowledge RMP as a unique for of accredited activity (conformity assessment

activity) as it crosses over into calibration (characterization of a property value), testing (homogeneity, stability and controls applied during production) and a bit of certification (a CRM is by title ‘certified’).

A certification body usually applies/promotes their own logo to a product and not the logo of their AB.

Outcome: The changes to ISO 17011 (proposed in the DIS) set out the requirements for the use of the accreditation symbol and need to ensure it does not imply certification or approval of products, so the decision was no further action / documentation is required. This is adequately covered in the DIS of ISO 17011. The accreditation symbol should not be put on the label of RM alone by itself. It may however go on the certificate/documentation of the RM.


Discussion paper #5




Name of party submitting the paper: NATA Discussion topic related to: Formulating a scope of accreditation for RMP (Session Three), structure of material categories. Related standard/APLAC document and clause number (where applicable): Appendix B of ILAC G12 (Withdrawn?) APLAC TC 008 clause 6.2 Question/Discussion item: Background

The categories and sub-categories of reference materials given in Appendix B of ILAC G12 were primarily divided into technical disciplines (chemical inorganic, chemical organic, biological, mechanical engineering/sciences) and then subdivided into either material type or reported property value, with inconsistency in the resolution for each subcategory. This may lead to inconsistency in application as many materials can relate to multiple subcategories. An Alternative approach?

Reference materials are often marketed to an industry group rather than to a technical discipline. Industry groups include Environmental Testing, Food Safety, Mining, Manufacturing, Construction Materials, etc. As an alternative, the structure of categories for reference materials could take a uniform designed based on;

Industry group Material category

Type of property value / characteristic

The type of property value / characteristic for the materials will then inform the technical discipline used in the evaluation of the material. For example;

Agribusiness Agrichemicals

Herbicides Pesticides Fertilizers

Veterinary drugs Steroids Growth promotants Antibiotics

Legal Controlled substances

Forensic drugs Sport doping control references Ethanol

Ethanol reference standards Aqueous ethanol standard Ethanol in nitrogen

Forensic reference materials Accelerants Explosives, primers Glasses Paints


Health and Healthcare Pathology Reference Materials

Bacteria Viruses Fungi, yeasts Parasites Blood and Blood Products Histology tissues; Cytology tissues; Body fluids

Pharmaceutical reference materials Antibiotics

For those reference materials used as a reference standard across any one of the industry groups, one of the industry groups could be titled ‘Reference Standards and Calibration references’. Other Industry groups may include;

Environment Food and Beverage Mining Construction Industry Industrial Products

For the next tier within the scope of accreditation the listing of property values and capabilities is covered by NATA Discussion Paper 3.

Outcome: With ILAC G18 currently under review and the understanding this will be expanded to include guidelines for scopes of RMPs and PTPs with each accreditation body to define categories as appropriate no further action is required. It was noted that the DIS of ISO 17011 now also includes scopes for RMPs.


Discussion paper #6




Name of party submitting the paper: NATA Discussion topic related to: Formulating a scope of accreditation for RMP (Session Three), the contents of a Scope of Accreditation Related standard/APLAC document and clause number (where applicable): APLAC TC 008 ISO/IEC DIS 17011 ILAC P14 Consistency of the contents of a scope of accreditation now appears to be managed via the revision of ISO/IEC 17011. This may negate the need for the detail provided in APLAC TC 008. ISO/IEC DIS 17011 Clause 7.8.2 states, The scope of accreditation shall, at least, identify the following; f) for reference material producers:

specific types of reference materials (certified, non-certified or both); the reference material matrix / artifact;

the property(ies) characterized; and the principle approach used to assign property(s) values.

Secretariat comment: REMCO informed CASCO secretariat that a change made at the 5th WG42 meeting was technically incorrect. The error was corrected by the secretariat by deleting the bullet point "range of assigned property values and for certified reference materials, range of assigned property values with associated uncertainties; and" and changing the last bullet from "the principle of the characterization techniques used to assign property(s) values" to see 4th bullet above. The Scope of ILAC P14; This document is applicable to calibration laboratories, reference measurement laboratories for laboratory medicine, and producers of certified reference materials that provide calibration and measurement services that refer to their accredited status under the ILAC MRA. May be read as RMPs accredited under ISO/IEC 17025 that provide characterization values used in production of a Certified Reference Material. Thus ILAC P14 would apply to the 17025 scope of accreditation for the characterization activity and not necessarily apply to an ISO/IEC 17034 scope of accreditation. The revision of 17011 is correct in not using the term CMC for CRMs. A Calibration and Measurement Capability is very different to a Property Value Capability. At the same time, by stating “the principal approach used to assign property(s) values” in a scope of accreditation is open to interpretation. Does this mean characterization technique or something more general, such as whether the characterization is undertaken internally or externally by an ILAC MRA accredited laboratory? I believe the intension is to provide an indication of property value capability which is not only based on the technique used in characterization but also informed by the RMPs material preparation capability, the stability/homogeneity of the material, the variation of the Stated Reference to which traceability is claimed and other factors. One approach would be simply to state the limits to the assigned property value where applicable. Outcome: The paper generated much discussion in conjunction with paper #5, with a similar outcome. See outcome for paper #5.


Discussion paper #7

APLAC Workshop on RMP and PTP Accreditation 7 – 10 November 2016

Hong Kong, China

Discussion Paper Name of party submitting the paper: Bureau of Laboratory Accreditation, Department of Science Service, Thailand Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO 13528: 2015 Annex B, B 2.4 ISO/IEC 17043: 2010 clause 4.8.2 i Question/Discussion item: 1. Assessment criteria for a homogeneity check (ISO 13528: 2015 Annex B, B 2.4)

- When σpt is not known in advance (such as when σpt is calculated from participants’ results, the PTP check for statistically significant differences between proficiency test item by using One-way ANOVA and shows that the result of F-test is not significant. Is it necessary to confirm that Ss ≤ 0.3σpt again after the σpt is calculated?

2. Proficiency Test Reports (ISO/IEC 17043: 2010 clause 4.8.2 i))

- According to clause 4.8.2 i), the standard required that “a clear description of the proficiency test items used, including necessary details of the proficiency test item’s preparation and homogeneity and stability assessment”

Does “the details” include the raw data of homogeneity and stability assessment? Outcome: From the discussion: 1. If σpt is satisfied then the sample difference are unlikely to affect the PT round; this is quite a tight

criteria. 2. The participant needs sufficient information to understand the report provided by the PTP. If the data

supporting the homogeneity and stability statements are not included in the report provided then it needs to be readily available to the participant should they request it.


Discussion paper #8




Name of party submitting the paper: NATA Discussion topic related to: Formulating a scope of accreditation for PTP (Session seven) Related standard/APLAC document and clause number (where applicable): EA-2/18 - Guidelines for Accreditation Bodies on the Contents of the Scopes of Accreditation for Proficiency Testing Providers, ISO/IEC DIS 17011 Question/Discussion item: Background

In August 2014 NATA revised the structure for the categories of proficiency testing schemes stated in the scope of accreditation.

In October 2015 European Accreditation released EA-2/18. Fortunately for NATA, the guidance provided in EA-2/18 is compatible with the approach taken by NATA. ISO/IEC DIS 17011 Clause 7.8.2 states, The scope of accreditation shall, at least, identify the following;

e) for proficiency testing providers: schemes that the proficiency testing provider is competent to provide; type of proficiency testing items; and the measurand(s) or characteristic(s) or where appropriate the type of measurand(s) or characteristic(s) that are to be identified, measured or tested.

NATA’s approach to formulating a scope for PTP

NATA’s approach to PTP scopes is similar to the approach taken for RMP scopes with a slight change in emphasis. While PT programs are fundamentally based on an item/artifact/materials being distributed, they are marketed to testing laboratories that generally specialize in a testing technical discipline, so more emphasis is given to technical discipline.

The scope of accreditation serves two functions. Firstly, it defines exactly which schemes the facility is accredited for. Secondly, it provides potential clients with information about available proficiency testing services.

Expressing the scope of accreditation

Scopes of accreditation will include essential information covering the following elements:

The discipline/industry group for which the scheme is targeted The service group or testing activity which the scheme supports The artifact, test item, equipment, matrix or material which is distributed to the

participants The determination, test result or measurement for which the proficiency testing

participants report To ensure these elements are covered and to assist in keeping consistency in the scopes of accreditation, NATA categorizes the schemes covering the first three dot points listed above.

Additionally scopes must also list the determinations and/or reported characteristics for which the scheme is based (the fourth dot point) as free text either aside each subcategory of scheme or against the main service category following information obtained from each facility. The facility may choose the level of detail they wish to express, depending on the openness/flexibility of its Scope of Accreditation. The level of detail in the Scope of Accreditation will inform the level of assessment effort. In general more detail in a scope leads to less assessment time as a more open Scope of Accreditation requires all possible applications


to be discussed during assessment.

For example, scheme providers who offer programs in environmental contaminates testing may list:

Environment Chemical Composition, Residues and Contaminants

Soils and Sediments Detection of Cadmium, Mercury, Lead

or Detection of Heavy Metals

More detail in the scope of accreditation allows potential customers to easily search for scheme providers. In the example above a customer searching for mercury detection will have a successful hit in the first example, but not the second.

Scheme providers may also wish to list the testing techniques for which the scheme is based and the statistical methods (as listed in Annex B of ISO/IEC 17043) each scheme uses in reporting. This additional information can be of assistance to potential customers. Even if this extra detail is not included in the scope of accreditation, NATA will seek this information before each assessment to ensure the assessment team has the required technical competence.

Categories of schemes proposed by NATA, this list is provided as a discussion point and is not exhaustive.

Environment

Chemical Composition, Residues and Contaminants • Waters • Soils and Sediments • Gases

Biological Composition • Waters

Detection of Asbestos • Soils and Sediments • Prepared Slides • Filters

Agriculture, Foods and Beverages

Chemical Composition, Residues and Contaminants • Livestock Feeds • Foods and Food Products • Cosmetics and Hygiene Products • Potable Water

Biological Composition and Contaminants Meat and Meat Products Dairy Products Grains Other Foods Plant Pathogens Potable Water Cosmetics and Hygiene Products

Nutritional Content Livestock Feeds Grains Foods and Food Products

Health and Community Services

Human and Veterinary Pathology Services Bacteria Parasites


Viruses Fungi and Yeasts Molecular Diagnostics Cytogenetics Chemical and Biochemical Analytes Drugs and Drug Metabolites Blood and Blood Products Histology and Cytology Tissues and Body Fluid Samples

Pharmaceuticals Active ingredients Contaminants

Clinical Services Medical Imaging Contaminated Equipment

Legal

Forensic Activities Accelerants Physical Materials Biological Materials Electronic Evidence Firearms Documents Fingerprints

DNA Testing Human Samples Non-Human Samples

Controlled Substances Forensic Drugs Sport Doping Controlled Substances Toxicology Samples

Calibration Schemes

Chemical Metrology Alcohol Standards

Length and Dimensional Metrology Gauges

Mass and Related Quantities Masses Pressure equipment

Electrical Metrology Digital Multi-Meters

Time and Frequency Frequency Analysers Communications Equipment

Optics and Radiometry Standard Lamps Filters

Temperature Metrology Temperature Sources Thermometers

Acoustics Vibration and Ultrasonics


Vibration Equipment Acoustic Equipment

Materials Testing and Industrial Products

Material Testing of Construction Materials Concrete, grout and mortar Aggregates Soils Asphalts

Tensile Testing Lifting Gear

Non Destructive Testing Welds

Consumer Products

Electrical Safety Testing Household Appliances and Components Medical Equipment

Gas Appliances Safety Testing Water Heaters Space Heaters Cookers

Energy Efficiency Water Heaters Space Heaters Air Conditioning Equipment White Goods Solar Appliances

Textile Testing Wool and Natural Fibres Fibres for identification Woven and Knitted Products

Chemical Analysis Detergents and Cleaning Products Surface Coatings and Adhesives •

Outcomes: The paper generated plenty of discussion around the format of scopes. The consensus being as per the RMP scope discussions that the DIS of ISO 17011 now includes requirements for scopes for PTPs and ILAC G18 is under review. Several representatives of the working group are on the ILAC G18 working group and so will be in a position to ensure the review includes, as proposed, PTP and RMP scopes while maintaining the flexibility this working group would like to see.


Discussion paper #9




Name of party submitting the paper: NATA Discussion topic related to: PTP & RMP top 5 deficiencies Related standard/APLAC document and clause number (where applicable): ISO/IEC 17043 and ISO Guide 34 Question/Discussion item: Top 5 deficiencies cited for RMP and PTSP assessments. Reviewing our assessment files will find the top 5 deficiencies identified during assessment of RMP and PTP facilities are very similar to our testing fields, that being (in no particular order);

Metrological Traceability / equipment calibration Test Method validation / measurement uncertainty Reporting Completeness of technical records Completeness of training records

The introduction of ILAC Policy P10 has presented problems with regard to a lack of understanding and inconsistency of application amongst not only the accredited facilities but also our assessment teams. Key issues regarding the application of ILAC P10 are;

Many facilities perform equipment checks on measuring instruments. In some instances these checks are in fact in-house calibrations.

The provenance (point of origin) and reliability of reference standards/materials. Very few reference materials are supported by an ISO Guide 34 accreditation.

Many reference materials obtained from a competent supplier (not certified) are being utilized as Certified Reference Materials by the facilities, without investigation of traceability to a stated reference and/or verification of measurement uncertainty.

The provenance of a test sample distributed as part of a PT program remains a key focus during assessment, particularly in the discipline of pathology testing where the test sample is obtained from a volunteer. Identity and anonymity of the volunteer is paramount. In PTP many of the medical providers make use of technical experts who are volunteers and not directly employed by the provider. With reference to ISO/IEC 17043 we consider them to be contracted staff under the direct technical control of the provider and not ‘sub contractors’ Outcomes: The paper was provided as part of session 8, experience sharing.


Discussion paper #10a



Discussion Paper #1

Name of party submitting the paper: N Venkateswaran

Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO/IEC 17043:2010

Question/Discussion item: Use of AB Symbol by accredited PT Provider’s in their PT Item / Artifacts Background: PT Providers are accredited for limited scope of accreditation. However, they may conduct PT programmes which are not in their scope of accreditation. To differentiate the PT item as well as for marketing purpose, PT Provider request for use of AB symbol. Example: The accredited scope is

Type of PT Scheme**

Proficiency Test Item (Matrix/Group/Field)*

Analyte or Parameter Range of Measurement (if applicable)

Periodicity (Minimum)

Simultaneo Metals & Alloys Si 0.01% to Once in a us Scheme Low Alloy Steel. C 0.07% to Year (Solid Block) Mn 0.15% to

1.80%

Whereas the PT Provider does PT Program for areas like

Metals & Alloys - Low Alloy Steel. (Solid Block) for Si, C, Mn parameter where range is beyond the scope (may be C- 0.05%; Si – 1.5% etc)

Metals & Alloys - Low Alloy Steel. (Solid Block) for P, S Cr, Mo parameter which is not the part of accredited scope of PT Provider

Metals & Alloys - Stainless Steel, Tool Steel etc for which PT Provider is not accredited. In this situation, if AB symbol is used for accredited PTP’s scope PT Item, it will enhance marketing / business opportunity. Outcomes: The DIS of ISO 17011 sets out the requirements for the use of the accreditation symbol and ensuring the use of it does not imply certification or product approval, so the decision of the working group was this is adequately covered. Also see outcomes for Paper #4.


Discussion paper #10b



Discussion Paper #2

Name of party submitting the paper: N Venkateswaran Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO/IEC 17043:2010

Question/Discussion item: Flexible Scope of accreditation for PT Provider NABL is granting scope of accreditation to PTP in the following form:

Sr. No.

Type of PT Sche me**

Proficiency Test Item (Matrix/Group/Field)*

Analyte or Parameter Range of Measurement (if applicable)

Periodicity (Minimum)

PT Provider scope of accreditation is restricted / limited wrt range / parameters / PT items. A

flexible scope is expected from the PT Provider.

Example:

Accreditation is granted as Requirement of PTP

Barium, Copper and Iron in Drinking Water

Metals in Water

Ethion, Chlorpyriphos, Phorate in C l

Residues in Food and Agriculture d t 5g Mass Mass only so entire Mass can be covered.

Outcomes: The question was related what types of information need to go on the scope for an accredited PTP. As per previous commentary, it is expected that ILAC G18 once the review has been completed will provide guidelines and this is also covered in the DIS of ISO 17011. The working group did not have any concerns with the type of scheme being listed on the scope i.e. simultaneous, sequential etc., or the name of the scheme i.e. Food micro as opposed to the type of scheme. The more flexible the scope, the more assessment activity that will be required to confirm the competency of the PTP.


Discussion paper 10c


Hong Kong, China

Discussion Paper #3



Question/Discussion item: Importance of PT scheme in PTP accreditation At present, the classification for PTP are based on testing (and Medical), Calibration. Focus is also on the testing and calibrations rather than PT schemes like Simultaneous, Sequential, Qualitative , Quantitative, Sampling, Data transformation & interpretation, Single Occasion exercise, Continuous (Refer Cl. 3.7 of ISO/IEC 17043:2010). The PT schemes are also not effectively used either in assessing PTP (or) granting accreditation (scope of accreditation) etc. Outcomes: There was discussion in the working group as to whether PTPs participated in their own schemes if they have the competencies to do i.e. also operate a testing laboratory. No consensus was reached.


Discussion paper #10d


Hong Kong, China

Discussion Paper #4



Question/Discussion item: Retention of Artifact used in Sequential PT schemes In sequential schemes (calibration), generally the artifact are procured from manufacturer, then calibrated from NMI & used for PT program. On completion of PT program, the PTP does not want to retain it for future use due to high cost. This may be used as standard /masters for calibrating other equipment / instrument. Outcomes: The consensus of the working group was that the artefact does not need to be retained.


Discussion paper #11


Hong Kong, China

Discussion Paper

Name of party submitting the paper: Taiwan Accreditation Foundation (TAF)

Discussion topic related to: RMP accreditation Related standard/APLAC document and clause number (where applicable): ISO Guide 34 5.14

Question/Discussion item: 1. The expiry date of a CRM was determined by the stability test usually. Are there any

reference materials whose expiry date can be considered as long as 30 years or unlimited just based on scientific theory? What kind of evidence should be provided?

Outcomes: The question of the lengthy expiry date is partially addressed by the ISO Guide 35 and having additional technical justification available to support such an expiry date. It would be the responsibility of the RMP to provide scientific justification for the expiry date assigned. Also see 7.11.2 of ISO 17034. There are plenty of examples of RMs with lengthy stability assignments for products that are inherently stable.




Hong Kong, China

Discussion Paper

Name of party submitting the paper: Taiwan Accreditation Foundation (TAF)

Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO/IEC 17043 5.5;4.4.3

Question/Discussion item: 1. A PTP conducted medical test PT program, and the testing samples were prepared by PTP

staff using space and equipment belonging to another hospital. Can we consider this part as subcontracting?

2. When a testing laboratory and a PTP belong to the same organization, under what kind of

condition, we can consider the testing laboratory as core lab of this PTP or as subcontractor?

3. A CRM was used as test sample in a PT program (tensile testing of plastic product), and no

stability test was conducted. The PTP provided the certificate of CRM, and showed the expiry date as evidence for stability. For this case, do we need further information?

Outcomes: Following discussion from the working group, the following responses were provided: 1. If the PTP and the laboratory are operating under the same management system then as a

general guide this would not be considered subcontracting.

2. It would generally not be considered subcontracting if the laboratory and the PTP are under the same management system and from the same organisation i.e. it would be considered subcontracting if they different management systems.

3. If the PTP was receiving the CRM with the requisite paperwork (as accompanies a CRM)

additional verification of stability would not be required, however if any changes were made to the CAR i.e. packaging etc. additional checking would need to be carried out.

There was also much discussion as to whether some of above scenarios were purchasing or subcontracted activities.





Discussion Paper 1 A2LA

Name of party submitting the paper: A2LA

Discussion topic related to: RMP/PTP Scopes of Accreditation Related standard/APLAC document and clause number (where applicable): TC008 (PTP equivalent if developed)

Question/Discussion Item:

ANAB and NATA have already raised a number of very valid points related to scopes for both PTP and RMP and the revision of 17011 and the current version of TC008. There needs to be further discussion and we would like to add that IAAC has a document for both PTP and RMP accreditation MD028 and MD038. We wanted to provide these documents for those who are not involved with IAAC as it might help facilitate discussion. IAAC MD028 is quite similar to past versions of TC008 and seems to be closer to the recent 17011 draft. IAAC MD038 has some requirements for scopes and they are as follows:

The scope of accreditation shall state the following: • Type of proficiency testing item • The measurand(s) or characteristic(s) that are to be identified measured or tested." • Techniques for determination of the assigned value and its uncertainty (See ISO/IEC 17043, Annex B, clause B.2.1. for common techniques used.)

Note: For quantitative schemes it may be necessary to specify the range of the measurand

A2LA does not necessarily agree with all these items above but we think it is important to be aware of what other regional bodies are doing as some ABs are part of both arrangements and both PTP and RMP accreditation will eventually be ILAC MRA options so there should be harmonization. (For the purposes of the working group report the IAAC documents have not been included with the discussion paper but are available from the IAAC website). Outcomes: This discussion paper was tabled to make participants aware that APLAC TC 008 had requirements over and above the requirements of the IAAC MD0028 document and that eventually all accreditation documents from the various regions should have similar requirements. No action or decision was required.



APLAC Workshop on RMP and PTP Accreditation 7 – 10 November 2016 Hong

Kong, China

Discussion Paper 1 A2LA

Name of party submitting the paper: A2LA

Discussion topic related to: Subcontractor vs. Supplier Related standard/APLAC document and clause number (where applicable): G34 and 17043

Question/Discussion Item: This is a question that comes up quite often and the lines can get blurred as to what category an outside service falls into. As an example, a PTP purchases CRMs or RMs to use as their PT samples. Is the manufacturer of the CRM/RM a subcontractor or supplier? Could they be either depending on how the PTP uses the product and associated data provided with the CRM/RM? This doesn’t really come up as often in RMP but some other accreditation bodies may have been asked these types of questions. A2LA has participated in peer evaluations and there seems to be differing opinions on what falls into which category. As so much of the process can be outsourced (subcontracted or purchased), especially in the PTP area, it may be beneficial to discuss this area during our session. In some instances, it is rather a moot point as many similar requirements apply to both subcontractors and suppliers but it is an area where there seems to be inconsistency and may be worth further discussion if there is sufficient time. Outcomes: This topic generated much discussion, with the outcome being very similar to Discussion Paper #12, where it was established that if the CAB is receipting a CRM with all the requisite paperwork for a CRM and no further manipulation of the item is carried out, then could be considered subcontracting. If the CAB carries out some modification i.e. re-packaging then additional verification could be required and this would be generally considered under purchasing.



APLAC Workshop on RMP and PTP Accreditation 7 – 10 November 2016 Hong

Kong, China

Discussion Paper

Name of party submitting the paper: Japan Accreditation Board（JAB） Discussion topic related to: RMP accreditation (*Please delete as appropriate) Related standard/APLAC document and clause number (where applicable): APLAC TC008 Issue 5 6.1

6.1 An AB shall provide a scope of accreditation that describes the specific types of RMs that the RMP is competent to produce. The scope shall state clearly whether the accreditation covers CRMs, non-certified RMs or both.

Question/Discussion item: I would like to confirm the possibility of the accredited RMP’s scope within the reference material that is not a calibrator.

For example, a reference material documentation says that this material is not a calibrator and is applied to check the trueness of measurement values measured by a chemical laboratory. Note: The term of “calibrator” is defined “measurement standard used in calibration” in ISO/IEC Guide 99：2007.

(1) Have you accredited the RMP with the scope including such as the above reference materials under the RMP accreditation activities based on ISO Guide 34?

(2) Can an Accreditation Body include the material in the scope of accredited RMP under

the RMP accreditation activities based on ISO Guide 34?

Outcomes: It was confirmed by the working group that as ISO 17034 allows for RMPs to produce both CRMs and RMs, it was possible to offer accreditation for both activities.




Hong Kong, China

Discussion Paper Name of party submitting the paper: ACCREDIA Discussion topic related to: RMP accreditation Related standard/APLAC document and clause number (where applicable): NA Question/Discussion item: Point 5.15 of VIM 3 - Commutability of a Reference material (RM) property of a reference material, demonstrated by the closeness of agreement between the relation among the measurement results for a stated quantity in this material, obtained according to two given measurement procedures, and the relation obtained among the measurement results for other specified materials NOTE 1 The reference material in question is usually a calibrator and the other specified materials are usually routine samples. NOTE 2 The measurement procedures referred to in the definition are the one preceding and the one following the reference material (calibrator) in question in a calibration hierarchy (see ISO 17511). NOTE 3 The stability of commutable reference materials should be monitored regularly. This means that RM are commutable when RM and routine samples give the same measurement response when tested with two different methods (one established as reference method). For assessors accrediting RMPs, there is a need to understand when commutability is a matter to have clear criteria to evaluate when commutability studies are necessary. ISO Guide 34:2009 reports in Appendix B that commutability is mainly used in clinical chemistry, but at the end of the point B1, the same guide enlarges the applicability of commutability reporting “The need for the establishment of the commutability of reference materials is not limited to clinical chemistry. It is desirable in any field where the measurement procedures in routine use are based on different physical or chemical principles in comparison to the reference method used to assign the property values of a reference material. It is particularly important to assess commutability where differences between the matrix of the reference material or changes to the secondary/tertiary structure of the analyte in the reference material relative to normal test samples could potentially introduce a bias between results obtained with one measurement procedure relative to the results obtained with (an) other procedure(s) when used on representative test samples. » The ISO-REMCO position paper on commutability recalled in the ISO/FDIS 17034 (point 7.2.3-note) clarifies the concept and gives some indication on where and when commutability becomes an issue. Furthermore in the Summary, the position paper recalls the ISO 15194, as the reference for commutability. This means that commutability assessment has relevance only for RMP for clinical chemistry?

Reading all the position paper, there are still some gray points regarding for example CRM for calibration of X-ray fluorescence measurements, calibration solutions prepared using materials certified for purity. In these cases the position paper suggests to take into consideration the commutability assessment. For analysis different from clinical chemistry studies on commutability are very rare and there is not any standard/guide on this issue. In this situation the evaluation on the need of commutability studies is strongly subjective. To overcome this situation the part on “intended use” of the CRM certificate (point 5.2.5 of the ISO Guide 31:2014) can include some indications on issues like as matrix effects. Could this transitory solution be acceptable?


Outcomes: This paper generated much discussion as to how / when commutability applies. After first clarifying with the presenters / speakers as to what was required for commutability. The working group was informed that the commutability issue is mainly concerning medical (biochemical) testing where the quality control and calibrants are not real samples. It is usually considered when artificial matrix materials are used instead of natural matrix materials. There are some simple statistics that can be applied to look at commutability (refer to CSLI). During RMP assessments, where relevant, the assessors should discuss the need to consider commutability of the RMs produced with the RMP, taking into account the intended use of those RMs.




Hong Kong, China

Discussion Paper

Name of party submitting the paper: ACCREDIA

Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO/IEC 17011 § 7.5.6, ISO/IEC DIS 17011 §7.4.4, 7.4.5, 7.4.6

Question/Discussion item: ISO/IEC 17011 § 7.5.6 states that The accreditation body shall establish procedures for sampling (if applicable) where the scope of the CAB covers a variety of specific conformity assessment services. The procedures shall ensure that the assessment team witness a representative number of examples to ensure proper evaluation of the competence of the CAB. ISO/IEC DIS 17011 § 7.4.4 states: The accreditation body shall establish documented procedures to assess the competence of a conformity assessment body to perform all activities in its scope of accreditation irrespective of where these activities are performed. These procedures shall describe the manner in which the scope of an applicant or an accredited conformity assessment body is covered through the use of a combination of on-site assessments and other assessment techniques sufficient to provide confidence in the conformance with the relevant accreditation criteria. Is there any accepted practice in order to assess the competence for one scheme/one technical area of PT Organization when there are no activity after the first assessment but the PT Provider can document that there are the resources (i.e. staff/experts/sub-contractors)? How can we accept to keep into the scope a sub-discipline that has not been active for sometime? I.e.: a specific scheme representing a specific technical field (i.e. GMO testing in food) was added 3 years ago in the scope of a PTP provider already accredited for chemical testing in food, environmental samples, fuels etc. There was a round before the accreditation in that scheme and one shortly after the accreditation. At the time of re-assessment, the PTP declares that although for its own reasons had not organized the scheme since the last time, wishes to keep it in the scope. What could be the policy in these cases? Outcomes: Following discussion in the work group the following points were made for consideration:

if a programme has not been run for several years accreditation could be maintained if thePTP retained, and it was evident, the competencies for running the programme, and that competency was able to be assessed

it was generally expected that at least one round be run prior to accreditation beinggranted for a particular programme

some accreditation bodies provide data sets as part of the assessment process to confirmcompetencies for particular types of statistical evaluation




Hong Kong, China

Discussion Paper

Name of party submitting the paper: BELAC

Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO IEC 17043 § 4.10.3

Question/Discussion item:

Interested parties

ISO/IEC 17043 § 4.10.3 states that “When an interested party requires the proficiency testing results to be directly provided by the proficiency testing provider, the participants shall be made aware of the arrangement in advance of participation." : who can be considered (or who cannot be considered....) to be an interested party? Could a national competent authority stakeholder be considered interested party (i.e. European Commission asking European Reference Laboratory to organize PT)? Outcomes: The consensus of the working group was yes, a stakeholder could be considered an interested party.



APLAC Workshop on RMP and PTP Accreditation 7 – 10 November 2016 Hong Kong, China

Discussion Paper


Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO IEC 17043 § 4.2.3


Personnel

ISO/IEC 17043 § 4.2.3 states that “The proficiency testing provider shall use personnel who are either employed by, or under contract to it.” How can be interpreted “personnel under contract”? Indeed, normally a subcontractor has a contract with its customer. The subcontractor’s personnel performing a specific task for the PT provider could be considered as contracted staff? Outcomes: The consensus of the working group was that using the definition of a subcontractor as set out in ISO Guide 30 i.e. operates under a separate management system, subcontractors personnel would not be considered contracted personnel of the PTP.



APLAC Workshop on RMP and PTP Accreditation 7 – 10 November 2016 Hong Kong, China

Discussion Paper


Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO IEC 17043 § 5.5


Subcontracting

1) How to evaluate a PT provider in case everything is subcontracted, except for planning of the PT scheme, the evaluation of performance & the authorization of the final report (which is possible according in the context of this standard)? Which attention points should be taken into consideration and how far the evaluation by an AB has to go in that case (in particular concerning the evaluation of non-accredited subcontractors)?

Should they comply with all the relevant clauses of the International Standard and other appropriate standards? 2) When a proficiency testing provider subcontracts a part of works such as the preparation of proficiency test items (4.4.2.1), how to assess homogeneity, stability (4.4.3) and the assigned value (with metrological traceability and MU §4.4.5) considering also the impact of the environmental conditions (both for accredited or non-accredited subcontracted activities)?

Outcomes: Points arising from the discussions were as follows:

The accreditation body needs to ensure that the PTP has assessed all the requirements comply with ISO 17043

The accreditation body needs to review how the PTP assesses its subcontractor, which could be done through record review, witnessing or other appropriate means.

The accreditation body can reserve the right to assess the subcontractor The contract between the PTP and subcontractor could be reviewed The PTP itself will need to be competent to assess the subcontracted activities




Hong Kong, China

Discussion Paper


Discussion topic related to: PTP accreditation Related standard/APLAC document and clause number (where applicable): ISO IEC 17043 § 4.4.1.4


Personnel/advisory group

In case an advisory or steering group is used (§4.4.1.4 ISO/IEC 17043), is this group considered as "personnel of the PT provider" to which all criteria related to personnel also apply (e.g. §4.2.2, 4.2.5, 4.2.6, 5.1.5.b....)? Outcomes: The consensus of the working group was that it would depend on the arrangements between the advisory group and the PTP. The question was also raised; does the advisory group need to be present for the assessment? The consensus being it would depend on the expertise of the PTP staff but they should at least be contactable should the need to speak with them arise during the assessment process.




Hong Kong, China

Discussion Paper



Question/Discussion item: Collusion and falsification of results. ISO/IEC 17043 § 4.4.1.3.j) states that “The proficiency testing provider shall document a plan before commencement of the proficiency testing scheme that addresses the objectives, purpose and basic design of the proficiency testing scheme, including the following........: reasonable precautions to prevent collusion between participants or falsification of results, and procedures to be employed if collusion or falsification of results is suspected" Which attention points should be taken into consideration in the context of "reasonable precautions"? Outcomes: Mr Tholen advised there are statistical analyses which can be performed to confirm that no collusion has been undertaken. The accreditation body needs to look at the PTP to see what precautions are in place to ‘not facilitate’ collusion.




Hong Kong, China

Discussion Paper




Homogeneity and stability

ISO/IEC 17043 § 4.4.3.6 states that “In circumstances where homogeneity and stability testing is not feasible, the proficiency testing provider shall demonstrate that the procedures used to collect, produce, package and distribute the proficiency test items are sufficient for the purpose of the proficiency testing.” What kind of circumstances can be accepted where homogeneity and stability testing is not feasible? Outcomes: Several scenarios were provided by the working group where homogeneity and stability testing may not be feasible, as follows:

Gas mixture in a cylinder with a single item being circulated; gases could stratify but this could be managed through the instructions provided to the participants

Where the PTP is also the manufacturer of the item; they have done studies to establish the shelf life of the product and its components

In the case of an RMP becoming a PTP, where this data is available for a given CRM /RM Where there is for each participant their own artifact which is compared with reference

criteria. Homogeneity is unlikely to be an issue but stability may be In some microbiology schemes, homogeneity and stability testing is carried out

concurrent with the scheme due to the short shelf life of the material



Outcomes: As for discussion paper #7, the participant needs sufficient information to understand the report provided by the PTP. If for example, the data supporting the homogeneity and stability statements are not included in the report provided then it needs to be readily available to the participant should they request it.

Report on the APLAC Workshop on Reference Material ...

Documents

Transcript of Report on the APLAC Workshop on Reference Material ...