Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities,...
Transcript of Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities,...
![Page 1: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/1.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessment of
Supervisory
Requirements
CMT
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 2: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/2.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessment of Supervision Requirements
Assessing facilities as to their technical competence in relation to the supervisory
requirements in the CMT FAD
• Technical Supervisory Personnel (refer ISO 17025 Clause 4.1.5, 5.2.1)
• Competency Records (refer ISO 17025 Clause 5.2.5)
National Association of Testing Authorities, Australia
![Page 3: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/3.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessing Technical Supervision
• Initially focussed on Level 5/4 supervisory staff
– Technical knowledge in relation to the type of testing, its performance, selection
of appropriate methods and depth of understanding of test outputs;
– Establishment of the equipment assurance program;
– Establishment and implementation of the quality assurance activity plan,
including proficiency testing and the analysis of the results and actions taken
from this program;
– Development of training plans for the development of staff and assessment of
staff technical competencies;
![Page 4: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/4.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessing Technical Supervision
• Focussed on Level 5/4
– Ability to critically review test data and reports;
– Ability to technically review tender documents, selection of appropriate test
methods and development of inspection and test plans, as appropriate;
– Supervision of technical staff, including field testing and sampling as well as
annex facilities if applicable;
– Authorisation of staff to sign test reports and the process of managing this.
![Page 5: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/5.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Competency Records
• How did the facility decide an operator was competent (discuss with L4/L5)
– Process
– Records
– By whom, e.g. supervisor
– Quality assurance or comparison
– Reading of methods and understanding check sheets
– New staff
– Assignment of key responsibilities
– Authority to use equipment and perform tasks
![Page 6: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/6.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Competency Records
• Supervision of staff
• Records of supervisory visits
• Records of what is done at supervisory visits
• Records of checks of work
• Training in laboratory’s quality system
![Page 7: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/7.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Competency Records
• Competency records (all levels as appropriate)
– Qualifications
– Certificates
– Previous signatory approval
– Relevance of qualifications and certificates of competency
– Supporting material for deciding competence
– Authorisations, e.g.
• signing reports
• delegations
![Page 8: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/8.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Competency Records
• Reports and worksheets (all levels as appropriate)
Who signed reports?
• Are these signatories authorised for the tests
– Do reports meet the technical standards (review sufficient across the full range of
methods)
• ISO/IEC 17025
• Test method
• NATA Rules
![Page 9: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/9.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Questions
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 10: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/10.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessment Plans
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 11: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/11.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessment Plans
• Plan based on
– Discipline – Class of test
– Core competency areas – sub-class of test
– Competency elements– standard test descriptors
![Page 12: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/12.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Assessment Plans
• Review plan
– Which tests are to be performed
– Which tests may be discussed
– At least one test from each subclass of scope will be witnessed
– Advise lead assessor of special needs for samples, timing, off-site visits required
– To be agreed prior to assessment and reinforced at entry meeting
![Page 13: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/13.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Witnessing and assessment plan
• Technical witnessing and questioning (all levels as appropriate)
– Demonstrated knowledge by operators of details of the tests they perform
– Knowledge by supervisors and those deemed to be competent of the feasibility
of the test results
– Discussion on what can go wrong and what is done to fix
– Number of staff involved
– If infrequently performed tests – how are they managed
![Page 14: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/14.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Questions
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 15: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/15.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Quality Assurance
Activities
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 16: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/16.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Quality assurance activities
• Quality assurance activities : How does a lab ensure competence is maintained?
– Proficiency and QAA Programs – planned and followed
– Infrequently performed tests
– Participation by whom?
– How were results evaluated? – Is this adequate
– What corrective action has been taken?
– Records
![Page 17: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/17.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Quality Assurance Activities (Lead assessor)
• Proficiency testing
– Plan
• Does it cover all classes of test every four years?
• Was the plan achieved?
• Does the plan cover next 18 months?
• Quality Assurance Testing
– Plan
• Does it cover each sub class of tests every three years?
• Was the plan achieved?
• Does the plan cover next 18 months?
![Page 18: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/18.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Quality Assurance Activities
(Technical assessor)
• Quality Assurance Testing
• Was the analysis of the results adequate?
• Do z-scores matter and where does the facility fit in?
• Were the differences for the facility practically significance
• Was the appropriate action taken and followed up?
– Check action taken and whether improvement has been measured.
![Page 19: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/19.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Quality Assurance Activities
(Technical assessor)
• Proficiency testing
• Was it adequate for the scope of accreditation?
• Review reports
![Page 20: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/20.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Questions
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 21: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/21.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Equipment Assurance
Plans
Calibrations and Checks
National Association of Testing Authorities, Australia
www.nata.com.au
![Page 22: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/22.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Equipment Assurance Programs
(Lead Assessor) • Procedure
• Plan
– Does it cover all equipment?
– How is out of service equipment managed?
– How is the movement of equipment to other facilities managed?
– Do the frequencies match CMT FAD?
• If not, refer to technical assessor for discussion with L5/4
– Was the equipment assurance plan followed?
• If not, refer to technical assessor for the action required
![Page 23: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/23.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Equipment Assurance Programs
(Technical assessor) • Are there variations from commonly accepted frequencies and methods?
• Is the plan adequate for equipment used
– Review facilities identification of critical/non critical equipment
• Has the plan been followed by using correct methods
– Review calibration records for frequency and adequacy for tests on scope
– Review check records for frequency and adequacy for equipment
– Ensure that the acceptance criteria shown are acceptable and have been
checked against the recorded measurements
• Identify missing or out of service equipment
![Page 24: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/24.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Equipment Assurance Programs
(Technical assessor) • Review records
– Have they met equipment requirements in methods?
– Are the intervals of calibrations/checks as detailed in plan?
– Have checks been performed correctly?
– Has equipment been noted as suitable for use in these records
![Page 25: Assessment of Supervisory Requirements CMT - … · National Association of Testing Authorities, Australia Assessment of Supervisory Requirements CMT](https://reader031.fdocuments.us/reader031/viewer/2022031107/5ba8d7d009d3f2fb228b6634/html5/thumbnails/25.jpg)
National Association of Testing Authorities, Australia
www.nata.com.au
Questions
National Association of Testing Authorities, Australia
www.nata.com.au