൮ing strategy team. - eastcoastmetrology.com · – Committee chartered ... – Designed and...
35
CMSC 2018 Boston, Ma. Scanner Certification CMS Certification Committee March 22-23, 2018
Transcript of ൮ing strategy team. - eastcoastmetrology.com · – Committee chartered ... – Designed and...
CMSC 2018 Boston, Ma.
Scanner Certification
CMS Certification Committee
March 22-23, 2018
Presenter
Presentation Notes
Good morning, I have had the pleasure of participating with a great team of 3D professionals on this effort the past 8 years. My background was in Tooling, beginning with conventional optics then CAT and both film-based and digital photogrammetry, as well as laser trackers. I transitioned to training these disciplines about 20 years ago at Boeing Puget Sound, and now lead a learning strategy team.
CMSC 2018 Boston, Ma.
Agenda
• Welcome Gary Confalone
• Introductions All
• CMS Certification Program Overview Keith Bevan
• NOCI – Assessment Workshop Dr. Katherine Manley (Kitty)
CMSC 2018 Boston, Ma.
CMS Certification Committee
The CMS Certification Committee was formed in 2009 to assess the need and feasibility of creating a Certification program for portable metrology professionals. A charter was developed that defined the committee membership, the reporting responsibility and updated the CMS bylaws to make the certification committee a standing committee. The committee researched existing certifications, surveyed the CMS membership (2009 conference) and began developing a preliminary Body of Knowledge (BOK). The committee also determined the role of the Coordinate Metrology Society as an issuing authority.
Presenter
Presentation Notes
Laser Trackers, Photogrammetry systems, Portable Arms, Radars, CMM’s and Scanners are all assessed for performance and usually certified by an independent or qualified process validating such. Operators however, though well intentioned, and committed to performing their jobs well did not have a similar, rigorous process to validate and demonstrating proficiency. Our mission began with a clear purpose: To design and develop a rigorous, defensible certification that would attest to a level of proficiency and professionalism missing in the field of portable 3D Metrology. A certification not just for professional engineers, but the technical professionals in the field, and factories, supporting every facet of manufacturing, quality assurance, and close tolerance measurement and data collection in every sector. Our research indicated a 3-5 year level of effort was not uncommon given the challenges and details required to meet this goal. We offered our first certification assessment 4 years into our journey. I’d say we fell within an acceptable range! I am going to share with you some of the detail and process that led to our success, as well as what we have left to accomplish.
CMSC 2018 Boston, Ma.
– Market Study – Membership surveys – Committee chartered – Created Bylaws – Created a Body of Knowledge – Accreditation Research – Selected ASPRS as certification partner – Development of skills study at CMSC 2010/2011 – CMS contracted National Occupational Competency
Testing Institute (NOCTI) to provide oversight of assessment development.
– Developed Level1- (General) and Level2 (Performance) PCMM & Laser Tracker assessments
– Designed and Developed CMS Certification Handbook
2009-2017 accomplishments
Presenter
Presentation Notes
The journey necessitated a detailed analysis including: Speak to each
CMSC 2018 Boston, Ma.
CMS Certification Committee 2017
Members Randy Gruver The Boeing Company, Employee Development
Al Metzel Northrop Grumman Electronic Systems
Bob Elliot Lockheed Martin Integration/Test Engineering Senior Staff, Alignment Engineer Matthew Ilardo Brookhaven National Lab, Alignment Engineer
Gary Confalone East Coast Metrology, President CEO
Ulysses Green The Boeing Company /GKN / RT
Chuck Pfeffer FARO Technologies, Director of Product Management - Metrology
Ben Rennison The Boeing Company, Boeing Research & Technology
Presenter
Presentation Notes
The following experts are the active members of the CMS Certification committee. Some with the effort since day 1, Al, Chuck and UG and Keith Bevan, shortly thereafter Gary, then Bob Matthew and Ben. All meet every other week and have dedicated themselves to contributing to professional growth in our field. Several folks were key to the effort but have moved on including many members of the Executive committee, Talion Edwards, Ron Rode, Ron Hicks and Pat Walsh.
CMSC 2018 Boston, Ma.
Timeline
The Certification Journey
CMSC 2018 Boston, Ma.
• Industry Requested Certification • Potential Growth Nuclear – Increase in production and reset Aerospace – Extended Service Life Automotive – Efficiency requirements
• Contractual Requirements • Standardized Practices • Customer Demand of Qualifications • Membership feedback • Supports a Quality Management System
Validate the Need
Presenter
Presentation Notes
We began by validating the need:
CMSC 2018 Boston, Ma.
• Baseline with CMS membership (survey) • Also contacted UAW International through
CMS automotive connected members • Contacted each represented company by
phone • Asked to provide how many 3D
Metrologists worked at the facility • Total number of users from survey: 2407
Market Research
Presenter
Presentation Notes
Performed copious amounts of research
CMSC 2018 Boston, Ma.
Portable 3D Metrology users by sector
64.37%
20.71%
5.36% 3.95%
1.00% 2.54% 2.08%
Aerospace Automotives Shipbuilding Misc
Nuclear H. Equipment Measurement
Services
Market Research
Presenter
Presentation Notes
Some of our data
CMSC 2018 Boston, Ma.
Accreditation Research
• ISO 17024 Personnel Certification • ANSI accredits to ISO 17024 in US • Attended ANSI workshop on ISO 17024 • The Council of Engineering & Scientific
Specialty Boards (CESB)
Presenter
Presentation Notes
Membership supported accreditation, we decided that due to the cost, we would develop our certification within the guidelines necessary to attain ISO or CESB accreditation, but would abstain.
CMSC 2018 Boston, Ma.
Workshop Feedback • 80% of those in attendance learned
OJT • 62% recommended proficiency
testing • 80% recommended graduated levels • Accreditation was strongly
recommended
Engage Membership
Presenter
Presentation Notes
We held workshops and sent surveys to membership.
CMSC 2018 Boston, Ma.
2009/2010 Workshop Attendees - Industry
What industry best describes your company?
0%10%20%30%40%50%
Gov
ernm
ent/M
il...
Aero
spac
eAu
tom
otive
Met
rolo
gy H
a...
Serv
ice p
rovid
erSc
ientifi
c/Res
...Po
wer g
ener
a...
Ship
build
ing
Perc
ent
2009
2010
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
Perc
ent
Sales MetrologyTechnical
Management Other
What is your job title?
2009
2010
0.00%
10.00%
20.00%
30.00%
40.00%
Perc
ent
None 1-5 6-10 11-20 21+
Years of Experience
How much experience do you have in Metrology?
2009
2010
Engage membership
Presenter
Presentation Notes
Attendee data supports survey distribution, largely aerospace sector and technical users of equipment.
CMSC 2018 Boston, Ma.
Perceived Value to Company • ~91% of respondents felt that certification had at
least some value
• Of the 159 indicating valuable/extremely, the highest % were in aerospace (among groups)
CMSC 2017 Snowbird, UT.
9% 91%
Engage Membership
Presenter
Presentation Notes
91% believed there would be some and extremely valuable
CMSC 2018 Boston, Ma.
Value to Customer vs. Company/Individual
• Interest in certification appears largely driven by personal/company benefits, not by customers – Only 29% indicated that customers request certification
CMSC 2017 Snowbird, UT.
Engage Membership
Presenter
Presentation Notes
106% believed there was a personal or company benefit (pause), while just under 30% have had customers request or ask about certification.
CMSC 2018 Boston, Ma.
Certification Process • Among the choices, most preferred ‘general certification’ or
‘general with add-on hardware modules’ (~70%) – Survey Responses: Certification should be:
• general and not include hardware specific knowledge. • a general certification that is hardware independent along with individual hardware
technology group modules that can be added to the general certification. • individual certifications for each hardware technology group.
• Similar results across all industry groups (see appendix)
CMSC 2017 Snowbird, UT.
Engage Membership
Presenter
Presentation Notes
Membership feedback indicated that a multiple level assessment was most effective. This sets CMS apart from many professional certifications. The practical, or psychomotor assessment (hands-on) demonstrates without doubt, a successful candidate can perform at exemplary levels.
CMSC 2018 Boston, Ma.
Certification Requirements • Of those indicating their company would seek certified
employees, majority desire a rigorous process – With both written exam and demonstration of proficiency
• One potential challenge: some existing certification bodies (such as ASQ) do not perform proficiency evaluations.
– Majority also wanted documented experience/ annual hours
CMSC 2009 Snowbird, UT.
Engage Membership
Presenter
Presentation Notes
A thorough and rigorous process was desired, and achieved!
CMSC 2018 Boston, Ma.
• 2010 - measurement study introduced - First principle measurement study based on a variety of ‘hand tools’ used in dimensional measurement. The objectives looked at the importance of core measurement principles, instilling the right measurement strategy, questioning and observing behaviours when dealing with measurements.
• 2011 - 3 Portable Co-ordinate Measurement Systems looking at the ‘influence of behaviours’ on the measurement result. • 2012 - 6 Portable Co-ordinate Measurement Systems looking at the ‘appropriateness’
and the ‘importance of practical testing’ to support the certification programme. • 2013 - Measurement study and pilot practical test in support of the cognitive
certification process
Measurement Study Timeline
Presenter
Presentation Notes
In addition to membership feedback and industry analysis, with the able support of NPL, Metrologic and more, the CMS hosted several measurement studies at the conference. 2010: Core principles, using calipers and micrometers on artifacts. 2011: A focus on the Influence of behaviours 2012: A focus on practical testing, and variations in data due to technique with and without guidance. 2013: As stated
CMSC 2018 Boston, Ma.
Performed measurement studies
Presenter
Presentation Notes
We were able to observe and assess, even in the exhibition hall environment practices like:
CMSC 2018 Boston, Ma.
Is a certifying an individual for knowledge and practical skills important? (1 = Low and 5 = High)
Evaluated results
Presenter
Presentation Notes
Careful evaluation of all the data collected supported the mission of developing certification.
CMSC 2018 Boston, Ma.
4 measurement tasks with and without instruction covering diameters, length, depths and gauge block wringing • Task 1: Without procedures 55 readings ranged from 0.006 to 0.0075
inches • Task 2: Without procedures 43 readings ranged from 0.0015 to 1.21
inches • Task 3: With procedures 39 readings ranged from 0.0003 to 0.002
inches • Task 4: Without procedures 28 readings varied by 0.0045 inches • Key observations and behaviours across a wide range of expertise
Evaluated results
Presenter
Presentation Notes
Results achieved illustrated benefit of performance-based competency testing.
CMSC 2018 Boston, Ma.
Level 2
Device-specific hands on exam
Level 1 General Knowledge exam (Proctored, online)
CMS Code of Ethics
Created Certification Structure
Presenter
Presentation Notes
The developed certification components are based upon a foundation of professional conduct, detailed in our code of ethics.
CMSC 2018 Boston, Ma.
Developed the process
Presenter
Presentation Notes
This is standard process for assessment development, academically sound and defensible.
CMSC 2018 Boston, Ma.
Defined Level 1 Requirements • Basic Requirements The basic requirements for a Level-One Certified
Portable 3D Metrologist are: • 1.A professional who uses portable 3D metrology technology to collect
measurements and interpret data from a variety of contact and non-contact portable 3D measurement devices. The Portable 3D Metrologist is responsible for all phases of data collection and other requirements such as planning and supervising survey activities requirements and interpretation.
• 2.Two years of experience in portable 3D metrology, one year of which was in a position of responsibility demonstrating professional knowledge and competence.
• 3.References from four people who are holding, or who have held, responsible positions in portable 3D metrology and have first-hand knowledge of the applicant’s professional and personal qualifications.
• 4.Declaration of compliance with the Code of Ethics of the Coordinate Metrology Society.
• 5.Successful completion of the online examination.
CMSC 2018 Boston, Ma.
Defined Recertification Requirements
• ‘Active’ Status Requires Re-Certification – Every 5 Years for Certified Professional
• Submit Completed Application • Provide 3 References • No Written Examination Required • Sub-Committee Evaluation
– Document Experience and Activity During the Previous 5 Years
– Must Satisfy Minimum Requirements of 75 PDH’s (Professional Development Hours)
CMSC 2018 Boston, Ma.
Focus Area % of questions
Interpret design documents/requirements 16%
Measurement device knowledge 16%
Pre-measurement planning 23%
Performing measurement operations 25%
Analyzing data 19%
Ethics and professional responsibility 1%
Certification Structure
Level 1 General Principles (Cognitive)
CMSC 2018 Boston, Ma.
Level 2 PCMM Performance Matrix (PCMM)
14%
Certification Structure
CMSC 2018 Boston, Ma.
Defined Recertification Requirements
Presenter
Presentation Notes
75 Personal Development Hours or PDH’s over a 5 year period are required to maintain certification. 50 PDH’s can be earned by using the equipment in your daily jobs. Our intent was to make the certification attainable and sustainable by technicians collecting measurements.
CMSC 2018 Boston, Ma.
Defined Recertification Requirements
Presenter
Presentation Notes
Many opportunities to earn PDH’s including attendance at the conference, and training sessions as an attendee or instructor.
CMSC 2018 Boston, Ma.
• Laser Tracker • PCMM • Photogrammetry • Area Scanner • Hand Scanner • Spherical Scanner • Post Processing
Certification Structure
Level 2 Hands-On (Psychomotor)
Presenter
Presentation Notes
Level 2 will cover the 5 Portable 3D systems listed…ya, the scanners will likely be combined. We will learn more from our members tomorrow at the scanner workshop in the education zone. If you are a scanner user please plan on attending.
CMSC 2018 Boston, Ma.
Authorized Level 1 Proctor Sites
Presenter
Presentation Notes
CMS has authorized these organizations to proctor Level 1 assessments. In addition to the conference you can test any time of the year at a convenient location.
CMSC 2018 Boston, Ma.
“The CMS Certification gave me tremendous credibility when meeting with customers. They understand that I possess the knowledge required to tackle their metrology challenges.” John Chan Application Engineering Manager | Chef Ingénieur d'Applications T 1-514-369-4055 x245 www.builditsoftware.com
Level 2 Certified Member
Presenter
Presentation Notes
Matt and John are not in attendance this year, but sent a few words about how certification has enhanced their careers.
CMSC 2018 Boston, Ma.
Level 2 Certified Member
Strenuous checks and balances have been set to ensure metrology hardware and software are traceable, accurate and precise. Yet, few standards have been set to ensure the operator is knowledgeable in the science of measurement. The CMS Certification instills reassurance in my customer that I am qualified in metrology best practices. Matt Chieppa | Account Manager, Faro Mobile: 434.856.2687 | Fax: 407.333.4181
Presenter
Presentation Notes
Matt and John are not in attendance this year, but Luke Turolski volunteered to share a few words about his experience. Luke is certified Level 1 and 2, and is a Level 2 proctor/assessor.
CMSC 2018 Boston, Ma.
Contributors
Keith Bevan National Physical Laboratory – UK Georgia Harris NIST, NCSLI Dr. Steve Phillips NIST J.-Angelo Beraldin National Research Council Canada Dr. David MacKinnon National Research Council Canada Ron Hicks API Mike Renslow ASPRS Randy Gruver Boeing Ulysses Green Boeing/GKN Aerospace Talion Edwards Boeing Paul Reed Boeing Ben Rennison Boeing Matthew Ilardo Brookhaven National Lab Mathieu Dube’-Dallaire BuildIt Software Michael Raphael Direct Dimensions Lukasz Turolski East Coast Metrology Gary Confalone East Coast Metrology
Thank you!
Presenter
Presentation Notes
The commitment necessary to develop a certification program is wholly dependent on participation of subject matter experts. Documenting the job tasks begins with identifying and then ranking each. We had the benefit of outstanding support from these experts,
CMSC 2018 Boston, Ma.
Contributors
Thank you! Jennifer Park FARO Michelle Edwards FARO Chad Crisostomo Faro Chuck Pfeffer Faro Eric Hollenbeck Hexagon Metrology Doug Sutphin Hexagon Metrology Pat Bass Hexagon Metrology Jean-Sebastion Blais Innovmetric David O’Neal New River Kinametics Nathalie Anderson Metrologic Jeffrey Bourque Nikon Metrology Pat Walsh MVS Group Dr. Katherine Manning NOCTI Al Metzel Northrop Grumman Ron Rode Planet Tool Mitch Fields TriMet Ernie Hustad Verisurf Scott Knoche Verisurf Michael Mock 3D Systems
CMSC 2018 Boston, Ma.
Download www.cmsc.org
Developed Certification Handbook
Presenter
Presentation Notes
The process for application and all information regarding certification are detailed in the certification handbook.
