Certified Aviation Manager - NBAA · PDF fileCertified Aviation Manager Behind the Credential...
Transcript of Certified Aviation Manager - NBAA · PDF fileCertified Aviation Manager Behind the Credential...
Certified AviationManagerBehind the Credential
Las Vegas, NV October 10, 2011
Schroeder Measurement Technologies
• Our roots go back to 1985 with the founding of AppliedMeasurement Services
• Presently work with nearly 200 licensing boards and professionalassociations
• Through IQT we have over 300 computer-based testing centersworld wide
• ESS – the only licensed private investigation firm that isexclusively focused on examination-related intellectual propertyloss prevention
With Iso Quality Testing and Examination Security Solutions
And, aviation supports our business
3
Certified Aviation Managers
• Certified Retinal Angiographers - Certified Neuroscience Nurse
• Certified Fire Protection Specialist - Certified Anesthesiologist
• Certified Orthopedic Technologist - Certified Professional Trainer
• Certified Alcohol & Drug Counselor - Certified Emergency Physician
• Certified Pain Management Educator – Certified Hemiodialysis Nurse
• Certified Forester - Certified Orthopedic Surgeon
• Certified Retirement Financial Advisor - Certified Professional Organizer
• Certified Optical Coherence Tomography Specialist
• Certified Nuclear Cardiology Technician - Certified Lymphedema Therapist
• Certified Personal Trainer - Certified Fire Inspector
• Certified Surgical Assistant - Diplomate in Veterinary Ophthalmology
In Good Company
4
Standards for Credentialing Programs
• American National Standards Institute (ANSI)
• National Commission for Certifying Agencies (NCCA)
5
CAM FACT: NBAA is presentlyin the process of seekingcertification through NCCA forthe CAM program.
Process of Developing a CredentialingProgram
• Role Delineation Study
• Examination Design
• Question Development
• Examination Assembly
• Establish Minimum Passing Score
• Determine Reporting Scale
Role Delineation Study• Initial Research into Role
– How are people trained for the role?
– Review of relevant textual materials
– Interviews with role incumbents
• Focus Group or Survey
– Define important elements of knowledge, skill and ability (KSAs)
– Organize KSAs into content areas
– Obtain Subject Matter Expert (SME) input
CAM FACT: The most recent roledelineation study was done in 2008 andused a focus group approach
Examination Design
• Determine How the KSAs will be tested
– Multiple-choice questions
– Performance-oriented scenarios
– Simulation-related
• Associate important KSAs with examination types
• Weight content areas
• All decisions made by SMEs
CAM FACT: The examination has 150 itemsthat are scored spread across 5 domains.The examination also contains 25 un-scoredquestions on which statistics are gathered.
Question Development
• SMEs trained in how to write questions
• Questions written by SMEs
• Questions critiqued by editors knowledgeable in testing
• Questions reviewed and revised by SMEs
• Questions approved for use by SMEs
• Questions associated with test design outline by SMEs
CAM FACT: Two item writing and reviewworkshops are held each year in June andOctober. Prior to the workshops, questionsare written on-line and edited. Items receivefinal approval for use at the workshops.
Examination Assembly
• Questions selected from bank by test developer using thefollowing criteria
– Conformance with content area weighting
– No question contains information that helps answer another question
• Assembled examination reviewed, revised and approved bySMEs
• After approval, the examination goes through a 4 to 6 weekprocess of editing, quality assurance checking and publishing tothe computer-based testing network.
CAM FACT: At least two SMEs checkeach test form for duplicative, out ofdate, and inaccurate content
Establish Minimum Passing Score
• Recognized procedures meeting testing industry standards usedto set minimum passing scores
• SMEs trained in the application of the procedure
• Procedures depend in some way on the judgment of SMEs inconnection with the definition of minimum required competenceand the relationship of test questions to that definition
Determine Reporting Scale
• Scores are reported on a scale
• The scale is a mathematical function of the number correct
• The use of a scale supports equating which accounts fordifficulty differences from examination form to form
CAM FACT: The minimum passingscaled score on the exam is 500.Scores are reported in the range from200 to 800.
Maintaining a Credentialing Program
• Continuing Question Development
• Continuing Form Assembly
• Statistical Equating and Psychometric Assessment
• Periodic Role Delineation Updating
• Periodic Test Design Revision
Continuing Question Development
• SMEs meet from time to time to write questions
• On-line item writing is used to reduce meeting costs
• Questions continue to be reviewed and revised by SMEs
• Questions continue to be approved for use by SMEs
CAM FACT: To be approved for pre-testing, at least 4 SMEs must bepresent.
Continuing Form Assembly
• On a regular basis, new forms are assembled
• The basis may depend upon the passage of time or the numberof candidates to which the form is exposed
CAM FACT: A new form of theexam is release every other year
Psychometric Assessment and StatisticalEquating
• Psychometric properties such as test reliability are assessed
• Statistical analyses of questions is conducted to determinequestion difficulty and discrimination
• Statistical form equating creates a new mathematical functionbetween raw scores and reported scores to eliminate theunfairness of form difficulty differences
CAM FACT: Statistical item analysis isperformed on all of the data availableimmediately prior to the assembly of anew form and immediately after the firstadministration of a new form.
Periodic Role Delineation Updating
• As technology, law and other factors change so do roles
• Every 3 to 5 years, the role delineation study is updated
• New KSAs are identified and added to the role while others areeliminated
• The relative importance of KSAs to competence in the role mayalso change
Periodic Test Design Revision
• As changes occur in the role study, changes in test design areneeded
• Such change may result in alteration of the content and/orcomposition of a multiple choice exam
• Such change may also result in adding or eliminating modes oftesting – such as performance tests or simulation
The NBAA CAM Credential
• NBAA has invested heavily in the CAM program
• NBAA has also ensured that the program meets testing industrystandards for credentialing programs
• NBAA is seeking recognition for the program through NCCA
• CAM credential holders should be very proud of their credential
CAM FACT: Over 86 percent of CAMcertificate holders renew their certification
Help Make CAM Even Better
• Credentials assist employers with hiring decisions
• Credentials provide practitioners with a means of havingcompetence recognized
• CAM credential holders can help by emphasizing the importanceof the credential to decision makers in their companies and totheir peers
Get Involved!
• Become a Subject Matter Expert (SME)
• Write examination questions on-line
• Help make the examination more authentic by reflecting yourunique knowledge and experience
21
CAM FACT: SMEs who author sixacceptable questions receive one of fourpoints required for recertification