Assuring assessments leaders team member’s quality and qualification
-
Upload
kobi-vider -
Category
Documents
-
view
216 -
download
0
description
Transcript of Assuring assessments leaders team member’s quality and qualification
![Page 1: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/1.jpg)
Assuring Our
Assessments, Leaders
and Team Member’s
Quality and
Qualification Process
and Quality
Kobi Vider
K.V.P Consulting
+972522946676
![Page 2: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/2.jpg)
Agenda and Topics
• Opening
• The Internal and External Assessors Community
• The Challenge Statement
• The ‘Project’ Goals / Success Factors
• Tool DEMO And Short Walkthrough
• Results and Benefits
• Conclusions and Recommendations
• Future Plans
• Discussion Points
![Page 3: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/3.jpg)
Opening
• As for today most of major industries mainly defense and
aerospace, as well as armed forces
• Which runs and mange large and complex programs
• Need to comply with more than just one quality standards
in many disciplines (e.g. HW, optics, software)
• They use large groups of internal and external assessors
that perform implementation checks, progress checks,
readiness reviews and formal appraisals.
• These communities are typically composed from groups of
very experienced and professional individuals
• These individuals have the best knowledge in their
professional domain but not necessarily on how to conduct
an efficient and effective appraisal which provide
meaningful results.
![Page 4: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/4.jpg)
The Internal and External
Assessors Community
• ~450 auditors (internal military staff and externals reserves)
• ~80 externals (contractors, that perform audits for and on the XYZ
ORG)
• Few tens that are the contractors internal audits and assessments teams
(quality wise)
• ~200 quality engineers (full time)
• ~150 quality practitioners / trustees (part time)
• ~X00 project / program and acquisitions officers
![Page 5: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/5.jpg)
The Challenge Statement
• The combination of the effort and expected
resources increase the risks on qualification
of auditors, domain knowledge, and
calibration of results and findings
effectiveness.
• When I have initial try to map the risks and
challenges, I have discovered that the main
ones are:
![Page 6: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/6.jpg)
The Challenge Statement
• Large diversity in tailoring the methods for ‘of the
same’ appraisal (such as scope, coverage, rating
understanding)
• Inconsistency in ratings
• Inconsistency of interpretations
• Reports calibration for the same unit / group and
on organizational level is very difficult if not
impassible
• Performing quality assurance of the performed
process is very challenging
![Page 7: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/7.jpg)
The ‘Project’ Goals / Success
Factors
• Role based profile and criteria
• Role development path
• Calibration mechanism and criteria
• Evaluation mechanism and criteria
• Leveling the different quality engineers and
‘auditors’ in the XYZ ORG by specific
performances and graduations
![Page 8: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/8.jpg)
The ‘Project’ Goals / Success
Factors
• Flowing specific trainings (on different levels) as
personal development and qualification criteria
• Listing specific performances as indicators for
leveling justifications
• Structuring the different audits and reporting
guidelines in a single mandatory to follow (based
on MDD and SLA-BOK) process, to increase
calibration and quality of professional conduct and
results (for process and meaningfully
improvements)
![Page 9: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/9.jpg)
3/14/2013
![Page 10: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/10.jpg)
![Page 11: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/11.jpg)
![Page 12: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/12.jpg)
PP
SSD
SCON
CAR
PMC
M&A OT
REQM
SST
DAR
SAM
IRP
OPD
QPM
SD
PPQA
OID
RSKM
OPF
CAM
STSM CM
OPP
IPM
ISO 9000:2008 Correlation Snapshot
Management
Responsibility
Measurement
Analysis &
Improvement
Product
Realization
Resource
Management
Quality
Management
System
20 are covered
![Page 13: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/13.jpg)
PP
SSD
SCON
CAR
PMC
M&A OT
REQM
SST
DAR
SAM
IRP
OPD
QPM
SD
PPQA
OID
RSKM
OPF
CAM
STSM CM
OPP
IPM
PMBOK Correlation Snapshot
Integration
Scope
Time
Cost
Quality
15 PAs are covered
Procurement Risk Communications Human
Resource
![Page 14: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/14.jpg)
PP
SSD
SCON
CAR
PMC
M&A OT
REQM
SST
DAR
SAM
IRP
OPD
QPM
SD
PPQA
OID
RSKM
OPF
CAM
STSM CM
OPP
IPM
ITIL – CMMI Correlation Snapshot Service Delivery
Financial Management
for
IT Services
Capacity
Management
IT Service
Continuity Management
Availability
Management
Service Level
Management
![Page 15: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/15.jpg)
PP
SSD
SCON
CAR
PMC
M&A OT
REQM
SST
DAR
SAM
IRP
OPD
QPM
SD
PPQA
OID
RSKM
OPF
CAM
STSM CM
OPP
IPM
ITIL – CMMI Correlation Snapshot Service Support
The Service Desk
Incident Management
Problem Management
Configuration
Management
Change Management
Release Management
![Page 16: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/16.jpg)
PP
SSD
SCON
CAR
PMC
M&A OT
REQM
SST
DAR
SAM
IRP
OPD
QPM
SD
PPQA
OID
RSKM
OPF
CAM
STSM CM
OPP
IPM
ISO 20000 – CMMI
Correlation Snapshot Control processes
new or changed
services
Service delivery
process
Requirements for a
management system
Relationship processes
Resolution processes Release process Planning and
implementing
service management
![Page 17: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/17.jpg)
![Page 18: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/18.jpg)
![Page 19: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/19.jpg)
![Page 20: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/20.jpg)
![Page 21: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/21.jpg)
Supporting Tools for
High Maturity
![Page 22: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/22.jpg)
HLM ATM Training agenda
![Page 23: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/23.jpg)
High Level Maturity Appraisal
Team Member References
![Page 24: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/24.jpg)
High Level Maturity
Appraisal Team
Member
References
Statistical Methods
Chi-Square ..................................................................................................................................................... 4
Chi-Square Test for Goodness of Fit ......................................................................................................... 4
Chi-Square Test for Independence ........................................................................................................... 5
Assumptions and Accuracy of the Chi-Square Test .................................................................................. 7
Common Cause of Variation ......................................................................................................................... 8
Continuous Probability Distribution ........................................................................................................... 11
Control Chart ............................................................................................................................................... 14
Control Charts for Variables .................................................................................................................... 15
Design of Experiments (DOE) ...................................................................................................................... 19
Description .............................................................................................................................................. 19
When to Use DOE.................................................................................................................................... 19
DOE Procedure ........................................................................................................................................ 20
Discrete Probability Distributions ............................................................................................................... 22
One-sided Test ............................................................................................................................................ 22
Example ................................................................................................................................................... 23
Example ................................................................................................................................................... 24
One Sample t-test ....................................................................................................................................... 25
Two Sample t-test ....................................................................................................................................... 25
P-Value ........................................................................................................................................................ 26
Discrete Data ............................................................................................................................................... 26
Exponential Distribution ............................................................................................................................. 26
Probability density function ................................................................................................................ 26
Hypothesis Testing ...................................................................................................................................... 28
Null Hypothesis ....................................................................................................................................... 29
Alternative Hypothesis ............................................................................................................................ 29
Simple Hypothesis ................................................................................................................................... 29
Composite Hypothesis ............................................................................................................................ 30
![Page 25: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/25.jpg)
High Level Maturity
Appraisal Team
Member
References
Acknowledgments ............................................................................................................................ 1
Table of Content .............................................................................................................................. 1
Abstract ............................................................................................................................................ 1
1.1. Purpose ............................................................................................................................ 2
1.2. Objectives ......................................................................................................................... 2
1.3. CMMI Terminology .......................................................................................................... 2
1.4. Industry Terminology ....................................................................................................... 2
1.5. Terms and definitions ...................................................................................................... 2
1.6. Process Performance Models References in the CMMI Suite ......................................... 2
1.7. Process Performance Models relationships with Measurements in the CMMI Suite ..... 2
1.8. Relationships from PSP and TSP to support CMMI Measurements and high maturity ... 2
1.1 high level listen for in process performance models ....................................................... 4
1.1.1 Overview .................................................................................................................. 4
1.1.2 In Small Setting Organizations ................................................................................. 4
1.1.3 In Agile like development and teams ....................................................................... 4
1.1.4 In different type Organizations ................................................................................ 4
1.1.5 At Level 2 & 3 Organizations .................................................................................... 4
1.1.6 At Level 4 & 5 Organizations .................................................................................... 4
1.2 Business Objectives; quantitative objectives, Needs and Between ................................ 4
1.2.1 Business Objectives .................................................................................................. 4
1.2.2 Quantitative objectives ............................................................................................ 4
1.3 Analyzing the organization lifecycle and process ............................................................ 4
1.3.1 Key players in the process ........................................................................................ 4
1.3.2 Key users of performance models ........................................................................... 4
1.3.3 Key elements (building blocks) of the process ......................................................... 4
1.3.4 Domain Specific Perspective .................................................................................... 4
1.3.5 Process perspective .................................................................................................. 4
1.3.6 Organizational Role Perspective .............................................................................. 4
3.1 Project Management Process Areas ................................................................................ 5
3.1.1 Project Planning (PP) 5
![Page 26: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/26.jpg)
PPM-BOK PROCESS PERFORMANCE MODELS ........................................................................................................... 1
APPLICATION AND APPROPRIATE USAGE (METHODOLOGY) .................................................................... 1
ACKNOWLEDGMENTS ....................................................................................................................................... 1
TABLE OF CONTENT .......................................................................................................................................... 1
ABSTRACT ...................................................................................................................................................... 6
INTRODUCTION ........................................................................................................................................ 7
1.1. Purpose ...................................................................................................................................... 7
1.2. Objectives .................................................................................................................................. 7
1.3. CMMI Terminology .................................................................................................................... 7
1.4. Industry Terminology ................................................................................................................. 7
1.5. Process Performance Models References in the CMMI Suite .................................................... 7
1.6. Process Performance Models relationships with Measurements in the CMMI Suite ................ 7
1.7. Relationships from PSP and TSP to support CMMI Measurements and high maturity ............. 7
INTRODUCTION TO PROCESS PERFORMANCE MODELS ............................................................................ 8
1.1 Why to Measure the process performance ............................................................................... 8
1.2 What is all about? ..................................................................................................................... 8
1.3 Benefits from using process performance models ..................................................................... 9 1.3.1 Overview .............................................................................................................................................. 9 1.3.2 Application in Small Setting Organizations ........................................................................................... 9 1.3.3 Application in Agile like development and teams ................................................................................ 9 1.3.4 Application in different type Organizations ......................................................................................... 9
1.3.4.1 Software ..................................................................................................................................... 9 1.3.4.2 System ........................................................................................................................................ 9 1.3.4.3 Services ....................................................................................................................................... 9 1.3.4.4 Acquisition .................................................................................................................................. 9 1.3.4.5 Maintenance ............................................................................................................................... 9 1.3.4.6 Support ....................................................................................................................................... 9
1.3.5 Application and support low maturity organization ............................................................................. 9 1.3.6 At Level 2 & 3 Organizations ................................................................................................................ 9 1.3.7 At Level 4 & 5 Organizations ................................................................................................................ 9
1.4 The Measurements and Analysis Team ................................................................................... 10 1.4.1 As Organizational Level Functional Team ........................................................................................... 10
1.4.1.1 Position in the Organizational Structure ................................................................................... 10 1.4.1.2 Team Suggested Structure ........................................................................................................ 10 1.4.1.3 Team Roles, Responsibilities and authority .............................................................................. 10
1.4.1.3.1 Roles ..................................................................................................................................... 10 1.4.1.3.2 Responsibilities ..................................................................................................................... 10 1.4.1.3.3 Authority .............................................................................................................................. 10
1.4.1.4 Principles of Professional Conduct............................................................................................ 10 1.4.1.5 Basic Activities .......................................................................................................................... 10 1.4.1.6 Additional Activities .................................................................................................................. 10 1.4.1.7 Knowledge and Skills ................................................................................................................ 10 1.4.1.8 Personal Development Path ..................................................................................................... 10
1.4.2 As Group of Individuals ...................................................................................................................... 11 1.4.2.1 Position in the Business Organization Structure ....................................................................... 11 1.4.2.2 Position in the Quality Organization Structure ......................................................................... 11
![Page 27: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/27.jpg)
High Level Maturity
Lead Appraisal
Guideline
Acknowledgments ............................................................................................................................ 1
Table of Content .............................................................................................................................. 1
Abstract ............................................................................................................................................ 1
1.1. Purpose ............................................................................................................................ 2
1.2. Objectives ......................................................................................................................... 2
1.3. CMMI Terminology .......................................................................................................... 2
1.4. Industry Terminology ....................................................................................................... 2
1.5. Terms and definitions ...................................................................................................... 2
1.6. Process Performance Models References in the CMMI Suite ......................................... 2
1.7. Process Performance Models relationships with Measurements in the CMMI Suite ..... 2
1.8. Relationships from PSP and TSP to support CMMI Measurements and high maturity ... 2
1.1 High level SCAMPI scripts for process performance models ........................................... 4
1.1.1 Overview .................................................................................................................. 4
1.1.2 In Small Setting Organizations ................................................................................. 4
1.1.3 In Agile like development and teams ....................................................................... 4
1.1.4 In different type Organizations ................................................................................ 4
1.1.5 At Level 2 & 3 Organizations .................................................................................... 4
1.1.6 At Level 4 & 5 Organizations .................................................................................... 4
1.2 SCAMPI scripts for Business Objectives; quantitative objectives, Needs and Between .. 4
1.2.1 Business Objectives .................................................................................................. 4
1.2.2 Quantitative objectives ............................................................................................ 4
1.3 SCAMPI scripts for analyzing the organization lifecycle and process .............................. 4
1.3.1 Key players in the process ........................................................................................ 4
1.3.2 Key users of performance models ........................................................................... 4
1.3.3 Key elements (building blocks) of the process ......................................................... 4
1.3.4 Domain Specific Perspective .................................................................................... 4
1.3.5 Process perspective .................................................................................................. 4
1.3.6 Organizational Role Perspective .............................................................................. 4
3.1 Project Management Process Areas ................................................................................ 6
3.1.1 Project Planning (PP) 6
![Page 28: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/28.jpg)
Results and Benefits Methodology conclusions.
• Model formulation is complex, but evaluation is straightforward.
• Standard tools, easy to integrate on domain cases.
• Method is better at capturing SMEs insights and project dynamics.
• Opportunity to further include additional domains
SME related conclusions (preliminary).
• Small units have better acceptance for audits
• Engineering related organizations have a good case for higher maturity levels
• Critical dependency on allowed investment horizon.
• Appraisal costs impacts on overall investment not dominant.
• Benefits for organizations operating at volatile (uncertain) segments.
![Page 29: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/29.jpg)
Conclusions and Recommendations
• Method can be implemented in order to develop a decision
support system in the management of new product
development domain
• This model addresses various aspects of new product
development over multiple stages, and easily can be
adjusted to other domains
• The model can deal with quantitative and qualitative input
and missing data
• Method can be implemented to address our managerial
problems and to support our managers with strong ‘gate
keepers’
• The implementation of a graphical user interface hiding the
complexities of the method and model is significant
![Page 30: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/30.jpg)
Future Plans
• Continue refinement of model.
• Parameter ranges.
• Roadmap involving multiple SCAMPI activities.
• Complete Validation and Verification.
• Integrate work and receive feed from other
initiatives.
• Continue sharing with other domains
![Page 31: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/31.jpg)
Discussion Points
• Performance data
• Cost of poor planning
• Quantifying the operational impact
• Effecting and effected stakeholders
mapping
• Quantifying the impact of support planning
on the development teams
• Appling this model on other domains
![Page 32: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/32.jpg)
Questions ?
![Page 33: Assuring assessments leaders team member’s quality and qualification](https://reader033.fdocuments.us/reader033/viewer/2022052820/54b6c6994a7959a0608b45bf/html5/thumbnails/33.jpg)
Contact
Kobi Vider
K.V.P Consulting
Phone: +972522946676