EDAR Training Fall 2015
-
Upload
patrick-mcquinn -
Category
Documents
-
view
256 -
download
1
Transcript of EDAR Training Fall 2015
Information Technology Solution Engineering
Engineering Decision Analysis and Resolution (EDAR)
Spring 2015
Rodney DuBosePatrick McQuinn
Engineering Decision Analysis and Resolution | Solution Engineering
Agenda
• High Level Overview• Procedure Overview• Exercise• Contact
Engineering Decision Analysis and Resolution | Solution Engineering2
EDAR High Level Overview
• Training Objective• EDAR Purpose• When EDAR Is Not Done Well• EDAR High Level Overview
• Relationship to other processes• EDAR Specific Goal• EDAR Specific Practices• EDAR Work Products
Engineering Decision Analysis and Resolution | Solution Engineering3
Training Objective
• To increase understanding of the Engineering Decision Analysis and Resolution (EDAR) procedure
• Institutionalize repeatable engineering design and development processes and procedures
Engineering Decision Analysis and Resolution | Solution Engineering4
EDAR Purpose
The purpose of EDAR is to analyze possible decisions using a formal evaluation procedure that evaluates identified alternatives given established criteria.
Engineering Decision Analysis and Resolution | Solution Engineering5
When EDAR Is Not Done Well• It is unclear who is authorized to make what decisions.
• Decisions are made subjectively.
• Rationale is unavailable when needed to understand an earlier decision.
• Too few choices are considered for major decisions.
• Missing a more optimal solution costs time, money, credibility, and perhaps even the whole project.
Engineering Decision Analysis and Resolution | Solution Engineering6
High Level OverviewRelationship to Other Processes
• EDAR is a Technical Solution (TS) Procedure
• EDAR is a CMMI Support Process Area Process
• The CMMI Support Process Area covers the activities that support product development and maintenance.
Engineering Decision Analysis and Resolution | Solution Engineering7
High Level OverviewRelationship to Other Processes
Engineering Decision Analysis and Resolution | Solution Engineering8
EDAR Specific Goal
Engineering Decision Analysis and Resolution | Solution Engineering9
SG1Evaluate Alternatives
Decisions are based on an evaluation of Alternatives using established criteria.
EDAR Specific Practices
Engineering Decision Analysis and Resolution | Solution Engineering10
SG1
Evaluate Alternatives
SP 1.1 Use The EDAR GuidelineSP 1.2 Establish Evaluation CriteriaSP 1.3 Identify Alternative SolutionsSP 1.4 Select Evaluation MethodsSP 1.5 Evaluate Alternative SolutionsSP 1.6 Select Solutions
EDAR Work Products
Engineering Decision Analysis and Resolution | Solution Engineering11
Problem Statement
Use the EDAR Guidelines
SP 1.1
Recommended Solutions
SelectSolutions
SP 1.6
Evaluation Results
Evaluate Alternative Solutions
SP 1.5
Methods
Select Evaluation Methods
SP 1.4
Proposed Alternatives
Identify Alternative Solutions
SP 1.3
Criteria
Establish Evaluation
Criteria
SP 1.2
Agenda
• High Level Overview• Procedure Overview• Exercise• Contact
Engineering Decision Analysis and Resolution | Solution Engineering12
EDAR Process Steps
1. Use the EDAR Guideline
2. Establish Evaluation Criteria
3. Identify Alternative Solutions
4. Select Evaluation Methods
5. Evaluate Alternative Solutions
6. Select Solutions
Engineering Decision Analysis and Resolution | Solution Engineering13
EDAR Step 1 ActivitiesUse the EDAR Guidelines
Engineering Decision Analysis and Resolution | Solution Engineering14
Role Description
Definition of Responsibility
Lead analyst •Obtain the EDAR Guidelines.
•Determine whether or not the stated problem falls within the scope of the EDAR Guidelines.
•Communicate the decision to conduct a formal evaluation.
Start Obtain the EDAR Guidelines.
A1
Use the EDAR Guidelines.
A2
Communicate the decision.
A3
Problem within
Guideline scope?
StopNo
Yes
1
ProblemStatement
EDAR Step 1 ActivitiesUse the EDAR Guidelines
Engineering Decision Analysis and Resolution | Solution Engineering15
Activity Steps RolesA1: Obtain the EDAR Guidelines
1. Download the EDAR Guidelines
Lead Analyst
A2: Use the EDAR Guidelines
1. Determine whether or not the stated problem falls within the scope of the EDAR Guidelines.
2. If not then stop.3. If yes, then proceed to step A3.
A3: Communicate the Decision
1. Communicate the decision to conduct a formal evaluation.
EDAR Guidelines, Summary
1. General
2. Insertion of New Technologies
3. Acquisition of New Hardware/Software Products or New Tools
Engineering Decision Analysis and Resolution | Solution Engineering16
EDAR Guidelines, cont., 1/4
1. General Guidelines – Any project engineering effort may employ formal evaluation methods and is welcome to use the EDAR procedure by accessing the IT PAL and executing the EDAR Procedure that is associated with Engineering Technical Solution. The procedure is generic enough that it will apply to any need to conduct a formal evaluation of technical solution alternatives. Each use of the EDAR procedure requires that evaluation criteria be established for the specific issue being evaluated. It is well known that evaluation criterion will vary depending on the specifics of the alternatives being evaluated.
Engineering Decision Analysis and Resolution | Solution Engineering17
EDAR Guidelines, cont., 2/4
2. Guidelines for Insertion of New Technologies – Whenever new technologies are being considered for introduction into the IRS computing environments, a formal evaluation is required. The function, organization, or project considering the new technology is required to engage Enterprise Architecture who will lead, participate, or give guidance in the conduct of the EDAR activities. In the event the evaluation will also select new HW/SW products or tools, Solution Engineering will also be engaged to either participate in the EDAR activities or give guidance as to the follow-on conduct of a Product Integration activity.
Engineering Decision Analysis and Resolution | Solution Engineering18
3. Guidelines for Purchase of New Hardware/Software Products or New Tools – Whenever new HW/SW products or tools are being considered for purchase and introduction into IRS computing environments, a formal evaluation is required. The function, organization, or project is required to engage Enterprise Architecture who will advise as to whether or not they should be involved in the EDAR activities. Solution Engineering must also be engaged for participation in the EDAR activities and to determine whether or not a Product Integration activity will be conducted.
Engineering Decision Analysis and Resolution | Solution Engineering19
EDAR Guidelines, cont., 3/4
Engineering Decision Analysis and Resolution | Solution Engineering20
EDAR Guidelines, cont., 4/4
EDAR Step 2 ActivitiesEstablish Evaluation Criteria
Engineering Decision Analysis and Resolution | Solution Engineering21
Role Description
Definition of Responsibility
Lead Analyst • Define the criteria for evaluating alternative solutions for the specific problem to be solved.
• Define the scale of relative importance for ranking evaluation criteria.
• Assign the weight of each criterion.• Document the rationale for the selection
of evaluation criteria.
Roles and Responsibilities 1
2
Document the rationale for the
selection of evaluation criteria.
A7
Assign the weight of each criterion.
A6
Define the scale of relative importance for
ranking evaluation criteria.
A5
Define the criteria for evaluating alternative
solutions for the specific problem to be
solved.
A4
EDAR Step 2 ActivitiesEstablish Evaluation Criteria
Engineering Decision Analysis and Resolution | Solution Engineering22
Activities Steps Roles
A4: Define the criteria for evaluating alternative solutions for the specific problem to be solved.
1. Define criteria for evaluation of the problem to be solved.
2. Consult with subject matter experts for project, programmatic, security, and vendor aspects as applicable.
3. Document the evaluation criteria.
Lead Analyst
A5: Define the scale of relative importance for ranking evaluation criteria.
1. Define a numerical scale to be applied in assigning relative importance to the criteria.
2. Define the numerical scale from lower relative importance to higher relative importance. i.e. lower numbers to be applied to criteria of lesser importance, higher numbers to criteria of greater importance.
A6: Assign the weight of each criterion.
1. Assign a relative weight percentage to each criterion.NOTE: the sum of all criteria weights must equal 100%.
A7: Document the rationale for the selection of evaluation criteria.
1. Document the rationale for criteria selection.
EDAR Step 3 ActivitiesIdentify Alternative Solutions
Engineering Decision Analysis and Resolution | Solution Engineering23
Role Description
Definition of Responsibility
Analysts • Discover candidate alternative solutions.
• Document proposed alternatives.
Roles and Responsibilities2
3
Discover candidate alternative solutions.
A8
Document proposed alternatives.
A9
EDAR Step 3 ActivitiesIdentify Alternative Solutions
Engineering Decision Analysis and Resolution | Solution Engineering24
Activities Steps Roles
A8: Discover candidate alternative solutions.
1. Perform discovery to uncover similar previous alternative analyses both inside and outside the organization to expose alternative candidates and barriers to implementation.
2. Discover relevant existing trade studies and lessons learned from previous formal decisions.
3. Identify priorities of relevant stakeholders and technical/logistics challenges using evaluation criteria.
4. Conduct interviews and workshops with relevant stakeholders to solicit and uncover candidate alternatives.
5. Use available research tools (e.g. Forrester).
Analyst(s)
A9: Document proposed alternatives.
1. Document the proposed alternatives.
EDAR Step 4 ActivitiesSelect Evaluation Methods
Engineering Decision Analysis and Resolution | Solution Engineering25
Role Description
Definition of Responsibility
Lead Analyst
• Select the evaluation method(s).
• Determine the measures needed to support the evaluation method(s).
• Document the selected valuation method(s).
Roles and Responsibilities3
4
Select evaluation method(s).
A10
Determine the measures needed to support the evaluation
method(s).
A11
Document the selected evaluation method(s).
A12
EDAR Step 4 ActivitiesSelect Evaluation Methods
Engineering Decision Analysis and Resolution | Solution Engineering26
Activities Steps Roles
A10: Select evaluation method(s).
1. Select methods based on the following:• The problem statement.• The level of the availability of and confidence in
the information used to support the method.• The ability to focus on the issues at hand without
being overly influenced by side issues.
Lead AnalystA11: Determine the
measures needed to support the evaluation method(s).
1. Develop measures for each proposed method that will, when collected, aid in the selection of the most appropriate evaluation method(s) through the analysis of the method’s impact on cost, schedule, performance, and risks.
A12: Document the selected evaluation method(s).
1. Document the selected evaluation method(s) and supporting justification.
EDAR Step 5 ActivitiesEvaluate Alternative Solutions
Engineering Decision Analysis and Resolution | Solution Engineering27
Roles and Responsibilities Role Description Definition of ResponsibilityLead Analyst • Identify the Decision Making Authority
• Identify the stakeholders• Select key personnel (Analysts) to perform
the process• Distribute the formal report, if required
Analyst • Perform the evaluation and provide a recommended solution (one person or a team, one of whom is designated leader)
• Request additional resources if required.
Higher Authority • Allocate additional resources, if required• Discuss the EDAR results with the Lead
Analyst
Stakeholders • Provide information as requested by the EDAR team to perform the analysis of alternative solutions
Decision Making Authority
• Receive the documented evaluation results
4
5
Develop evaluation artifacts.
A14
Evaluate the assumptions related to the evaluation criteria and the evidence that support
the assumptions.
A15
Evaluate the identified alternatives.
A16
Document the evaluation results and recommend alternative
solutions.
A17
Distribute the evaluation results and
recommendations.
A18
Prepare for evaluation of proposed alternative solution(s).
A13
EDAR Step 5 ActivitiesEvaluate Alternative Solutions – 1/3
Engineering Decision Analysis and Resolution | Solution Engineering28
Activities Steps Roles
A13: Prepare to evaluate proposed alternative solutions.
1. Identify Analyst(s)2. Identify Stakeholders Lead Analyst
3. Acknowledge Roles Analyst(s)
Stakeholders
A14: Develop evaluation artifacts.
1. Populate the EDAR evaluation matrix with evaluation criteria, criteria weights, identified alternatives and selected method(s).
2. Develop man-hour estimates for development and operations.
3. Develop cost estimates for development and operations.
4. Develop expected technology end of life estimate.
Analyst(s)
A15: Evaluate the assumptions related to the evaluation criteria and the evidence that support the assumptions.
1. Develop assumptions and supporting arguments.
2. Document assumptions related to evaluation criteria.
Analyst(s)
EDAR Step 5 ActivitiesEvaluate Alternative Solutions (cont. 2/3)
Engineering Decision Analysis and Resolution | Solution Engineering29
Activities Steps Roles
A16: Evaluate the identified alternatives.
1. Using the method(s) selected, and evaluation criteria, evaluate the identified alternative solutions.
2. Request additional funding if necessary. 3. Assign raw scores using the numerical
scale developed in EDAR step 2 for each criterion relative to each alternative. Record scores in the evaluation matrix.
4. Calculate the value of each criterion relative to each alternative. (value=(raw score) *(criterion weight) Record values in the evaluation matrix.
5. Calculate the rank of each alternative solution by summing the values of each alternative’s criteria. Record ranks in the evaluation matrix.
Analyst(s)
6. Disposition requests for additional funding.
Higher Authority
EDAR Step 5 ActivitiesEvaluate Alternative Solutions (cont. 3/3)
Engineering Decision Analysis and Resolution | Solution Engineering30
Activities Steps Roles
A17: Document the evaluation results and recommend alternative solution(s).
1. Document the evaluation results. 2. Document the recommended
alternative solution(s). Analyst(s)
A18: Distribute evaluation results and recommendations.
1. Receive and distribute the completed EDAR evaluation documentation form.
Lead Analyst
2. Receive the evaluation results. Decision Making Authority
EDAR Step 6 ActivitiesSelect Solutions
Engineering Decision Analysis and Resolution | Solution Engineering31
Role Description
Definition of Responsibility
Decision Making Authority
• Select solutions.• Document and
communicate the recommended solution.
Roles and Responsibilities
End
5
Select Solution(s).
A19
Document and communicate the recommended solution.
A20
EDAR Step 6 ActivitiesSelect Solutions
Engineering Decision Analysis and Resolution | Solution Engineering32
Activities Steps Roles
A19: Select solutions.
1.Examine the documented evaluation results and recommended solutions.2.Accept security risks associated and documented with the recommended solutions.3.Instruct the Lead Analyst to repeat EDAR steps required to mitigate identified risks or concerns.4.Select solutions.
Decision Making Authority
A20: Document and communicate the recommended solutions.
1.Document the solutions selected.
Agenda
• High Level Overview• Procedure Overview• Exercise• Contact
Engineering Decision Analysis and Resolution | Solution Engineering33
Exercise Problem StatementTo build and install a solution to the following:To build and install a solution to the following:
•Business NeedBusiness Need: To process paper checks in a timely basis.: To process paper checks in a timely basis.
•Business CaseBusiness Case: Every day, IRS Taxpayer Assistance Centers, Consolidated : Every day, IRS Taxpayer Assistance Centers, Consolidated Campuses, and Revenue Officers accept tax payments in the form of paper Campuses, and Revenue Officers accept tax payments in the form of paper remittances (i.e., checks, money orders). As these IRS entities are not equipped to remittances (i.e., checks, money orders). As these IRS entities are not equipped to fully process and deposit the remittance, so paper checks must be controlled, fully process and deposit the remittance, so paper checks must be controlled, packaged, and transported via overnight mail to a Submission Processing Center. packaged, and transported via overnight mail to a Submission Processing Center. This adds days to the time a payment can be deposited, and creates more employee This adds days to the time a payment can be deposited, and creates more employee effort than would otherwise be necessary. effort than would otherwise be necessary.
•Customer RequirementsCustomer Requirements::• Process Paper checks at the point of receipt.• Maintain a copy of the check and associated information in a central
repository.• Deposit funds.
Engineering Decision Analysis and Resolution | Solution Engineering34
Exercise: Procedure Step 1Exercise: Procedure Step 1Use the EDAR GuidelinesUse the EDAR Guidelines
Questions for determination:Questions for determination:•Does the Problem Statement indicate that a new technology Does the Problem Statement indicate that a new technology will be introduced into the IT enterprise? (Answer: NO)will be introduced into the IT enterprise? (Answer: NO)
•Does the Problem Statement indicate that new HW/SW will be Does the Problem Statement indicate that new HW/SW will be introduced into the IT enterprise? (Answer: NO)introduced into the IT enterprise? (Answer: NO)
•Does the Project Management believe that a EDAR is indicated Does the Project Management believe that a EDAR is indicated justified by the EDAR guideline General clause? (Answer: Yes)justified by the EDAR guideline General clause? (Answer: Yes)
Engineering Decision Analysis and Resolution | Solution Engineering35
Exercise: Procedure Step 2/3Exercise: Procedure Step 2/3Selection Criteria and Alternative Selection Criteria and Alternative SolutionsSolutions
Some questions for discussion:Some questions for discussion:• What Criteria should be used to select the approach used?What Criteria should be used to select the approach used?• What are some alternative approaches to meet the customer What are some alternative approaches to meet the customer
requirements?requirements?• Consider:Consider:• What technologies are Available?What technologies are Available?• What can be reused?What can be reused?• Operations Concept for approachOperations Concept for approach
• What are the alternative solutions?What are the alternative solutions?
Engineering Decision Analysis and Resolution | Solution Engineering36
Exercise: Procedure Step 2/3 Identify Criteria and Alternatives
• Derived Criteria:• Cost, Schedule, Risk, Performance, EA Compliance,
Maintainability• (NOTE: Criteria used in this exercise are arbitrary but
plausible.)• For the purpose of the exercise assume the approach uses the
following 3 alternatives:• Current System: (Keeping current system)• New Centralized System• Modify Existing ISRP System
• Make sure all requirements are allocated to each alternative
Engineering Decision Analysis and Resolution | Solution Engineering37
Weight the Evaluation Criteria
Overview• Weighting criteria involves assigning a ranking or relative
weight to each criterion to reflect its importance to the customer.
• NOTE: The term “weight” as used in the statistical evaluation of criteria is a verb, not a noun. This usage is common to the EDAR procedure and has become a best practice in the discipline.
Engineering Decision Analysis and Resolution | Solution Engineering38
Weight the Evaluation Criteria, Cont.
Analytic Hierarchy MethodThe Analytic Hierarchy Method (AHM) is a decision-making process
developed by Dr. Thomas L. Saaty, University of Pittsburgh, in the early 1980s. Rather than prescribing a "correct" decision, the AHM helps decision makers find one that best suits their goal and their understanding of the problem. It provides a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions. This section shows how to derive weighting factors (W) for the selection criteria using the Analytic Hierarchy Method. The Analytic Hierarchy Method is the preferred method for deriving weighting criteria and consists of the following steps that may be implemented on a standard PC-based spreadsheet.
Engineering Decision Analysis and Resolution | Solution Engineering39
Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix
• Perform Pair-Wise Comparison of Selection Criteria• The basis of the weighting process is to perform a pair-wise comparison of all of the selection criteria.
For example, suppose you have six selection parameters: cost, schedule, performance risk, EA compliance, and Maintainability. You can construct a 6x6 matrix as follows:
Engineering Decision Analysis and Resolution | Solution Engineering40
NOTE: 1.Number of Pair-Wise Comparisons = (N2 –N)/22.Number of Pair-Wise Comparisons increases exponentially with the number of criteria. (Motivation to keep the number of criteria to a minimum!)
Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix
•Assess the relative priority of one parameter to the other for each pair, using the sliding pair-wise comparison scale shown below.
Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 2Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 3Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 3Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6
Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix
• Assess the relative priority of one parameter to the other for each pair, using the sliding pair-wise comparison scale shown below.
Engineering Decision Analysis and Resolution | Solution Engineering42
Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix
• The numerical values on the scale have the following meanings:1: Equal importance2-4: More important5-7: Much more important7-9: Extremely more important
• If the two parameters in a pair are of equal importance, designate the 1 on the scale. If one is more important than the other, designate the appropriate number (as defined above) on the side where the more important parameter exists. For instance, if cost is more important than schedule, designate the 4 nearest the cost, as shown in the next slide.
Engineering Decision Analysis and Resolution | Solution Engineering43
Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix
Engineering Decision Analysis and Resolution | Solution Engineering44
Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ScheduleCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 PerformanceCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 PerformanceSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskPerformance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.EA Compl. 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.
The completed sliding pair-wise comparison matrix:
Weight the Evaluation Criteria, Cont.The Priority Matrix
Engineering Decision Analysis and Resolution | Solution Engineering45
Cost Schedule Performance Risk EA Compl. Maintain.
Cost 1 4 3 4 1/5 1/4
Schedule 1/4 1 1/5 1/4 1/5 4
Performance 1/3 5 1 5 1/5 5
Risk 1/4 4 1/5 1 1/5 1
EA Compl. 5 5 5 5 1 5
Maintain. 4 1/4 1/5 1 1/5 1
Complete the Priority MatrixWhen the pair-wise comparisons have all been completed, enter the results into the priority matrix using the following rules: *The cells along the diagonal of the matrix (pairing a parameter with itself) are always assigned a value of 1 .*If the two parameters are rated as equally important, enter the value 1 into the cell;*If the parameter along the row in the matrix is more important than the parameter in the column, enter the value (2 to 9) into the cell;*If the parameter in the column is more important than the parameter in the row, enter the reciprocal of the value (1/2 to 1/9) into the cell.
Weight the Evaluation Criteria, Cont.The Priority Matrix
Engineering Decision Analysis and Resolution | Solution Engineering46
Cost Schedule Performance Risk EA Compl. Maintain.Cost 1 4 3 4 1/5 1/4Schedule 1/4 1 1/5 1/4 1/5 4 Performance 1/3 5 1 5 1/5 5 Risk 1/4 4 1/5 1 1/5 1 EA Compl. 5 5 5 5 1 5 Maintain. 4 1/4 1/5 1 1/5 1 Column Total 10.83 19.25 9.60 16.25 2.00 16.25
Normalize the Priority MatrixNormalization is the statistical method used to equalize the scale used to measure each criteria parameter. In this case the scale chosen for normalization is 1 or 100%. To normalize the priority matrix, sum each column as shown below.
Weight the Evaluation Criteria, Cont.Calculate the Weights
Engineering Decision Analysis and Resolution | Solution Engineering47
Cost Schedule Performance Risk EA Compl. Maintain. WeightCost 0.092 0.208 0.313 0.246 0.100 0.015 16.24%Schedule 0.023 0.052 0.021 0.015 0.100 0.246 7.62%Performance 0.031 0.260 0.104 0.308 0.100 0.308 18.50%Risk 0.023 0.208 0.021 0.062 0.100 0.062 7.91%EA Compl. 0.462 0.260 0.521 0.308 0.500 0.308 39.29%Maintain. 0.369 0.013 0.021 0.062 0.100 0.062 10.44%
Normalization 1.000 1.000 1.000 1.000 1.000 1.000 100.00%
• Next, divide the value in each cell by its column total. • For example, in the cell Cost:Schedule: (4)/(19.25) = 0.208
• Compute Weighting Factors• The weighting factor (W) for each parameter may now be computed by summing along
the parameter's row in the matrix and dividing by the number of selection parameters, N = 6 (in this case).
• W (cost) = (0.092 + 0.208 + 0.313 + 0.246 + 0.1 + 0.15)/6 = 0.1624• W (schedule) = (0.023 + 0.052 + 0.021 + 0.015 + 0.1 + 0.246)/6 = 0.0762• W (performance) = (0.031 + 0.26 + 0.104 + 0.308 + 0.1 + 0.308)/6 = 0.185• etc…
• The sum of the weighting factors should add up to the whole number one. [NOTE: Multiply the weighting factors by 10 if you want them on a 10-point scale or 100 for percentage.]
Weight the Evaluation Criteria, Cont. Checking the Consistency
Engineering Decision Analysis and Resolution | Solution Engineering48
Check Consistency• Occasionally, the pair-wise comparisons may not be consistent. Instead of the sample values used
above, suppose that you obtained the following results:Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Schedule
Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Performance
Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.
Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Performance
Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskPerformance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.EA Compl. 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.
• Here, cost is more important than schedule and equally as important as performance. Schedule is also rated as equally as important as performance. This result seems inconsistent. If Cost > Schedule and Schedule = Performance, then logically you should have Performance > Schedule. Instead, you have Schedule = Performance. Some inconsistency is to be expected when comparing such diverse parameters as schedule, performance, and cost, and reflects subjective human judgment, which cannot be held to such mathematical precision as: if a = b and b = c, then a = c. The question is how such inconsistencies affect the derived weighting factors (W).
Identify the Alternatives
Engineering Decision Analysis and Resolution | Solution Engineering49
Discover candidate alternative solutions.
•Current System
•New Centralized System
•Modify Existing ISRP System
Select Evaluation Methods• Overview• Using the previously identified alternative solutions, select the evaluation methods to be
used during the evaluation step. The below example matrix is intended to aid in the selection of methods.
Engineering Decision Analysis and Resolution | Solution Engineering50
Evaluation Method
Alt 1 Current ISRP System
Alt 2 New Centralized System
Alt 3 Modified ISRP System
Cost Schedule Perf. risk EA Compl. Maintain.
• Example evaluation method types for typical IT Problems:• Testing• Modeling and simulation• Engineering Studies• Manufacturing studies• Extrapolations based on field experience and prototypes• Judgment provided by an expert or group of experts (e.g. Forrester)
Scoring the Alternatives• Overview• The purpose of this exercise is to present a method for scoring the
alternative solutions given that the units and ranges of the evaluation criteria vary widely.
• The evaluation criteria consist of a diverse set of parameters, such as cost, schedule, and performance. The units of these evaluation criteria vary widely. For instance, schedule may be expressed as time consumed by humans in hours, while cost is given in dollars.
• The method used in this exercise is called “Normalization”
Engineering Decision Analysis and Resolution | Solution Engineering51
Scoring the Alternatives, Cont. Calculate the Score
• Alternatives Scoring Method: NormalizationNormalization involves taking a ratio between the value of a given alternative and
the alternative with the best value. If the best value is defined as a maximum, as for schedule in the table below, the ratio is given by the following formula. In this case, alternative 1 has the best value (200 hours).
• If the best value is defined as a minimum, as for cost in the table, the ratio is given by the following formula. For cost, alternative 1 has the best value ($600K).
S (alternative n) = (alternative n value)/(best alternative value) • If the best value is defined as a maximum, as for performance in the table, the
ratio is given by the following formula. For cost, alternative 1 has the best value ($600K). S (alternative n) = (best alternative value)/(alternative n value)
• In either case, the alternative with the best value will receive a score of 1, and all other alternatives will receive a score between 0 and 1. If you wish to operate on a 10-point scale, multiply the normalized score by 10.
Engineering Decision Analysis and Resolution | Solution Engineering52
Scoring the Alternatives, Cont. Calculate the Score
Engineering Decision Analysis and Resolution | Solution Engineering53
Selection Criteria
Alt 1 Current ISRP System
Alt 2 New Centralized
SystemAlt 3
Modified ISRP System
Value (S) Score Value (S) Score Value (S) ScoreCost 600K$ 1 2.2M$ 0.27 1.2M$ 0.5Schedule 200hrs. 1 2000hrs. 0.1 1000hrs. 0.2Performance 21Mbps 0.84 25Mbps 1 22Mbps 0.88Risk 300K$ 1 1.4M$ 0.21 700K$ 0.43EA Compl. 200hrs. 1 2000hrs 0.1 1600hrs. 0.13Maintain. 2.6M$/yr 0.54 1.4M$/yr. 1 1.7M$/yr. 0.82
Alternatives Analysis Matrix• Alternatives Analysis Matrix• The Alternatives Analysis Matrix presents all of the information required
by the Decision Making Authority to select a solution for recommendation. Include this matrix in the EDAR Report.
Engineering Decision Analysis and Resolution | Solution Engineering54
Alternatives Analysis
CriteriaWeigh
t
Alt 1 Alt 2 Alt 3Scor
e ValueScor
e ValueScor
e Value
Cost W1 S11 V11=(W1*S11) S21 V21=(W1*S21) S31 V31=(W1*S31)
Schedule W2 S12 V12=(W2*S12) S22 V22=(W2*S22) S32 V32=(W2*S32)
Performance W3 S13 V13=(W3*S13) S23 V23=(W3*S23) S33 V33=(W3*S33)
Risk W4 S14 V14=(W4*S14) S24 V24=(W4*S24) S34 V34=(W4*S34)
EA Compl. W5 S15 V15=(W5*S15) S25 V25=(W5*S25) S35 V35=(W5*S35)
Maintain. W6 S16 V16=(W6*S16) S26 V26=(W6*S26) S36 V36=(W6*S36)
Rank: R1=∑ V11:V16 R2 =∑ V21:V26 R3 =∑ V31:V36
Normalized Rank: = R1/ ∑ (R1:R3)*100% = R2/ ∑( R1:R3)*100% = R3/ ∑( R1:R3)*100%
Alternatives Analysis Matrix,Rank the Alternatives
Engineering Decision Analysis and Resolution | Solution Engineering55
Alternatives Analysis
Criteria WeightAlt 1
Current ISRP SystemAlt 2
New Centalized SystemAlt 3
Modified ISRP System
Score Value Score Value Score ValueCost 0.1624 1 0.1624 0.27 0.043848 0.5 0.0812Schedule 0.762 1 0.762 0.1 0.0762 0.2 0.1524Performance 0.185 0.84 0.1554 1 0.185 0.88 0.1628Risk 0.791 1 0.791 0.21 0.16611 0.43 0.34013EA Compl. 0.3929 1 0.3929 0.1 0.03929 0.13 0.051077Maintain. 0.1044 0.54 0.056376 1 0.1044 0.82 0.085608
Rank: 2.320076 0.614848 0.873215Normalized Rank: 61% 16% 23%
Select Solutions
• Select solutions. 1. Current ISRP System (Rank 61%)
2. Modified ISRP System (Rank 23%)
3. New Centralized System (Rank 16%)
• Document and communicate the recommended solutions.
Engineering Decision Analysis and Resolution | Solution Engineering56
Engineering Decision Analysis and Resolution | Solution Engineering57
Questions
Engineering Decision Analysis and Resolution | Solution Engineering58
Contact for EDAR