Dickey.alan

Determining Requirements Compliance During the Design Phase for a System of Systems

February 10, 2011

Alan Dickey, Wyle Integrated Science and Engineering

Stephen Chan, NASA JSC

Shana McElroy, Booz Allen Hamilton

Requirements Design Compliance

Outline

Introduction and Background

Purpose and Objectives

Methodology Details

Influencing Design

Connecting to Verification

Making it Work for a System of Systems Architecture

- -


Introduction – Project Life Cycle Phases

SRR SDR PDR CDR SAR

KeySRR – System Requirements ReviewSDR – System Definition ReviewPDR – Preliminary Design ReviewCDR – Critical Design ReviewSAR – System Acceptance Review

Requirements Development/ Achievability

Design Qualification and Verification

DesignFocus on establishing program requirements that demonstrate capability to meet Agency goals and

objectives

Period of time used to define the detailed design

of the systemFocus on developing confidence that the integrated system will be

able to satisfy technical requirements


Introduction - How Requirements Design Compliance Fits

Design

Requirem

ents

Risk and Safety Operatio

ns

The design phase must adeptly balance the interplay between hardware/software design and the requirements while managing an acceptable level of safety/risk, all within the context of the defined operations concepts


Introduction - Why Requirements Design Compliance?

A system of systems is complex and has many variables – making an objective decision to proceed from one design phase to the next is hard and Requirements Design Compliance is a systematic method to bring more objectivity into that decision making process

Risk can be found early by measuring the requirement achievability of design concepts with the same measuring stick that will be used in the final product

Requirement verification is the finish-line by which we declare success for product delivery, therefore requirement compliance assessments can be used to provide periodic dry-runs for verification

NPR 7123 Success Criteria (as it relates to requirements design compliance):

At PDR: The preliminary design is expected to meet the requirements at an acceptable level of risk

At CDR: The detailed design is expected to meet the requirements with adequate margins at an acceptable level of risk

Introduction – Constellation Program Structure


Constellation (Cx) Program Office


Background - Requirement Flow Down and Decomposition

ISS Crew Capacity

Orion

Ares 1

Orion Element

EVAMission Systems

Ex-0010-01Cx

Orion Habitable Volume

Orion Low Earth Orbit Crew Capability

Mission Systems Crew Training

CA0426-PO

CA1000-PO

CA5129-PO

EVA1067

CA0447-PO

CV0085

SS-SC-2256

CV0874

Orion Automated De-Orbit from Low Earth Orbit

Low Earth Orbit Capability

Orion Un-Crewed Flight Configuration

CA0447-PO Orion Automated Rendezvous, Prox Ops, Dock/Undock

EVA System Crew Size

Ares 1 Lift Mass for ISS Missions

Requirement decomposition, flow down and traceability of top level Architecture requirements set the stage for Requirements Design Compliance● For Cx, Architecture requirements are implemented through Program and allocated

to Projects (systems), and flowed down to the design implementation level● All Cx requirements are assigned owners and mapped to stakeholders

Exa

mp

le

Requirements Design Compliance Purpose & Objectives

Purpose of Requirements Design Compliance is to identify and establish resolution paths for integrated architectural performance/design issues as early in the design cycle as possible Proactively manage ‘acceptable level of risk’ Early issue resolution/prevention reduces cost and schedule impacts End-to-End mission architecture perspective

Key objectives of Requirements Design Compliance include: Early identification of design aspects that are at high risk to meet the established

requirements and perform end-to-end Utilize objective design evidence to determine success of design cycle, from

architecture perspective of the design reference mission (DRM) and operations concepts (ops con)

Facilitate vertical integration of design compliance data with Projects and horizontal integration across Program (i.e. provide bigger picture for lower level requirements)

Resolve, report and track significant architecture compliance issues (includes impact analysis based on lover level non-compliances)

Engage architecture requirement owners in how the design will be verified and create a relationship between the tools/process for compliance with that used for verification (‘get their hands dirty’ early)


Connects the beginning (requirements) to the end (verification) and provides a means to track the health of the architecture and minimize risk

Requirements Design Compliance Methodology

ProjectDCMs

ProjectDCMs

ProjectDCMs

ProjectDCMs


Requirements Design Compliance Methodology - Issue Handling Processes

Reqmt Owners

ProjectData

RequirementStakeholders

Risk Plans

TPMs

Analyses

Requirement Owners Design

Compliance Assessments

New issue handling measures created to mitigate new issues

Status used as assessment input

Change Request

Technology

Requirement Owners use objective data provided by projects, stakeholders, architecture analyses and issue handling processes to conduct requirement design compliance assessments as an integrated team/process activity. Issues fed back to

architecture or system issue resolution processes as appropriate.

Architecture/System Processes


Requirements Design Compliance Methodology, continued

Requirements Design Compliance relies on objective evidence to identify issues that could impact verifying the architecture requirements

• Objective evidence helps limit subjectivity when assessing realistic level of risk; however, objective evidence must be combined with sound judgment – design phase data may be incomplete and sound judgment is needed to help bridge the gap

• As the Project moves forward through its life cycle, more objective data is necessary to lead to verification compliance

• Lack of any objective evidence indicative of either non-compliant design or design behind schedule

Acceptable objective evidence can be of many types, such as:• Analysis and Independent Assessment Results – approved and peer reviewed• Technical Performance Metric (TPM) Reports – e.g., monthly mass status• Inspection and Demonstration Reports• Test Reports• Design Data Deliverables • Engineering judgment based on historical similarity

Traceability permits objective evidence to be readily linked to requirements

Objective evidence is key to Requirements Design Compliance but thinking must stay in the equation

Requirements Design Compliance Methodology - Definitions

Architecture Compliance Definitions Compliant (Green) – The overall technical design is expected to meet the

requirement at an acceptable level of risk.

Watch (Yellow) – The overall technical design does not currently meet the requirement,

but an acceptable mitigation plan has been identified and documented. There exists a specific significant threat(s) that requires additional

mitigation plans to be developed – actions assigned and accepted.

Non-Compliant (Red) – The overall technical design is not expected to meet the requirement and an acceptable mitigation plan has not been identified. In addition, the requirement has systemic threats that permeates the overall design compliance.

Acceptable Level of Risk - No known technical issues exist which will impact meeting the requirement; or, appropriate issue management processes (risk mitigation plans, Technical Performance Measures, etc.) are in place and indicate issue(s) will be mitigated to meet program baseline schedule. See graphic on next chart.


Requirements Design Compliance Methodology - Acceptable Level of Risk

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0

Task 1Task 2

Task 3 Task 4

Task 5Task 6

Time

RiskLevel

High

Moderate

Low

The Approved Mitigation Plan – Day 0kg (1000)*

Requirement*

Current Design

Level of risk is acceptable if: • A technically feasible/funded path forward exists to provide a compliant design prior to a required date

• Plan is executed and remains on schedule

Required Date

Predicted path forward should be described via a Risk Plan, TPM, or other project plan and statused in the appropriate forum.

Jul 1

1

Jan

12

Jul 1

2

Jan

13

Jul 1

3

Jul 1

4

Jan

14


*Hypothetical requirement


Requirements Design Compliance Methodology -Tools Used

Requirements Design Compliance embedded in Architecture requirement database (Cradle)

Full linkages between requirements and assessments

Assessment reports and requirement traceability reports exported to Excel

Any database/excel can be used as tool

+

An intuitive, uncomplicated tool which links directly to requirements database will yield the best results

Using Requirements Design Compliance to Influence Design - Example

Requirement Design Compliance at Ares PDR proved effective at identifying significant integrated requirement non-compliances


Example 1 – analysis revealed the liftoff clearance between the integrated stack vehicle and the launch facility was insufficient and caused pad re-contact & plume damage Requirement validation and model refinement (e.g.

cantilever vehicle, thrust misalignment update) did not resolve issue

Implemented thrust vector control (TVC) on liftoff to preclude re-contact (e.g. command to counter deterministic TVC bias, command to bias to the South)

Example 2 – GN&C team analysis indicated a roll attitude error build up during First Stage flight that exceeded the 27 deg requirement Defined forward work such as evaluating OML

changes to improve aero roll torques, re-working CFD, and re-examining vehicle performance using Monte Carlo Flight simulations to gain compliance

Combined hardware (aerodynamic strake) and software (load relief algorithms) options to optimize resolution of roll attitude error issue Example from Ares I-X Roll Control System

CFD Analysis


Connecting Requirements Design Compliance to Verification

During PDR phase, verification planning is in early development● The Requirements Design Compliance effort is separate from verification

planning due to disparate maturity levels between the two efforts– Verification and requirements compliance have a lot of overlaps– Requirements compliance helps clarify verification planning aspects

During CDR phase, verification is in final planning stage● Requirements design compliance combined with verification pre-work

– Capture objective evidence in same database – Look for and capture requirements compliance data that could be used as part of

verification closure data For example, integrated verification starts at CDR – integrated analyses to be

used for verification are often complete; you validate that the systems are performing as you expected through qualification

Requirements design compliance can help identify constraints for verification work and/or potential operational constraints due to verification limitations

Assessments supporting Requirements Design Compliance can provide the integrated analysis for verification, leaving only validation of the system to

be done through qual


Making it Work for a System of Systems Architecture

The integration effort is easily underestimated, with specific integration needed to● Agree on roles and responsibilities between architecture and systems

– System data is needed at specific points in time to determine architecture compliance at major milestones

– Many of the pieces from this effort came from collaboration with system implementers– Agree on both the what and the how– Create an overall community of practice focused on overall architecture requirements

design compliance

● Clarify that it isn’t just filling in a database with compliance data – it is a technical assessment

– Time must be allocated for the technical team members to perform the assessment

● Learn from the different system engineering perspectives in order to improve the success of the effort

– Ares was first system to be assessed - provided significant insight into what to do and not do

● Communicate definitions and objectives across all teams


Making it Work for a System of Systems Architecture, cont.

Showing design is compliant to requirements is needed by all systems to move from one phase to the next – communicate value/purpose at all levels

It always takes more time than you think it should – maximize time for stakeholders and requirement owners assessment

The earlier in the project lifecycle you start, the easier it is to implement● Starting during requirements development/achievability phase using available data

enables full PDR phase implementation and facilitates understanding of requirements achievability


Making it Work for a System of Systems Architecture, cont.

Simplicity should be a high priority – people/teams get frustrated by complicated processes and tools● The objective is not to have a process/tool that does the engineer’s thinking, but

to have a simple enough process/tool that it enables the engineers to think

Requirement traceability is a factor in the accuracy of the effort

Acceptable level of risk has different meanings to different people

It is not a panacea of all integration – it is more like a periodic health report to catch anything integration missed● While such a process can be intuitive for a small project, for a large Project you

don’t want to leave it to chance


Questions


If you would like further information…

Alan Dickey: [email protected], 281-244-5680

Stephen Chan: [email protected], 281-483-1083

Shana McElroy: [email protected], 281-483-9580

mailto:[email protected]




Backup

Why Integration Is Needed

As proposed by the

customer.As specified by the

Program.Design as interpreted

by Project A.

Design as interpreted by Project B.

Design as interpreted by Project C.

What was really needed.


- -

CxP PDR Architecture Metrics - Example

Color Definition

Green Compliant

Yellow Watch items

Red Non-compliant

Cx Architecture Requirement Owner Green Yellow Red TOTAL

DIO (Architecture Integration) 2 1 3

Environments & Constraints 7 1 8

Flight Performance 10 2 12

Ground and Mission Operations 17 3 1 21

Human Systems 1 1

Integrated Systems Performance 2 4 6

Integrated Loads, Structures and Mechanisms 4 2 1 7

Integrated Power 1 1

Integrated Thermal/ECLSS 1 2 3

Software and Avionics 16 8 2 26

Safety, Reliability & Quality Assurance 2 2 4

Supportability, Operability, and Affordability 3 3

TOTAL CORE REQUIREMENTS 58 32 5 95


Requirements Design Compliance (RDC) Lifecycle

CustomerConstellation

Program, NASA

Headquarters

PDR Products

Weekly RDC Planning Meeting

SOA SIG

T/ECLSS SIG

ILSM SIG

Power SIG

E&C SIG

HSIG

FP SIG

HTI

Trai

ned

Sta

ff

Exp

erie

nce

Best

Practices

Subject

Matter

Experti

se

RIDsProjects

Orion

Altair

Ares

MS

GS

EVA

Level IIIPDR Board Actions

Analyses Risks

CxP PDR RDC

Report

Projects

Orion

Altair

Ares

MS

GS

EVA

GOALTo objectively determine if the preliminary design

can be expected to meet requirements to an acceptable level of risk.

Official Path of Issue Resolution

GMO SIG

SR&QA

SAVIO

SIG/Project Community of Practice Efforts

ASET

SIG/Project Community of Practice Efforts

RIDsAnalysis Cycles

Level II Actions

DIO

TraceVerific

ation


NASA Center and Project Locations


Dickey.alan

Technology

Transcript of Dickey.alan