8:15 AM Tuesday September 15, 2009

Karl Frank, Point of Contact for Constellation Projects

Establishing IV&V Expectations

Diagrams for illustrative purposes only.

Section Overview1. Goals in workshop context2. Expectations: whose? With respect to what?3. Goal-based capture4. Role of Validation in model-based process 5. Model as cage for captured understanding6. Model as share point7. Role of Verification8. Lessons learned

GOALS IN WORKSHOP CONTEXT

In context of the workshop, this presentation aims at providing an overview of the many ways in which modeling has been done in the Fairmont IV&V facility, both how and why, so that you will be able to fit particular topics at a more detailed level, into a big picture.If successful, you will later be able to see how the use some particular diagram or tool, is part of an overall approach, not a detached effort.

Section 1

Basis for Understanding• Modeling serves many purposes

– Not an end in itself, serves V & V• Can be done in many different ways

– Different “ways” as different languages– Within a given language different “ways”:

• Levels of detail, “analysis” vs. “implementation”

• Here, language is UML 2– Levels of detail evolve to finer granularity

• But never for purpose of implementation• Validation itself moves through “Levels”

EXPECTATIONS: WHOSE? WRT ?

Collectively staff of an IV&V facility needs shared understanding of the goals and approach, and staff on a given project needs a shared understanding of that project, starting with project goals, and at the level of artifacts presented for validation and verification.

Section 2

Expectations

• expectations in black box testing, is particularly focused on how the software under test should behave, not how it should have been implemented. – Behavior model takes precedence

• Architectures & Interfaces require V&V– Upon presentation of an IRD what

expectations do we bring to its analysis?

GOAL-BASED CAPTURE

Valid expectations wrt project artifacts begin with project goals. These are form the foundation of our project models. As there is a commonality to all NASA projects, the common goals of space transportation systems form a common root for all project models.

Section 3

Goal-Based Capture• Popularly called SysGoals Model• Adapted from use case modeling

– Text document with use-case like structure– Graphic overview as UML use case diagram– System goals are elaborated as use cases– Use Cases bridge into the UML model

• Head of the model navigation traces• Use case flow of events elaborated in Activity Diagrams,

one per use case– UML model can be traced back to Goals

Common Model Pattern

Goals at head of Navigation

Number of Levels will vary with the Project

Links down to Activity Diagrams.

Swimlanes typically disappear at “functional” levels of behavior

Diagram Example

Horizontal LinksBetween alternativeViews of same Behavior

Diagram Example

Diagram Example One Project Use Cases Linked to Goals

Diagram Example, Different Project, Shared Goals

ROLE OF VALIDATION

Modeling process is tightly coupled with validation in steps. Goal statements from project sponsors or used to validate Concept of Operations documents from NASA and these are the sources that are the basis of the first iteration of the SRM, then used in validating next round of project requirements docs and specs.

Section 4

Validation• Project process presents successive levels of

documents for validation• Approach focused on validating behaviors,

modeled from validated sources, instead of text document focus, reporting on behaviors

• Progress from System ConOps to sub-system, component, element level requirements, Interface Requirement Docs

Elaborated Behavior Example

Ports

Discrete RIU remote interface unit

Notation Example for Interface Validation

MODEL AS CAGE FOR CAPTURED

For each project, a UML Model links goal and use case documents to behaviors and, thru behaviors, to source documents (Concept of Operations), architectures, issues raised in validation, and defects reported in verification. Do not think of Model as collection of diagrams: the traces to correlated requirements cannot be graphic.Progress on different IV&V tasks are traced thru model.

Section 5

Cage for Captured Understanding

• Diagrams as a training resource• Model as a shared database

– Can report links to target requirements related to behaviors or other model elements

• Same model elements can be viewed at different levels of detail for high-level or detailed views.

MODEL AS SHARE POINT

Different project tasks are best served by different kinds of diagram, based on different modeling concepts, and at different levels of detail. These are developed as views of a single ever-expanding underlying model, which can be audited programmatically for consistency, so that instead of producing a variety of distinct documents with diagrams, which need to be versioned as distinct resources, each project has a single shared and evolving UML model, organized into packages and subpackages, containing the gold copies of all the diagrams.

Section 6 Heading

UML Model database

• Linked to Goal and Use Case Documents• Linked to Requirements Documents• Linked to Issues• Linked to exported Assertions• Linked to tests and test result.

– The behaviors and logical architectures are the IV&V focus, and the model defines those, and links them to the other workproducts

Model as Share Point

Correlation between expectations on behaviors -> requirements

ROLE OF VERIFICATION

Software verification requires models sufficiently detailed to make it possible to compare the model with the behavior or architecture of software. Our goals-based approach to model elaboration means that the needs of verification are met by the models only when they reach an advanced state of elaboration.In particular, statemachine diagrams are particularly useful in verification of software behavior, but for reasons to be explained, are among the last products of our modeling work.

Section 7 Heading

Verification

• Statemachines in UML 2 made sufficiently detailed to be used as definitions of required behavior and prohibited behavior.

• Oracle concept various external tools can interpret these statemachines as executable, and compare the behavior of software under test, to behaviors mandated or prohibited by the statemachine: assertion testing.

Statemachines

Diagram Example: Statemachine exportable Assertion for verification

LESSONS LEARNED

Ongoing challenges:Many different UML modeling concepts and notations compete as ways to represent system behaviors and architectures. Up front agreement on what modeling concepts and graphic diagram standards is difficult to get, but important. Have modeling standards and model-based IV&V processes place.

Section 8

Lessons

• Distinction between Model, as a single organized XML database of model elements, and Diagrams which present graphic views of model contents, is difficult to master.

• Many different UML modeling concepts and notations compete as ways to represent system behaviors and architectures. Up front agreement on what diagrams to use for what is difficult to get, but important.

• Defining the process for applying concepts of model driven development to IV&V, while continuing to perform traditional IV&V functions.

• Training modeling staff in validation and verification, and training validation staff in modeling, is a challenge.

Related Article

• Latest issue IEEE Computer– Cover Feature seems to be based on approach

NASA IV&V has been pioneering for years– Faultless Systems: Yes We Can!

• Jean-Raymond Abriel, Swiss Federal Institute of Technology, Zurich

Beware of arbitrary verbal distinction “horizontal” vs.“vertical” refinement