Verification and Validation (V&V)

Federal AviationAdministration

Presented To: Verification and Validation

6th Annual Summit 2010

By: Maureen Molz

Date: October 13, 2010

Verification and Validation (V&V)

System of Systems (SoS)

Assessment Platform

2Federal AviationAdministration

Verification & Validation System of Systems Assessment PlatformOctober 13, 2010

Net-Centric

V&V Strategies and Processes

System of Systems Assessment Platform

Agenda



What is Net-Centric?A large set of desperate pieces of data provided by differing systems and/or sensors littered across a network awaiting multiple users, service providers or applications to

use them to create information to be used by humans to solve problems

VALUE CHAIN

DATA

INFORMATION

KNOWLEDGE

DECIDE

ACT

Individual facts, measurements, or observations which may or may not be sufficient to make a particular decision

Obtained when elements of data are assembled, reconciled fused and placed in an operational context

Derived from being able to use information to construct and use an exploratory model based on understanding of the situation or phenomenon



Implications of Net-Centric• Better situation awareness (SA) and situation understand (SU)

• Decision support tools (synthesized and analyzed information):– Focus on relevant information to reduce uncertainty– Reduce process time and clearly visualize current and future air

traffic states– Improve/reduce decision time– Offload spatial and temporal integration to hardware/software

environment

• Research to understand how to carve the problem space between pilots and air traffic controllers

• Scaleable, adaptable, flexible, agile = effective and efficient



Conceptual View of Current NAS

Tools/Systems(Automation)

Data Generated

from Systems

TCAS

ASDE-X

CARTS/STARS

HOST/DSR

Apriorito

Flight

Real-TimeFlight

Real-TimeRunway

Mgt.

Real-time Conflict Mgt

& TrafficDepart/Arrive

Real-Time & Near Future

ConflictMgt.

Airlines/General Aviation

Pilots

Towers

En Route Ctrs

Correlate/Control/Decide

TRACONs

TRAFFIC FLOW MGT

Future StateMgt.

Interoperable systems that exchange data in a point-to-point manner with human intensive manual processes

TFM-M

Flight ServicesApriori

toFlight

FS21

Data/Info

Data/Info

Data/Info

Data/Info

Data/Info

Data/Info

Communication

Navigation

Surveillance

Weather



Conceptual View of a Net-Centric NAS: NextGen Net-centric systems that gather, fuse and analyze multiple pieces of data from

multiple sources to meet unique user needs

Data: Individual facts, measurementsor observations which may or may notbe sufficient to make a particular decision

Information: Obtained when elementsof data are assembled, reconciled, fusedand placed in operational context

Knowledge: Derived from being ableto use information to construct and use anexploratory model based on understandingof the situation or phenomenon

Correlate

NetCentric has 2 components: The transport layers and The information layers.

This is about the information layers



Test for New Capabilities

• NextGen represents a fundamental paradigm shift from:– stove-piped, analog, point to point systems to ...– an integrated, digital, net centric architecture

requiring a system of systems (SoS) development strategy that does not exist in its entirety today

• NextGen has unique requirements which increase the need to:– Verify NextGen and its systems work correctly– Validate NextGen and its systems work in their

intended environment



V&V Strategies and Processes



Maureen MolzManager, V&V

Strategies and Practices TeamAJP-7C2

V&V Strategies Test Standards Board

John Frederick, ChairHarry BilickiWanda Lopez-LaBarberaEric LowySteve OshelAmnon Pollak

Contractor Support: Viola Gray

V&V Strategies PM – VacantV&V Strategies Engineer – Vacant

Contractor Support: Howard Shectman

Team Organizational Structure

Team Summary 7 FAA Employees

3 Contractor Employees

Team Summary 7 FAA Employees

3 Contractor Employees

Secretary: Kim Fitzpatrick (CTR)

Contractor Support to AJP-7C1Vy BuiNoreen MillerKristin Snyder

Contractor Support to AJP-7C1Vy BuiNoreen MillerKristin Snyder



Acquisition V&V Process

Current:• Stovepiped• Limited interdependency issues

NetCentric:• SoS level performance• Performance relies on data integration



V&V Strategies and Practices Team Plan

• Continue TSB Support

• Continue AEB V&V Working Group Support

• Begin Development of Prototype SoS Assessment Platform Awarded Contract: 24 Sep 2010 Kick-off meeting: 6 Oct 2010 1st Working Group Meeting: Oct 2010



Advantages

• Provide acquisition decision makers more reliable information

• Ability to track operational improvements through out the lifecycle process

• Ability to mitigate risks associated with the increased system of system complexity

• Common set of tools (data, models, scenarios) to assess system from research through system engineering into test



System Of Systems Assessment Platform



V&V effort

LABNET

STARS TFMERAM

DataComm

SOS Toolset

Integrated Laboratories

NextGenV&V Environment(variable fidelity)

High

NextGen V&V: Current & Future Capabilities

ERAM

TFM

DataComm

Stove Pipe Laboratories

UAS

WJHTC Laboratories

STARS TFM

Terminal

ERAM

SOSNEXTGEN

V&V Capability

PhysicalLaboratory

Transformation

NIEC

WX

NVS

Wx DataComm

UAS

Medium-High

NASData

Scenario

Models

STARS

SystemUnderTest

LowLow-Medium

Medium

Full Scale ScenariosSymptomatic Modeling

Thread TestingConstructive Simulation

One Integrated Toolset

Project Toolsets

ModelsScenariosNAS Data



System-of-Systems V&V Capability:

NextGen Synthetic Environment



SoS Assessment Platform Activities FY11 FY12 FY13 FY14 FY15

I. V&V Assessment Phase

II. V&V Definition Phase

III. V&V Design Phase

IV. V&V Development Phase

SimNet

NetCentric Net

NAS Systems

NAS SystemAgents

CollectiveIntelligent Agents

Cellx Net

Detail of Cell

Agent Sim

Agent Sim

Agent Sim

Virtual Sims

Agent Sim

Agent Sim

Persistent Data

Region A

Region B

Zone GRegion D

GeoLocX

Region C

Region E

GeoLoc Y

Region F

Zone F

NetCentric Net

SimulationNetwork

(DIS/HLA/TENA)

Multiple Cells running at different fidelity based on

scenario requirements

C2 Control

GPS etc.

Constructive Sim

Live (NAS systems controlled by humans)

Constructive Simulationwith Intelligent Agents

Collective Agents Decision Support Agents

EnRoute

Target Generation OneSAF Tower

SimsHuman Sims (Pilot, ATC,

etc)Virtual SimulationWith Intelligent Agents

UAS Sims

Weather

Tier 1DSS at the Commander

Tier 2DSS acrossthe Network

Tier 3DSS atthe Platform

Intelligent Agents

Cockpit Sims

NetCentric V&V ExperienceNetCentric V&V Design



Backups

Verification and Validation (V&V)

Documents

Transcript of Verification and Validation (V&V)