Model=Based$Design$Approach$for$the$COTS$...

Ian T Mitchell

Draper Laboratory

[email protected] 8th December, 2010

Model-‐Based Design Approach for the COTS Guidance, NavigaJon and TargeJng (GN&T)

Algorithm and SoNware Development

 Overview

 Development Process  Design and ImplementaJon

 Unit Test Framework  Monte-‐Carlo SimulaJon Framework

 Metrics

  Lessons Learned

Agenda

2

Overview: Cygnus Mission and Team Members

3

Mission Partners

Orbital Sciences Corpora=on Prime contractor; engineering and development; Cygnus service module, mission and cargo operaJons

Thales Alenia Space Pressurized cargo module

Draper Laboratory Guidance, navigaJon and fault tolerant computer support

Mitsubishi Electric Corpora=on (MELCO) Proximity locaJon system

Odyssey Space Research VisiJng vehicle requirements support

SAS SoNware Independent VerificaJon and ValidaJon (IV&V), engineering support

Cimarron Mission Control Center -‐ Houston (MCC-‐H) gateway developer

Quick Facts

http://www.orbital.com/NewsInfo/Publications/Cygnus_fact.pdf

 David Benson (SimulaJon)

  Chris Besse[e (FDI Algorithms/SoNware)   Louis Breger (Proximity Guidance Algorithms/SoNware)

  Sungyung Lim (Proximity Guidance Algorithms)   Ian Mitchell (Task Lead)

  Piero Mio[o (Program Manager)

 Dick Phillips (FDI Algorithms)   Russell Sargent (TargeJng Algorithms/SoNware)

  Kurt Winikka (SoNware)  Dave Woffinden (Algorithms)

  Renato Zaneb (NavigaJon Filter Algorithms/SoNware)

Overview: Draper GN&T Team

4

Overview: Model Based Design Workflow

5

Detailed Algorithm

Design

Design Validation

•  Perform detailed design and verify performance •  Ensure guidelines adhered to with the detailed design •  Establish requirement traceability to the design •  Build and execute test cases based on requirements

Preliminary Algorithm

Design

Requirement Validation

Algorithm Requirements

•  Capture algorithm performance and functional requirements •  Establish and follow style guidelines •  Perform preliminary design •  Validate requirements via simulation

Code Generation

Integration and Test

•  Perform software integration •  Establish requirement traceability to the generated code •  Run requirements based test cases

  Process   Model-‐based design, graphical

programming, Integrated Quality Management System (IQMS)

  Methods   Modeling, simulaJon, prototyping,

automaJc code generaJon, model tesJng and coverage, soNware-‐in-‐the-‐loop tesJng, processor-‐in-‐the-‐loop tesJng

  Tools   DOORS for requirements management

  Simulink® , Stateflow ® and Embedded MATLAB™

  Simulink VerificaJon and ValidaJon ™

  Real-‐Time Workshop Embedded Coder ™

  C based tools (gcc, gcov and Perl) for source code verificaJon

Overview: Development Process

6

GN&T Algorithm Specification and Design Description

GN&T Software Requirements

GN&T Software Design

GN&T Software Implementation and

Unit Test

GN&T Software Unit Integration and Test

GN&T Software Verification

GN&T Algorithm Validation

GN&T Algorithm Validation Plan

GN&T Software Test Plan

GN&T Software Test Plan

  V Model emphasizes requirements-‐driven design and tesJng   All GN&T high level design elements (CSCs) are traceable to one or more high level soNware requirement

captured in the SRS/IRS

  All GN&T low level design elements (CSUs) are traceable to one or more low level soNware requirement captured in the SDD

  Every low level and high level requirement is addressed by one or more test cases captured in the STD   Bi-‐direcJonal traceability ensures nothing is done unnecessarily and everything that is necessary is

accomplished

Development Process: Subsystem Design

7

•  COTS to ISS IRD •  COTS System Specification •  Cygnus Specification •  Service Module Specification

COTS System Requirements

Develop GN&C Subsystem

Requirements

COTS System Requirements

Documents

GN&C Subsystem Technical Data

Package

•  GN&C Subsystem Specification •  GN&C Concept of Operations (CONOPS) •  GN&C Preliminary Design Notes (DNs) •  GN&C Preliminary Simulation Study Results (LinCov, 3-dof, 6-dof)

Develop GN&C Subsystem Design


Package


Package

•  GN&C Design Notes (DNs) •  GN&C Interface Control Documents (ICDs) •  GN&C Algorithm Design Descriptions (ADDs) •  Allocated GN&T Algorithm Functional and Performance Requirements •  Updated GN&C Simulation Study Results (LinCov, 3-dof, 6-dof)

R R GN&T Software

Development

•  GN&C Subsystem Specification •  GN&C Concept of Operations •  GN&C Preliminary Design Notes (DNs)

Legend

Process map Process element Work Product Review Gate Configuration Management

Orbital Technical Review

Orbital Technical Review

  System and GN&C subsystem performance requirements were derived from NASA requirements   Prototype algorithms were developed in Simulink and analyzed and simulated in order to validate the

subsystem performance requirements

Development Process: SoNware Design

8

GN&C Subsystem

Requirements and Design

Develop and Analyze GN&T Software Requirements

•  GN&T Algorithm Specification and Design Description (GN&T ASDD) •  GN&T Software Development Plan (SDP) •  GN&T Software Test Plan (STP)

GN&T Software Development Files •  GN&T Software Requirements Specification (SRS) •  GN&T Interface Requirements Specification (IRS) •  Draft GN&T Software Test Description (STD)

Develop GN&T Software Architecture

and Design

R

GN&T Software Development Files •  Preliminary GN&T Software Design Description (SDD) •  GN&T Simulink CSC models, allocated SRS high level requirements, defined interfaces •  GN&T Software Data Dictionary (DD)

GN&T Software Development Files •  GN&T Software Requirements Specification (SRS) •  GN&T Interface Requirements Specification (IRS) •  GN&T Software Development Plan (SDP) •  GN&T Prototype Algorithms (Simulink block diagrams, eML code) •  GN&T Model-Based Design Software Development Standard

ADevelop GN&T Software Item

Detailed Design

GN&T Software Development Files •  Updated GN&T Software Design Description (SDD) •  GN&T Simulink CSU models, derived low level requirements, low level defined interfaces •  Updated GN&T Software Data Dictionary (DD)

Draper Peer Review

  Algorithm components were defined according to overall GN&C states and modes   FuncJonal or behavioral requirements for each of the algorithm components were derived

  Interfaces between Draper soNware components and Orbital soNware components were defined

Development Process: Implementa=on and Test

9

Develop GN&T Models and

AutoCode and Perform Unit Test

R

GN&T Software Development Files •  Unit tested GN&T Simulink models (CSUs) and auto-generated flight code •  GN&T Simulink models and auto-generated test code •  Updated GN&T Software Test Description (STD) including low level requirements/test traceability •  GN&T Software Test Report (STR) •  GN&T software metrics

GN&T Software Development Files •  GN&T Prototype Algorithms (Simulink block diagrams, eML code) •  GN&T Model-Based Design Software Development Standard •  GN&T unit test framework •  Low level CSU requirements

Perform GN&T Unit Integration and Test A

GN&T Software Development Files •  Unit tested GN&T Simulink models (CSUs) and auto-generated flight code •  GN&T Model-Based Design Software Development Standard •  GN&T unit test framework •  Low level CSU requirements

B

GN&T Software Development Files •  Integrated and tested GN&T Simulink models (CSUs and CSCs) and auto-generated flight code •  GN&T Simulink models and auto-generated test code •  Updated GN&T Software Test Description (STD) including high/low level requirements/test traceability •  Updated GN&T Software Test Report (STR) •  Updated GN&T software metrics

Draper Peer Review

  Unit Test Framework used for model implementaJon and unit test

  Models, unit tesJng and unit integraJon work products peer reviewed for each soNware component

Development Process: Qualifica=on

10

B Perform GN&T CSC Qualification

GN&T Software Development Files •  Integrated and tested GN&T Simulink models (CSCs) and auto-generated flight code •  Updated GN&T Software Test Description (STD) including SRS requirements/test traceability •  Updated GN&T Software Test Report (STR) •  GN&T Algorithm Validation Results

GN&T Software Development Files •  Integrated and tested GN&T Simulink models (CSUs) and auto-generated flight code (flight_gnt) •  GN&T simulation framework (sim_gnt) •  GN&T Algorithm Validation Plan

GN&C Subsystem Integration

R

Orbital Acceptance

Review

  Draper developed unit test framework and simulaJon framework used for verificaJon and validaJon

  Final GN&C subsystem integraJon performed by Orbital using an Orbital simulaJon environment

Development Process: Build Planning

11

GN&T Software Development Activities Build 1 Build 2 Build 3

GN&T Software Requirements X X X

GN&T Software Design X X X

GN&T Software Implementation and Unit Test X X X

GN&T Software Unit Integration and Testing X X X

GN&T Software Qualification X

GN&T Software Delivery X X X

GN&C/GN&T Software Integration X X X

GN&C/GN&T Software/Hardware Integration and Testing X

GN&C/GN&T Subsystem Qualification Testing X

GN&T Software Configuration Management X X X

GN&T Software Product Evaluation X X X

GN&T Software Quality Assurance X X X

  Incremental soNware life cycle model   Build 1: Interface build to miJgate risk associated with soNware integraJon

  Build 2: Core guidance and targeJng funcJonality to be used with perfect navigaJon

  Build 3: Remaining navigaJon and FDI funcJonality

Development Process: Peer Review

12

Prepare for a Peer Review

GN&T Peer Review Package

Analyze and Disposition Peer

Review Data

GN&T Peer Review Package

GN&T Peer Review Data

•  Recorded defects •  Defect severity •  Review summary: Project, work product, review type, participants

Plan for a Peer Review

Execute the Peer Review

Perform Peer Review Follow Up

GN&T Peer Review Plan

•  GN&T work product •  GN&T review checklist •  Backup/reference material and supporting documentation

•  Participants: Draper, Orbital, Odyssey (IV&V) •  Moderators: Draper inspection moderators •  Meeting schedule

•  GN&T work product •  GN&T review checklist •  Backup/reference material and supporting documentation


•  Action items •  Dispositioned defects: Fix, Do Not Fix, Action Item, PTS •  Work product status: Release or further review


•  Defect resolution •  Plan defect correction



•  Recorded defects •  Defect severity •  Review summary

•  Action items •  Dispositioned defects •  Work product status: Release or further review

  Peer review work products   All GN&T flight soNware documentaJon: SDP, STP, SRS, IRS, SDD, STD, STR, VDD

  All GN&T Simulink modeling arJfacts: Models, CSU/CSC test cases, test results, V&V results (coverage and complexity)

Development Process: Peer Review Checklist

13

  Model Peer Review Checklist   No peer review checklist existed for

model-‐based design   GN&T team defined an iniJal peer

review checklist to be used for all GN&T model peer reviews

  Automated Style Checking   Peer review defects can be substanJally

reduced with automated style checking   Simulink’s style checker was not

extensively used during the COTS program

  Some custom scripts were developed to check specific items

  Not sufficient Jme or resources to modify and/or develop own style checker

Development Process: Test Approach

14

Perform Code Verification

Perform Model Verification

Perform Algorithm Validation

  Three Test Phases for GN&T VerificaJon and ValidaJon   GN&T Model VerificaJon: Simulink environment for both analysis and test (Simulink® , Real-‐Time

Workshop Embedded Coder ™, Simulink VerificaJon and ValidaJon ™) using unit test framework   GN&T Code VerificaJon: C code environment for both analysis and test (gnu gcc, gnu gcov, Perl)

using unit test framework   GN&T Algorithm ValidaJon: SimulaJon environment (Simulink® , Real-‐Time Workshop Embedded

Coder ™) using simulaJon framework

Auto-generate model code and test harness

code

Auto-generate model code and integrate as 8

S-functions for each individual CSC

Verified GN&T Code Validated GN&T Models and Code Verified GN&T Models

Verified GN&T Models Verified GN&T Models and Code GN&T Models

Unit Test Framework Simulation Framework


15

Test Phase

Test Levels Test Classes

Sim

ulin

k U

nit V

erifi

catio

n

Cod

e U

nit V

erifi

catio

n

CS

C In

tegr

atio

n Te

st

Reg

ress

ion

Test

CS

C Q

ualif

icat

ion

Test

Func

tiona

l

Mon

te-C

arlo

Per

form

ance

Stre

ss

Bou

ndar

y

Err

or C

ondi

tions

Mod

el C

over

age

& A

naly

sis

Cod

e C

over

age

& A

naly

sis

GN&T Model Verification 1-3 2,3 2,3 3 1-3 3 2,3 3 2,3

GN&T Code Verification 1-3 2,3 2,3 3 1-3 2,3 3 2,3 3 2,3

GN&T Algorithm Validation 2,3 2,3 3 2,3 2,3

  Test Levels   Unit tesJng, integraJon tesJng, regression tesJng and qualificaJon or acceptance tesJng

  Test Classes   FuncJonal, Monte-‐Carlo, performance, stress, boundary, error condiJons, coverage and

analysis


16

  Simulink Unit VerificaJon   Uses Draper developed unit test framework and employs Simulink V&V toolbox for model

coverage analysis and cyclomaJc complexity metrics.   Test classes include funcJonal, error condiJons, boundary and structural tesJng

  Code Unit VerificaJon   Uses Draper developed unit test framework and gnu tools for compiling auto-‐generated flight

code and test code and generaJng code coverage, cyclomaJc complexity, code Jming and code stack usage metrics.

  Test cases are all repeats of the Simulink unit verificaJon test cases

  CSC IntegraJon Test   Repeats Simulink Unit VerificaJon and Code Unit VerificaJon for integrated units. Top level

CSC tests integrate Simulink S-‐funcJon within unit test framework.   Uses simulaJon framework to validate algorithms and ensure performance requirements are

met. CSCs are integrated as 8 S-‐funcJons for each individual CSC.

  Regression Test   Repeats test levels for previously tested CSUs and CSCs

  CSC QualificaJon Test   Formal execuJon of unit verificaJon and CSC integraJon tests


17

  FuncJonal Tests   FuncJonal test which verifies generated output in response to selected inputs and expected

output   Monte-‐Carlo Tests

  FuncJonal test which evaluates algorithm performance against validaJon criteria

  Performance Tests   FuncJonal test which evaluates Jming performance and stack usage of units, integrated units and

CSCs

  Stress Tests   FuncJonal tests which verifies soNware funcJonality under stress condiJons such as extreme

input values. Ensure that these condiJons do not cause abnormal behavior (divide by zero excepJons for example) or that such faults are detected, isolated and reported.

  Boundary Tests   Structural test which verifies boundary values are tested. For example, loop boundaries at 0, 1,

many, max-‐1, max, max+1 iteraJons

  Erroneous Input Tests   FuncJonal test verifies soNware responds correctly to erroneous input values that may cause

algorithm excepJons


18

  Simulink Model Coverage and Analysis   Structural test which evaluates MC/DC model coverage by a parJcular test data set

  Analysis performed to determine requirement traceability, test coverage, model complexity analysis, and model standards compliance

  Code Coverage and Analysis   Structural test which evaluates percentage of code exercised at least once by a parJcular test

data set (the same test data set used in Simulink coverage tesJng)   Analysis performed to determine test coverage, code complexity, Jming and memory usage

Design and Implementa=on: Requirements

19

COTS GN&C Subsystem Specification

COTS GN&T Algorithm

Specification and Design Description

  COTS GN&C Subsystem SpecificaJon   NASA requirements analyzed to

determine system and GN&C subsystem requirements

  AllocaJons to GN&T

  COTS GN&T Algorithm SpecificaJon and Design DescripJon   Captured algorithm performance

requirements and algorithm descripJons

  COTS GN&T SoNware Requirements SpecificaJon   IdenJfied GN&T soNware components

(CSCs) and high level funcJonal requirements and interfaces

  COTS GN&T SoNware Design DescripJon   IdenJfied GN&T soNware units (CSUs)

and low level funcJonal requirements including error handling

COTS GN&T Software

Requirements Specification

COTS GN&T Software Design

Description

Prior to Approach Initiation, the GN&C shall maintain nominal approach and planned contingency trajectories, including expected dispersions, to keep the COTS vehicle outside the Approach Ellipsoid.

Examples Specification Tree

The GN&T shall autonomously compute proximity operations targeted maneuvers to acquire and maintain the 1.4 km co-elliptic such that the 3-sigma trajectory dispersions represented in the ISS LVLH frame after acquisition of the 1.4 km co-elliptic do not exceed +/- 150 m in the out-of-plane and radial directions.

The gnt_targeting CSC shall provide targeting modes of operation for Idle, Initial Altitude Adjust, First Mid-Course Correction, Second Mid-Course Correction, Final Altitude Adjust, Final Mid-Course Correction and R-Bar Acquisition.

The gnt_chuggerBy CSU shall propagate the Cygnus J2000 ECI orbital state iteratively over N cycles defined by the delta-V burn duration and a parameterized maximum step size.

The gnt_chuggerBy CSU shall detect an algorithm exception if the computed N cycles is greater than or equal to a parameterized threshold

Design and Implementa=on: Requirements

20

  DOORS used for managing requirements and maintaining bidirecJonal traceability   Snapshot above is the COTS GN&T SoNware Requirements SpecificaJon (SRS) on the leN and the COTS GN&T SoNware

Design DescripJon (SDD) on the right. Trace links to the COTS GN&T Algorithm SpecificaJon and Design DescripJon (ASDD), the SDD and SRS are shown.

  The COTS GN&T SoNware Test DescripJon (STD) not shown is also maintained within DOORS and provides trace links from test cases to requirements.

  CSC modules and CSU names are also shown for each of the CSU requirements.   Chose not to include requirements in the Simulink models themselves (via Simulink V&V toolbox )

Design and Implementa=on: Architecture

21

Navigation

Sensor Processing

GN&C Executive

ALG CSC

Attitude Determination

ALG

Orbit and Relative Navigation

ALG CSC

Guidance

Attitude Guidance

ALG

Orbit Guidance

ALG

Targeting

CSC

CSC

Control

Attitude Control

ALG

Position & Velocity Control

ALG

Jet Selection

ALG

  Orbital using Rhapsody® modeling tool for algorithm block development and state chart tool for GN&C states and modes sequencer development

  Orbital using an object oriented approach for the GN&C execuJve and algorithm blocks. There are 40 algorithm blocks including the Orbital wrappers for the Draper CSCs.

  Orbital GN&C execuJve calls algorithm blocks and CSCs at the appropriate rate (5 Hz, 1 Hz, 0.1 Hz) and in the appropriate GN&C state and mode

Legend

Orbital

Draper

Design and Implementa=on: Architecture

22

CSC

CSU

Rate Dependent or State/Mode

Dependent

CSU CSU CSU

CSU CSU CSU CSU CSU

CSU CSU

First Level of Integrated CSUs

Next Level of CSUs

Lowest Level of CSUs

  Total of 8 Draper CSCs: gnt_imuFdir, gnt_rgpsFdir, gnt_lidarFdir, gnt_rgpsFilt, gnt_lidarFilt, gnt_targeJng, gnt_proxGuid and gnt_targetProp. Each CSC has a common interface (INP, PRM, CON, OUT and TLM buses) and common methods for iniJalizaJon, calling and terminaJon.

  All CSCs and CSUs are defined as Model Reference blocks. Model Reference blocks allow models to be reused and shared, facilitate parallel development and configuraJon management.

  CSUs were defined according to a funcJonal decomposiJon and the general guideline to have a CSU cyclomaJc complexity < 20   Depth of decomposiJon is different for each CSC: gnt_imuFdir (2 deep), gnt_lidarFdir (2), gnt_rgpsFdir (3), gnt_rgpsFilt (5),

gnt_lidarFilt (4), gnt_targeJng (5), gnt_proxGuid (4) and gnt_targetProp (2)

Design and Implementa=on: CSC Defini=ons

23

  Model Components (8 CSCs)   InerJal Measurement Unit (IMU) Failure DetecJon and

IsolaJon (FDI) CSC: gnt_imuFdir   Global PosiJoning System (GPS) FDI CSC: gnt_rgpsFdir

  Lidar FDI CSC: gnt_lidarFdir

  GPS Based RelaJve NavigaJon CSC: gnt_rgpsFilt

  Lidar Based RelaJve NavigaJon CSC: gnt_lidarFilt

  TargeJng CSC: gnt_targeJng

  Proximity Guidance CSC: gnt_proxGuid

  Back-‐up Target PropagaJon CSC: gnt_targetProp

  Model Units (96 CSUs)   Each CSC is comprised of unique and/or common CSUs

  Allowable primiJve blocks defined in SCLIB

  Model UJliJes (51 UJliJes)   Low level eML uJliJes for quaternion math, GPS Jme

calculaJons, guarded funcJons, table lookup etc

gnt_cscName

INP

PRM

CON TLM

OUT

CSC Model Reference

CSC Total CSUs

gnt_imuFdir 7(1)

gnt_rgpsFdir 211)

gnt_lidarFdir 8 (1)

gnt_rgpsFilt 19 (2)

gnt_lidarFilt 15 (2)

gnt_targeting 23 (3)

gnt_proxGuid 20 (3)

gnt_targetProp 2 (4)

Notes: (1)  Includes 6 CSUs common to each Fdir CSU (2)  Includes 4 CSUs common to each Filt CSU (3)  Includes 1 CSU common to targeting and guidance

CSUs (4)  Includes 2 CSUs common to each Filt CSU

Design and Implementa=on: CSC Defini=ons

24

  CSC Common Interface DefiniJon   Each of the 8 CSCs use a common interface which

is defined by a data structure that contains all the data associated with the CSC

  Input data structure: contains all the dynamic inputs to the CSC

  Parameter data structure: contains algorithm parameters that may change during the mission

  Constant data structure: contains physical constants that change infrequently if at all

  Output data structure: contains all the dynamic outputs from the CSC that are desJned for other CSCs

  Telemetry data structure: contains all the telemetry data that is transmi[ed to the ground

  CSC Common Method DefiniJon   Each of the 8 CSCs is invoked using 3 methods   The iniJalizaJon method is called before

starJng to use the call method

  The call method is used to call the CSC and generate the output and is executed at the parJcular rate of the CSC

  The terminate method may be called as part of a system shutdown. Currently, terminate funcJons are not uJlized.

Design and Implementa=on: CSC Interfaces

25

  A Data DicJonary, captured in MicrosoN Office Excel, was created to manage CSC I/O. A representaJve bus definiJon for the gnt_lidarFilt CSC input bus is shown above and the bus itself, INP_gnt_lidarFilt, is shown at right.

  MATLAB™ scripts were developed to automaJcally create the Simulink bus I/O definiJons for each CSC. Each CSC has a dedicated worksheet within the data dicJonary.

  Orbital naming convenJons (including drinking camel style) were adopted at the CSC interface

  A common worksheet contains all the common bus types that are used for input, parameters, constants, output and telemetry, BUS_imu and BUS_quat for example

Design and Implementa=on: Error Handling

26

  Algorithm ExcepJons   Each CSC detects and reports potenJal algorithm excepJons via the CSC interface

  Algorithm developers define the types of algorithm excepJons that are unique to his/her algorithm

  Math ExcepJons   Each CSC detects and reports potenJal math excepJons via the CSC interface

  Guarded funcJons for division, square root, exponenJal, arc-‐sine, and arc-‐tangent are part of the model uJliJes and are used extensively within eML blocks.

  Common ExcepJon ReporJng   Algorithm excepJons or asserts are issued directly through a EXASSERT_EML funcJon that is

embedded within an eML block.   Math excepJons are issued indirectly by each of the uJlity funcJons. Each uJlity funcJon includes

a call to the EXASSERT_EML funcJon if a potenJal math excepJon is detected.   Each excepJon, whether it is an algorithm excepJon or a math excepJon, has a unique ID

associated with it, referred to as the asserJon number.   The asserJon number is reported as an output of each CSC

Design and Implementa=on: Error Handling

27

  Example: for loops within gnt_targeJng   Each for loop instance determines whether the

maximum number of iteraJons allowable is reached

  An algorithm excepJon is reported if i reaches iMax

  EXASSERT_EML is called when this condiJon is detected

  EXASSERT_EML funcJon performs different acJons depending on calling environment

  When auto-‐coding, EXASSERT_EML is autocoded as a macro defined as the C funcJon gnt_exAssert

  The C funcJon gnt_getExAssert called within an eML blocks is used to access the global assert variable and report as an output

Design and Implementa=on: Standards

28

  GN&T Model-‐Based Design SoNware Development Standard   Defines development tools to be used

  Defines standards for model architecture, use of Simulink and use of Embedded MATLAB™ (eML)

  Similar to MAAB standards but with some differences: Stateflow is restricted to test harness and simulaJon components as needed


29

  All Simulink block diagrams include a banner block that contains the standard distribuJon statement required for deliverable items

  Banner block also includes: Program Name, Customer, Author, Date of Last ModificaJon, Version Number and Model Name.

  Model revision number for each .mdl is managed by version control tool and callbacks within each model. Auto-‐code for each model also has its own revision number


30

  Auto-‐generated code revision number for each .mdl is also managed by version control tool.

  The declaraJons at top of each .c and .h file show revision number for the auto-‐generated code as well as the revision number of the .mdl from whence it came

Unit Test Framework

31

  CSC/CSU Unit TesJng   CSUs, integrated CSUs and top level CSCs

each have an associated test harness (testBase.mdl)

  Test script executes the Model Reference block with pre-‐defined inputs and compares actual output to pre-‐defined expected output

  Test script auto-‐generates code for the Model Reference block, executes the auto-‐code and compares actual output with Simulink output

  The whole test harness is auto-‐coded and the same test cases are run during Code VerificaJon

  Model Reference is run in “Normal” mode for code coverage and “Accelerator” mode for code generaJon or when at the lower level in an integrated unit

  A CSC is auto-‐coded and re-‐integrated as an S-‐funcJon

Unit Test Framework

32

  CSC/CSU Unit TesJng   TesJng is script intensive both for Model

VerificaJon and Code VerificaJon   Each test script has a standard format for

defining test inputs, expected test outputs, and any expected asserts during the execuJon of the test

  Test script can execute both a Simulink test and auto-‐code test or Simulink test only

  Test input and test output are stored as .mat files

  Test log and coverage report is created for each executed test case

Monte-‐Carlo Simula=on Framework

33

  Overview   Modified Draper Simulink® simulaJon

environment (ARAD sim aka GIDE) integrated with Monte-‐Carlo framework for both development and algorithm validaJon

  Simulink® models for Target, Chaser, Planet, Ground. Data logging has been augmented with Monte-‐Carlo framework.

  Model Reference blocks are used for mulJple model instanJaJon, 3x Lidars, 3x GPS receivers, 3x IMUs. Also used for model version control, parallel development etc.

  Flight algorithms are integrated into the simulaJon framework either as Model Reference blocks during development or as S-‐funcJons during algorithm validaJon

  Flight algorithms are executed in their respecJve rate groups: 0.1 Hz, 1 Hz and 5 Hz.


34

  Scheduler provides triggers for each of the 3 triggered subsystems: 0.1 Hz, 1 Hz and 5 Hz

  Scheduler triggers include Jme offsets to model dispatching of rate groups by a rate monotonic scheduler   Sample and hold blocks are used to model rate group delays inherent in a rate monotonic scheduler


35

  Flight Algorithm IntegraJon   Each CSC is integrated as a S-‐funcJon for

final algorithm validaJon   S-‐funcJon wrapper calls the CSC call method

previously described   Ensures actual CSC flight code is used when

performing algorithm validaJon

Metrics

36

  Model Metrics   Complexity metrics were used to decompose CSCs into CSUs   Each CSC and CSU needed to have 100% model coverage for their test cases

  Code Metrics   Complexity and code coverage was analyzed for the auto-‐code   Several reasons for code complexity being greater than the model complexity. Draper guideline was to have the

model complexity < 20 with less regard for the resulJng auto-‐code complexity.

  Structural differences between model and auto-‐code accounted for code coverage < model coverage. For example, uJlity funcJons were unit tested as models but these were in-‐lined in the auto-‐code.

  Peer Review Metrics   Number of defects, defect type, defect severity, defect density, defect rate

Lessons Learned: Overall Process

  Mandate Team-‐wide use of the same development plaxorm and environment (desktop PC, MicrosoN Visual C++, R2008b etc)

  Baseline Matlab/Simulink version and minimize dependencies on addiJonal toolboxes   Avoid frequent version upgrades and new toolboxes.

  Develop and use customized libraries of blocks and models   Pre-‐built blocks and models ensure adherence to standards and efficient auto-‐generated code.

  Modeling standards must be clearly documented and maintained   Basic training and familiarizaJon is needed for developers. Major part of peer review process.

  CSUs and CSCs should be defined as Model Reference blocks   Allows CSUs/CSCs to be developed, tested, maintained and configuraJon controlled as their

own .mdl file   Each CSU/CSC should have a designated owner. Minimizes issues related to graphical merging and

differencing.

  Data dicJonary concept was essenJal to defining and controlling model interfaces   Model data managed and maintained in MicrosoN Excel.

  Model I/O automaJcally generated based on data dicJonary.

37

Lessons Learned: Overall Process

  SoNware engineers are an essenJal part of the team   Auto-‐generaJng code is not just pushing a bu[on!

  Real-‐Time Workshop skills, process skills, standards development for efficient auto-‐generated code etc

  Code verificaJon is sJll necessary

  Typical soNware documentaJon requirements need to be reviewed   Draper IQMS required DIDs for MIL-‐STD-‐498 to be used.

  Tailor DIDs for model-‐based development and use model content to the maximum extent to auto-‐generate design/test documentaJon and requirement/design/test bi-‐direcJonal traceability.

  Model analysis and code analysis can yield different results   Model complexity versus code complexity

  Code path coverage versus model path coverage

  Effort esJmaJon   Need for non-‐SLOC based effort esJmaJon methods

  Techniques based on measures of model complexity are required.

  COTS project is a good basis for the next Draper MBD project.

38

Model=Based$Design$Approach$for$the$COTS$...

Documents

Transcript of Model=Based$Design$Approach$for$the$COTS$...