VERIFYING IMPLEMENTATION PROTOTYPE

Independent Test Capability Team - Bill Stanton- Jarrod Petersavage- Justin Morris- Steven Seeger- Mike Wise

1

VERIFYING IMPLEMENTATION

2

GOAL

IVV 09-1: Verify System Behavior

…Verify actual system behavior in the implemented system against expected (or designed) behavior…

Engineering Services Initiative #10:

Provide a capability to dynamically assess mission’s software against expected software behaviors.

STEPS

1. Understand the Problem a. Vision b. Concept of Operations2. Practical Example: Case

Study a. Proof of Concept b. Role of the SRM c. Effort, Lessons Learned3. Capture System Behaviors a. Models b. GUI Mockups c. Design Documentation 4. Acquire Supporting Tools

and Develop Framework5. Maintain Capability and

Supports Applicable Projects

OBJECTIVES

CASE STUDY APPROACH

ROLE OF THE PBRA & SRM

CASE STUDY RESULTS

LESSONS LEARNED

FUTURE WORK

3

CASE STUDY APPROACHNavigator Software on GPM Project4

WHAT DID WE DO?

Chose a Project GPMExamined PBRA Results

o GPM PBRA Profile: March 5, 2009o GPM PBRA-Lite: May 28, 2009

Chose a small example Navigator Softwareo Requirements:

o Code is Availableo Some Documentation Availableo Supporting Tool(s) Available o Modeling Artifacts Available

Successfully Run Code Executed using EDGE IDE with SimTest Simulator

o Develop and Execute Test Cases using Sequences from GPM SRM (Working) 5

ROLE OF THE PBRA AND SRM

6

GPM PBRA PROFILEMARCH 5, 2009, MACAULAY, DUNKERLEY

System CapabilitiesJ1: Launch and Achieve Initial OrbitJ2: Checkout SpacecraftJ3: Fly in Required OrbitsJ4: Obtain Science DataJ5: Maintain Health and Safety of SpacecraftJ6: Process Science DataJ7: Decommission Spacecraft

J4J1

J5

J6

J3J7

J2

Impact

7

ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)

ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)

“Maintain Health and Safety of Spacecraft Activity Diagram”

8

H H

None

H

H

H

Maintain Propulsion System

H

ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)

9

H

H

H

H

H

M

M

M

M

L

Maintain Propulsion System Activity Diagram

Determine Position & Delta-V

M

ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)

Determine Position and Delta-V Behavior is implemented in the GPS Navigator The SC determines its position using GPS

position information.

10

RESULTS & LESSONS LEARNED

11

CASE STUDY RESULTS

Identified a capability using PBRA & SRM to drive Verification Implementation Activities (Case Study)

Executed Navigator Flight Software using a trial version of a COTS toolset in 2 months Duration includes obtaining

all required tools and artifacts and configuring the environment

Develop serial interface to provide conduit for testing

Utilize Elaborated Sequence Diagrams (to be developed by SRMV PL) to drive test cases

Accomplishments Future Work

12

GPM Electrical Block Diagram

13

Navigator Software Simulation

Navigator software runs on a FreeScale ColdFire 5307.

On GPM, the Navigator is part of GN&C and connects to C&DH via RS-422.

GPM has two Navigator units, but we only need to test one.

Initial trial run used EDGE Development suite from Mentor Graphics. Compiled code as x86 without Nucleus OS code. Nucleus runtime provided by simtest.

14

Navigator Commands

All Navigator commands and responses are transmitted over RS-422.

ITC team determined that Navigator would be a good place to start because sending and receiving serial data is not difficult.

Navigator commands over serial include read and write memory, patience, and ephemeris data. Easy proof of concept with simple write and

read-back operation. Can expand simulation further with more

complex commands. CCSDS message format.

15

Issues with Hardware Simulation

Navigator has an RF board that receives GPS signals. Difficult to simulate.

Developer usescomplex hardwareand softwaresimulation solution.

16

Software Simulation End Goal

Run binaries provided by vendors on an instruction set simulator. No need to compile Navigator software for x86. No chance of results varying by build process.

More hardware interaction. Simulation Real hardware

Headless operation with all simulations driven by test scripts.

17

LESSONS LEARNED Working with Trial Versions of Tools often Proves

Difficult Vendor Support 30-day to 60-day Trial Window Limited Tool Capabilities

Importance of Communications between Product Lines Modeling Artifacts help drive Verification Implementation

Activities Importance of SRMV and Verification Task Scheduling

Initial setup time will vary depending on test environment and requirements

Project Leads & Product Lines need to identify Verification Implementation targets early in lifecycle to allow time for tool acquisition, development time, and training

Leveraging of Developer’s Capabilities may prove Beneficial Parallel Activities

18

PARALLEL ACTIVITIESOTHER ITEMS BEING WORKED BY THE ITC TEAMNOT SCOPE OF PRESENTATION – INFORMATION SHARING

SoftSim All-digital system simulation with flight-like

interfaces Juno and GRAIL missions

VxWorks Utilized by almost all Science missions ITC is obtaining trial version, inquiring about

licensing, and training License required to support SoftSim testing

19

ANY QUESTIONS?

20

BACKUP SLIDES21

PROVIDES ADDITIONAL CAPABILITY“STATIC VERSUS DYNAMIC ANALYSIS”

Static Analysis Dynamic Analysis

• Finds weaknesses in exact location• Allows quicker turnaround for fixes• Finds errors earlier in lifecycle• Automated Tools• Relatively fast• Can scan all of code

•Assess Mission Software against Expected Software Behaviors•Finds run-time vulnerabilities•Provide increased flexibility of what to look for•Identifies vulnerabilities that may have been false negatives in static analyses•Validation of Static Analysis Findings

22