Automation for System Safety Analysis Jane T. Malin, Principal Investigator Project: Automated Tool...
-
Upload
ada-patrick -
Category
Documents
-
view
217 -
download
1
Transcript of Automation for System Safety Analysis Jane T. Malin, Principal Investigator Project: Automated Tool...
Automation for System Safety Analysis
Jane T. Malin, Principal Investigator
Project: Automated Tool and Method for System Safety Analysis
Software Assurance SymposiumSeptember, 2007
Complex systems typically fail because of the unintended consequences of their design, the things they do that were not intended to be done. - M. Griffin, System Engineering and the “Two Cultures” of Engineering, March 28, 2007
SAS 07 Automation for System Safety Analysis Malin 2
Problem• Need early evaluation of software requirements
and design– Assess test and validation plans
• Assess system failures and anomalous conditions that may challenge software in system integration testing
• Identify software-system interaction risks– Identify requirements gaps– Perform virtual system integration tests prior to software-
hardware integration• Benefits
– Reduce software-system integration risks early– Reduce requirements-induced errors and rework in later
development phases – Improve efficiency and repeatability of analyzing system
and software risks• Reduce time spent reanalyzing when specifications and
designs change– Reduce contention for software-hardware integration
laboratory resources
SAS 07 Automation for System Safety Analysis Malin 3
Technical ApproachSystematic semi-automated analysis for early
evaluation and rapid update– Capture model of the controlled system architecture
• Abstract physical architecture models with subsystems, functions, interfaces, connections
– Extracted directly from requirements and design text and data
– Capture risks and hazards in model• Constraints, hazards, risks from requirements and design • Risk and failure libraries
– Analyze model and risk data to identify relevant risks and constraints
• Analyze and simulate risk propagation in the system• Use operational and off-nominal scenarios and
configurations– Identify possible test scenarios for virtual system
integration testing
SAS 07 Automation for System Safety Analysis Malin 4
Relevance to NASA
• This work leverages component tools that have been used in NASA applications
• Goal: Integrate and enhance these tools for software assurance during requirements and design phases
• Project test case is NASA Constellation Launch Abort System (LAS)
SAS 07 Automation for System Safety Analysis Malin 5
Extend and Integrate Existing Technology
Requirements and Constraints Text
Risks & Mitigations
Physical/Functional Architecture Models
Discrete Time Simulation Model
Extraction Tool:
Model Parts, Interfaces, Risks, Scenarios
Library
Components, Connections, States & Risks
Functional Diagrams
Aerospace Ontology Taxonomy, Thesaurus, Classes, Synonyms
Modeling Tool:
- Map
-Connect
- Visualize
- Embed problems and states
Analyze and Simulate:
- Identify interaction-risk pairs
- Estimate severity in nominal and fault scenarios
- Investigate influence of timing
ReportsPairs, Paths, Risky Scenarios,Test Cases for Virtual System Integration Testing
Virtual System Integration Lab (VSIL)
Inputs Extraction Modeling Analysis Simulation Testing
Interaction Model
SAS 07 Automation for System Safety Analysis Malin 6
Extraction Tool and Nomenclature
• Reconciler Extractor– Extract model parts from requirements text and data from
functional analysis and threat/risk analysis– Semantic parsing for text analysis and word/phrase classification– Extract operational scenarios from functional analysis data
• Aerospace Systems Library and Ontology – Classes of model elements with properties and defaults– Taxonomy with synonym lists, for parsing and mapping to types
of model elements– Extensive problem taxonomy and thesaurus that includes hazard
types from Constellation Hazard Analysis handbook• Current NASA use: Semantic text mining to classify JSC
Discrepancy Reports (DRs) for trend analysis– Discrepancy Reports describe mechanical, electrical, software
and process discrepancies in engineering and operating NASA-furnished equipment
SAS 07 Automation for System Safety Analysis Malin 7
Discrepancy Report Analysis Tool
Cross-Cutting Teams Receive Subsets of DRs in Excel File and Browsers
Extract DRs from Database
• Analyze text in each DR Problem Description
• Identify categories of problems described
• Sort DRs into subsets for cross-cutting teams: Mechanical, Electrical, Software, Process, Other
Browsers for Each Cross-Cutting Team, with links to Database
Filterable Excel File
SAS 07 Automation for System Safety Analysis Malin 8
Model-Based Safety Analysis Case
• Model extraction and hazard analysis were demonstrated in 2005– Case: Generic unmanned spacecraft;
concerns about transmitter noise– Requirements from SpecTRM and risks from
Defect Detection and Prevention (DDP) Tool
– Reference: J. T. Malin, D. R. Throop, L. Fleming and L. Flores, “Transforming Functional Requirements and Risk Information into Models for Analysis and Simulation,” 2005 IEEE Aerospace Conference Proc., March 2005.
SAS 07 Automation for System Safety Analysis Malin 9
Reconciler Information Extractor
Requirements
Risks and Mitigations
Parse and Extract:
Model Parts
Interfaces
Vulnerabilities Threats/Risks Mitigations
Scenarios
Functional Diagrams XML- Structured Data
Aerospace Ontology
Classes, Synonyms
SAS 07 Automation for System Safety Analysis Malin 10
Reconciler Tool Extracts Model Parts from Text
• Parses the Process and Requirements sentences from SpecTRM or Cradle
• Extracts functions and objects• Classifies functions (uses Aerospace Ontology)• Formats the parsed knowledge
– In XML format or OWL format
• Passes results for mapping into models
OWL XML
SAS 07 Automation for System Safety Analysis Malin 11
Requirements Model (Shift Info)
• Operation/Function: Transfer (“Downlink”)
• Agent/contributor: ?
• Affected Operand: Information
• Operand Source: ?
• Operand Destination/Goal: ?
• Path Type: Information
• Effect value/measures: “Successful”
• …
Problem Model (Failure of Function)
• Problem: Failure of function (“Failure”)
•Agents/contributors: “Transmission Subsystem, Transmitter…”
• Impacted Entity: “Telecom Subsystem”
• Impacted Objective (link to): “Downlink Successful”
• …
Reconciler Tool Extracts Risks
RAP or ARM Risk Analysis and Matrix
DDP Analysis and Visualization of Risks, Mitigations and Costs
Mitigation Model (Replace)
• Function Type: Replace (“Redundancy”)
• Replaced: “Transmitter”
• Replacement: “Transmitter Spare”
• Counteraction Type: Recover
• Counteracted Problem (link to): “Telecom Sub… Failure… Transmitter”
• …
Objective: “Downlink successful”
Risk: “Telecom Subsystem Failure: Transmission: Transmitter”
Mitigation: “Redundant Systems: Transmitter”
Transmitter Failure Mitigation: Redundant Transmitter
Telesub: Failure(Transmission sub: Transmitter)
SAS 07 Automation for System Safety Analysis Malin 12
Modeling and Analysis Tools• Hazard Identification Tool (HIT) identifies threats
and risks– Model mapper and developer – Hazard path analyzer– Model diagram visualizer– Least mature tool in the suite
• Hazard Identification Tool was demonstrated in SpecTRM spacecraft case – Use Reconciler output to develop interaction
architecture and risk model – Identify pairs that are not intended to interact
• Hazard sources • Sensitive or vulnerable objects or functions
– Analyze paths between pairs and estimate severity
SAS 07 Automation for System Safety Analysis Malin 13
Hazard Identification Tool
Architecture Visualizer
Library
Components
Functions
Problems
Extracted Model Data
(XML from Reconciler)
Modeler:
- Map
- Connect
- Embed problems and states
Aerospace Ontology
Classes, Synonyms Path Analyzer:
- Find pairs
- Search graph of paths in scenarios
- Estimate Severity
ReportPairs, Paths
Risky Scenarios,Test Cases
SAS 07 Automation for System Safety Analysis Malin 14
Modeler: Each Requirement Provides
Pieces of the Architecture[C.1] Telecommunication Subsystem (Telesub)• [C.1.1] The CDHC sends the TeleSub a compressed picture.
[FG.1] [TeleSub C.1.4]
• [C.1.2] The CDHC sends the TeleSub telemetry. [FG.2] [FR.1] [FR.5] [TeleSub C.1.5]
• [C.1.3] The CDHC sends In View of Ground alerts to the TeleSub. [DP.5.6] [TeleSub C.1.6]
• [C.1.4] The CDHC receives plan files from the TeleSub. [FR.3] [TeleSub C.1.3]
• [C.1.5] The CDHC receives ground commands from the TeleSub. [FR.3] [TeleSub C.1.2]
• [C.1.6] The CDHC receives the TeleSub operating state from the TeleSub. [DP.5.5] [TeleSub C.1.1]
…
[C.2] Camera Subsystem• [C.2.1] The CDHC sends the Camera a "take picture" command.
[FG.2] [FR.1] [FR.3] • [C.2.2] The CDHC sends the Camera x, y and z gimballing
coordinates. [FG.2] [FR.1] [FR.3] • [C.2.3] The CDHC sends a turn on command to the Camera.
[DP.5.3] [H Constraint 1.1.4]• [C.2.4] The CDHC sends a turn off command to the Camera.
[DP.5.3] • [C.2.5] The CDHC receives a compressed picture file from the
Camera. [FG.1] [FG.2] [FR.1]
…
[C.4] Attitude Determination Subsystem• [C.4.1] The CDHC receives an In View of Ground alert from the
ADS. [DP.5.6] [ADS]• [C.4.2] The CDHC receives the ADS operating state from the
ADS. [DP.5.5] [ADS]
Requirements Model (Shift Info)
• Function Type: Transfer (“Send”)
• Agent/Contributor: Subsystem (“CDHC”)
• Affected Operand: Information (“Telemetry”)
• Operand Source: Subsystem (“ CDHC”)
• Operand Destination/Goal: Subsystem (“ Telesub”)
• Path Type: Information
• …
CDHCFn: Send
Telesub
Telemetry
Physical/Functional Architecture Fragment
SpecTRM: Spacecraft Command and Data Handling Computer (CDHC) Send/Receive Requirements
SAS 07 Automation for System Safety Analysis Malin 15
Modeler: Architecture Model and Visualization of a Set of Requirements
[C.1] Telecommunication Subsystem (TeleSub)• [C.1.1] The CDHC sends the TeleSub a compressed
picture. [FG.1] [TeleSub C.1.4]• [C.1.2] The CDHC sends the TeleSub telemetry. [FG.2]
[FR.1] [FR.5] [TeleSub C.1.5] • [C.1.3] The CDHC sends In View of Ground alerts to the
TeleSub. [DP.5.6] [TeleSub C.1.6]• [C.1.4] The CDHC receives plan files from the TeleSub.
[FR.3] [TeleSub C.1.3]• [C.1.5] The CDHC receives ground commands from the
TeleSub. [FR.3] [TeleSub C.1.2]
• [C.1.6] The CDHC receives the TeleSub operating state
from the TeleSub. [DP.5.5] [TeleSub C.1.1] …
[C.2] Camera Subsystem• [C.2.1] The CDHC sends the Camera a "take picture"
command. [FG.2] [FR.1] [FR.3] • [C.2.2] The CDHC sends the Camera x, y and z gimballing
coordinates. [FG.2] [FR.1] [FR.3] • [C.2.3] The CDHC sends a turn on command to the
Camera. [DP.5.3] [H Constraint 1.1.4]• [C.2.4] The CDHC sends a turn off command to the
Camera. [DP.5.3] • [C.2.5] The CDHC receives a compressed picture file from
the Camera. [FG.1] [FG.2] [FR.1]
…
[C.4] Attitude Determination Subsystem (ADS)• [C.4.1] The CDHC receives an In View of Ground alert from
the ADS. [DP.5.6] [ADS]• [C.4.2] The CDHC receives the ADS operating state from
the ADS. [DP.5.5] [ADS]
Note: CDHC is Command and Data Handling ComputerPhysical/Functional Architecture Model
SAS 07 Automation for System Safety Analysis Malin 16
Modeler: Seed the Spacecraft 1 (SC1) Model with Problems and Mitigations
• Libraries of objects (components) and functions
– Typical components and operating modes– Typical functions and failures– Typical output that may be a problem– Typical sensitivities and tolerances– Typical mitigations
• Manual additions to model– Add spare transmitter (xmitter)– Transmission performance (rate)
degradation due to noise– CDHC Comm Controller controls mitigation:
switch to spare transmitter– Add Comm Network, Ground data
components– Remove Reaction Control System (RCS)
and camera – Add Power (PwrSpply) and Thermal Control
(ThermalSys) subsystems, with new risks and mitigations
• ThermalSys is noise source (when on)• Power lines can transmit noise
SAS 07 Automation for System Safety Analysis Malin 17
Path Analyzer: Find Potential Interaction Problems
1. Find matching pairs of components (hazard source-vulnerable sink)
2. Find system interaction paths that permit hazards to impact sensitive components and functions
3. Estimate local and integrated system hazard impact severity
SAS 07 Automation for System Safety Analysis Malin 18
Path Analyzer: Incremental Quick Look Approach
• Simple early threat analysis, refined as design information becomes available– Identify risky matching pairs from component or
function vulnerabilities, threats and hazards– Search for paths between pairs along connections
or dependencies – Make search dependent on configuration
information, with changeable configuration and operational states
• Estimate impact severity from local estimates of severity
SAS 07 Automation for System Safety Analysis Malin 19
Simulator: CONFIG Simulation Tool to Assess Timed Scenarios
NASA experience with CONFIG hybrid discrete event simulation tool: Used for software virtual validation testing for 1997 90-day manned Lunar Life Support Test
• Software: Intelligent control for gas storage and transfer • Testing: Simulated failures and imbalances that would
not be tested in hardware-software integration• Too slow to develop, too expensive, too destructive
• Results: Identified software requirements deficiencies
SAS 07 Automation for System Safety Analysis Malin 20
Add Timing to Selected Scenarios and Narrow Potential Problem Set
Model data
Integrated Architecture Model
Mapped Timed Simulation Model
Log/Report Specifications
Scenario Scripts
• Map components and connections
• Reuse scenario scripts and report specifications
SAS 07 Automation for System Safety Analysis Malin 21
Virtual System Integration Lab (VSIL)
• Triakis has used VSIL in >25 avionics verification projects
• Models and problem configurations for new tests and test suite models
Models and Test Definitions
DE: detailed executable, the simulation of the embedded controller hardware
ES: executable specifications
V&V: verification and validation
SAS 07 Automation for System Safety Analysis Malin 22
Accomplishments: First 9 Months
• Drafted Concept of Operations• Enhanced tools• Completed a simple integration of tool
functions, inputs and outputs– Based on SpecTRM-style requirements text
• Selected Constellation Launch Abort System Case– Gained access to Cx Windchill materials 9/07
• Takes time, but requirements may now be mature enough
SAS 07 Automation for System Safety Analysis Malin 23
Concept of Operations• Drafted and iterated a draft Concept of
Operations Document with Safety and Mission Assurance (S&MA) (Due 12/07) – Data flow diagram shows use of tools to support S&MA
software processes and virtual system integration testing
SAS 07 Automation for System Safety Analysis Malin 24
Tool Enhancements
• Refined Reconciler parsing and extraction capabilities
• Re-implemented Hazard Identification Tool functions for constructing hierarchical models from extracted model parts – No longer uses Protégé– Uses elements of CONFIG simulation tool for
automatic and manual model construction and visualizing architecture models
• Re-implemented risk path analyzer code, to make planned extensions feasible
SAS 07 Automation for System Safety Analysis Malin 25
Aerospace Ontology Library Objects
• Enhanced Aerospace Ontology class objects for modeling risks and qualitative dependency relationships– General for multiple types of influences among
entities and functions/actions • Capability, integrity/reliability, performance timing and quality
or controllability
Influencing Factor Relationships – Positive-Negative (signed) relation to influenced variable or problem– Importance (degree of worst-case impact)– Likelihood (probability of occurrence of factor)– Cross-reference to Requirements and Constraints
SAS 07 Automation for System Safety Analysis Malin 26
Aerospace Ontology Action Primitives• Enhanced Aerospace Ontology taxonomy for
straightforward mapping to primitives used in path analysis
Place/Arrange– Move + EntityOperand + Path
• Transport + SourcePlace + DestinationPlace– Change “Owner”
• Transfer + EntityOperand + Source + Sink • Input/Output + EntityOperand
– Output» Emit (Active-Output)» Release (Passive-Output)
– Take-In» Input (Active Take-In)» Receive (Passive Take-In)
Process– Transform + EntityOperand + Parameter
• Phase change, change in composition…– Change Position on a Scale + EntityOperand + Parameter
• Increase• Decrease
Control– Regulate + EntityOperand + Parameter
SAS 07 Automation for System Safety Analysis Malin 27
Simple “Hello World” Architecture Case
<MODEL name="CSRL Spacecraft" type="SYSTEM-UNIT"> <COMPONENTS> <COMPONENT name="CDHC" type="SYSTEM-UNIT"> </COMPONENT> <COMPONENT name="TELESUB" type="SYSTEM-UNIT"> </COMPONENT> </COMPONENTS> <CONNECTIONS> <CONNECTION> <FROM_DEVICE name="CDHC"> </FROM_DEVICE> <TO_DEVICE name="TELESUB"> </TO_DEVICE> <ENTITY-TRANSFERRED name="DATA"> </ENTITY-TRANSFERRED> </CONNECTION> </CONNECTIONS> </MODEL>
CDHC
Telesub
Data
CSRL Spacecraft
CDHC: Command and Data Handling Computer
Telesub: Telemetry subsystem
– Extracted model parts from small set of requirements (2 components, 1 connection) – Defined output specifications for XML model files from HIT for
VSIL– Expanded “Hello World” example case definition to include risk information in components
SAS 07 Automation for System Safety Analysis Malin 28
Potential Applications
• Visualize integrated requirements• Evaluate completeness and consistency of
requirements and risk• Quickly reanalyze each revision of
requirements and risk• Validate failure modes and effects analysis
(FMEA) and fault trees• Validate and test early with low-fidelity
simulation
SAS 07 Automation for System Safety Analysis Malin 29
Next Steps• Complete first version of Launch Abort System
case and evaluate– Text extraction from requirements and risks – Model construction and visualization– Model analysis to identify interaction risks and test
configurations for virtual software integration testing• Complete Concept of Operations• Enhance tool suite capabilities, integration and
user interfaces– Achieve Technology Readiness Level (TRL) 6 – Prepare for other uses for Constellation software
assurance
SAS 07 Automation for System Safety Analysis Malin 30
ReferencesJ. T. Malin and D. R. Throop, “Basic Concepts and Distinctions for an Aerospace Ontology of
Functions, Entities and Problems,” 2007 IEEE Aerospace Conference Proc., March 2007.
J. T. Malin and L. Fleming, “Vulnerabilities, Influences and Interaction Paths: Failure Data for Integrated System Risk Analysis,” 2006 IEEE Aerospace Conference Proc., March 2006.
T. L. Bennett and P. W. Wennberg, “Eliminating Embedded Software Defects Prior to
Integration Test,” CROSSTALK: The Journal of Defense Software Engineering, December 2005.
J. T. Malin, D. R. Throop, L. Fleming and L. Flores, “Transforming Functional Requirements and Risk Information into Models for Analysis and Simulation,” 2005 IEEE Aerospace Conference Proc., March 2005.
D. Throop, “Reconciler: Matching Terse English Phrases,” Proceedings of 2004 Virtual Iron Bird Workshop, NASA Ames Research Center, April, 2004.
J. T. Malin, D. R. Throop, L. Fleming and L. Flores, “Computer-Aided Identification of System Vulnerabilities and Safeguards during Conceptual Design,” 2004 IEEE Aerospace Conference Proc., March 2004.
J. T. Malin, L. Fleming and T. R. Hatfield, “Interactive Simulation-Based Testing of Product Gas Transfer Integrated Monitoring and Control Software for the Lunar Mars Life Support Phase III Test,” In Proceedings of SAE 28th International Conference on Environmental Systems. SAE Paper No. 981769, 1998.