JOINT SERVICES SOFTWARE SYSTEMS SAFETY HANDBOOK Michael L. Brown, Chairperson JOINT SOFTWARE SYSTEMS...
-
date post
18-Dec-2015 -
Category
Documents
-
view
232 -
download
3
Transcript of JOINT SERVICES SOFTWARE SYSTEMS SAFETY HANDBOOK Michael L. Brown, Chairperson JOINT SOFTWARE SYSTEMS...
JOINT SERVICES SOFTWARE JOINT SERVICES SOFTWARE SYSTEMS SAFETY HANDBOOKSYSTEMS SAFETY HANDBOOK
Michael L. Brown, ChairpersonJOINT SOFTWARE SYSTEMS SAFETY COMMITTEE (JSSSC)
March 2001
presented to the
Safety Critical Systems Club
INTRODUCTIONINTRODUCTIONThis presentation provides an overview of the Software Safety Handbook and its Status. The following topics are addressed:
PurposeBackgroundHandbook Layout and ContentsSoftware Systems Safety ProcessesApplicabilityProject StatusAdditional tasksRecommendations
Provide management and engineering “how-to” guidelines to achieve a reasonable level of assurance that software will execute in a system context with an acceptable level of safety risk.
Initial process and methodology based on Independent Software Nuclear Safety Analysis process tailored to conventional systems and experience from many programs.
Process successfully applied to wide range of systems
PURPOSEPURPOSE
BackgroundBackground
Inconsistent processes– Most incomplete– Many good points but not tied together well
Lessons learned– SSSTRP– Took good points from each process– Developed comprehensive systems engineering
based software systems safety process
HandbookSections
Introductionto System
Safety
Introductionto the
Handbook
Executive Summary
4.0
3.0
2.0
1.0
Appendices
SoftwareSystemSafety
Purpose, Scope, Authority, Overview and Handbook Organization
Motivation and Direction for the Program Director and Program Manager
Overview of System Safety and Safety Risk Management
Definition, References, Supplemental Information, Generic Guidelines, Sample Documents, Lessons Learned, Synopsis of Analysis Techniques and Methods, and Agency Specific Information
Planning, Task Implementation, Risk Assessment and Acceptance, Configuration Management and Reusable Software
HANDBOOK LAYOUT & HANDBOOK LAYOUT & CONTENTSCONTENTS
System and Software System and Software EngineersEngineers
Written for all members of safety team
Purpose and scope of handbook
Authority for software safety program requirements
Organization of handbook
Software Safety EngineerSoftware Safety Engineer
How-to guidance on establishing and conducting software systems safety program
Process based on DODI-5000.1 and DOD-5000.2R System Acquisition Process and Requirements, MIL-STD-498/IEEE STD 1498/12207 Software Development Process, MIL-STD-882 Standard Practice for System Safety, and NATO Standardization Agreement requirements
Section 4.0Section 4.0
Main part of the document– “How-to” Guidance
Complete– However, needs expansion in certain areas (e.g., CDI in
safety critical applications) Covers entire software systems safety process
from concept to system retirement Refers reader to examples in appendices and
reference documents
Software Systems Software Systems Safety ProcessSafety Process
Program management– Planning through execution
Requirements derivation– Generic Requirements from Lessons Learned– System Specific Safety Requirements from system
level analyses Requirements verification and validation
– Detailed analyses– Safety Testing
Safety Assessment
SWS ProcessesSWS ProcessesProgram Phases & Milestones
CSUCSCITest
Software RequirementsAnalysis PD DD
Inter-Operability
Test
SystemRequirements & Design
HardwareDesign
PrototypeManufacturing
ConceptExploration
Program Definition& Risk Reduction Engineering and Manufacturing Development
OPEV AL
MS 0 MS 1 MS 2 MS 3
SRR
SDR
PDR CDR TRR
MIL-STD-498
Phase I Phase II Phase IIIPhase 0
Configuration Management
LifecycleSupport Activity
SPRA SPRA SPRASPRA SPRA SPRA
Code &CSUTest
SystemIntegration
Test
DODI 5000.2
ECPPCRSCN
AnalysisPIPs
Safety Processes for Phase 4 Are an Iteration of the Processes Performed for Phases 0-3.
Preliminary Hazard List (PHL) Development
Tailor the Generic Safety Critical Software Requirements List
SSR
MIL-STD882C
Software Safety Program Planning
Software Safety Program Management
Preliminary Hazard Analysis - (PHA)
Derive System Specific SoftwareSafety Critical Requirements
Preliminary Software Design SubsystemHazard Analysis - SSHA
Software Detailed Design Subsystem Hazard Analysis SSHA
SSR PDR TRRCDR
Verify Software Developed IAW Applicable Standards and Criteria
Software Safety Testing & Analysis
SRR SDR
Software Safety Test Planning
System Hazard Analysis (SHA)
Software Safety Assessment
Code Level Analysis
Production, Fielding/Deployment & Operational
Support
Copy Media & DistributionRecovery System Upgrades
MaintenancePIPs
Technial ReviewObsolescence
SWS PROCESSESSWS PROCESSESSoftware Safety Requirements Derivation
Develop Generic Safety Critical Software Guidelines &
Requirements
Obtain Generic Software Safety Requirements Lists
Tailor Generic Software Safety Requirements/Guidelines List for the specific system and/or subsystem
Categorize and Prioritize Generic Software Requirements/Guidelines
Derive Functional Safety Critical Requirements
Develop Safety Critical Functions List
Develop Potential Functional Hazard List
Preliminary Hazard List (PHL)
Preliminary Hazard Analysis (PHA) Categorize & Prioritize System
Functional Hazards Determine System Level H/W,
S/W and HF Causal Factors Execute System Level Trade
Study Begin Determining all of the
Software Specific Causal Factors
Begin Software & Architectural Detailed Design Trade StudyDerive System-Specific Software Safety Critical Requirements
Requirements Hazard Analysis/Software Criteria Requirements Analysis
Tag Safety-Critical Software Requirements Establish Methods for Tracing Software Safety Requirements to
Test Provide Evidence for Each Functional Hazard Mitigated by
Comparing to Requirements Verify Software Developed IAW Applicable Standards and Criteria
SWS PROCESSESSWS PROCESSESSoftware Safety Program Planning - Customer
• Acquisition Policy• OCD/OR/MENS• DOP• Proposal• Safety Policy• Generic Requirements• Lessons Learned• PHL
Inputs (Later Milestones)
• Draft PHA• Draft TEMP• Draft SEMP• Draft ILSP• Draft CRLCMP• Draft SSS• Draft S/SDD• Draft SOW/RFP
Software Safety Program PlanningProcuring Agency - Customer
• Define Acceptable Level of Risk (HRI, Acceptance Authority, Risk Assessment Methodology)
• Establish System Safety Program:– Identify and Establish Interfaces to Other Program Disciplines– Identify Contract Requirements– Develop POA&M– Establish Safety Data Library– Establish Hazard Tracking Process– Resource Requirements Determination:
»Analyses Required»Tests Required (U)»Resources Required
– Develop Safety Statement of Work: (U)»Define Safety Program Requirements (U)»Define Safety Reporting Requirements (U)»Define Safety Review Requirements (U)
– Proposal Evaluation»Develop Evaluation Criteria»Evaluate Proposals
• Input to SOW/SOO• Input to RFP• Safety Proposal Evaluation Plan• Safety POA&M• Review Requirements• System Safety Management Plan• System Safety Program Plan with
SwSPP Appendix• SSWG Charter and Subgroups• Input to SDP• Input to TEMP• Input to SEMP• Input to ILSP • Input to PHL• Input to PHA• Input to CRLCMP
• Program Initiation • Safety Program Management
• Program Initiation
PM, Principal for SafetySE, SWE, SWSE (EVAL)SSWG (Later Milestones)
Identify and Establish System Safety Program Tasks and Requirements
Purpose:
Primary Sub-Processes
Inputs (Suppliers) Outputs (Customers)
Players
Exit ObjectiveEntry Criteria
References in Addition to ¶ 4.1.1
Comments:
Related Sub-Processes
Proceeding Process Next Process
Service Specific GuidanceIEEE 1228FAR’s
• System Safety Program Management Plan
• Proposal Evaluation and Acceptance• Established System Safety Program
• Program Planning and Management• System Safety Management Plan development
SWS PROCESSESSWS PROCESSESPreliminary Hazard Analysis
Preliminary Hazard Analysis - (PHA)
• Identify System Level Causal Factors (I)• Identify Software Level Causal Factors (I)
– Apply Analysis Techniques (for Example SFTA’s) (I)– Develop recommendations to minimize software induced hazards (I)
• Apply HRI and Prioritize Hazards (I)• Apply Risk Assessment Criteria (Categorize) Hazards (I)• Link hazard causal factors to requirements (I)• Develop design recommendations to mitigate hazards (I)
• RHA/SCRA Inputs• PHA Update• Input to S/W Design• Inputs to S/W Development Plan• Input to Preliminary Software Design
Analysis SSHA• Input to Trade-Off Studies• Design Implementation
Recommendations• HARs• Inputs to Software Test Plans
– Test Requirements• Input to SPRA Reviews• Prioritized Hazard List• Input to SSS, SRS, S/SDD, IRS (I)• Input to ECP, SENs, PIPs (I)
• SOW/SOO/RFP
• Risk Assessment Criteria, HRI
• Draft SSS, S/SDD
• Lessons Learned:
– Analyses on Similar Systems
– Incident Reports
– Previous Mishap Causes
• Life Cycle Environment Profile
• PHL • Tailored Generic Software Safety
Requirements List
Inputs Later Milestones)• HARs
• ECPs
• Safety Data Library (SDL) Maintenance• System Level Trade Studies• Software Level Trade Studies
• Tailor The Generic Software Safety Requirements List
• Functional Hazard List Development
• Program Planning/Management
SwSWG, Domain Experts IEEE 1498
• Derived System Specific Software Safety Critical Requirements
• Preliminary Software Design Analysis
• Upon Completion of PHL • Completion of hazard categorization, prioritization, and determination of all causal factors (initial drafts)
• Resolution of identified hazards (completion)
Identification, Classification and Tracking of System Level Software Hazards
Purpose:
Primary Sub-Processes
Inputs (Suppliers) Outputs (Customers)
Players
Exit ObjectiveEntry Criteria
References
Comments
Related Sub-Processes
Proceeding Process Next Process
References in Addition to ¶ 4.1.1
Handbook ChartsHandbook Charts
Preliminary HazardAnalysis (PHA)
Primary Task
Inputs Outputs
Primary Sub-Tasks Critical Interfaces• Identify Sytem-Level Causal Factors• Identify Software Level Causal Factors• Apply HRI and Prioritize Hazards• Apply Risk Assessment Criteria and Categorize Hazards• Link Hazard Causal Factors to Requirements• Develop Design Recommendations
• Software Safety Working Group• Domain Engineers
• SOW/ SOO/ RFP• Risk Assessment Criteria• Draft SSS, S/ SDD• Lessons Learned• Similar Systems Hazard Analyses• Mishap Data• Life Cycle Environmental Profile• PHL• Tailored Rqmts Lists
• Input to RHA/ SCRA• Update PHA• Input to S/ W Design• Input to SDP• Input to Preliminary S/ W Design Analysis • Input to Trade Studies• Safety-specific S/ W Design Requirements• Hazard Action Records• Prioritized Hazard List
Iterative Loop
Developed OutlineDeveloped Outline
Used Process Charts as basisCovered all items in chartAssigned specific sections to primary
authorsAssigned secondary authors to each section
AuthorsAuthors
Selected for particular expertise
Sources sought from each service, industry, and academia
Coordinating author– Selected for expertise in field– Provided “common voice”
through out handbook
AuthorsAuthors Michael Brown,
NAVSEA Dahlgren John Bozarth, EG&G Janet Gill, NAWC AD Brenda Hyland, NAWC
AD Archibald McKinlay
– BAH
Lenny Russo, CECOM
Coordinating Author– Steve Mattern, SEA
Task AssignmentsTask Assignments
Based on author’s area of expertise– Prior experience– Interest in topical area
Secondary author– Prior experience– Area of Expertise
ApplicabilityApplicability
Process applicable to wide range of military and non-military systems– Weapon Systems– Fire Control and Guidance Systems– Operational Flight Control Programs– Any system containing safety critical software
Handbook provides tailoring guidance for wide range of programs
StatusStatus
Submitted draft to community for review and comment October 1997
Received and collated commentsComments adjudicated by committeeDeveloped revisions based on comments
and additional input from authorsForward revised draft to reviewersFinalized handbook in November 1999
Collating commentsCollating comments
Comments are annotated next to applicable paragraph– Source identified– Comment as provided entered– Rationale for acceptance, rejection, or
modification provided.
Handbook with comments placed on web page
4.2.3 TAILORING GENERIC SAFETY-CRITICAL REQUIREMENTS
Figure 4-20 depicts the software engineering process for tailoring the generic safety-related software requirements list. Generic software safety requirements are those design features, development processes, "best practices", coding standards and techniques, and other general requirements that are levied on a system containing safety-critical software, regardless of the functionality of the application. The PHL, as described above, may help determine the disposition or applicability of many individual generic requirements. The software safety analysis must identify the applicable generic software safety requirements necessary to support the development of the Software Requirement Specification (SRS). A tailored list of these requirements should be provided to the developer for inclusion into the software requirements document.
MLB1: Include references to coding standards and guidelines.
Rationale: Provide validity and additional reference material.
Comment: References provided in appendix X. Hyperlinks to appendix to be provided in final document.
JAG3: Include requirements for evidence supporting tailoring of generic safety critical requirements.
Comment: Concur. Guidelines provided in appendix Y. Appropriate hyperlinks provided in final document.
PROJECT STATUS - Mar PROJECT STATUS - Mar 20012001
First Publication - 31 December 1999– Handbook text complete
All topical areas addressed Some areas require additional work
– Appendices approximately 75% complete Need to develop additional examples for guidance
– Based on previous programs and lessons learned
Need to incorporate additional reference documents
Funding Sources to dateFunding Sources to date
Joint Systems Engineering Steering Group– Funding source: Army
Naval Ordnance Center– WSESRB
USAF HQ-AFSC– Lt. Col. Alberico
Naval Facilities Engineering CommandPersonal time by authors
Additional TasksAdditional Tasks
Commercial Developed Items, Government off-the-shelf (legacy) items, and Non-Developmental Items (Hardware and Software) in safety critical applications
– Selection criteria– Design requirements and
guidelines– “How to” guidelines on
influencing system and software architecture, design, and implementation
– “How-to” analysis and testing guidelines
– Configuration Control Requirements
Additional TasksAdditional Tasks
Software Safety Risk Assessment
– Relating software causal factors and control categories to Hazard Risk Indices
– Relating software metrics to hazard risk assessment
– Guidelines for assessing adequacy of safety program efforts
Hazard Risk IndexFor Example Purposes Only
SeverityProbability
I II III IVCatastrophic Critical Marginal Negligible
1 3 7 13
2 5 9 16
4 6 11 18
8 10 14 19
12 15 17 20
(A) FREQUENT 1 IN 100
(B) PROBABLE 1 IN 1,000
(C) OCCASIONAL 1 IN 10,000
(D) REMOTE 1 IN 100,000
(E) IMPROBABLE 1 IN 1,000,000
Unacceptable Risk - Acquisition Executive Resolution Or Risk Acceptance
Marginal Risk - Design To Eliminate, Requires Program Manager Resolution Or Risk Acceptance
Minimum Risk - Design To Minimize, Requires Program Manager Resolution Or Risk Acceptance
Software Hazard Criticality Matrix
For Example Purposes Only
High Risk - Significant Analyses and Testing Resources
Medium Risk - Requirements and Design Analysis and Depth Testing Required
Moderate Risk - High Levels of Analysis and Testing Acceptable With Managing Activity Approval
SeverityControl Category Catastrophic Critical Marginal Negligible
(I ) Software exercises autonomous control over potentially hazardous hardware systems, subsystems or components without the possibility of intervention to preclude the occurrence of a hazard. Failure of the softwareor a failure to prevent an event leads directly to a hazards occurrence.
(I Ia) Software exercises control over potentially hazardous hardware
systems, subsystems, or components allowing time for intervention by independent safety systems to mitigate the hazard. However, these systems by themselves are not considered adequate.
(I Ib) Software item displays information requiring immediate operator
action to mitigate a hazard. Software failure will allow or fail to prevent the hazard’s occurrence.
(I I Ia) Software items issues commands over potentially hazardous
hardware systems, subsystem, or components requiring human action to complete the control function. There are several, redundant, independent safety measures for each hazardous event.
(I I Ib) Software generates information of a safety critical nature used to make
safety critical decisions. There are several, redundant, independent safetymeasures for each hazardous event.
(IV) Software does not control safety critical hardware systems, subsystems,
or components and does not provide safety critical information.
1 1 3 5
1 2 4 5
1 2 4 5
2 3 5 5
2 3 5 5
3 4 5 5
Extracted from Mil-Std 882C
Moderate Risk - High Levels of Analysis and Testing Acceptable With Managing Activity Approval
Low Risk - Acceptable
Additional TasksAdditional Tasks
Guidelines for selection and/or implementation of:– Language– Operating Systems– Operating
Environments– Middleware
– Pros and Cons for each language, OS, OE, etc.
– Selection and implementation guidelines and criteria
– Analysis and testing guidelines
– Caveats and requirements for each language, etc.
– Configuration control
Additional TasksAdditional Tasks
“Software like” hardware: FPGAs, PLDs
Define safety assessment process and develop guidelines
FUTURE OBJECTIVESFUTURE OBJECTIVES
Hypertext CD-ROM Integrate into DoD Acquisition Deskbook Software Safety WWW Home Page Revisions & updates to handbook On-Site Software Systems Safety Training System Safety Handbook Tool Development
ConclusionsConclusions
The JSSSH provides a comprehensive, Systems Engineering based approach to ensuring that software executes safely within the system context.
The process is designed for application to a wide range of systems without the need for highly specialized expertise (e.g., formal methods)
The JSSSH provides a basis against which to evaluate the thoroughness of Software Systems Safety Programs
The JSSSH is a useful guideline for any safety critical system