Assessment for Learning. What is the purpose of assessment? To educate or to select?
Purpose of the Assessment
description
Transcript of Purpose of the Assessment
Brussels, 23rd March 2010
Digital SNOWTAMSafety Impact AssessmentViewed from a non-safety expert
Purpose of the Assessment
SNOWTAM Trial Safety Impact Assessment 3
Safety Impact Assessment Purpose
Demonstrate that the Digital SNOWTAM infrastructure is providing a satisfactory service to the users involved
The test facility fulfils its intended function
It is acceptably safe
SNOWTAM Trial Safety Impact Assessment 4
Overall Safety Argument in GSN*
GSN: Goal Structuring Notation
Arg0Digital SNOWTAM infrastructure provides a satisfactory service for trials activities
Arg1Digital SNOWTAM infrastructure fulfils its intended function
Arg2Digital SNOWTAM infrastructure is acceptably safe
Argue on the basis that the Digital SNOWTAM infrastructure fulfils its intended function and
is acceptably safe
Cr001Satisfactory means:- fulfils the intended function- is acceptably safe
C001Digital SNOWTAM infrastructure is used to support trials activities
C002Digital SNOWTAM infrastructure is not considered as an operational ATM system
‘Success’ approach
Normal operations
‘Failure’ approach
Failure modes
SNOWTAM Trial Safety Impact Assessment 5
Safety ‘coverage’ of the V-cycle
Intended Functions
Specification
Design
Implementation
Operations
Arg1: Nominal Mod(success approach)
Arg2: Failure Mode(failure approach)
‘Success’ approach
SNOWTAM Trial Safety Impact Assessment 7
Users and Intended Functions
SNOWTAM Trial Safety Impact Assessment 8
System Fulfils its Intended Functions
Arg1Digital SNOWTAM infrastructure fulfils its intended function
Argue that Digital SNOWTAM infrastructure fulfils its intended
functions all along its lifecycle
Arg1.1Digital SNOWTAM infrastructure service is specified to be fulfil the intended function
Arg1.2Digital SNOWTAM infrastructure logical design satisfies the specification
Arg1.3Digital SNOWTAM infrastructure has been implemented completely and correctly for trials
Arg1.4Digital SNOWTAM infrastructure continues to fulfil the intended function during trials
Test infrastructure fulfils intended functions
Specification Design Implementation OperationHow to help the Safety Expert checking this?
Traceability
SNOWTAM Trial Safety Impact Assessment 9
Specifications
Specification
Related IF#
AirlineNOTAM office
AirportSystem Developer
S01: Automatic conversion between current SNOWTAM messages and digital SNOWTAM encodings, i.e.:To exploit the semi-structured format of the SNOWTAM messages in order to automatically convert them into digital SNOWTAM encodings;
A.1 A.2
B.1, C.1, D1
S02: Generation of digital SNOWTAM encodings either from existing current SNOWTAM messages or through direct input (using graphical and forms tools)
B.2 C.2 C.3
S03: Output of the generated SNOWTAM text message (after conversion from digital SNOWTAM encodings)
B.2 C.2 C.3
(*) Table partially reproduced
SNOWTAM Trial Safety Impact Assessment 10
Design
Design Related S#
D02: Conversion module for SNOWTAM messages, which creates AIXM 5.1 Surface Contamination data from messages received from EAD INO (4).
S01 S05
D03: Graphical User Interface (GUI) for data provider, which enables: The direct input by NOTAM operators and airport managers in the application database of surface contamination data;Manage Rejected SNOWTAM – indicating the syntax errors that have stopped the automatic interpretation of the message and allowing the correction and re-submission of the SNOWTAM message to the interpreter (only for selected users); All the functionality available for the data user, as described at D5.
S02 S04 S05
‘Failure’ approach
SNOWTAM Trial Safety Impact Assessment 12
System is Acceptably Safe
Arg2Digital SNOWTAM infrastructure is acceptably safe
Arg2.1Digital SNOWTAM infrastructure has been specified and designed to be safe in case of failure or misuse
Arg2.2Digital SNOWTAM infrastructure has been implemented completely and correctly
Cr002:Acceptably safe means “no impact on real ATM related operations”.
Hazard analysis▼
Safety requirements
Checksafety requirements
are covered
SNOWTAM Trial Safety Impact Assessment 13
Simplified process
Hazard analysis
Mitigation means
Safety requirements
Specification, Design
Test infrastructure, Operations
SNOWTAM Trial Safety Impact Assessment 14
Identified HazardsHazards list Related S#
H1 Incorrect or incomplete information is graphically provided by Digital SNOWTAM application
S04, S01
H2 Incorrect detailed information is graphically provided by Digital SNOWTAM application
S04, S01
H3 Lack of information in Digital SNOWTAM application with respect to an airport for which an official SNOWTAM message has been issued
S04, S05, S01
H4 An official SNOWTAM message is incorrectly rejected by the Digital SNOWTAM application
S05, S01
H5 Incorrect SNOWTAM text message is provided by Digital SNOWTAM application
S03
H6 Incorrect information is encoded by the user in the Digital SNOWTAM infrastructure
S02
H7 Incorrect detailed information is encoded by the user in the Digital SNOWTAM infrastructure
S02
H8 Incorrect SNOWTAM encoding is provided by Digital SNOWTAM application
S06
SNOWTAM Trial Safety Impact Assessment 15
Mitigation means => Safety Requirements
Safety RequirementsAirline
NOTAM office
AirportSyst
. Dev.
A B1 B2 C1 C2 C3 D1
SR-1 The evaluation of the Digital SNOWTAM application shall be carried out by users only when time permitted and shall not be detrimental to operational tasks they must conduct.
X X X X X X
SR-2 The access to information for pilots and airline operational centres has to be limited, without possibility of modifying data in the Digital SNOWTAM application. X
SR-3 Official SNOWTAM information shall prevail, for airline operational centres and for pilots, to support decision making. X
SR-4 For pilots the information shall not be directly accessible during the flight (only accessible in the pre-flight phase). X
SR-5 Pilots shall use last update of the surface contamination and friction coefficient provided by the corresponding airport controller (TWR) or by the ATIS. X
SR-6 In case of inconsistency between the official SNOWTAM messages and the Digital SNOWTAM information is detected, NOTAM office uses current means to check its validity and modify it if and as necessary.
X X X
(*) Table partially reproduced
SNOWTAM Trial Safety Impact Assessment 16
Checking coverage of safety requirements
Implementation elementsSR-1
SR-2
SR-3
SR-4
SR-5
SR-6
SR-7
SR-8
SR-9
SR-10
SR-11
SR-12
PI-1 The Digital SNOWTAM application is only accessible thought Internet at the EUROCONTROL portal http://extranet.eurocontrol.int.
A
PI-3 The logo “TRIAL SNOWTAM” is presented to the user in all the screenshots of the application in order to remind the user that the information provided by the Digital SNOWTAM application is not to be used for operational purposes.
X
UI-4 In complement to the dedicated training, supporting documentation is provided to the different users in order to be able to manipulate the Digital SNOWTAM application ([10]).
AA A B B B
B
UI-5 Contractual agreement to be signed by NOTAM office stating that the use of Digital SNOWTAM application is for trial purpose only, i.e. official SNOWTAM information prevails, and current means are to be used to check its validity and modify it if/as necessary.
B B
(*) Table partially reproduced
Assessment caveats
SNOWTAM Trial Safety Impact Assessment 18
Caveats
- Assumptions
- Outstanding Issues
- Limitations
Digital SNOWTAM performances depend on the availability and the quality of some external data as the SNOWTAM information itself and some static aeronautical information (e.g. airport layout)
The real effectiveness for most of the Safety Requirements mainly depends on each user and their awareness on the use they can do of Digital SNOWTAM infrastructure
Conclusion
SNOWTAM Trial Safety Impact Assessment 20
Assessment Conclusions
- The proposed Digital SNOWTAM infrastructure fulfils the intended functions for the Digital SNOWTAM trial.
- There is no impact on real ATM related operations while using Digital SNOWTAM infrastructure during the trial.
SNOWTAM Trial Safety Impact Assessment 21
Developer’s ‘mantras’
- Integrate safety aspects in your development as soon as possible
- Traceability
Questions?