Towards Automatic Instrumentation by Learning to Separate ...
Towards Model-Based AHMI Automatic Evaluation
-
Upload
jean-vanderdonckt -
Category
Technology
-
view
904 -
download
0
description
Transcript of Towards Model-Based AHMI Automatic Evaluation
* Université catholique de Louvain (UCL) Belgian Laboratory of Computer-Human Interaction (BCHI)
{juan.m.gonzalez, jean.vanderdonckt}@uclouvain.be† OFFIS e.V., Escherweg 2, 26127,
Oldenburg, Germany{luedtke, osterloh}@offis.de
Towards Model-Based AHMI Automatic Evaluation
Juan Manuel Gonzalez-Calleros*, Jean Vanderdonckt*, Andreas Lüdtke†, Jan-Patrick Osterloh†
• Existing work on automated evaluation of AHMI • Some studies suggested that AHMI and software user interfaces
share several similarities:• In terms of interaction techniques for input/outputWilliges, R.C., Williges, B.H., and Fainter, R.G., Software interfaces for aviation systems, in
Human Factors in Aviation, E.L. Wiener and D.C. Nagel (Eds.) 1988, Academic Press: San
Diego: pp. 463-493. • In terms of automatic analysis of workloadEldredge, D., S. Mangold, and R.S. Dodd, A Review and Discussion of Flight Management
System Incidents Reported to the Aviation Safety Reporting System 1992, Battelle/U.S. Dept. of Transportation.
• In terms of automatic evaluation based on goal modelsIrving, S., Polson, P., Irving, J.E., A GOMS Analysis of the Advanced Automated Cockpit, Proc.
of ACM Conf. CHI’94, ACM Press, New York, 1994, pp. 344-350
Introduction
10 April 2023 Page 2
• Several challenges for Aircraft cockpit design, such as: • Introduction of new technologies break previous user experience.
Introduction
10 April 2023 Page 3
• New challenges for AHMI: analysis, design, implementa-tion, and evaluation
• Integrating evaluation in the loop of AHMI life cycle involves the use of pilots and a physical simulator. • Shortcomings: availability, cost, time,…
• Human project: new ways to substitute pilots and physical simulator are explored by coupling them to virtual simulator
• Our goal: to introduce UI automated evaluation (workload, execution time, usability guidelines) in the life cycle
Introduction
10 April 2023 HMAT Workshop, June 2010 Page 4
• Interactive Cooperative Objects (ICOs)• Used to model aircraft interactive systems (air traffic workstations, civil
aircraft cockpit and military aircraft cockpit)• Formal description for interactive cockpit applications using Petri nets• Behavioural aspects of systems in the cockpit are modelled• Recent work on usability, reliability, and scalability
• Navarre, D., Palanque, Ph., Ladry, J.-F., Barboni, E., ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability, November 2009, Transactions on Computer-Human Interaction (TOCHI) , Volume 16 Issue 4
• To be added • More widgets, more than system and task models• Usability guidelines• Notion of resilience
• David Navarre, Philippe Palanque, Eric Barboni, Jean-François Ladry, Célia Martinie, Designing for Resilience to Hardware Failures in Interactive Systems:A Model and Simulation-Based Approach Reliability Engineering and System Safety, vol. 95, n°6, 2010
State of the Art
10 April 2023 HMAT Workshop, June 2010 Page 5
• Interactive Cooperative Objects (ICOs)
State of the Art
10 April 2023 HMAT Workshop, June 2010 Page 6
• ARINC 661 Standard• Defines protocols to communicate the dialogue and the functional core
of cockpit display system:
• a user application is defined as a system that has two-way communication with the CDS (Cockpit Display System):
• Transmission of data to the CDS, possibly displayed to the flight deck crew• Reception of input from interactive items managed by the CDS
• A set of widgets is included as a recommendation
• Problems• No design guidelines for the widgets,
• No method to design UIs are considered in the standard
• Each manufacturer is free to implement their own understanding about the standard
• The ARINC standard is not used for primary cockpit applications (AHMI)
• Only deals with secondary applications involved in the management of the flight such as the ones allocated to the Multiple Control Display Unit
State of the Art
10 April 2023 HMAT Workshop, June 2010 Page 7
• A Methodology is suggested that is composed of:• Models (Traditional Widgets + AHMI specific UI elements)• Language (User Interface Description Language)• Method (Model-Based UI Design applied to the AHMI)• Software support
• A Language Engineering approach• Semantics as meta-models (UML class diagrams)• Syntax as XML Schemas• Stylistics is the visual syntax, in our context not used.
• A Structured Method compliant to the Cameleon Reference Framework for UI development• Just focus in the layer that concerns to the concrete description
model.
Model-Based AHMI Design
10 April 2023 HMAT Workshop, June 2010 Page 8
• The Concrete UI Model (CUI) allows:• Specification AHMI presentation and behaviour independently of
any programming toolkit
Model-Based AHMI Design
10 April 2023 HMAT Workshop, June 2010 Page 9
X3D OpenGL
• Automated evaluation of• Static aspects (UI layout, position of objects)• Dynamic concepts (state of a button during the interaction, color of
the label).
• Any UI is represented in a UsiXML model submitted to automated evaluation (automatic or manual)
• Usability guidelines over the UI objects (distribution of the widgets composing the UI) are evaluated.
• Special attention was paid to those guidelines for standard certification and quality assurance and to express them in the Guideline Definition Language (GDL)
Evaluating the AHMI User Interface
10 April 2023 HMAT Workshop, June 2010 Page 10
http://www.usixml.orghttp://www.w3.org/2005/Incubator/model-based-ui/
Integrating UI Evaluation in a Simulation Environment
10 April 2023 HMAT Workshop, June 2010 Page 11
UI timeline
10 April 2023 HMAT Workshop, June 2010 Page 12
• Evaluating the AHMI User Interface
• Execution Time
• Workload
• Guidelines
Outline
10 April 2023 Page 13
Evaluation-Execution time
10 April 2023 Page 15
Evaluation-Workload
10 April 2023 Page 16
Evaluation-Execution Workload
1.2 + 3x(1.5 + 0.075) + 1.5 = 7.4254.6 + 4x(4.6 + 2.2) = 31.8
WorkloadRemember dateSelect daySelect monthSelect year
Execution Time Select daySelect monthSelect yearValidate
(Use mouse to point at object on screen1.5 second)+Execute a mental “step”.075 secondRetrieve a simple item from a long-term memory1.2 second
4.6Evaluation/Judgment (consider single aspect) + 2.2Discrete Actuation (button, toggle, trigger)
Compilation
UsiXML Evaluation
Evaluation report
Formal rules file
Original ruleUsability guidelines
Smith & MosierRule interpretation
WCAG 2.0
Natural language
Formal language
…
A Rule
A Header
Contrainte A
Variable AVariable AA Variable
Contrainte AA constraint
Formal des-cription of A
A Widget
A Quantifier
UA: evaluation process
Compiled evaluationrules
Compilation
UsiXML Evaluation
Evaluation report
Formal rules fileSource file
Lexical analysis
Syntactic tree
Sequence ofLexical units
SyntacticAnalysis
Header:1, “White and yellow should not be used together for text“,high,partial, "Legibility and perception",must,enabled
rule:no inputtext i: i has_properties(bgcolor == "#ffffff") and i has_properties(fgcolor == "#ffff00")
UA: evaluation process
Compiled evaluationrules
Compilation
UsiXML Evaluation
Evaluation report
Formal rules fileCréation d’algorithmes testant le contenu de l’information des “GraphicalWidget”InputText AInputText A
InputText BInputText B
InputText C
UA: evaluation process
Compiled evaluationrules
Compilation
UsiXML Evaluation
Evaluation report
Formal rules fileInputText C
UA: evaluation process
Compiled evaluationrules
Some guidelines
• Cockpit display systems should at least be consistent with systems of our daily life [Singer 2002]
• Usability guidelines from ISO 9126 could be evaluated• Messages should follow always the nomenclature: first letter in
capital and the rest in lower case• AHMI display systems such as the consistency in the roll index in
the compass rose [Singer 2001]
10 April 2023 HMAT Workshop, June 2010 Page 21
• A model-based method for the development of the AHMI was presented allowing:• Structured development process (making design more explicit)• UI evaluation
• Traditional measurements can be assessed like UI workload and execution time,
• More complex automated evaluation based on guideines.
• Explore design options, for instance, modality of interaction of the UI can be object of evaluation.
• The original 2D rendering can be equally rendered in 3D.
• A future plan is to automatically generate the AHMI from its model and to submit it to run-time analysis.
Conclusion
10 April 2023 HMAT Workshop, June 2010 Page 22
10 April 2023 HMAT Workshop, June 2010 Page 23