DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin
33
July 21th, 200 9 Marco Blumendorf I Marco.Blumendorf@dai- labor.de Towards a Model-Based Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin Marco Blumendorf and Sahin Albayrak
description
Towards a Model-Based Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments. Marco Blumendorf and Sahin Albayrak. DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin. Agenda. Motivation - PowerPoint PPT Presentation
Transcript of DAI-Labor Fakultät IV – Elektrotechnik und Informatik Technische Universität Berlin
July 21th, 2009 Marco Blumendorf I [email protected]
Towards a Model-Based Framework for the Development of Adaptive Multimodal User
Interfaces for Ambient Assisted Living Environments
DAI-LaborFakultät IV – Elektrotechnik und InformatikTechnische Universität Berlin
Marco Blumendorf and Sahin Albayrak
HCI International 2009 2July 21th, 2009
Agenda
• Motivation
• A Runtime Architecture for Adaptive Multimodal User Interfaces
• User Interface Models
• The Multi Access Service Platform
• Conclusion
HCI International 2009 3July 21th, 2009
Ambient Assisted Living Environments
are
Smart Environments
and reflect
Ubiquitous Computing Paradigms
HCI International 2009 4July 21th, 2009
“Ubiquitous Computing”(Mark Weiser, 1991)
“In the 21st century the technology revolution will move into the everyday,
the small and the invisible.”
HCI International 2009 5July 21th, 2009
Smart Objects Smart Devices
Smart NetworksSmart Environments
HCI International 2009 6July 21th, 2009
Interaction in Smart Environments
used interaction modalities
HCI International 2009 7July 21th, 2009
Challenges
• Complex, distributed system to interact with• Different capabilities and skills of users• Personalization of UIs• Context adaptivity• Continuous, flexible and coherent interaction• Excellent interaction experience
Lack of development tools, methodologies and runtime support
HCI International 2009 8July 21th, 2009
Agenda
• Motivation
• A Runtime Architecture for Adaptive Multimodal User Interfaces
• User Interface Models
• The Multi Access Service Platform
• Conclusion
HCI International 2009 9July 21th, 2009
Requirements
• Connect interaction resources
• Integrate context
• Host the applications
• Support UI features(Multimodality, Distribution, Adaptation, …)
HCI International 2009 10July 21th, 2009
Connecting Interaction Resources
HCI International 2009 11July 21th, 2009
Integrating Context
Sensors
HCI International 2009 12July 21th, 2009
Application Hosting
Sensors
HCI International 2009 13July 21th, 2009
Model State
Sensors
HCI International 2009 14July 21th, 2009
Monitoring State
Sensors
Monitoring
HCI International 2009 15July 21th, 2009
Stimulating Model Execution
Sensors
Stimulation
HCI International 2009 16July 21th, 2009
Processing Components
Sensors
HCI International 2009 17July 21th, 2009
Agenda
• Motivation
• A Runtime Architecture for Adaptive Multimodal User Interfaces
• User Interface Models
• The Multi Access Service Platform
• Conclusion
HCI International 2009 18July 21th, 2009
Mediating between human and computer
10011010101111100100
“turn the light on”
HCI International 2009 19July 21th, 2009
Mediating between human and computer
10011010101111100100 ?
“turn the light on”
HCI International 2009 20July 21th, 2009
Mediating between human and computer
10011010101111100100
“turn the light on”
HCI International 2009 21July 21th, 2009
Abstraction Layers
See also Szekely 1996, Cameleon Reference Framework (Calvary et al. 2003)
Similar concepts can be found e.g. in UsiXML (Limbourg et al. 2004)and TERESA (Berti et al. 2004)
HCI International 2009 22July 21th, 2009
Mediation Process
See also Szekely 1996, Cameleon Reference Framework (Calvary et al. 2003)
Similar concepts can be found e.g. in UsiXML (Limbourg et al. 2004)and TERESA (Berti et al. 2004)
HCI International 2009 23July 21th, 2009
DefinitionElement
SituationElement
ExecutionElement
Adding State and Execution Logic
Dynamic Executable Model
See e.g. Breton & Bézivin 2001 for details
• Definition Elements - static structure and constant elements
• Situation Elements - state of the model and changing elements
• Execution Elements - interpretation process altering state (and definition) at runtime
HCI International 2009 24July 21th, 2009
Adding State and Execution Logic
DefinitionElement
SituationElement
ExecutionElement
DefinitionElement
SituationElement
ExecutionElement
DefinitionElement
SituationElement
ExecutionElement
DefinitionElement
SituationElement
ExecutionElement
DefinitionElement
SituationElement
ExecutionElement
HCI International 2009 25July 21th, 2009
Agenda
• Motivation
• A Runtime Architecture for Adaptive Multimodal User Interfaces
• User Interface Models
• The Multi Access Service Platform
• Conclusion
HCI International 2009 26July 21th, 2009
Implementation
The Multi-Access Service Platform (MASP) implements the presented concepts
Features• Multimodal interaction• Dynamic distribution• Adaptation
Different applications have been implemented.
HCI International 2009 27July 21th, 2009
A Meta UI for Smart Environments
HCI International 2009 28July 21th, 2009
The Multimodal Cooking Assistant
HCI International 2009 29July 21th, 2009
Agenda
• Motivation
• A Runtime Architecture for Adaptive Multimodal User Interfaces
• User Interface Models
• The Multi Access Service Platform
• Conclusion
HCI International 2009 30July 21th, 2009
Results
• We can build models of multimodal applications that dynamically change the used modalities.
• Applications can be distributed and dynamically configured at runtime.
• A follow me mode shows automatic reconfiguration based on context information.
• Dynamic layout changes according to the distance of the user to the screen have been realized.
HCI International 2009 31July 21th, 2009
Open issues and future work
• Technology and software engineering centric
• Next Step: Study how to deploy the features properly to provide integration into environment and the required excellent user experience!
• Additional Aspects:
– multi-user and multi-application scenarios have not been adequately addressed yet
– tools also covering runtime issues and moving towards end-user development and customization
– integration of natural language processing and models-based development
HCI International 2009 32July 21th, 2009
References• [Berti 2004] Silvia Berti, Francesco Correani, Giulio Mori, Fabio
Paternò, and Carmen Santoro. Teresa: A transformation-based environment for designing and developing multi-device interfaces. In ACM CHI 2004, volume II, pages 793–794, 2004. ACM Press.
• [Breton&Bézivin 2001] Erwan Breton and Jean Bézivin. Towards an understanding of model executability. In FOIS '01: Proceedings of the international conference on Formal Ontology in Information Systems, pages 70–80, New York, NY, USA, 2001. ACM.
• [Calvary 2003] Gaëlle Calvary, Joëlle Coutaz, David Thevenin, Quentin Limbourg, Laurent Bouillon, and Jean Vanderdonckt. A unifying reference framework for multi-target user interfaces. Interacting with Computers, 15(3):289–308, 2003.
• [Limbourg 2004] Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, and Víctor López-Jaquero. Usixml: A language supporting multi-path development of user interfaces. In EHCI/DS-VIS, volume 3425 of Lecture Notes in Computer Science, p. 200–220. Springer, 2004.
• [Szekely 1996] Pedro A. Szekely. Retrospective and challenges for model-based interface development. In DSV-IS, pages 1–27. Springer, 1996.
• [Weiser 1991] Mark Weiser, The computer for the 21st century, Scientific American, Morgan Kaufmann Publishers Inc., 1991, 265, 66-75
HCI International 2009 33July 21th, 2009
The End …
Thank you for your attention!
Your questions please …
[email protected]://masp.dai-labor.de
Please visit our booth at the break!
Dynamic Executable Model
DefinitionElement
SituationElement
ExecutionElement
