F. Paternò et al. (Eds.): AmI 2012, LNCS 7683, pp. 328–337, 2012. © Springer-Verlag Berlin Heidelberg 2012

Towards Accessibility in Ambient Intelligence Environments

George Margetis1, Margherita Antona1, Stavroula Ntoa1, and Constantine Stephanidis1,2

1 Foundation for Research and Technology – Hellas (FORTH) Institute of Computer Science

N. Plastira 100, Vassilika Vouton, GR-700 13 Heraklion, Crete, Greece

{gmarget,antona,stant,cs}@ics.forth.gr 2 University of Crete, Department of Computer Science

Abstract. This paper aims to set a landscape and elaborate a roadmap for accessible user interaction in AmI environments, including and beyond personal computational devices (PCs, mobiles, etc.), by identifying and addressing the new needs that emerge in the above context. This line of work is currently being pursued in the context of the AmI Research Programme of ICS-FORTH, and will be implemented in the new AmI Research Facility, which provides an ideal environment for developing the proposed solutions and a test-bed for validating them in a realistic simulation environment.

Keywords: ambient intelligence, accessibility, universal access, assistive technologies, personalized interaction, user model.

1 Introduction

In the years ahead, the potential of Ambient Intelligence (AmI) environments to ad-dress older and disabled people’s everyday life needs is expected to have a radical impact on independent living and e-Inclusion. Many applications and services are al-ready becoming available, for example in the domain of Ambient Assisted Living (AAL), which address a wide variety of issues critical for older and disabled people and are targeted to make possible and enjoyable a more independent, active and healthy life. The European strategy in ICT for Ageing Well of 2010 [1] identifies a number of ICT solutions addressing daily and independent living in areas such as so-cial communication, daily shopping, travel, social life, public services, safety, remind-ers, telecare and telemedicine, personal health systems, and support for people with cognitive problems and their carers. The same report also points out the importance of user interaction for ICT solutions, and mentions user-friendly interfaces for all sorts of equipment in the home and outside, taking into account that many older people have impairments in vision, hearing, mobility or dexterity. Clearly, the benefits of AmI en-vironments can only be fully achieved and accepted by their target end-users if such technologies can demonstrably be developed in such a way as to guarantee inclusive accessibility for a wide variety of functional limitations brought about by age or disabilities.

Towards Accessibility in Ambient Intelligence Environments 329

Accessibility in the context of individual applications and services has been de-fined as follows: for each task a user has to accomplish through an interactive system, and taking into account specific functional limitations and abilities, as well as other relevant contextual factors, there is a sequence of input and output actions which leads to successful task accomplishment [2]. However, the accessibility of AmI envi-ronments poses different problems and is more complex than currently available ap-proaches to the accessibility of desktop or web applications and services, as AmI environments do not simply introduce a new technology, but an integrated set of technologies. Different levels of accessibility may be distinguished. A first level con-cerns accessibility of individual devices. Interactive devices need to be accessible to their owners according to their needs, but basic accessibility should also be provided for other users with potentially different needs. A second level concerns the accessi-bility of the environment as a whole, intended as equivalent access to content and functions for users with diverse characteristics, not necessarily through the same de-vices, but through a set of dynamic interaction options integrated in the environment.

It is likely that some of the built-in features of AmI environments, such as multi-modality, will facilitate the provision of solutions that will be accessible by design ([3], [4]). For example, blind users will benefit from the wider availability of voice input and output. Different modalities can be used concurrently, so as to increase the quantity of information made available or present the same information in different contexts, or redundantly, to address different interaction channels, both to reinforce a particular piece of information or to cater for the different abilities of users. A novel aspect is that in AmI environments, the accessibility of the physical and of the virtual world need to be combined. For example, for blind, visually impaired and motor-impaired users, requirements related to interaction need to be combined with require-ments related to physical navigation in the interactive environment.

Although several interaction technologies, such as, for example voice output, are already widely available, and other, such as, for example, eye-tracking, are reaching a maturity stage where they can be robustly exploited for accessibility purposes, a num-ber of fundamental obstacles still hinders the provision of alternative and personalized accessibility solutions in AmI environments. These include:

• Limited knowledge of user requirements and of the appropriateness of different solutions for different combinations of user characteristics / functional limitations and environment characteristics / functions (for example, age-related factors).

• Lack of ready-to-use accessibility solutions supporting alternative interaction techniques for various combinations of user abilities / functional limitations. Even when optimal combinations of interaction devices and techniques are known, embedding them in AmI environments is far from easy, since most exist-ing assistive technologies are limited in use to specific devices, and cannot be easily made compatible with complex environments including various devices.

• Lack of architectural frameworks, taking into account the need for practical ac-cessibility solutions and supporting their integration and management.

• Lack of tools supporting various phases of the development lifecycle of accessi-ble smart environments (e.g., requirements analysis, design and prototyping, evaluation, content creation).

330 G. Margetis et al.

As a result, developing truly accessible AmI environments is currently very expensive in terms of time, efforts, costs and required knowledge, and the results are often of limited flexibility and reusability in terms of accessibility solutions and addressed target user groups.

This paper aims to set a landscape and elaborate a roadmap towards Universally Accessible AmI environments, providing on one hand a user centered but also con-text-aware methodology for enabling accessibility in such environments and on the other hand modern tools and personalized assistive solutions that will constitute the building blocks for the development of independent living AmI environments ad-dressing interaction needs of older and disabled persons.

This line of work is currently being pursued in the context of the AmI Research Programme of ICS-FORTH, and will be implemented in the new AmI Research Facil-ity, which provides an ideal environment for developing the proposed solutions and a test-bed for validating them in a realistic simulation environment.

2 Related Work

Accessibility in the context of AmI environments is usually intended as inclusive mainstream product design (e.g., [5]), although a contextual definition is not yet available. AmI environments are expected to have profound consequences on the type, content and functionality of the emerging products and services, as well as on the way people will interact with them, bringing about multiple new requirements (e.g., [6], [7], [8]).

The issue of accessible interaction in AmI environments has been mainly explored so far through scenarios [9], while the impact of AmI environments on users with activity limitations has been studied in [10]. In [11] an overview is provided of EC-funded projects which include some concepts related to accessibility in AmI environ-ments, although mainly focusing on services and applications set up in order to sup-port people (including people with activity limitations). In [12], an AmI home care approach is presented aiming to address elderly and disabled persons’ needs at home, mainly through smart users’ monitoring in order for the proposed system to be able to notify on time their carers or doctors. An example of the AmI potential on helping people with particular needs in their everyday living is presented in [13]. Its major goal is the training of elderly people in order to handle modern interfaces for Assisted Living and evaluate the usability and suitability of these interfaces. A more systematic approach towards interactive personalization of ambient assisted living environments is presented in [14]. In more detail, a framework is presented providing interactive configuration of comprehensive AAL Environments at the level of authoring tools, focusing on the application of AAL at home.

Despite the aforementioned efforts towards supporting accessibility in AmI envi-ronments, a systematic and comprehensive approach to inclusive accessibility in AmI environments is still needed. This paper proposes some basic steps towards a radical new dimension of accessible user interaction in AmI environments for older and disabled inhabitants. The main idea behind the proposed roadmap is to make easy

Towards Accessibility in Ambient Intelligence Environments 331

available, usable and “pluggable” in the intelligent environment mature and emerging assistive technology solutions, as well as alternative multimodal interaction tech-niques, applying the appropriate input and output modalities according to the user, the task at hand, and the current context of use.

3 The AmI Research Facility

ICS-FORTH is creating a state-of-the-art AmI Research Facility, targeted to support research, experimentation and multidisciplinary scientific collaboration. Such a facili-ty is intended, amongst other things, to support the establishment and conduct of a line of research targeted towards the provision of accessibility in AmI technologies and environments. The Facility occupies a three-floor 3,000 square meters building (see Fig. 1), and comprises simulated AmI environments and support spaces, includ-ing as a house, a class-room, a collaborative workplace, a multifunctional doctor’s office, and a multipurpose “exhibition” space.

Fig. 1. ICS-FORTH AmI Facility

The entire building has been designed to be accessible by people with disabilities, and follows DfA guidelines concerning stairs, elevators, ramps, corridors width, ac-cessible bath-room facilities, multimodal signs and labels, etc. In particular, the house simulator will constitute a prototype accessible house for disabled and elderly people. Additionally, the building design takes into account issues of easy orientation and navigation in the physical environment. Designed to accommodate and simulate eve-ryday life environments and to provide the necessary flexibility for testing news tech-nologies, the AmI Facility will constitute a real test bed for accessible AmI spaces.

4 Indicative Scenario

The following is an indicative scenario which illustrates some basic requirements for accessible AmI environments.

Mary is a middle age blind teacher. She has just moved to her new AmI home where she lives alone. Today, she has a day off and she decides to spend most of her

332 G. Margetis et al.

time listening to her favorite music. She moves to the smart living room using the “magic wand”, a plastic stick which can control the environment’s devices. Using the wand, she starts pointing to diverse directions. Every time she points to an artifact of the smart living room, audio feedback explains which is the artifact she pointed at, and what is its state.

After a few tries, she finds out the direction to the living room couch and starts moving towards it. When she approaches the couch, the smart living room informs her how close she is and if there is an obstacle in her way. It is important to note that the wand can provide similar information for every smart environment Mary may visit, thus making unfamiliar environments easier for her to explore.

Mary reaches the couch, sits down and then uses the “magic wand” again to find the direction of the smart audio system in order to listen to her favorite music. As soon as she finds it, she starts using the set of four buttons on the wand, with which she is able to control any device in the room in a seamless way. Using the “universal control” functionality of the “magic wand” she can interact with the display to choose the music category she wishes to listen to, as well as specific songs. The system con-stantly provides audio feedback for the available options and the interaction carried out. After selecting the list of songs she likes to listen to, she chooses the play function and sits back to the couch, relaxing and enjoying the music.

5 Roadmap Towards Accessible AmI Environments

In the context of the scenario outlined above, several challenges need to be addressed in order to elaborate a systematic approach to accessibility in AmI environments:

• Advancing knowledge of user requirements and of the appropriateness of differ-ent solutions for different combinations of user characteristics / functional limita-tions and environment characteristics / functions, and creating related ontological models

• Developing a reference architectural model that will address user needs for inclusive design in AmI environments, allowing for accessible multi – modal interaction

• Providing ready-to-use accessibility solutions supporting alternative interaction techniques for various combinations of user abilities / functional limitations

• Developing a design tool supporting the implementation of accessible AmI environments

• Developing AmI accessible applications in three fundamental everyday life domain, namely home, work and self-care.

• Evaluating the developed assistive solutions tools and applications in order to assess their accessibility, usability and added value for the target users.

The above challenges are further analyzed in the next subsections.

Towards Accessibility in Ambient Intelligence Environments 333

5.1 Ontology-Based User and Interaction Modeling

An extensible context ontology [15] for user and interaction techniques modeling in Ambient Intelligence environments is necessary in order to effectively address the needs for context-aware personalization in AmI environments. To this end, building on existing knowledge and best practices, a generic ontology framework for personalized assistive solution modeling needs to support the interconnection of heterogeneous domain ontologies, integrate single services using ontological layering and provide contents-aware update and maintenance mechanisms. This framework is intended to provide a simplified and highly abstract model of ontology which is independent of a specific ontology representation language and operates with ontologies on a conceptual rather than syntactic level, thus supporting compatibility and interoperability with on-tologies described in diverse language forms, and ensuring significant advantages in simplicity of software development.

Fig. 2. User and interaction techniques model’s implementation steps

Fig. 2 illustrates the steps to be followed in order to implement the ontology based framework. In more details, requirements elicitation for accessibility interaction tech-niques, analysis of end-users’ experience on AmI environments and use cases and application development will be conducted. The consolidate results of these activities will constitute the input for the design and implementation of the aforementioned user and interaction techniques model.

5.2 Ready-to-Use Accessibility Solutions

In the context of accessible AmI environments, the assistive technologies that are currently used to make interactive applications and services accessible to disabled users will continue to constitute the basis for users’ interaction, but they will also be extended and enriched in terms of functionality and use in order to provide multimod-al and personalized accessible interaction beyond conventional ICT environments.

334 G. Margetis et al.

Fig. 3 illustrates the main categories of assistive technologies and interaction tech-niques that constitute the state of the art today and offer significant potential of pro-viding accessibility means in AmI environments.

Fig. 3. Assistive technology categories

Assistive solutions and accessibility technologies have so far supported the aug-mentation of the capabilities of the individuals and the adaptation of single artifacts for accessibility. In the context of AmI, it is necessary to build on current state of the art in the field and further research new interaction methods as they emerge, taking advantage of built-in features in these living spaces, such as multimodality. A number of personalized cross – domain AmI assistive solutions that will address the accessi-bility and usability needs of older and disabled users can be designed and developed, building upon state-of-the-art solutions in user interface visual adaptation, voice-based interaction, scanning-based interaction, touch interaction, haptic feedback, handwriting, and eye tracking technologies (see Fig. 3). Different modalities can be provided based on user's preferences and needs according to the ontology model.

Personalized assistive solutions for AmI environments are necessary to provide al-ternative input, output and information rendering modalities capable of addressing each individual's needs (see Fig. 4). An innovative aspect relates to the interaction and co-operation between different / changing applications and devices used in each environ-ment. Furthermore, such an approach will provide the building blocks for the harmo-nized interoperability of assistive technologies and conventional computational devices (PCs, smart phones, etc.) with innovative high end technology artifacts in the context of AmI accessible environments. The developed assistive solutions will be available through an open architecture that will allow their interoperation over different applica-tion domains, enabling use through a combination of personal and ambient devices.

Towards Accessibility in Ambient Intelligence Environments 335

Fig. 4. AmI Personalized Assistive Solution concept

5.3 Accessible AmI Applications

The provision of accessibility solutions addressing the interaction needs of older users and people with activity limitations will foster the adoption of effective approaches for the design of AmI accessible environments’ applications. Innovative multi-modal application concepts, using personalized AmI assistive solutions, will be investigated in three domains encompassing fundamental activities of daily life, namely home, work and self-care. Through the requirements analysis and the assistive solutions and infrastructure that will be developed, new AmI accessible interactive applications will be proposed addressing everyday needs of older people and people with activity limitations.

Fig. 5. AmI home navigation and control for blind people

The adopted approach will move beyond AmI – home / office / self-care technolo-gies to integrating assistive solutions into things of everyday objects.

One of the applications under development is the implementation of the “magic wand” [16] and “universal control” described in the scenario of section 4. As depicted in Fig. 5 (a), a laboratory AmI living room has already been set-up able to monitor and give auditory feedback about the location and state of the room’s objects to blind inhabitants pointing with the plastic stick of Fig. 5 (b). The remote mounted on the stick enables to control the room’s devices and applications in a seamless way.

336 G. Margetis et al.

5.4 Deployment and Pilot Evaluation

The evaluation of AmI technologies and environments will need to go beyond tradi-tional usability evaluation in a number of dimensions, concerning assessment methods and tools as well as metrics. AmI technologies and systems challenge traditional usa-bility evaluation methods, because the context of use can be difficult to recreate in a laboratory setting. This suggests that the evaluation of user’s experience with AmI technologies should take place in real world contexts. However, evaluation in real settings also presents difficulties, as there are limited possibilities of continuously monitoring users and their activities. In this respect, the AmI Facility will offer an ideal experimental environment, combining user experience in context with the avail-ability of the necessary technical infrastructure for studying the users’ behavior over extended periods of time. The current deployment and evaluation plans include accessible AmI applications in the home, work, learning and health-care domains.

6 Conclusions

This paper has discussed a landscape and roadmap to enable accessible user interaction in AmI environments for people with disabilities and older inhabitants in a systematic way. To achieve these objectives a number of implementation steps are proposed in-cluding: the development of a reference architectural model, which will address user needs for inclusive design in AmI environments and the implementation of AmI perso-nalized assistive solutions, which will enable accessible user interaction in AmI envi-ronments. Moreover, design tools will be implemented in order to enable developers of AmI environments to effectively design accessible everyday life applications, while a number of test AmI accessible applications will be designed and developed aiming to enable the users to interact in the AmI environment using various conventional devices and AmI artifacts available in the environment, according to their needs and require-ments. The adopted approach goes in the direction of catering for accessibility at design time in a systematic fashion, as due to their complexity AmI environments can-not be made accessible ‘a posteriori’ after development.

Finally, evaluations of the developed assistive solutions tools and applications will be carried out in interconnected simulation environments, in order to assess their ac-cessibility and added value for the target users.

References

1. Overview of the European strategy in ICT for Ageing Well, http://ec.europa.eu/ information_society/activities/einclusion/docs/ageing/overview.pdf

2. Savidis, A., Stephanidis, C.: Unified User Interface Design: Designing Universally Acces-sible Interactions. Int. J. Interact. Comput. 16(2), 243–270 (2004)

3. Carbonell, N.: Ambient multimodality: towards advancing computer accessibility and as-sisted living. Univ. Access. Inf. Soc. 5, 96–104 (2006)

Towards Accessibility in Ambient Intelligence Environments 337

4. Richter, K., Hellenschmidt, M.: Position paper: Interacting with the ambience: Multimodal interaction and ambient intelligence. In: W3C Workshop on Multi-modal Interaction, Sophia Antipolis, France, July 19/20 (2004)

5. Kemppainen, E., Abascal, J., Allen, B., Delaitre, S., Giovannini, C., Soede, M.: Ethical and legislative issues with regard to Ambient Intelligence. In: Roe, P.R.W. (ed.) Impact and Wider Potential of Information and Communication Technologies, COST, Brussels, pp. 188–205 (2007)

6. Han, J.Y.: Multi-touch interaction wall. In: ACM SIGGRAPH 2006 Emerging technolo-gies (SIGGRAPH 2006). ACM, New York (2006)

7. Coroama, V., Bohn, J., Mattern, F.: Living in a Smart Environment – Implications for the Coming Ubiquitous Information Society. In: Proceedings of the International Conference on Systems, Man and Cybernetics 2004 (IEEE SMC 2004), vol. 6, pp. 5633–5638 (2004)

8. Edwards, W.K., Grinter, R.E.: At Home with Ubiquitous Computing: Seven Challenges. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) UbiComp 2001. LNCS, vol. 2201, pp. 256–272. Springer, Heidelberg (2001)

9. Antona, M., Burzagli, L., Emiliani, P.-L., Stephanidis, C.: The ISTAG scenarios: a case study. In: Roe, P.R.W. (ed.) Towards an Inclusive Future: Impact and Wider Potential of Information and Communication Technologies, Ambient Intelligence and implications for people with disabilities, ch.4, sec. 4.1, pp. 158–187. COST219ter, Brussels (2007)

10. Emiliani, P.-L., Burzagli, L., Billi, M., Gabbanini, F., Palchetti, E.: Report on the impact of technological developments on eAccessibility. DfA@eInclusion Deliverable D2.1. (2009), http://www.dfaei.org/deliverables/D2.1.pdf

11. Emiliani, P.-L., Aalykke, S., Antona, M., Burzagli, L., Gabbanini, F., Klironomos, I.: Document on necessary research activities related to DfA. DfA@eInclusion Deliverable D2.6 (2009), http://www.dfaei.org/deliverables/D2.6.pdf

12. Nehmer, J., Becker, M., Karshmer, A., Lamm, R.: Living assistance systems: an ambient intelligence approach. In: Proceedings of the 28th International Conference on Software Engineering (ICSE 2006), pp. 43–50. ACM, New York (2006)

13. Kleinberger, T., Becker, M., Ras, E., Holzinger, A., Müller, P.: Ambient Intelligence in Assisted Living: Enable Elderly People to Handle Future Interfaces. In: Stephanidis, C. (ed.) UAHCI 2007 (Part II). LNCS, vol. 4555, pp. 103–112. Springer, Heidelberg (2007)

14. Marinc, A., Stocklöw, C., Braun, A., Limberger, C., Hofmann, C., Kuijper, A.: Interactive Personalization of Ambient Assisted Living Environments. In: Smith, M.J., Salvendy, G. (eds.) HCII 2011, Part I. LNCS, vol. 6771, pp. 567–576. Springer, Heidelberg (2011)

15. Preuveneers, D., Van den Bergh, J., Wagelaar, D., Georges, A., Rigole, P., Clerckx, T., Berbers, Y., Coninx, K., Jonckers, V., De Bosschere, K.: Towards an Extensible Context Ontology for Ambient Intelligence. In: Markopoulos, P., Eggen, B., Aarts, E., Crowley, J.L. (eds.) EUSAI 2004. LNCS, vol. 3295, pp. 148–159. Springer, Heidelberg (2004)

16. Zabulis, X., Koutlemanis, P., Baltzakis, H., Grammenos, D.: Multiview 3D Pose Estima-tion of a Wand for Human-Computer Interaction. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Wang, S., Kyungnam, K., Benes, B., Moreland, K., Borst, C., DiVerdi, S., Yi-Jen, C., Ming, J. (eds.) ISVC 2011, Part II. LNCS, vol. 6939, pp. 104–115. Springer, Heidelberg (2011)