Design of interactive technologies for a

mobile and ubiquitous worldNikolaos Avouris

University of Patras Human-Computer Interaction Grouphttp://hci.ece.upatras.gr

Invited talk, Athens 29 May 2010

Human-Computer InteractionGroupUniversity of Patras, Greece

Mobile and ubiquitous computing Ubiquitous computing is a post-desktop model of human-computer interaction in which information processing has been integrated into everyday objects and activities.– Reliance on context / physical environment– New kinds of applications (leisure related,

everyday activities)– Many variations, flavours, technologies– Focus on devices or specific services

How to design such applications?3

Ubiquitous computing application domains

4

Ubiquitous Computing Architecture

Spaces Brokers

ProvidersDirectory

Directory services

Service Broker

Service Provider 1Service

Provider i

Digital Information Providers

User Profiles User

Agency

Profile Management

Local Service Provider j

Semantics of space

Space information

systemHistorical user interaction data

and sensory data

Space Owner System

Dynamic Service Binding

Physical space

Physical hyperlink 5

Ubiquitous Computing applications examples

6

Museum Game: Inheritance (1/2)

• Collaborative game that requires the players to discover a special exhibit in a museum based on hints that must be discovered and collected from the descriptions of exhibits.

• Direct interaction with physical artifacts- that become augmented artefacts that are easier to perceive using the mobile device and physical hyperlinks

A. Stoica, G. Fiotakis, D. Raptis, I. Papadimitriou, V. Komis, N. Avouris (2007), Field evaluation of collaborative mobile applications, chapter LVIX in J. Lumsden (ed.), “Handbook of Research on User Interface Design and Evaluation for Mobile Technology”, pp. 994-1011, Hershey, PA, IGI Global

Papadimitriou I., Komis V., Tselios N., Avouris N., (2006) Designing PDA Mediated Educational Activities for a Museum Visit, Proceedings of Cognition and Exploratory Learning in Digital Age (CELDA 2006), December 2006, Barcelona, Spain.(http://www.iadis.org/celda2006/)

7

Museum Game: Inheritance (2/2)

8

Mobile library assistant (1/2)

• Mobile application to assist library visitors:– retrieving information about the books by

directly interacting with them– Searching books similar to those collected

by direct interaction– Multiple modalities of interaction with the

physical artifacts have been chosen

9

Mobile library assistant (2/2)

10

Museum Guide (1)• Investigate multiple modalities for delivering

information (text, images, audio)• Implement legacy connectors to leverage existing

information.• Build user profiles• Physical space navigation support• Content authoring support

• C. Sintoris, D. Raptis, A. Stoica and N. Avouris, (2007) Delivering Multimedia Content in Enabled Cultural Spaces,Proceedings of 3rd International Mobile Multimedia Communications Conference, Mobimedia 2007 , August 27-29, 2007, Nafpaktos, Greece, ACM press.

11

Museum Guide (2)

12

Game: MuseumScrabble (1)

• A team game involving linking exhibits and connecting them to topics covered by the museum

• Deepen the understanding of interaction with digitally augmented physical artefacts

• Yiannoutsou N., Papadimitriou I., Komis V. and Avouris N., “Playing with” museum exhibits: designing educational games mediated by mobile technology, Proc. of IDC 2009, 8th International Conference of Interaction Design and Children, June 3–5, 2009, Como, Italy, pp. 230-233. ACM Press, New York, NY

• Sintoris C., Stoica A., Papadimitriou I., Yiannoutsou N., Komis V., Avouris N. (2010). MuseumScrabble: Design of a mobile game for children's interaction with a digitally augmented cultural space, International Journal of Mobile Human Computer Interaction, 2(2), 53-71, April-June 2010

13

MuseumScrabble (2)

14

BeNatural (1)

• Personalized mobile shopping assistant.• Exploration of a new space with different

dynamics.• Investigation of interaction with a different

physical artefacts (items bought in a supermarket)

15

The beNatural architecture (2)

16

Playful Narrative: Who killed Hanae(Paay et al. 2008)

• Location based narrative. Goal: find the killer• Navigation in Aalborg city through clue collection

and “suspect interrogation”• Chapters of a mystery story are “attached” to

different city areas

images from Paay et al., 2008

Street activity Screenshot from the mobile device17

City Game: Frequency 1550 (Huizenga et al. 2009)

• Location based game. Goal: earn citizenship (366 days in Amsterdam)

• Navigation in the historical centre of Amsterdam• Assignment undertaking in specific areas of the

city

images from http://freq1550.waag.org/

the Headquarter view during game playing timeLocation activity18

Ubiquitous Computing design and evaluation

19

Abstract design frameworks: An example Design Principles of Playful Narratives (Yiannoutsou et al. 2010)

Narrative

• Role of user • Structure of story

Space

• setting • motion Interaction

• with the technology • with the story • with the physical space • social interactions

Playful character

Learning

• thinking • decision making • factual knowledge

building • procedural knowledge

building

• Rules of game • Engagement • Joy

20

Overview of mobile applications design & evaluation practice ( Avouris et al. 2008) Source System Number of

Participants Evaluation

Method Metrics Used

Jokela et al. (2008)

Mobile Multimedia Presentation Editor

24 (lab) 15 (field)

Laboratory evaluation and Field study

Qualitative measures of user behaviour.

Guo et al. (2008)

Nintendo Wiimote and Nunchuk – based controller of a robot

20 Lab based comparative user study

Speed and accuracy in both tasks and user preference through questionnaire

Riegelsberger et al. (2008)

Use of Google Maps in Mobile Devices

24 Field study in four different locations.

Qualitative measures and usability problems found using group briefing sessions, recorded usage, multiple telephone interviews and debriefs in a lab setting

Sanchez et al. (2008)

AudioNature, A pocketPC device for science learning for the blind

10 Case study in lab involving typical users

Qualitative measures of effectiveness and performance through pre and post tests and questionnaires

Bellotti et al. (2008)

Leisure guide Magitti

11 Field study over a period of several days

Qualitative measures of user experience recorded through questionnaires

21

Issues to be taken into accountsee Framework of (de Sa et al. 2008)

• (1) Locations and Settings: Lighting, Noise, Weather, Obstacles, Social Environment.

• (2) Movement and Posture: variations for sitting, standing and walking

• (3) Workloads, Distractions and Activities: Critical activities, settings or domains requiring different degrees of attention, Cognitive distractions (e.g., phone ringing, etc), to study cognitive recovery, Physical distractions

• (4) Devices and Usages: Single vs Dual handed interaction, and Stylus/Finger/Keyboard/Numeric Pad different mobile devices (e.g., PDAs, Smart Phones, Portable Media Players, etc).

• (5) Users and Personas: movement and visual impairment, Heterogeneity – Age, dexterity, cultural background, profession etc. 22

Adaptation of existing measures of user performance

• Task Load Index (TLX) extended for mobile applications, in particular for in car tasks, by addition of the Distraction scale (Cook 2006)

• Measures related to Field Experimentation method (Goodman et al. 2004), e.g. Percentage Preferred Walking Speed (PPWS)

• Measures related to multitasking characteristics (measures of divided attention)

23

New settings for lab evaluation(e.g. Kjeldskov & Stage 2003)

24

New requirements for Content Generation: Create-Attach to Space

25

Different evaluation methods and findings according to the phase of the design process

• early design evaluation (storyboarding, enactment in physical setting)• low fidelity prototype evaluation (a desktop simulation – no spatial aspect)• high fidelity prototype evaluation (field study)

26

Design through enactment (Jacucci, 2004)

27

Storyboard and sketching in design and evaluation (Buxton, 2007)

28

beNatrural Conceptual mobile application design (1st phase)

29

beNatrural Conceptual mobile application design (2nd phase)

30

2 dimensional simulationof space

Mobile device

simulation

Low fidelity prototype evaluation

31

High fidelity prototype evaluation in the field (Inheritance MuseumGame)

32

Multimedia ManagerMultimedia

Repository

context System screen

User’s discussions

Analyst’s comments

User 1

User 2

User 3

User n

33

Field studies: Analysis of user action

Dimensions of Analysis

34

Use of ActivityLens for data analysis (Fiotakis et al. 2007)

35

Discussion

Discussion: On design and evaluation

• .. So far emphasis on user centered interactive systems design.

• .. typically done by using an evaluation method to measure or predict how effective, efficient and/or satisfied people would be when using the system to perform one or more tasks.

• Usability evaluation methods range from laboratory-based user observations, controlled user studies, and/or inspection techniques

37

Some criticism of current practice• Usability Evaluation can be ineffective and

even harmful if naively done ‘by rule’rather than ‘by thought’. If done during early stage design, it can mute creative ideas that do not conform to current norms, especially to new fields like ubiquitous computing

Is usability evaluation considered harmful ? (Greenberg and Buxton, 2008) .

38

Some criticism of current practice

• Current approaches are often assuming formal task structures more related to work, while today's’ systems are more often related to no structured human activities

• need to move to a more design centered approach in evaluation (Cockton 2008)

39

Some criticism of current practice

• Usability should be related to the value a product has for its users, as often usable products are not useful (Cockton 2007, 2008)

• The usability evaluation results should be judged in terms of their downstream utility to designers, (Law et al., 2007, Howarth, et al. 2007)

40

Challenges: the way ahead

l Reseting the relation between evaluation and designl Redefinition of content authoring – interaction

metaphores, evaluation of user experiencel We need to put the activity to the centre of the design

process and not just the userl There is need for an abstract design method for

ubiquitous computing.• We need to support the designer and increase the

objectivity with measures of usability, while at the same time increase the pace of development

41

thanks

for more: hci.ece.upatras.gr

42