Rapid prototyping of ubiquitous interaction

A comprehensive framework for the rapid prototyping of ubiquitous interaction

Candidate: Andrea Bellucci

Supervisor: Dr. Ignacio AedoCo-supervisor: Dr. Alessio Malizia

Doctoral degree in Ciencia y Tecnología InformáticaDepartamento de Informática, Universidad Carlos III de MadridLeganés, October 21st, 2013

REsearch QUestion

REsearch QUestionHow to lighten the prototyping of ubiquitous interactive systems?

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

SynopsisPart I: How things are• Framing• State of the art• Exploration• Open issues

Part II: How THINGS COULD BE• Hypothesis• Objective • Research methodology• The proposal• Evaluation

PART III: CONCLUSIONs• Contributions• Potential for future research

PART I: HOW THINGS ARE• Framing• State of the art• Exploration• Open issues

• Texto

UbiComp: merging the physical and the digitalFraming

Digitally-augmented environments

• Texto

Technology-mediated human-to-human interactionPEOPLE

• Texto

Device EcologiesTEchnology

[...] a massive noncentralized agglomeration of devices, connectivity and electricity means, applications, services, and interfaces.

[The] infrastructure is not homogenous or seamless, but fragmented into several techniques that the user has to study and use.

Oulasvirta, A. (2008). When users “do” the ubicompTHE REAL UBICOMP

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

EXPERIENCE

The exploration has been carried out within three research projects:1. INTEGRA (national, AMPER)2. TIPEx (national, MINECO)3. meSch (european, FP7)

EXPERIENCE (2)Don’t touch me

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.TESIS: Turn Every Surface into an Interactive SurfaceEXPERIENCE (3)

LESSONS LEARNT

1. Difficult knowledge reuse2. High technical expertise

CONTEXTTechnological perspective

OPEN ISSUES

Issue #1. Integration of hetherogeneous devices and interaction modalities

Issue #1

OPEN ISSUES

Issue #2. Lowering the technical expertise

Issue #2

OPEN ISSUES

Issue #1. Integration of hetherogeneous devices and interaction modalities

Issue #2. Lowering the technical expertise

Issue #2

OPEN ISSUES

Issue #1

PART II: HOW THINGS COULD BE• Hypothesis• Objective• Research methodology• The proposal• Evaluation

RESEARCH QUESTION

How to lighten the prototyping of ubiquitous interactive systems?

Encompassing heterogeneous devices into a unique design can reduce users efforts to develop ubiquitous interactive systems

HYPOTHESIS

Issue #1Issue #2

Hypothesis

The definition of a conceptual and software framework that provides users with a comprehensive environment for managing interactions between heterogeneous networked devices...

...to support the rapid prototyping of ubiquitous interactive systems

OBJECTIVE

The six steps of Design and Development ResearchHasan, H. “Information systems development as a research method”, (2003)

METHODOLOGY

PART II: HOW THINGS COULD BE• Hypothesis• Objective • Research methodology• The proposal• Evaluation

Materialization of design ideasRAPID PROTOTYPING

Exploration of the design space

The output is not the artifact, but the feedbacks

Rapid prototyping allows to test: appearance, functionality and user experience

Why does it matter?

Desktop environments have the same I/O capabilities, while UbiComp devices vary in shape, size and I/O design

Usability and UX cannot be evaluated with simulations on a screen

PROTOTYPING UBIQUITOUS INTERACTION

1. Interaction Developerslong time/high ceiling; high eXperience; coding

2. Interaction Designersmedium/rapid operation; medium X; design

3. End-Usersrapid operation; X?; usage

STAKEHOLDERS

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

REQUIREMENTS

Interviews

Literature Survey

Personal Experience

REQUIREMENTS

Interviews

REQUIREMENTS

Interviews

REQUIREMENTS

Interviews

REQUIREMENTS

Interviews

REQUIREMENTS

Interviews

REQUIREMENTS

Six categoriesREQUIREMENTS (2)

1. Input/Output Hardware: Support Heterogeneous I/O; Device Abstraction

2. Interaction Modalities: Support Heterogeneous Interaction Modalities

3. Interactive Space: Spatial Awareness; Multi-display environment

4. Architectural Traits: Distributed Architecture; Agnosticism of Legacy Middlewares

5. Developing/Coding: Low Viscosity; Programming Alternatives; Hide Low-Level Details

6. Application/User Interface: Cross-Device UI; Easy Configuration of I/O Devices

PHYSICAL

DIGITAL

INTERACTION MODEL

PHYSICAL

DIGITAL

Environment

INTERACTION MODEL

PHYSICAL

DIGITAL

Environment

PObject

INTERACTION MODEL

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

TOutputTObjectTInput

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

DInput

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

DInput

Network

DObjectDObject

PHYSICAL

DIGITAL

Environment

PObject

Model Model

TInputTObject

INTERACTION MODEL

DInput

Network

DObjectDObject

AN EXAMPLE

DINPUT

Support Heterogeneous Input/Output Hardware;

Device Abstraction

Support Heterogeneous Interaction Modalities

Agnosticism of LegacyMiddlewares

DINPUT

Spatial Awareness

DINPUT

For developers: an Application Programming Interface for the direct coding of devices behaviors.

For designers/researchers: an XML-based language for defining object creation, management, communication and interaction within a multiuser, multidevice environment.

PRACTICAL FEASIBILITY

PART II: HOW THINGS COULD BE• Hypothesis• Objective• Research methodology• The proposal• Evaluation

FramingA COMPREHENSIVE EVALUATION

1. Use case

2. USER STUDY

USE CASEDigitally-augmented product shelf

Depth Camera

RGB Camera

I/O Hardware MAGIC MOUSE(FIDUCIAL MARKER)

Hardware Abstraction

Pico-Projector

USER’S FINGER

RGB CAMERA

IR CAMERA

+PICO PROJECTOR

IR EMITTER

Depth Camera

RGB Camera

I/O Hardware MAGIC MOUSE(FIDUCIAL MARKER)

Hardware Abstraction

Pico-Projector

USER’S FINGER

RGB CAMERA

IR CAMERA

IR EMITTER

PICO PROJECTOR

Gesture Interpreter

Depth to Touch

Input Interpretation

Application

FIDUCIAL MARKER

FINGER

DPANELDPANELDPANEL DBUTTONDBUTTONDBUTTON

Null hypothesis

The framework does not affect positively the efficiency in the development of prototypes that make an integrated use of different devices.

USER STUDY

Independent variables

Hardware Technology (2)Wiimote or iPad

Software Technology (3) wrj4P5 library, oscP5 library or the framework

Interaction Component (3)real button (the physical button of the Wiimote), virtual button (a button widget on the iPad) or accelerometer

USER STUDY (2)

Dependent variables

Time to complete the task

USER STUDY (3)

• Texto

USER STUDY (4)Experiment setup

TIME TO COMPLETE THE TASK

TIME TO COMPLETE THE TASKAverage times for each task under each condition

Wiimote w/ framework

Wiimote w/o framework

iPad w/ framework

iPad w/o framework

Results

The null hypothesis has been rejected using the non-parametric test Wilcoxon Rank Sum for Large Samples

USER STUDY (5)

PART III: CONCLUSIONS• Contributions• Potential for future research

Objective. The definition of a conceptual and software framework that provides u s e r s w i t h a c o m p r e h e n s i v e environment for managing interactions between heterogeneous networked devices

MAIN CONCLUSION

Hypothesis. Encompassing heterogeneous devices into a unique design can reduce user efforts to develop ubiquitous interactive systems

MAIN CONCLUSION

Hypothesis. Encompassing heterogeneous devices into a unique design can reduce user efforts to develop ubiquitous interactive systems

MAIN CONCLUSION

DISSEMINATION

Journals and MagazinesIEEE Pervasive Magazine 2010,

ACM Computing Surveys 2013

ConferencesTEI 2014, Physicality 2012, ITS 2011, AVI 2010

FUTURE WORK

Implementation of a wide range of sensors, actuators and platforms

Web-based user interface for the definition of sensors-actuators rules

In which ways a tool influences the exploration and understanding of a problem space?

Thanks for your attention!¡Gracias por su atención!Grazie per l’attenzione!

Rapid prototyping of ubiquitous interaction

Technology

Transcript of Rapid prototyping of ubiquitous interaction

Wiring: Prototyping Physical Interaction Designpeople.interactionivrea.org/h.barragan/thesis/thesis_low_res.pdf · Wiring: Prototyping Physical Interaction Design. put/output board

Prototyping Ubiquitous Imaging Surfaces · Prototyping Ubiquitous Imaging Surfaces Kyle Montague1, Daniel Jackson1, Tobias Brühwiler2, Tom Bartindale1, ... spaces are being overlooked

towards a ubiquitous semantics of interaction

Virtual prototyping in human-machine interaction …...Virtual prototyping in human-machine interaction design In recent years, the use of virtual prototyping (VP) has increased in

Virtual prototyping in human-machine interaction design · · 2015-03-18Virtual prototyping in human-machine interaction design ... Time savings by using virtual prototyping ...

Virtual Sensors: Rapid Prototyping of Ubiquitous ...rod/publications/NorMur11.pdf · Virtual Sensors: Rapid Prototyping of Ubiquitous Interaction with a Mobile Phone and a Kinect

Amarino:a toolkit for the rapid prototyping of mobile ubiquitous computing

Bury the Wireframe: A Primer in Interaction Prototyping

User Interaction: Ubiquitous Computing

Wiring: Prototyping Physical Interaction Designwiring.org.co/download/Wiring_thesis.pdf · Wiring: Prototyping Physical ... Wiring is both a programming environment and an electronics

Interaction Management for Ubiquitous Augmented · PDF fileInteraction Management for Ubiquitous Augmented Reality User ... Ubiquitous Augmented Reality user interfaces incorporate

UBIQUITOUS INTERACTION AND COLLABORATION WITH … · 2013-02-19 · ubiquitous access, interaction and bookability of such services through mobile devices may therefore be key to

Prototyping for Interaction Design

Ubiquitous Computing and Interaction Group

Essay on Interaction Design for Ubiquitous Computing

Diplomarbeit Interaction Management for Ubiquitous Augmented Reality User Interfaces

Rapid Prototyping in User Interaction Development & Evaluation

THE APEX FRAMEWORK: PROTOTYPING OF UBIQUITOUS ENVIRONMENTS BASED ON PETRI NETS

GAINING DESIGN INSIGHT THROUGH INTERACTION PROTOTYPING TOOLSbjoern/... · Prototyping is the fundamental activity that structures innovation in design. While prototyping tools are

Prototyping and analysing ubiquitous computing ...eprint.ncl.ac.uk/file_store/production/197701/718C... · Prototyping and Analysing Ubiquitous Computing Environments using Multiple