Rapid Prototyping of Distributed User Interfaces J. P. Molina 1,2, J. Vanderdonckt 1, P. González 2...

Rapid Prototyping of Rapid Prototyping of Distributed User Distributed User

InterfacesInterfacesJ. P. MolinaJ. P. Molina1,21,2, J. Vanderdonckt, J. Vanderdonckt11, P. González, P. González22

A. FernándezA. Fernández22 and M. D. Lozano and M. D. Lozano22

11 Université catholique de Louvain Université catholique de Louvain22 University of Castilla-La Mancha University of Castilla-La Mancha

www.usixml.org

6th International Conference on 6th International Conference on Computer-Aided Computer-Aided Design of User Interfaces CADUIDesign of User Interfaces CADUI´2006´2006Bucharest, Romania, June 5-8, 2006Bucharest, Romania, June 5-8, 2006

June 5-8 2CADUI 2006, Bucharest, Romania

ContentsContents

IntroductionIntroduction FoundationsFoundations Distributed UI Design SpaceDistributed UI Design Space Software Tool for Rapid PrototypingSoftware Tool for Rapid Prototyping ExamplesExamples ConclusionConclusion


ContentsContents



IntroductionIntroduction

Distributed User InterfacesDistributed User Interfaces ( (DUIsDUIs) ) apply the notion of distributing parts or apply the notion of distributing parts or whole of a UI across several places or whole of a UI across several places or locations like Distributed Systems.locations like Distributed Systems.

Designing a DUI remains a complex Designing a DUI remains a complex problem which may prevent designers problem which may prevent designers from exploring design issues.from exploring design issues.



First, advances in DUIs do not necessarily First, advances in DUIs do not necessarily rely on the same concepts of distribution, rely on the same concepts of distribution, and the axes according to which the and the axes according to which the distribution may take place vary from one distribution may take place vary from one research to another.research to another.

Second, as developing DUIs require a Second, as developing DUIs require a sophisticated architecture, it is not sophisticated architecture, it is not surprising DUIs are slow to obtain, surprising DUIs are slow to obtain, expensive to produce, and probably expensive to produce, and probably equally complex to use.equally complex to use.



Rapid prototyping of DUIsRapid prototyping of DUIs turns to be turns to be an important issue, not only it could be an important issue, not only it could be used as a vehicle for developing and used as a vehicle for developing and demonstrating visions of innovative DUIs demonstrating visions of innovative DUIs but also…but also…

… … it could help showing various it could help showing various distribution configurations before going to distribution configurations before going to full implementation.full implementation.



However, rapid prototyping is also However, rapid prototyping is also challenging, as the design space of DUIs challenging, as the design space of DUIs covers a wide range of different covers a wide range of different possibilities.possibilities.

In order to tackle this complexity, the In order to tackle this complexity, the software tool presented here supports software tool presented here supports rapid prototyping of DUIs based on a rapid prototyping of DUIs based on a limited, but significant, design space of limited, but significant, design space of four design dimensionsfour design dimensions..


ContentsContents



FoundationsFoundations

The software tool and the design space The software tool and the design space presented here relies on the following presented here relies on the following concepts:concepts: Cameleon frameworkCameleon framework Context of useContext of use Interaction surfaceInteraction surface Environment modelEnvironment model


Cameleon Reference Cameleon Reference FrameworkFramework

Defines UI development steps for multi-Defines UI development steps for multi-context interactive applications [Calvary context interactive applications [Calvary et et al.al., 2003]., 2003]. Final UI (FUI)Final UI (FUI)..

UI running on a particular computing platform (HTML, UI running on a particular computing platform (HTML, Java, VRML).Java, VRML).

Concrete UI (CUI)Concrete UI (CUI).. A mock-up that is independent of any computer platform.A mock-up that is independent of any computer platform.

Abstract UI (AUI)Abstract UI (AUI).. Defines abstract containers and individual components, Defines abstract containers and individual components,

independent of any modality of interaction.independent of any modality of interaction. Task & Concepts (T&C)Task & Concepts (T&C). .

Describe tasks and domain-oriented concepts.Describe tasks and domain-oriented concepts.


Cameleon reference Cameleon reference frameworkframework

Multi-path UI development: Abstraction, Multi-path UI development: Abstraction, Reification and Translation.Reification and Translation.

Task & Concepts

Abstract UI (AUI)

Concrete UI (CUI)

Final UI (FUI)

Task & Concepts

Abstract UI (AUI)

Concrete UI (CUI)

Final UI (FUI)

Context of use A Context of use B

Reification TranslationAbstraction

Task & Concepts

Abstract UI (AUI)

Concrete UI (CUI)

Final UI (FUI)

Task & Concepts

Abstract UI (AUI)

Concrete UI (CUI)

Final UI (FUI)

Context of use A Context of use B

Reification TranslationAbstraction


Cameleon reference Cameleon reference frameworkframework

The selected language for this work is The selected language for this work is UsiXMLUsiXML, which stands for , which stands for User Interface User Interface eXtensible Markup LanguageeXtensible Markup Language..

www.usixml.org

IdealXML[Montero et al., 2005]

GrafiXML[B. Michotte]


Context of useContext of use

The context of use describes all the The context of use describes all the entities that may influence how the user’s entities that may influence how the user’s task is carried out.task is carried out.

Takes into account three aspects:Takes into account three aspects: User typeUser type, ,

e.g. system experience, task experience, task e.g. system experience, task experience, task motivation.motivation.

Computing platform typeComputing platform type, , e.g. mobile platform, stationary platform.e.g. mobile platform, stationary platform.

Physical environment typePhysical environment type, , e.g. office conditions, outdoor conditions.e.g. office conditions, outdoor conditions.


Interaction surfaceInteraction surface

This term, introduced by [Coutaz This term, introduced by [Coutaz et al.et al., , 2003], refers to any physical surface 2003], refers to any physical surface which can be “acted on or observed” so as which can be “acted on or observed” so as to support user interaction with a system, to support user interaction with a system, whether visible or embedded.whether visible or embedded.

An interaction surface could be a screen, An interaction surface could be a screen, a monitor, a wall display, a table equipped a monitor, a wall display, a table equipped with a camera tracking system, etc. with a camera tracking system, etc.


Environment ModelEnvironment Model

UsiXML’s existing UsiXML’s existing physical physical environment model environment model has been expanded has been expanded with the concept of with the concept of interaction surface.interaction surface.

+id : string(idl)+name : string(idl)

Context

+id : string(idl)+stereoType : string(idl)+taskExperience : string(idl)+systemExperience : string(idl)+device Experience : string(idl)+taskMotivation : string(idl)

userStereotype

+id : string(idl)+name : string

Platform

+id : string(idl)+type : string(idl)+name : string(idl)+isNoisy : boolean(idl)+isStressing : boolean(idl)+lightingLevel : string(idl)

Environment

Context Model

+Composes1+isComposedOf1..*

0..1

0..*

0..1

0..*

+surfaceID : string(idl)+name : string(idl)+title : string(idl)+xPosition : float(idl)+yPosition : float(idl)+zPosition : float(idl)+height : float(idl)+frameColor : float(idl)+transparencyLevel : float(idl)+texture : float(idl)+xOrientation : float(idl)+yOrientation : float(idl)+zOrientation : float(idl)+angleOrientation : float(idl)+top : float(idl)+right : fixed(idl)+width : float(idl)

Surface

-xPosOnSurface : float(idl)-yPosOnSurface : float(idl)-zPosOnSurface : float(idl)

Location0..1

-isLocatedOn

0..*

-Locates

+isFluid : boolean(idl)+isRigid : boolean(idl)+isRotatable : boolean(idl)+Material : string(idl)

InteractionSurface

+isConnectedTo

0..*

+Connects0..*1*

+id : string(idl)+name : string(idl)+xPosition : float(idl)+yPosition : float(idl)+zPosition : float(idl)+xSize : float(idl)+ySize : float(idl)+zSize : float(idl)+Geometry : string(idl)

Shape

1

*

+shapes

0..*

+isShapedAs 1

+hardwareID : string(idl)+category : string(idl)+colorCapability : string(idl)+pointingDevice : string(idl)+pointingResolution : float(idl)+screenWidth : fixed(idl)+screenHeight : fixed(idl)+screenSizeChar : fixed(idl)

Hardware Platform

-isPossessedBy

1

-possesses

0..*


Environment ModelEnvironment Model The The physical physical

environmentenvironment is is described as a scene described as a scene which is in turn which is in turn decomposed into decomposed into surfacessurfaces, to be , to be connected together or connected together or not by position and not by position and orientation. Each orientation. Each surface can be defined surface can be defined as an as an interaction interaction surfacesurface..


Context


userStereotype


Platform


Environment

Context Model


0..1

0..*

0..1

0..*


Surface


Location0..1

-isLocatedOn

0..*

-Locates


InteractionSurface

+isConnectedTo

0..*

+Connects0..*1*


Shape

1

*

+shapes

0..*

+isShapedAs 1


Hardware Platform

-isPossessedBy

1

-possesses

0..*


Environment ModelEnvironment Model An environment also An environment also

comprises one or several comprises one or several computing platformscomputing platforms, , holding one or more holding one or more hardware platformshardware platforms. .

Hardware platforms are Hardware platforms are considered as rectangular considered as rectangular interaction surfaces that interaction surfaces that can be acted on (by using can be acted on (by using pointers) and/or observed pointers) and/or observed

(by looking at the (by looking at the screen).screen).


Context


userStereotype


Platform


Environment

Context Model


0..1

0..*

0..1

0..*


Surface


Location0..1

-isLocatedOn

0..*

-Locates


InteractionSurface

+isConnectedTo

0..*

+Connects0..*1*


Shape

1

*

+shapes

0..*

+isShapedAs 1


Hardware Platform

-isPossessedBy

1

-possesses

0..*


ContentsContents



Distributed UI Design Distributed UI Design SpaceSpace

The proposed DUI design space is The proposed DUI design space is decomposed into four design axes or decomposed into four design axes or dimensions:dimensions: Type of computing platformType of computing platform

Stationarization … MobilizationStationarization … Mobilization Amount of interaction surfacesAmount of interaction surfaces

Decentralization … CentralizationDecentralization … Centralization Type of interaction surfaceType of interaction surface

Physicalization … DigitizationPhysicalization … Digitization Type of user interfaceType of user interface

Materalization … DemateralizationMateralization … Demateralization


Distributed UI Design Distributed UI Design SpaceSpace


ContentsContents

IntroductionIntroduction FoundationsFoundations Distributed UI Design SpaceDistributed UI Design Space Software Tool for Rapid Software Tool for Rapid

PrototypingPrototyping ExamplesExamples ConclusionConclusion


Software Tool for Rapid Software Tool for Rapid PrototypingPrototyping

A software tool named A software tool named VUIToolkitVUIToolkit has been has been developed above UsiXML and expanded with developed above UsiXML and expanded with the previous environment model. the previous environment model.

With this tool, the environment model is With this tool, the environment model is produced as a Virtual Reality scene, thus produced as a Virtual Reality scene, thus allowing the rendering of both software (e.g. allowing the rendering of both software (e.g. widgets) and hardware (e.g. physical buttons), widgets) and hardware (e.g. physical buttons), but just for rapid prototyping purposes.but just for rapid prototyping purposes.

The tool provides basic operations such as The tool provides basic operations such as copy a CUI from one surface to another one, copy a CUI from one surface to another one, thus migrating it. thus migrating it.


Software Tool for Rapid Software Tool for Rapid PrototypingPrototyping

VUIToolkit offers a set of VUIToolkit offers a set of VRML97/X3D PROTOs VRML97/X3D PROTOs that transform the that transform the standard plain widgets standard plain widgets into a truly 3D into a truly 3D representation, referred representation, referred as as Virtualized User Virtualized User InterfaceInterface..


Mapping from CUI to Mapping from CUI to VRML/X3DVRML/X3D

When mapping a CIO from CUI to a When mapping a CIO from CUI to a VRML or X3D world, different cases VRML or X3D world, different cases are assessed:are assessed: Direct mapping.Direct mapping.

It is not possible to set a one-to-one It is not possible to set a one-to-one mapping, as there is no VRML/X3D mapping, as there is no VRML/X3D primitives that can be matched with a CIO. primitives that can be matched with a CIO. These Web3D languages specify basic These Web3D languages specify basic elements that must be used together to elements that must be used together to create interactive elements.create interactive elements.



When mapping a CIO from CUI to a When mapping a CIO from CUI to a VRML or X3D world, different cases are VRML or X3D world, different cases are assessed:assessed: Direct mapping.Direct mapping. New mapping.New mapping.

There is a need to fill a gap by introducing a new There is a need to fill a gap by introducing a new widget by appropriate implementation.widget by appropriate implementation.

Some attributes defined in the CUI model are not Some attributes defined in the CUI model are not used in the 3D world, while new attributes must be used in the 3D world, while new attributes must be added in order to describe the interface elements added in order to describe the interface elements in the 3D world, such as position and size.in the 3D world, such as position and size.



When mapping a CIO from CUI to a When mapping a CIO from CUI to a VRML or X3D world, different cases VRML or X3D world, different cases are assessed:are assessed: Direct mapping.Direct mapping. New mapping.New mapping. No possible mapping.No possible mapping.

There are CIOs whose VRML/X3D There are CIOs whose VRML/X3D counterpart have not been done yet or are counterpart have not been done yet or are impossible or difficult to implement. An impossible or difficult to implement. An example of this case is the CIO example of this case is the CIO boxbox..



Input/output mapping from desktop-oriented Input/output mapping from desktop-oriented 2D GUI to VRML/X3D virtual world.2D GUI to VRML/X3D virtual world. Display = “Screen” PROTO.Display = “Screen” PROTO.

Rectangular area where screen units (pixels) are Rectangular area where screen units (pixels) are converted into world units (meters).converted into world units (meters).

Serves as a container of UI elements (Serves as a container of UI elements (childrenchildren field). field). Mouse = User’s pointing device.Mouse = User’s pointing device.

VRML/X3D sensors.VRML/X3D sensors. Keyboard?Keyboard?

VRML: requires non-standard extensions (PG’s VRML: requires non-standard extensions (PG’s KbdSensorKbdSensor).).

X3D: X3D: StringSensorStringSensor.. Virtual keyboard.Virtual keyboard.


ContentsContents



ExamplesExamples

The user’s environment is a small The user’s environment is a small office, with five different platforms office, with five different platforms


Across Multiple Computing Across Multiple Computing PlatformsPlatforms

StationarizationStationarization … … MobilizationMobilization Moving along this axis means Moving along this axis means

rendering a FUI on a target rendering a FUI on a target platform which is more platform which is more stationarystationary / / mobilemobile than the than the source platform.source platform.

It can be achieved by performing It can be achieved by performing an abstraction from CUI to AUI, an abstraction from CUI to AUI, followed by a reification from followed by a reification from AUI to CUI. A simple translation AUI to CUI. A simple translation from CUI to CUI can also work.from CUI to CUI can also work.


Across Multiple Computing Across Multiple Computing PlatformsPlatforms

StationarizationStationarization MobilizationMobilization


Across Multiple Interaction Across Multiple Interaction SurfacesSurfaces

DecentralizationDecentralization … … CentralizationCentralization Moving along this axis means Moving along this axis means rendering a FUI rendering a FUI

on more / less interaction surfaces than on more / less interaction surfaces than previously, the displays being connected to the previously, the displays being connected to the same computing platform or not.same computing platform or not.

It can be achieved by performing a It can be achieved by performing a decomposition of the source UI (by graceful decomposition of the source UI (by graceful degradation, fragmentation, etc) or a re-degradation, fragmentation, etc) or a re-composition of gathered pieces (by union, composition of gathered pieces (by union, merging, etc).merging, etc).


Across Multiple Interaction Across Multiple Interaction SurfacesSurfaces

DecentralizationDecentralization CentralizationCentralization


Across Different Interaction Across Different Interaction SurfacesSurfaces

PhysicalizationPhysicalization … … DigitizationDigitization Moving along this axis means Moving along this axis means

rendering a FUI on a target rendering a FUI on a target platform which is more platform which is more physicalphysical / / digitaldigital than the source platform. than the source platform.

Projection surfaces are Projection surfaces are considered as physical ones since considered as physical ones since the UI is rendered by projection the UI is rendered by projection onto a surface, and user’s actions onto a surface, and user’s actions are tracked by camera are tracked by camera recognition techniques.recognition techniques.


Across Different Interaction Across Different Interaction SurfacesSurfaces

PhysicalizationPhysicalization DigitizationDigitization


Across Different Types of User Across Different Types of User InterfacesInterfaces

MaterializationMaterialization … … DematerializationDematerialization Moving along this axis means Moving along this axis means changing the changing the

distribution of the FUI towards more distribution of the FUI towards more physicalphysical / / immaterialimmaterial components. components.

It can be achieved by re-assigning pieces of It can be achieved by re-assigning pieces of the FUI to physical or software objects. the FUI to physical or software objects. Materialization obviously touches the area of Materialization obviously touches the area of tangible UIs. tangible UIs.


Across Different Types of User Across Different Types of User InterfacesInterfaces

MaterializationMaterialization DematerializationDematerialization


ContentsContents



ConclusionsConclusions A design space for DUIs consisting of four dimensions A design space for DUIs consisting of four dimensions

has been introduced to better understand this field.has been introduced to better understand this field. That design space is supported in a tool for rapid That design space is supported in a tool for rapid

prototyping of DUIs, using VUIToolkit to render in a prototyping of DUIs, using VUIToolkit to render in a virtual world the concrete UI specified in UsiXML.virtual world the concrete UI specified in UsiXML.

A richer environment model has been defined so as to A richer environment model has been defined so as to represent the world in which distribution may occur.represent the world in which distribution may occur.

The functionality given by this combination enable The functionality given by this combination enable designers to explore different design issues and designers to explore different design issues and keeping the distribution which is found adequate to keeping the distribution which is found adequate to the goals.the goals.

MMulţumesc!ulţumesc! Thank youThank you

MerciMerciGraciasGracias

More info:More info:www.usixml.orgwww.usixml.org

Contact:Contact:[email protected]@dsi.uclm.es

The UsiXML Consortium

Rapid Prototyping of Distributed User Interfaces J. P. Molina 1,2, J. Vanderdonckt 1, P. González 2...

Documents

Transcript of Rapid Prototyping of Distributed User Interfaces J. P. Molina 1,2, J. Vanderdonckt 1, P. González 2...