1 Private Ph.D. defense – Louvain-la-Neuve, June 25th, 2007 A Methodological Framework for...

1 Private Ph.D. defense – Louvain-la-Neuve, June 25th, 2007

A Methodological Frameworkfor Multi-Fidelity Sketching

of User InterfacesAdrien Coyette

Université catholique de Louvain (UCL)Louvain School of Management (IAG-LSM)

Information Systems Unit (ISYS)Belgian Lab. of Computer-Human Interaction (BCHI)

http://www.isys.ucl.ac.be/bchi


Presentation plan

MotivationsMotivations

PrototypingPrototyping

State of the ArtState of the Art

SketchiXML SketchiXML

MethodologiesMethodologies

SurveysSurveys

ConclusionConclusion


Motivations: UI importance

User interface determines how easily a user may control underlying functions of a computer program

A program equipped with powerful functionalities and low quality user interfaces may be under-exploited or misused

In an interactive application, the UI is probably the portion which affects the most the general acceptability of the system by end usersOn the average, 48% of the code written for the 69 applications studied was devoted to the user interface portion

[Nielsen 93; Myers & Rosson 92]


Motivations: UI development life cycle

Software plans and requirements

Validation

Validation

Product design

Verification

Detailed design

Verification

Code

Unit test

Integration

Product verification

Operations and maintenance

Revalidation

Implementation

System test

System feasibility

[Boehm88]

UI Design occupied 45% of the totalamount of time devoted to each activity

50%

37%

Mean = 44%


Motivations: methods

As in software engineering, HCI community wants to develop a well-structured and step-wise method for developing user interfaces.

The systematization and the reproducibility found in software engineering methods however cannot be transferred straightforwardly in HCI

The development life cycle remains inherently:

Open, Ill-defined, Highly iterative

Interaction is of higher complexity than algorithms

[Sumner et al.Sumner et al., 1997; Wegner, 1997]


Motivations: iterative design

Designers often consider that informal techniques based on mock up is the best alternative for the early design phase.

It allows to cope with the fact the such process is highly iterative in 3 stages

[Norman, 1996Norman, 1996]


Motivations: empirical findings

UI Drawing or sketching increases participatory design and subjective satisfaction with respect to UI editors

[Landay & Myers, 1995, Plimmer & Apperley, 2004, Buxton, 2005]

Equal usability problems found for a sketched UI than with an edited one[Virzi et al., 1996]

Explanatory power of a sketched UI does not decrease with respect to edited one; tool support is preferred over paper prototyping

[Walker et al., 2002 ]

A sketched UI delivers same quantity and quality of outputs than an edited one[Sefelin et al., 2006]

UI sketching fosters alternative designs and communication [Tohidi et al., 2006]

Supports throw-away prototyping due to low cost[Schneider & Petrie, 2006, Bailey & Konstan, 2003]


Presentation plan






SurveysSurveys



Prototyping: when?

Early design

Late design

Low fidelity paper prototypes

Content: Mainly Presentation

Use: Exploration, Communication

Medium fidelity prototypes

Content: Presentation, content, basic navigation

Use: Simulation, refinement, user testing…

High fidelity prototypes

Content: Presentation, navigation, functionalities, content

Use: Final specifications, marketing, documentation

A paper based prototype

A PowerPoint prototype

The final user interface with few functionalities


Prototyping: sketches

• Paper prototypes– Familiar and unconstrained approach – Fast to learn and quick to produce – Focus on basic structural issues instead of unimportant

details – Support collaboration between designers and end-users – …

• Purpose– Early design phase– Get a sense of the user’s needs and goals– Supports brainstorm competing representations– …


Prototyping:Limiting prototype functionality

• Vertical prototypes– Includes in-depth functionality for only a few selected

features– Common design ideas can be tested in depth

• Horizontal prototypes– The entire surface interface with no underlying

functionality– A simulation; no real work can be performed

• Qcenario– Scripts of particular fixed uses of the system; no deviation allowed

Vertical prototype

Scenario

Horizontal prototype

Full interface

[Nielsen 1993]


Prototyping:Integrating prototypes and products

• Throw-away– Prototype only serves to elicit user reaction– Creating prototype must be rapid, otherwise too expensive

• Incremental– Product built as separate components (modules)– Each component prototyped & tested, then added to the

final system

• Evolutionary– Prototype altered to incorporate design changes– Sometimes becomes the final product


Presentation plan






SurveysSurveys



State of the Art:Computer Assisted Prototyping

Several levels of granularity

Support for scenario-based design

Good documentation

No shape recognition and interpretation

No code generationNo preview mode

DENIM

[Lin,2000]


State of the Art:Computer Assisted Prototyping

Performance (speed and accuracy)Multi-stroke

gesturesRecognizes rotated shapes

Interpretation in Java

Only generates Java

No scenario editor

Limited widget setMono-window

JavaSketchIt

[Caetano,2002]


State of the Art:Eliciting requirements

17 different software and approaches surveyedSources: scientific literature and commercial

softwareComparative analysis relying on consistent

analysis gridTool identification (7)Install and first use (4)Tool functions (8)Shape recognition (4)Shape interpretation (7)UI editors (9)

Applied to the same case study

(eCommerce app.)


State of the Art:List of final requirements

Avoidance of Effort loss

Well defined editing functionalities

Language neutrality

Robust recognition

Large conceptual coverage

Recognition and process flexibility

Design history

Expressive scenario editor

Ease of use (naturalness)

Preview (Run-mode)


Presentation plan






SurveysSurveys



SketchiXML: Demonstation


SketchiXML: Shape Recognition



Recognition engine: Cali libraryRecognition engine: Cali library

A fast, simple and compactA fast, simple and compact approach to identify Scribbles (multi-stroke geometric shapes)

Combines temporal adjacency, Fuzzy Logic and geometric temporal adjacency, Fuzzy Logic and geometric featuresfeatures to classify scribbles.

Recognizes elementary geometric shapes, such as Triangles, Rectangles, Diamonds, Circles… and some gesture commands, such as, Delete, Cross, WavyLine, Move, Copy and Tap.

Shapes are recognized independently of changes in rotation, recognized independently of changes in rotation, size or number of individual strokessize or number of individual strokes

The recognizer has a recognition rate of 94%94% and a fast response



Trainable Gesture RecognizerTrainable Gesture Recognizer Handdrawn input (sketch) is

superimposed with a grid Each sampling point is mapped to

its closest grid node For adjacent nodes, a stroke

direction (out of 8 different directions) is taken

A string is generated from direction coding of input

Complex task of comparison of drawing inputs is reduced to simple calculation of string edit distances



Distance is shortest sequence of edit commands that transform s to t Simplest set of operations:

• Copy character from s over to t • Delete a character in s (cost 1) • Insert a character in t (cost 1) • Substitute one character for another (cost 1)

Levenstein Levenstein distance distance


SketchiXML: Shape Interpretation


SketchiXML: Grammar

<widget type="ListBox"> <representation id="0">

<constraint id="0" shape1="Triangle_3" shape2="Rectangle_0" condition="isInsideInUpperRightCorner" /> <constraint id="1" shape1="Triangle_4" shape2="Rectangle_0" condition="isInsideInLowerRightCorner" />

<shape id="Rectangle_0" type="Rectangle" /> <shape id="Triangle_3" type="Triangle" /> <shape id="Triangle_4" type="Triangle" /> </representation><representation id="1"><constraint id="0" shape1="Triangle_0" shape2="Rectangle_2" condition="isInsideInTop"/>

<constraint id="1" shape1="Triangle_1" shape2="Rectangle_2" condition="isInsideInBottom" /> <constraint id="2" shape1="Rectangle_2" shape2="Rectangle_3" condition="isInsideOnTheRight" />

<shape id="Triangle_0" type="Triangle" /> <shape id="Triangle_1" type="Triangle" /> <shape id="Rectangle_2" type="Rectangle" /> <shape id="Rectangle_3" type="Rectangle" /> </representation> <representation id="2"><constraint id="0" shape1="Line_0" shape2="ListBox_1" condition="isInside" /> <constraint id="1" shape1="Line_0" shape2="-" condition="isHorizontal" /> <shape id="Line_0" type="Line" /> <shape id="ListBox_1" type="ListBox" /> </representation></widget>


Presentation plan






SurveysSurveys



Methodologies: framework

• Prototyping types

User interface

Control layer

Functional core

Completeinteractiveapplication

Horizontal prototypeVertical prototype

User interface

Control layer

Functional core


Diagonal prototype

[Nielsen93]




User interface

Control layer

Functional core


PresentationGlobal

navigationLocal

navigation




Control layer

Functional core

PrésentationGlobal

navigationLocal

navigation

Presentational prototype first

Control layer

Functional layer

PresentationNavigation

globaleLocal

navigation

Global navigation prototype first

Control layer

Functional core

PrésentationNavigation

globaleNavigation

locale

Local navigation prototype first


Methodologies: Support by fidelity

High fidelity prototype : GrafiXML

Medium fidelity prototype : VisiXML

Low fidelity prototype: SketchiXML


Methodologies: Application to development methodologies

Analyst claimed in the past that prototyping should be considered as an alternative to the systems development life cycle…

More recent analysis advocate the use of prototyping as a part of the traditional systems development life cycle

Existing methodologies tend to integrate prototyping as part of the approach

Based on the framework we are thus able to support effectively the current development methodologies. SketchiXML permit to involve end users without pre-requisite

[Kendall 97]


Best practices:

• Collect information

• Develop simple prototype

• Discuss with the end-users

• Test interaction between end user and prototype

• Adapt the fidelity level

• Avoid polished prototype

• Iterate

Applied to the latest methodologiesApplied to the latest methodologies

[Meyer 2005]

Methodologies: Application to development methodologies


Presentation plan






SurveysSurveys



Surveys: Widget catalogue

Building the widget catalogue Building the widget catalogue


Surveys: widget catalogue


Surveys: User testing of fidelity

Method:

• 12 participants

• Wacom graphic tablet

• 12 widgets randomly selected

• 4 fidelity levels

• Repeated twice

Factors observed:

• Time required

• Number of errors

• Number of delete operations


Surveys: User testing of fidelity

Results:

• Fidelity has no impact on the sketching of individual widget (time or error)

• User profile influence the widget sketching time (non UI designers performed even better !)

• Proved learning effect over time


Surveys: Performances

Widget

Recognition rateSketching Time

Q1 Q2 Q3 Q4 total Q1 Q2 Q3 Q4 total

Button 0.88 0.94 1.00 1.00 0.95 6654 2646 1952 2036 3322

Checkbox 1.00 0.94 0.88 1.00 0.95 1221 2040 2030 1486 1694

Combox 0.75 0.88 0.94 0.88 0.86 4501 2445 2859 3178 3246

Label 0.94 0.94 1.00 1.00 0.97 2513 353 35 37 734

Listbox 0.88 0.94 0.88 0.94 0.91 7325 7250 8535 5852 7241

Picture 1.00 1.00 1.00 0.87 0.97 1858 1541 1378 2836 1903

Progress 0.69 0.81 0.94 0.94 0.84 11429 5697 4118 4547 6448

Radio 1.00 0.94 0.88 0.94 0.94 1085 2030 1913 2029 1764

Slider 0.94 0.88 0.94 0.94 0.92 3313 2789 2474 1851 2607

Textarea 0.94 1.00 0.81 1.00 0.94 3545 2342 4838 2312 3259

Textfield 0.94 1.00 1.00 1.00 0.98 3379 1080 1089 1204 1688

Toggle 0.69 0.69 0.88 0.94 0.80 16041 6641 7445 4283 8603

Total 0.89 0.91 0.93 0.95 0.92 5238,62 3071,19 3222,15 2637,61 3542,39


Surveys: Representations

Group Widget Gesture repres

Recognition rate

Spe

cific

or

ienta

tion

Sim

ple

incl

usi

on

Com

plex

in

clusi

on

Juxt

apos

itio

n

Inte

rsec

tion

sequ

ence

of

com

ponen

ts

Siz

e

1

Text field 0,98 X X

1 Label 0,97 X

1 Picture 0,97 X

2 Button 0,95 X

2 Checkbox 0,95 X X X

2 Radio button 0,94 X X X

2 Text area 0,94 X X X

2 Slider 0,92 X X X

3 List box 0,91 X X X X

3 Combobox 0,86 X X

3 Progress bar 0,84 X X X X

3 Toggle button 0,80 X X


Presentation plan






SurveysSurveys



1) Fundamental

• Notion of fidelity level (low, medium, high) in multi-fidelity prototyping

• Graphical definition of widgets in terms of representations• Formal definition of representations in a grammar• Gesture recognition algorithm based on Levenstein distance

2) Methodological

• Multiple prototype types (diagonal=vertical+horizontal) and paths• Intertwining of presentational and navigational prototyping• Integration within various development life cycles• Transition between the fidelity levels (within SketchiXML and

other tools based on UsiXML) at Concrete User Interface level

Conclusion: contributions


3) Empirical

• User and designer acceptance of widget representations• User testing of sketching tool• Complexity of widget representations

4) Implementation

• Implementation of SketchiXML (50000 LOC)• Multi-agent shape recognizer

– 8 basic shapes (i.e., triangle, rectangle, cross, line, wavy line, arrow, ellipse, and circle)

– 6 basic commands (i.e., undo, redo, copy, paste, cut, new window)– 27 widgets

• Trainable gesture recognition engine• Multi-fidelity support by smooth transition between (none, low, medium, high)• Multi-agent interpreter• All 10 requirements addressed (partially or completely)

Conclusion: contributions


Conclusion: future work

• Extending the coverage of sketching artifacts• Improving the Text Divider• Tuning the Recognition Engine more extensively• Support for Multi-windowing Design• Augmenting the Support for Design Memory• Extending to other domains than Computer Science

• General Improvement of sketching facilities


Conclusion: example of extension

• Sketching Physical User Interfaces with transition between digital and physical worlds


For more information and downloading,http://www.isys.ucl.ac.be/bchi

http://www.usixml.orgUser Interface eXtensible Markup Language

http://www.similar.ccEuropean network on Multimodal UIs

Special thanks to all members of the team!

This work has been conducted in the context of the following projects

Projet WIST 1 ReQuestRapid prototyping of e-commerce applications

http://www.isys.ucl.ac.be/bchi

http://www.usixml.org/

http://www.similar.cc/


Thank you very much for your attention!Questions?

1 Private Ph.D. defense – Louvain-la-Neuve, June 25th, 2007 A Methodological Framework for...

Documents

Transcript of 1 Private Ph.D. defense – Louvain-la-Neuve, June 25th, 2007 A Methodological Framework for...