The Glass Class: Rapid Prototyping for Wearable

Computing

Mark BillinghurstHIT Lab NZ

University of Canterbury

March 25th 2014

How can I quickly build my own wearable experiences?

Wearables Vuzix M100

$999, profession Recon Jet

$600, more sensors, sports Opinvent

500 Euro, multi-view mode Motorola Golden-i

Rugged, remote assistance

Common Elements Different form factor but common

elements Android OS Monocular Display Camera On-board sensors

- Compass, Accelerometer, etc Connectivity Designed for wearability

WTF is This?

Rapid Prototyping for Non-Programmers

Interaction Design Process

Paper Prototyping: Widgets (Low Fidelity)

Tethered Prototypes (High Fidelity)

Goal: Communicate Vision

Rapid Prototyping Approach Quick design Capture key interactions Focus on user interface and

experience No programming required

BUNRATTY FOLK PARKSharon BrosnanSharon Brosnan

06518690651869

Bachelor of Science in Digital Media DesignBachelor of Science in Digital Media Design

EXAMPLE: BUNRATTY PARKIrish visitor attraction run by Shannon Heritage

19th century life is recreatedBuildings from the mid-west have been relocated to Bunratty Castle

30 buildings are set in a rural or village setting there.

AUGMENTED REALITYWant to develop AR tour guideRuns on smart phonesOverlay virtual information on real objects

Provides navigation aidAdds story/game elements

STORYBOARD

SKETCHES – PHOTOSHOPPros:• Good for idea generation• Cheap• Concepts seem feasible

Cons:• Not great feedback gained• Photoshop not fast enough for making changes

POST IT PROTOTYPING

First Draft

Camera View with 3D

Second Draft Third Draft• Selection highlighted in blue • Home button added for easy

navigation to main menu

POWERPOINT PROTOTYPINGBenefits • Used for User Testing• Quick, Interactive• Functionalities work with storyboard• Easy arrangement of slides

User Testing• Participants found• 15 minute sessions screen captured• ‘Talk Allowed’ technique used • Notes taken, Post-Interview

WIKITUDEPopular AR browser Mapping Point of Interest abilities

MultiplatformShows the points of interest of Bunratty Folk Park

Benefits of RAPID Prototyping

Fast and inexpensive Identifies problems before they’re

coded Elicits more and better feedback

from users Helps developers think creatively Gets users involved early in the

process Helps decide design directions

Rapid Prototyping Tools for Wearables

Rapid Prototyping Tools Low-fi/Static Tools

Sketching User interface templates Storyboards/Application flows

High-fi/Interactive Tools Wireframing tools Mobile prototyping Coding

GlassSim – http://glasssim.com/

Simulate the view through Google Glass Multiple card templates

GlassSim Card Builder Use HTML for card

details Multiple templates Change background

Own image Camera view

GlassSim Samples

Glass UI Templates

Google Glass Photoshop Templates http://glass-ui.com/ http://dsky9.com/glassfaq/the-google-glass-psd-template/

Sample Slides From Templates

Glass Application Storyboard

http://dsky9.com/glassfaq/google-glass-storyboard-template-download/

Application Flow

Wireframing Tool Add transitions, limited interactions UXPin Wireframing Tool

http://uxpin.com/ Web based Glass templates

UXpin

Proto.io - http://www.proto.io/

Web based mobile prototyping tool Features

Prototype for multiple devices Gesture input, touch events, animations Share with collaborators Test on device

Proto.io - Interface

Demo Building a Simple Flow

Gesture Flow

Scr1

Scr2 Scr3

Scr4 Scr5 Scr6

Tap

Swipe

Start Transitions

Demo

Justinmind – justinmind.com

Wireframing tool – interactive prototypes Google Glass widget library

Wireframe Limitations Can’t deploy on Wearable No access to sensor data

Camera, orientation sensor No multimedia playback

Audio, video Simple transitions

No conditional logic

App Inventor http://appinventor.mit.edu/ Visual Programming for Android Apps Features

Access to Android Sensors Multimedia output

Drag and drop web based interface Designer view – app layout Blocks view – program logic/control

Device Setup Emulator

Use aiStarter on Windows/Linux Live view (run on connected device)

Make sure device is in Debug mode Install companion app (sideload on

Glass)- MITAI2Companion.apk

App Inventor Designer View

App Inventor Blocks View

Orientation Demo

Use orientation sensor

Rapid Prototyping using Processing

Processing Easy to use language for Interaction Programming tool for Artists/Designers

http://processing.org Easy to code, Free, Open source, Java based 2D, 3D, audio/video support

Processing For Android http://wiki.processing.org/w/Android Generates Android Ready .apk file

Development Environment

Hello World//called initially at the start of the Processing sketch

void setup() {

size(640, 360);

background(0);

}

//called every frame to draw output

void draw() {

background(0);

//draw a white text string showing Hello World

fill(255);

text("Hello World", 50, 50);

}

Demo

Hello World ImagePImage img; // Create an image variable

void setup() {

size(640, 360);

//load the ok glass home screen image

img = loadImage("okGlass.jpg"); // Load the image into the program

}

void draw() {

// Displays the image at its actual size at point (0,0)

image(img, 0, 0);

}

Demo

Touch Pad Input Tap recognized as DPAD inputvoid keyPressed() {

if (key == CODED){

if (keyCode == DPAD) {

// Do something ..

Java code to capture rich motion events import android.view.MotionEvent;

Motion Event//Glass Touch Events - reads from touch pad

public boolean dispatchGenericMotionEvent(MotionEvent event) {

float x = event.getX(); // get x/y coords

float y = event.getY();

int action = event.getActionMasked(); // get code for action

switch (action) { // let us know which action code shows up

case MotionEvent.ACTION_DOWN:

touchEvent = "DOWN";

fingerTouch = 1;

break;

case MotionEvent.ACTION_MOVE:

touchEvent = "MOVE";

xpos = myScreenWidth-x*touchPadScaleX;

ypos = y*touchPadScaleY;

break;

Demo

SensorsKetai Library for Processing

https://code.google.com/p/ketai/Support all phone sensors

GPS, Compass, Light, Camera, etcInclude Ketai Library

import ketai.sensors.*; KetaiSensor sensor;

Using Sensors Setup in Setup( ) function

sensor = new KetaiSensor(this); sensor.start();

sensor.list(); Event based sensor reading

void onAccelerometerEvent(…)

{

accelerometer.set(x, y, z);

}

Sensor Demo

PhoneGap - http://phonegap.com/

Developing mobile apps using web tools HTML, CSS, JavaScript

Access to sensor data Camera, compass, accelerometer

Cross platform iOS, Android, Windows Phone, etc

WearScript – wearscript.com

JavaScript for Glass Access to sensors, touch input, etc

Power of Android, Ease of website development

Design Guidelines

"Computing should just be more comfortable"

"Google should do the hard work, and you should have a chance to live, have a good life, and get on with it."

As technology becomes more personal and immediate, it can start to disappear.

Distant Intimate

Last year Last week NowForever

The Now machine

Focus on location, contextual and timely information, and communication.

1. Design For the Device

Simple, relevant information Complement existing devices

2. Don’t Get in the Way

Enhance, not replace, real world interaction

3. Keep it Relevant

Information at the right time and place

4. Avoid the Unexpected

Don’t send unexpected content at wrong times Make it clear to users what your glassware does

5. Build for People

Use imagery, voice interaction, natural gestures Focus on fire and forget interaction model

Micro Interactions

The position of the display and limited input ability makes longer interactions less comfortable.

Using it shouldn’t take longer than taking out your phone.

It's like a rear view mirror

Don't overload the user. Stick to the absolutely essential, avoid long

interactions. Be explicit.

Examples

Virtual Exercise Companion

GlassFitGames http://www.glassfitgames.com

GlassFitGames Video

CityViewAR

Using AR to visualize Christchurch city buildings 3D models of buildings, 2D images, text,

panoramas AR View, Map view, List view Available on Android market

CityViewAR on Glass

AR overlay of virtual buildings in Christchurch

CItyViewAR on Glass Demo

Conclusions

Conclusions Rapid prototyping vital for wearables

Display concepts, user feedback Range of different tools available

Sketching, storyboarding, wireframe, interactive

Need to follow design guidelines Don’t get in the way, build for people

Books Programming Google Glass

Eric Redmond

Rapid Android Development: Build Rich, Sensor-Based Applications with Processing Daniel Sauter

Building Android Apps in Easy Steps: Using App Inventor Mike McGrath

Web Resources Main Developer Website

https://developers.google.com/glass/ Glass Apps Developer Site

http://glass-apps.org/glass-developer Google Glass Emulator

http://glass-apps.org/google-glass-emulator AR for Glass Website

http://www.arforglass.org/

More Information Mark Billinghurst

Email: mark.billinghurst@hitlabnz.org Twitter: @marknb00

HIT Lab NZ http://www.hitlabnz.org/