Pres ented by

8ninths HoloLens Design Patterns

Version 1.0July 2015

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 2

Introduction

A new medium brings with it a new palette, new design possibilities, new composition methods, and an invitation to create defining new experiences.

HoloLens Design Patterns is a design research document intended to serve as a dynamic foundation for the ground-breaking, in-depth, iterative processes that will forge the first generation of HoloLens experiences. This is a living document that will grow and evolve as we continue to explore and discover what this phenomenal new technology is capable of. In its initial iteration, four foundational aspects are represented:

A Table of Elements defines the palette for composing HoloLens experiences, delineating elements in the categories of Physical Space, User Input, Holographic Form, and 3D Sound.

Mixed Reality Design Methods identifies pivotal creative strategies for combining the elements and composing mixed reality experiences with HoloLens.

Case Studies analyzes early Microsoft HoloLens experiences using the Mixed Reality Design Methods and Table of Elements.

An Example Flow Diagram applies the language of the HoloLens Design Patterns to a moment-by-moment flow analysis of a HoloLens experience.

Building upon the foundation of the Design Patterns, 8ninths is currently in the process of applying this emerging vocabulary and set of compositional strategies to blueprints for new HoloLens experiences.

Physical Space

Walls

Tabletops

Ambient Surfaces

Ambient Space

Floor

Ceiling

User Input

Gaze

Gesture

Voice Commands

Mouse & Keyboard

Tablet

Locomotion

3D Sound

User Interaction Feedback Audio

Audio-Cued Spatial Navigation

Ambient Sound

Object-Specific Sound

Spatialized 3D Sound Composition

User Dialogue

Holographic Form

3D Animation Using Ambient Space

3D Objects

3D Structures

3D Environments

3D People, Animals, and Characters

3D Data

2D Data

Holographic UI Elements

Holographic Avatars

3D Animation Using Spatial Mapping

Holographic Video

2D Video

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 3

Table of Elements

Using holographic form to transform the function and visual perception of surfaces in a physical space.

Translating data traditionally constrained within a 2-dimensional screen into 3-dimensional physical space.

Composing the holographic layer in a physical environment as an immersive environment to explore and interact with.

Moving information from screen into space – seamless integration between monitor or tablet and holographic expression of its contents. Also encompasses 3D printing of holographic objects.

Playful engagement with the holographic layer, including shooting, dodging, chasing, catching, following, finding hidden objects, etc.

Transforming Surfaces

Spatializing Data

Mixed Reality Immersion

Screen to World

Mixed Reality Play

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 4

Mixed Reality Design Methods

Superimposing annotation or other expressions of information onto objects and structures in a physical environment.

Collaboratively viewing and interacting with shared spaces, including remote annotation of a physical environment.

Building objects holographically, encompassing the intent to export them into the physical world.

Using a holographic object as a collective focal point for collaborative work, dialogue, learning, etc.

Information Overlay

Annotating Shared Spaces

Holographic Building

Holographic Collaboration

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 5

Mixed Reality Design Methods

Physical Space

+

+

+

+

User Input

3D Sound

Holographic Form

Mixed Reality Design Methods

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 6

Formula

Physical Space

Walls Ambient Space

GazeGestureVoice CommandsMouse & KeyboardLocomotion

3D Environments3D DataHolographic AvatarsHolographic UI Elements

User DialogueUser Interaction Feedback Audio Audio-Cued Spatial Navigation

Mixed Reality Design Methods

Spatializing DataScreen to WorldMixed Reality ImmersionInformation OverlayAnnotating Shared Spaces

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 7

Case Study – NASA OnSight

HOLOLENS MIXED REALITY DESIGN PATTERNS

Physical Space

Walls Ambient SpaceAmbient Surfaces

GazeGestureVoice CommandsMouse & KeyboardTablet

2D Data3D Animation Using Spatial Mapping2D VideoHolographic UI Elements

User DialogueUser Interaction Feedback Audio Audio-Cued Spatial Navigation

Mixed Reality Design Methods

Spatializing DataScreen to WorldInformation OverlayAnnotating Shared Spaces

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 8

HOLOLENS MIXED REALITY DESIGN PATTERNS

Case Study - HoloNotes in Skype

Physical Space

Walls TabletopsAmbient SurfacesFloor

GazeGestureVoice Commands

3D Objects3D Structures3D Environments3D Characters3D Animation Using Spatial Mapping

User Interaction Feedback AudioAmbient SoundObject-Specfic SoundUser Dialogue

Mixed Reality Design Methods

Transforming SurfacesScreen to WorldMixed Reality ImmersionMixed Reality Play

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 9

HOLOLENS MIXED REALITY DESIGN PATTERNS

Case Study – HoloLens Minecraft

Physical Space

Walls TabletopsAmbient SurfacesAmbient Space

GazeGestureVoice CommandsLocomotion

3D Data2D Data2D VideoHolographic UI Elements3D Objects3D Animation Using Spatial Mapping

User InteractionFeedback AudioObject-Specfic Sound

Mixed Reality Design Methods

Transforming SurfacesSpatializing DataInformation Overlay

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 1 0

HOLOLENS MIXED REALITY DESIGN PATTERNS

Case Study – HoloLens Home Scenario

Physical Space

Walls TabletopsAmbient Space

GazeGestureVoice CommandsMouse & Keyboard

3D Data2D Data3D Objects3D Structures3D EnvironmentsHolographic UI ElementsHolographic Avatars

User Interaction Feedback AudioUser Dialogue

Mixed Reality Design Methods

Spatializing DataScreen to WorldInformation OverlayAnnotating Shared SpacesHolographic CollaborationHolographic Building

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 1 1

HOLOLENS MIXED REALITY DESIGN PATTERNS

Case Study – HoloLens Trimble Project

Physical Space

TabletopsAmbient Space

GazeGestureVoice CommandsMouse & Keyboard

3D Objects3D StructuresHolographic UI Elements3D Animation UsingAmbient Space

User Interaction Feedback AudioAmbient Sound

Mixed Reality Design Methods

Spatializing DataScreen to WorldMixed Reality PlayHolographic Building

User Input 3D SoundHolographic Form

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 1 2

HOLOLENS MIXED REALITY DESIGN PATTERNS

Case Study – Holo Studio

Screen to World

Information Overlay

TRANSITION

Gaze + air tap to select physical bike chassis.

User Input: Gaze

Physical Space:

Ambient Surfaces

Render virtual bike body and fuel tank onto physical chassis.

Holographic Form:

3D Objects

Gaze + voice command to select fuel tank (hypothesis).

Gesture to resize fuel tank. Feedback audio (hypothesis).

User Input: Gesture

Interaction Feedback AudioUser Input: Gesture

User Input: Gaze

User Input: Voice

Interaction Feedback Audio

Starting Assumptions:Room is scanned. Surfaces are spatially mapped.

Display bike on 2D monitor. Drag bike off screen to table surface.

User Input: Mouse

Physical Space: Tabletop

Display table model bike hologram.

Holographic Form:

3D Objects

Display hologram manipulation tools.

Holographic Form:

Holographic UI Elements

Mouse click to collapsehologram (hypothesis).

User Input: Mouse

Interaction Feedback Audio

HOLOLENS DESIGN PATTERNS

8 N I N T H S T h i s wo r k i s l i c e n s e d u n d e r a Cr e a t ive C o m m o n s At t r i b u t i o n - S h a r e A l i ke 4 . 0 I n te r n a t i o n a l L i c e n s e . 1 3

Example Flow Diagram Based on HoloLens Motorcycle Video

Pres ented by

Thank Youhttp://8ninths.com

partners@8ninths.com