Q: A Tactile ApproachTo Digital Organization

COMP 410December 2009

COMP 410 – What is it? Semester-long project “Customer” poses a challenge Student-run Dr. Wong and 4 TAs Warmup project: BallNet

Resources – Sharepoint

TFS + Visual Studio

Resources – Development Servers

Our Team

Nick BridleInput

Jaewoo JeonClient

Lee MarinelliServer, Input

Rick ManningServer, Client

Kevin MontroseServer, Plugin

Seth O’BrienClient

Jim Russell3D Interface

Ivan VanServer

Our Organization

Every two weeks we: Presented new prototype to customer Reassessed progress Set goals for next two weeks

Advantages: Regular schedule More upfront progress

The Challenge

Search is useful, but users need a way to structure information.

Our Solution

Workspaces Multi-user collaboration Categories, Tags, and Links

Gestures!

Demonstration

System Overview – Modules

Server

GestureInput

Plugin Client

The Server

Server

GestureInput

Plugin Client

Server – Architecture

Workspace

Workspace

Workspace

Workspace

Task Queue Task Queue Task Queue Task Queue

Server Worker Threads

HTTP Communications Layer

Server – HTTP Communication HTTP Requests

GET to get data POST to send data

JSON encoding used for structured queries:

Server – Concurrency

Many-reader, single-writer Number of threads proportional to

number of processor cores

Server – Workspace Instances Each instance manages:

Document content Document metadata Full-text indexing Version tracking Lock tracking

Resource Desc. Framework

Semantic Web Triples:

Subject Predicate Object

Easy to represent many different types of relationships

“Tagged with”

www.Q-is-the-best.com

“COMP 410”

The Plugin

Server

GestureInput

Plugin Client

Plugin – Overview

Extend the browser, rather than replace it

Familiar interface for keyboard and mouse

User “shares” items from the browser, optionally specifying title, category, and tags

Plugin – Design Firefox extension

Well-exercised platform

Lots of flexibility XUL for UI Javascript for logic

Plugin – The Guts

Wraps all requests into JSON objects Data transmitted as base-64 encoded

strings Tracks browsing history on a per-tab

basis Tracks user actions like reload,

back/forward Browsing histories are non-linear!

The Client

Server

GestureInput

Plugin Client

Client – Overview

Browser-hosted application Windows Presentation Foundation

Reference implementation Wide variety of clients could use same

server Platform independence!

Client – Design

Scalability

Download data only as needed Workspace structure Thumbnails Full documents

Demonstration

The Input

Server

GestureInput

Plugin Client

Input – Motivation

Interesting Substitute for traditional keyboard and

mouse Uses Wiimotes!

Intuitive Meshes well with 3D view

Flexible Doesn't restrict the user to one input

interface

Input – Inspiration

Input – Physical Setup

2 Wiimotes on tripods Gloves with IR LEDs on the fingers

Wiimote Stereovision

Like depth perception in humans Calibrate with cube

Human w/glove

s

MonitorWiimote Wiim

ote

Input – Demonstration

Gesture Recognition

Project points in 3D space to two 2D planes

Apply 2D recognizers

Custom Gestures

Pre-existing libraries used to analyze strokes

Super Annoying ProblemsInteresting Challenges Interfacing different platforms

Browser/client communication Client/input communication

Firefox Poor documentation No official API for extensions!

Hardware (Wiimotes)

Future Improvements

Security HTTPS SQL queries

More 3D layouts Improved gesture fidelity Improved gloves

Reflected IR approach

Acknowledgements

Questions