InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern...

34
InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, Western University, London Ontario, Canada

Transcript of InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern...

Page 1: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

InkChatStephen M. WattWestern University

Teaching with Technology21 May 2015, Western University, London Ontario, Canada

Page 2: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Teaching and Learning areCollaborations

Page 3: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Collaborative Software

Page 4: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Common Features

• Slide shows• Whiteboarding• Voice chat• Video chat• Image capture

Page 5: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Technical Content

Page 6: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Technical Content

Missing:• Mathematics• Diagrams, graphs• Geometric figures• Technical knowledge base• Document markup

• Scientific software connections (Maple, Mathematica, GeoGebra, R,…)

Page 7: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Technical Content

Isn’t a shared whiteboard, with the ability to save images enough????

Page 8: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Technical Content

No!

Page 9: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

The Treachery of Images(La trahison des images)

Page 10: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.
Page 11: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Einstein’s Blackboard

• Einstein to receive honorary doctorate at Oxford, May 1931.

• Lecture at Rhodes House.

• Board retrieved and preserved by Edmund (“Ted”) Bowen.

• Nice to look at, but content is trapped.

Page 12: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Digital Ink

• Location, time information, sometimes also pressure and angles.

• Capture online pen strokes, not images.

• Suitable for• Recognition algorithms• Semantic grouping • Annotation• Manipulation: search, transformation, archival.

• Problem: Multiple vendor-specific formats.

Page 13: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.
Page 14: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Pen-Based Math

• Input for CAS and document processing.• 2D editing.• Computer-assisted collaboration.

Page 15: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Pen-Based Math•Different than natural language recognition:

• 2-D layout is a combination of writing and drawing.• Many similar few-stroke characters.• Many alphabets, used idiosyncratically.• Many symbols, each person uses a subset.• No fixed dictionary for disambiguation.

Page 16: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

The Research

Page 17: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

The Software

Page 18: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Prior Generations

• 2002 Pocket PC:

Page 19: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Prior Generations

• 2002-2008 Tablet PC:

Page 20: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Prior Generations

• 2008-2013 Java Application:

Page 21: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

InkChat (Java Version)

• Skype and GTalk add-on to the Java application.

Page 22: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Problems

•Requires installation:• Big hassle for someone to use only once in a while or on all their

machines.

•Limited portability:• Users expect versions on Android, iOS, Windows,

Mac OSX, Linux, etc… • Incompatible software bases• Flakey, moving APIs

•Need to support multiple devices.• Nowadays a single user will want to work across many devices.

Page 23: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Solution

•Use browser infrastructure.

• JavaScript is not a great language for large projects, but…..• It is ubiquitous: Telephones, tablets, laptops, …• Libraries for many UI elements• Our new recognition algorithms are fast enough

• Rapid development:• Prototype developed in 3 months by 3 students.

Page 24: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Current Generation

Desktop Telephone

Page 25: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Simple Interface with device-adapted menus

Page 26: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Ink Controls

Page 27: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Collaboration:Different Viewports from Different Devices

Page 28: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

CollaborationDocument Annotation Pointers for Discussions

Page 29: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Embedding in Other Environments

Page 30: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Cloud Integration

• Save or load files to cloud storage• DropBox• Google Keep• Others possible

• Previous work to store user profiles• Save cloud of ground-truth labelled symbols

(corrected/accepted)

• Future work to store user-defined brushes

Page 31: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Application Web Site

Page 32: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Application Web Site: Found Objects

Page 33: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Conclusions• Teaching technical subjects requires new tools.• Drawing, mathematics and technical text is required.

• The treachery of images.• Needed:

• Math handwriting recognition.• Easy geometry and diagrams.

• Even a little goes a long way….• … there is a lot of opportunity for future development.

• Document mark up.• APIs to scientific software.

Page 34: InkChat Stephen M. Watt Western University Teaching with Technology 21 May 2015, W estern University, London Ontario, Canada.

Bruce CharJoseph Choi

Michael FriesenOleg Golubitsky

Rui HuVadim Mazalov

Shirley Miao

MaplesoftMicrosoftMITACSNSERC

Thanks

Jeliasko PolihronovMaya Ramamurthy

Elena SmirnovaClare So

Stephen SolisCoby Viner

James Wake