2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning

Tangible Objects for E-Learning

Abdulmotaleb El Saddik

FIEEE, FCAE, FEIC

At a glanceCreation of Advanced Multimedia LO

Protection of LO

Navigation of LOR by VR metaphors

Adaptation of LO according to user context/profile & network QoS

LO Delivery to user

Introduction

Haptics Derived from the Greek verb “haptesthai” Refers to sensing and manipulation through touch Comprises both “Tactile” and “Kinesthetic” senses

Applications Education and Training Medicine: Tele-surgery, rehabilitation, etc. Entertainment Arts and Design …

Multimedia and Authoring

Multimedia contents Graphic images 3D models Audio and video files And recently haptic stimuli

Multimedia authoring tools Integrate the disparate media elements into a

cohesive multimedia application

Multimedia Authoring Tools

A major challenges in multimedia authoring is to make authoring complex multimedia titles as easy as using a word processor or drawing program

We propose a multimedia authoring tool for hapto-visual applications to make the development process a non-programming experience

HAMLAT

Blender: Open source Full-fledged 3D graphical renderer Physics and game engine Adaptive user-interface

Plug-ins incorporated in Blender source code Haptic panel in Blender GUI Haptic rendering extension Implemented in C++ using OpenHaptics and OpenGL

HAMLAT Implementation (demo)

A Multimedia Handwriting Learning and Evaluation Tool

Learning sensorimotor skills is difficult with graphical or auditory feedback

Sense of touch will be a must for physical guidance

Examples of physical guidance learning: Handwriting for visually impaired people Medical procedures and rehabilitation training Painting/sculpting techniques

Multimedia Learning Tool

Distinguished features Language choice: Arabic, French, English, Japanese

and Spanish

Multimodal: haptic, visual, and audio information for

each letter

Learning mode choice: full or partial guidance, or test

mode.

Evaluation Mode: quantitative evaluation using dynamic

time warping

Multimedia Learning Tool

Learning Mode -1-

Language Repository Including symbols images,

pronunciation files, and the

haptic stimuli of each letter

(using XML-based schema)

<?xml version="1.0" encoding="UTF-8" ?> <Language Name="Japanese_Katakana" Flow="left" xmlns="urn:language">

<MetaData> <AuthorName> Steve </AuthorName > <ContactInfo> [email protected]

</ContactInfo> <Copywrite> LGPL </Copywrite> <CreationDate> 06-05-2007

</CreationDate> </MetaData> <HapticDevice>

<DeviceName> Phantom </DeviceName> <Workspace> 160X120X0 </DeviceName> <MaxForce> 3.0 </MaxForce> <DOF> 3 </DOF>

</HapticDevice> <Alphabets>

... <Item Name="ka" /> <Item Name="ki" /> <Item Name="ku" /> <Item Name="ke" /> <Item Name="ko" /> <Item Name="sa" /> ...

</Alphabets> </Language>

Learning Mode -2-

Haptic player: Three modes of

operation: No guidance

Partial guidance

Full guidance

Evaluation Mode -1-

Test GUI

Evaluation Mode -2-

Dynamic Time Warping Measure the similarity between

test and reference handwritings

Tolerate both global and local shifts in the time domain

Startup position of the test pattern is shifted to that of the reference pattern

Given two time series// xi is a point in the test trajectoryX = x0, x1, …, xN-1 // yi is a point in the ref. trajectoryY = y0, y1, …, yT-1

D(xi,yi) = sqrt((abs(xi)-abs(yi))^2 + (ord(xi)-ord(yi))^2)DTW[0,0] = D(x0,y0)

For i=1 to T-1 DTW[0,i] = D(x0,yi) + DTW[0,i-1]For i=1 to N-1 DTW[i,0] = D(xi,y0) + DTW[i-1,0]

For i=1 to N-1 For j=1 to T-1DTW[i,j]=D(xi,yj)+min{DTW[i,j-1],DTW[i-1,j],DTW[i-1,j-1]}

Return (DTW[N-1,T-1])

Performance Evaluation -1-

Experimental Setup The Phantom Omni Device Application is developed with .NET 2.0 Framework and

programmed using C#. Bipolar grading scheme (Pass/fail) Threshold distance is computed based on subjective

analysis Three threshold values were estimated:

Difficult Medium Easy


The DTW distance for the “Tha” Arabic character


The DTW distance for the “i” Japanese character

Performance Evaluation -4-(Demo)

The DTW distances for five Japanese characters

CharacterThreshold Distance (x103 mm)

Easy Medium Difficult

Sa 5.5 2.5 1

Shi 4 1 0.5

Chi 5 2.5 1

Tsu 4 1.5 1

Na 10 6 1


Protection of LO



LO Delivery to user

Haptic signal prediction, compression, and watermarking

Haptic data stream

Compressed bitstream

Haptic Compression

Encoder

Adaptive Golomb-

Rice Coding

Code Table

Run Counter

Run-length Coder

Bitstream Formatting

Entropy Coding

8-samples Blocks

along x, y and z

F-DCT

Uniform Scalar

Deadzone Quantization

Simple Differential Predictive

Coding

Haptic Data Acquisition and Pre-

processing

Watermark haptic signals to protect copyright and authenticate content.

Case Study: Identifying Human Pattern with Haptics

The Experiment

Methodology

Dynamic Time Warping :

+ Nelder-Mead non-linear minimization

Spectral analysis: Fast Fourier Transform

Unsupervised Method: K-Means

3

1c

N

1i

2p2c,i

1c,i tldMS

Graphic Representation

Trial Timestamp Position X Position Y … Force X

( N)

1 0 0.23344 0.56768 0.00456

1 … .. … …

1 0.0123090 0.37676 0.98976 0,03767 Data

Verifying such feasibility (demo)

Virtual Check

Virtual MobilePhone

Performance of Classifier

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

FAR error rate

Virtual Check Maze Virtual Phone

Ambient Intelligent Engine

Performance


Protection of LO by Digital Watermarking



LO Delivery to user

Navigation

Tangible Objects: Demo1

Haptics: Demo 2

Mobile Phone: Demo 3





LO Delivery to user

E-Learning courses

Toddler Interfaces

Toddler Interfaces Main idea

Whenever a child utters an object’s name or taps on an object, a set of appropriate multimedia representations is displayed (i.e. images, Multi-lingual texts, audio…)

Techniques investigated Voice-based technique (Speech detection) Magic Stick technique (RFID and Bluetooth technologies)

Evaluation Several games were developed: i.e., Book game, Alphabet Learning, and

Object Identification

Alphabet Learning Evaluation

12 children, ages 4-5, evaluated the system

We Observed the following:

Children’s discussions

Their Writings Their Questions Simplicity of

using the Magic Stick.

Demo 1

Demo 2





LO Delivery to user

Clustering and 3D Mapping

a) MST tree b) Graph construction

T

CT1

CT3

CT2

CT4

CT5

T

CT1 CT2

CT5CT4CT3Semantic keyword metrics MST tree

Yahoo, Google Web-services

P2PDirectory

Remote Repository

Keyword terms1. Query Parsing and Search

2. Analyze each of the result links

Xml results3. Determine the high ranked keywords

4. Commit search with first n high ranked keywords

PFs

5. Calculation of the relevance between the high ranked keywords

6. Grouped search result Xml results

Overall clustering process design

Advanced search options

Sharing LO paramers

Searching/Sharing Learning Objects

Group interaction menu. Comprising options viz:a) Search related b) Save allc) Share alld) Make parent

Group sign

Dynamic road for each group of search results

Dynamic navigation layout

Sample Interface: Dynamic Traffic Layout

Multimodal Interaction (demo)

Search Result Clustering:A) KeywordB) User tagC) Term frequency

Remote Repository

Web-Search: Yahoo/Google

P2P DirectoryList

[ title ]

(a)

(b)

(c)

(e)

(d)

Multi-modal Interactions

Visualization metaphors Information organization Information repository

[ title ]

At a glance

Creation of Advanced Multimedia LO




LO Delivery to user

Last thoughts

Acknowledgment

October 4, 2005 متشکرم

2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning

Education

Transcript of 2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning