Multimedia Specification Design and Production

2013 / Semester 2 / week 8Lecturer: Dr. Nikos Gazepidisgazepidis@ist.edu.gr

2

Outline

Human Computer Interface.

Multimedia interfaces and applications.

Vision based interfaces Systems using vision based interfaces. Some basic methods in vision interfaces

Background subtraction. Color blob detection and tracking. Face detection (Viola and Jones face detector).

Multimedia and HCI

3

Human Computer Interface

Different meanings of HCI:

1. The study of people, computer technology and the ways they interactive with each other.

2. The design, evaluation and implementation of computer systems for human use.

3. The research to study how to make computer systems more use-able.

Multimedia and HCI

4

Aspects of HCI

DesignIntuition, experiences and design principles.

Human factorsExperiments testing the perception, memory and cognition models of human being.

DevicesPhysical devices such as mouse, keyboards and other fancy HCI hardware.

Software and systemsSuch as windows GUI, shell commands in Unix etc.

Multimedia and HCI

5

HCI Studies

ComputersWhat are they good at and bad at?Input, output, processing, networking, etc.

PeopleWhat are people good at?What is the limitation of human abilities?Input, output, memory, perception and cognition model.No average answer. Must accommodate diversities.

UsabilityMore than the size of widgets, where are the buttons and syntax of commands.

ContextEnvironment, physical constraints, cognition constraints, task, etc.

Multimedia and HCI

6

The quality of HCI

There are fives factors that affect the design and implementation of HCI

Time to learn. Speed of performance. Rate of errors. Retention over time. Subjective satisfaction

Multimedia and HCI

7

Multimedia Interfaces

Multimedia interface combines different sorts of media modalities to achieve more natural and more efficient human computer communication.

Speech, audio, touch, video input (patterns, posture, gesture) etc.

“Put that there” is an example multimedia HCI that combines speech, gestures and graphics

Multimedia and HCI

Put that there, MIT 1981http://www.media.mit.edu/speech/sig_videos.html

8

Advantages of Multimedia Interface

More alternative methods. The flexibility of combining different schemes. More efficient for specific tasks. Greater precision of information. Help handicapped people to use computers

Multimedia and HCI

http://www.youtube.com/watch?v=d7eGypGOlOc&feature=related http://www.youtube.com/watch?v=Gnq-8iaOcXk

Pen gesture interface Virtual Reality

9

Vision Based Interfaces Vision based interfaces uses computer vision techniques to achieve more effective HCI.

Vision based interfaces are based on techniques such as:

Multimedia and HCI

Hand tracking,Hand gesture,Arm gestureHand Pointing

Face detectionHead trackingGazeExpressionLip reading

Posture, body gesture and action recognition

10

Example Systems

Multimedia and HCI

MotionCapturing

11

Example Systems (cont.)

Multimedia and HCI

The MIT Kidsroom

Posture recognition

Actiondetection

12

Methods in Vision Interfaces

Multimedia and HCI

Background subtraction

Estimating the background B(n) = B(n)*alpha + I(n)*(1-alpha)

Foreground Map = |Image – Background|>threshold

alpha is a positive floating point number close to 1.

13

Mean Shift Tracking Result

Multimedia and HCI

http://www.caip.rutgers.edu/~comanici/Papers/KernelTracking.pdf

14

Viola and Jones face detector

Multimedia and HCI

The Viola – Jones object detection framework is the first object detection framework to provide competitive object detection rates in real-time proposed in 2001 by Paul Viola and Michael Jones.

Although it can be trained to detect a variety of object classes, it was motivated primarily by the problem of face detection. This algorithm is implemented in OpenCV as cvHaarDetectObjects().

15

Viola and Jones face detector (cont.)

Multimedia and HCI

These features can be computed using integral images similarly to the color histograms discussed before.

16

Summary

Multimedia and HCI

We only touch the very surface of multimedia interfaces.

Currently, the dominant HCI is still GUI with keyboard and mouse.

Multimedia interfaces are beginning to attract more and more interests.

We need more robust and efficient media recognition schemes to support effective multimedia interfaces.