Lec02, Introduction, v1.08.ppt -...

Multimedia Systems

Introduction

Mahdi Amiri

September 2015

Sharif University of Technology

Course Presentation

Multimedia Systems, Introduction

Multimedia System

Review course website resources

Announcements, Password, Handling large files

Previous session


Multimedia System

Text

SMS address.

Describing Feelings and Emotions.

Drawing

Let me see the bar chart for this data table.

Audio

Can’t chat, I have to call you.

Still image

See to believe.

Video

Wonder how it works.

Media

Also inside above categories: Graphs, Maps,

Animations, 3D Objects or Models, Diagrams, etc.


Media

Ascii art

Early printers often lacked graphics ability.

Ultrasonography

Visualizing subcutaneous body structures.

Spectrogram

Audio signal analysis.

Magnetic Resonance Imaging (MRI)

Visual Representation


Media

We have well matured acquisition and representation

methods available for these two senses.

Sense of Sound and Sight


Media

Haptics

The sense of touch. It is a form of nonverbal

communication.

Hugging, Shaking hands, Vibration, Tickling.

Haptics are enabled by actuators that apply forces

to the skin for touch feedback, and controllers.

Common Haptic Feedback: Cause the handlebars

to vibrate during a collision with another vehicle in

racing video games.

Now are gaining widespread acceptance as a key

part of Virtual reality systems.

Sense of Touch


Media

Example Haptics: Tactus Technology

Sense of Touch


Media

Example Haptics: Apple “Taptic Engine”

Sense of Touch

Apple created what it calls a “taptic engine”

to deliver physical sensations to your wrist.

According to the company:

"The Taptic Engine creates a discreet,

sophisticated, and nuanced experience by

engaging more of your senses. It also

enables some entirely new, intimate ways for

you to communicate with other Apple Watch

wearers. You can get someone’s attention

with a gentle tap. Or even send something as

personal as your heartbeat."


Media

Example Haptics: Apple 3D Touch

Sense of Touch

New gestures introduced with 3D Touch —

Peek and Pop — let you view contextual-

style menus from your home screen icons,

allowing access to the features within.

Peek,lets you preview information while Pop

moves you inside the app for additional actions.

Peek example: With Mail, you can press on an email to

get a preview of the message — and lift up your finger if

you want to return to your inbox untouched.

How does this happen? Capacitative sensors are

integrated into the backlight of the phone’s Retina

HD display so that each touch measures

“microscopic changes” in finger pressure between

the glass and the backlight.Ref.: www.forbes.com/sites/jvchamary/2015/09/12/3d-touch-iphone-6s/


MediaSense of Touch

Strain gauge: A strain gauge (or strain

gage) is a device used to measure strain on

an object.

Foil strain gauges: An excitation voltage is

applied to input leads of the gauge network,

and a voltage reading is taken from the

output leads. Typical input voltages are 5 V

or 12 V and typical output readings are in

millivolts.

Ref.: en.wikipedia.org/wiki/Strain_gauge

Visualization of the working concept

behind the strain gauge on a beam

under exaggerated bending.


Media

Example Haptics: LDWS

Sense of Touch

Lane Departure Warning System (LDWS)

Ref.: http://mashable.com/2012/08/05/haptic-car-technology/

Turn when the steering wheel tells you

(AT&T Labs Research)

Hyundai Genesis offers a Lane Departure Warning System (LDWS). Whenever the system is activated,

the haptic steering wheel will provide a vibration warning and dial-integrated warning light to alert

drivers of their lane departure, allowing them to reposition the car smoothly into the correct position.

LDWS in Hyundai Genesis 2015

LDWS in Genesis 2012

LDWS module

and sample video

Note: LDWS operates when vehicle speed exceeds 43 miles per hour.


Media

Methods: All computerized scent output systems work in the same

way: the computer, be it a full-featured desktop machine or a simple

embedded chip, sends a signal out through a serial or parallel port to a relay,

which turns on the output device itself for a designated period of time.

Olfaction, The Sense of Smell

Nasal Ranger, an

olfactometer, in use.

An olfactometer is an instrument used to detect and measure ambient odor dilution. Olfactometers are

used in conjunction with human subjects in laboratory settings, most often in market research, to

quantify and qualify human olfaction (en.wikipedia.org/wiki/Olfactometer).

Scentography is the technique of creating and storing odor by

artificially recreating a smell using chemical and electronic means.

iSmell Personal Scent Synthesizer, one of

the "25 Worst Tech Products of All Time"

by PC World Magazine.

How the Nasal Ranger Works

Ref.: St. Croix Sensory, Inc., www.fivesenses.com


MediaOlfaction, The Sense of Smell

Madeleine, a prototype of a desktop device to record aromas (Jul. 2013).

www.amyradcliffe.co.uk

en.wikipedia.org/wiki/Scentography

Amy Radcliffe


MediaThe Sense of Smell

scentee, Smart Phone Smell Attachment.

Ref.: scentee.com


Media

Sight, Hearing, Touch, Smell, and now Taste!

Sense of Taste

Led by Nimesha Ranasinghe, researchers at the National University of Singapore have cooked up a

tasty new technology: a digital simulator that translates the taste of virtual food and drink to the tip

of the user’s tongue.review information while Pop moves you inside the app for additional actions.

Ref.: www.nimesha.info

Digital Taste Interface

Nimesha Ranasinghe

Or Digital lollipop

Also see: Taste, gustatory perception, en.wikipedia.org/wiki/Taste


Media

Example: Tellspec

Sense of Taste

Ref.: TellSpec.com

TellSpec: uses a spectrometer

to analyze the ingredients of

any dish we aim the device at

This information is transmitted to an application on our mobile and a cloud service developed by the manufacturer analyzes the data and offers

diverse conclusions. TellSpec allows us to know what any food is made of and it also provides us with nutrition facts like the calories of a certain

dish. It is even able to detect ingredients or elements that may be harmful to our health, for example, if we are allergic to certain types of food.


MediaAll of the Senses

Try and you will find more new

examples online.What about sixth sense?


Multimedia System

Computer controlled integration of

different media

Text Audio Graphic

Drawing Still image Video

Haptics Animation

Every other type of

data representation forms

Multimedia…


Multimedia System

Where multimedia used to be … digitally.

Processed

Generated

Represented

Stored

Transmitted

Retrieved

Delivered

System


Multimedia System

HyperText

A text which contains links to other texts

Ted Nelson, 1965

Non-linear traversal

HyperMedia

Can include other media too

Text, HyperText, …, HyperMedia


Multimedia Systems

WWW

PowerPoint

Adobe Acrobat

iSilo

Text compression, Hyperlinks,

Images, Tables, Formatted text

ePUB

Free and open e-book standard

HyperMedia Examples

Tim Berners-Lee

first proposal for www: 1989


Multimedia Systems

HTMLNo version management, no rights management.

Project XanaduA word processor capable of storing multiple

versions, and displaying the differences between

these versions.

Find more at: http://transliterature.org/

ZigZagData Model

, Database and

visualization system

HyperMedia: True generalization!

Ted

Nelson

Computer documents are not just

to SIMULATE paper. They should

allow us to improve on paper. Ref.: xanadu.com/zigzag


Multimedia SystemsHyperMedia: What is ZigZag?

Ref.: gzigzag.sourceforge.net/nutshell.html

ZigZag (or ZZstructure) is a way of representing the structure of information. Compared with

the previous ways, ZigZag is very different, for example the concepts of 'file', 'folder' and

'application' are abandoned.

A ZigZag structure consists of cells and dimensions. A cell is the basic unit of information of

a ZigZag structure. Cells containing related information can be connected with each other

along dimensions, the number of which is unlimited. A ZigZag structure is separate from its

visualisation (= the way the data is presented on the screen), which means that a ZigZag

structure can have many visualisations designed for different purposes.

Every piece of information stored in a digital device using based on ZigZag is in the same

space: the same cells can be connected on several dimensions created for different structures.

For example, the cells containing the names of one's relatives can be connected on

dimensions created for a family tree structure, an address book structure and a photo album

structure.


Multimedia SystemsHyperMedia: Linked Data

Ref.: en.wikipedia.org/wiki/Linked_data

Describes a method of publishing structured data so that it can be interlinked and

become more useful through semantic queries. It builds upon standard Web

technologies such as HTTP, RDF and URIs, but rather than using them to serve web

pages for human readers, it extends them to share information in a way that can be

read automatically by computers. This enables data from different sources to be

connected and queried.

The Resource Description Framework (RDF) is a

general method for conceptual description or

modeling of information that is implemented in

web resources, using a variety of syntax notations

and data serialization formats.


Multimedia SystemsLinked Data: Google Knowledge Graph

Ref.: en.wikipedia.org/wiki/Knowledge_Graph

The Knowledge Graph is a knowledge base used

by Google to enhance its search engine's search

results with semantic-search information gathered

from a wide variety of sources.

Another example is Facebook Open Graph

protocol.


Multimedia Systems

Computer controlled

IntegratedSeamless

Data are represented digitally

Interactive interfaceEnd-user is able to control the elements of media

Non-linear

Characteristics


Multimedia Systems

Digital representation

Analog to digital conversion, sampling.

Large data requirements

Storage, compression.

Distributed networks

Bandwidth, QoS.

Maintaining temporal relationship between data

Sequencing (playing frames in correct order/time frame in video).

Synchronization (inter-media scheduling, e.g. Lip synchronization).

Challenges


Multimedia Systems

Hardware

Very high processing power.

Efficient and high performance I/O.

Large storage and memory units.

Network support.

Software

Multimedia capable file system.

Special operating system.

User friendly software tools:

Easy to use, Easy to understand.

Network support.

Desirable Features


Multimedia Systems

Capture device

Video camera, Audio microphone, Keyboard

, Mouse, Tablet, OCR, different sensors.

Storage device

Hard-disk, CD-ROMs, DVD-ROM.

Communication Network

LAN, Intranets, Internet.

Computer System

Multimedia desktop machine.

Display Device

Speaker, Monitor, Printer.

Components


Multimedia Systems

World Wide Web.

Video conferencing.

Interactive TV, Video-on-demand.

E-learning, Hypermedia courseware.

Home shopping.

Games, Virtual reality.

Digital video editing and production systems.

Collaborative software , Groupware.

Edutainment, Infotainment.

Applications


Multimedia Systems

Format

Raw text, Rich text format.

Bitmap graphics (Image raster), Vector graphics.

Time independent (text), Time dependent (video).

Size

Typ. PAL digital video: 720 by 576 pixels for each one of 3 color

components, 25 frames per sec. �� ~ 30 MB per sec., ~ 104GB per

hour.

Standards

JPEG

MPEG-2, MPEG-4

Multimedia Data

largest capacity of a commercially

available HDD at the end of …

2010

2014


Multimedia SystemsData Format: Graphics Example

vector vs. raster (bitmap)

graphics

Raster graphics

Are composed of pixels.

.gif, .jpg, .png

Data file contains: an array of pixels of

various colors.

Pro.: Small file size for lower resolutions.

Con.: Will become "blocky" if scaled up.

Vector graphics

Are composed of paths.

.eps file or Adobe Illustrator (.ai) file, .svg

Data file contains: the points where the paths

start and end, how much the paths curve, and

the colors that either border or fill the paths.

Pro.: The images can be scaled to be very

large without losing quality (logos are typically

created in vector forma).

Con.: Not the best format for photographs.


Multimedia SystemsScalable Vector Graphics (.SVG)

Ref.: en.wikipedia.org/wiki/Scalable_Vector_Graphics

SVG is an XML-based vector image format for two-dimensional graphics with support for

interactivity and animation. The SVG specification is an open standard developed by the

World Wide Web Consortium (W3C) since 1999.

Vector Graphics Example: .svg

SVG images can be produced by the use of a vector graphics editor, such as Inkscape, Adobe Illustrator,

Adobe Flash Professional or CorelDRAW, and rendered to common raster image formats such as PNG

using the same software. Inkscape uses a (built-in) potrace to import raster image formats.


Thank You

1. http://ce.sharif.edu/~m_amiri/

2. http://www.aictc.ir/

FIND OUT MORE AT...

Multimedia Systems

Introduction

Next Session: Review of Signals and Systems

Lec02, Introduction, v1.08.ppt -...

Documents

Transcript of Lec02, Introduction, v1.08.ppt -...