Imagery CS 110 Fall 2005. Image Technology Always Changing Paint / pencil Paint / pencil Cave...

ImageryImagery

CS 110CS 110

Fall 2005Fall 2005

Image Technology Always Image Technology Always ChangingChanging

Paint / pencilPaint / pencil• Cave drawingsCave drawings• Perspective renditionsPerspective renditions• ImpressionismImpressionism

OpticsOptics• Photographs Photographs • Photo volumes that shift lighting, time, Photo volumes that shift lighting, time,

and spaceand space

Early Imagery: In the Mind’s Early Imagery: In the Mind’s EyeEye

http://www.metmuseum.org/toah/hd/iptg/hob_60.173.htm

Improving Improving ImageryImagery

Equations dictateEquations dictateimage image

Mastering PerspectiveMastering PerspectiveLorenzetti (1342)Lorenzetti (1342)

Rules for a New EraRules for a New Era

PhotorealismPhotorealism

Perspective, therefore, should be Perspective, therefore, should be preferred above all man’s discourses preferred above all man’s discourses and disciplines. and disciplines. (Da Vinci, 1400)(Da Vinci, 1400)

Image Generation TechnologyImage Generation Technology

Camera ObscuraCamera Obscura• 1515thth Century Century

http://en.wikipedia.org/wiki/Image:Camera_obscura.jpg

Camera’s Inhibit Art?Camera’s Inhibit Art?

There are some who… trace outlines on a There are some who… trace outlines on a surface of a transparent medium.surface of a transparent medium.

Such an invention is to be condemned in Such an invention is to be condemned in those who do not know how to portray those who do not know how to portray things without it, nor how to reason about things without it, nor how to reason about nature with their mind.nature with their mind.

They are always poor and mean in every They are always poor and mean in every invention and in the invention and in the composition of composition of narrativesnarratives, which is the final aim of this , which is the final aim of this science.science.

(Da Vinci)

Can a Camera do This?Can a Camera do This?

Artistic controlArtistic controlof vanishingof vanishingpointpoint

(Da Vinci, 1498)

Can a Camera do This?Can a Camera do This?

Yes!Yes!

We’ll discussWe’ll discussexciting newexciting newtechnologiestechnologies

(Da Vinci, 1498)

Photographs vs. MomentsPhotographs vs. Moments PhotographPhotograph: an (almost : an (almost objectiveobjective) )

representation of a point in time and representation of a point in time and spacespace

MomentMoment: a visual representation of the : a visual representation of the subjectivesubjective reality at some time and place reality at some time and place

From M. Cohen, Microsoft Researchhttp://research.microsoft.com/~cohen/TheMoment.pdfhttp://research.microsoft.com/~cohen/FindingMagicInAnImageStack.pdf

Photographs vs. MomentsPhotographs vs. Moments

Artists capture moments so wellArtists capture moments so well

Photographs vs. MomentsPhotographs vs. Moments

With introduction of cameras, art With introduction of cameras, art became more abstract (to became more abstract (to compensate)compensate)

Have cameras lost the moment?Have cameras lost the moment? Can we recapture it?Can we recapture it?

Some InspirationSome Inspiration

The old days: 35mm film, $0.30 / picThe old days: 35mm film, $0.30 / pic Now: DigitalNow: Digital

• Disk storage = $1 / GigabyteDisk storage = $1 / Gigabyte• DVD = $0.50 / GigabyteDVD = $0.50 / Gigabyte• Images =~ 1 MbyteImages =~ 1 Mbyte

• 1 picture < $0.001 =~ free1 picture < $0.001 =~ free

From M. CohenMicrosoft Research

PhotosPhotos

Our roots: Edison’s 35mm film (1889)Our roots: Edison’s 35mm film (1889)

• Chemical coatingChemical coatingreacts to lightreacts to light

• Specific chemicalsSpecific chemicalsreact to uniquereact to uniquewavelengthswavelengths

Analog recording of visible wavelengths and intensities (continuous data)

ImagesImages

An image is a 2D function An image is a 2D function II(x, y) that (x, y) that specifies intensity for each point (x, specifies intensity for each point (x, y)y)

From L. McMillanMIT Comp. Graphics

Analog to DigitalAnalog to Digital

Our goal is to convert the continuous Our goal is to convert the continuous image to a discrete set of samplesimage to a discrete set of samples

Point Sampling an ImagePoint Sampling an Image

Simplest sampling is on a gridSimplest sampling is on a grid Sample dependsSample depends

solely on valuesolely on valueat grid pointsat grid points

Point SamplingPoint Sampling

Multiply sample grid by image Multiply sample grid by image intensity to obtain a discrete set of intensity to obtain a discrete set of points, or points, or samples.samples.

A to D, 2-D ExampleA to D, 2-D Example

Analog to digital conversionAnalog to digital conversion

location on image slice

inte

nsi

ty

Analog Signal

image slice

A to D, 2-D ExampleA to D, 2-D Example

Analog to digital conversionAnalog to digital conversion


inte

nsi

ty

Analog Signal

image slice


inte

nsi

ty

samples

Digital Signal

From Computer Graphics: Principles and Practice

Sensor ArraysSensor Arrays

Capturing “samples” of lightCapturing “samples” of light


Storing amount of light contacting Storing amount of light contacting squares on a surfacesquares on a surface• Each square tuned toEach square tuned to

sense red, green, or bluesense red, green, or blue• More green squares thanMore green squares than

othersothers• Humans see green bestHumans see green best

(lots of tigers in the grass)(lots of tigers in the grass)

Human PerceptionHuman Perception

We see these colors (mostly green)We see these colors (mostly green)


Very smallVery small Lenses focus lightLenses focus light Light generates Light generates

electricity that is electricity that is sensed by a silicon sensed by a silicon chipchip


Pixel: each square in sensor arraysPixel: each square in sensor arrays Resolution: number of pixelsResolution: number of pixels Color depth: number of bits that Color depth: number of bits that

describe the “reading” at each pixeldescribe the “reading” at each pixel


Aliasing: a common problem when Aliasing: a common problem when discretizing analog datadiscretizing analog data

Moire Pattern


Noise: at low light, small errors in Noise: at low light, small errors in pixel “readings” are very apparentpixel “readings” are very apparent• Human perception is more sensitive to Human perception is more sensitive to

small errors at low intensitiessmall errors at low intensities

Digital cameras have more trouble Digital cameras have more trouble than film cameras in low lightthan film cameras in low light

CompressionCompression

Light intensity at all sensors can be Light intensity at all sensors can be recordedrecorded• Large: RAW and TIFF file formatsLarge: RAW and TIFF file formats

Small, local intensity variations can Small, local intensity variations can be lumped togetherbe lumped together• JPEG (most common of digital cameras)JPEG (most common of digital cameras)• GIF (common web format)GIF (common web format)

Digital to AnalogDigital to Analog

New Display TechnologiesNew Display Technologies

Display WallsDisplay Walls• PrincetonPrinceton• StanfordStanford• UVa – Greg UVa – Greg

HumphreysHumphreys

Interesting Display Interesting Display TechnologiesTechnologies

Wood chips Chrome spheresWood chips Chrome spheres

Daniel Rozin – NYU: (movies) http://fargo.itp.tsoa.nyu.edu/~danny/art.html

Human PerceptionHuman Perception

A monitor can display these colorsA monitor can display these colors• Note how limitingNote how limiting

they arethey are

Human PerceptionHuman Perception A photograph is a poor A photograph is a poor

model of realistic lighting model of realistic lighting conditionsconditions• Cannot convey shadows Cannot convey shadows

and bright spots of real and bright spots of real scenesscenes

• Artists use optical illusions Artists use optical illusions to fool us into sensing to fool us into sensing lighting conditionslighting conditions

Photographers have Photographers have limited abilities to limited abilities to replicate artistic featuresreplicate artistic features

Jack Tumblin, SIGGRAPH ‘99

Creating MomentsCreating Moments

Automatically capitalizing on artistic Automatically capitalizing on artistic techniques to emphasize photostechniques to emphasize photos

Creating interactive photo Creating interactive photo techniquestechniques

Modern TechnologyModern TechnologyStanford

New ResultsNew Results

Ng – Stanfordhttp://graphics.stanford.edu/papers/lfcamera/


Ng - Stanford


http://graphics.stanford.edu/papers/http://graphics.stanford.edu/papers/lfcamera/lfcamera.wmvlfcamera/lfcamera.wmv

Modern TechnologyModern Technology

Mitsubishi Electric Research Labs


Artist Rendition

MERL Cam

Shadows indicate edgesShadows indicate edges


http://www.merl.com/people/raskar/http://www.merl.com/people/raskar/NprCamera/nprMovie320short2.aviNprCamera/nprMovie320short2.avi


Multi-ExposureMulti-Exposure

Durand - MIT


M. CohenMicrosoft Research


Automatic PanoramasAutomatic Panoramas


Modern TechnologyModern Technology

10 Mpixel Sensor10 Mpixel Sensor 200 fps200 fps Buffers 5 seconds of captureBuffers 5 seconds of capture

• 10 Gigapixel circular buffer10 Gigapixel circular buffer• Continuously “on” Continuously “on”


Merge Multiple ShotsMerge Multiple Shots


Light StageLight Stage


http://gl.ict.usc.edu/research/LS5/http://gl.ict.usc.edu/research/LS5/LS5_Siggraph_052005.aviLS5_Siggraph_052005.avi

Imagery CS 110 Fall 2005. Image Technology Always Changing Paint / pencil Paint / pencil Cave...

Documents

Transcript of Imagery CS 110 Fall 2005. Image Technology Always Changing Paint / pencil Paint / pencil Cave...