CG Architectures, Image Formation, and Models Angel, Chapter 1 CSCI 6360.

CG Architectures, Image Formation,

and ModelsAngel, Chapter 1

CSCI 6360

Elements of Computer Graphics

User, i.e., Human“Interaction” “Perception” “Information”

An illusion: analog world represented

digitally

DisplayRecall …


Programming systems-windows-graphics

Hardware andgraphics engines

3-DViewing

Transformation

Representation ofcurves and surfaces

“Models”,(Foley et al. term)

DrawingAlgorithms

Lights Rendering



digitally

Programming is a craft

OpenSystem

DisplayClosedSystem

Computer Graphics, Spring 2015• Angel, page 1:

– Perhaps the dominant characteristic of this new millennium is how computer and communication technologies have become dominant forces in our lives.

– “Computer graphics is concerned with all aspects of producing pictures or images using a computer.”

• … all aspects …– Hardware– Software– Applications

Overview

• “more later” … about all of what we’ll talk about tonight!– Angel’s “top down” (“gentle”) presentation

• CG System Hardware Overview– But, how do you get a pixel on the display?

• Images and Image Formation– Pinhole Camera– Human visual system

• And a bit of depth to illustrate how an expert might exploit known elements

• CG Programming Framework and Architecture– Pipeline architectures– OGL and the synthetic camera

CG Hardware System Overview

• Components of a CG system

• Raster displays

• Display devices: crt, etc.

• CG system architectures– GPU

• Frame buffers– Addressability and resolution

• “Scanning out” the image

• Color look up tables

• Input devices

(Really) Basic Graphics System Angel … note frame buffer

Input devices

Output device

Image formed in FB

Conceptual (Programming) Framework for Interactive Computer Graphics

• Overview – much more detail in 2nd half of class

– Application program • maps application objects to views (images) of those objects by calling on graphics library

– Graphics library/package (e.g., OpenGL) • is intermediary between application and the display hardware

– User interaction • results in modification of model and/or image

– Images • are usually means to an end: synthesis, design, manufacturing, visualization, ...

Raster Architecture (last time)

• Early systems– Used part of system memory (system

address space) for frame buffer– Single CPU set frame buffer and performed

“graphics system” commands

• Current systems– Separate processor gets cmds from CPU

• Graphics Processing Unit (GPU) – “In hardware”

• Scan conversion• Texturing• Dedicated texture memory• Lighting calculations• 3D transformation calculations

• GPU more power than CPU– Interesting implications!

Current Graphics Workstations

• Historically, small volume, high memory systems were necessary and expensive to do computer graphics defined as some frame rate with some number of polygons

• Now, PC + high performance graphics card works fine for most applications– Coprocessor + large, multiple frame

buffers

• Not “hard line” between “workstations” and “pcs”

Current GPU – “Quick Look”, 1Where we’re going tonight

• GPU architecture– Note cpu – gpu boundary– Will talk about “graphics pipeline” shortly

Current GPU – “Quick Look”, 2A complete architecture, FYI

• GPU architecture detail– Gee whiz …– Nvidia …

• But there is a frame buffer in there!

• Also, note vertex and fragment processing

Graphics Display HardwareHistory, for completeness

• Two classes of output technology

• Vector (calligraphic, stroke, random scan) – Historical – Still used in some plotters

• Raster (TV, bitmap, pixmap), – Used in displays and laser

printers – “dots”– Picture elements - pixels

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRw&url=http%3A%2F%2Fwww.nycresistor.com%2F2012%2F09%2F03%2Fvector-display%2F&ei=e_THVNieKYSnyQTD_ILoDA&bvm=bv.84349003,d.aWw&psig=AFQjCNEavYvPq5JAZtmPPAi8NDTefxFQAw&ust=1422476751191041

Vector Architecture

• Historic in interactive displays

• However, useful in understanding some, now, software terminology

• Vector display works with a display list / file stored in refresh buffer

• Display controller draws all vectors at < 60 Hz– often at variable rate – flicker is a problem

Raster Definitions

• Raster– A rectangular array of

points or dots

• Pixel (or, arcanely, Pel)– One dot or picture element

of the raster

• Scan Line– A row of pixels– Called that because in a crt

an electron stream is moved across (scans) the crt face

Frame Buffer – “Memory Mapped”

• Image, i.e., what shown on display, exists as way memory values set in frame buffer

– Image “mapped” to memory• (supposed to be house and

tree)– In practice, not 0’s and 1’s

• “Frame buffer”, “refresh buffer”– like “frame” of a movie– Represents image,

• what shows up on display

– “Buffer” is a term for memory• Hence, “frame buffer”

• Simple electronic process goes through (scans) this memory and control output device

– Video – scan - controller– “scan out the image”

BTW - Addressability and Resolution

• Fairly straightforward mapping of memory to display

• Addressability– Number of individual dots per inch

that can be created– May differ in horizontal and vertical

• Resolution– Number of distinguishable lines

per inch that a device can create– Closest spacing at which adjacent

black and white lines can be distinguished

• In fact, resolution usually less than addressability

– Smooths out jaggies, etc.

(1,1)

(9,7)

“Scanning out the Image”, Overview

• Again, frame buffer holds memory values representing pixel values

• Scan controller goes through memory locations

FYI - Scanning out Image, Details

• 1 bit Bilevel Display– Recall, frame buffer holds value for display

• Color, intensity– Simplest

• Black and white (or any 2 colors) • 1000 x 1000 display needs 1,000,000 bits or 128k memory (8 bits/byte)

• Memory access time > display time, so fetch chunks and control with registers– 1024x1024 memory 15 nanosec/pixel => 32 pixel chunks stored in shift registers/buffers

• Digital intensity value -> Digital to Analog Conversion (DAC) ->analog signal to drive display

Frame Buffer “Depth”(and OpenGL has much much more, later …)

• 1 bit– For 2 levels– Display is 0-off or 1-on

• 4 or 8 bits– For 16 levels or 256 levels of

intensity (gray)– Gun is varied in intensity

• 00-off … • 0011-gray …• 1111-bright

• Also, 3 bits– For 8 levels– Or color look up table– Provides index into table of colors

Frame Buffer “Depth”“True Color”

• 24 bits– “True color”

• Enough eye can’t distinguish

• > 24 bits– True color +

• Alpha blending, later …• Fog• Text overlay• …

• In general, n-bits / pixel

• Total frame buffer memory required– Addressability x depth– E.g., 1000 x 1000 x 24-bits = 3 mb– Also, Multiple frame buffers for animation, etc.

• Currently, large amounts of graphics memory used for textures for texture mapping, etc.

OpenGL Buffers

• Will consider buffers in detail later

• Buffer depth typically a few hundred bits and can be much more– Color, and double buffering

• Front: from which image is scanned out • Back: “next” image written into• 2 x 32-color + 8-alpha

– Depth• Hidden surface removal using z-buffer

algorithm• 16 min, 32 for fp arithmetic

– Color indices• Lookup tables

– Accumulation• Blending, compositing, etc.

• Pixel at mi, nj is all bits of the k bit-planes

FYI - Color Look up Table, 1

• Display hardware may not have sufficient capacity (memory) to store all, e.g, cell phone– So, choose some to store

• e.g., lots of shades of blue for water or sky (and not too many green)

• Any specific 2n colors may not be adequate – (n typically 16 24 in low end systems)

• Look up table allows 2n colors out of 224 to be used in one picture, – some other 2n in another picture

• Color table is a resource managed (usually) by window manager

FYI - Color Look Up Table, 2

• Pixel value is indexed to color look up table (CLUT) where color is stored. – Here use only 12 bits (4bits per color) for clarity typically, 24bits are used

• CLUT look up done at video rates, overlapped with fetch and DAC

• In 24 bit ``true''color systems, 3 x 8 bits for R, G, B; each color has its own 8 bit CLUT (0 255)

• CLUT allows variety of effects

– pseudo coloring (LandSat images, stress diagrams, thermograms...) – fast image changes: change table rather than stored image – multiple images: select or composite/blend

Display Technologies …. Briefly:

• Cathode ray tube– Monochrome– Color

• LCD

• Solid State– Active matrix

• Plasma, projection, etc.

Cathode Ray Tube• Obsolete, but cg terms from

• Electron gun emits stream of electrons

– Accelerated toward phosphor-coated screen by high positive voltage at face of tube

– Forced into narrow stream and directed at a point by focusing system

– Intensity controlled, also

• Electron hitting face of screen causes phosphor to emit visible light

– Decays, and must be refreshed > 60 times/sec

– “Flicker” results when eye can no longer fuse the (many) images

– Critical Fusion Frequency• Typically 60 times/sec for raster displays• Varies with intensity, individuals, phosphor

persistence, lighting, …

FYI - Scanning – for CRT Display

• Interlaced scanning– Assume can only scan 30 times/sec– To reduce flicker, divide frame into two “fields” (odd and even lines)

• … Also, terminology– Vertical sync pulse

• Signals the start of the next field– Vertical retrace

• Time needed to get from the bottom of the current field to the top of the next field– Horizontal sync pulse

• Signals the start of the new scan line– Horizontal retrace

• Time needed to get from the end of the current scan line to the start of the next scan line.

Color CRT

• Single electron gun for monochrome– Color seen depends on color of phosphor

• Multiple electron guns for color

• Shadow mask– Phosphors arranged in triads

– Each triad has one R/G/B phosphor dot– Typically 2.3 to 2.5 triads per pixel– Shadow mask has one small hole for each

phosphor triad

• Aperture grill– i.e. Sony Trinitron– Phosphors arranged in vertical stripes– Shadow mask is a vertical “grill”

FYI - Flat-Panel Displays

• Emissive versus nonemissive

• Emissive– Plasma panels

• mixture of gases between two glass plates• vertical and horizontal conducting ribbons• apply voltage to two ribbons to make plasma glow

– Thin-film electroluminescent displays similar, but phosphor instead of gas

– LED's• matrix of diodes, one per pixel• apply voltage and they produce light

• Nonemissive– LCD

• LC substance flow like a liquid, but have crystaline molecular structure• Usually use nematic LC's (threadlike)• Two polarizers, two conductors, reflector• LC in normal state twists the light, so is reflected back to viewer• apply voltage to conductors to turn off (see figure)• Active Matrix LCD - transistor at each pixel (stores)

Liquid Crystal Display

• 6 layers– Vertical polarizer plate– Layer with thin vertical grid wires electrodeposited on the surface adjoining the crystals– Thin liquid-crystal layer– Layer with horizontal grid wires– Horizontal polarizer– Reflector

• Liquid-crystal material - long crystalline molecules, normally in spiral arrangement

• Address x, y point with voltage– Causes molecules to align– Polarizing effect of crystal lost at this point– So, “dark spot” appears

FYI - Stereoscopic Viewing• Compelling effect!

– But, ergonomics challenges to apply– “promise of the future … always has been always will be” ???

• Left and right eyes get different images

• Change in eye separation with depth

• But, only perceive one image– Perceptual processes integrate different views from the two eyes

• E.g, LCD Shutter Glasses– >120 hz monitor refresh– Different images on odd and even– LCD “Shutters” open and close for left and right eyes

• (used to be metal shutters!)

Input: Logical and Physical DevicesAngel and interaction terminology – “logical” and “physical”

• Locator– Logical: position and/or orientation– Physical: Tablet, mouse, trackball, touch panel, Light pen

• Keyboard– Logical: characters and strings– Physical: Alphanumeric keyboard (coded or unencoded)

• Valuator– Logical: single values in the space of real numbers– Physical: Rotary dials (bounded or unbounded), sliders

• Choice– Logical: select from a set of actions or choices– Physical: Function keys

Images and Image Formation


• Objects and viewers

• Lights and Images

• Image formation models

• Physical systems– Pinhole camera– Human visual system

• A bit of example detail to illustrate how knowing about human system is useful in computer graphics

• None in Angel


• Physical (real or physical world) images have been “captured” for some time now …– Optical systems:

• Cameras, telescopes, microscopes• Human visual system

• Synthetic (synthesized, artificial) images may or may not exist– Can be created by computer, painting (through human imagination), etc.

• The methods for creating computer images, especially in “real time” - that allow fully interactive computer graphics:• In some ways similar to optical methods, e.g., camera model and viewing• … and in other ways not! …

• computationally tractable shading, discrete approach in general

• Goals and techniques of interactive cg & photorealistic cg often differ– “good enough” is good enough

Image Formation: Objects and Viewers

• Image formation process (either physical or mathematical abstraction) requires:

– Object • Exists in a space independent of any image-

formation process or viewer– Viewer

• Physical– camera, human,

• Abstractly, mathematically– location, orientation

• In cg can specify synthetic objects as primitives with attributes and positions in space

– Line: end points (vertices) and color– Sphere with some radius and segmentation– CAD systems provide an interface to

facilitate construction of synthetic model of the (physical) world

– OpenGL next chapter to do such things

glBegin(GL_POLYGON) glvertex3f(0.0, 0.0, 0.0) /* vertex A */ glvertex3f(0.0, 1.0, 0.0) /* vertex B */ glvertex3f(0.0, 0.0, 1.0) /* vertex C */glEnd();

Image Depends on Location(s)

• Viewer forms the image

• Image formed depends on perspective (“viewing angle”, …) of image– Formed on film, retina– Or in cg system– Different viewers form different images

• (or same viewer from different location)

• Of course, the image formed (even of a three-dimensional object) is two-dimensional

– Film plane, retina are 2-d, so is computer screen– This formation/mapping of three dimensions to two dimensions is one of the

fundamental challenges of computer graphics

Lights and Images

Lights and Images

• Of course, without light there would be no image!

– Image formed (physically) by changing chemicals on film, exciting neurons, etc.

• Light from a light source strikes various surfaces of the object

– A portion (very small in fact) of the light striking the surfaces is reflected to the viewer, e.g., camera

– Details of the interaction between light and surfaces of object determine how much (and what color, etc.) light enters camera

• Light is a form of electromagnetic radiation

– Visible (to humans) spectrum

• Geometric optics models light sources as emitters of light energy, each with a fixed intensity

– Point sources are typically used

Image Formation Models

• Multiple approaches to how form images from set of objects

– Using also properties of light sources and light-reflecting properties of the objects

• E.g., ray tracing imaging model might follow all rays emanating from a single point source

– Like we said, not many would reach viewer

• Ray tracing in essence “does it backwards”– Follows “rays” through pixels of display– Radiosity also a physically based technique

• Time constraints of interactive cg precludes use of these and related techniques

– However, used for photorealistic images, e.g., “global affects, including accurate shadows and reflection

• Image formation techniques that are “fast” are focus in techniques we will look at in detail

– And are those implemented “in hardware” – And so accessed reasonably efficiently by OpenGL

Imaging System: Pinhole CameraPhysical image formation

• Box with small hole in one side– Expose film by uncovering hole

• Assume oriented to z-axis, and a single ray of light enters– Film plane at distance d from hole – a single ray of light isn’t interesting, but many are!

• Can use similar triangles to determine projection of any point outside camera on to film plane

– “side” view– Down x-axis

www.silverprint.co.uk/acc21.html

-

http://www.silverprint.co.uk/acc21.html

Point Location and Projectors

• Can use similar triangles to determine projection of any point outside camera on to film plane– “side” view– Down x-axis

1. Find y coordinate of the image yp (figure for y – only - is at right)

– yp = - y / (z/d)

2. And use top view to find xp

– xp = - x / (z/d)

3. Point (xp, yp, -d) is the projection of the point (x, y, z)

– This idea is at core of cg techniques!

-

A little about humans




digitally

Display

Just a little bit …

Imaging System: Human EyePhysical Reception

• Mechanism for receiving light and transforming it into nerve transmissions

– A transducer: light -> nerves

• Light reflects from objects– Some strikes retina– Images are focused upside-down on

retina

• Ganglion cells (from brain!) detect pattern and movement

• As camera, has:– equivalent of lens, aperture (pupil)– and film (retina)

• Note – image upside down on retina– … perception!

Environment: Visible Light

• Generally, the body’s sensory systems are the way they are because had survival value

– Led to success • survival and reproduction

• Focus on human vision (because of computer), but all share basic notions

• Humans have receptors for (a small part of) the electromagnetic spectrum

– Have receptors sensitive to (fire when excited by) energy 400-700nm

– Snakes “see” infrared, some insects ultraviolet

• i.e., have receptors that fire

Human Eye: Retinal Receptors

• Two types of (photo) receptors on retina: rods and cones– Rods look like, well, rods …

• Rods:– spread all over the retinal surface (75 - 150 million)– low resolution, no color vision, but very sensitive to low light (scotopic or dim-light vision)

• Cones:– dense array around the central portion of the retina, the fovea centralis (6 - 7 million)– high-resolution, color vision, but require brighter light (photopic or bright-light vision)

FYI - Human Eye: Lens

• Eye has compound lens: – cornea (power) and lens (adjust focal length)

f = focal length of lens

d = distance to object

r = distance to image that is formed

– Flexibility of lens changes with age, approaching 0 at 60 years

Human Eye: Depth of Field and Focus

• Depth of focus– Distance over which objects are in focus without change in focus

• Note: Different colors have different depths of focus – chromatic aberration• E.g., blue vs. red

– Varies with size of pupil– Range of focus:

Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

– Rarely do computer systems model

Human Eye: Depth of Field - Example

• Photographic Images:– Depth of field longer with small aperture (f

stop)– Range of focus:

Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17

cm 1 m 75 cm 1.5 m 75

cm 2 m 1.2m 6.0m 1.8

m 3 m 1.5m Infinity

Large

Human Eye: Chromatic Aberration

• Demo

Most people see red

Closer than blue

Human Eye: Chromatic Aberration

• Different wavelengths of light focused at different distances within eye

– Short-wavelength blue light refracted more than long-wavelength red light

• Focusing on a red patch, an adjacent blue patch will be significantly out of focus

• Strong illusory depth effects

• Human eye has no correction for chromatic aberration!

– Inadvisable: fine blue patterns in visualizations!

– Visual effects in soap bubbles, crystal sculptures, etc.

Computer Graphics Programming Framework and Architecture

Computer Graphics Programming Framework and Architecture

• CG Software Architectures and APIs– Especially, where OpenGL fits– Last time, a very detailed view of cg and software architectures– Tonight, Angel’s (and others) schema

• Overview of OpenGL

• Computer Graphics Pipeline

• Synthetic camera

Recall, Raster Architecture for CG (… and more next time)

• Early systems– Used part of system memory (system

address space) for frame buffer– Single CPU set frame buffer and performed

“graphics system” commands

• Current systems– Separate processor gets cmds from CPU

• Graphics Processing Unit (GPU) – Operations “in hardware (firmware):”

• Scan conversion• Texturing• Dedicated texture memory• Lighting calculations• 3D transformation calculations

• GPU can have more power than CPU– Interesting implications!

CG Software Architecture, APIsBoth window system and graphics system api’s

Graphics API OpenGl (no win sys func) DirectX

Window system API MS windows X tools

CPU GPUGraphics Processing Unit

Software Arch., Window System“The familiar” – No Graphics API Use

“Application Progs” E.g., student programs MS Excel, etc.

Tools: UI Program. MFC, Swing, Qt (widget toolkit and more)




Software Arch., Graphics – 1“Simple” Use of Graphics API



GLUTOpenGL Utility Toolkit



Software Arch., Graphics – 2Tool/package Use of Graphics API





Tools: Vis. Program. VTK s3s

Software Arch., Graphics – 3Graphics API Use






Tools: Vis. Program. VTK

Software Architecture, All


Tools: Graphic Maya, 3ds

Tools: UI Program. MFC, Swing, Qt (widget toolkit and more)





Tools: Vis. Program. VTK

Raster Architecture for CG (now its next time)

• Current systems – at right

• Graphics Processing Units (GPUs) - multiple “cores”– And CPUs for that matter

• Also, operations (lighting, texture handling) now programmable

• OpenGL Shader Language (GLSL)

Programming Framework for CG

• In practice, writing “computer graphics” programs entails several programming systems

– “old familiar things”, application programs– + windowing system elements which may be relatively new, GLUT, etc., help– Graphics API, new– “GPU programming”, very new

• Relatively new paradigm (>2000), more next time

GPU programs, API Opengl Shader Lang.

Graphics API Opengl, DirectX, etc.

Window system API c/c++, java, etc.

Application programs c/c++, java, etc.

Programming Framework for CGFrom Angel – same ideas as last slide, focus on cg

• Several different kinds of “programming” and communication– Application, windowing system, high level graphics, low level

graphics, input devices, ….

“Domain” (and models) - CAD, movie, …

Application Program - C/C++, Java, …

Graphics API - OGL, DirectX, …

Window. Sys. API - MS Wind., X Wind., …

Graphics hardware - Algs for lines, … - in firmware - (co)processor

- Timed voltages output

Graphics Pipeline - GLSL

Programming Framework for CG

• Will be programming OpenGL – Standard for more than pcs– Will also examine hw architecture, etc.– “Convenience” ui programming

• Algorithms for drawing– Can do much/all in cg without specifying (just use “built-in”, or, default)– However, standards are changing and approach widely used with other

systems, e.g., WebGL, mobile devices


Application Program - C/C++, Java, … - “graphics” programs



Graphics hardware - Algs for lines, … - Programmable - (co)processor


GLUT - “convenience”


Graphics Library/PackageOpenGL

Things to program in OpenGL:

1. Primitives (2D and 3D)- All about vertices, meshes, and polygons

2. Attributes – color – line style – ... – material properties for 3D

3. Lights 4. Transformations 5. Immediate mode vs. retained mode


Application Program - C/C++, Java, … - “graphics” programs



Graphics hardware - Algs for lines, … - in firmware - (co)processor


GLUT -“convenience”


Opengl API ContentsPreview

• Functions that specify what needed to form an image– Objects– Viewer– Light Source(s)– Materials

• Other information– Input from devices such as mouse and keyboard– Capabilities of system

• Object Specification– Most APIs support a limited set of primitives including

• Points (0D object)• Line segments (1D objects)• Polygons (2D objects)• Some curves and surfaces

– Quadrics– Parametric polynomials

– All are defined through locations in space or vertices

Graphics Library FunctionalityPreview

1. Primitives – 2D and 3D– All about vertices, meshes,

and polygons• “In hardware”

– Libraries used seamlessly with OpenGL in programming

2. Attributes – color – line style – ... – material properties for 3D

Example: 3D primitives

Photorealism and Complexity• Recall, from 1st lecture … examples below exhibit range of “realism”

• In general, trade off realism for speed – interactive computer graphics– Wireframe – just the outline – historically interesting– Local illumination models, polygon based

• Flat shading – same illumination value for all of each polygon• Smooth shading (Gouraud and Phong) – different values across polygons

– Global illlumination models• E.g., Raytracing – consider “all” interactions of light with object

Polygons – Flat shading Ray tracingWireframe Polygons - Smooth shading

Graphics Library FunctionalityOpenGL Preview

3. Lights - Local, vs. global

4. Transformations - Concerned with view, etc.

5. Immediate mode vs. retained mode

– Immediate mode: no stored representation, package holds only attribute state, and application must completely draw each frame

– retained mode: library compiles and displays from a “scene graph” Example: Multiple Lights in Scene

Lights and MaterialsOpengl Preview

• Types of lights– Point sources vs. distributed sources– Spot lights– Near and far sources– Color properties

• Material properties– Absorption: color properties– Scattering

• Phong, considers ambient, diffuse, specular

Light Interactions with a Solid

• Most (interactive) cg systems model only reflected light– And then in a manner that allows reasonably fast computation

• E.g., Phong model– “Off-line” rendering (model, render, repeat) is also used, at expense of interactivity

• In fact, there are some complexities in modeling light …– But, good enough is good enough…

Vertices and Geometry

• Objects usually defined by sets of vertices (gpu primitive)

– OpenGL at right

• Note – vertices give only locations in space, not specify geometric entity that they define

– Could be, e.g., triangle, two line segments, three pixels

• Must specify geometric entity– E.g., GL_TRIANGLES,

GL_LINE_STRIP, GL_POINTS

• Program only specifies geometry, leaves it to graphics system to determine which pixels to color in frame buffer

float vertices [3][3]

vertices[0][0] = 0.0; // vertex A

vertices[1][0] = 0.0;

vertices[2][0] = 0.0;

vertices[0][1] = 0.0; // vertex B

vertices[1][1] = 0.0;

vertices[2][1] = 0.0;

vertices[0][2] = 0.0; // vertex C

vertices[1][2] = 0.0;

vertices[2][2] = 0.0;

OpenGL (synthetic) Camera

OpenGL (synthetic) Camera

• Forming am image: – determine where projectors

• from an image • to the center of projection

– intersect the image plane

– … and turn on that pixel with appropriate color

• Natural to think of the cop, as a camera

• Not a real camera, but a synthetic camera

projectorsimage

image plane

center ofprojection (cop)

OpenGL (synthetic) CameraPlacing and Aiming

• So, just position camera in “world space” (world coordinates) as if real camera1. Identify the eye point where the camera is located 2. Identify the look-at point that we wish to appear in the center of our view3. Identify an up-vector oriented upwards in the final image (blue arrow in figure

• Specify camera configuration with:– gluLookAt(ex, ey, ez, ax, ay, az, ux, uy, uz)– 3 camera vectors: lookFrom (ex, ey, ez), lookAt (ax, ay, az), vUp (ux, uy, uz)

• Other alternatives exist in OpenGL, e.g., figure on right

OpenGL (synthetic camera)Clipping planes

• Mimics “synthetic camera” model to design graphics hardware software

– Will position and “point” (and tilt and rotate) the camera

• Clipping planes (clip = exclude)– Pragmatic considerations– If < 1 pixel when all calculations done,

no need to do calculations!

• How is the API implemented?– Pipeline architecture– A “practical” approach

• Not a “physical” approach• Trade off photorealism for display

speed

Physical Approaches

• Ray tracing– Follow rays of light from

center of projection until they either are absorbed by objects or go off to infinity

– Can handle global effects• Multiple reflections• Translucent objects

– Slow– Must have whole data base

available at all times

• Radiosity– Energy based approach– Very slow

Pipeline ArchitecturePractical Approach

Pipeline ArchitecturePractical Approach

• Process objects one at a time in the order generated by application– Can consider only local (one direction) lighting – not photorealistic from outset

• Pipeline architecture:

• All steps can be implemented “in hardware” on the graphics card

• CG in this “practical approach”, using pipeline architecture, allows effective parallelization

• Pipeline deals with geometry, image formation still requires buffer manipulations

Application program Display

GPU Pipeline Architecture Vertex Processing

• Recall, geometry based on vertices (or, just a bunch of points):

• MUCH of work in graphics deals with “just points”– E.g., vertex processing and clipping

• “Effective parallelization” plus several to dozens to hundreds of graphics processors (cores) leads to fast graphics

• Also, much of the work in pipeline is in converting object representations from one coordinate system to another (next slide)

– Object coordinates, Camera (eye) coordinates• Screen coordinates

– Every change of coordinates is equivalent to a matrix transformation – Also computes vertex colors

vertices[0][0] = 0.0; // vertex A

vertices[1][0] = 0.0;

vertices[2][0] = 0.0;

vertices[0][1] = 0.0; // vertex B

vertices[1][1] = 0.0;

vertices[2][1] = 0.0;

vertices[0][2] = 0.0; // vertex C

vertices[1][2] = 0.0;

vertices[2][2] = 0.0;

Coordinate Systems in ViewingTransformations to change

• Coordinate Systems in the Graphics Pipeline – OCS – object coordinate system – WCS – world coordinate system – VCS – viewing coordinate system – CCS – clipping coordinate system – NDCS - normalized device coordinate system – DCS – device coordinate system

• image plane– Where images formed

GPU Pipeline Architecture Primitive Assembly

• Projection is the process that combines the 3D viewer with the 3D objects to produce the 2D image

– Perspective projections: all projectors meet at the center of projection

– Parallel projection: projectors are parallel, center of projection is replaced by a direction of projection

• Primitive assembly– Vertices -> objects– Vertices must be collected into geometric objects

before clipping and rasterization can take place• Line segments• Polygons• Curves and surfaces

GPU Pipeline Architecture Clipping

• Just as real camera cannot “see” whole world, virtual camera can only see part of the world or object space

• Objects that are not within this volume are said to be clipped out of the scene

– Or, ignored– No need to process something that will not be displayed!

GPU Pipeline Architecture Rasterization

• Rasterization

• If an object is not clipped out, the appropriate pixels in the frame buffer must be assigned colors

• Rasterizer produces a set of fragments for each object

• Fragments are “potential pixels”– Have a location in frame bufffer– Color and depth attributes

• Vertex attributes are interpolated over objects by the rasterizer

GPU Pipeline Architecture Fragment Processing

• Fragment Processing

• Fragments may be blocked by other fragments closer to the camera – Hidden-surface removal– Or, visible surface determination

• Fragments are processed to determine the color of the corresponding pixel in the frame buffer

• Colors can be determined by texture mapping or interpolation of vertex colors

Programmable Pipelines… and OpenGL

• Recall, “in hardware” means in “in firmware” …

• And with current gpu’s, graphics routines, formerly only in firmware, are now accessible to programmer

– Current graphics systems have ability to change algorithms that implement low level algorithms

• E.g., changing shading algorithm … or anything

• Indeed, current versions of OpenGL require program to supply low level algorithms for the graphics pipeline:

– Defaults available

for all … for now– Text requires

from 1st program!– We will use a

“middle ground”

The Graphics PipelineAnother look

Rasterization

Application

Commands

Geometry

Fragment

Display

“Application Program” Logic and processes Networking User input events Etc. …

GPU

CPU

Vertex transformationsVertex lightingClippingPrimitive assembly

Convert triangles to fragmentsTex coordinate interpolationColor interpolation

Color blendingTexturingDepth test

From Pfister, 2006, Harvard E-234 notes

Send OpenGL commands OpenGL driver: Process GL command stream, talk to GPU

The Graphics PipelineProgrammable Pipeline

Rasterization

Application

Commands

Geometry

Fragment

Display

“Application Program” Logic and processes Networking User input events Etc. … Send OpenGL commands OpenGL driver: Process GL command stream, talk to GPU

GPU

CPU

Vertex transformationsVertex lightingClippingPrimitive assembly

Convert triangles to fragmentsTex coordinate interpolationColor interpolation

Color blendingTexturingDepth test

Programmable vertex units

Programmable fragment units

Hence, the term “shaders”


Vertex Shaders


Pixel Shaders


• End of lecture

Term Project

• See web site materials

Getting Started with OpenGL(much more next time)

• Instructions on web site and BB

• Example programs

• Try to load software and execute

• Again, more next time

Questions?

• .

CG Architectures, Image Formation, and Models Angel, Chapter 1 CSCI 6360.

Documents

Transcript of CG Architectures, Image Formation, and Models Angel, Chapter 1 CSCI 6360.