Graphics and Imaging Architectures · 2011. 9. 15. · graphics pipeline (e.g., OpenGL/Direct3D) on...

Graphics and Imaging Architectures

Kayvon Fatahalian

http://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/

Kayvon Fatahalian CMU 15-869, Fall 2011

About Kayvon▪ New faculty, just arrived from Stanford

▪ Dissertation: Evolving real-time graphics pipeline for micropolygon rendering(micropolygon meshes = high-resolution meshes used in o!-line rendering for "lm)

▪ In addition to rendering: involved in many research projects related to computing on GPUs

- Brook (precursor to CUDA/OpenCL)

- Sequoia (locality-aware parallel programming language)

- GRAMPS (stream computing for heterogeneous HW)


COMPUTING IS BECOMING HIGHLY VISUAL!

Visually rich user interfaces(that require 3D graphics!)

Innovative new input modalities(touch, gyro, cameras)

Ubiquitous, high-resolution cameras(emergence of computational photography)

Games, entertainment(continuous push towards higher visual realism)


COMPUTING EFFICIENTLY IS INCREASINGLY CRITICAL!

Power constraints

Ubiquitous parallelism

Heterogeneous parallelism

Mobile, mobile, mobile

Core i7 (Nahalem)

Real-time 3D graphics has always found a way to consume all available compute: GPUs are e#cient parallel, heterogeneous systems (that are relatively easy to program)


My Macbook Pro 2011 (two GPUs)

From i"xit.com teardown

AMD Radeon HD GPU

Quad-core Intel Core i7 CPU(Contains integrated Intel GPU)


iPad 2 main board

From i"xit.com teardown

Image processing DSP

Flash memory

Dual-core ARM CPU

GPUVideo

Encode/Decode

Apple A5 SoC


Touchscreen controller (integrated DSP) ~ $1

From US Patent Application 2006/0097991


My Nikon D7000 camera

16.2 MP sensor

6 fps burst

14 bits per pixel


What this course isThis is a course about how real-time graphics systems work

GRAPHICSALGORITHMS

(the workload)

MACHINE ORGANIZATION

geometry processingrasterization

texture mappinganti-aliasing

ParallelismLocality

CommunicationThe design of throughput processing cores

The role of "xed-function HW

mapping/scheduling


What this course isThis is a course about how real-time graphics systems work

GRAPHICSALGORITHMS

(the workload)

MACHINE ORGANIZATION

mapping/scheduling

ABSTRACTIONS(the real-time graphics pipeline)

choice of primitiveslevel of abstraction


What this course is not

▪ This is not an [OpenGL, CUDA, OpenCL] programming course

- But we will be discussing the design of these abstractions (and their future) in great detail!


Logistics 1: schedule▪ Expect it to be $uid- it’s my "rst time doing this

- students in this class have wide variation in background

- http://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/

▪ First half of class: focused on real-time 3D graphics- 3D graphics abstractions and the implementation of the real-time 3D

graphics pipeline (e.g., OpenGL/Direct3D) on modern GPUs

▪ Second half of class: broader sampling of applications and systems- future 3D graphics pipelines, GPU programming model issues (OpenCL/

CUDA), hybrid CPU/GPUs, image processing architectures, cameras

▪ Several external speakers (AMD, Intel, NVIDIA)


Logistics 2: expectations

▪ (40%) Paper readings

- Weekly: you will be asked to submit brief reviews of selected papers (or answer a speci"c questions)

- Participate in class discussions

▪ (10%) Two quick (2-3 day) C++ programming assignments

- Meant to concretize concepts, not stress you out

▪ (50%) Semester-long, self-selected research project (teams of 1-2)

- Aim for publishable results

This is a reading/project course


3D rendering

Image credit: Henrik Wann Jensen

Model of a scene:3D surface geometry (e.g., triangle mesh)

surface materialslights

camera

Image

How does each triangle contribute to each pixel in the image?


What is an architecture?(white board)


Real-time graphics pipeline (entities)

Vertices Primitives(triangles, points, lines)

Fragments Pixels

1

2

3

4


Real-time graphics pipeline (operations)

Primitive Generation

Vertex Generation

Vertex Processing

Fragment Generation(Rasterization)

Fragment Processing

Pixel Operations

Primitive Processing

Vertex stream

Vertex stream

Primitive stream

Primitive stream

Fragment stream

Fragment stream

Vertices

Primitives

Fragments

Pixels

Triangles positioned on screen

Fragments (one per each covered pixel)

Shaded fragments

Output image (pixels)

Vertices in positioned on screen

Vertices in 3D space1

2

34


Real-time graphics pipeline (state)

Primitive Generation

Vertex Generation

Fragment Generation(Rasterization)

Pixel Operations

Output image bu!er

Vertex stream

Vertex stream

Primitive stream

Primitive stream

Fragment stream

Fragment stream

Vertices

Primitives

Fragments

Pixels

Vertex Data bu!ers1

2

34

Memory Bu!ers (system state)

Texture bu!ersVertex transform matrices

Texture bu!ers

Texture bu!ers

Vertex Processing

Fragment Processing

Primitive Processing


History:Evolution of interactive 3D graphics

Following slides borrowed from Kurt Akeley and Pat Hanrahanhttp://graphics.stanford.edu/wikis/cs448-07-spring

Kayvon Fatahalian CMU 15-869, Fall 2011Slide credit Kurt Akeley and Pat Hanrahan


Early years ▪ Evans and Sutherland built machines for $ight simulators

throughout the 70s▪ Early 80s: Custom ASIC for geometry processing

- The Geometry Engine: a VLSI Geometry System for Graphics(Jim Clark, SIGGRAPH 82)

- Work started at Stanford (led to Silicon Graphics, 1982)


O!-line rendering for "lm

▪ O!-line graphics advancing as well- Pixar created in 1986- Initially thought they were going to build hardware (Pixar image computer)

▪ Reyes graphics system at Pixar (Renderman)- Feature rich: smooth/complex surfaces, programmable shading, anti-aliasing,

motion blur- Implemented as a software application on CPUs- Hours/frame

Image credit: Pixar, Toy Story 1 , 1995


Early PC graphics

Wolfenstein 3D, 1992 Doom, 1993

Software applications running on CPUExtreme amount of software optimization, algorithm development (and hacks)


PC 3D graphics▪ 3D graphics acceleration for PC motivated by games▪ add-in boards made by 3DFX, NVIDIA, ATI, PowerVR, and more- 3DFX founders were from SGI

- 3DFX Voodoo (1997)

▪ standardized around accelerating OpenGL/Direct3D- o%oaded rasterization, texture mapping, frame-bu!er processing

- initially only subset of GL implemented by Quake

Quake 1


Fourth generation▪ Programmable shading


Current generation▪ Tessellation (smooth/complex surfaces)

Image credit: NVIDIA/Unigine


Current generation▪ Some GPU computations now driven by alternative

programming interface (not 3D graphics pipeline) - to augment 3D rendering (game physics, global lighting)

- for non-graphics applications (scienti"c computing)

Image credit: NVIDIA Image credit: Folding @ Home


Today: high versatility, high peak compute

Intel Core i7 (quad-core CPU)~100 GFLOPS

730 million transistors

AMD Radeon HD 5870 GPU~2.7 TFLOPS

2.2 billion transistors

More compute power than Pixar’s entire render farm for Toy Story 1!


Readings▪ D. Blythe, The Rise of the Graphics Processor

Proceedings of the IEEE, 2008

▪ Real-Time Rendering, Ch 2. The Graphics Rendering Pipeline(handout)

Graphics and Imaging Architectures · 2011. 9. 15. · graphics pipeline (e.g., OpenGL/Direct3D) on...

Documents

Transcript of Graphics and Imaging Architectures · 2011. 9. 15. · graphics pipeline (e.g., OpenGL/Direct3D) on...