Post on 11-Sep-2021
(12) United States Patent Duluk et al.
US006771264B1
(10) Patent N0.: (45) Date of Patent:
US 6,771,264 B1 Aug. 3, 2004
(54) METHOD AND APPARATUS FOR PERFORMING TANGENT SPACE LIGHTING AND BUMP MAPPING IN A DEFERRED SHADING GRAPHICS PROCESSOR
(75) Inventors: Jerome F. Duluk, Palo Alto, CA (US); Stephen L. Dodgen, Boulder Creek, CA (US); Joseph P. Bratt, San Jose, CA (US); Matthew Papakipos, Menlo Park, CA (US); Nathan Tuck, San Diego, CA (US); Richard E. Hesse], Pleasanton, CA (US)
(73) Assignee: Apple Computer, Inc., Cupertino, CA (Us)
( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days.
(21) Appl. No.: 09/213,990
(22) Filed: Dec. 17, 1999
(Under 37 CFR 1.47)
(51) Int. Cl.7 .............................................. .. G06T 15/60
(52) U.S. Cl. ...................................... .. 345/426; 345/584
(58) Field of Search ............................... .. 345/426, 582,
345/583, 584
(56) References Cited
U.S. PATENT DOCUMENTS
4,783,829 A 11/ 1988 Miyakawa et 211. 4,945,500 A 7/1990 Deering 4,970,636 A 11/1990 Snodgrass et 81. 5,083,287 A 1/1992 Obata et 81. 5,123,085 A 6/1992 Wells et 81. 5,402,532 A 3/1995 Epstein et 81. 5,448,690 A 9/1995 Shiraishi et 81. 5,455,900 A 10/1995 Shiraishi et 81. 5,481,669 A 1/1996 Poulton et 81. 5,574,836 A 11/1996 Broemmelsiek .......... .. 395/127
5,579,455 A 11/1996 Greene et 81. 5,596,686 A 1/1997 Duluk, Jr.
(List continued on next page.)
SM
OTHER PUBLICATIONS
Angel (interactive computer graphics: a top—doWn approach With OpenGl: ISBN: 0—201—85571—2—sections 6.8 &
7.7.2). Schilling et al., “Texram: a smart memory for texturing,” IEEE computer graphics and applications, 5/96, 32—41.
Primary Examiner—MattheW Luu (74) Attorney, Agent, or Firm—R. Michael Ananian; Dorsey & Whitney LLP
(57) ABSTRACT
A system and method for performing tangent space lighting in a deferred shading graphics processor (DSGP) encom passes blocks of the DSGP that preprocess data and a Phong shader that executes only after all fragments have been preprocessed. A preprocessor block receives texture maps speci?ed in a variety of formats and converts those texture maps to a common format for use by the Phong shader. The preprocessor blocks provide the Phong shader With interpo lated surface basis vectors (v5, v,, n), a vector Tb that represents in tangen/obj ect space the texture/bump data from the texture maps, light data, material data, eye coordinates and other information used by the Phong shader to perform the lighting and bump mapping computations. The data from the preprocessor is provided for each fragment for Which lighting effects need to be computed. The Phong shader computes the color of a fragment using the information provided by the preprocessor. The Phong shader performs all lighting computations in eye space, Which requires it ?rst to transform bump data from tangent space to eye space. In one embodiment the Phong hardWare does this by multiplying a matrix M Whose columns comprise eye space basis vectors (b5, b,, n) derived from the surface basis vectors (v5, v,, n) and the vector Tb of bump map data. The eye space basis vectors are derived by the DSGP preprocessor so that the multiplication (M><Tb) gives the perturbed surface normal N‘ in eye space, re?ecting the bump effects. The perturbed surface normal N‘ is subsequently used in the lighting computations.
35 Claims, 41 Drawing Sheets
US 6,771,264 B1 Page 2
US. PATENT DOCUMENTS 5,920,326 A 7/1999 Rentschler et a1. ....... .. 345/503 _ 5,949,424 A * 9/1999 Cabral et a1. ............. .. 345/426
576647071 A 9/1997 Nagashlma 5,949,428 A 9/1999 Toelle et 211.
27233723? 2 15/133; g‘?alllllet al-l 5,977,977 A 11/1999 Kajiya et a1. 7 7 / I a e‘ a‘ 5,990,904 A 11/1999 Grif?n
5,710,876 A 1/1998 Peercy et a1. ............. .. 395/126 . 6,002,410 A 12/1999 Battle
5,767,859 A 6/1998 ROSSlIl et a1. _ . 6,002,412 A 12/1999 Schlnnerer 5,798,770 A 8/1998 Baldwln
. . . 6,118,452 A 9/2000 Gannett 5,828,378 A 10/1998 Shlralshl _
6,128,000 A 10/2000 Jouppl et 211. 5,841,447 A 11/1998 DreWs 6,167,486 A 12/2000 Lee et 211.
5,850,225 A 12/1998 Cosman , 6,204,859 B1 3/2001 Jouppl et 211. 5,854,631 A 12/1998 Akeley et a1.
. 6,229,553 B1 5/2001 Duluk, Jr. et 211. 5,860,158 A 1/1999 Pal et a1. .. 6,246,415 B1 6/2001 Grossman et 211.
5,864,342 A 1/1999 Kapya et a1. 6259 452 B1 72001 C t 1 RE36,145 E 3/1999 DeAguiar et a1. 7 7 / 0mg 6 a'
5,880,736 A 3/1999 Peercy et a1. ............. .. 345/426 5,889,997 A 3/1999 Strunk * cited by examiner
U.S. Patent
GENERIC 3D GRAPHICS PIPELINE
Aug. 3, 2004 Sheet 2 0f 41
l
US 6,771,264 B1
>_ FLOATING-POINT INTENSIVE EUNCT IONS
I >_ DRAWING INTENSIVE FUNCTIONS
\( DATA INPUT ) 2|4 2|6\’ musroamnou ‘ m\ cum“ m\ FACE ommmmou 224\ vmaxusmme m\ CLIP(|FNEEDED) \ PERSPECTIVE DIVIDE
228\ SCREEN SPACE 232 common \ SET UPFOR
KINCREMENTAL moan J 234 m\r EDGEWALK “ m\ SPAN INTERPOLATION 24 \\ Z-BUFFERED BLEND J
z \ FRAME BUFFER
244\rnouBLE BUFFERED Mux“ k oumn LOOKUP TABLE J
“K DIGITALTO ANALOG CONVERSION
(moa ART)
U.S. Patent Aug. 3, 2004 Sheet 3 0f 41 US 6,771,264 B1
US 6,771,264 B1
525
m mass: W595;
m
M A E222: =25: 56.55
We
=22: w :35;
m “II n
e l"
m Ea .:
P SH U
U.S. Patent Aug. 3, 2004 Sheet 7 0f 41 US 6,771,264 B1
52 A .5:
5:2;
#252 #255 Ill: maze. :25: IL u n
:25; | m; _m;.
5.5;. H 22.5: | _II 5:22. I + I“? gang .2 H a
“52;: If m.
U.S. Patent Aug. 3, 2004 Sheet 8 0f 41 US 6,771,264 B1
1010 2000 10000 / I /
' IIIIIHO§TII'.-- __T\COMMAND 2D :>‘ .............. -. >20 APPLICATION OGUD3D DRNER GHC ___MEM0RY___ GHC , g COMMANDS &
""""""" " GUI
VERTEX
3000\ GEOMETRY 20 ENGINE PIXELS
GOURAUD SHADED, CLIPPED PRINITIvEs AND MODES 10000 . 3D ~ DISPLAY
________ __ /5°°° / 150$ PIxELs CONTROL
EXTRACT MODES /...MEMORY-_. MODES / MODE‘ VSP * """"""" " ' INJECT g-?’zmm \ 4000
SPATIAL VSP INFO T000 _/ COLOR
TILE :::I'soR'T'.'.'.. VSPCOVER SORT SORTED -------------- -- PRIM. COLOR
, PR'M- :MEMQEYZ: A MODES
MM‘ \6000 11000 PER 9000 FRAGMENT /
TILE / INTEIITPE0LTATI0N PHgNG - _OR
I PRIM.|NFO ' BUMP sETuP “mp5s CULL VSPX,Y.Z 19,0
3000 TEXTURE 'IITEXTURE--
12000 / IINE'NI6RY5-T
FIG. 7
U.S. Patent
DEFERRED SHADING GRAPHICS PROCESSOR VERSION l
Aug. 3, 2004
m -
\( DATA INPUT ) 2|4 “?r TRANSFORMATION ‘ m\ CLIPTEST m\ FACEDETERMINATION 22} vmixusmmo m\ CLIP(|FNEEDED) \ PERSPECTIVE own):
228\ SCREEN SPACE m CONVERSION \ SETUPFOR I
LINCREMENIAL among a
25o\ SCENEMEMORY zsl\? EXACT HIDDEN
SURFACE REMOVAL 25.2 253 \?mmm comma ‘ \ BLENDING J
z42\ FRAME BUFFER
244Ynoum: BUFFERED nux? L ouwur LOOKUP mu;
246\ DIGITAL T0 ANALOG CONVERSION
FIG. 8
Sheet 9 0f 41 US 6,771,264 B1
F FLOATING-POINT INTENSIVE FUNCTIONS
U.S. Patent Aug. 3, 2004 Sheet 10 0f 41
I000
2000
COMMAND FETCH AND DECODE
IN FROM l/O BUS
FRAGMENT
TEXTURE IN FROM |/0 ' FRAMEBUFFER MEMORY BUS 2” GMPH'“ MEMORY
/ / moo laooo nooo
FIG. 9
US 6,771,264 B1
7000 6000 (J
SORT MEMORY
OUT TO DISPLAY
U.S. Patent Aug. 3, 2004 Sheet 11 0f 41 US 6,771,264 B1
7 FOUR
FRAGMENTS
WHITE BLACK FILLED FRAGMENTS FRAGMENT BACKGROUND PIXELS
FIG. 11A FIG. 118 FIG. 11C FIG. 11D
U.S. Patent Aug. 3, 2004 Sheet 12 0f 41 US 6,771,264 B1
2010
20l2 I cm BLOCK ”
zooo BUS INTERFACE
r l + $ ‘ READ/WRITE 20l6 lNTERRlIPT
m8 ’ comm \zom 2m \ summon L \ / -+ DNA CONTROLLER
BACKE 0mg? ‘ ‘ I DNA DATA ‘
BUS commas VIA Pl0 ' l 20g? 202\2 ‘ 2031 ' 202\6 v 2035
Bus DNA DNA WRITE 2n REQUEST 20 3o mum 30
[2036 TO HOST cm: azsrouss cm: RESPONSE QUEUE QUEUE QUEUE QUEUE QIIEUE
‘7 “k k
|8000 1034 l l R V V R
= ‘ 2o ENGINE mtfgc'gwn ' l 2
F“" _ * i "M = > m 3000
noool BUFFER H5002 W [6004 GEO f
y DISPLAY =
U.S. Patent Aug. 3, 2004 Sheet 14 0f 41 US 6,771,264 B1
a1&‘ 31m 3112 ZLiR- 31% N Nx N , 315B
8 6 28 CURRENT N° 3x3 NORMALTRANSFO?iMElWiEYQCOORDINATES ) 3m
INVERSE 312a ' ' n
NORMAL Go MODEL- G - 3138 3154 TANGENT VIEW 9 NORMAUZE 3160
MATRIX Gt BINORMAL Bo & 6 m MMV' Be SCALE ' 9
3120 31% 3116 311*“3130 3152 m 3162 Ve Xe, Ye, Ze We AUTOMATIC e Bt VER‘IEX EYE (£0022. SURFACE SI mb
Se Te TANGENT TEXTURE COOR GENERATION B9 _
3122 3132 3136
FIG. 14
U.S. Patent Aug. 3, 2004 Sheet 15 0f 41 US 6,771,264 B1
BUMP VECTOR
TRIVIAL REJECT BACK FACE CULLING USER DEFINED CLIP VIEW VOLUME CLIP
PERSPECTIVE DIVIDE VIEWPOEAT PROJECT AR SIGN (FRONT 0R BACK)
LINE STIPPLE DEPTH OFFSET
FIG. 15
US 6,771,264 B1 US‘ Patent Aug- 3, 2004 Sheet 16 0f 41
SORT MEMORY
LARRY
WATER‘ \' Y cm STP ->
T ——> MEX | m] I
+
AA ‘ CURLY _> [FD PIX ‘‘ +
Y PHB <- TEX MOE m
[B MEMORY TEXTURE MEMORY
3000 \
GEOMETRY
SPATIAL CULL_MODE, COLOR AND MLM
COLOR AMD MLM DATA 500R‘) DATA
MODE & 4000f EXTRAC'HON POLYGON MEMORY
SPATIAL DATA WITH COLOR POINTERS, CULL__MODE DATA
.__> P 6000/ 30'“ T0 57
Sheet 17 0f 41 US 6,771,264 B1
REUSE TYPE FRONTFACE CULLMODE1 MATFRONT1 MATBACK1
VTXMODE
Aug. 3, 2004
THE FOLLOWING PACKETS WILL BE SENT T0 MEX IN TIME ORDER:
VERTEX PACKET
M. KEEP MIDDLE O: KEEP OLDEST
N:NOTCOMPLETE P: POINT LI LINE T:TR|ANGLE
I 0000000 FIG. 18
NNNNT TTNNTTT T NT NNNNTTTTT MMMMM MMMMOOMMMO OOOOOOMMV 2 2T.
E wmw D 00% DOuTWTWOTWTTTTWTTTMRFHMWTWTWTMWT mmmmmmwmmmmmmmmmmmmwmwmmmmmm CSCSC SCSSSSCSSSCMTUCSCSCSCS WWAVIWW WWWAVEWWWWWW WWWWWWWW 4 W V ..
U.S. Patent
U.S. Patent Aug. 3, 2004 Sheet 18 0f 41 US 6,771,264 B1
STATE APPROXIMATE PARAMETER DESCRIPTION SIZE IN I44
SET BIT ORDS) IF THE FRONT OR BACK FACING PRIMITIIIES ARE NOT CULLED OUT,
SOFTWARE MAV C OOSE T ATTACH DIFFERENT TEXTURE AND MATERIAL TEXT UREAFRONT PARAMETER TO THE FRONT AND BACK FACING PRIMITIVES. Z
TEXT UREAFRONT PARTITION OF THE STATE VECTOR CONTAINS THE TEXA STATE APPLICABLE IF THE CURRENT PRIMITIVE IS FRONT FACING.
TEXTUREBFRONT PARTITION 0T THE STATE VECTOR CONTAINS THE TEXB IEXIUREBFRONT sTATE APPLICABLE IF THE CURRENT PRIMITIVE IS FRONT FACING. 6
TEXTUREABALK TEXTUREABACK PARTITION OF THE STATE VECTOR CONTAINS THE TEXA STATE APPLICABLE IF THE CURRENT PRIMITIVE IS BACK FACING.
TEXTUREBBACK TEXTUREBBACK PARTITION OR THE STATE vEcroR CONTAINS THE TEXB 6 STATE APPLICABLE IF THE CURRENT PRIMITIVE IS BACK FACING. MATERIALFRONT PARTITION OF THE STATE VECTOR CONTAINS THE
N
"ATEIIIMTRONT MATERIAL STATE APPLICABLE IF THE cuRREMT PRIMITIVE IS FRONT FACING. '0
MATERIMBACK MATERIALBACK PARTITION or THE mm mm CONTAINS THE MATERIAL ,0 WE APPLICABLE IF THE CURRENT PRIMITIVE IS BACK FACING.
LIGHT CONTAINS THE CURRENT LIGHT sTATE. I8 PIXELMODES CONTAINS THE CURRENT PIXEL MODES. 2
STIPPLE CONTAINS CURRENT STIPPLE PATTERN a CULLMODES CONTAINSCURRENTCULLMODES 0.33
"('2 FIG. 19 DIRTY POLVGON MEMORY 4m ‘198 _
MExsTATEvEcToR FLAGS POINTER WHOM MLMPOINTERS
4'°Z\_FR0NTMATER|AL l2] IFRONTMATERIALPTRI- Mmmuo h FACE POLYGON MEMORY, MATERIALPTR
4'°‘'\_ W mm [2 BACIIMATERIALPTII. UPDATE PTR,AND ‘"1 4m CLEARDIRTYBIT 4l46 4'48
4I°T\FRONUEXTUREA [:l IFRONTTEXTUREAPTRI- 4M0 _ 4|24 TAcE TEXTUREAPTR
4 ' 08X IZI WKTEXT'JREAPTR sAvE FRONTTEXTB TO BMKIEXTUREA 4m POLYGON MEMORY
4I'°\FRONHEXTUREB E] FRONTTEXTUREBPTR UPDATE PTR,AHO’ IEXIUREBPIR mg m; CLEARDIRTVBIT 4'50
4"“- IZI BMKTEXTUREBPTR SAVELIGIITTO 4M2 4'52 BACIITEXTUREB f 4|3o POLVGON MEMORY, m
4"4\_ LIGHT |Z| | LIGHTPTR t- UPDATE PTR,AND 4“, ‘ mLAiwfm 4m IXE E D~ PIXELMODES IZI P'XMODEPTI 4'34 SHIILIGOHQROITIO PIXMODEPTR 4| l8 UPDATE PTR. AND
\_ SHPPLE I] STIPPLE4PIT3R6 CLEARDIRWB" STIPPLEPTR ASSUMES THAT POLVGONS ARE FRONT FACING. 4'56
FIG. 20