Numerical Robustness for Numerical Robustness for Geometric CalculationsGeometric Calculations

Christer EricsonChrister Ericson

Sony Computer Sony Computer EntertainmentEntertainment

Slides @Slides @ http://realtimecollisiondetection.net/pubs/http://realtimecollisiondetection.net/pubs/

Numerical Robustness for Numerical Robustness for Geometric CalculationsGeometric Calculations

Christer EricsonChrister Ericson

Sony Computer Sony Computer EntertainmentEntertainment

Slides @Slides @ http://realtimecollisiondetection.net/pubs/http://realtimecollisiondetection.net/pubs/

EPSILON is NOT

0.00001!

TakeawayTakeaway An appreciation of the pitfalls An appreciation of the pitfalls

inherent in working with floating-inherent in working with floating-point arithmetic.point arithmetic.

Tools for addressing the robustness Tools for addressing the robustness of floating-point based code.of floating-point based code.

Probably something else too.Probably something else too.

Somewhat shameless Somewhat shameless plug!plug!

The contents of this presentation is The contents of this presentation is discussed in further detail in my discussed in further detail in my book:book:

THE PROBLEMTHE PROBLEMFloating-point arithmeticFloating-point arithmetic

Floating-point numbersFloating-point numbers Real numbers must be approximatedReal numbers must be approximated

Floating-point numbersFloating-point numbers Fixed-point numbers (integers)Fixed-point numbers (integers) Rational numbersRational numbers

Homogeneous representationHomogeneous representation

If we could work in real arithmetic, I If we could work in real arithmetic, I wouldn’t be having this talk!wouldn’t be having this talk!

Floating-point numbersFloating-point numbers IEEE-754 single precisionIEEE-754 single precision

1 bit sign1 bit sign 8 bit exponent (biased)8 bit exponent (biased) 23 bits fraction (24 bits mantissa, including hidden 23 bits fraction (24 bits mantissa, including hidden

bit)bit)

ss Exponent (e)Exponent (e) Fraction (f)Fraction (f)

This is a This is a normalizednormalized format format

31 31 23 22 0

127( 1) (1. ) 2s eV f

Floating-point numbersFloating-point numbers IEEE-754 representable numbers:IEEE-754 representable numbers:

Exponent

Fraction

Sign

Value

0<e<255

e=0 f=0 s=0

e=0 f=0 s=1

e=0 f≠0

e=255 f=0 s=0

e=255 f=0 s=1

e=255 f≠0

127( 1) (1. ) 2s eV f 0V

0V 126( 1) (0. ) 2s eV f

V Inf

V Inf

V NaN

Floating-point numbersFloating-point numbers In IEEE-754, domain extended with:In IEEE-754, domain extended with:

––Inf, +Inf, NaNInf, +Inf, NaN Some examples:Some examples:

a/0 = +Inf, if a > 0a/0 = +Inf, if a > 0 a/0 = –Inf, if a < 0a/0 = –Inf, if a < 0 0/0 = Inf – Inf = ±Inf · 0 = NaN0/0 = Inf – Inf = ±Inf · 0 = NaN

Known as Known as Infinity ArithmeticInfinity Arithmetic ( (IAIA))

Floating-point numbersFloating-point numbers IA is a potential source of robustness IA is a potential source of robustness

errors!errors! +Inf and –Inf compare as normal+Inf and –Inf compare as normal But NaN compares as unorderedBut NaN compares as unordered

NaN != NaN is trueNaN != NaN is true All other comparisons involving NaNs are falseAll other comparisons involving NaNs are false

These expressions are not equivalent:These expressions are not equivalent: if (a > b) X(); else Y();

if (a <= b) Y(); else X();

Floating-point numbersFloating-point numbers But IA provides a nice feature tooBut IA provides a nice feature too Allows not having to test for div-by-Allows not having to test for div-by-

zerozero Removes test branch from inner loopRemoves test branch from inner loop Useful for SIMD codeUseful for SIMD code

(Although same approach usually (Although same approach usually works for non-IEEE CPUs too.)works for non-IEEE CPUs too.)

Floating-point numbersFloating-point numbers Irregular number lineIrregular number line

Spacing increases the farther away Spacing increases the farther away from zero a number is locatedfrom zero a number is located

Number range for exponent Number range for exponent k+1k+1 has has twice the spacing of the one for twice the spacing of the one for exponent exponent kk

Equally many representable numbers Equally many representable numbers from one exponent to anotherfrom one exponent to another

0

Floating-point numbersFloating-point numbers Consequence of irregular spacing:Consequence of irregular spacing:

––101020 20 + (10+ (102020 + 1) = 0 + 1) = 0 (–10(–1020 20 + 10+ 1020 20 ) + 1 = 1) + 1 = 1

Thus, not associative (in general):Thus, not associative (in general): (a + b) + c != a + (b + c)(a + b) + c != a + (b + c)

Source of endless errors!Source of endless errors!

0

Floating-point numbersFloating-point numbers All discrete representations have All discrete representations have

non-representable pointsnon-representable points

A

BC

D

Q

P

The floating-point gridThe floating-point grid In floating-point, behavior changes based In floating-point, behavior changes based

on position, due to the irregular spacing! on position, due to the irregular spacing!

EXAMPLEEXAMPLEPolygon splittingPolygon splitting

Polygon splittingPolygon splitting Sutherland-Hodgman clipping algorithmSutherland-Hodgman clipping algorithm

A

B BCI

D

C

A DJ

Polygon splittingPolygon splitting Enter floating-point errors!Enter floating-point errors!

I

P

Q

P

Q

IF

Polygon splittingPolygon splitting ABCDABCD split against a plane split against a plane

A

BD

C

A

BD

J

I

C

Polygon splittingPolygon splitting ThickThick planes to the rescue! planes to the rescue!

P

Q

Desired invariant:OnPlane(I, plane) = true

where:I = IntersectionPoint(PQ, plane)

Polygon splittingPolygon splitting Thick planes also help bound the errorThick planes also help bound the error

PQ P'Q'

PQ

e

e

P'Q'

A

BD

C

Polygon splittingPolygon splitting ABCDABCD split against a split against a thickthick plane plane

A

BD

C

I

Polygon splittingPolygon splitting CracksCracks introduced by inconsistent ordering introduced by inconsistent ordering

A

B

C

D

A

B

C

D

EXAMPLEEXAMPLE(K-d) tree robustness(K-d) tree robustness

(K-d) tree robustness(K-d) tree robustness Robustness problems for:Robustness problems for:

Insertion of primitivesInsertion of primitives Querying (collision detection)Querying (collision detection)

Same problems apply to:Same problems apply to: All spatial partitioning schemes!All spatial partitioning schemes! (BSP trees, grids, octrees, quadtrees, …)(BSP trees, grids, octrees, quadtrees, …)

(K-d) tree robustness(K-d) tree robustness Query robustnessQuery robustness

1

2I

P

Q

IF

CA

B

(K-d) tree robustness(K-d) tree robustness Insertion robustnessInsertion robustness

1

2

CA

B

1

2

CA

B

I

IF

(K-d) tree robustness(K-d) tree robustness How to achieve robustness?How to achieve robustness?

Insert primitives conservativelyInsert primitives conservatively Accounting for errors in querying and Accounting for errors in querying and

insertioninsertion Can then ignore problem for queriesCan then ignore problem for queries

EXAMPLEEXAMPLERay-triangle testRay-triangle test

Ray-triangle testRay-triangle test Common approach:Common approach:

Compute intersection point Compute intersection point PP of ray of ray RR with plane of triangle with plane of triangle TT

Test if Test if PP lies inside boundaries of lies inside boundaries of TT Alas, this is not robust!Alas, this is not robust!

Ray-triangle testRay-triangle test A problem configurationA problem configuration

R

P

A

BC

D

Ray-triangle testRay-triangle test Intersecting Intersecting RR against one plane against one plane

R

P

Ray-triangle testRay-triangle test Intersecting Intersecting RR against the other plane against the other plane

R

P

( )R t O t d( )OA OB d d

Ray-triangle testRay-triangle test Robust test must Robust test must share calculationsshare calculations

for shared edge for shared edge ABAB Perform test directly in 3D!Perform test directly in 3D!

Let ray beLet ray be Then, sign of says whether Then, sign of says whether

is left or right of is left or right of ABAB If If RR left of all edges, left of all edges, RR intersects CCW intersects CCW

triangletriangle Only then Only then compute compute PP

Still errors, but managableStill errors, but managable

( )R t O t d( )OA OB d d

Ray-triangle testRay-triangle test ““Fat” tests are also robust!Fat” tests are also robust!

P

EXAMPLES SUMMARYEXAMPLES SUMMARY Achieve robustness through…Achieve robustness through…

(Correct) use of tolerances(Correct) use of tolerances Sharing of calculationsSharing of calculations Use of fat primitivesUse of fat primitives

TOLERANCESTOLERANCES

Tolerance comparisonsTolerance comparisons Absolute toleranceAbsolute tolerance Relative toleranceRelative tolerance Combined toleranceCombined tolerance Integer testInteger test

Absolute toleranceAbsolute tolerance

Almost never used correctly!Almost never used correctly! What should EPSILON be?What should EPSILON be?

Typically arbitrary small number used! OMFG!!Typically arbitrary small number used! OMFG!!

if (Abs(x – y) <= EPSILON) …

Comparing two floats for equality:Comparing two floats for equality:

Absolute tolerancesAbsolute tolerances

Delta step to next representable number:Delta step to next representable number:

Decimal Hex Next representable number

10.0 0x41200000

x + 0.000001

100.0 0x42C80000

x + 0.000008

1,000.0 0x447A0000

x + 0.000061

10,000.0 0x461C4000

x + 0.000977

100,000.0 0x47C35000

x + 0.007813

1,000,000.0 0x49742400

x + 0.0625

10,000,000.0

0x4B189680

x + 1.0

Absolute tolerancesAbsolute tolerances

Möller-Trumbore ray-triangle code:Möller-Trumbore ray-triangle code:

#define EPSILON 0.000001#define DOT(v1,v2) (v1[0]*v2[0]+v1[1]*v2[1]+v1[2]*v2[2]).../* if determinant is near zero, ray lies in plane of triangle */det = DOT(edge1, pvec);...if (det > -EPSILON && det < EPSILON) // Abs(det) < EPSILON return 0;

Written using doubles.Written using doubles. Change to float without changing epsilon?Change to float without changing epsilon? DOT({10,10,10},{10,10,10}) breaks test!DOT({10,10,10},{10,10,10}) breaks test!

Relative toleranceRelative tolerance

Comparing two floats for equality:Comparing two floats for equality:

Epsilon scaled Epsilon scaled byby magnitude of inputs magnitude of inputs But consider Abs(x)<1.0, Abs(y)<1.0But consider Abs(x)<1.0, Abs(y)<1.0

if (Abs(x – y) <= EPSILON * Max(Abs(x), Abs(y)) …

Combined toleranceCombined tolerance

Comparing two floats for equality:Comparing two floats for equality:

Absolute test for Abs(x)≤1.0, Abs(y)≤1.0Absolute test for Abs(x)≤1.0, Abs(y)≤1.0 Relative test otherwise!Relative test otherwise!

if (Abs(x – y) <= EPSILON * Max(1.0f, Abs(x), Abs(y)) …

Integer testInteger test

bool AlmostEqual2sComplement(float A, float B, int maxUlps){ // Make sure maxUlps is non-negative and small enough that the // default NAN won't compare as equal to anything. assert(maxUlps > 0 && maxUlps < 4 * 1024 * 1024); int aInt = *(int*)&A; // Make aInt lexicographically ordered as a twos-complement int if (aInt < 0) aInt = 0x80000000 - aInt; // Make bInt lexicographically ordered as a twos-complement int int bInt = *(int*)&B; if (bInt < 0) bInt = 0x80000000 - bInt; int intDiff = abs(aInt - bInt); if (intDiff <= maxUlps) return true; return false;}

Also suggested, an integer-based test:Also suggested, an integer-based test:

Floating-point numbersFloating-point numbers

Caveat: Intel uses 80-bit format Caveat: Intel uses 80-bit format internallyinternally Unless told otherwise.Unless told otherwise. Errors dependent on what code Errors dependent on what code

generated.generated. Gives different results in debug and Gives different results in debug and

release.release.

EXACTEXACTARITHMETICARITHMETIC

(and semi-exact ditto)(and semi-exact ditto)

Exact arithmeticExact arithmetic Hey! Integer arithmetic is exactHey! Integer arithmetic is exact

As long as there is no overflowAs long as there is no overflow Closed under +, –, and *Closed under +, –, and * Not closed under / but can often remove Not closed under / but can often remove

divisions through cross multiplicationdivisions through cross multiplication

Exact arithmeticExact arithmetic Example: Does Example: Does CC project onto project onto AB AB ??

A B

C

D

float t = Dot(AC, AB) / Dot(AB, AB);if (t >= 0.0f && t <= 1.0f) ... /* do something */

int tnum = Dot(AC, AB), tdenom = Dot(AB, AB);if (tnum >= 0 && tnum <= tdenom) ... /* do something */

,AC AB

D A tAB tAB AB

,

AC ABD A tAB t

AB AB

Floats:Floats:

Integers:Integers:

Exact arithmeticExact arithmetic Another example:Another example:

A

B

C

D

Exact arithmeticExact arithmetic TestsTests

Boolean, can be evaluated exactlyBoolean, can be evaluated exactly ConstructionsConstructions

Non-Boolean, cannot be done exactlyNon-Boolean, cannot be done exactly

Exact arithmeticExact arithmetic Tests, often expressed as determinant Tests, often expressed as determinant

predicates. E.g.predicates. E.g.

Shewchuk's predicates well-known exampleShewchuk's predicates well-known example Evaluates using extended-precision arithmetic Evaluates using extended-precision arithmetic

(EPA)(EPA) EPA is expensive to evaluateEPA is expensive to evaluate

Limit EPA use through “floating-point filter”Limit EPA use through “floating-point filter” Common filter is interval arithmeticCommon filter is interval arithmetic

( , , ) 0 ( ) 0x y z

x y z

x y z

u u u

P v v v

w w w

u v w u v w( , , ) 0 ( ) 0x y z

x y z

x y z

u u u

P v v v

w w w

u v w u v w

Exact arithmeticExact arithmetic Interval arithmeticInterval arithmetic

x = [1,3] = { x x = [1,3] = { x R | 1 ≤ x ≤ 3 } R | 1 ≤ x ≤ 3 } Rules:Rules:

[a,b] + [c,d] = [a+c,b+d][a,b] + [c,d] = [a+c,b+d] [a,b] – [c,d] = [a–d,b–c][a,b] – [c,d] = [a–d,b–c] [a,b] * [c,d] = [min(ac,ad,bc,bd), [a,b] * [c,d] = [min(ac,ad,bc,bd),

max(ac,ad,bc,bd)]max(ac,ad,bc,bd)] [a,b] / [c,d] = [a,b] * [1/d,1/c] for 0[a,b] / [c,d] = [a,b] * [1/d,1/c] for 0[c,d][c,d]

E.g. [100,101] + [10,12] = [110,113]E.g. [100,101] + [10,12] = [110,113]

Exact arithmeticExact arithmetic Interval arithmeticInterval arithmetic

Intervals must be rounded up/down to Intervals must be rounded up/down to nearest machine-representable numbernearest machine-representable number

Is a reliable calculationIs a reliable calculation

ReferencesReferences Ericson, Christer. Ericson, Christer. Real-Time Collision DetectionReal-Time Collision Detection. Morgan Kaufmann, . Morgan Kaufmann,

2005. 2005. http://http://realtimecollisiondetection.net/realtimecollisiondetection.net/ Hoffmann, Christoph. Hoffmann, Christoph. Geometric and Solid Modeling: An IntroductionGeometric and Solid Modeling: An Introduction. .

Morgan Kaufmann, 1989.Morgan Kaufmann, 1989. Ratschek, Helmut. Jon Rokne. Ratschek, Helmut. Jon Rokne. Geometric Computations with Interval Geometric Computations with Interval

and New Robust Methodsand New Robust Methods. Horwood Publishing, 2003.. Horwood Publishing, 2003.

Hoffmann, Christoph. “Hoffmann, Christoph. “Robustness in Geometric ComputationsRobustness in Geometric Computations.” JCISE 1, 2001, .” JCISE 1, 2001, pp. 143-156. pp. 143-156. http://www.cs.purdue.edu/homes/cmh/distribution/papers/Robustness/robust4.pdfhttp://www.cs.purdue.edu/homes/cmh/distribution/papers/Robustness/robust4.pdf

Santisteve, Francisco. “Santisteve, Francisco. “Robust Geometric Computation (RGC), State of the Robust Geometric Computation (RGC), State of the ArtArt.” Technical report, 1999. .” Technical report, 1999. http://www.lsi.upc.es/dept/techreps/ps/R99-19.ps.gzhttp://www.lsi.upc.es/dept/techreps/ps/R99-19.ps.gz

Schirra, Stefan. “Schirra, Stefan. “Robustness and precision issues in geometric computation.Robustness and precision issues in geometric computation.” ” Research Report MPI-I-98-004, Max Planck Institute for Computer Science, 1998. Research Report MPI-I-98-004, Max Planck Institute for Computer Science, 1998. httphttp://domino.mpi-sb.mpg.de/internet/reports.nsf/NumberView/1998-1-004://domino.mpi-sb.mpg.de/internet/reports.nsf/NumberView/1998-1-004

Shewchuk, Jonathan. “Shewchuk, Jonathan. “Adaptive Precision Floating-Point Arithmetic and Fast Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates.Robust Geometric Predicates.” Discrete & Computational Geometry 18(3):305-” Discrete & Computational Geometry 18(3):305-363, October 1997. 363, October 1997. http://www.cs.cmu.eduhttp://www.cs.cmu.edu/~quake-papers/robust-arithmetic.ps/~quake-papers/robust-arithmetic.ps

BOOKS

PAPERS