Post on 26-Mar-2015
Numerical Robustness for Numerical Robustness for Geometric CalculationsGeometric Calculations
Christer EricsonChrister Ericson
Sony Computer Sony Computer EntertainmentEntertainment
Slides @Slides @ http://realtimecollisiondetection.net/pubs/http://realtimecollisiondetection.net/pubs/
Numerical Robustness for Numerical Robustness for Geometric CalculationsGeometric Calculations
Christer EricsonChrister Ericson
Sony Computer Sony Computer EntertainmentEntertainment
Slides @Slides @ http://realtimecollisiondetection.net/pubs/http://realtimecollisiondetection.net/pubs/
EPSILON is NOT
0.00001!
TakeawayTakeaway An appreciation of the pitfalls An appreciation of the pitfalls
inherent in working with floating-inherent in working with floating-point arithmetic.point arithmetic.
Tools for addressing the robustness Tools for addressing the robustness of floating-point based code.of floating-point based code.
Probably something else too.Probably something else too.
Somewhat shameless Somewhat shameless plug!plug!
The contents of this presentation is The contents of this presentation is discussed in further detail in my discussed in further detail in my book:book:
THE PROBLEMTHE PROBLEMFloating-point arithmeticFloating-point arithmetic
Floating-point numbersFloating-point numbers Real numbers must be approximatedReal numbers must be approximated
Floating-point numbersFloating-point numbers Fixed-point numbers (integers)Fixed-point numbers (integers) Rational numbersRational numbers
Homogeneous representationHomogeneous representation
If we could work in real arithmetic, I If we could work in real arithmetic, I wouldn’t be having this talk!wouldn’t be having this talk!
Floating-point numbersFloating-point numbers IEEE-754 single precisionIEEE-754 single precision
1 bit sign1 bit sign 8 bit exponent (biased)8 bit exponent (biased) 23 bits fraction (24 bits mantissa, including hidden 23 bits fraction (24 bits mantissa, including hidden
bit)bit)
ss Exponent (e)Exponent (e) Fraction (f)Fraction (f)
This is a This is a normalizednormalized format format
31 31 23 22 0
127( 1) (1. ) 2s eV f
Floating-point numbersFloating-point numbers IEEE-754 representable numbers:IEEE-754 representable numbers:
Exponent
Fraction
Sign
Value
0<e<255
e=0 f=0 s=0
e=0 f=0 s=1
e=0 f≠0
e=255 f=0 s=0
e=255 f=0 s=1
e=255 f≠0
127( 1) (1. ) 2s eV f 0V
0V 126( 1) (0. ) 2s eV f
V Inf
V Inf
V NaN
Floating-point numbersFloating-point numbers In IEEE-754, domain extended with:In IEEE-754, domain extended with:
––Inf, +Inf, NaNInf, +Inf, NaN Some examples:Some examples:
a/0 = +Inf, if a > 0a/0 = +Inf, if a > 0 a/0 = –Inf, if a < 0a/0 = –Inf, if a < 0 0/0 = Inf – Inf = ±Inf · 0 = NaN0/0 = Inf – Inf = ±Inf · 0 = NaN
Known as Known as Infinity ArithmeticInfinity Arithmetic ( (IAIA))
Floating-point numbersFloating-point numbers IA is a potential source of robustness IA is a potential source of robustness
errors!errors! +Inf and –Inf compare as normal+Inf and –Inf compare as normal But NaN compares as unorderedBut NaN compares as unordered
NaN != NaN is trueNaN != NaN is true All other comparisons involving NaNs are falseAll other comparisons involving NaNs are false
These expressions are not equivalent:These expressions are not equivalent: if (a > b) X(); else Y();
if (a <= b) Y(); else X();
Floating-point numbersFloating-point numbers But IA provides a nice feature tooBut IA provides a nice feature too Allows not having to test for div-by-Allows not having to test for div-by-
zerozero Removes test branch from inner loopRemoves test branch from inner loop Useful for SIMD codeUseful for SIMD code
(Although same approach usually (Although same approach usually works for non-IEEE CPUs too.)works for non-IEEE CPUs too.)
Floating-point numbersFloating-point numbers Irregular number lineIrregular number line
Spacing increases the farther away Spacing increases the farther away from zero a number is locatedfrom zero a number is located
Number range for exponent Number range for exponent k+1k+1 has has twice the spacing of the one for twice the spacing of the one for exponent exponent kk
Equally many representable numbers Equally many representable numbers from one exponent to anotherfrom one exponent to another
0
Floating-point numbersFloating-point numbers Consequence of irregular spacing:Consequence of irregular spacing:
––101020 20 + (10+ (102020 + 1) = 0 + 1) = 0 (–10(–1020 20 + 10+ 1020 20 ) + 1 = 1) + 1 = 1
Thus, not associative (in general):Thus, not associative (in general): (a + b) + c != a + (b + c)(a + b) + c != a + (b + c)
Source of endless errors!Source of endless errors!
0
Floating-point numbersFloating-point numbers All discrete representations have All discrete representations have
non-representable pointsnon-representable points
A
BC
D
Q
P
The floating-point gridThe floating-point grid In floating-point, behavior changes based In floating-point, behavior changes based
on position, due to the irregular spacing! on position, due to the irregular spacing!
EXAMPLEEXAMPLEPolygon splittingPolygon splitting
Polygon splittingPolygon splitting Sutherland-Hodgman clipping algorithmSutherland-Hodgman clipping algorithm
A
B BCI
D
C
A DJ
Polygon splittingPolygon splitting Enter floating-point errors!Enter floating-point errors!
I
P
Q
P
Q
IF
Polygon splittingPolygon splitting ABCDABCD split against a plane split against a plane
A
BD
C
A
BD
J
I
C
Polygon splittingPolygon splitting ThickThick planes to the rescue! planes to the rescue!
P
Q
Desired invariant:OnPlane(I, plane) = true
where:I = IntersectionPoint(PQ, plane)
Polygon splittingPolygon splitting Thick planes also help bound the errorThick planes also help bound the error
PQ P'Q'
PQ
e
e
P'Q'
A
BD
C
Polygon splittingPolygon splitting ABCDABCD split against a split against a thickthick plane plane
A
BD
C
I
Polygon splittingPolygon splitting CracksCracks introduced by inconsistent ordering introduced by inconsistent ordering
A
B
C
D
A
B
C
D
EXAMPLEEXAMPLE(K-d) tree robustness(K-d) tree robustness
(K-d) tree robustness(K-d) tree robustness Robustness problems for:Robustness problems for:
Insertion of primitivesInsertion of primitives Querying (collision detection)Querying (collision detection)
Same problems apply to:Same problems apply to: All spatial partitioning schemes!All spatial partitioning schemes! (BSP trees, grids, octrees, quadtrees, …)(BSP trees, grids, octrees, quadtrees, …)
(K-d) tree robustness(K-d) tree robustness Query robustnessQuery robustness
1
2I
P
Q
IF
CA
B
(K-d) tree robustness(K-d) tree robustness Insertion robustnessInsertion robustness
1
2
CA
B
1
2
CA
B
I
IF
(K-d) tree robustness(K-d) tree robustness How to achieve robustness?How to achieve robustness?
Insert primitives conservativelyInsert primitives conservatively Accounting for errors in querying and Accounting for errors in querying and
insertioninsertion Can then ignore problem for queriesCan then ignore problem for queries
EXAMPLEEXAMPLERay-triangle testRay-triangle test
Ray-triangle testRay-triangle test Common approach:Common approach:
Compute intersection point Compute intersection point PP of ray of ray RR with plane of triangle with plane of triangle TT
Test if Test if PP lies inside boundaries of lies inside boundaries of TT Alas, this is not robust!Alas, this is not robust!
Ray-triangle testRay-triangle test A problem configurationA problem configuration
R
P
A
BC
D
Ray-triangle testRay-triangle test Intersecting Intersecting RR against one plane against one plane
R
P
Ray-triangle testRay-triangle test Intersecting Intersecting RR against the other plane against the other plane
R
P
( )R t O t d( )OA OB d d
Ray-triangle testRay-triangle test Robust test must Robust test must share calculationsshare calculations
for shared edge for shared edge ABAB Perform test directly in 3D!Perform test directly in 3D!
Let ray beLet ray be Then, sign of says whether Then, sign of says whether
is left or right of is left or right of ABAB If If RR left of all edges, left of all edges, RR intersects CCW intersects CCW
triangletriangle Only then Only then compute compute PP
Still errors, but managableStill errors, but managable
( )R t O t d( )OA OB d d
Ray-triangle testRay-triangle test ““Fat” tests are also robust!Fat” tests are also robust!
P
EXAMPLES SUMMARYEXAMPLES SUMMARY Achieve robustness through…Achieve robustness through…
(Correct) use of tolerances(Correct) use of tolerances Sharing of calculationsSharing of calculations Use of fat primitivesUse of fat primitives
TOLERANCESTOLERANCES
Tolerance comparisonsTolerance comparisons Absolute toleranceAbsolute tolerance Relative toleranceRelative tolerance Combined toleranceCombined tolerance Integer testInteger test
Absolute toleranceAbsolute tolerance
Almost never used correctly!Almost never used correctly! What should EPSILON be?What should EPSILON be?
Typically arbitrary small number used! OMFG!!Typically arbitrary small number used! OMFG!!
if (Abs(x – y) <= EPSILON) …
Comparing two floats for equality:Comparing two floats for equality:
Absolute tolerancesAbsolute tolerances
Delta step to next representable number:Delta step to next representable number:
Decimal Hex Next representable number
10.0 0x41200000
x + 0.000001
100.0 0x42C80000
x + 0.000008
1,000.0 0x447A0000
x + 0.000061
10,000.0 0x461C4000
x + 0.000977
100,000.0 0x47C35000
x + 0.007813
1,000,000.0 0x49742400
x + 0.0625
10,000,000.0
0x4B189680
x + 1.0
Absolute tolerancesAbsolute tolerances
Möller-Trumbore ray-triangle code:Möller-Trumbore ray-triangle code:
#define EPSILON 0.000001#define DOT(v1,v2) (v1[0]*v2[0]+v1[1]*v2[1]+v1[2]*v2[2]).../* if determinant is near zero, ray lies in plane of triangle */det = DOT(edge1, pvec);...if (det > -EPSILON && det < EPSILON) // Abs(det) < EPSILON return 0;
Written using doubles.Written using doubles. Change to float without changing epsilon?Change to float without changing epsilon? DOT({10,10,10},{10,10,10}) breaks test!DOT({10,10,10},{10,10,10}) breaks test!
Relative toleranceRelative tolerance
Comparing two floats for equality:Comparing two floats for equality:
Epsilon scaled Epsilon scaled byby magnitude of inputs magnitude of inputs But consider Abs(x)<1.0, Abs(y)<1.0But consider Abs(x)<1.0, Abs(y)<1.0
if (Abs(x – y) <= EPSILON * Max(Abs(x), Abs(y)) …
Combined toleranceCombined tolerance
Comparing two floats for equality:Comparing two floats for equality:
Absolute test for Abs(x)≤1.0, Abs(y)≤1.0Absolute test for Abs(x)≤1.0, Abs(y)≤1.0 Relative test otherwise!Relative test otherwise!
if (Abs(x – y) <= EPSILON * Max(1.0f, Abs(x), Abs(y)) …
Integer testInteger test
bool AlmostEqual2sComplement(float A, float B, int maxUlps){ // Make sure maxUlps is non-negative and small enough that the // default NAN won't compare as equal to anything. assert(maxUlps > 0 && maxUlps < 4 * 1024 * 1024); int aInt = *(int*)&A; // Make aInt lexicographically ordered as a twos-complement int if (aInt < 0) aInt = 0x80000000 - aInt; // Make bInt lexicographically ordered as a twos-complement int int bInt = *(int*)&B; if (bInt < 0) bInt = 0x80000000 - bInt; int intDiff = abs(aInt - bInt); if (intDiff <= maxUlps) return true; return false;}
Also suggested, an integer-based test:Also suggested, an integer-based test:
Floating-point numbersFloating-point numbers
Caveat: Intel uses 80-bit format Caveat: Intel uses 80-bit format internallyinternally Unless told otherwise.Unless told otherwise. Errors dependent on what code Errors dependent on what code
generated.generated. Gives different results in debug and Gives different results in debug and
release.release.
EXACTEXACTARITHMETICARITHMETIC
(and semi-exact ditto)(and semi-exact ditto)
Exact arithmeticExact arithmetic Hey! Integer arithmetic is exactHey! Integer arithmetic is exact
As long as there is no overflowAs long as there is no overflow Closed under +, –, and *Closed under +, –, and * Not closed under / but can often remove Not closed under / but can often remove
divisions through cross multiplicationdivisions through cross multiplication
Exact arithmeticExact arithmetic Example: Does Example: Does CC project onto project onto AB AB ??
A B
C
D
float t = Dot(AC, AB) / Dot(AB, AB);if (t >= 0.0f && t <= 1.0f) ... /* do something */
int tnum = Dot(AC, AB), tdenom = Dot(AB, AB);if (tnum >= 0 && tnum <= tdenom) ... /* do something */
,AC AB
D A tAB tAB AB
,
AC ABD A tAB t
AB AB
Floats:Floats:
Integers:Integers:
Exact arithmeticExact arithmetic Another example:Another example:
A
B
C
D
Exact arithmeticExact arithmetic TestsTests
Boolean, can be evaluated exactlyBoolean, can be evaluated exactly ConstructionsConstructions
Non-Boolean, cannot be done exactlyNon-Boolean, cannot be done exactly
Exact arithmeticExact arithmetic Tests, often expressed as determinant Tests, often expressed as determinant
predicates. E.g.predicates. E.g.
Shewchuk's predicates well-known exampleShewchuk's predicates well-known example Evaluates using extended-precision arithmetic Evaluates using extended-precision arithmetic
(EPA)(EPA) EPA is expensive to evaluateEPA is expensive to evaluate
Limit EPA use through “floating-point filter”Limit EPA use through “floating-point filter” Common filter is interval arithmeticCommon filter is interval arithmetic
( , , ) 0 ( ) 0x y z
x y z
x y z
u u u
P v v v
w w w
u v w u v w( , , ) 0 ( ) 0x y z
x y z
x y z
u u u
P v v v
w w w
u v w u v w
Exact arithmeticExact arithmetic Interval arithmeticInterval arithmetic
x = [1,3] = { x x = [1,3] = { x R | 1 ≤ x ≤ 3 } R | 1 ≤ x ≤ 3 } Rules:Rules:
[a,b] + [c,d] = [a+c,b+d][a,b] + [c,d] = [a+c,b+d] [a,b] – [c,d] = [a–d,b–c][a,b] – [c,d] = [a–d,b–c] [a,b] * [c,d] = [min(ac,ad,bc,bd), [a,b] * [c,d] = [min(ac,ad,bc,bd),
max(ac,ad,bc,bd)]max(ac,ad,bc,bd)] [a,b] / [c,d] = [a,b] * [1/d,1/c] for 0[a,b] / [c,d] = [a,b] * [1/d,1/c] for 0[c,d][c,d]
E.g. [100,101] + [10,12] = [110,113]E.g. [100,101] + [10,12] = [110,113]
Exact arithmeticExact arithmetic Interval arithmeticInterval arithmetic
Intervals must be rounded up/down to Intervals must be rounded up/down to nearest machine-representable numbernearest machine-representable number
Is a reliable calculationIs a reliable calculation
ReferencesReferences Ericson, Christer. Ericson, Christer. Real-Time Collision DetectionReal-Time Collision Detection. Morgan Kaufmann, . Morgan Kaufmann,
2005. 2005. http://http://realtimecollisiondetection.net/realtimecollisiondetection.net/ Hoffmann, Christoph. Hoffmann, Christoph. Geometric and Solid Modeling: An IntroductionGeometric and Solid Modeling: An Introduction. .
Morgan Kaufmann, 1989.Morgan Kaufmann, 1989. Ratschek, Helmut. Jon Rokne. Ratschek, Helmut. Jon Rokne. Geometric Computations with Interval Geometric Computations with Interval
and New Robust Methodsand New Robust Methods. Horwood Publishing, 2003.. Horwood Publishing, 2003.
Hoffmann, Christoph. “Hoffmann, Christoph. “Robustness in Geometric ComputationsRobustness in Geometric Computations.” JCISE 1, 2001, .” JCISE 1, 2001, pp. 143-156. pp. 143-156. http://www.cs.purdue.edu/homes/cmh/distribution/papers/Robustness/robust4.pdfhttp://www.cs.purdue.edu/homes/cmh/distribution/papers/Robustness/robust4.pdf
Santisteve, Francisco. “Santisteve, Francisco. “Robust Geometric Computation (RGC), State of the Robust Geometric Computation (RGC), State of the ArtArt.” Technical report, 1999. .” Technical report, 1999. http://www.lsi.upc.es/dept/techreps/ps/R99-19.ps.gzhttp://www.lsi.upc.es/dept/techreps/ps/R99-19.ps.gz
Schirra, Stefan. “Schirra, Stefan. “Robustness and precision issues in geometric computation.Robustness and precision issues in geometric computation.” ” Research Report MPI-I-98-004, Max Planck Institute for Computer Science, 1998. Research Report MPI-I-98-004, Max Planck Institute for Computer Science, 1998. httphttp://domino.mpi-sb.mpg.de/internet/reports.nsf/NumberView/1998-1-004://domino.mpi-sb.mpg.de/internet/reports.nsf/NumberView/1998-1-004
Shewchuk, Jonathan. “Shewchuk, Jonathan. “Adaptive Precision Floating-Point Arithmetic and Fast Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates.Robust Geometric Predicates.” Discrete & Computational Geometry 18(3):305-” Discrete & Computational Geometry 18(3):305-363, October 1997. 363, October 1997. http://www.cs.cmu.eduhttp://www.cs.cmu.edu/~quake-papers/robust-arithmetic.ps/~quake-papers/robust-arithmetic.ps
BOOKS
PAPERS