1

Computation of scalar accuracy metrics LE, CE, and SE as both predictive and sample-

based statistics

John Dolloff [C] and Jacqueline Carr [C]

National Geospatial-Intelligence Agency (NGA)

7500 GEOINT Drive, Springfield, VA

Government POC: Christopher O’Neill (Christopher.M.Oneill@nga.mil); PA case #: 16-277

ABSTRACT

Scalar accuracy metrics often used to describe geospatial data quality are Linear Error (LE), Circular Error (CE),

and Spherical Error (SE), which correspond to vertical, 2d horizontal, and 3d radial errors, respectively, in a local

tangent plane coordinate system (e.g. East-North-Up). In addition, they also correspond to specifiable levels of

probability; for example, CE_50 is Circular Error probable, or CE at the 50% probability level. Typical probability

levels of interest are 50, 90, and 95%. Scalar accuracy metrics are very important for both near-real time accuracy

predictions corresponding to derived location information, and for post-analysis verification and validation of

system performance (location accuracy) requirements. The former normally correspond to predictive scalar

accuracy metrics and the latter usually correspond to sample-based scalar accuracy metrics. Predictive metrics are

generated from a priori error covariance matrices and mean-values, if non-zero, corresponding to the underlying (up

to 3) error components. Sample-based metrics are generated from independent and identically distributed (i.i.d.)

samples of error relative to “ground truth”. This paper details the proper but practical computation of both types of

scalar accuracy metrics, including corresponding confidence intervals – techniques with non-trivial computational

approximation errors are not included nor needed. Recommended sample-based metrics utilize order statistics

which require no assumptions regarding the underlying probability distributions of errors.

Keywords: Linear Error (LE), Circular Error (CE), Spherical Error (SE), probability, predictive statistics, sample

statistics, order statistics

1.0 Introduction Scalar accuracy metrics are commonly used to express the accuracy or corresponding error in an estimate of

geospatial location in a local tangent plane coordinate system (e.g. East-North-Up) in a way that is easy to visualize,

comprehend, and communicate. Common scalar accuracy metrics used include Linear Error (LE) for describing

vertical or elevation data accuracy and Circular Error (CE) for describing two dimensional horizontal data accuracy.

Accuracy of geospatial location in three dimensions is often described using either a combination of the LE and CE

scalar metrics into a representative CE-LE cylinder or by the single metric representation of Spherical Error (SE)

providing a radial representation of accuracy in three dimensions. Representing accuracy through the use of a

single number, or even pair of numbers, has many advantages as it provides simplified and intuitive representations,

a clear means of comparison, and minimal bandwidth requirements. This simplification of sometimes complex

information, however, is not without risk. There is not clear guidance or broad agreement on the best practices for

calculation of these commonly desired metrics which results in representations which may not be reliably compared,

such as to a desired requirement. Also, common computations of predictive statistics are typically excessive in their

approximations. Similarly, computations based on sample-based statistics may rely on incorrect assumptions of

error distribution and subsequently incorrectly reflect system or process performance. This paper considers these

challenges in scalar accuracy metric representation and provides guidance to fast, rigorous, and accurate methods to

robustly calculate these representative values.

2.0 Defining Scalar Accuracy Metrics LE, CE, and SE To clarify the discussion presented in this paper, it is important to set the stage with explicit descriptions of the

popular scalar accuracy metrics, LE, CE, and SE.

Linear Error (LE) - LE corresponds to the length of a vertical line (segment) such that there is a 90%

probability that the absolute value of vertical error resides along the line. If the line is doubled in length

and centered at the position solution, there is a 90% probability the true position vertical location resides

along the line. LE_XX corresponds to LE at the XX % probability level. See Figure 2.0-1.

2

Circular Error (CE) - CE corresponds to the radius of a circle such that there is a 90% probability that the

horizontal error resides within the circle, or equivalently, if the circle is centered at the position solution,

there is a 90% probability the true position horizontal location resides within the circle. CE_XX

corresponds to CE at the XX % probability level. See Figure 2.0-1.

Spherical Error (SE) - SE corresponds to the radius of a 3D sphere such that there is a 90% probability

that 3d error resides within, or equivalently, if the sphere is centered at the position solution, there is a 90%

probability the true position location resides within the sphere. SE_XX corresponds to SE at the XX %

probability level.

Figure 2.0-1: Graphic depiction of CE (left) and LE (right) for examples where CE = 4m and LE = 5m

For the above scalar accuracy metrics:

It is assumed that the underlying x-y-z coordinate system is a local tangent plane system, i.e., x and y are

horizontal components and z the vertical component.

CE-LE corresponds to the CE-LE error cylinder. There is a probability between 81 to 90 percent that 3d

radial error resides within the cylinder. The former value corresponds to uncorrelated horizontal and

vertical errors, the latter value to highly correlated horizontal and vertical errors. See Figure 2.0-2.

Figure 2.0-2: Example of CE-LE cylinder compared to corresponding error ellipsoid based on full covariance matrix

References for the definitions of scalar accuracy metrics include [6-7]; an application of scalar accuracy metrics

which also discusses error ellipsoids is [5].

3

As important means of conveying both near-real time accuracy predictions corresponding to derived location

information, and for post-analysis verification and validation of system performance (location accuracy)

requirements, scalar accuracy metrics are developed based on both predictive statistics and sample statistics.

Predictive statistics correspond to the mathematical modeling of assumed a priori error characteristics and are part

of a statistical error model. Predictive scalar accuracy metrics are generated from a priori error covariance matrices

and mean-values, if non-zero, corresponding to the underlying (up to 3) error components. Sample statistics

correspond to the analysis of a collection of error samples. Sample-based scalar accuracy metrics are generated

from independent and identically distributed (i.i.d.) samples of error relative to “ground truth”.

The following sections of this paper detail the proper but practical computation of both types of scalar accuracy

metrics, including corresponding confidence intervals – techniques with non-trivial computational approximation

errors are not included nor needed with the availability of today’s computer systems. Sample-based techniques

include the recommended use of order statistics which require no assumptions regarding the underlying probability

distributions of errors.

3.0 Scalar Accuracy Metrics: Computations based on Predictive Statistics This section of the document presents the evaluation of the scalar accuracy metrics based on the predictive statistics:

(1) 𝑛𝑥1 mean-value 𝜖𝑋̅̅̅̅ , and (2) 𝑛𝑥𝑛 error covariance matrix 𝐶𝑋; where 𝑛 = 1 for the computation of LE_XX,

𝑛 = 2 for the computation of CE_XX, and 𝑛 = 3 for the computation of SE_XX. The mean-value is typically zero

for predictive statistics, although a non-zero mean-value is also accounted for in the following. A (multi-variate)

Gaussian (normal) distribution is also assumed for the underlying random variables corresponding to location error

(𝜖𝑋 = [𝜖𝑥 𝜖𝑦 𝜖𝑧]𝑇). This is necessary in order to assign probabilities; e.g., CE_90 corresponds to 90%

probability. Note that radial error 𝜖𝑟 is defined as follows: 𝜖𝑟 = |𝜖𝑧|, 𝜖𝑟 = √𝜖𝑥2 + 𝜖𝑦2, ,𝜖𝑟 = √𝜖𝑥2 + 𝜖𝑦2 + 𝜖𝑧2,

for LE, CE, and SE, respectively. (Note that confidence intervals are not computed for predictive-based scalar

accuracy metrics as the underlying mean-value and error covariance matrix are assumed a “given”.)

3.1 LE LE_XX is defined as that line length L such that:

𝑝 =1

(2𝜋)1/2𝜎𝑧∫ 𝑒−1/2((𝑧−�̅�)2/𝜎𝑧

2) 𝑑𝑧, (3.1-1)

integrated over the region √𝑧2 ≤ 𝐿, and where probability 𝑝 = 𝑋𝑋/100, 1d error 𝜖𝑋 = 𝜖𝑧 is defined as 𝑧 for

notational convenience, with mean-value 𝜖𝑋̅̅̅̅ defined as 𝑧̅, and 1 × 1 error covariance matrix 𝐶𝑋 about the mean

defined as 𝜎𝑧2. Note that if the mean-value is not zero, the length 𝐿 is still relative to the origin per the standard

definition of LE_XX.

If we assume that the mean-value of error is zero, and change variables such that 𝑧/(𝜎𝑧√2) → 𝑧∗, Equation 3.1-1

can be rewritten as:

𝑝 =2𝜎𝑧√2

(2𝜋)1/2𝜎𝑧∫ 𝑒−𝑧∗2 𝑑𝑧∗𝐿∗

0 =

2

(𝜋)1/2 ∫ 𝑒−𝑧∗2 𝑑𝑧∗ ≡𝐿∗

0erf (𝑧 ∗), where 𝐿∗ = 𝐿/(𝜎𝑧√2). (3.1-2)

Thus, since erf (Error Function) is a well-tabulated function and its inverse available via MATLAB and other

programming languages, we have by definition, erf 𝑖𝑛𝑣(𝑝) = erf 𝑖𝑛𝑣(𝑋𝑋/100) = 𝐿∗; thus, and accounting for the

change of variables:

𝐿𝐸_𝑋𝑋 = 𝜎𝑧√2 × 𝑒𝑟𝑓𝑖𝑛𝑣(𝑋𝑋

100), and specifically, (3.1-3)

𝐿𝐸_𝑋𝑋 = 𝐿(𝑝)𝜎𝑧, where 𝑝 = 𝑋𝑋/100 and the multiplier L(p) is listed in Table 5.1-1: (3.1-4)

4

Table 3.1-1: Linear Error (LE) multiplier L(p) versus probability level p

The light blue entries include the standard probability levels of interest. Thus, for example, L(p)=1.6499 is

applicable for LE_90. The violet entries are others of general interest. For example, p=0.9973 is the “three-sigma”

level of probability. If the desired probability level is different than any of the above, simply evaluate Equation 3.1-

3 using the desired value for XX. If the mean-value for error is not equal to zero, solve Equation 3.1-1 directly

using iteration and numerical integration.

Examples are as follows:

(1) Assume a desired probability level of 90%, a mean error of zero, and 𝐶𝑋 ≡ 𝜎𝑧2 = [9] meters-squared: 𝐿𝐸_90 =

𝐿 × 𝜎𝑧 = 1.6499 × 3 = 4.95 meters.

(2) Assume a desired probability level of 90%, a mean predictive error equal to �̅�𝑇 ≡ 𝑧̅ = [−2], and 𝐶𝑋 ≡ 𝜎𝑧2 = [9]

meters-squared. Thus, the Integral Equation 3.1-1 is applicable:

𝐿𝐸_90 = 5.976 meters.

The solution corresponding to the first example was virtually instantaneously, while the solution corresponding to

the second example took on the order of 0.02 seconds using non-optimized MATLAB code on a notebook computer.

The calculation error was negligible for all.

3.2 CE CE_XX is defined as that circular radius 𝑅 such that:

𝑝 =1

(2π))det (𝐶𝑋)1/2 ∬ 𝑒−1/2((𝑋−�̅�)𝑇𝐶𝑋−1(𝑋−�̅�) 𝑑𝑥𝑑𝑦, (3.2-1)

integrated over the region √𝑥2 + 𝑦2 ≤ 𝑅, and where probability 𝑝 = 𝑋𝑋/100, 2d error 𝜀𝑋𝑇 = [𝜖𝑥 𝜖𝑦] is defined

as 𝑋𝑇 = [𝑥 𝑦] for notational convenience, with mean-value 𝜖𝑋̅̅̅̅ defined as �̅�𝑇 = [�̅� �̅�], and 2 × 2 error

covariance matrix about the mean defined as 𝐶𝑋. Note that if the mean-value is not zero, the radius 𝑅 is still relative

to the origin [0 0]𝑇, per the standard definition of CE_XX.

Assuming a mean-value of zero and an additional change of variables (x*,y* ) relative to a eigenvector aligned

system and scaled by the square-roots of the two eigenvalues of the covariance matrix, Equation 3.2-1can also be

written as:

𝑝 =1

(2𝜋)∬ 𝑒−1/2(𝑥∗2+𝑦∗2)𝑑𝑥∗𝑑𝑦∗ (3.2-2)

integrated over the region √𝑥∗2 + 𝑟2𝑦∗2 ≤ 𝑅/𝜎𝑚𝑎𝑥 , where 𝑟 = 𝜎𝑚𝑖𝑛/𝜎𝑚𝑎𝑥 , and 𝜎𝑚𝑖𝑛 and 𝜎𝑚𝑎𝑥 are the square-

root of the minimum eigenvalue and maximum eigenvalue, respectively, of the covariance matrix 𝐶𝑋.

Therefore, the value 𝑅 = 𝑅(𝑝, 𝑟), such that the above integral equals the desired level of probability p, is related to

CE_XX as follows:

𝐶𝐸_𝑋𝑋 = 𝑅(𝑝, 𝑟)𝜎𝑚𝑎𝑥. (3.2-3)

The following table presents the pre-computed values of 𝑅(𝑝, 𝑟) for various probability levels. In particular,

columns 2-5 correspond to p=0.5, 0.9. 0.95, 0.99, and 0.999, respectively, or alternatively, to XX=50, 90, 95, 99,

and 99.9 %, respectively.

p=0.5 p=0.6827 p=0.90 p=0.95 p=9545 p=0.99 p=0.9973 p=0.999

L(p) 0.6745 1.0000 1.6499 1.9600 2.0000 2.5758 3.0000 3.2905

Probabilities

5

Table 3.2-1: Circular Error (CE) multiplier 𝑅(𝑝, 𝑟) versus probability level p and ratio r

CE_XX is computed as 𝐶𝐸_𝑋𝑋 = 𝑅∗𝜎𝑚𝑎𝑥, where the normalized radius 𝑅∗ is computed as the linear interpolation

of 𝑅(𝑋𝑋/100, 𝑟) from the corresponding column of Table 3.2-1. Computation time is virtually instantaneous and

its accuracy relative to the true value of CE_XX is typically on the order of 0.01 % (experimental max of 0.1%).

Therefore, if the true value of CE_90=2 meters, we expect the computed value to be CE_XX=2 +/- 0.0002 meters.

If the mean-value for error is not equal to zero, one can solve Equation 3.2-1 directly using iteration and numerical

integration. However, this solution may have convergence problems, so the Monte-Carlo method of subsection

3.2.1 is recommended instead. Its evaluation via MATLAB pseudo-code takes on the order of 0.08 seconds on a

high-end notebook computer, and its accuracy relative to the true value of CE_XX is typically on the order of 0.05

% (experimental max of 0.6%).

3.2.1 Monte-Carlo Method for Computation of CE

The following approach to the computation of CE_XX is applicable to arbitrary mean-values and arbitrary

probability levels, is computationally accurate and reasonably fast:

(1) Compute 1E6 independent samples of the 2x1 horizontal error: 𝑠𝑖 = �̅� + 𝐶𝑋1/2

𝑛𝑖 , 𝑖 = 1, . . ,1𝐸6, (3.2.1-1)

where �̅� and 𝐶𝑋 are the 2x1 mean and the 2x2 error covariance about the mean relative to the original (non-

eigenvector aligned) coordinate system, 𝑛𝑖 is a two-element vector with each element the realization of an

independent Gaussian or normal 𝑁(0,1) random variable, and where the superscript “1/2” indicates principal

matrix square root. �̅� and 𝑛𝑖 are 2x1 vectors, and 𝐶𝑋1/2

is a 2x2 matrix. Also, 𝑠𝑖 is a Gaussian distributed random

vector with mean �̅� since it is a linear function of the mean-zero random vector 𝑛𝑖 and added to �̅�.

(2) Order the magnitudes of the error samples 𝑠𝑖 from smallest to largest, and designate 𝑅𝐸𝑋𝑋 the XX_th percent

largest, and 𝑅𝐸𝑋𝑋∗ the next largest magnitude. (Magnitudes equal √𝑠(1)𝑖

2 + 𝑠(2)𝑖2.)

(3) CE_XX=(𝑅𝐸𝑋𝑋 + 𝑅𝐸𝑋𝑋∗ )/2.

Note that the symmetric 𝐶𝑋1/2

is computed once prior to generating the independent samples, and the samples

𝑠𝑖 = 𝐶𝑋1/2

𝑛𝑖 are consistent with the error covariance matrix about the mean:

p=0.5 p=0.9 p=0.95 p=0.99 p=0.999

0.00 0.6745 1.6449 1.9600 2.5758 3.2905

0.05 0.6763 1.6456 1.9606 2.5763 3.2910

0.10 0.6820 1.6479 1.9625 2.5778 3.2921

0.15 0.6916 1.6518 1.9658 2.5803 3.2940

0.20 0.7059 1.6573 1.9704 2.5838 3.2967

0.25 0.7254 1.6646 1.9765 2.5884 3.3003

0.30 0.7499 1.6738 1.9842 2.5942 3.3049

0.35 0.7779 1.6852 1.9937 2.6013 3.3104

0.40 0.8079 1.6992 2.0051 2.6099 3.3172

0.45 0.8389 1.7163 2.0190 2.6203 3.3252

0.50 0.8704 1.7371 2.0359 2.6326 3.3346

0.55 0.9021 1.7621 2.0564 2.6474 3.3459

0.60 0.9337 1.7915 2.0813 2.6653 3.3595

0.65 0.9651 1.8251 2.1111 2.6875 3.3759

0.70 0.9962 1.8625 2.1460 2.7151 3.3965

0.75 1.0271 1.9034 2.1858 2.7492 3.4227

0.80 1.0577 1.9472 2.2303 2.7907 3.4570

0.85 1.0880 1.9936 2.2791 2.8401 3.5018

0.90 1.1181 2.0424 2.3318 2.8974 3.5594

0.95 1.1479 2.0932 2.3881 2.9625 3.6310

1.00 1.1774 2.1460 2.4478 3.0349 3.7169

Ratio

r

Probabilities

6

𝐸{(𝑠𝑖 − �̅�)(𝑠𝑖 − �̅�)𝑇}=𝐸{ 𝐶𝑋1/2

𝑛𝑖(𝐶𝑋1/2

𝑛𝑖 )𝑇}=𝐶𝑋

1/2𝐸{𝑛𝑖 𝑛𝑖

𝑇}𝐶𝑋1/2

=𝐶𝑋1/2

𝐼2𝑥2 𝐶𝑋1/2

=𝐶𝑋,where 𝐸{ } is the expected

value operator.

Note that MATLAB pseudo-code for the above takes advantage of efficiencies such as: (1) evaluating 𝐶𝑋1/2

only

once and use of the MATLAB sqrtm function, (2) generating all 1E6 samples of 𝑛𝑖 simultaneously using the

MATLAB function randn(2,1E6), and (3) ordering the magnitude samples using the MATLAB sort function.

(Magnitudes-squared can also be used in step 2 of Equation/Algorithm 3.2.1-1 to avoid taking square-roots with

step 3 modified appropriately to reduce computation time about 15%.)

The following examples assumes a mean-value �̅�𝑇 = [10 5] meters, and an error covariance matrix about the

mean 𝐶𝑋 = [102 0.75 × 10 × 12. 122 ] meters-squared. Equation/Algorithm 3.2.1-1 was applied twice: once for

CE_50 and once for CE_95.

Figure 3.2.1-1 CE_50 circle (red), CE_95 circle (black), and 10,000 of 1,000,000 random samples

The results are plotted in Figure 3.2.1-1, including the first 10,000 of the 1,000,000 independent samples used in the

calculation of CE_50 for context. (The CE_50 circle in the figure was computed using all 1,000,000 independent

samples. The CE_95 circle was computed similarly, but used a different set of 1,000,000 independent samples for

convenience. Both circles are centered at zero by definition.)

Note that sample results (blue points) are not centered about zero and in a non-symmetric fashion due to a mean-

value with different non-zero components in the x and y directions, as is correct. Also note that the actual statistical

significance is greater than that implied by the figure, which displays only 1/100_th the actual number of samples

used in the calculation of CE_50.

3.2.2 CE Evaluation Examples

Examples are as follows:

(1) Assume a desired probability level of 90%, a mean error of zero, and 𝐶𝑋 = [4 22 3

] meters-squared. Eigenvalues

equal 5.562 and 1.438 meters-squared (per MATLAB pseudo code eig(A), where 𝐴 ≡ 𝐶𝑋).

𝜎𝑒𝑖𝑔_𝑚𝑎𝑥 = 2.36 meters, 𝑟 = .509;

𝑅∗ = 1.74 (via linear interpolation: 0.041

0.051.7371 +

0.009

0.051.7621 = 1.7416);

𝐶𝐸_90 = 𝑅∗𝜎𝑚𝑎𝑥 = 4.11 meters.

7

(2)As above except wi a mean-value �̅�𝑇 = [1 −3] . Thus, since the mean-value is not zero, the Monte-Carlo

Matrix Square Root method (Equation/Algorithm 3.2.1-1) is applicable:

𝐶𝐸_90 = 5.69 meters.

3.3 SE The definition and derivations/computations for SE are similar to that described above for CE, but extended from

two dimensions to three dimensions.

In particular, SE_XX is defined as that spherical radius R such that:

𝑝 =1

(2𝜋)3/2det (𝐶𝑋)1/2 ∭ 𝑒−1/2((𝑋−�̅�)𝑇𝐶𝑋−1(𝑋−�̅�) 𝑑𝑥𝑑𝑦𝑑𝑧, (3.3-1)

integrated over the region √𝑥2 + 𝑦2 + 𝑧2 ≤ 𝑅, and where probability 𝑝 = 𝑋𝑋/100, 3d error 𝜀𝑋𝑇 = [𝜖𝑥 𝜖𝑦 𝜖𝑧]

is defined as 𝑋𝑇 = [𝑥 𝑦 𝑧] for notational convenience, with mean-value 𝜖𝑋̅̅̅̅ defined as �̅� = [�̅� �̅� 𝑧̅], and

3 × 3 error covariance matrix 𝐶𝑋 about the mean. Note that if the mean-value is not zero, the radius 𝑅 is still

relative to the origin [0 0 0]𝑇 , per the standard definition of SE_XX.

Corresponding practical evaluation techniques for SE are analogous to those for CE. In particular, assuming a

mean-value equal to zero and eigenvalue square roots of 𝜎𝑚𝑖𝑛, 𝜎𝑚𝑖𝑑 , 𝜎𝑚𝑎𝑥:

𝑆𝐸_𝑋𝑋 = 𝑅(𝑝, 𝑟1, 𝑟2)𝜎𝑚𝑎𝑥 , where 𝑟1 = 𝜎𝑚𝑖𝑑/𝜎𝑚𝑎𝑥 and 𝑟2 = 𝜎𝑚𝑖𝑛/𝜎𝑚𝑎𝑥. (3.3-5)

The following are pre-computed tables of 𝑅(𝑝 = 0.5, 𝑟1, 𝑟2), 𝑅(𝑝 = 0.9, 𝑟1, 𝑟2), and 𝑅(𝑝 = 0.95, 𝑟1, 𝑟2). All table

entries are presented although each table is symmetric. Tables are interpolated using bilinear interpolation.

Table 3.3-1: Spherical Error (SE) multiplier 𝑅(𝑝 = 0.5, 𝑟1, 𝑟2) versus ratios r1 and r2

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00

0.00 0.6745 0.6763 0.6820 0.6916 0.7059 0.7254 0.7499 0.7779 0.8079 0.8389 0.8704 0.9021 0.9337 0.9651 0.9962 1.0271 1.0577 1.0880 1.1181 1.1479 1.1774

0.05 0.6763 0.6782 0.6838 0.6934 0.7076 0.7271 0.7516 0.7795 0.8094 0.8404 0.8719 0.9035 0.9350 0.9664 0.9975 1.0283 1.0589 1.0891 1.1192 1.1489 1.1784

0.10 0.6820 0.6838 0.6894 0.6989 0.7130 0.7324 0.7567 0.7844 0.8141 0.8449 0.8762 0.9077 0.9390 0.9703 1.0013 1.0320 1.0625 1.0926 1.1225 1.1522 1.1817

0.15 0.6916 0.6934 0.6989 0.7084 0.7223 0.7414 0.7654 0.7927 0.8221 0.8526 0.8836 0.9147 0.9459 0.9768 1.0077 1.0381 1.0684 1.0984 1.1282 1.1578 1.1870

0.20 0.7059 0.7076 0.7130 0.7223 0.7359 0.7546 0.7781 0.8048 0.8336 0.8636 0.8941 0.9248 0.9556 0.9862 1.0167 1.0469 1.0769 1.1067 1.1362 1.1655 1.1947

0.25 0.7254 0.7271 0.7324 0.7414 0.7546 0.7727 0.7952 0.8211 0.8491 0.8783 0.9081 0.9382 0.9684 0.9986 1.0286 1.0584 1.0881 1.1174 1.1466 1.1756 1.2045

0.30 0.7499 0.7516 0.7567 0.7654 0.7781 0.7952 0.8167 0.8414 0.8684 0.8966 0.9256 0.9549 0.9844 1.0140 1.0434 1.0728 1.1019 1.1309 1.1597 1.1883 1.2168

0.35 0.7779 0.7795 0.7844 0.7927 0.8048 0.8211 0.8414 0.8651 0.8909 0.9181 0.9462 0.9748 1.0035 1.0324 1.0612 1.0899 1.1185 1.1470 1.1753 1.2035 1.2315

0.40 0.8079 0.8094 0.8141 0.8221 0.8336 0.8491 0.8684 0.8909 0.9157 0.9420 0.9692 0.9970 1.0251 1.0533 1.0814 1.1096 1.1376 1.1656 1.1934 1.2211 1.2488

0.45 0.8389 0.8404 0.8449 0.8526 0.8636 0.8783 0.8966 0.9181 0.9420 0.9675 0.9939 1.0210 1.0484 1.0760 1.1036 1.1313 1.1588 1.1863 1.2137 1.2409 1.2681

0.50 0.8704 0.8719 0.8762 0.8836 0.8941 0.9081 0.9256 0.9462 0.9692 0.9939 1.0197 1.0462 1.0730 1.1002 1.1273 1.1545 1.1816 1.2086 1.2356 1.2625 1.2893

0.55 0.9021 0.9035 0.9077 0.9147 0.9248 0.9382 0.9549 0.9748 0.9970 1.0210 1.0462 1.0722 1.0985 1.1251 1.1519 1.1788 1.2055 1.2322 1.2589 1.2854 1.3119

0.60 0.9337 0.9350 0.9390 0.9459 0.9556 0.9684 0.9844 1.0035 1.0251 1.0484 1.0730 1.0985 1.1245 1.1508 1.1772 1.2037 1.2302 1.2567 1.2830 1.3093 1.3355

0.65 0.9651 0.9664 0.9703 0.9768 0.9862 0.9986 1.0140 1.0324 1.0533 1.0760 1.1002 1.1251 1.1508 1.1767 1.2029 1.2291 1.2554 1.2817 1.3078 1.3339 1.3599

0.70 0.9962 0.9975 1.0013 1.0077 1.0167 1.0286 1.0434 1.0612 1.0814 1.1036 1.1273 1.1519 1.1772 1.2029 1.2288 1.2549 1.2810 1.3070 1.3330 1.3590 1.3848

0.75 1.0271 1.0283 1.0320 1.0381 1.0469 1.0584 1.0728 1.0899 1.1096 1.1313 1.1545 1.1788 1.2037 1.2291 1.2549 1.2807 1.3067 1.3325 1.3585 1.3843 1.4101

0.80 1.0577 1.0589 1.0625 1.0684 1.0769 1.0881 1.1019 1.1185 1.1376 1.1588 1.1816 1.2055 1.2302 1.2554 1.2810 1.3067 1.3324 1.3582 1.3840 1.4098 1.4355

0.85 1.0880 1.0891 1.0926 1.0984 1.1067 1.1174 1.1309 1.1470 1.1656 1.1863 1.2086 1.2322 1.2567 1.2817 1.3070 1.3325 1.3582 1.3840 1.4098 1.4356 1.4611

0.90 1.1181 1.1192 1.1225 1.1282 1.1362 1.1466 1.1597 1.1753 1.1934 1.2137 1.2356 1.2589 1.2830 1.3078 1.3330 1.3585 1.3840 1.4098 1.4355 1.4612 1.4869

0.95 1.1479 1.1489 1.1522 1.1578 1.1655 1.1756 1.1883 1.2035 1.2211 1.2409 1.2625 1.2854 1.3093 1.3339 1.3590 1.3843 1.4098 1.4356 1.4612 1.4869 1.5125

1.00 1.1774 1.1784 1.1817 1.1870 1.1947 1.2045 1.2168 1.2315 1.2488 1.2681 1.2893 1.3119 1.3355 1.3599 1.3848 1.4101 1.4355 1.4611 1.4869 1.5125 1.5382

r1

r2

8

Table 3.3-2: Spherical Error (SE) multiplier 𝑅(𝑝 = 0.9, 𝑟1, 𝑟2) versus ratios r1 and r2

Table 3.3-3: Spherical Error (SE) multiplier 𝑅(𝑝 = 0.95, 𝑟1, 𝑟2) versus ratios r1 and r2

SE_XX is computed as 𝑆𝐸_𝑋𝑋 = 𝑅∗𝜎𝑚𝑎𝑥, where the normalized radius 𝑅∗ is computed as the bi-linear

interpolation of 𝑅(𝑋𝑋/100, 𝑟1, 𝑟2) from the corresponding row and column of the appropriate table. Computation

time is virtually instantaneous. Its accuracy relative to the true value of SE_XX is typically on the order of 0.02%

(experimental max 0.15%)

If the mean-value for error is not equal to zero, one can solve Equation 3.3-1 directly using iteration and numerical

integration. However, this solution may have convergence problems, so the Monte-Carlo method of subsection

3.3.1 is recommended instead. Its evaluation via MATLAB pseudo-code takes on the order of 0.08 seconds on a

high-end notebook computer, and its accuracy relative to the true value of SE_XX is typically on the order of 0.2 %

(experimental max of 0.4%).

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00

0.00 1.6449 1.6456 1.6479 1.6518 1.6573 1.6646 1.6738 1.6852 1.6992 1.7163 1.7371 1.7621 1.7915 1.8251 1.8625 1.9034 1.9472 1.9936 2.0424 2.0932 2.1460

0.05 1.6456 1.6464 1.6487 1.6525 1.6581 1.6654 1.6745 1.6860 1.6999 1.7170 1.7378 1.7628 1.7922 1.8258 1.8632 1.9040 1.9478 1.9942 2.0429 2.0938 2.1466

0.10 1.6479 1.6487 1.6509 1.6548 1.6604 1.6676 1.6769 1.6882 1.7021 1.7192 1.7400 1.7650 1.7944 1.8279 1.8652 1.9060 1.9497 1.9961 2.0448 2.0956 2.1483

0.15 1.6518 1.6525 1.6548 1.6587 1.6642 1.6714 1.6806 1.6920 1.7059 1.7229 1.7436 1.7686 1.7979 1.8314 1.8687 1.9094 1.9530 1.9993 2.0479 2.0987 2.1512

0.20 1.6573 1.6581 1.6604 1.6642 1.6697 1.6769 1.6861 1.6974 1.7113 1.7282 1.7489 1.7738 1.8030 1.8364 1.8735 1.9141 1.9576 2.0039 2.0523 2.1029 2.1555

0.25 1.6646 1.6654 1.6676 1.6714 1.6769 1.6841 1.6932 1.7045 1.7183 1.7352 1.7558 1.7806 1.8097 1.8429 1.8799 1.9204 1.9638 2.0098 2.0581 2.1086 2.1610

0.30 1.6738 1.6745 1.6769 1.6806 1.6861 1.6932 1.7023 1.7135 1.7273 1.7441 1.7646 1.7892 1.8182 1.8513 1.8881 1.9283 1.9715 2.0173 2.0654 2.1156 2.1678

0.35 1.6852 1.6860 1.6882 1.6920 1.6974 1.7045 1.7135 1.7247 1.7383 1.7550 1.7755 1.7999 1.8286 1.8614 1.8981 1.9380 1.9809 2.0265 2.0743 2.1243 2.1762

0.40 1.6992 1.6999 1.7021 1.7059 1.7113 1.7183 1.7273 1.7383 1.7519 1.7685 1.7887 1.8130 1.8414 1.8740 1.9102 1.9498 1.9923 2.0375 2.0850 2.1347 2.1862

0.45 1.7163 1.7170 1.7192 1.7229 1.7282 1.7352 1.7441 1.7550 1.7685 1.7849 1.8049 1.8289 1.8569 1.8890 1.9248 1.9639 2.0060 2.0506 2.0977 2.1469 2.1981

0.50 1.7371 1.7378 1.7400 1.7436 1.7489 1.7558 1.7646 1.7755 1.7887 1.8049 1.8245 1.8481 1.8757 1.9071 1.9422 1.9807 2.0221 2.0663 2.1127 2.1614 2.2120

0.55 1.7621 1.7628 1.7650 1.7686 1.7738 1.7806 1.7892 1.7999 1.8130 1.8289 1.8481 1.8710 1.8979 1.9287 1.9630 2.0007 2.0413 2.0847 2.1304 2.1783 2.2282

0.60 1.7915 1.7922 1.7944 1.7979 1.8030 1.8097 1.8182 1.8286 1.8414 1.8569 1.8757 1.8979 1.9240 1.9539 1.9873 2.0240 2.0637 2.1061 2.1510 2.1980 2.2472

0.65 1.8251 1.8258 1.8279 1.8314 1.8364 1.8429 1.8513 1.8614 1.8740 1.8890 1.9071 1.9287 1.9539 1.9827 2.0151 2.0507 2.0894 2.1308 2.1746 2.2207 2.2689

0.70 1.8625 1.8632 1.8652 1.8687 1.8735 1.8799 1.8881 1.8981 1.9102 1.9248 1.9422 1.9630 1.9873 2.0151 2.0464 2.0809 2.1185 2.1587 2.2015 2.2464 2.2936

0.75 1.9034 1.9040 1.9060 1.9094 1.9141 1.9204 1.9283 1.9380 1.9498 1.9639 1.9807 2.0007 2.0240 2.0507 2.0809 2.1143 2.1506 2.1898 2.2314 2.2753 2.3214

0.80 1.9472 1.9478 1.9497 1.9530 1.9576 1.9638 1.9715 1.9809 1.9923 2.0060 2.0221 2.0413 2.0637 2.0894 2.1185 2.1506 2.1858 2.2237 2.2642 2.3070 2.3520

0.85 1.9936 1.9942 1.9961 1.9993 2.0039 2.0098 2.0173 2.0265 2.0375 2.0506 2.0663 2.0847 2.1061 2.1308 2.1587 2.1898 2.2237 2.2605 2.2998 2.3415 2.3854

0.90 2.0424 2.0429 2.0448 2.0479 2.0523 2.0581 2.0654 2.0743 2.0850 2.0977 2.1127 2.1304 2.1510 2.1746 2.2015 2.2314 2.2642 2.2998 2.3380 2.3786 2.4213

0.95 2.0932 2.0938 2.0956 2.0987 2.1029 2.1086 2.1156 2.1243 2.1347 2.1469 2.1614 2.1783 2.1980 2.2207 2.2464 2.2753 2.3070 2.3415 2.3786 2.4180 2.4597

1.00 2.1460 2.1466 2.1483 2.1512 2.1555 2.1610 2.1678 2.1762 2.1862 2.1981 2.2120 2.2282 2.2472 2.2689 2.2936 2.3214 2.3520 2.3854 2.4213 2.4597 2.5003

r1

r2

r1

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00

r 2

0.00 1.9600 1.9606 1.9625 1.9658 1.9704 1.9765 1.9842 1.9937 2.0051 2.0190 2.0359 2.0564 2.0813 2.1111 2.1460 2.1858 2.2303 2.2791 2.3318 2.3881 2.4478

0.05 1.9606 1.9612 1.9632 1.9664 1.9711 1.9771 1.9848 1.9943 2.0058 2.0197 2.0365 2.0570 2.0819 2.1117 2.1466 2.1864 2.2309 2.2796 2.3324 2.3887 2.4482

0.10 1.9625 1.9632 1.9651 1.9683 1.9729 1.9791 1.9867 1.9962 2.0077 2.0215 2.0383 2.0589 2.0837 2.1135 2.1483 2.1881 2.2325 2.2813 2.3339 2.3902 2.4498

0.15 1.9658 1.9664 1.9683 1.9716 1.9762 1.9823 1.9899 1.9994 2.0108 2.0247 2.0415 2.0620 2.0868 2.1165 2.1513 2.1910 2.2354 2.2841 2.3367 2.3929 2.4524

0.20 1.9704 1.9711 1.9729 1.9762 1.9808 1.9868 1.9945 2.0039 2.0153 2.0292 2.0459 2.0664 2.0912 2.1208 2.1555 2.1952 2.2394 2.2880 2.3406 2.3967 2.4561

0.25 1.9765 1.9771 1.9791 1.9823 1.9868 1.9929 2.0005 2.0099 2.0213 2.0351 2.0518 2.0722 2.0969 2.1265 2.1611 2.2006 2.2448 2.2932 2.3457 2.4016 2.4609

0.30 1.9842 1.9848 1.9867 1.9899 1.9945 2.0005 2.0081 2.0175 2.0288 2.0425 2.0592 2.0795 2.1041 2.1336 2.1682 2.2075 2.2515 2.2998 2.3520 2.4078 2.4669

0.35 1.9937 1.9943 1.9962 1.9994 2.0039 2.0099 2.0175 2.0268 2.0381 2.0518 2.0683 2.0885 2.1131 2.1425 2.1767 2.2160 2.2598 2.3079 2.3598 2.4154 2.4743

0.40 2.0051 2.0058 2.0077 2.0108 2.0153 2.0213 2.0288 2.0381 2.0493 2.0630 2.0795 2.0995 2.1239 2.1531 2.1873 2.2262 2.2697 2.3175 2.3692 2.4246 2.4831

0.45 2.0190 2.0197 2.0215 2.0247 2.0292 2.0351 2.0425 2.0518 2.0630 2.0764 2.0929 2.1129 2.1371 2.1660 2.1999 2.2385 2.2816 2.3291 2.3804 2.4353 2.4935

0.50 2.0359 2.0365 2.0383 2.0415 2.0459 2.0518 2.0592 2.0683 2.0795 2.0929 2.1092 2.1290 2.1529 2.1816 2.2150 2.2532 2.2959 2.3429 2.3936 2.4481 2.5058

0.55 2.0564 2.0570 2.0589 2.0620 2.0664 2.0722 2.0795 2.0885 2.0995 2.1129 2.1290 2.1485 2.1722 2.2004 2.2333 2.2708 2.3129 2.3592 2.4093 2.4631 2.5202

0.60 2.0813 2.0819 2.0837 2.0868 2.0912 2.0969 2.1041 2.1131 2.1239 2.1371 2.1529 2.1722 2.1953 2.2229 2.2551 2.2919 2.3332 2.3786 2.4279 2.4809 2.5371

0.65 2.1111 2.1117 2.1135 2.1165 2.1208 2.1265 2.1336 2.1425 2.1531 2.1660 2.1816 2.2004 2.2229 2.2497 2.2810 2.3168 2.3570 2.4014 2.4497 2.5017 2.5570

0.70 2.1460 2.1466 2.1483 2.1513 2.1555 2.1611 2.1682 2.1767 2.1873 2.1999 2.2150 2.2333 2.2551 2.2810 2.3112 2.3460 2.3850 2.4281 2.4752 2.5259 2.5801

0.75 2.1858 2.1864 2.1881 2.1910 2.1952 2.2006 2.2075 2.2160 2.2262 2.2385 2.2532 2.2708 2.2919 2.3168 2.3460 2.3794 2.4170 2.4589 2.5046 2.5539 2.6067

0.80 2.2303 2.2309 2.2325 2.2354 2.2394 2.2448 2.2515 2.2598 2.2697 2.2816 2.2959 2.3129 2.3332 2.3570 2.3850 2.4170 2.4533 2.4937 2.5379 2.5858 2.6371

0.85 2.2791 2.2796 2.2813 2.2841 2.2880 2.2932 2.2998 2.3079 2.3175 2.3291 2.3429 2.3592 2.3786 2.4014 2.4281 2.4589 2.4937 2.5325 2.5751 2.6214 2.6713

0.90 2.3318 2.3323 2.3339 2.3367 2.3406 2.3457 2.3520 2.3598 2.3692 2.3804 2.3936 2.4093 2.4279 2.4497 2.4752 2.5046 2.5379 2.5751 2.6162 2.6609 2.7091

0.95 2.3881 2.3887 2.3902 2.3929 2.3967 2.4016 2.4078 2.4154 2.4246 2.4353 2.4481 2.4631 2.4809 2.5017 2.5259 2.5539 2.5858 2.6214 2.6609 2.7040 2.7506

1.00 2.4478 2.4482 2.4498 2.4524 2.4561 2.4609 2.4669 2.4743 2.4831 2.4935 2.5058 2.5202 2.5371 2.5570 2.5801 2.6067 2.6371 2.6713 2.7091 2.7506 2.7955

9

3.3.1 Monte-Carlo Method for Computation of SE

The algorithm/equation for computation of SE based on the Monte-Carlo method is basically the same as

Equation/Algorithm 3.2.1-1 for CE, except that �̅� and 𝐶𝑋 are the 3x1 mean-value and 3x3 error covariance matrix,

respectively, and 𝑛𝑖 is a three-element vector with each element the realization of an independent 𝑁(0,1) random

variable.

3.3.2 SE Evaluation Examples

Examples are as follows:

(1) Assume a desired probability level of 90%, a mean error of zero, and 𝐶𝑋 = [4 −5.4 6

−5.4 9 −96 −9 25

] meters-squared.

Eigenvalues equal 31.2, 6.22, and 0.55 meters-squared.

𝜎𝑒𝑖𝑔_𝑚𝑎𝑥 = 5.59 meters, 𝑟1 = 0.446, 𝑟2 = 0.132;

𝑅∗ = 1.72 ( via bilinear linear interpolation: 0.018

0.05(

0.004

0.051.7021 +

0.046

0.051.7192) +

0.032

0.05(

0.004

0.051.7059 +

0.046

0.051.7229) = 1.7202 );

𝑆𝐸_90 = 𝑅∗𝜎𝑒𝑖𝑔_𝑚𝑎𝑥 = 9.61 meters.

(2) As above except with a mean-value �̅�𝑇 = [1 0 −1]. Thus, since the mean-value is not zero, the Monte-Carlo

Matrix Square Root method (Equation 3.3.2-1) is applicable:

𝑆𝐸_90 = 9.76 meters.

4.0 Scalar Accuracy Metrics: Computations based on Sample Statistics Order statistics are recommended for the computation of sample-based scalar accuracy metrics. Order statistics are

not only used to compute the scalar accuracy metric values, but confidence intervals around those values as well

since confidence in the values is highly dependent on the number of samples used.

Subsection 4.1 describes the relevant portions of order statistics for this application. Subsection 4.2 then specifically

addresses the computation of scalar accuracy metrics based on the material of subsection 4.1. It equates a radial

error random variable 𝜖𝑟 to the general random variable 𝑥 of subsection 4.1. The radial error random variable

corresponds to different combinations of geolocation error for each scalar accuracy metric: (1) 𝜖𝑟 = |𝜖𝑧| if LE,

𝜖𝑟 = √𝜖𝑥2 + 𝜖𝑦2 if CE, and 𝜖𝑟 = √𝜖𝑥2 + 𝜖𝑦2 + 𝜖𝑧2 if SE.

Order statistics require no assumptions regarding the probability distribution of the corresponding random variable

𝑥, or the probability distributions of the underlying errors 𝜖𝑥, 𝜖𝑦 , and 𝜖𝑧. That is, whether they are Gaussian

distributed or not, have a non-zero mean-value or not, etc.

4.1 Order Statistics The 𝑝 percentile of a random variable 𝑥 is defined as the smallest number 𝑥𝑝 such that 𝑝 = 𝑝𝑟𝑜𝑏{𝑥 ≤ 𝑥𝑝}. Thus,

the probability distribution function (typically unknown) of the random variable 𝑥 evaluated at 𝑥𝑝 is equal to 𝑝 [8,

pg. 69]. 𝑥𝑝 is a deterministic parameter with typically unknown value.

We are interested in estimating 𝑥𝑝 given a set of independent and identically distributed (i.i.d.) samples of the

random variable 𝑥 using order statistics. This approach to estimating 𝑥𝑝 is sometimes termed the “percentile

method”.

Assume that 𝑛 i.i.d. samples 𝑥𝑖 of the random variable 𝑥 are ordered (increasingly) as 𝑦𝑖 (see Figure 4.1-1 below).

Samples can be signed, although for our application of interest (subsection 4.2), they are strictly positive.

10

Figure 4-1: Samples versus corresponding ordered samples

The collection of 𝑦𝑖 are the “order statistics” of the random variable 𝑥 [8, pg. 254]. In general, they too are

considered random variables, but when assigned specific values corresponding to the samples 𝑥𝑖, they are

considered samples as well. The appropriate interpretation of 𝑦𝑖 should be clear from context.

At first glance, it seems obvious what the best estimate of 𝑥𝑝 should be given the collection of 𝑦𝑖 . For example, if

we had 𝑛 = 200 i.i.d. samples and the 𝑝 percentile 𝑥0.90 was of interest, we would expect the best estimate of 𝑥0.90

to be 𝑦180, give or take an ordered sample or two, since 𝑝 ∙ 𝑛 = 0.90 ∙ 200 = 180. However, we need to be more

precise. Also, this does not address the considerable effects of small 𝑛 (all too typical), the computation of

corresponding confidence intervals, and underlying theory – all described in the following subsections.

4.1.1 Fundamental Equations

We now present various equations [8, pg. 254] for the probability that 𝑥𝑝 is contained in various intervals of the

form (𝑦𝑘 < 𝑥𝑝 < 𝑦𝑘+𝑟), where 𝑦𝑘 and 𝑦𝑘+𝑟 are considered random variables. The equations are based on the

binomial distribution, and if 𝑛 is large, by approximation of the binomial distribution by the Gaussian distribution

[8, pg. 50]. The formulas are exact (other than the Gaussian approximation if large 𝑛) – somewhat amazing when

one considers that there is no a priori information of the random variable 𝑥 and its probability distribution function.

The equations presented below also assume a continuous-type random variable 𝑥, i.e., a random variable with a

continuous probability distribution function.

If (𝑛 ≈≤ 50) we can use direct binomial evaluation to obtain:

𝑝𝑟𝑜𝑏{𝑦𝑘 < 𝑥𝑝 < 𝑦𝑘+𝑟} = ∑ (𝑛𝑚

) 𝑝𝑚(1 − 𝑝)𝑛−𝑚𝑘+𝑟−1𝑚=𝑘 ; (4.1.1-1)

else, use Gaussian approximation for large 𝑛 (≈≥ 50):

𝑝𝑟𝑜𝑏{𝑦𝑘 < 𝑥𝑝 < 𝑦𝑘+𝑟} ≅ 𝐺 (𝑘+𝑟−0.5−𝑛𝑝

√𝑛𝑝(1−𝑝)) − 𝐺 (

𝑘−0.5−𝑛𝑝

√𝑛𝑝(𝑛−𝑝)) = (4.1.1-2)

1

2𝑚𝑎𝑡𝑙𝑎𝑏 erf (

1

√2

𝑘+𝑟−0.5−𝑛𝑝

√𝑛𝑝(1−𝑝)) −

1

2𝑚𝑎𝑡𝑙𝑎𝑏 erf (

1

√2

𝑘−0.5−𝑛𝑝

√𝑛𝑝(1−𝑝)) , in terms of MATLAB pseudo-code.

Note that for Equation 4.1.1-1, in general, 𝑘 and 𝑘 + 𝑟 are integers between the values 1 and 𝑛. However, the case

𝑘 = 0 and 𝑟 = 1 is defined as corresponding to 𝑝𝑟𝑜𝑏{𝑥𝑝 < 𝑦1} and the case 𝑘 = 𝑛 and 𝑟 = 1 is defined as

corresponding to 𝑝𝑟𝑜𝑏{𝑦𝑛 < 𝑥𝑝}.

Note that 𝐺(𝑥) =1

√2𝜋∫ 𝑒−𝑦2/2𝑑𝑦

𝑥

−∞ is the standard normal cumulative distribution function.

. . . −

5.1 6.2 1.5 3.3 . . . 7.2 2.2

. . . −

1.1 1.5 3.3 4.0 . . . 6.9 7.2

samples :

ordered samples :

11

(Caution: “𝑝𝑟𝑜𝑏” in the above equations corresponds to the probability that 𝑥𝑝, the p-percentile of the random

variable 𝑥, is within the specified interval , and does not correspond to the value of “𝑝” associated with the p-

percentile itself.)

Also, as demonstrated in [8, pg. 46], relative to all single intervals of the form (𝑦𝑘 , 𝑦𝑘+1), the single interval

(𝑦𝑘′ , 𝑦𝑘′+1) contains the maximum probability that the 𝑝 percentile 𝑥𝑝 resides within, where:

𝑘′ = 𝑓𝑙𝑜𝑜𝑟[ (𝑛 + 1)𝑝 ], i.e., the integer part of (𝑛 + 1)𝑝. (4.1.1-3)

An exception occurs when (𝑛 + 1)𝑝 equals and integer, in which case the adjacent interval (𝑦𝑘′−1, 𝑦𝑘′) also

contains the maximum probability.

4.1.2 Best Estimate of Percentile and Two-Sided Confidence Intervals

The best (maximum likelihood) estimate of the percentile 𝑥𝑝 corresponds to the mid-point of the interval that

contains the maximum probability that 𝑥𝑝 resides within. It is designated 𝑦𝑘 𝑚𝑙 with corresponding ordered sample

location designated 𝑘 𝑚𝑙. Based on Equation 4.1.3-1, it is equal to:

𝑦𝑘 𝑚𝑙 ≡ 0.5(𝑦𝑘′ + 𝑦𝑘′+1), 𝑘 𝑚𝑙 ≡ 𝑘′ + 0.5., if (𝑛 + 1)𝑝 does not equal an integer; else (4.1.2-1)

𝑦𝑘 𝑚𝑙 ≡ 𝑦𝑘′, 𝑘 𝑚𝑙 ≡ 𝑘′.

The following general algorithm computes the two-sided confidence interval (𝑐𝑜𝑛𝑓 = 𝛾) for the percentile 𝑥𝑝:

Find the integer 𝑘1 with the smallest possible corresponding integer 𝑘2 that satisfies: (4.1.2-3)

𝑝𝑟𝑜𝑏{𝑦𝑘1 < 𝑥𝑝 < 𝑦𝑘2} ≥ 𝛾, where 0 < 𝛾 < 1.

The order sample indices 𝑘1 and 𝑘2 represent the two-sided confidence interval (𝑦𝑘1, 𝑦𝑘2).

For large 𝑛 [8, pg. 255]: (4.1.2-4)

𝑘1 = 𝑟𝑜𝑢𝑛𝑑_𝑡𝑜_𝑛𝑒𝑎𝑟𝑒𝑠𝑡_𝑖𝑛𝑡𝑒𝑔𝑒𝑟[𝑛𝑝 − (𝑧0.5+𝛾/2)√𝑛𝑝(1 − 𝑝) ],

𝑘2 = 𝑟𝑜𝑢𝑛𝑑_𝑡𝑜_𝑛𝑒𝑎𝑟𝑒𝑠𝑡_𝑖𝑛𝑡𝑒𝑔𝑒𝑟[𝑛𝑝 + (𝑧0.5+𝛾/2)√𝑛𝑝(1 − 𝑝) ],

where by definition: erf (𝑧0.5+𝛾

2) = 𝛾/2, and similarly in terms of MATLAB pseudo-code

1

2𝑚𝑎𝑡𝑙𝑎𝑏 erf (

𝑧0.5+

𝛾2

√2) =

𝛾/2. Therefore 𝑧0.5+𝛾

2= √2𝑚𝑎𝑡𝑙𝑎𝑏 𝑒𝑟𝑓𝑖𝑛𝑣(𝛾) can be used in Equation 4.1.2-4 above. Note that the 𝑒𝑟𝑓 function

corresponds to the standard normal (Gaussian) distribution.

Furthermore, for large 𝑛, the best estimate of 𝑥𝑝 is set to 𝑦𝑘 𝑚𝑙 , where:

𝑘 𝑚𝑙 = 𝑘1 + (𝑘2 − 𝑘1)/2. (4.1.2-5)

4.1.3 Corresponding Tables

The appropriate equations in subsection 4.1.2 were evaluated as a function of the number of samples 𝑛 with results

presented in Table 4.1.3-1 through Table 4.1.3-3, assuming a desired percentile value of 𝑝 = 0.50, 𝑝 = 0.90, and

𝑝 = 0.95, respectively. Each table includes the order sample index 𝑘 𝑚𝑙 corresponding to the best (maximum

likelihood) estimate of the 𝑥-percentile 𝑥𝑝, and three two-sided confidence intervals corresponding to confidence

values 𝛾 = 0.50, 𝛾 = 0.90, and 𝛾 = 0.95. Each of these intervals is designated by corresponding order sample

indices 𝑘1 and 𝑘2.

In general, the larger 𝑛 the better the estimate 𝑦 𝑚𝑙 and the smaller the relative size of the confidence intervals.

Note that for confidence intervals, table entries are only populated for values of 𝑛 such that the confidence interval is

contained within the spanning interval (1, 𝑛). Also, for large 𝑛 with no corresponding table entries, evaluate

Equation 4.1.2-5 for 𝑘 𝑚𝑙 and Equation 4.1.2-4 for (𝑘1, 𝑘2).

12

As a general example of how to interpret the tables: Table 4.1.3-2 corresponds to 𝑥0.90 (percentile value 0.90). For

𝑛 = 35, 𝑘 𝑚𝑙 = 32.5, and for confidence 𝛾 = 0.50 , (𝑘1, 𝑘2) = (31,34). Thus, the best estimate of 𝑥0.90 equals

𝑦𝑘 𝑚𝑙 = 𝑦32.5 = 0.5(𝑦32 + 𝑦33), and there is a 50% confidence (probability) that the true value of 𝑥0.90 resides

within the interval (𝑦31, 𝑦34).

Table 4.1.3-1:Order statistics 𝒑 = 𝟎. 𝟓𝟎 best estimate and two-sided confidence intervals

n k ml k1 k2 k1 k2 k1 k2 n k ml k1 k2 k1 k2 k1 k2

5 3 2 4 1 5 36 18.5 16 21 13 23 12 24

6 3.5 2 4 1 6 1 6 37 19 17 22 14 24 13 25

7 4 3 5 2 7 1 7 38 19.5 17 22 14 25 13 26

8 4.5 3 6 2 7 1 7 39 20 18 23 15 26 14 27

9 5 4 7 2 8 2 8 40 20.5 18 23 15 26 14 27

10 5.5 4 7 2 8 2 9 41 21 19 24 16 27 15 28

11 6 5 8 3 9 3 10 42 21.5 19 24 16 27 15 28

12 6.5 5 8 3 9 3 10 43 22 20 25 17 28 16 29

13 7 6 9 4 10 3 11 44 22.5 20 25 17 28 16 29

14 7.5 6 9 4 11 3 11 45 23 21 26 17 29 16 30

15 8 7 10 5 12 4 12 46 23.5 21 26 17 29 16 30

16 8.5 7 10 5 12 4 12 47 24 22 27 18 30 17 31

17 9 8 11 6 13 5 13 48 24.5 22 27 18 30 17 31

18 9.5 8 11 6 13 5 14 49 25 23 28 19 31 18 32

19 10 8 12 6 14 6 15 50 25 23 27 19 31 18 32

20 10.5 8 12 6 14 6 15 51 25.5 23 28 20 31 19 32

21 11 9 13 7 15 6 16 52 26 24 28 20 32 19 33

22 11.5 9 13 7 15 6 16 53 26.5 24 29 21 32 19 34

23 12 10 14 8 16 7 17 54 27 25 29 21 33 20 34

24 12.5 10 14 8 17 7 17 55 27.5 25 30 21 34 20 35

25 13 11 15 9 18 8 18 56 28 25 31 22 34 21 35

26 13.5 11 15 9 18 8 19 57 28.5 26 31 22 35 21 36

27 14 12 16 10 19 9 20 58 29 26 32 23 35 22 36

28 14.5 12 16 10 19 9 20 59 29.5 27 32 23 36 22 37

29 15 13 17 11 20 10 21 60 30 27 33 24 36 22 38

30 15.5 13 17 11 20 10 21 75 37.5 35 40 30 45 29 46

31 16 14 18 11 21 10 22 100 50 47 53 42 58 40 60

32 16.5 14 18 11 21 10 22 125 62.5 59 66 53 72 52 73

33 17 15 19 12 22 11 23 150 75 71 79 65 85 63 87

34 17.5 15 19 12 22 11 23 175 87.5 83 92 77 98 75 100

35 18 16 20 13 23 12 24 200 100 95 105 88 112 86 114

p= 0.50

0.50 conf 0.90 conf 0.95 conf 0.50 conf 0.90 conf 0.95 conf

13

Table 4.1.3-2: Order statistics 𝒑 = 𝟎. 𝟗𝟎 best estimate and two-sided confidence intervals

Table 4.1.3-3: Order statistics 𝒑 = 𝟎. 𝟗𝟓 best estimate and two-sided confidence intervals

n k ml k1 k2 k1 k2 k1 k2 n k ml k1 k2 k1 k2 k1 k2

5 36 33.5 32 35 30 36 29 36

6 37 34.5 33 36 31 37 30 37

7 7 4 7 38 35.5 34 37 32 38 30 38

8 8 6 8 39 36 35 38 32 39 31 39

9 9 7 9 40 36.5 35 38 33 40 32 40

10 9.5 8 10 41 37.5 36 39 34 41 33 41

11 10.5 9 11 42 38.5 37 40 35 42 34 42

12 11.5 10 12 43 39.5 38 41 36 43 35 43

13 12.5 11 13 44 40.5 39 42 37 44 36 44

14 13.5 12 14 45 41.5 40 43 38 45 37 45

15 14.5 13 15 46 42.5 41 44 39 46 38 46

16 15.5 14 16 47 43.5 42 45 40 47 39 47

17 16.5 15 17 48 44.5 43 46 40 48 39 48

18 17.5 16 18 49 45 44 47 41 49 40 49

19 18 17 19 50 45 44 46 42 48 41 49

20 18.5 17 20 51 45.5 44 47 42 49 42 50

21 19.5 18 21 52 46.5 45 48 43 50 43 51

22 20.5 19 22 15 22 53 47.5 46 49 44 51 43 52

23 21.5 20 23 17 23 54 48.5 47 50 45 52 44 53

24 22.5 21 24 18 24 55 49.5 48 51 46 53 45 54

25 23.5 22 25 19 25 56 50.5 49 52 47 54 46 55

26 24.5 23 26 20 26 57 51.5 50 53 48 55 47 56

27 25.5 24 27 21 27 58 52.5 51 54 48 56 48 57

28 26.5 25 28 22 28 59 53.5 52 55 49 57 49 58

29 27 26 29 23 29 21 29 60 54 52 56 50 58 49 59

30 27.5 26 29 24 30 22 30 75 67.5 66 69 63 72 62 73

31 28.5 27 30 25 31 24 31 100 90 88 92 85 95 84 96

32 29.5 28 31 26 32 25 32 125 112.5 110 115 107 118 106 119

33 30.5 29 32 27 33 26 33 150 135 133 137 129 141 128 142

34 31.5 30 33 28 34 27 34 175 157.5 155 160 151 164 150 165

35 32.5 31 34 29 35 28 35 200 180 177 183 173 187 172 188

0.95 conf

p= 0.90

0.50 conf 0.90 conf 0.95 conf 0.50 conf 0.90 conf

n k ml k1 k2 k1 k2 k1 k2 n k ml k1 k2 k1 k2 k1 k2

5 36 35.5 34 36

6 37 36.5 35 37

7 38 37.5 36 38

8 39 38 37 39

9 40 38.5 37 40

10 41 39.5 38 41

11 42 40.5 39 42

12 43 41.5 40 43

13 44 42.5 41 44

14 14 11 14 45 43.5 42 45 37 45

15 15 13 15 46 44.5 43 46 39 46

16 16 14 16 47 45.5 44 47 41 47

17 17 15 17 48 46.5 45 48 42 48

18 18 16 18 49 47.5 46 49 43 49

19 19 17 19 50 47.5 46 49 45 50 44 51

20 19.5 18 20 51 48 47 49 46 51 45 52

21 20.5 19 21 52 49 48 50 47 52 46 52

22 21.5 20 22 53 50 49 51 48 53 47 53

23 22.5 21 23 54 51 50 52 49 54 48 54

24 23.5 22 24 55 52 51 53 50 55 49 55

25 24.5 23 25 56 53 52 54 51 56 50 56

26 25.5 24 26 57 54 53 55 51 57 51 57

27 26.5 25 27 58 55 54 56 52 58 52 58

28 27.5 26 28 59 56 55 57 53 59 53 59

29 28.5 27 29 60 57 56 58 54 60 54 60

30 29.5 28 30 75 71.5 70 73 68 74 68 75

31 30.5 29 31 100 95 94 96 91 99 91 99

32 31.5 30 32 125 118.5 117 120 115 123 114 124

33 32.5 31 33 150 142.5 141 144 138 147 137 148

34 33.5 32 34 175 166 164 168 162 171 161 172

35 34.5 33 35 200 190 188 192 185 195 184 196

p= 0.95

0.50 conf 0.90 conf 0.95 conf 0.50 conf 0.90 conf 0.95 conf

14

4.2 Scalar Accuracy Metric Computations Order statistics, as described in Section 4.1, are a “natural” for the computation of the sample-based scalar accuracy

metrics LE, CE, and SE. For a specified probability level XX (%), e.g., CE_50, XX/100 corresponds to the value of

𝑝 percentile. Application of order statistics not only provides a best estimate of the desired scalar accuracy metric,

but a confidence interval around it as well. The following presents the appropriate procedure: (4.2-1)

Define 3d errors as a multi-variate random variable 𝜖𝑋 = [𝜖𝑥 𝜖𝑦 𝜖𝑧]𝑇relative to “ground truth”,

assumed to be in a local tangent plane coordinate system, e.g., East-North-Up. We then equate the

appropriate radial error of interest to the general random variable 𝑥 of subsection 4.1. If we are interested

in LE, define the random variable 𝑥 ≡ |𝜖𝑧|; if in CE, 𝑥 ≡ √𝜖𝑥2 + 𝜖𝑦2; if in SE, 𝑥 ≡ √𝜖𝑥2 + 𝜖𝑦2 + 𝜖𝑧2

Given 3d samples 𝜖𝑋𝑖 = [𝜖𝑥𝑖 𝜖𝑦𝑖 𝜖𝑧𝑖]𝑇, 𝑖 = 1, . . , 𝑛, generate appropriate scalar samples 𝑥𝑖 per the

above definitions, and then generate corresponding ordered samples 𝑦𝑖 .

o For example, if CE is of interest, 𝑥1 = √𝜖𝑥12 + 𝜖𝑦1

2, 𝑥2 = √𝜖𝑥22 + 𝜖𝑦2

2, ... , 𝑥𝑛 = √𝜖𝑥𝑛2 + 𝜖𝑦𝑛

2; 𝑦𝑖 ,

𝑖 = 1, . . , 𝑛, their ordered counterparts.

Specify the probability level XX (XX=50, 90, or 95%) for the desired scalar accuracy metric LE, CE, or

SE, and compute the equivalent 𝑝 percentile as 𝑝 = 𝑋𝑋/100. Specify the desired level of confidence 𝛾

(𝛾 = 0.50, 0.90, 𝑜𝑟 0.95) for the two-sided confidence interval.

Utilize the appropriate table of subsection 4.1 based on the value of 𝑝 and the value of 𝛾 (in conjunction

with the ordered samples 𝑦𝑖 , 𝑖 = 1, . . , 𝑛) as detailed in subsection 4.1 and with examples presented below.

This yields the best estimate of the percentile 𝑥𝑝, i.e., the value of either LE_XX, CE_XX, or SE_XX,

𝑝 = 𝑋𝑋/100. This also yields the appropriate confidence interval at confidence level 𝛾.

Examples:

The following examples utilize simulated i.i.d. samples of geolocation error, based on specified (but unknown to the

scalar accuracy metric computation algorithm) probability distributions for these errors. Therefore, the true answer

for the scalar accuracy metric of interest is known as well for comparison. For CE and Gaussian distributed

geolocation errors (example 2), the Monte-Carlo method for the predictive statistic CE (subsection 3.2.1) was used

to compute the true answer. For LE and uniform distributed geolocation errors (example 1), the method of

subsection 3.2.1 was modified in a straight-forward manner to compute LE instead of CE and to use simulated

uniform errors instead of Gaussian errors, in order to compute the true answer. (Direct analysis of the specified

uniform distribution could also have been used instead to compute the true LE.)

(1) LE_90 is of interest; a priori uniform (1,4) probability distribution for the random variable 𝑥 ≡ |𝜖𝑧| (hence, a

mean-value of 2.5 meters) ; 𝑛 = 30 and 0.90 percentile 𝑝; 30 independent samples of 𝑥𝑖 generated via simulation

consistent with the a priori probability distribution, subsequently ordered to yield 𝑦𝑖, 𝑖 = 1: 30:

Because the 0.90 percentile 𝑝 is of interest, we use Table 4.1.3-2 which states that 𝑘 𝑚𝑙 = 27.5 with corresponding

value 𝑦𝑘 𝑚𝑙 = 𝑦27.5 = 0.5(𝑦27 + 𝑦28). Thus, the best estimate of 𝑥0.90 is 𝑦 𝑘 𝑚𝑙 = 0.5(3.65 + 3.80) = 3.73; or

equivalently, the best estimate of LE_90 equals 3.73 meters.

50% confidence interval for LE_90 is (𝑦𝑘1 , 𝑦𝑘2) = (𝑦26, 𝑦29) or (3.52,3.81); 90% confidence interval for LE_90 is

(𝑦𝑘1, 𝑦𝑘2) = (𝑦24, 𝑦30) or (3.20,3.86); 95% confidence interval for LE_90 is (𝑦𝑘1 , 𝑦𝑘2) = (𝑦22, 𝑦30) or (3.04,3.86).

It is known that 𝑥0.90𝑡𝑟𝑢𝑒 = 𝐿𝐸_90_𝑡𝑟𝑢𝑒 = 3.70 meters, since the probability distribution was specified for

simulation of the samples.

(2) CE_50 is of interest; a priori Gaussian probability distribution of 2d random variable 𝜖𝑋 = [𝜖𝑥 𝜖𝑦]𝑇 with

mean-value [1 0]𝑇 and Covariance matrix [4 33 5

]; 𝑛 = 24 and 0.50 percentile 𝑝; 24 independent samples of 𝜖𝑋𝑖

i = : 1 2 .. 22 23 24 25 26 27 28 29 30

x_i : 2.10 1.77 .. 2.46 2.49 1.14 2.02 3.81 2.81 2.48 3.65 1.88

y_i : 1.03 1.14 .. 3.04 3.15 3.20 3.48 3.52 3.65 3.80 3.81 3.86

15

generated via simulation consistent with the a priori probability distribution function; corresponding 24 independent

samples of (horizontal error) 𝑥𝑖 = √𝜖𝑥𝑖2 + 𝜖𝑦𝑖

2 generated and subsequently ordered to yield 𝑦𝑖 , i=1:24:

Because the 0.50 percentile 𝑝 is of interest, we use Table 4.1.3-1 which states that 𝑘 𝑚𝑙 = 12.5 with corresponding

value 𝑦𝑘 𝑚𝑙 = 𝑦12.5 = 0.5(𝑦12 + 𝑦13). Thus, the best estimate of CE_50 equals 0.5(𝑦12 + 𝑦13) = 2.14 meters.

Corresponding two-sided confidence intervals are:

(𝑦𝑘1, 𝑦𝑘2) = (𝑦10, 𝑦14) or (1.86,2.25), for 50% confidence; (𝑦𝑘1, 𝑦𝑘2) = (𝑦8, 𝑦17) or (1.68,2.99), for 90%

confidence; (𝑦𝑘1 , 𝑦𝑘2) = (𝑦7, 𝑦17) or (1.49,2.99), for 95% confidence. Thus, for example, there is a 90%

probability that the true value of CE_50 is within the interval (1.68,2.99) meters.

It is known that 𝑥0.50𝑡𝑟𝑢𝑒 = 𝐶𝐸_50_𝑡𝑟𝑢𝑒 = 2.47 meters, since the probability distribution was specified for

simulation of the samples. Thus, the 50% two-sided confidence interval “fails” in this example, i.e., the true value is

not within; not unexpected since we expect the interval to fail approximately 50% of the time.

Other references

In terms of order statistics, [4] is another good reference; their applications to scalar accuracy metrics are also

discussed in [1-3].

5.0 Summary and Conclusions This paper has presented methodologies for proper but practical computation of LE, CE, and SE based on both

predictive and sample statistics, including corresponding confidence intervals, which address known risks in the

application and use of scalar accuracy metrics. The calculations presented are fast, rigorous and accurate methods to

describe and evaluate data. These methods meet a broad community need and support the contemporary necessity

to be practical and efficient. To effectively utilize scalar accuracy metrics to characterize accuracy and error of

geolocation data, such practices must be broadly adopted by both geospatial data producers and users. This paper is

intended to support the necessary research and documentation that will provide needed references and standardize

the community approach to describing and evaluating the accuracy of geospatial data.

References

[1] Ager, T., “An Analysis of Metric Accuracy Definitions and Methods of Computation, NGA white paper, 2004.

[2] Bresnahan, P.C and Jamison, T.A., “A Monte Carlo Simulation of the Impact of Sample Size and Percentile

Method Implementation on Imagery Geolocation Accuracy Assessments”, ASPRS Annual Conference, Tampa, FL,

7-11 May, 2007.

[3] Bresnahan, P., Brown, E., HenryVazquez, L., “WorldView-3 Absolute Geolocation Accuracy Evaluation”,

JACIE Workshop, 5-7 May, 2015.

[4] Conover, W., Practical Nonparametric Statistics, 3rd

Edition, John Wiley and Sons, Inc., New York, 1999.

[5] Dolloff, J., and Theiss, H., The Specification and Validation of Predictive Accuracy Capabilities for Commercial

Satellite Imagery, Proceedings of ASPRS Annual Conference, 2014.

[6] Greenwell, C.R., and M.E. Schultz, “Principals of Error Theory and Cartographic Applications, ACIC Technical

Report #96, Feb. 1962.

[7] MIL-STD-6000001, Department of Defense Standard Practice; Mapping, Charting and Geodesy Accuracy,

February 26, 1990.

[8] Papoulis, A., Probability, Random Variables, and Stochastic Processes, 3rd

Edition, McGraw-Hill, 1991.

i = : .. 7 8 9 10 11 12 13 14 15 16 17 .. 24

x_i : .. 1.68 1.77 2.99 1.86 1.21 3.80 0.87 6.38 4.57 3.62 1.49 .. 1.28

y_i : .. 1.49 1.68 1.77 1.86 1.87 2.04 2.23 2.25 2.76 2.77 2.99 .. 6.38