Experimental Errors

True value – the value that would be obtained in the absence of errors – can only be

known in a practical sense through standards, which are true values by agreement, rather

than real true values. Measured values can approach true values to more and more

significant figures through careful experimentation designed to eliminate more sources of

error, and confirmed by duplication of results in other experimental setups.

Two types of errors: systematic errors and random errors

Systematic errors – deviations between the mean of a large number of measured

values and the true value – due to limitations of the measurement equipment or

improper calibration – sometimes called “bias” – always present in measurements

– must be estimated by the person making the measurements – can not be

analyzed using statistics

truesystematic xxe −=

Random errors – deviations between measured values and the mean value of a

large number of repeated measurements – due to limitations of the measurement

equipment or operator technique – can be accounted for by statistics through

repetitions of the measurement

xxeirandom−=

In practice, we can not know the true value, and we can not afford to make large

numbers of measurements to pin down the mean, thus, exact values of errors

should be considered to be unknowable. However, we can estimate the

magnitude of the errors one might expect from a given experiment, which is

called the “experimental uncertainty”.

Two ways of expressing the uncertainty of experimental data: accuracy and precision

Accuracy – how close the value of a measurement is believed to be to the true

value – depends on the estimated magnitudes of both systematic and random

errors

Precision – how big the scatter of repeated measurements about the mean value –

depends on the magnitude of random errors only - different from sensitivity,

which is the smallest change in the variable being measured that can be sensed by

an instrument – can be represented by a statistical uncertainty (variance or

standard deviation)

Standards and calibration of instruments

Standards – based on international agreements on defined fundamental units of

measurement – made practical through defined calibration instruments and

methods of measurement – forms the basis for instrument manufacturers’

specifications of accuracy

Calibration – 1) comparison of a measurement instrument with one traceable to

standards; or 2) comparison of instrument readings with measurements of more

fundamental dimensions of the process variable, which can presumably be

measured more accurately (for example, comparison of volumetric flowmeter

readings with direct measurements of volume delivered in a time interval)

Calibration curve – a plot of the best estimate of the true value of the process

variable versus instrument reading

Estimation of errors

Total Error = Systematic Error + Random Error (usually assumed that variances

in systematic and random errors add to give the variance in total error – not a

good basis for this assumption, as systematic errors can not be rigorously treated

by statistics). If u represents the uncertainty in a measurement, which is the

estimated error in the measurement,

5.022 )( systematicrandomtotal uuu +≈

Random Errors are usually expressed in terms of some standard deviation (square

root of variance), which may be based on manufacturer specifications, repeated

measurements, or engineering experience.

For a set of repeated measurements (of the same quantity) in the same apparatus,

all with the same uncertainty,

Variance =

( )

1

2

1

−

−∑=

N

yyN

i

i

where y is the mean N

y i∑

Std Devn (σ ) = (Variance)^0.5

Maximum Random Error (urandom) = 2σ (95 % level - often assumed)

Note that proper use of statistics implies many data points. If only 2 or 3

repetitions are made, then just use the maximum spread as a measure of

Random Error.

Note that statistical analyses are based on the assumption of a Gaussian

distribution of measurements about a mean value (see Appendix).

For a set of repeated measurements yi (of the same quantity) in different

experimental systems, all with different uncertainties σi, a common procedure is

to minimize the sum of squares of the deviations divided by weighting factors

(taken to be the uncertainties of the individual measurements). This leads to the

following relation:

Weighted Mean =

∑

∑=

2

2

1

i

i

i

WM

y

y

σ

σ

The standard deviation is then calculated from the square root of the

variance in the usual way. Note that this standard deviation will be larger

than the one calculated assuming all data points have the same uncertainty,

but it will be a better representation of the data (including uncertainties).

Example. The following table contains recent measurements of the Universal

Gravitational Constant in different laboratories, with estimated uncertainties.

Note that the weighted mean of these values is significantly different from the

true mean, but is in good agreement with the recommended value by CODATA.

This is an example of a standard that has gotten less certain (increased

uncertainty) over time, as a series of new measurements have been made in

various labs around the world. Of course, it means that the uncertainties in earlier

experiments were underestimated.

Recent Measurements of Gravitational Constant (G = G-Value*1E-11 N m2 kg

-2)

CODATA: Set in 1986 as 6.672 +/- 0.001; Revised in 2000 to 6.672 +/- 0.010

(from Analysis of Data by J. M. Haile)

Year G-Value Error Devn^2 (Err^2)^-1 Wtd Value Devn^2 (Wtd)

1994 6.670 0.0050 0.000020 40000 266800 0.000005

1995 6.666 0.0005 0.000072 4000000 26664000 0.000039

1995 6.672 0.0010 0.000006 1000000 6672000 0.000000

1997 6.674 0.0010 0.000000 1000000 6674000 0.000003

1998 6.670 0.0010 0.000020 1000000 6670000 0.000005

1999 6.674 0.0010 0.000000 1000000 6674000 0.000003

1999 6.673 0.0030 0.000002 111111 741444 0.000001

1999 6.683 0.0120 0.000072 6944 46410 0.000116

1999 6.687 0.0090 0.000156 12346 82556 0.000218

1999 6.676 0.0020 0.000002 250000 1669000 0.000014

1999 6.675 0.0010 0.000000 1000000 6675000 0.000008

2000 6.674 0.0003 0.000000 11111111 74155556 0.000003

80.094 0.000353 20531512 136990765 0.000415

Mean = 6.6745 Wtd Mean = 6.6722

Std Devn = 0.0057 Std Devn = 0.0061

If a linear equation y = ax + b is fit by least squares to a set of y vs. x data (all

points with the same uncertainty, all of which is assumed to be in y), then the

numerator in the Variance becomes the sum of the squares of the deviations

between the experimental and calculated y values, and the denominator becomes

N - 2, because two constants (a, b) have been fit rather than one (the mean), and

the equation for the standard deviation for the fit becomes

2

)( 2

exp

−

−=∑

N

yy calc

yσ

Systematic Errors are usually estimated – based on manufacturer specifications as

modified by age and usage of the instrument – can be minimized through

calibration

Propagation of errors

Errors in calculated process variables – result from individual errors in all

variables that are used in the calculation – errors propagate through formulas in

regular ways

Note: All formulas given below assume that the variables are all independent, that

is, that they are not correlated with each other.

Addition or subtraction – variances (squares of standard errors) in the individual

measurements add

⋅⋅⋅++=

⋅⋅⋅++=2

2

2

1

2

21

xxy

xxy

σσσ

For multiplication or division, squares of fractional errors add. A slightly more

general form is considered here:

( )⋅⋅⋅= 21

21

aaxxy α

where α is a pure constant (no error involved with it, e.g., π/4)

⋅⋅⋅+

+

=

2

2

2

2

2

1

2

1

2

21

xa

xa

y

xxyσσσ

Example. Suppose the length and diameter are measured for a short

cylinder. How is the uncertainty in its volume related to uncertainties in

these measurements?

22

2

2

2

2

4

+

=

=

LdV

LdV

LdV σσσ

π

This shows that if length and diameter are comparable, or if it is a long

cylinder, the measure of the diameter is most critical. However, for a thin

wafer, the length (thickness) is likely to be the critical measurement.

For other functions, the uncertainties must be determined through differentiation.

The general relation (which yields the two special cases above) is

⋅⋅⋅+

∂

∂+

∂

∂≈

⋅⋅⋅=

2

2

2

2

2

1

2

21

21

),,(

x

a

x

a

yx

f

x

f

xxfy

σσσ

where "a" is the point of interest (x1(a)

, x2(a)

, …). For a derivation of this

formula, see the Appendix for this document.

Note: If xi represents a derivative (e.g., dT/dx in a heat conduction

equation), there may be as much as an order of magnitude greater error

than would be indicated by the errors in the measured variables involved

in the derivative (T and x).

2/1

22

/ 10

+

≈

xTdx

dT xT

dxdT

σσσ

On the other hand, if T is linear in x, then the factor of 10 may be too

large. For example, if T = ax + b, then the uncertainty in dT/dx will be

equal to the uncertainty in a, which can be found from standard statistical

analysis. Similarly, more complicated functions could be fit to the T-x

data, with the uncertainties in dT/dx determined from the uncertainties in

the coefficients.

Note: If xi represents an integral, then the random error will likely be

significantly less than the error predicted from the errors in the variables

involved.

Example. Estimate the standard and maximum errors for the amount of heat

transferred to a flowing fluid in a heat exchanger, calculated from TCmQ p ∆= & .

Assume that a good estimate of uncertainty is 2 times the standard error.

Applicable Relation

2222

∆+

+

=

∆

TCmQ

T

p

CmQ p σσσσ

&

&

Flow Rate

The flow rate was measured by timing delivery of 100 lb of fluid, with

estimated maximum errors of 1 lb in mass delivered and 0.3 s in time.

The measured time was 23.2 s.

00007.02.23

15.0

100

5.022222

=

+

=

+

=

=

tmm

t

mm

tmm σσσ&

&

&

Heat Capacity

A value of 1.56 Btu/lb-oF was taken from a correlation for fluids like the

one being used in the exchanger, and the estimated accuracy of the

correlation is 2 %.

( ) 0004.002.02

2

==

p

C

C

pσ

Change in Temperature

The inlet temperature was measured as 94 oF and the exit temperature as

41 oF, with a maximum estimated uncertainty in 2

oF for each

measurement. Thus, the standard error in each temperature is 1 oF. Since

the variances add, the standard error in ∆T is √2 oF, and

0007.053

222

=

=

∆∆

T

Tσ

Error in Q

%7/1009.02.

/1004.0

/1028.1)3600)(53)(56.1(32.4

0012.00007.00004.000007.0

6

6

6

2

===

=

==

=++=

hrBtuxQinUncert

hrBtux

hrBtuxQ

Q

Q

Q

Q

σ

σ

σ

Appendix.

Gaussian Distribution Functions

Define a probability function P by letting Pdx be the fraction of many measurements of x

that lie between x and x+dx. Thus, the integral of Pdx over all possible values of x will

equal unity. The probability distribution function is called a Gaussian function if it is

given by the formula

dxedxxxP x

xx

x

x

2

2

2

)(

2

1),,(

σ

πσσ

−−

=

In this formula, x is the mean of the measurements, and x

σ is the standard deviation in x.

The Gaussian distribution is a peak-shaped, symmetrical function about the mean, and the

standard deviation is the “half width” of the peak (half of the peak width where P has

dropped to e/1 of its peak value). From the formula above, the standard deviation can

also be written in terms of the peak value as

π

σ2

1

maxPx =

If a new function is defined in terms of x, then you may or may not get a Gaussian

distribution back for the new function. For example, if Axy = , then the distribution for

y will be Gaussian, as can be easily verified by substitution into the above definition.

However, if xAy /= is the new function, then you will not get a Gaussian distribution

for y, rather one with a longer tail on the right side (like some chromatograph peaks that

exhibit “tailing”).

Derivation of Propagation of Errors Formula

Assume a functional relationship involving two measured variables (x1 and x2),

each of which can be described by a Gaussian distribution

),( 21 xxfy =

For example, this could be an equation of state (e.g., PR or SRK), where we

desire to calculate the density of a gas from measured values of temperature and

pressure. Imagine we have the gas at equilibrium and we make repeated

measurements of the temperature and the pressure (with a thermometer and a

pressure gauge).

Assume we have made N replicate measurements of the general variables (as

pairs), so we can calculate the mean value of each by the relations

N

xx

N

xx

i

i

∑

∑

=

=

)(2

2

)(1

1

Define the functions yi and y by the following

),(

),(

21

)(2)(1

xxfy

xxfy iii

=

=

Expand yi about y, neglecting higher order terms,

.)()(

212

2)(2

1

1)(1 neglx

yxx

x

yxxyy

x

i

x

ii +∂

∂−+

∂

∂−+=

By definition, the variance in y can be written as

1

)()(

1

)(

2

2

2)(2

1

1)(12

2 21

−

∂

∂−+

∂

∂−

=−

−=

∑∑

N

x

yxx

x

yxx

N

yy x

i

x

i

i

yσ

Since the derivatives are evaluated at the mean values, they can be pulled out of

the summations. The remaining summations are by definition equal to the

following:

))((1

2

)(1

1

)(1

1

2)(21)(1

2

2)(2

2

2

1)(1

2

21

2

1

xxxxN

xxN

xxN

iixx

ix

ix

−−−

=

−−

=

−−

=

∑

∑

∑

σ

σ

σ

With these definitions, the variance in y can be written as

21

21

2

2

1

121

2

2

2

2

2

1

2

xx

xx

x

x

x

x

yx

y

x

y

x

y

x

yσσσσ

∂

∂

∂

∂+

∂

∂+

∂

∂=

If it is assumed that the variables x1 and x2 are independent (uncorrelated), then

the last term can be dropped. Additional independent variables (e.g., x3, etc.)

would contribute additional terms like the first two.

This general formula can be used to obtain the estimated error in the mean of N

repeated measurements of the same quantity, all with the same uncertainty σ.

N

NN

NN

xxxN

yy

y

y

N

σσ

σσσσ

=

=+

+

=

+++==

2

2

2

2

2

22

21

111

)(1

L

L

Thus, the uncertainty in the mean value decreases with the square root of the

number of measurements. In this formula, σ is the standard deviation for the

Gaussian distribution describing a very large number of measurements of the

variable x, and it should not be confused with yσ , the uncertainty in the mean.

This may be confusing, as we often associate the uncertainty in the mean value of

the measured variable with the standard deviation of the distribution. The point is

that the mean can be obtained more accurately, if lots of repeated measurements

are made, which is usually not practical in engineering project work.