Flux uncertainty and qualitycriteria

Üllar Rannik, University of Helsinki

Outline

• Basic characteristics of turbulent records

• Flux random uncertainty

• Methods to calculate random errors

• Flux quality criteria

• Examples and statistics

Examples of turbulent records

Examples of probability distributions and correlation functions

An example of the “same flux“ measured by two different EC systems

• Measured at SMEAR II

• Two EC systems located approx. at 30 m distance

• Measuring almost the same flux footprint

• But not exactly the same realisations of turbulence

• Turbulence not (fully) independent

Random errors: general considerations• Imagine time series x with pdf p(x)

A) white noise (no correlation in time)

• Standard deviation (SD) of the average is the measure for the error due to finite ( ), random sampling:

• N is the number of independent observations

B) A time series with integral time scale

• In this case number of independent observations is decreased and

where the factor accounts for reduction of number of independent observations

In case of ensemble averaging no random error occurs!

2/1-= Nxx ssò¥

=0

2 ')'(1 dttRxx

x st

tNx

xx D=

tss

2

tx

Dt2

tTND

=

Random uncertainty of flux - definition

• If w(t), w(t+t), c(t) and c(t+t) are mutually joint Gaussian processes, then error variance of flux

><-= wcwcj

• Valid for

• Integral time scale

• Variance

• Covariance function

This is equivalent to

where

T<<jt

Flux uncertainty as random error (δ), being the measure of one standard deviation of the random uncertainty of turbulent flux observed over an averaging period T, can be evaluated by following different approaches:

Methods for estimating Flux uncertaintyMethods for estimating Flux uncertainty

( )[ ]22 ''''2

cwcwTIF -= jtd• “instantanous flux” (Wyngaard,

1973)

2/1-F= NSE sd• “standard error” (Vickers and

Mahrt, 1997)

ò¥

¥-

- += dffSfSfST wccwFM21 )()()(d• “Fourier method”(Rannik and

Vesala, 1999)

FrrorFlux RelativeNFF

RFE iFSEsd

==

Random errors, particle fluxes

The random flux uncertainty

depends in addition to turbulent flux variance also on the time-correlation of instantaneous flux events

Then the integral time scale of j is defined according to

(EQ. 1)

Where is the auto-covariance

function of φ.

))(('' ccwwcw --==j

ò¥

=0

2 ')'(1 dttRjj

j st

))'()()(()'( jjjjj -+-= ttttR

The integral time scale of fluxThe integral time scale of flux

( )[ ]22 ''''2

cwcwTIF -= jtd

A common parameterization used for this timescale is

where is the effective measurement height above the displacement height and u the mean wind speed (Pryor et al., 2007).

uze /=jt

jtWe estimated by numerical approach (EQ. 1) and we present

our results in term of normalized frequency

ez

uzn e

jpt2=

• n = 0.27+/-0.35 (0.008 as se) for unstable conditions

• for stable conditions÷÷ø

öççè

æ÷øö

çèæ+=

26.0

4.3121.0Lzn e

Using as an estimate of ,

implies that

uze / jt

16.021

==p

n

Here we have observed higher values for the corresponding frequency implying that times scale is smaller roughly by a factor of 2 in unstable (n = 0.27) and neutral conditions and up to 6 in very stable conditions.

However, for random uncertainty this implies the difference by square root of the factor.

( )[ ]22 ''''2

cwcwTIF -= jtd

uzn e

jpt2=

Another method to calculate random flux error

Flux uncertainty estimations: comparisonFlux uncertainty estimations: comparison

y = 0.965 x, R = 0.96 y = 0.974 x, R = 0.92

The best correlation was observed between the flux error estimates δSE and δIF .

In the following analysis the error estimate δIF is used.

EC system setup - SMEAR II

Measurement level: 23.3 m

CPC: TSI model 3010

Gas analyzer: LI-6262, LiCor Inc

Sonic anemometer: Solent 1012R2, Gill

Aerosol size distribution (from 3 to 500 nm particles in diameter) measurements were performed within the canopy (at 2 m height) using Differential Mobility Particle Sizer(DMPS) system.

EXAMPLE: LONGEXAMPLE: LONG--TERM AEROSOL PARTICLE TERM AEROSOL PARTICLE FLUX OBSERVATIONS AT SMEAR IIFLUX OBSERVATIONS AT SMEAR II

Formation of new aerosol particles

A day with no apparent particle formation but still significant variation in concentrations and fluxes in May 31 2002

About 32.5 % flux estimates emission

Frequency distribution of fluxes (measured by EC)

Flux uncertainty estimationsFlux uncertainty estimations

Typical random flux error (EC) in the order of 20%

Can be much larger depending on the instrumental noise (signal to noise ratio) and level of the variability due to “local” turbulent transfer and non-local forcings

For aerosol particle flux, the counting error due to discrete nature of aerosols is another source of random uncertainty [see e.g. Fairall, 1984]. The uncertainty of average flux can be evaluated as

TQcF wcounting sd =

, where Q is the volumetric flow rate through the counting device and T the averaging period (assuming that virtually all particles are counted). For a CPC with the flow rate Q = 1 LPM used in our study the typical flux error due to counting statistics is from 104 to 105 m-2 s-1, which is much smaller than the observed average flux values.

Additional random error source for Additional random error source for particle fluxesparticle fluxes

IFFDjt IFFD uZ /=jt SPFD ''cw ''cw ''cwFDFDFD IFFDjt IFFD uZ /=jt SPFD ''cw ''cw ''cwFDFDFDwithwith% % % withwith% % %

L < 0Unstable with estimated with

% passing %F< 0

100* vd/u*

% passing %F< 0

100* vd/u*

% passing %F< 0

100* vd/u*

RFE < 1 32.3 72.0 0.274 28.1 73.4 0.305 33.9 71.4 0.263

RFE< 0.5 17.6 82.0 0.596 13.3 85.5 0.821 20.1 80.6 0.494

RFE< 0.3 7.4 93.2 1.08 4.55 96.6 1.50 10.0 89.8 0.839

L > 0Stable % passing %

F < 0100* vd/u*

% passing %F < 0

100* vd/u*

% passing %F < 0

100* vd/u*

RFE < 1 38.1 76.0 0.247 30.1 78.9 0.300 40.3 74.8 0.240

RFE < 0.5 26.0 81.6 0.409 15.0 86.6 0.568 29.3 80.1 0.319

RE F< 0.3 15.6 87.5 0.556 4.8 92.6 0.937 19.2 85.6 0.444

IFFD jt IFFD Uze=jt SPFD

Particle flux data selection according to Particle flux data selection according to random flux error estimates and thresholdsrandom flux error estimates and thresholds

22

Flux quality criteriaVariable Description Apply to Allowed

valuesReferences

Flux instationarity

Stationarity Co-variances FS < 0.30 Foken and Wichura, 1996

Flux intermittency

Intermittency Co-variances FI < 1 Mahrt et al., 1998

Friction velocity

turbulence well developed?

Co-variances U* > 0.1-0.3 Not discussed here

Kurtosis Is the probability distribution narrow

Single variable time series

1 < KU < 8 Vickers and Mahrt, 1997

Skewness Is the probability distribution skewed

Single variable time series

-2 < SK < 2 Vickers and Mahrt, 1997

σu/u*,

σT/T*,…

Integral turbulence characteristics

Single variable time series

ITC<0.3 Foken and Wichura, 1996

Spectra Power spectraCo-spectra

Visual inspection

Not discussed here

Quality statistics applied for time series and fluxes. X = W, C denotes time series of vertical wind speed and concentration with duration T = 30 min, Xi subrecord with duration T/N = 5 min (N = 6). Correspondingly X , 'X , Xs and ''CWF = are the time average and deviation from it,

standard deviation of X and flux for the entire record of duration T; ix , iXs and '' iii CWF = are

time average, standard deviation and flux calculated over i’th subrecord. denotes averaging

over N subrecord values and ( ) 2/122 ><-><= iiF FFi

s is the standard deviation of iF . XS

denotes the spectrum of X ( ò¥

¥-

= XX dffS s)( ) and WCS is the cross-spectrum.

Acronym Test statistic Definition Range of quality values

SK Skewness 33' -XX s (-1,1)

K Kurtosis 44' -XX s (2,5)

HM Haar mean [ ] [ ] 11 4/))min()(max(min||max -+ -- XXXX Xii s

< 2

HV Haar variance [ ]2221

max -

+- XXX ii

sss < 2

FI Flux instationarity 1)( -- FFFi < 1

24

Flux (in)stationarity (Foken and Wichura, 1996)

• Measure the quality of co-variances

• Often about 40% of data omitted due to these, especially during night

– x = u, T, CO2, H2O, etc.

– The flux is often considered non-stationary if FS>0.3 and the Reynolds’ decomposition is not valid

5min 30min

30min

' ' ' '' '

w x w xFSw x

-=

The covariance calculated for the whole period (e.g.

30min)

The covariance calculated as a mean of the co-variances of

5min periods

Single time series quality criteria (1)

• Higher order moments (SK, K): possible instrument or recording problems and physical but unusual behaviour

• Haar transforms: discontinuities

Time series quality criteria (2): physical but unusual behavior

• A) Hard flagged by K and Haar variance

• B) Flagged by Haar mean and variance

• C) Flagged by Haar variance

Table 2. Percentages of observations not satisfying the time series quality criteria given in Table

1. Time series are denoted as following: W – vertical wind speed; P - particle concentration

measured by EC system; T – temperature; CO2- carbon dioxide concentration; H2O – water

vapour concentration. Statistics: SK –skewness; K – kurtosis; HM - Haar mean; HV – Haar

variance; Any – one or several of the statistics (SK, K, HM, HV) not satisfied. Total number of

analysed 30 min periods was 4933 for unstable (L < 0) and 5554 for stable (L > 0) stratifications.

Stability Quantity SK K HM HV Any L < 0

Unstable W 0.1 1.5 0.0 0.4 1.7 P 17.4 23.0 4.4 7.1 27.6 T 8.7 8.6 1.7 1.1 12.9

CO2 8.9 32.1 0.3 1.8 32.9 H2O 3.8 8.8 5.0 2.6 15.5

L > 0 Stable

W 0.4 5.7 0.0 1.3 5.9 P 12.2 18.5 2.0 3.8 21.0 T 3.2 5.5 0.7 0.9 6.9

CO2 15.4 46.7 0.5 1.6 47.5 H2O 4.3 19.5 2.0 1.0 21.7

Performance of (single) time series quality criteria

28

Integral turbulence test

• Measure the quality of the time series of a single variable (Wichura and Foken, 1995)

• Is the turbulence well developed? Is the flux variance similarity followed?

• Normalized standard deviation for wind components and a scalar as a function of stability

2 2, ,

1 1* *

c cu v w xz d z dc cu L X L

s s- -æ ö æ ö= =ç ÷ ç ÷è ø è ø

An example from SMEAR IIIFrom Vesala et al. 2008a

29

• If the measured normalized standard deviation deviates less than 30% from the model, the turbulence is considered well developed

mod*

*mod*

)/()/()/(

XXXITC

x

mesxx

sss

s-

=

From Lee et al. 2004, p.192Originally from other papers.

Parameter z/L c1 C2

σw/u* 0>z/L>-0.0032 1.3 0

-0.0032>z/L 2.0 1/8

σu/u* 0>z/L>-0.0032 2.7 0

-0.0032>z/L 4.15 1/8

σT/T* 0.02<z/L<1 1.4 -1/4

0.02>z/L>-0.062 0.5 -1/2

-0.062>z/L>-1 1.0 -1/4

-1>z/L 1.0 -1/3

• Instrumental noise – white noise

• Assumes no correlation btw. vertical wind speed and noise signal

• Could be estimated as (from std of noise c)

ncw

cw

sss =''

Performance of flux quality criteria

Table 3. Percentages of observations satisfying the scalar flux quality thresholds for different

statistics for unstable (L < 0) and stable (L > 0) stratification conditions. RFE – random flux

error; TRFE – theoretical random flux error; FI – flux intermittency. Number of all analysed 30

min periods 10400, unstable 4900, stable 5500.

Stability Scalar Threshold 1.0 Threshold 0.3

RFE TRFE FI RFE TRFE FI L < 0

Unstable P 76.2 57.6 72.0 33.6 8.2 30.4 T 91.4 88.2 91.8 71.3 64.4 73.9

CO2 94.7 93.3 98.3 79.7 69.4 78.7 H2O 94.4 88.0 92.5 72.7 49.2 67.5

L > 0 Stable

P 80.5 63.1 72.8 41.9 14.1 34.2 T 92.4 89.0 91.8 68.8 60.7 74.9

CO2 93.0 87.3 89.8 70.6 58.7 71.5 H2O 80.7 59.5 72.4 41.2 9.9 30.7

FI – flux (in)stationarity