Cambridge 2014 Complexity, tails and trends
-
Upload
prof-nick-watkins -
Category
Science
-
view
120 -
download
0
description
Transcript of Cambridge 2014 Complexity, tails and trends
![Page 1: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/1.jpg)
Complexity, tails & trends ...Nick Watkins ([email protected])
Max Planck Institute for the Physics of Complex Systems, Dresden, Germany (2013-14)
MCT, Open University
Centre for the Analysis of Time Series, LSE
Centre for Fusion Space & Astrophysics, University of Warwick
![Page 2: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/2.jpg)
Addendum:
These slides were given as a contributed talk at the Trends 2014 meeting at Clare College, Cambridge, UK on 28th July 2014. I have also included the spare slides which were prepared in case of technical questions etc.
Nick Watkins29/7/2014
![Page 3: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/3.jpg)
BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?
![Page 4: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/4.jpg)
BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?
![Page 5: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/5.jpg)
BIG QUESTION: How big, how long lasting, and how serially dependent are real world risks and hazards ?
![Page 6: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/6.jpg)
SUBTOPIC: How do 2 types of complexity affect trend detection ?
This presentation complements those by Sandra and Christian and discusses aspects of two problems we face in real world time series:
• Long range dependence, if present in a system, would complicate the
trend detection, because it implies the presence of low
frequency "slow" fluctuations.
• Another effect, the presence of heavier than Gaussian tails in probability distributions, is, meanwhile, a source of "wild" fluctuations.
• Important to distinguish these two concepts, a process begin in the 60s by Benoit B Mandelbrot.
![Page 7: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/7.jpg)
I will mainly be surveying ideas today, the detail is here:
[1] Watkins, “Bunched black (and grouped grey) swans”, Frontiers review article, GRL, 2013, http://onlinelibrary.wiley.com/doi/10.1002/grl.50103/full[2] Franzke et al, Phil. Trans. A, 2012, doi:10.1098/rsta.2011.0349[3] Graves et al, “A brief history of long memory”, submitted 2014 arXiv:1406.6018v1 [stat.OT][4] Graves et al, “Efficient Bayesian inference for long memory processes”, submitted 2014, arXiv:1403.2940v1 [stat.ME]
and will be developed in:
[5] Franzke et al, “Hurst without Joseph”, in prep.[6] Watkins and Franzke, in prep.
![Page 8: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/8.jpg)
2Variance
Correlation length
2 2( )x
( ) ( ) ( )
e.g. ( ) ~ exp( )
x t x t
How do we quantify “big”, “correlated”?
![Page 9: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/9.jpg)
2Variance
Correlation length
2 2( )x
( ) ( ) ( )
e.g. ( ) ~ exp( )
x t x t
iid Gaussian
white ~ ( )
(0, )N
![Page 10: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/10.jpg)
2
Correlation length
Damped Brownian motion
"Physics"mv v
1
First order autoregressive AR(1)
(1 ) "stats"N N N
x x x
,x v
![Page 11: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/11.jpg)
Variance But what if fluctuations “wild” in amplitude. Consider fat tailed pdf , wildest case of which is power law with small tail exponent ?
Light tail
Power law tail
(1 )~
stable
p
range
df (
: 0 2
) xp x
![Page 12: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/12.jpg)
Fat tails not just curiosity: a well known topic in financial risk
[S&P 500] Mantegna &
Stanley, Nature, 1996
![Page 13: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/13.jpg)
Fat tails not just curiosity: a well known topic in financial risk … lead to erratically fluctuating moments
[S&P 500] Mantegna &
Stanley, Nature, 1996
1963
![Page 14: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/14.jpg)
We also observe fat(tish) tails in solar wind, ionosphere and magnetosphere
AE: Chapman, Hnat, Rowlands
& Watkins, NPG, 2005
Polar UVI: Uritsky et al, reviewed in
Freeman & Watkins, Science, 2002
SW Poynting flux: Freeman,
Watkins & Riley, PRE, 2000
See also Chapman, et al, GRL, 1998;
Watkins et al, GRL, 1999;
Lui et al, GRL, 2000 etc
![Page 15: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/15.jpg)
Correlation length
But what if fluctuations “slow” in time, consider long rangedependence
Light tail
Power law tail
] ~
( ) ( ) ( )
( )
( ) F[
x t x t
S f f
(1 )~
stable
p
range
df (
: 0 2
) xp x
We just saw case
Variance
![Page 16: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/16.jpg)
Nile river minima
600 700 800 900 1000 1100 1200 13009
10
11
12
13
14
15Annual minimum level of Nile: 622-1284
Annual m
inim
um
:
Time in years
Nile minima, 622-1284
![Page 17: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/17.jpg)
Long range dependence contentious topic since Hurst reported anomalous growth of range in Nile river minima …
600 700 800 900 1000 1100 1200 13009
10
11
12
13
14
15Annual minimum level of Nile: 622-1284
Annual m
inim
um
:
Time in years
Hurst, Nature, 1957
Nile minima, 622-1284
![Page 18: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/18.jpg)
Long range dependence contentious topic since Hurst reported anomalous growth of range in Nile river minima … and Mandelbrot proposed LRD as one possible explanation …d parameter … J=d+1/2
600 700 800 900 1000 1100 1200 13009
10
11
12
13
14
15Annual minimum level of Nile: 622-1284
Annual m
inim
um
:
Time in years
Hurst, Nature, 1957
Walk built from noise with d=-1/2
As above for d=1/2
Nile minima, 622-1284
![Page 19: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/19.jpg)
Joseph Effect:
29 July 2014 19
Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle
... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ...
Genesis: 41, 29-30.
![Page 20: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/20.jpg)
Joseph Effect:
29 July 2014 20
Pharoah’s dream of 7 years of plenty and 7 years of drought. Now shuffle
Point is that marginal distribution, of sample at least, unaffected by
shuffling, but that the two series represent very different worlds for insurers,
or Pharoahs. Former unlikely to happen in random trend free process without LRD.
... there came seven years of great plenty throughout the land of Egypt. And there shall arise after them seven years of famine ...
Genesis: 41, 29-30.
![Page 21: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/21.jpg)
We see some classic long range dependence indicators in our space weather data …
AE power spectrum
Tsurutani et al, GRL, 1991
![Page 22: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/22.jpg)
We see some classic long range dependence indicators in our space weather data …
AE power spectrum .
Tsurutani et al, GRL, 1991
ACF. Watkins, NPG, 2002
after Takalo and Timonen
( )
( ) F[ ] ~S f f
![Page 23: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/23.jpg)
So how have I been modelling competing effects of LRD and heavy tails so far ? LFSM
• Use linear fractional stable motion (LFSM) model
• Self-similarity exponent H depends both on memory parameter d and tail exponent alpha in this class of models.
1 11( ) ( ) ( ) ( )
H H
H HR
X t C t s s dL s
1/ d H
Memory kernel: d measures LRD
α-stable jump: heavy tails
![Page 24: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/24.jpg)
LFSM model unpicks the different scaling of solar wind Poynting flux driver & response in AE/U/L
Watkins et al, Space Science
Reviews, 2005
Data
Model
Peak pdf vs. diff. time Std dev vs tau
![Page 25: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/25.jpg)
Need better model with LRD & heavy tails & ability to add HF damping: α-stable AutoRegressive Fractionally Integrated Moving Average (ARFIMA(p,d,q)):
29 July 2014 25
Ionospheric index
power spectrum .
Tsurutani et al, GRL,
1991
( )(1 ) ( )d
t tB B X B
1
( ) 1p
j
j
j
z z
1 t tBX X
Granger (& Joyeux), 1980
![Page 26: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/26.jpg)
Need better model with LRD & heavy tails & ability to add HF damping: α-stable AutoRegressive Fractionally Integrated Moving Average (ARFIMA(p,d,q)):
29 July 2014 26
0.15(1 ) t tB X 1.5 Ionospheric index
power spectrum .
Tsurutani et al, GRL,
1991
( )(1 ) ( )d
t tB B X B
1
( ) 1p
j
j
j
z z
1 t tBX X
Granger (& Joyeux), 1980
![Page 27: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/27.jpg)
Need better inference: Cambridge-BAS PhD Tim Graves developed Bayesian method: tested on α-stable ARFIMA(0,d,0) where heavy tails & LRD co-exist
Graves, Gramacy, Franzke & Watkins, submitted 2014; and in prep.
1.5 0.15 d
![Page 28: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/28.jpg)
BUT …
• Both Mandelbrot (1965, 1967), and his critics (notably VitKlemes, WRR, 1974) noted that there were nonstationarymodels that also produced 1/f spectra-close relatives of what is now called Alternating Fractional Renewal Process.
• Mandelbrot was at pains to emphasise that needed to use our eyes as well as mathematical rigour (e.g. his Selecta series of annotated collected papers).
• Work in progress with Christian on how best to distinguish “true” LRD from models which share the power spectral and/or growth of range features but actually show much shorter typical lengths for dependent runs …
29 July 2014 28
![Page 29: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/29.jpg)
Why does LRD matter in already fat-tailed hazards ? [Riley, Space Weather, 2012 ]: • Knowing relative frequency of a coronal mass ejection of a given
magnitude …
![Page 30: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/30.jpg)
Why does LRD matter in already fat-tailed hazards ? [Riley, Space Weather, 2012 ]: … Knowing relative frequency of a coronal mass ejection of a given magnitude doesn’t fully specify the hazard:
Bunching, whether short range, or full blown LRD, affects overall hazard.
“Grouped grey swan” problem …[Watkins, GRL Frontiers, 2013]
Link to interacting hazards problem, c.f. UCL-Kings-Southampton Workshop & EGU session, 2013
![Page 31: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/31.jpg)
Conclusions:
Bold paradigms, including some imported from outside space
physics
Stimulus to statistical inference-confrontation
of early claims with rigour
Critical thinking about
assumptions and methods
Risk and hazard questions: how big, how correlated ? Basic statistical concepts (finite) variance and (finite) correlation length. How we might relax these limits, and why we might want to [Watkins, GRL Frontiers, 2013]. Showed you some examples of work in this area [see also Watkins et al, GRL, 1999; Freeman, Watkins et al, PRE, 2000; Watkins et al, PRE, 2009 and outcomes of BAS Natural Complexity project & Warwick collaboration].
Statistical methodology: Importance of confronting these exciting new results and ideas with statistical inference, and more flexible models such as ARFIMA: e.g. Bayesian inference of long range dependence [Graves, Gramacy, Franzke & Watkins, first 2 papers now submitted].
Closing the loop: Better inference alone is not enough, though-work is in progress on how to distinguish between competing models. [Franzke et al; Watkins and Franzke]
![Page 32: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/32.jpg)
SPARESSubtitle
![Page 33: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/33.jpg)
Ideal reservoir • Average influx over
years, need to ensure annual released
volume equals mean influx:
Accumulated deviation
of the influx from the
mean:
29 July 2014 33
1
1( )
t
t
1
) { ( )( },t
u
uX t
![Page 34: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/34.jpg)
Range:
Standard deviation:
Form against interval Plot loglog.
White Gaussian noise prediction
(Rescaled) range
29 July 2014 34
R
S
0 0max ) min ( ), ,(t tx t x t
1S
1/2/ ~R S
![Page 35: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/35.jpg)
AR(1): 1st order AutoRegressive
29 July 2014 35
1 1t t tX X
0 100 200 300 400 500 600 700 800 900 1000-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9
![Page 36: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/36.jpg)
AR(1): 1st order AutoRegressive
29 July 2014 36
1(1( ) ) t tB B X
1 1t t tX X
1ttBX X 0 100 200 300 400 500 600 700 800 900 1000
-8
-6
-4
-2
0
2
4
6
8Example series of AR(1)
1 0.9
1
( ) 1p
j
j
j
z z
![Page 37: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/37.jpg)
AutoRegressive Fractionally Integrated Moving Average [ARFIMA(p,d,q)]
29 July 2014 37
( )(1 ) ( )d
t tB B X B
Autoregressive term of order p
Moving average of order q
1
( ) 1q
j
j
j
z z
Fractional integration of order d
Granger (& Joyeux), 1980
(1 )d
t tB X Pure LRD
ARFIMA(0,d,0 ):
![Page 38: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/38.jpg)
Exact Bayesian inference on ARFIMA for d in Gaussian special case
• Have data x, assumed to be a realization from a distribution with likelihood function L, parameters psi are the objects of interest.
• ARFIMA has parameters μ (location), σ (scale), d (order of fractional difference), φ (autoregression), θ. All but d, and its sign are essentially nuisance parameters here.
• First assume Gaussian innovations.
• Assume flat priors for μ, log σ and d …
• Even with this, likelihood for d very complex
• No analytic posterior p --- use MCMC sampling 29 July 2014 38
( | ) ( ) ( | )x p L x
![Page 39: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/39.jpg)
Key features
• Don’t want to assume form of p, q – use reversible jump MCMC [Green, Biometrika 1995]
• Reparameterisation of model to enforce stationarityconstraints on φ and θ.
• Efficient calculation of Gaussian likelihood (long memory correlation structure prevents use of standard quick methods)
• Necessary use of Metropolis-Hastings requires careful selection of proposal distribution
• Parameter correlation (φ,d) requires blocking
29 July 2014 39
![Page 40: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/40.jpg)
Approximate inference in more general case
• Drop Gaussianity assumption.
• Go to more general distribution (t,α-stable,…)
• Seek joint inference on d, α
• Approximate long memory process as very high order AR
• Construct likelihood sequentially
• Use auxiliary variables to integrate out unknown history
29 July 2014 40
![Page 41: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/41.jpg)
Further developments
29 July 2014 41
~ 1/d n• In addition to joint inference problem.
have studied how posterior variance
of d depends on sample size n,
Motivated by Kiyani, Chapman & Watkins PRE,
2009 study of how errors in structure functions
depend on sample size. Watkins et al, in prep.
![Page 42: Cambridge 2014 Complexity, tails and trends](https://reader035.fdocuments.us/reader035/viewer/2022062708/558cafcdd8b42a27188b4759/html5/thumbnails/42.jpg)
LFSM model allowed initial burst scaling study
H2
1
) '( 't
ts x t dt
Prediction: ( ) ~ s 1/ (2 H) p s
Watkins et al,
PRE, 2009