Data-drivenModeling,PredictionandPredictability:TheComplexSystemsFramework

Surja Sharma,ErinLynchandEugeniaKalnayUniversityofMaryland,CollegePark

V.KrishnamurthyGeorgeMasonUniversity,Fairfax

GSFC AI WorkshopNov. 2018

Nonlinear Dynamics and Complexity Dynamics (Lorenz, 1963)

Deterministic dynamics, ChaosQuantitative resultsWeak connection with data

Structure (Mandelbrot, 1977)Real objects in nature

(Trees, clouds, coastline, etc.)Fractals and Multifractals

Dynamics + StructureSpatio-temporal DynamicsData-driven modeling in Complex Systems Framework

Machine Learnimg / Artificial Intelligence

Reconstruction of Dynamics

“Geometry from a time series”(Packard et al., 1980)

Embedding theorem (Takens, 1981)Time series data: x(t)Time-delay embedding:

xk(ti) = x(ti + (k-1)τ)Reconstructed space:

Xi = {x1(ti), x2(ti), x3(ti), ..}(Broomhead and King, J. Phys. A, 1986)

Actual

Reconstructed

Time series data

Complex Systems Framework:• Low-dimensional (data-driven) modeling• Dynamical prediction (Dynamics)• Predictability analysis (Statistics)

SpaceWeather:PredictionandPredictability

Data-drivenModeling:Phasespacereconstructionofdriver(solarwind)– response(magnetosphere)• Storms(Dst)• Substorms (AL)• Killerelectronflux

FirstPredictions[Sharma,Rev.Geophys.,1995]

EarlycontributionstoAI/MachineLearning

11/29/18 GSFCAI 4

Solar wind (VBs)

Past data: auroral electrojet index AL

Predicted ALandPredictability

[Ukhorskiy et al., 2002, 2004].

PredictabilityofSpaceWeatherGlobalorCoherentaspectsoftheMagnetosphere

Demonstratedby• Low-dimensionality– reconstructionof

phasespace[Vassiliadis etal.,1990;Sharmaetal.,1993]

• Modeling[Bakeretal.,1990]• Phenomenology[Siscoe,1991]• Phasetransition-likebehaviorfromdata-

drivenmodelingandMHDsimulations

impliesPredictability

Fundamentalcontributionbasedondata-drivenmodeling(AI/MachineLearning)

Similartoearlyresultsondynamicalbehavioroftheatmosphere

11/29/18 GSFC/AI 5

Transition from higher (orange)to lower (green) level.AL index data – observed andfrom global MHD simulations.

ExtremeeventsandEnsembleforecasting

• Data-drivenmodelswithoutgoverningequations

• ForecastsusingEnsembleTransformKalman Filter(ETKF)

• Ensemblespreadasanindicatorofextremeevents

Forecast ensemble

Erin Lynch, Ph. D. thesis (2018)

Data-drivenPredictionofMonsoonPhase space reconstruction model (PSRM).

Rainfall data on 0.25 deg longitude × 0.25 deg latitude grid for 1901-2009(1800 stations)

Climate Forecasting System (CFSv2 )State of the art numerical model (NOAA )

Modeling by Reconstruction using Rainfall and CFSv2 data.

Improvement of predictability

Krishnamurthy and Sharma, Geophys. Res. Lett. (2017)

ComparisonofpredictionsofPSRMandCFSv2

Key results and conclusions:

Intraseasonal oscillations are predictable

Predictability of intraseasonalphenomena such as MJO and midlatude processes

Data-driven modeling provides higher predictability

Modeling and prediction of spatio-temporal structure of space weather

Spatio-temporal Data: Networks of monitoring stations

Krishnamurthy and Sharma, Geophys. Res. Lett., 2017

ComplexSystemsFrameworkØ Data-drivenModelingØ Dynamicalprediction(globalandspatiallyextended)Ø CharacterizationofpredictabilityØ Extremeevents:QuantificationofpredictabilityØ PredictabilityofextremeeventsfromBigDataØ Quantitativemeasureofthelikelihoodofextremespaceweatherevents(data-drivenmodeling)

Ø PredictionofIntraseasonal climate(IndianMonsoon)Ø Applicationsandaframeworkforartificialintelligence, andmachinelearning

Ø Fourthparadigm– Data–enabledscience

11/29/18 9