Skill Assessment for Coupled Physical-Biological Models

of Marine Systems

Daniel R. Lynch

Dennis J. McGillicuddy, Jr.

Francisco E. Werner

Sponsors:

NOAA - CSCOR

NSF - CMG

Prepared for:

U.S. GLOBEC Pan-Regional Synthesis Workshop

27 November - 1 December 2006

NCAR, Boulder CO

OverviewGoals

Assess the state-of-the-art

Provide recommendations in support of Agency programs

Deliverables

Special volume of peer-reviewed contributions

Report to NOAA summarizing progress

Topical Organization

Scientific

Carbon Cycle

Harmful Algal Blooms

Ecosystem Dynamics and Fisheries

Estuarine/Coastal Water Quality

Cross -Cutting Themes

Skill Vocabulary

Metrics

Data Assimilation

Participation

Apex Contributions, Invited

GLOBECECOHABSABJGOFSEuropean Shelf Seas

Contributions

18 -- 30 papers42 et al -- 55 et al people

TimelineJanuary '06 Invitations out

July '06 Authors' Workshop 1Vocabulary Rev. 1Working Groups: DA, Metrics

Dec ‘06 Working Group Reports to Editors

Feb ‘07 Vocabulary Rev. 2 + Working Group Report Distribution

March '07 Authors’ Workshop 2

April ‘07 MS Submission; Peer Review Start

April ‘08 Final Copy to PrinterReport goes to NOAA

Peer-Reviewed Publication

Journal of Marine Systems

Coordination

3 Community Pieces

Vocabulary

Metrics

Data Assimilation

http://www-nml.dartmouth. edu/

Publications/internal_reports/

NML-06-Skill/

Vocabulary

Vocabulary

The first Bloom!

55 GLOBEC Contributions

Dartmouth

WHOI

UNH

UNC

Dalhousie

Rutgers

NMFS - WH

NMFS - Narragansett

NMFS - Sandy Hook

DFO - Halifax

DFO - St Andrew’s

DFO - Victoria

Reused in ECOHAB, SAB, EIRE, SWVI, NERRS, CICEET, RMRP, SeaGrant

,

Skill: Conformance to Truth • State of Model and Truth

• Processes - Internal Dynamics

• Modes of Expression - Properties, Features• Equilibria• Instabilities• Spectra• Covariance• Population Structure

The Realm of Error

Skill Assessment

• Judgement about Skill

• Future, Past

The realm of Mistake

What is Truth?

What is Truth?

Data Model

d m

Misfit

What is Truth?

Data Model

Truth real but unknowable

Errors unknowable

Prediction a credible blend:

Data + Model

Blend: Invokes statistics of d , m

Prediction Error: blend of d , m

Misfit: = d - m

d m

Misfit

Prediction

p

What is Truth?

Data Model

d m

Misfit

Prediction

p

Skill:Misfits

Small, Noisy

Deduced Inputs Small, Smooth

Features Credible

Truth real but unknowable

Errors unknowable

Prediction a credible blend:

Data + Model

Invokes statistics of d , m

Prediction Error: blend of d , m

Misfit: = d - m

Features Ex: a Retentive Gyre

• Physical Features– Is there a gyre?

– Size?

– Location?

– Timing?

– Residence Time?

– Entrance Paths?

– Exit Paths?

• Relative to Organism– Cohort

– Density

– Scale

– Age / Stage

– Onset / Demise

– Vital Rates

Bloom!

Misfit MetricsQuadratic Form

= W

W = Cov-1()

= d+

m

Importance of

Data Error

Model Error (Unmodeled part of Truth)

“Dictatorship of Measurement”

Regularization

Data Sparse --> Indeterminacy

= W p* Wp p

Importance of Prior

= W p* Wp p

Joint estimation of and p

Regularization adds bias toward prior

BPE - Best Prior Estimate

BPE is PDG --> small, p small

Post-Optimality Judgement

• Beyond Misfit

• Model - Truth

• Criterion?

Causality

Prior / Posterior

Logical

Previous / Subsequent

Temporal

Forward / Inverse

Influence in Classic Initial/Boundary Value Problem

StatisticsDistributions by Moments

Value of Moments: mean, variance, …

Ensemble within which Moments occur

Ex: 3 different ensembles

all previous realizations of a field

“Field variability”

all possible observations of this field

“Instrument Error”, “Noise”

all possible estimates of this field

“Inverse Noise”

Data and State Estimation

Time of Occurrence(Ocean)

Time of Availability(Information)

Future

(Now)

Past

Time

Time of Occurrence(Ocean)

Time of Availability(Information)

Forecast

Nowcast

Hindcast

All Data

Time of Occurrence(Ocean)

Time of Availability(Information)

Forecast

Nowcast

Hindcast

All Data

Time of Occurrence(Ocean)

Time of Availability(Information)

Forecast

Nowcast

HindcastAll Data

Model‘Data Product’

Time of Occurrence(Ocean)

Time of Availability(Information)

Forecast

Nowcast

Hindcast

Data Used

Bell

Time of Occurrence(Ocean)

Time of Availability(Information)

Forecast

Nowcast

Hindcast

Data Used

Bell Publication