Median Filtering and Median Filtering and Morphological Filtering
Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of...
-
Upload
miguel-brown -
Category
Documents
-
view
217 -
download
0
Transcript of Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of...
![Page 1: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/1.jpg)
Filtering for High Dimension Spatial Systems
Jonathan BriggsDepartment of StatisticsUniversity of Auckland
![Page 2: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/2.jpg)
Talk Outline
• Introduce the problem through an example• Describe the solution• Show some results
![Page 3: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/3.jpg)
Example
• Low trophic level marine eco-system• 5 System states:
– Phytoplankton– Nitrogen– Detritus– Chlorophyll– Oxygen
Det
Phy
Nit
ChlOxyPhy Growth Phy Growth
air
sea
Phy Mortality
Climate
![Page 4: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/4.jpg)
Example
• 5 System states:– Phytoplankton– Nitrogen– Detritus– Chlorophyll– Oxygen
• 350 layers• 1750 dimension state space
350 layers
5 states per layer
1 metre
Surface
350m
…
![Page 5: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/5.jpg)
Example
• Assume ecosystem at time t completely defined by 1750 dim state vector:
• Objective is to estimate at discrete time points {1:T} using noisy observations
• Using the state space model framework:
- evolution equation
- observation equation
- initial distribution
![Page 6: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/6.jpg)
Example
• Observations provided by BATS (http://bats.bios.edu/index.html)
• Deterministic model for provided by Mattern, J.P. et al. (Journal of Marine Systems, 2009)
![Page 7: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/7.jpg)
Deterministic Model
• Coupled physical-biological dynamic model• One hour time-steps• Implemented in GOTM (www.gotm.net)
![Page 8: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/8.jpg)
Deterministic model
Concentration Concentration
Dep
thD
epth
![Page 9: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/9.jpg)
Deterministic model
Concentration Concentration
Dep
thD
epth
![Page 10: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/10.jpg)
Problems
1. To improve state estimation using the (noisy) observations
2. To produce state estimate distributions, rather than point estimates
![Page 11: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/11.jpg)
Solution – state space model
• Evolution equation provided by deterministic model + assumed process noise
• Define the likelihood function that generates the observations given the state
• Assume the state at time 0 is from distribution h( )
- evolution equation
- observation equation
- initial distribution
.
![Page 12: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/12.jpg)
Currently Available Methods
• Gibbs Sampling
• Kalman Filter• Ensemble Kalman Filter• Local Ensemble Kalman Filter• Sequential Monte Carlo/Particle Filter
Sequential methods
All time steps at once
![Page 13: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/13.jpg)
Currently Available Methods
Sequential methods
[E.g. Snyder et al. 2008, Obstacles to high-dimensional particle filtering, Monthly Weather Review]
All time steps at once
![Page 14: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/14.jpg)
Solution – prediction
• Select a sample from an initial distribution
• Apply the evolution equation, including the addition of noise to each sample member to move the system forward one time-step
• Repeat until observation time
• Same as SMC/PF and EnKF
![Page 15: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/15.jpg)
Time Stepping
Concentration
Dep
thSurface
Deep
Phy d=0
![Page 16: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/16.jpg)
Time Stepping
Concentration
Dep
th
Phy d=0 Phy d=1
Surface
Deep
![Page 17: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/17.jpg)
Time Stepping
Concentration
Dep
th
Phy d=0 Phy d=1 Phy d=2
Surface
Deep
![Page 18: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/18.jpg)
Time Stepping
Concentration
Dep
th
Phy d=0 Phy d=2Phy d=1 Phy d=3
Surface
Deep
![Page 19: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/19.jpg)
Time Stepping
Concentration
Dep
th
Phy d=0 Phy d=1 Phy d=3Phy d=2 Phy d=26
Surface
Deep
…
![Page 20: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/20.jpg)
Solution – data assimilation
• We want an estimate of • We could treat as a standard Bayesian update:
– Prior is the latest model estimate: – Likelihood defined by the observation equation
• However, 1750 dimension update and standard methodologies fail
![Page 21: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/21.jpg)
Solution – data assimilation
• We can solve this problem sequentially:• Define a sequence of S layers
• Each is a 5-dim vector• Estimate using a particle
smoother (a two-filter smoother)
![Page 22: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/22.jpg)
Results - priors
Concentration
Dep
th
![Page 23: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/23.jpg)
Results - priors + observations
Concentration
Dep
th
![Page 24: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/24.jpg)
Results – forward filter quantiles
Concentration
Dep
th
![Page 25: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/25.jpg)
Results – backwards filter quantiles
Concentration
Dep
th
![Page 26: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/26.jpg)
Results – smoother quantiles
Concentration
Dep
th
![Page 27: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/27.jpg)
Results – smoother sample
Concentration
Dep
th
![Page 28: Filtering for High Dimension Spatial Systems Jonathan Briggs Department of Statistics University of Auckland.](https://reader033.fdocuments.us/reader033/viewer/2022052522/5513bf4c5503465b298b48b9/html5/thumbnails/28.jpg)
Conclusion
• I have presented a filtering methodology that works for high dimension spatial systems with general state distributions
• Plenty of development still to do…– Refinement– Extend to smoothing solution– Extend to higher order spatial systems