Diel Oxygen Analysis

Tim Kratz, Laurence Choi, Barbara Benson, Yu Hen Hu

University of Wisconsin-Madison

Center for Limnology

The Midnight Surge

153 153.5 154 154.5 1559.2

9.4

9.6

9.8

10

10.2

10.4

10.6

Time

DO

Outline

Preprocessing

Feature Extraction

Classification

Verification

Correlation

Algorithm proceeds in 5 steps:

Problem Statement Given a set of high frequency data points and

timestamps, objectively detect and quantify the presence of the midnight surge.

150 151 152 153 154 155 156 1579

10

11

157 158 159 160 161 162 163 1648

9

10

164 165 166 167 168 169 170 1718

9

10

171 172 173 174 175 176 177 1788

9

10

Sparkling Lake 2004

Some other lakes Lake Taihu - 2006

212 213 214 215 216 217 218 2195

10

15

219 220 221 222 223 224 225 2265

10

15

20

226 227 228 229 230 231 232 2330

5

10

15

233 234 235 236 237 238 239 2400

10

20

Some other lakes Lake Ormajarvi - 2006

236 237 238 239 240 241 242 243 244

6.8

7

7.2

7.4

7.6

7.8

8

8.2

243 244 245 246 247 248 249 250 251

6.5

7

7.5

8

8.5

9

9.5

10

Data Preprocessing

Missing data points Need to choose whether to interpolate

or just to leave data sequences fragmented

Differences in sampling frequencies Resample the data at appropriate rates

– we also need to know when this is feasible

Preprocessing

Feature Extraction

Classification

Verification

Correlation

Algorithm Development

1. Define the time domain in which a surge in oxygen concentration is unexpected. Here, we define it to be the time between half an hour after sunset and half an hour before sunrise.

2. The structure of a rise generally requires a string of mostly positive gradients between data points. Since raw data often contains jitter, we need to pass the data through a low-pass filter.

Filter Selection We want to smooth the data (remove high

frequency noise) by passing the raw data through a low-pass filter.

We don’t want to smooth out the feature of interest!

224 224.2 224.4 224.6 224.8 225 225.2 225.4 225.6 225.8 2268

8.1

8.2

8.3

8.4

8.5

8.6

8.7

8.8

8.9

Algorithm Design

After filtering, we can use the gradients to find the local minima in our data set.

Using these minima, we segment our data set and extract features such as volume and height from the curve.

Given that we’re trying to detect deviation from a negative slope, we need to choose a baseline appropriately.

Feature Extraction Baseline Selection When the next minima is

above the previous minima, use a linear interpolation for the baseline.

When the next data point is below the previous minima, assume the general decrease has taken over again – use a horizontal line as the baseline.

256.6 256.7 256.8 256.9 257 257.1 257.2 257.38.1

8.2

8.3

8.4

8.5

8.6

8.7filteredminimabaseline

Preprocessing

Feature Extraction

Classification

Verification

Correlation

Classification

With just the volume metric, we implemented a classification system based upon a fixed threshold.

This does not take into account lakes with characteristically different amplitudes; a lake with typically smaller daily variation will have less chance of triggering the classifier.

Preprocessing

Feature Extraction

Classification

Verification

Correlation

Bump Height vs. Daily Amplitude Now, we take into account the height of

our feature with respect to the range of a window of days around our feature.

224 224.2 224.4 224.6 224.8 225 225.2 225.4 225.6 225.8 2268

8.1

8.2

8.3

8.4

8.5

8.6

8.7

8.8

8.9

Results – Sparkling 2004

150 151 152 153 154 155 156 157 158

9

9.5

10

10.5

157 158 159 160 161 162 163 164 165

8.5

9

9.5

164 165 166 167 168 169 170 171 1728.48.68.8

99.29.4

171 172 173 174 175 176 177 178 179

8.28.48.68.8

99.2

Results –Taihu 2006

212 213 214 215 216 217 218 219 2206

8

10

12

219 220 221 222 223 224 225 226 227

5

10

15

226 227 228 229 230 231 232 233 234

468

101214

233 234 235 236 237 238 239 240 241

468

101214

Results – Ormajarvi 2006

236 237 238 239 240 241 242 243 244

6.8

7

7.2

7.4

7.6

7.8

8

8.2

243 244 245 246 247 248 249 250 251

6.5

7

7.5

8

8.5

9

9.5

10

Expert Comparison

Using expert analysis, have a set of ideal classifications to compare against. Our experts classified each day regarding the presence of the bump as either “Yes”, “Maybe” or “No”.

These results were compared against the algorithm’s “Yes” or “No”.

Preprocessing

Feature Extraction

Classification

Verification

Correlation

Classifier Accuracy

Yes Maybe No

Yes 371 37 31

No 4 8 133

Expert Opinion

Classifier Decision

Miss Rate: 1.1% False Alarm Rate: 18.9%

ResultsLake Year Days Detected Total Days % Days

Crystal Bog 06 33 43 76.74

Ormajarvi 06 11 12 91.67

Sparkling Lake 04 120 168 71.43

Sparkling Lake 06 28 45 62.22

Sunapee 06 40 60 66.67

Taihu 06 28 32 87.50

Trout Bog 04 152 183 83.06

Trout Bog 05 67 94 71.28

Trout Russ 06 18 32 56.25

TOTAL 497 669 74.29

The Future

We now have a method to objectively detect the presence of the midnight surge.

Our new question is: Why is there a surge on some days, and not on other days?

To answer this question, we have to look at other types of data readings, for example: water temperature; wind speed; PAR etc.

Preprocessing

Feature Extraction

Classification

Verification

Correlation

The Future Search for hidden correlations between other data types

and our feature to formulate/validate an hypothesis.

153 154 155 156 157 158 159 1605

10

15

20

153 154 155 156 157 158 159 1608.5

9

9.5

10

10.5

Sparkling Lake 04

Dis

solv

ed

Oxy

gen

Con

cent

ratio

nW

ater

T

empe

ratu

res

Therm

istor S

ensor D

epth

Questions?