Dynamic Time Warping improves sewer flow...
Transcript of Dynamic Time Warping improves sewer flow...
Dynamic Time Warping improves sewer flow monitoring
David Dürrenmatt Dario Del Giudice
Jörg Rieckermann
What’s the matter?
Quality control of discharge measurements
Problem • Flow meters show considerable errors under normal operating
conditions in sewers.
• Discharge − Ultrasonic flow meters: 10% − Tracer dilution methods: 6% to 16% − Venturi: 12% to 20%
• Ultrasonic velocity sensors
− Single-point: 14 - 18% − Multi-point: 4 - 5%
Hoppe (2009), Smits (2008)
Quality control of discharge measurements
• Manual
calibration in the lab or field is expensive.
• Usually only point calibration once per year or 3 months.
• During dry weather conditions!
Hoppe (2009), Smits (2008)
Quality control of discharge measurements Create independent information on flow velocities
Idea • Use “natural” tracers in wastewater to obtain independent
information on average flow velocities ⇒ Time shift of characteristic patterns between 2 measuring
locations A and B contains information on travel time θ.
A
Quality control of discharge measurements Create independent information on flow velocities
Idea • Use “natural” tracers in wastewater to obtain independent
information on average flow velocities ⇒ Time shift of characteristic patterns between 2 measuring
locations A and B contains information on travel time θ.
⇒ Length of the sewer section is obtained from map or field measurements.
θLtv ≈)(
Methods
Ideal plug-flow reactor Concept
t t
Δt
V 𝜃 𝑡 =𝑉
𝑄(𝑡)
A B
C
D
Equations: Packet A: 2𝛥𝑡 → Δ𝑡 𝑄1 + 𝑄2 = 𝑉 Packet B: 2𝛥𝑡 → Δ𝑡 𝑄2 + 𝑄3 = 𝑉 Packet C: 3𝛥𝑡 → Δ𝑡 𝑄3 + 𝑄4 + 𝑄5 = 𝑉 Packet D: 4𝛥𝑡 → Δ𝑡 𝑄4 + 𝑄5 + 𝑄6 + 𝑄7 = 𝑉 …
𝑄1 𝑄2 𝑄3 𝑄4 𝑄5 𝑄6 𝑄7
Ideal plug-flow reactor
Δ𝑡 𝑄1 + 𝑄2 = 𝑉 Δ𝑡 𝑄2 + 𝑄3 = 𝑉
Δ𝑡 𝑄3 + 𝑄4 + 𝑄5 = 𝑉 Δ𝑡 𝑄4 + 𝑄5 + 𝑄6 + 𝑄7 = 𝑉
Matrix notation:
𝑨 =
1 1 0 0 0 0 00 1 1 0 0 0 00 0 1 1 1 0 00 0 0 1 1 1 1
,
Q = 𝑄1,𝑄2, … ,𝑄7 𝑇 and b = 𝑉
Δ𝑡1, 1, 1, 1 𝑇
4 Equations:
𝑨𝑄 = 𝑏 with
How do we get the residence times of the water packets?
Dynamic Time Warping
• “Warps” two sequences non-linearly in the time domain so that the dissimilarity is minimized
• Was originally developed for speech recognition • Is a standard technique for non-linear pattern matching
• Has been used to successfully estimate flow distribution in hydraulic flow dividers at WWTPs.
Dürrenmatt (2011)
Dynamic Time Warping
(Keogh und Pazzani, 2000)
„Warping-Path“
Time series B
Time series A p
q
(p,q)=(8/10)
→ 𝜃𝐷𝑇𝐷 = 10 − 8 Δt = 2Δ𝑡 Δ𝑡
∑=kw
mnDBAd ,),(
Dynamic Time Warping
Conditions for the warping path: • Starts and ends in
opposite corners • continuous • Steps are restricted • Pattern appears first in
A, then in B.
B
A
Dynamic Time Warping
Added stochasticity to avoid local optima: 1. Add noise to
observations 2. Iterate computation of
warping paths
B
A
Dynamic Time Warping
Added stochasticity to avoid local optima: 1. Add noise to
observations 2. Iterate computation of
warping paths 3. Compute average
warping
B
A 𝜃𝐷𝑇𝐷 = 𝜃
?
… 𝜽𝑫𝑫𝑫 = 𝜽 ?
Illustration: Tracer experiment
Error: 𝐸 = 𝑉𝑡𝜇− 𝑉
𝑡𝑚≠ 0 if 𝑡𝜇 ≠ 𝑡𝑚
• The estimated travel time does not equal the true travel time in real systems (Dispersion, Reaction).
Numerical experiments
Numerical experiments
• Testing the method on virtual data to determine the field of application
• “Benchmark Simulation Environment” − Inflow generator (Discharge, Temperature) − Hydrodynamic heat transport model (Aquasim) − Sensor model (BSM 1, Class “A”)
A
B
T
t T
t
Results (1) Example
Time [s]
Results (1) Example
Results (1) Accuracy of DTW velocity estimates
Confirms the theorectical considerations.
Results (2) Dispersion and heat exchange
Results (3) Sensor response time and error
Results (4) Sampling frequency
Application
Real-world case study
• Testing the performance of 2 flow meters
• Measurement campaign: 2 weeks
• 2x Onset TMC6-HD thermistor w. HOBO logger • Accuracy: 0.25 °C • Resolution: 0.03 °C • T10/90: 30s (90%)
Results (5) Online analysis
Results (6) Offline analysis
This looks nice. But...
Discussion
• Pre-processing is important! High-pass filtering is better than normalization of the Temperature signals.
• Results chould be improved by using other or multiple tracers with near-conservative behaviour (e.g., Conductivity).
• Using a physically-based model for data analysis could also be promising.
Dürrenmatt, D.J., D. Del Giudice, J. Rieckermann et al., Dynamic time warping improves sewer flow monitoring (submitted to Water Research)
Discussion Comparison to Cross-correlation with sliding windows
Conclusions
Conclusions
• Dynamic time warping (DTW) can retrieve sewer flow velocities from online measurements of wastewater quality.
• DTW extracts travel times from the temporal shift between upstream and downstream patterns by computing a non-linear warping path which maximizes the similarity between both patterns.
• The method is very well suited for the conditions found in typical sewer systems. Errors are estimated to less than 7.5%.
• The simple set-up and low experimental costs for sensors make it a practicable approach to diagnose sewer flow monitoring devices.