Post on 16-Feb-2017
Quality Control and Measurement Uncertainty
Ympäristötiedosta palveluihin – seminaari, 23.9.2015
Mauno Rönkkö (UEF), Okko Kauhanen (UEF), Markus Stocker (UEF), Mikko Kolehmainen (UEF), Harri Hytönen (Vaisala), Olli Ojanperä (Vaisala), Esko Juuso (UO), Markku Ohenoja (UO), Ville Kotovirta (VTT), Maija Ojanen (VTT), Petri Koponen (VTT), Teemu Näykki (SYKE), Jari Koskiaho (SYKE), Niina Kotamäki (SYKE), Jari Silander (SYKE), Hanna Huitu (LUKE), Jussi Nikander (LUKE),…
24.9.2015 Mauno Rönkkö 2
Contents
1. Sources for Uncertainty in Environmental Monitoring
2. Quality Flagging by Nordic Meteorological Institutes
3. Extended Quality Flagging Scheme to Environmental Data
4. Automatic Monitoring – case Väänteenjoki
5. Water Quality Monitoring and MUkit
6. Conclusion
Sources for Uncertainty in
Environmental Monitoring
24.9.2015 Mauno Rönkkö 3
1. Case #1: Incomplete Understanding [1/8]
24.9.2015 Mauno Rönkkö 4
http://www.mmea.fi
1. Case #2: Indirect Measurements [2/8]
•The Finnish Environmental Institute (SYKE) monitors total phosphorus of lakes and rivers in Finland.
•Currently, there is no device that measures total phosphorus of a lake.
•The amount of total phosphorus is (1) estimated based on the amount of suspended solids (2) which is estimated based on measured turbidity at (3) a given location on a lake/river.
24.9.2015 Mauno Rönkkö 5
http://wwwi3.ymparisto.fi/i3/sakylapyhajarvi/sakylapyhajarvi.htm
1. Case #3: Heterogeneous Measurement Methods [3/8]
24.9.2015 Mauno Rönkkö 6
http://www.vaisala.com/en/products/ automaticweatherstations/Pages/default.aspx
http://www.biltema.fi/fi/Toimisto---Tekniikka/Kellot-ja-Lampomittarit/Lampomittari/Langaton-saaasema-84086/
1. Case #4: Sampling [4/8]
24.9.2015 Mauno Rönkkö 7
Is this a good enough sample?
1. Case #5: Inconsistent Treatment of Measurement Errors [5/8]
24.9.2015 Mauno Rönkkö 8
ERRORS (random errors + systematic errors): -Devices are individuals -Measurements drift over time -Devices are positioned badly -Devices are used in non-optimal conditions -Measurement noise is too large for measured quantity -Person measuring affects the measurements -Environmental conditions vary and affect measurements -Several different measurement devices used to get a dataset -Spatially and temporally different measurements are not preprocessed -Wrong devices are used in a specific measurement method -Different measurement methods with different devices are used -No calibration -Standards and not used or they are used improperly -Measurement data is treated with wrong statistical methods …
1. Case #5: Inconsistent Treatment of Measurement Errors [6/8]
24.9.2015 Mauno Rönkkö 9
ERRORS (the less recognized but most important): -Non-synchronized measurement clocks -Cognitive pitfalls
1. Case #6: Semantically Inconsistent Interoperability [7/8]
24.9.2015 Mauno Rönkkö 10
1. Case #7: Poorly Understood Uncertainties and Validities [8/8]
24.9.2015 Mauno Rönkkö 11
Quality Flagging by
Nordic Meteorological Institutes
24.9.2015 Mauno Rönkkö 12
F. Vejen (ed), C. Jacobsson, U. Fredriksson, M. Moe, L. Andresen, E. Hellsten, P. Rissanen, T. Palsdottir, and T. Arason. Quality Control of Meteorological Observa- tions. Automatic Methods Used in the Nordic Countries. Climate Report 8/2002, Norwegian Meteorological Institute, 2002.
2. Why bother?
24.9.2015 Mauno Rönkkö 13
THIS IS WATER CONSUMPTION!?
2. Why bother?
24.9.2015 Mauno Rönkkö 14
IS THIS WATER CONSUMPTION!?
2. Why bother?
24.9.2015 Mauno Rönkkö 15
THIS IS WATER CONSUMPTION!
2. Data to information
24.9.2015 Mauno Rönkkö 16
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
2. Quality checks
24.9.2015 Mauno Rönkkö 17
QC0 QC1
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
2. Quality checks
24.9.2015 Mauno Rönkkö 18
QC1
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range, step and
consistency
2. Quality checks
24.9.2015 Mauno Rönkkö 19
QC2
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency
QC1: real-time quality control on individual data
points using statistical methods, including missing
and expected values
2. Quality checks
24.9.2015 Mauno Rönkkö 20
HQC
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency QC1: real-time quality
control on individual data points using statistical
methods, including missing and expected
values
QC2: non-real-time quality control on data sets
including spatial and temporal analysis with
corrective computations
2. Quality checks
24.9.2015 Mauno Rönkkö 21
Measurement device
Server
Data storage
Data analysis and refinement
Human operator
QC0: real-time quality control on
individual data points about range,
step and consistency QC1: real-time quality
control on individual data points using statistical
methods, including missing and expected
values
QC2: non-real-time quality control on data sets
including temporal and spatial analysis with
corrective computations
HQC: non-real-time quality inspection
including visualization; the final word
2. Measurement Data
24.9.2015 Mauno Rönkkö 22
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42
time
humidity temperature
location
device-id
2. Measurement Data with a Quality Flag
24.9.2015 Mauno Rönkkö 23
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
time quality flag
humidity temperature
location
device-id
2. The Flag Values
24.9.2015 Mauno Rönkkö 24
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
time
quality flag
humidity temperature
location
device-id
C = 1000EHQC + 100EQC2
+ 10EQC1 + EQC0
2. Multiple Data Points
24.9.2015 Mauno Rönkkö 25
556 2014-09-27T09:00:30 62.8925, 27.678333 22.6 42 9330
C = 1000EHQC + 100EQC2
+ 10EQC1 + EQC0
556 2014-09-27T09:00:30 62.8925, 27.678333 15.3 55 4000
Extending the Quality Flagging Scheme
to Environmental Data
24.9.2015 Mauno Rönkkö 26
M. Rönkkö, O. Kauhanen, M. Stocker, H. Hytönen, V. Kotovirta, E. Juuso, M. Kolehmainen. Quality Control of Environmental Measurement Data with Quality Flagging. IFIP Advances in Information and Communication Technology, 2015, Volume 448, Environmental Software Systems. Infrastructures, Services and Applications, pages 343-350.
3. Generic Interpretation
24.9.2015 Mauno Rönkkö 27
Flag Original interpretation Generic interpretation
0 No check performed Value not checked
1 Observation is ok Approved value
2 Suspected small difference Suspicious value
3 Suspected big difference Anomalous value
4 Calculated value Corrected value
5 Interpolated value
Imputed value
6 (Not defined originally) Erroneous value
7 (Not defined originally) Frozen value
8 Missing value Missing value
9 Deleted value Deleted value
3. Example Service Architecture
24.9.2015 Mauno Rönkkö 28
3. Quality Control of Water Consumption Data
24.9.2015 Mauno Rönkkö 29
3. Quality Control of Water Consumption Data
24.9.2015 Mauno Rönkkö 30
QC1: measured and checked once a minute QC2: runs every 2 hours, used for spotting leaks and malfunctions HQC: done once a month, aims at resolving frozen data values
Automatic Monitoring - case Väänteenjoki
24.9.2015 Mauno Rönkkö 31
24.9.2015 Mauno Rönkkö 32
4. Automatic monitoring – case Väänteenjoki • The challenge in automating water quality monitoring is that the measurement data
have not only significant seasonal variation, but also erroneous values
• Thus, without proper quality control and reliable uncertainty estimation, the data has little value
• As a solution, we have implemented a computation service based on an Enterprise Service Bus Architecture. The service provides means for online quality control and integration of uncertainty estimation
• Case study: In the Karjaanjoki River Basin the Väänteenjoki site equipped with an OBS3+ turbidity sensor (Campbell Scientific inc.)
• OBS3+ sensor emits a near-infrared
light into the water, measures the light that scatters back from the suspended particles, and transforms this information into turbidity values in Nephelometric Turbidity Units (NTU)
24.9.2015 Mauno Rönkkö 33
4. Automatic monitoring – case Väänteenjoki • The “raw” turbidity recorded by the OBS3+ sensor had to be calibrated against the
turbidity determined from water samples taken near the sensor • Calibration equation was determined by linear regression between the values of the
water samples and the simultaneous values recorded by the sensor • Then, because turbidity does not denote the content of substance in water, the
calibrated turbidity data had to be converted to concentrations of susp.solids and total P
• We have implemented a computational service that automates and integrates uncertainty estimation to the sequence of operations
Water Quality Monitoring and MUkit
24.9.2015 Mauno Rönkkö 34
24.9.2015 Mauno Rönkkö 35
24.9.2015 Mauno Rönkkö 36
24.9.2015 Mauno Rönkkö 37
24.9.2015 Mauno Rönkkö 38
Conclusion
24.9.2015 Mauno Rönkkö 39
5. Conclusion [1/2]
•What you cannot measure, you cannot control.
•Sources for uncertainties
Incomplete understanding, Indirect measurements, Heterogeneous measurement methods, Sampling, Inconsistent treatment of measurement errors, Semantically inconsistent interoperability, Poorly understood uncertainties and validities
•Quality Flagging
– scheme by the Nordic Meteorological Institutes
– Quality checks at various stages; Real-time and non-real-time checks
•Quality Flagging of Environmental Data
– Generic interpretation
– ESB based architecture
24.9.2015 Mauno Rönkkö 40
5. Conclusion [2/2]
•Automatic monitoring – case Väänteenjoki
– proper quality control and reliable uncertainty estimation required
– implemented a computation service based on an ESB
•Water quality monitoring and MUkit
– Based on the Nordtest TR 537 guide and on the standard SFS-EN ISO 11352
– Automated turbidity measuring system for ”real-time” uncertainty estimation using AutoMUkit
– Several international publications available!
24.9.2015 Mauno Rönkkö 41