Consumer Centered Calibration End Use Water Monitoring

consumer centered calibration in end-use water monitoring

eric c. larson | eclarson.comAssistant Professor Computer Science and Engineering

on average, a consumer can save 15-25%

lake mead 1983

lake mead 2011

we are using water faster than it is being replenished

Pacific Institute for Studies in Development, Environment, and Security, 2011

$2,994.83

more awareness

Fabricated"Unit

16

Water"Meter"Design

15

TurbineInsert

Thermistor

Flow

image: LBNL

metersflow rate fixture flowinline waterFabricated"Unit

16

metersflow rate fixture flowinline water

water pressure

pressure sensor

Fabricated"Unit

16

metersflow rate fixture flowinline water

water pressure

pressure sensor

machine learning

estimated

Fabricated"Unit

16

HydroSense

• central sensing point

• easy to install

• low cost

• can observe every fixture

HydroSense


• easy to install

• low cost


HydroSenseBelkinEcho


• easy to install

• low cost


HydroSense

40#

50#

60#

70#

80#

Cold Line Pressure (Hose Spigot)

0 9 4.5

time (s)

psi

BelkinEcho


• easy to install

• low cost


HydroSense

40#

50#

60#

70#

80#

Cold Line Pressure (Hose Spigot)

0 9 4.5

time (s)

psi

open close

BelkinEcho

kitchen sink

upstairs toilet

downstairs toilet

kitchen sink

upstairs toilet

downstairs toilettotals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

= totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

kitchen sink

upstairs toilet

template matching

unknown eventdownstairs toilet

kitchen sink

upstairs toilet

template matching

unknown event

downstairs toilet

feasibility study

Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.

Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).

feasibility study

• 10 homes



feasibility study

• 10 homes• staged calibration



feasibility study

• 10 homes• staged calibration• ~98% accuracy



70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

initial study: staged events

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

initial study: staged events

kitchen sink kitchen sink

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

natural water use

how well does single point sensing work in a natural setting?

longitudinal evaluation

totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

data collection

Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.

totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

data collection

Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.

most comprehensive labeled dataset of hot and cold water ever collected

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

8AM 2 minutes

bathroom sink

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

8AM 2 minutes

toilet

bathroom sink

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

8AM 2 minutes


toilet

bathroom sink

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

template matching: 98% 74%

8AM 2 minutes


toilet

bathroom sink

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

template matching: 98% 74%10 fold cross validation

8AM 2 minutes


toilet

bathroom sink

natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

template matching: 98% 74%10 fold cross validation

35%

8AM 2 minutes

minimal

two step process: segmentation70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

standard deviation smoothing operation

pressure deltas local maxima (for adjustments/compound)

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

pressure drop min pressure

duration min local maxima

max derivative max amplitude

LPC frequency characteristics sink flow meter

two step process: features extraction

x

s1

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

pressure drop min pressure

duration min local maxima

max derivative max amplitude

LPC frequency characteristics sink flow meter

two step process: features extraction

x

s1

x

d2

x

d3

x

d4

x

d5x

d6

x

d7

machine learning overview

labeled

RF classifier

predicted fixture

machine learning overviewminimal set of labels for calibrationlabeled

RF classifier

predicted fixture


RF classifier

predicted fixture

one or two examples per fixture


RF classifier

predicted fixture


accuracy 40-60%


RF classifier

predicted fixture


accuracy 40-60%

need more labels!


RF classifier

predicted fixture


accuracy 40-60%

need more labels!but which ones?

active learning

labeled

classifier 1 classifier N…

rando

m for

est

active learning

labeled

classifier 1 classifier N

unlabeled

…

rando

m for

est

active learning

labeled


unlabeled

disagree?

…

rando

m for

est

active learning

labeled


unlabeled

disagree?

max margin of committee

…

rando

m for

est

need a labelmargin < 20%

active learning leveraging the homeowner


• select low confidence margin examples



• ask homeowner for label




AT&T LTEAT&T LTE 5:23 PM

did you just use water?

YesNo






YesNo


Fixture Selection

Do Not Disturb

Select Notification Times

OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher

Master Sink Select Temp.

Kitchen Sink Select Temp.

Master Shower Select Temp.

More

Half Bath Sink Select Temp.

simulating labels from homeowner



YesNo


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More





YesNo

• ask for two labels every other day


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More





YesNo


• one morning and one eveningAT&T LTEAT&T LTE 5:23 PM

Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More





YesNo


• one morning and one evening

• only from 8AM-9PMAT&T LTEAT&T LTE 5:23 PM

Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


10 15 20 25 30 35 40 45

0.65

0.7

0.75

0.8

0.85Co−Labeling in H1

Number of Labels

Valv

e Le

vel A

ccur

acy

of C

oLab

el−H

MM

Co−LabelingRandom Labeling

simulating labels from homeowneractive learning for H1 totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

fixtu

re p

redi

ctio

n ac

cura

cy

number of labeled examples

10 15 20 25 30 35 40 45

0.65

0.7

0.75

0.8


Number of Labels

Valv

e Le

vel A

ccur

acy

of C

oLab

el−H

MM

Co−LabelingRandom Labeling

simulating labels from homeowneractive learning for H1

mini

mal

traini

ng s

ettotals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

fixtu

re p

redi

ctio

n ac

cura

cy


10 15 20 25 30 35 40 45

0.65

0.7

0.75

0.8


Number of Labels

Valv

e Le

vel A

ccur

acy

of C

oLab

el−H

MM

Co−LabelingRandom Labelingchosen labels random labeling

iteration 1

iteration 3

iteration 5

iteration 10

simulating labels from homeowneractive learning for H1

mini

mal

traini

ng s

ettotals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

fixtu

re p

redi

ctio

n ac

cura

cy


totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

60%

70%

80%

90%

100%

0 1 2 3 4 5 6 7 8 9 10 11 12 13

valve fixture category

error bars=std err.

active learning iteration

accu

racy

133

followed by the toilet. This advocates the importance of using the DBN-VE as a baseline classifier,

because co-labeling only marginally increases the diversity of class examples. Even so, a shower example

is typically asked for in the first two iterations, but washing machines and dishwashers are not asked for

until typically the fifth or sixth iteration. This is not a problem, however, because the rule based classifier

and DBN-VE are able to leverage prior knowledge in classifying these fixtures and appliances.

A final investigation of the confusions reveals that the system, after 10 iterations of co-labeling, is

highly accurate among each fixture, although temperature confusions still exist (Figure 8-20). The most

common confusion is the secondary bathroom shower for the master bathroom shower.

Figure 8-20. The final confusion matrix for the CoLabel-DBN algorithm

For comparison to the other algorithms, I also show the improvement in the across fixture accuracy at

the valve, lumped fixture, and fixture category level, shown in Figure 8-21.

-32974

8.5

1.5

-328

6.1

72

1.1

0.8

5.0

0.7

1.5

0.6

-7180

18

92

0.6

5.4

2.6

4.9

3.4

2.3

3.5

0.9

5.1

-7211

18

0.9

92

1.4

4.9

0.7

4.4

1.4

1.4

7.3

1.0

0.7

3.2

-72053

2.1 -1651

4.2

67

0.6

2.4

0.8

31

2.5

5.2

-5253

1.2

5.0

0.6

18

4.2

88

1.8

7.5

1.5

1.6

5.7

1.0

-5430

1.2

0.5

5.3

4.6

10

1.8

89

1.7

1.7

1.4

0.7

3.2

-2398

2.9

0.6

81

12

-2433

0.8

94

0.6

8.1

-133

6.7

68

-259

2.8

42

-1628

1.9

2.9

0.5

25

0.8

95

0.7 -1625

2.2

3.1

10

1.3

96

6.6 -27290

-316

1.1

0.7

59

18 -308

0.9

0.9

15

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Dishwasher close,1

open,2

KitchenSink close,3

open,4

M.BathroomShower close,5

open,6

M.BathroomSink close,7

open,8

M.BathroomToilet close,9

open,10

S.BathroomShower close,11

open,12

S.BathroomSink close,13

open,14

S.BathroomToilet open,15

WashingMachine close,16

open,17

133

followed by the toilet. This advocates the importance of using the DBN-VE as a baseline classifier,

because co-labeling only marginally increases the diversity of class examples. Even so, a shower example

is typically asked for in the first two iterations, but washing machines and dishwashers are not asked for

until typically the fifth or sixth iteration. This is not a problem, however, because the rule based classifier

and DBN-VE are able to leverage prior knowledge in classifying these fixtures and appliances.

A final investigation of the confusions reveals that the system, after 10 iterations of co-labeling, is

highly accurate among each fixture, although temperature confusions still exist (Figure 8-20). The most

common confusion is the secondary bathroom shower for the master bathroom shower.

Figure 8-20. The final confusion matrix for the CoLabel-DBN algorithm

For comparison to the other algorithms, I also show the improvement in the across fixture accuracy at

the valve, lumped fixture, and fixture category level, shown in Figure 8-21.

-32974

8.5

1.5

-328

6.1

72

1.1

0.8

5.0

0.7

1.5

0.6

-7180

18

92

0.6

5.4

2.6

4.9

3.4

2.3

3.5

0.9

5.1

-7211

18

0.9

92

1.4

4.9

0.7

4.4

1.4

1.4

7.3

1.0

0.7

3.2

-72053

2.1 -1651

4.2

67

0.6

2.4

0.8

31

2.5

5.2

-5253

1.2

5.0

0.6

18

4.2

88

1.8

7.5

1.5

1.6

5.7

1.0

-5430

1.2

0.5

5.3

4.6

10

1.8

89

1.7

1.7

1.4

0.7

3.2

-2398

2.9

0.6

81

12

-2433

0.8

94

0.6

8.1

-133

6.7

68

-259

2.8

42

-1628

1.9

2.9

0.5

25

0.8

95

0.7 -1625

2.2

3.1

10

1.3

96

6.6 -27290

-316

1.1

0.7

59

18 -308

0.9

0.9

15

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Dishwasher close,1

open,2

KitchenSink close,3

open,4

M.BathroomShower close,5

open,6

M.BathroomSink close,7

open,8

M.BathroomToilet close,9

open,10

S.BathroomShower close,11

open,12

S.BathroomSink close,13

open,14

S.BathroomToilet open,15

WashingMachine close,16

open,17

dishwasher laundry

dishwasher

laundry

implications for NILM

implications for NILMAT&T LTEAT&T LTE 5:23 PM

Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More



Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


did you plug in laptop?


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


did you plug in laptop?did you recently start the oven?


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


did you plug in laptop?did you recently start the oven?are you watching TV?


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


did you plug in laptop?did you recently start the oven?are you watching TV?

-can potentially start using more home specific features

limitations

limitationsresponsive cloud architecture

limitationsresponsive cloud architecturemultiple people in a home

limitationsresponsive cloud architecturemultiple people in a hometrust in user as “oracle”


Fixture Selection

Do Not Disturb


OFF

Master Toilet

Recently Used:

Half Bath Toilet

Dishwasher




More


consumer centered calibration in end-use water monitoring

eric c. larson | eclarson.comAssistant Professor Computer Science and Engineering

Thank You!

eclarson.com [email protected] @ec_larson

acknowledgements Belkin, Inc.

Shwetak Patel Jon Froehlich Sidhant Gupta

Les Atlas

Consumer Centered Calibration End Use Water Monitoring

Engineering

Transcript of Consumer Centered Calibration End Use Water Monitoring