Job Talk

enabling eco-feedback and out-of-clinic health sensingindirect

University of WashingtonEric C. Larson

UbiComp Lab

electrical engineering

computerscience and engineering

ubiquitous sensing

sensingdesiredquantity

outputdirect


outputdirect

calories burned=


outputdirect

auxiliaryquantity sensing

calories burned=


outputdirect

auxiliaryquantity sensing processing

calories burned=


outputdirect

auxiliaryquantity sensing processing

estimated

calories burned=

indirect sensing

indirect sensing• not exact


• calibration


• calibration

• low cost


• calibration

• low cost

• easier to deploy


• calibration

• low cost

• easier to deploy

• readily accepted

digital signal processingdigital signal processing

machine learningmachine learning

HCI mobile phone embedded references

image processing

time series

evol.comp.

ensemble/graphical


thermal imaging

CHI 2010ESPA 2012

facialanalysis IJAEC 2010

image fidelity

ICIP 2009JEI 2010

power harvesting UbiComp 2010

gas sensing Pervasive 2010

water sensing

UbiComp 2009Pervasive 2011

CHI 2012

cough sensing UbiComp 2011

lungfunction

UbiComp 2012DEV 2013

inter

actio

n &

imag

e an

alysis

sust

ainab

ility

healt

h

digital signal processingdigital signal processing

machine learningmachine learning


image processin

g

time series

evol.comp.

ensemblegraphic

al


thermal imaging

CHI 2010ESPA 2012

facialanalysis IJAEC 2010

image fidelity

ICIP 2009JEI 2010

power harvesting UbiComp 2010

gas sensing Pervasive 2010

UbiComp 2009Pervasive 2011

CHI 2012

cough sensing UbiComp 2011

UbiComp 2012DEV 2013

inter

actio

n im

age

analy

sissu

stain

abilit

yhe

alth

water sensing

lungfunction

water sensing

lungfunction

futureresearch

ubicomplabU N I V E R S I T Y O F W A S H I N G T O N

Professor Shwetak N. Patel

Healthsensing

Novelinteractiontechniques

Low-power sensing

Sustainabilitysensing

AWARDS

Professor Shwetak N. Patel awarded:

STRONG INDUSTRY COLLABORATIONS

IN THE NEWS

Computer Science & Engineering

Electrical Engineering

design + use +build

17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards

Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”

MacArthur ‘Genius’ AwardSloan Research Fellow

TR-35 Top innovatorSeattle’s Top Innovator

MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American

CNNThe EconomistWiredGizmodoEngadgetNew Scientist

ElectriSense InGen

SqueezeBlock

uTouch

LightWave

Humantenna

GripSense

HeatWave

WalkType

SNUPI

Static E-Field

CoughSense

SpiroSmart

ElderCare

Lullaby

MobilityRehabilitation

GasSense

HydroSense

InnerSol

water sensing

lungfunction

how can indirect sensing and machine learning be used for sustainability?

we are using water faster than it is being replenished

Pacific Institute for Studies in Development, Environment, and Security, 2011

$2,994.83

water usage is vastly misunderstood

eco-feedback

Geographic Comparisons Dashboards

Metaphorical Unit Designs Recommendations

eco-feedback in electricity

0%

5%

10%

15%

20%

1 2 3 4 5 Untitled 1

20%

12%9.2%8.4%

6.8%

3.8%

Enhanced Billing

Web Based

Daily Feedback

Realtime Feedback

Appliance Level + Personalized

Feedback

Annu

al %

Sav

ings

Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al.>20% reduction: Gardner et al. (2008) and Laitner et al. (2009)

Appliance Level


0%

5%

10%

15%

20%


20%

12%9.2%8.4%

6.8%

3.8%

Enhanced Billing

Web Based

Daily Feedback

Realtime Feedback


Feedback

Annu

al %

Sav

ings


Appliance Level


aggregate

0%

5%

10%

15%

20%


20%

12%9.2%8.4%

6.8%

3.8%

Enhanced Billing

Web Based

Daily Feedback

Realtime Feedback


Feedback

Annu

al %

Sav

ings


Appliance Level


aggregate

disaggregated

Courtesy: Belkin, Inc.

metersflow rate fixture flowinline water


waterpressure

pressuresensor


waterpressure

pressuresensor

machine learning

estimated

HydroSense

• single sensor

HydroSense

• single sensor

• easy to install

HydroSense

• single sensor

• easy to install

• low cost

HydroSense

• single sensor

• easy to install

• low cost

• can observe every fixture

HydroSense

HydroSense

40#

50#

60#

70#

80#

Cold Line Pressure (Hose Spigot)

0 9 4.5

time (s)

psi

HydroSense

40#

50#

60#

70#

80#


0 9 4.5

time (s)

psi

HydroSense

open close

40#

50#

60#

70#

80#


0 9 4.5

time (s)

psi

HydroSense

open close

kitchen sink

40#

50#

60#

70#

80#


0 9 4.5

time (s)

psi

HydroSense

open close

kitchen sink

upstairs toilet

40#

50#

60#

70#

80#


0 9 4.5

time (s)

psi

HydroSense

open close

kitchen sink

downstairs toilet

upstairs toilet

kitchen sink

upstairs toilet

downstairs toilet

template matching

unknown event

initial study

Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.

Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).

initial study

• 10 homes



initial study

• 10 homes

• staged calibration



initial study

• 10 homes

• staged calibration

• ~98% accuracy at identifying fixtures



70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

initial study: staged events

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

initial study: staged events

kitchen sink kitchen sink

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

natural water use

how well does HydroSense work in a natural setting?

longitudinal evaluation

totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

data collection

Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.

totals

days 33 33 30 27 33 156

events 2374 3075 4754 2499 2578 14,960

events/day 71.9 93.2 158.5 92.6 78.1 95.9

compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%

data collection

Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.

most comprehensive labeled dataset of hot and cold water ever collected

labeled natural water usage

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)


toilet

bathroom sink


70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)


toilet

bathroom sink


70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

template matching: 98% 70%


toilet

bathroom sink


70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

template matching: 98% 70%10 fold cross validation

a new approach

a new approach

templates feature vectors

a new approach


matching statistical approach

a new approach


matching statistical approach

minimal training

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

feature vectors

70

50

30

pres

sure

(psi)

feature vectors

70

50

30

pres

sure

(psi)

10 psi

feature vectors

70

50

30

pres

sure

(psi)

10 psi7.32 psi

feature vectors

70

50

30

pres

sure

(psi)

10 psi7.32 psi

15 Hz

feature vectors

70

50

30

pres

sure

(psi)

10 psi7.32 psi

15 Hz

200 ms

feature vectors

70

50

30

pres

sure

(psi)

10 psi

7.32 psi

15 Hz

200 ms

feature vectors

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

feature vectors

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

feature vectors: sequence

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

statistical model

observed

hidden

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

statistical model

p(x,y) =TY

t=1

p(yt|yt�1)| {z }transition

p(xt|yt)| {z }emission

observed

hidden

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

supervised training

p(x,y) =TY

t=1


p(xt|yt)| {z }emission

observed

observed

calibration, per homea few days of training labels

calibration, per homea few days of training labels

remainder of data is test set

fixtu

re le

vel a

ccur

acy

(%)

training amount (days)

0 2 4 6 8 10 12 14 1640

50

60

70

80

90

100

fixtu

re le

vel a

ccur

acy

(%)


0 2 4 6 8 10 12 14 1640

50

60

70

80

90

100

fixtu

re le

vel a

ccur

acy

(%)


ideally

0 2 4 6 8 10 12 14 1640

50

60

70

80

90

100

fixtu

re le

vel a

ccur

acy

(%)


ideally

error bars = ±�error bars

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

p(x,y) =TY

t=1


p(xLt |yt)| {z }

emission

p(xUt )| {z }

unlabeled

semi-supervised training

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

semi-supervised training

p(x,y) =TY

t=1


p(xLt |yt)| {z }

emission

p(xUt )| {z }

unlabeled

40

50

60

70

80

90

100

0 2 4 6 8 10 12 14 16

fixtu

re le

vel a

ccur

acy

(%)



y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6

decoding: add pairing

1

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6


11111

y1 y2 y3 y4 y5 y6

x1 x2 x3 x4 x5 x6


40

50

60

70

80

90

100

0 2 4 6 8 10 12 14 16

fixtu

re le

vel a

ccur

acy

(%)



Currently Using 0.00 GPM



Healthsensing


Low-power sensing


AWARDS



IN THE NEWS



design + use +build







ElectriSense InGen

SqueezeBlock

uTouch

LightWave

Humantenna

GripSense

HeatWave

WalkType

SNUPI

Static E-Field

CoughSense

SpiroSmart

ElderCare

Lullaby


GasSense

HydroSense

InnerSol

HydroSense

• low cost• easily installed• accurate• quickly calibrated• potential for high impact

futureresearch



Healthsensing


Low-power sensing


AWARDS



IN THE NEWS



design + use +build







ElectriSense InGen

SqueezeBlock

uTouch

LightWave

Humantenna

GripSense

HeatWave

WalkType

SNUPI

Static E-Field

CoughSense

SpiroSmart

ElderCare

Lullaby


GasSense

HydroSense

InnerSol

water sensing

lungfunction

how can indirect sensing and machine learning be used for health?

SpiroSmart

a smartphone based spirometer that leverages on-device microphone to help you keep track of your lung function.

SpiroSmart

a smartphone based spirometer that leverages on-device microphone to help you keep track of your lung function.lung function

spirometer

spirometer

device that measures amount of air inhaled and

exhaled.

lung function

asthmaCOPDcystic fibrosis

evaluates severity of pulmonary impairments

using a spirometer

flow

volume

volum

e

time

volume-time graphvo

lume

time

volume-time graphvo

lume

time

FEV1

FVC

FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity

volume-time graphvo

lume

time1 sec.

FEV1

FVC


volume-time graphvo

lume

time1 sec.

FEV1

FVCFEV1% = FEV1/FVC



FEV1% = FEV1/FVC


FEV1% = FEV1/FVC

> 80% healthy60 - 79% mild40 - 59% moderate

< 40% severe

flow-volume graphflo

w

volume

flow

volumeFEV1 FVC

1 sec.

PEF

PEF: Peak Expiratory FlowFEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity

flow-volume graph

flow

volume

normal

flow-volume graph

flow

volume

normalobstructive

flow-volume graph

obstructive diseases

resistance in air path leads to reduced air flow

restrictive diseases

lungs are unable to pump enough air and pressure

flow-volume graphFlo

w

Volume

normalobstructive

flow-volume graphFlo

w

Volume

normal

restrictiveobstructive

clinical spirometry

home spirometry

home spirometry

faster detectionrapid recovery

trending

home spirometry

high cost barrierpatient compliance

less coachinglimited integration

challenges with

flow ratevolume

lung functionairflowsensor

flow ratevolume


soundpressure microphone

flow ratevolume


soundpressure microphone processing

estimated

SpiroSmart

availabilitycostportabilitymore effective coaching interfaceintegrated uploading

Using SpiroSmart

Using SpiroSmart

]

Using SpiroSmart

initial study design

x 3

x 3

need for attachments

mouthpiece

study design

x 3

x 3

+

study design

x 3 x 3 x 3

+ +

x 3

x 3

dataset

+ + +

participants 52duration 45 minutes

first session 29

abnormals 12

revisits 10

dataset

+ + +

audio

flow features

audio

flow features

measuresregression

audio

flow features

measuresregression

FEV1FVCPEF

audio

flow features

measuresregression

curveregression

FEV1FVCPEF

0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measuresregression

curveregression

FEV1FVCPEF

0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measuresregression

curveregression

lung functionFEV1FVCPEF

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking

filtersource output

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking

filtersource outputestimated

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking0 1 2 3 4 5 6 7

−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

auto-regressive estimate


0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

auto-regressive estimate


lung function

audio

flow features

measuresregression

curveregression

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

measuresregression

ground truth

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measuresregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

ground truth

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measuresregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

PEF featuresground truth

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measuresregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35


feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measuresregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35

3.2

FEV1 features


feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

measuresregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e7.1

0.330.35

3.2

0.120.17

FEV1 features


feature 1

feature 2

measuresregression

FEV1 features PEF features

measuresregression

FEV1 features

bagged decision tree

PEF features


measuresregression

FEV1 features


output

PEF features


output

resultsmeasuresregression

Mea

n %

Erro

rLower is better


FEV1 FVC FEV1% PEF0

2

4

6

8

10

Mea

n %

Erro

rLower is better


FEV1 FVC FEV1% PEF0

2

4

6

8

10

No PersonalizationPersonalization

Mea

n %

Erro

rLower is better


general model = 8.8% error

ATS criteria = 5-7%personal model = 5.1% error


general model = 8.8% error

ATS criteria = 5-7%

norm

al

attachments have no effect

personal model = 5.1% error

lung function

audio

flow features

measuresregression

curveregression

curveregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature 2

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

curveregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature 2

curveoutput

curveregression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

curveregression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

CRF

curveregression


−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

CRF

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

example curvescurveregression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

can spirosmart curves be used for diagnosis?

survey

• 10 subjects curves

survey

• 5 pulmonologists


survey

• 5 pulmonologists


• unaware if from SpiroSmart / spirometer

survey

survey

results

normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive

restrictive

inadequate

identical32 / 50

results

normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive

restrictive

inadequate

identical

one off

32 / 50

5 / 18

FDA study underway

• part a: head to head clinical test

FDA study underway

• part a: head to head clinical test

• part b: home spirometry

FDA study underway

futureresearch



Healthsensing


Low-power sensing


AWARDS



IN THE NEWS



design + use +build







ElectriSense InGen

SqueezeBlock

uTouch

LightWave

Humantenna

GripSense

HeatWave

WalkType

SNUPI

Static E-Field

CoughSense

SpiroSmart

ElderCare

Lullaby


GasSense

HydroSense

InnerSol

water sensing

lungfunction

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction

activity detectionelder care

daily activity logscongestive heart failure

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction



health markers via phoneblood pressurepulse oximetrystress

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction




opportunistic sensingpain managementdetecting circulation

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction




opportunistic sensingpain managementdetecting circulation

developing world

summary

• sustainable water use, eco-feedback

• lung function via mobile phone

• future work in high impact areas

enabling eco-feedback and out-of-clinic health sensing

indirect

University of WashingtonEric C. Larson

UbiComp Lab


computerscience and engineering

ubiquitous sensing

eclarson.com [email protected]@ec_larson

Thank You!

thermal imaging

facialanalysis

image fidelity

power harvesting

gas sensing

cough sensing

inte

ract

ion

& im

age

anal

ysis

sust

aina

bilit

yhe

alth

water sensing

lungfunction

mailto:[email protected]


enabling eco-feedback and out-of-clinic health sensingindirect

University of Washington

Eric C. LarsonPhD Candidate in School of Electrical and Computer Engineering

UbiComp Lab


computerscience

ubiquitous sensing

eclarson.com [email protected]@ec_larson

acknowledgments:Jon FroehlichLeah FindlaterElliot SabaEric SwansonTim CampbellGabe CohnMayank Goel

TienJui LeeSidhant GuptaJosh PetersonConor HaggertyJeff BeorseShwetak PatelLes AtlasJames FogartyJeff BilmesMargaret Rosenfeld



water tower water tower

thermal expansion

tank

hose spigot

utility water meter

pressure regulator

laundry

bathroom 1

hot water heater bathroom 2

dishwasher

incoming cold water from supply line

kitchen

bath%

toilet&

shower'

kitchen(sink(

features: single instance

stable pressure (psi)

max

am

plitu

de


stable pressure (psi)

max

am

plitu

de

Kitchen(Sink(Basement(Sink(

Bathroom(Sink(

Basement(

Shower(

((((((((((((((((((((Upstairs(S

hower(

Upstairs(Bath(


42#

44#

46#

48#

50#

0# 5# 10# 15# 20# 25# 30# 35# 40# 45#

raw pressure (psi)

44"

46"

48"smoothed pressure

(psi)

bandpass derivative

(psi/s)

!4#

0#

4#

detrended

derivative

bandpass derivative

const-Q cepstrum

signa

l tra

nsfo

rms


frequency

42#

44#

46#

48#

50#

0# 5# 10# 15# 20# 25# 30# 35# 40# 45#

raw pressure (psi)

*10#

10#

30#cepstral

amplitude

index

detrended

derivative

bandpass derivative

const-Q cepstrum

signa

l tra

nsfo

rms



time

frequ

ency


]]

time

frequ

ency

]

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

feature extraction:sequence

70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

10 sec


70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

10 sec compound


70

50

30

pres

sure

(psi)

70

50

30

pres

sure

(psi)

10 sec compound time of day


accuracy

fixture levele.g., upstairs bathroom faucet

accuracy


accuracy

fixture50

60

70

80

90

100

92.2


accuracy

fixture50

60

70

80

90

100

92.2

what does this tell us?

10 fold cross validation

cycles

cyclesheated dry

cyclesheated dry

washing dishes

~7:30AM

filling coffee pot

coffee pot running

~7:30AM

bathroom usage

bathroom usage

leaky flapper valve

flow features

0 1 2 3 4−1

0

1

0 1 2 3 4−0.5

0

0.5

Amplitude

0 1 2 3 4−1

0

1

time(s)

flow features

0 1 2 3 4−1

0

1

0 1 2 3 4−0.5

0

0.5

Amplitude

0 1 2 3 4−1

0

1

time(s)

−1 0 1 2 3 4 50

2

4

6

8

time(s)flow

(L/s)

flow features

0 1 2 3 4−1

0

1

0 1 2 3 4−0.5

0

0.5

Amplitude

0 1 2 3 4−1

0

1

time(s)

~audio, pressure

−1 0 1 2 3 4 50

2

4

6

8

time(s)flow

(L/s)

flow features

0 1 2 3 4−1

0

1

0 1 2 3 4−0.5

0

0.5

Amplitude

0 1 2 3 4−1

0

1

time(s)

~audio, pressure

~pressure at lips

−1 0 1 2 3 4 50

2

4

6

8

time(s)flow

(L/s)

flow features

0 1 2 3 4−1

0

1

0 1 2 3 4−0.5

0

0.5

Amplitude

0 1 2 3 4−1

0

1

time(s)

~audio, pressure

~pressure at lips

~flow at lips

−1 0 1 2 3 4 50

2

4

6

8

time(s)flow

(L/s)

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature 2

linear chain regression

−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature 2

curveoutput


CRF

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e


CRF

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e


−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)



0 1 2 30

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

10

volume(L)flow(L/s)


0 1 2 30

2

4

6

8

volume(L)

flow(L/s)

0 1 2 30

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

10

volume(L)flow(L/s)

−1 0 1 2 3 40

2

4

6

8

10

volume(L)flow(L/s)

Job Talk

Technology

Transcript of Job Talk