Post on 31-Oct-2014
description
enabling eco-feedback and out-of-clinic health sensingindirect
University of WashingtonEric C. Larson
UbiComp Lab
electrical engineering
computerscience and engineering
ubiquitous sensing
sensingdesiredquantity
outputdirect
sensingdesiredquantity
outputdirect
calories burned=
sensingdesiredquantity
outputdirect
calories burned=
sensingdesiredquantity
outputdirect
calories burned=
sensingdesiredquantity
outputdirect
auxiliaryquantity sensing
calories burned=
sensingdesiredquantity
outputdirect
auxiliaryquantity sensing processing
calories burned=
sensingdesiredquantity
outputdirect
auxiliaryquantity sensing processing
estimated
calories burned=
indirect sensing
indirect sensing• not exact
indirect sensing• not exact
• calibration
indirect sensing• not exact
• calibration
• low cost
indirect sensing• not exact
• calibration
• low cost
• easier to deploy
indirect sensing• not exact
• calibration
• low cost
• easier to deploy
• readily accepted
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processing
time series
evol.comp.
ensemble/graphical
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
water sensing
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
lungfunction
UbiComp 2012DEV 2013
inter
actio
n &
imag
e an
alysis
sust
ainab
ility
healt
h
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processing
time series
evol.comp.
ensemble/graphical
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
water sensing
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
lungfunction
UbiComp 2012DEV 2013
inter
actio
n &
imag
e an
alysis
sust
ainab
ility
healt
h
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processing
time series
evol.comp.
ensemble/graphical
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
water sensing
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
lungfunction
UbiComp 2012DEV 2013
inter
actio
n &
imag
e an
alysis
sust
ainab
ility
healt
h
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processing
time series
evol.comp.
ensemble/graphical
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
water sensing
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
lungfunction
UbiComp 2012DEV 2013
inter
actio
n &
imag
e an
alysis
sust
ainab
ility
healt
h
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processing
time series
evol.comp.
ensemble/graphical
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
water sensing
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
lungfunction
UbiComp 2012DEV 2013
inter
actio
n &
imag
e an
alysis
sust
ainab
ility
healt
h
digital signal processingdigital signal processing
machine learningmachine learning
HCI mobile phone embedded references
image processin
g
time series
evol.comp.
ensemblegraphic
al
HCI mobile phone embedded references
thermal imaging
CHI 2010ESPA 2012
facialanalysis IJAEC 2010
image fidelity
ICIP 2009JEI 2010
power harvesting UbiComp 2010
gas sensing Pervasive 2010
UbiComp 2009Pervasive 2011
CHI 2012
cough sensing UbiComp 2011
UbiComp 2012DEV 2013
inter
actio
n im
age
analy
sissu
stain
abilit
yhe
alth
water sensing
lungfunction
water sensing
lungfunction
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
how can indirect sensing and machine learning be used for sustainability?
we are using water faster than it is being replenished
Pacific Institute for Studies in Development, Environment, and Security, 2011
we are using water faster than it is being replenished
Pacific Institute for Studies in Development, Environment, and Security, 2011
$2,994.83
water usage is vastly misunderstood
eco-feedback
Geographic Comparisons Dashboards
Metaphorical Unit Designs Recommendations
eco-feedback
Geographic Comparisons Dashboards
Metaphorical Unit Designs Recommendations
eco-feedback in electricity
0%
5%
10%
15%
20%
1 2 3 4 5 Untitled 1
20%
12%9.2%8.4%
6.8%
3.8%
Enhanced Billing
Web Based
Daily Feedback
Realtime Feedback
Appliance Level + Personalized
Feedback
Annu
al %
Sav
ings
Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al.>20% reduction: Gardner et al. (2008) and Laitner et al. (2009)
Appliance Level
eco-feedback in electricity
0%
5%
10%
15%
20%
1 2 3 4 5 Untitled 1
20%
12%9.2%8.4%
6.8%
3.8%
Enhanced Billing
Web Based
Daily Feedback
Realtime Feedback
Appliance Level + Personalized
Feedback
Annu
al %
Sav
ings
Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al.>20% reduction: Gardner et al. (2008) and Laitner et al. (2009)
Appliance Level
eco-feedback in electricity
aggregate
0%
5%
10%
15%
20%
1 2 3 4 5 Untitled 1
20%
12%9.2%8.4%
6.8%
3.8%
Enhanced Billing
Web Based
Daily Feedback
Realtime Feedback
Appliance Level + Personalized
Feedback
Annu
al %
Sav
ings
Based on 36 studies between 1995-2010. Summarized by Ehrhardt-Matinez et al.>20% reduction: Gardner et al. (2008) and Laitner et al. (2009)
Appliance Level
eco-feedback in electricity
aggregate
disaggregated
Courtesy: Belkin, Inc.
Courtesy: Belkin, Inc.
metersflow rate fixture flowinline water
metersflow rate fixture flowinline water
waterpressure
pressuresensor
metersflow rate fixture flowinline water
waterpressure
pressuresensor
metersflow rate fixture flowinline water
waterpressure
pressuresensor
machine learning
estimated
HydroSense
• single sensor
HydroSense
• single sensor
• easy to install
HydroSense
• single sensor
• easy to install
• low cost
HydroSense
• single sensor
• easy to install
• low cost
• can observe every fixture
HydroSense
HydroSense
HydroSense
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
open close
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
open close
kitchen sink
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
open close
kitchen sink
upstairs toilet
40#
50#
60#
70#
80#
Cold Line Pressure (Hose Spigot)
0 9 4.5
time (s)
psi
HydroSense
open close
kitchen sink
downstairs toilet
upstairs toilet
kitchen sink
upstairs toilet
downstairs toilet
template matching
unknown event
kitchen sink
upstairs toilet
downstairs toilet
template matching
unknown event
initial study
Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.
Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).
initial study
• 10 homes
Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.
Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).
initial study
• 10 homes
• staged calibration
Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.
Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).
initial study
• 10 homes
• staged calibration
• ~98% accuracy at identifying fixtures
Froehlich, J., Larson, E., Campbell, T., Haggerty, C., Fogarty, J., and Patel, S.N. HydroSense: infrastructure-mediated single-point sensing of whole-home water activity. Proceedings of the 11th ACM international conference on Ubiquitous computing, (2009), 235–244.
Larson, E., Froehlich, J., Campbell, T., et al. Disaggregated water sensing from a single, pressure- based sensor: An extended analysis of HydroSense using staged experiments. Pervasive and Mobile Computing, (2010).
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
initial study: staged events
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
initial study: staged events
kitchen sink kitchen sink
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
natural water use
how well does HydroSense work in a natural setting?
longitudinal evaluation
totals
days 33 33 30 27 33 156
events 2374 3075 4754 2499 2578 14,960
events/day 71.9 93.2 158.5 92.6 78.1 95.9
compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%
data collection
Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.
totals
days 33 33 30 27 33 156
events 2374 3075 4754 2499 2578 14,960
events/day 71.9 93.2 158.5 92.6 78.1 95.9
compound 22.2% 21.8% 16.6% 32% 21.3% 21.9%
data collection
Larson, E., Froehlich, J., Saba, E., et al. A Longitudinal Study of Pressure Sensing to Infer Real- World Water Usage Events in the Home. Pervasive Computing, Springer (2011), 50–69.
most comprehensive labeled dataset of hot and cold water ever collected
labeled natural water usage
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
kitchen sink kitchen sink
toilet
bathroom sink
labeled natural water usage
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
kitchen sink kitchen sink
toilet
bathroom sink
labeled natural water usage
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
template matching: 98% 70%
kitchen sink kitchen sink
toilet
bathroom sink
labeled natural water usage
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
template matching: 98% 70%10 fold cross validation
a new approach
a new approach
templates feature vectors
a new approach
templates feature vectors
matching statistical approach
a new approach
templates feature vectors
matching statistical approach
minimal training
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
10 psi
feature vectors
70
50
30
pres
sure
(psi)
10 psi7.32 psi
feature vectors
70
50
30
pres
sure
(psi)
10 psi7.32 psi
15 Hz
feature vectors
70
50
30
pres
sure
(psi)
10 psi7.32 psi
15 Hz
200 ms
feature vectors
70
50
30
pres
sure
(psi)
10 psi
7.32 psi
15 Hz
200 ms
feature vectors
70
50
30
pres
sure
(psi)
10 psi
7.32 psi
15 Hz
200 ms
feature vectors
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors: sequence
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature vectors: sequence
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
statistical model
observed
hidden
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
statistical model
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xt|yt)| {z }emission
observed
hidden
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
supervised training
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xt|yt)| {z }emission
observed
observed
calibration, per homea few days of training labels
calibration, per homea few days of training labels
remainder of data is test set
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
0 2 4 6 8 10 12 14 1640
50
60
70
80
90
100
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
0 2 4 6 8 10 12 14 1640
50
60
70
80
90
100
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
ideally
0 2 4 6 8 10 12 14 1640
50
60
70
80
90
100
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
ideally
error bars = ±�error bars
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xLt |yt)| {z }
emission
p(xUt )| {z }
unlabeled
semi-supervised training
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
semi-supervised training
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xLt |yt)| {z }
emission
p(xUt )| {z }
unlabeled
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
semi-supervised training
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xLt |yt)| {z }
emission
p(xUt )| {z }
unlabeled
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
semi-supervised training
p(x,y) =TY
t=1
p(yt|yt�1)| {z }transition
p(xLt |yt)| {z }
emission
p(xUt )| {z }
unlabeled
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
error bars = ±�error bars
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
error bars = ±�error bars
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
decoding: add pairing
1
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
decoding: add pairing
11111
y1 y2 y3 y4 y5 y6
x1 x2 x3 x4 x5 x6
decoding: add pairing
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
error bars = ±�error bars
40
50
60
70
80
90
100
0 2 4 6 8 10 12 14 16
fixtu
re le
vel a
ccur
acy
(%)
training amount (days)
error bars = ±�error bars
Currently Using 0.00 GPM
Currently Using 0.00 GPM
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
HydroSense
• low cost• easily installed• accurate• quickly calibrated• potential for high impact
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
how can indirect sensing and machine learning be used for health?
SpiroSmart
a smartphone based spirometer that leverages on-device microphone to help you keep track of your lung function.
SpiroSmart
a smartphone based spirometer that leverages on-device microphone to help you keep track of your lung function.lung function
spirometer
spirometer
device that measures amount of air inhaled and
exhaled.
lung function
asthmaCOPDcystic fibrosis
evaluates severity of pulmonary impairments
using a spirometer
flow
volume
volum
e
time
using a spirometer
flow
volume
volum
e
time
using a spirometer
flow
volume
volum
e
time
volume-time graphvo
lume
time
volume-time graphvo
lume
time
volume-time graphvo
lume
time
FEV1
FVC
FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
volume-time graphvo
lume
time1 sec.
FEV1
FVC
FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
volume-time graphvo
lume
time1 sec.
FEV1
FVCFEV1% = FEV1/FVC
FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
FEV1% = FEV1/FVC
FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
FEV1% = FEV1/FVC
> 80% healthy60 - 79% mild40 - 59% moderate
< 40% severe
flow-volume graphflo
w
volume
flow-volume graphflo
w
volume
flow
volumeFEV1 FVC
1 sec.
PEF
PEF: Peak Expiratory FlowFEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity
flow-volume graph
flow
volume
normal
flow-volume graph
flow
volume
normalobstructive
flow-volume graph
obstructive diseases
resistance in air path leads to reduced air flow
obstructive diseases
resistance in air path leads to reduced air flow
restrictive diseases
lungs are unable to pump enough air and pressure
restrictive diseases
lungs are unable to pump enough air and pressure
flow-volume graphFlo
w
Volume
normalobstructive
flow-volume graphFlo
w
Volume
normal
restrictiveobstructive
clinical spirometry
home spirometry
home spirometry
faster detectionrapid recovery
trending
home spirometry
high cost barrierpatient compliance
less coachinglimited integration
challenges with
flow ratevolume
lung functionairflowsensor
flow ratevolume
lung functionairflowsensor
soundpressure microphone
flow ratevolume
lung functionairflowsensor
soundpressure microphone processing
estimated
SpiroSmart
availabilitycostportabilitymore effective coaching interfaceintegrated uploading
Using SpiroSmart
Using SpiroSmart
Using SpiroSmart
]
Using SpiroSmart
]
Using SpiroSmart
]
Using SpiroSmart
initial study design
x 3
x 3
need for attachments
need for attachments
mouthpiece
sling
study design
x 3
x 3
+
study design
x 3 x 3 x 3
+ +
x 3
x 3
+
study design
x 3 x 3 x 3
+ +
x 3
x 3
+
study design
x 3 x 3 x 3
+ +
x 3
x 3
+
study design
x 3 x 3 x 3
+ +
x 3
x 3
+
study design
x 3 x 3 x 3
+ +
x 3
x 3
dataset
+ + +
participants 52duration 45 minutes
first session 29
abnormals 12
revisits 10
dataset
+ + +
participants 52duration 45 minutes
first session 29
abnormals 12
revisits 10
dataset
+ + +
participants 52duration 45 minutes
first session 29
abnormals 12
revisits 10
dataset
+ + +
participants 52duration 45 minutes
first session 29
abnormals 12
revisits 10
dataset
+ + +
participants 52duration 45 minutes
first session 29
abnormals 12
revisits 10
dataset
+ + +
audio
audio
flow features
audio
flow features
measuresregression
audio
flow features
measuresregression
FEV1FVCPEF
audio
flow features
measuresregression
curveregression
FEV1FVCPEF
0 1 2 3 40
5
10
15
Flow
(L/s
)
Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
0 1 2 3 40
5
10
15
Flow
(L/s
)Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
audio
flow features
measuresregression
curveregression
FEV1FVCPEF
0 1 2 3 40
5
10
15
Flow
(L/s
)
Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
0 1 2 3 40
5
10
15
Flow
(L/s
)Volume(L)
0 2 4 6 8 100
1
2
3
4
time(s)
Volu
me(
L)
audio
flow features
measuresregression
curveregression
lung functionFEV1FVCPEF
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
time(s)
frequency(Hz)
1 2 3 4 5 60
500
1000
1500
2000
2500
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
filtersource output
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
filtersource outputestimated
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking0 1 2 3 4 5 6 7
−1
−0.5
0
0.5
1lpc8raw
time(s)
amplitude
auto-regressive estimate
filtersource outputestimated
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
flow features
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
envelope detection
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1
time(s)
amplitude
resonance tracking
0 1 2 3 4 5 6 7−1
−0.5
0
0.5
1lpc8raw
time(s)
amplitude
auto-regressive estimate
filtersource outputestimated
lung function
audio
flow features
measuresregression
curveregression
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
measuresregression
ground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measuresregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
ground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measuresregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measuresregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measuresregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
3.2
FEV1 features
PEF featuresground truth
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
measuresregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e7.1
0.330.35
3.2
0.120.17
FEV1 features
PEF featuresground truth
feature 1
feature 2
measuresregression
FEV1 features PEF features
measuresregression
FEV1 features
bagged decision tree
PEF features
bagged decision tree
measuresregression
FEV1 features
bagged decision tree
output
PEF features
bagged decision tree
output
resultsmeasuresregression
Mea
n %
Erro
rLower is better
resultsmeasuresregression
FEV1 FVC FEV1% PEF0
2
4
6
8
10
Mea
n %
Erro
rLower is better
resultsmeasuresregression
FEV1 FVC FEV1% PEF0
2
4
6
8
10
Mea
n %
Erro
rLower is better
resultsmeasuresregression
FEV1 FVC FEV1% PEF0
2
4
6
8
10
No PersonalizationPersonalization
Mea
n %
Erro
rLower is better
resultsmeasuresregression
general model = 8.8% error
ATS criteria = 5-7%personal model = 5.1% error
resultsmeasuresregression
general model = 8.8% error
ATS criteria = 5-7%
norm
al
attachments have no effect
personal model = 5.1% error
lung function
audio
flow features
measuresregression
curveregression
curveregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature 2
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
curveregression
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature 2
curveoutput
curveregression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
curveregression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
CRF
curveregression
bagged decision tree
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
CRF
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
example curvescurveregression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
example curvescurveregression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
example curvescurveregression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
example curvescurveregression
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−2 0 2 4 60
5
10
15
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
volume(L)
flow(L/s)
can spirosmart curves be used for diagnosis?
survey
• 10 subjects curves
survey
• 5 pulmonologists
• 10 subjects curves
survey
• 5 pulmonologists
• 10 subjects curves
• unaware if from SpiroSmart / spirometer
survey
survey
survey
survey
survey
survey
results
normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive
restrictive
inadequate
identical32 / 50
results
normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive
restrictive
inadequate
identical
one off
32 / 50
5 / 18
results
normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive
restrictive
inadequate
identical
one off
32 / 50
5 / 18
results
normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive
restrictive
inadequate
identical
one off
32 / 50
5 / 18
results
normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive
restrictive
inadequate
identical
one off
32 / 50
5 / 18
FDA study underway
• part a: head to head clinical test
FDA study underway
• part a: head to head clinical test
• part b: home spirometry
FDA study underway
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
futureresearch
ubicomplabU N I V E R S I T Y O F W A S H I N G T O N
Professor Shwetak N. Patel
Healthsensing
Novelinteractiontechniques
Low-power sensing
Sustainabilitysensing
AWARDS
Professor Shwetak N. Patel awarded:
STRONG INDUSTRY COLLABORATIONS
IN THE NEWS
Computer Science & Engineering
Electrical Engineering
design + use +build
17 best paper awards & nominations in 3 years2 National Science Foundation Fellows2 Microsoft Research Fellows3 UW College of Engineering Innovator Awards
Belkin acquires work in Sustainability SensingSidhant Gupta featured Forbes “30 under 30”
MacArthur ‘Genius’ AwardSloan Research Fellow
TR-35 Top innovatorSeattle’s Top Innovator
MIT Technology ReviewNew York TimesNPRPopular SciencePopular MechanicsScienti!c American
CNNThe EconomistWiredGizmodoEngadgetNew Scientist
ElectriSense InGen
SqueezeBlock
uTouch
LightWave
Humantenna
GripSense
HeatWave
WalkType
SNUPI
Static E-Field
CoughSense
SpiroSmart
ElderCare
Lullaby
MobilityRehabilitation
GasSense
HydroSense
InnerSol
water sensing
lungfunction
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
activity detectionelder care
daily activity logscongestive heart failure
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
activity detectionelder care
daily activity logscongestive heart failure
health markers via phoneblood pressurepulse oximetrystress
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
activity detectionelder care
daily activity logscongestive heart failure
health markers via phoneblood pressurepulse oximetrystress
opportunistic sensingpain managementdetecting circulation
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
activity detectionelder care
daily activity logscongestive heart failure
health markers via phoneblood pressurepulse oximetrystress
opportunistic sensingpain managementdetecting circulation
developing world
summary
• sustainable water use, eco-feedback
• lung function via mobile phone
• future work in high impact areas
enabling eco-feedback and out-of-clinic health sensing
indirect
University of WashingtonEric C. Larson
UbiComp Lab
electrical engineering
computerscience and engineering
ubiquitous sensing
eclarson.com eclarson@uw.edu@ec_larson
Thank You!
thermal imaging
facialanalysis
image fidelity
power harvesting
gas sensing
cough sensing
inte
ract
ion
& im
age
anal
ysis
sust
aina
bilit
yhe
alth
water sensing
lungfunction
enabling eco-feedback and out-of-clinic health sensingindirect
University of Washington
Eric C. LarsonPhD Candidate in School of Electrical and Computer Engineering
UbiComp Lab
electrical engineering
computerscience
ubiquitous sensing
eclarson.com eclarson@uw.edu@ec_larson
acknowledgments:Jon FroehlichLeah FindlaterElliot SabaEric SwansonTim CampbellGabe CohnMayank Goel
TienJui LeeSidhant GuptaJosh PetersonConor HaggertyJeff BeorseShwetak PatelLes AtlasJames FogartyJeff BilmesMargaret Rosenfeld
water tower water tower
thermal expansion
tank
hose spigot
utility water meter
pressure regulator
laundry
bathroom 1
hot water heater bathroom 2
dishwasher
incoming cold water from supply line
kitchen
water tower water tower
thermal expansion
tank
hose spigot
utility water meter
pressure regulator
laundry
bathroom 1
hot water heater bathroom 2
dishwasher
incoming cold water from supply line
kitchen
bath%
toilet&
shower'
kitchen(sink(
features: single instance
bath%
toilet&
shower'
kitchen(sink(
features: single instance
bath%
toilet&
shower'
kitchen(sink(
features: single instance
bath%
toilet&
shower'
kitchen(sink(
features: single instance
stable pressure (psi)
max
am
plitu
de
features: single instance
stable pressure (psi)
max
am
plitu
de
Kitchen(Sink(Basement(Sink(
Bathroom(Sink(
Basement(
Shower(
((((((((((((((((((((Upstairs(S
hower(
Upstairs(Bath(
features: single instance
42#
44#
46#
48#
50#
0# 5# 10# 15# 20# 25# 30# 35# 40# 45#
raw pressure (psi)
44"
46"
48"smoothed pressure
(psi)
bandpass derivative
(psi/s)
!4#
0#
4#
detrended
derivative
bandpass derivative
const-Q cepstrum
signa
l tra
nsfo
rms
features: single instance
frequency
42#
44#
46#
48#
50#
0# 5# 10# 15# 20# 25# 30# 35# 40# 45#
raw pressure (psi)
*10#
10#
30#cepstral
amplitude
index
detrended
derivative
bandpass derivative
const-Q cepstrum
signa
l tra
nsfo
rms
features: single instance
features: single instance
time
frequ
ency
features: single instance
]]
time
frequ
ency
]
features: single instance
]]
time
frequ
ency
]
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature extraction:sequence
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
feature extraction:sequence
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
10 sec
feature extraction:sequence
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
10 sec compound
feature extraction:sequence
70
50
30
pres
sure
(psi)
70
50
30
pres
sure
(psi)
10 sec compound time of day
feature extraction:sequence
accuracy
fixture levele.g., upstairs bathroom faucet
accuracy
fixture levele.g., upstairs bathroom faucet
accuracy
fixture50
60
70
80
90
100
92.2
fixture levele.g., upstairs bathroom faucet
accuracy
fixture50
60
70
80
90
100
92.2
what does this tell us?
10 fold cross validation
cycles
cyclesheated dry
cyclesheated dry
washing dishes
~7:30AM
filling coffee pot
coffee pot running
~7:30AM
bathroom usage
bathroom usage
leaky flapper valve
flow features
0 1 2 3 4−1
0
1
0 1 2 3 4−0.5
0
0.5
Amplitude
0 1 2 3 4−1
0
1
time(s)
flow features
0 1 2 3 4−1
0
1
0 1 2 3 4−0.5
0
0.5
Amplitude
0 1 2 3 4−1
0
1
time(s)
−1 0 1 2 3 4 50
2
4
6
8
time(s)flow
(L/s)
flow features
0 1 2 3 4−1
0
1
0 1 2 3 4−0.5
0
0.5
Amplitude
0 1 2 3 4−1
0
1
time(s)
~audio, pressure
−1 0 1 2 3 4 50
2
4
6
8
time(s)flow
(L/s)
flow features
0 1 2 3 4−1
0
1
0 1 2 3 4−0.5
0
0.5
Amplitude
0 1 2 3 4−1
0
1
time(s)
~audio, pressure
~pressure at lips
−1 0 1 2 3 4 50
2
4
6
8
time(s)flow
(L/s)
flow features
0 1 2 3 4−1
0
1
0 1 2 3 4−0.5
0
0.5
Amplitude
0 1 2 3 4−1
0
1
time(s)
~audio, pressure
~pressure at lips
~flow at lips
−1 0 1 2 3 4 50
2
4
6
8
time(s)flow
(L/s)
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature 2
linear chain regression
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
feature 1
feature 2
curveoutput
linear chain regression
CRF
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
linear chain regression
CRF
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
bagged decision tree
linear chain regression
CRF
−1 0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
0 1 2 3 4 50
0.1
0.2
0.3
0.4
time(s)
feat
ure
valu
e
bagged decision tree
−1 0 1 2 3 4 50
2
4
6
8
time(s)
flow(L/s)
linear chain regression
example curvescurveregression
0 1 2 30
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
10
volume(L)flow(L/s)
example curvescurveregression
0 1 2 30
2
4
6
8
volume(L)
flow(L/s)
0 1 2 30
2
4
6
8
volume(L)
flow(L/s)
−1 0 1 2 3 40
2
4
6
8
10
volume(L)flow(L/s)
−1 0 1 2 3 40
2
4
6
8
10
volume(L)flow(L/s)