Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al....
-
Upload
andres-gush -
Category
Documents
-
view
219 -
download
0
Transcript of Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al....
![Page 1: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/1.jpg)
Whole-Home Gesture Recognition Using Wireless Signals
——MobiCom’13
Author: Qifan Pu et al. University of WashingtonPresenter: Yanyuan Qin & Zhitong Fei
![Page 2: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/2.jpg)
•Why interactive technique?Control music volume with gesture when showering or cooking.Turn on the thermostat with gesture while in bed.The disabled need it more.
•Options?Kinect at each room? Expensive…
Wearable Devices? Wear a device in a shower?…
Background
![Page 3: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/3.jpg)
• Can we use ?
Can we use it to recognize gestures?
![Page 4: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/4.jpg)
How to detect gestures using Wi-Fi?
• Wi-Fi Doppler shifts• Humans reflect Wi-Fi signals, thus can be treated as signal sources• Human motion introduce Wifi Doppler shifts
• Different gestures exhibit different patterns.
![Page 5: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/5.jpg)
Two Challenges• Challenge 1: How to detect small Doppler shifts within
WiFi’s bandwidth (typically 20MHz)
2 / ,f fv c 5
0.5 /
_
f GHz
v m s
c light speed
17f Hz
• Challenge 2: How to determine the relationship between Doppler shifts and gestures
In WiSee the target human performs a repetitive gesture, which we use as that person’s preamble.
![Page 6: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/6.jpg)
WiSee• WiSee is a wireless system that enable whole-home gesture reconigtion.
• Three main questions: First, how does WiSee extract Doppler shifts from conventional wireless signals like Wi-Fi?
Second, how does it map the Doppler shifts to the gestures performed by the user?
Third, how does it enable gesture recognition in the presence of other humans in the environment?
![Page 7: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/7.jpg)
Extracting Doppler shifts from Wireless Signals
Doppler shift is the change in the observed frequency as the transmitter and the receiver move relative to each other.
• Move towards crests arrive at a faster rate
• Move from crests arrive at a slower rate
• Where c is the speed of light in the medium and f is the transmitter’s center frequency.
![Page 8: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/8.jpg)
Extracting Doppler shifts from Wireless SignalsHow does WiSee deal with frequency offsets?
![Page 9: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/9.jpg)
Mapping Doppler Shifts to Gestures
WiSee extracts the Doppler information by computing the frequency-time Doppler profile of the narrowband signal.
![Page 10: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/10.jpg)
Segmentation
Gestures
WiSee leverages the stucture of the Doppler profiles. Set of segments that have positive and negative Doppler shifts.
![Page 11: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/11.jpg)
Gestures Classification
The receiver can classify gestures by matching the pattern.
Positive “1”
Negative “-1”
Both “2”
![Page 12: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/12.jpg)
Challenge 3: Interference from other people in the environment• WiSee uses MIMO in 802.11n to improve accuracy • But MIMO requires a known preamble• In WiSee, a repeated gesture acts as preamble to specify certain user
User pushes hand toward
and away
WiSee detect target user
WiSee iteratively finds MIMO channel that maximizes Doppler energy
![Page 13: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/13.jpg)
Multi-path EffectsThe reflections usually arrive at the receiver along multiple paths.
A gesture towards receiver can create both positive and negative Doppler shifts at the receiver.
WiSee automatically finds a proper MIMO direction
because of iteration algorithm
Strong reflectors (metal) can flip the positive and negative Doppler shifts.
The receiver can calibrate the sign of the subsequent Doppler shifts based on preamble gestures.
![Page 14: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/14.jpg)
Experiments
•ScenariosOffice buildingTwo-bed apt
•Many conditionsLine-of-sight, non-line-of-sight, through-the-wall,
through-the-corridor, through-the-room
![Page 15: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/15.jpg)
TX: transmitter RX: Wisee receiver
Wall
Feasibility of Wireless Gesture Detection
![Page 16: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/16.jpg)
Feasibility of Wireless Gesture Detection
![Page 17: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/17.jpg)
Gesture Recognition in Two-bed apt
![Page 18: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/18.jpg)
Gesture Recognition Accuracy
Confusion matrix
![Page 19: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/19.jpg)
Gestures in the Presence of Other Humans (No target user, only other 12 people)
False detection rate for a whole day
![Page 20: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/20.jpg)
Influence from the number of interfering user (Target user and other people)
![Page 21: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/21.jpg)
Influence from the distance of interfering user
![Page 22: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/22.jpg)
Summary
•Design and evaluate, WiSee, a gesture recognition system that leverages wireless signals to enable whole-home sensing and recognition of human gestures
•WiSee can extract a rich set of gesture information from wireless signals and enable whole-home gesture recognition using only two wireless sources placed in the living room
![Page 23: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/23.jpg)
Disadvantages
Whole-home gesture recognition with few wireless sources. Without sensing device on human body or many devices in
environmentHigh accuracy: 94%(on average) for 9 gestures.
Advantages
Heavily influenced by the number of other people.The number of gestures are limited (9 gestures) and hard to
extend.Using scale is smaller than wearable devices.
Comments on WiSee
![Page 24: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/24.jpg)
Future Works
•Security? •Information fusion: Add other information to make the system more robust, like sound•Explore whether the shape of body will affect the performance.
![Page 25: Whole-Home Gesture Recognition Using Wireless Signals —— MobiCom’13 Author: Qifan Pu et al. University of Washington Presenter: Yanyuan Qin & Zhitong Fei.](https://reader030.fdocuments.us/reader030/viewer/2022032517/56649c7f5503460f94935ea6/html5/thumbnails/25.jpg)