Automatic user state recognition for hand gesture based low cost television control system
-
Upload
martina-thampan -
Category
Engineering
-
view
1.027 -
download
3
description
Transcript of Automatic user state recognition for hand gesture based low cost television control system
![Page 1: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/1.jpg)
Automatic User State Recognitionfor Hand Gesture Based Low-CostTelevision Control System
Automatic User State Recognitionfor Hand Gesture Based Low-CostTelevision Control System
![Page 2: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/2.jpg)
2 CONTENTS
INTRODUCTION
HAND FREE CONTROL SYSTEM
PROPOSED SYSTEM
COMPONENTS OF AUSR SYSTEM
USER STATE DEFINITION
USER STATE INITIALIZATION
USER STATE TRANSITION
HAND GESTURE RECOGNITION
EXPERIMENT RESULTS
CONCLUSION
REFERENCES
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 3: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/3.jpg)
3 INTRODUCTION
Innovation in TV display screen TV controller remains unchanged Fitt's Law Hand Free control system
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 4: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/4.jpg)
4 HAND FREE CONTROL SYSTEM
Voice controlled devices
Hand gesture control system
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 5: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/5.jpg)
5 PROPOSED SYSTEM
Low cost hand gesture in automatic manner
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 6: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/6.jpg)
6
PROPOSED SYSTEM(contd..)
Automatic user state recognition(AUSR)
Ultra sonic distance array and camera module
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 7: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/7.jpg)
7 IMPLEMENTATION OF THE SYSTEM
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 8: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/8.jpg)
8 COMPONENTS OF AUSR SYSTEM
Hardware TVset-top boxsensors(RGB or depth cameras) ultrasonic distance array
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 9: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/9.jpg)
9 COMPONENTS OF AUSR(contd…)
Embedded software componentsconnects sensors with TVProcess the sensor dataDrive the TV control process
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 10: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/10.jpg)
10 USER STATE DEFINITION
Classification of user stateAbsentOther actionControllingWatching
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 11: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/11.jpg)
11 USER STATE DEFINITION (contd..)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 12: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/12.jpg)
12 USER STATE INITIALIZATION
Action detection Distance detection Movement detection Single sensor based & sensor array based
action decision.
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 13: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/13.jpg)
13 USER STATE INITIALIZATION (contd..)
L(t)=C.T(t)/2
L(t)=distance between object and sensor
C=sonic speed in air(340 m/s).
T(t)=time between the sensor emits and
receives the ultrasonic waves.
∆Lt =|L(t+P)-L(t)|
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 14: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/14.jpg)
14 USER STATE INITIALIZATION (contd..)
At = 0, ∆Lt <R (considering the threshold)
1,∆Lt ≥ R
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 15: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/15.jpg)
15 USER STATE INITIALIZATION (contd..)
Considering action sequence :
St=[∆ Lt-k+1 ,∆Lt-k+2 ,..., ∆Lt-1, ∆Lt ]
Based on threshold R for non intentional movements
SRt =[At-k+1 ,At-k+2 ,..,At-1 , At ]
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 16: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/16.jpg)
16 USER STATE INITIALIZATION (contd..)
Action status
A= 0, SUM(SRt )< Q
1, SUM(SRt )≥ Q
SUM(SRt )=∑ i=0K-1 At-i
S=2.√2.r=2√2.D.tgθ Action decision will be according to A= 0, max [A0 ,A1 ,A2 ,A3 ]=0 1, max [A0 ,A1 ,A2 ,A3 ]=1
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 17: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/17.jpg)
17 USER STATE INITIALIZATION (contd..)
Presentation detection
Face detection
Horizontal view angle α
α= arctg(W/2L)
arctg- inverse trigonometric function
W- width of tv screen
L-distance between user eye and TV screen.
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 18: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/18.jpg)
18 USER STATE INITIALIZATION (contd..)
Vertical view angle β
Β = arctg(H/2L)
H- height of TV screen Roll angle[φ] Lf ={F|-α≤Fh ≤α, -β ≤Fv ≤β and –φ ≤ Fr ≤φ }
B= o, if the face is not detected
1, if the face is detected
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 19: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/19.jpg)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
19 USER STATE INITIALIZATION (contd..)
![Page 20: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/20.jpg)
20 USER STATE INITIALIZATION (contd..)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 21: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/21.jpg)
21 USER STATE INITIALIZATION (contd..)
User state decision decided from results of action detection
and presentation detection.
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 22: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/22.jpg)
22
USER STATE INITIALIZATION (contd..)
ALGORITHIM OF DECISION OF USER STATE
If (the intentional motion is detected)
If (the face watching TV is detected)
Set (A=1,B=1) - Controlling
Else
Set (A=1,B=0) - Other Action
Else if (the face watching TV is detected)
Set (A=0,B=1) - Watching
Else
Set (A=0,B=0) - Absent
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 23: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/23.jpg)
23USER STATE INITIALIZATION (contd..)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 24: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/24.jpg)
24USER STATE TRANSITION If (the intentional motion is detected) If (A=1 and B=1)
Detect hand gesturesElse if (A=0 and B=0)
Set (A=1,B=0)Else if (A=1 and B=0)
If (the face watching TV is detected)Set (A=1,B=1)
ElseIf (the face watching TV is detected)Set (A=1,B=1)
Else if (the intentional motion is not detected for a long time)If (A=1,B=0)
Set (A=0,B=0)Else if(A=1,B=1)
Set (A=0,B=1)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 25: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/25.jpg)
25 USER STATE TRANSITION(contd..)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 26: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/26.jpg)
26 HAND GESTURE RECOGNITION
Hand gesture recognition module & camera activated when user state is controlling.
Gesture continuously detected in Tf
Tf is the threshold for restrict the time period permitted for the user to do hand gestures
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 27: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/27.jpg)
27 EXPERIMENTAL RESULTS
User study
Out of 6 people 2young,2middle age and 2 old users are considered
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 28: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/28.jpg)
28 EXPERIMENTAL RESULTS (contd..)
Automatic user state recognition
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 29: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/29.jpg)
29 EXPERIMENTAL RESULTS (contd..)
Computational cost and power consumption
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 30: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/30.jpg)
30 CONCLUSIONS
TV user’s states are defined according to finite state machine.
Camera system and gesture recognition is activated or closed by recognised user state.
Reduction in computational cost and power consumption
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 31: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/31.jpg)
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
31 REFERENCES
[1]Shiguo Lian, Member, IEEE, Wei Hu, Kai Wang,” Automatic User State Recognition for Hand Gesture Based Low-Cost Television Control System”, IEEE Transactions on Consumer Electronics, Vol. 60, No. 1, February 2014.
[2]I. S. Mackenzie and W. Buxton, "Extending fitts' law to two dimensional tasks," in Proceedings of CHI'92, pp. 219-226, May 1992.
[3]S. K. Kim, G. H. Park, S. H. Yim, S. M. Choi, and S. J. Choi, “Gesture recognizing hand-held interface with vibrotactile feedback for 3D interaction,” IEEE Trans. Consumer Electronics, vol. 55, no. 3, pp.1169-1177, Aug. 2009.
[4]H. Heo, E. C. Lee, K. R. Park, C. J. Kim, and M. C. Whang, “A realistic game system using multi-modal user interfaces,” IEEE Trans. Consumer Electronics, vol. 56, no. 3, pp. 1364-1372, Aug. 2010.
[5]D. W. Lee, J. M. Lim, S. W. John, I. Y. Cho, and C. H. Lee, “Actual remote control: a universal remote control using hand motions on a virtual menu,” IEEE Trans. Consumer Electronics, vol. 55, no. 3, pp.1439-1446, Aug. 2009
![Page 32: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/32.jpg)
32
Questions please??
AUSR FOR HAND GESTURE BASED LOW COST TV CONTROL SYSTEM
![Page 33: Automatic user state recognition for hand gesture based low cost television control system](https://reader033.fdocuments.us/reader033/viewer/2022052904/558300a7d8b42a50628b457a/html5/thumbnails/33.jpg)
THANK YOU