The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1...
-
Upload
brett-matthews -
Category
Documents
-
view
216 -
download
0
Transcript of The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1...
![Page 1: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/1.jpg)
The Challenges and Potential of End-User Gesture Customization
Uran Oh1 and Leah Findlater2
1 Department of Computer Science2 College of Information StudiesUniversity of Maryland, College Park
�
![Page 2: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/2.jpg)
Touchscreen gestures are widely used…Who designs these gestures?Design experts.
Apple’s touchpad gestures
![Page 3: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/3.jpg)
1) Tools for supporting designers (developers)to create gestures with ease
Previous Research:
A figure from Gesture Coder
MAGIC:[Ashbrook et al. 2010]
Proton++:[Kin et al. 2012]
Gesture Coder:[Lü et al. 2012]
![Page 4: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/4.jpg)
2) Methods for creating a gesture set that are intuitive and guessable by a wide range of users
[Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011]
Previous Research:
A figure from [Wobbrock et al. 2009]
![Page 5: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/5.jpg)
(2) Methods for creating a gesture set that are intuitive and guessable by a wide range of users
[Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011]
Previous research:
A figure from [Wobbrock et al. 2009]
Our focus: Supporting end-users
Personal gestures for a single user
![Page 6: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/6.jpg)
(2) Methods for creating a gesture set that are intuitive and guessable by a wide range of users
[Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011]
Previous research:
A figure from [Wobbrock et al. 2009]
Our focus: Supporting end-users
Personal gestures for a single userWhy?
![Page 7: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/7.jpg)
MemorabilityEfficiency
Accessibility
Potential Advantages of Self-defined Gestures…
![Page 8: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/8.jpg)
[Nacenta et al. 2013]
Memorability
�
Self-defined gestures improve memorability over predefined gestures
![Page 9: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/9.jpg)
[Ouyang et al. 2012]
Efficiency
�
Gestural shortcuts can be used as an efficient mean of accessing information
![Page 10: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/10.jpg)
Accessibility
[Anthony et al. 2013]
�
Customized gestures may improve accessibilityfor people with physical disabilities
![Page 11: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/11.jpg)
Our Goal: To investigate the feasibility of end-user gesture creation
�
![Page 12: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/12.jpg)
Our Goal: To investigate the feasibility of end-user gesture creation
�
How do typical users create gestures?
What are the challenges therein?
How can we support the process?
![Page 13: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/13.jpg)
v
Task 1 Task 2 Task 3Task 2
Open-EndedGesture Creation
Action-SpecificGesture Creation
Saliency ofGesture Features
Study With Three Tasks
![Page 14: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/14.jpg)
Controlled lab study - 20 participants (age from 20 to 35,
M=29.3)- Prior experience with touchscreen devices- Single one-hour session with 3 tasks- Think-aloud protocol
Study Method
�
Apparatus- Samsung Galaxy Tab
2 (10.1’’ running Android 4.0.4)
![Page 15: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/15.jpg)
v
Task 1
Task 2 Task 3Task 2
Open-EndedGesture Creation
Action-SpecificGesture Creation
Saliency ofGesture Features
Q. Are users able to create new gestures easily?If not, what are the barriers?
![Page 16: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/16.jpg)
�
Task 1: Open-ended Gesture Creation
“Create as many gestures as possible”
![Page 17: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/17.jpg)
�
“Create as many gestures as possible”
Task 1: Open-ended Gesture Creation
• For any purpose
• Any number of strokes, fingers, hands
• As long as they are:easy to draw, easy to remember,
distinguishable
![Page 18: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/18.jpg)
�
Task 1: Open-ended Gesture Creation
“Create as many gestures as possible”
![Page 19: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/19.jpg)
12.2 gestures created on average
(SD = 8.1, range of 5 to 36)
Gestures Created
p3
Total number of gestures and the number of arbitrary gestures are correlated
(Pearson’s r=.47, p=.037)
Task 1: Open-ended Gesture Creation
p5
![Page 20: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/20.jpg)
12.2 gestures created on average
(SD = 8.1, range of 5 to 36)
Gestures Created
p3 p5
Total number of gestures and the number of arbitrary gestures are correlated
(Pearson’s r=.47, p=.037)
Task 1: Open-ended Gesture Creation
p5
![Page 21: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/21.jpg)
Tendency to focus on the familiar“I just thought of gestures my tablet PC had.” (P1)“These gestures are all I use, I cannot be more creative” (P8)
Difficulties Creating Gestures
�
Opaque nature of gesture recognizer“Can I use all fingers?” (P2)
Task 1: Open-ended Gesture Creation
![Page 22: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/22.jpg)
v
Task 1
Task 2 Task 3Task 2
Open-EndedGesture Creation
Action-SpecificGesture Creation
Saliency ofGesture Features
A. Users felt difficulty in creating new gesturesBetter understanding of recognizer is needed
![Page 23: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/23.jpg)
23
Task 1
Task 2
Task 3
Task 2
Open-EndedGesture Creation
Action-SpecificGesture Creation
Saliency ofGesture Features
Q. What is a “good gesture” to end-users?How is it different from recognizer’s perspective?
![Page 24: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/24.jpg)
Task 2: Action-Specific Gesture Creation
Brainstorm gesturesper action
12 Specific ActionsZoom-inZoom-outRotateCopyCutPasteSelect-singleSelect-multiplePreviousNextCall-MomLaunch a web-browser
![Page 25: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/25.jpg)
12 Specific ActionsZoom-inZoom-outRotateCopyCutPasteSelect-singleSelect-multiplePreviousNextCall-MomLaunch a web-browser
Task 2: Action-Specific Gesture Creation
Brainstorm gesturesper action
![Page 26: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/26.jpg)
Task 2: Action-Specific Gesture Creation
Compose custom set of gestures, one per
action
Brainstorm gesturesper action
![Page 27: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/27.jpg)
Task 2: Action-Specific Gesture Creation
Compose custom set of gestures, one per
action
Brainstorm gesturesper action
![Page 28: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/28.jpg)
Task 2: Action-Specific Gesture Creation
Compose custom set of gestures, one per
action
Brainstorm gesturesper action
![Page 29: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/29.jpg)
Task 2: Action-Specific Gesture Creation
Brainstorm gesturesper action
Compose custom set of gestures, one per
action
Create training examples
(4 per selected gesture)
![Page 30: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/30.jpg)
Task 2: Action-Specific Gesture Creation
Brainstorm gesturesper action
Compose custom set of gestures, one per
action
Create training examples
(4 per selected gesture)
Rate satisfaction with the custom gesture
set
![Page 31: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/31.jpg)
Brainstorm gesturesper action
Compose custom set of gestures, one per
action
Create training examples
(4 per selected gesture)
Rate satisfaction with the custom gesture
set
Test recognition accuracy with $N
recognizer
Initial example
Training examples
Task 2: Action-Specific Gesture Creation
![Page 32: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/32.jpg)
Generally Preferred
Accurate
Familiar
Simple/Easy
Intuitive/Natural/Obvious
0 5 10 15 20 25 30
11.34
12.18
15.55
22.69
27.73
Percentage of Gestures (%)
Reasons for selecting a gesture for custom set
Others reasons: Generally preferred, fast, consistent, easy to remember, etc.
Task 2: Action-Specific Gesture Creation
![Page 33: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/33.jpg)
Need for improvement
Participants gave up the opportunity to edit their gesture set to make improvements
Task 2: Action-Specific Gesture Creation
Only two participants were fully satisfied
( M=5.3, SD = 1.1 where 1=negative, 7=positive)
Inability to improve gesture sets
![Page 34: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/34.jpg)
Low Recognition Potential of the Custom Sets$N recognizer (default setting) with 5-fold cross validation
1 2 3 40.7
0.8
0.9
Number of Training Examples
Re
cog
nit
ion
Ac-
cura
cy
�
76–88% accuracy depending on amount of training
Task 2: Action-Specific Gesture Creation
![Page 35: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/35.jpg)
35
Task 2
Task 3
Task 2
Action-SpecificGesture Creation
Saliency ofGesture Features
Customized set can be improved for both user’s and recognizer’s perspectiveA.
Task 1
Open-EndedGesture Creation
![Page 36: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/36.jpg)
36
Task 2
Task 3
Task 2
Action-SpecificGesture Creation
Saliency ofGesture Features
What features do users rely on to distinguish between gestures?Q.
Task 1
Open-EndedGesture Creation
![Page 37: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/37.jpg)
Gesture Features Judged
Orientation
Very slow Very fast slow fast moderate
Scale
Aspect Ratio
Speed
Task 3: Saliency of Gesture Features
Curviness
Pattern Repetition
6 features from Rubine’s recognizer [Rubine. 1991]
![Page 38: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/38.jpg)
Gesture Features Judged
Orientation
Scale
Aspect Ratio
Task 3: Saliency of Gesture Features
Curviness
Pattern Repetition
Finger Count
Stroke Count
Stroke Order
3 touchscreen features
6 features from Rubine’s recognizer [Rubine. 1991]
Very slow Very fast slow fast moderate
Speed
![Page 39: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/39.jpg)
Orientation
Scale
Aspect Ratio
Task 3: Saliency of Gesture Features
Curviness
Pattern Repetition
Finger Count
Stroke Count
Stroke Order
3 touchscreen features
6 features from Rubine’s recognizer [Rubine. 1991]
“Rank the distinguishability of 9 features”
Very slow Very fast slow fast moderate
Speed
![Page 40: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/40.jpg)
Objective features are more distinguishableFeatures that can be consistently interpreted/manipulated are considered distinguishable
“Even if the same person is performing the gesture, it might not have the same speed and size” (P7)
More distinctive
Very fast
Speed
Scale
Pattern
Repetit
ion
Aspect
Ratio
Curvin
ess
Orienta
tion
Stroke
Ord
er
Stroke
count
Finger c
ount
Task 3: Saliency of Gesture Features
![Page 41: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/41.jpg)
Objective features are more distinguishableFeatures that can be consistently interpreted/manipulated are considered distinguishable
“Even if the same person is performing the gesture, it might not have the same speed and size” (P7)
More distinctive
Very fast
Speed
Scale
Pattern
Repetit
ion
Aspect
Ratio
Curvin
ess
Orienta
tion
Stroke
Ord
er
Stroke
count
Finger c
ount
Task 3: Saliency of Gesture Features
![Page 42: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/42.jpg)
Objective features are more distinguishableFeatures that can be consistently interpreted/manipulated are considered distinguishable
“Even if the same person is performing the gesture, it might not have the same speed and size” (P7)
More distinctive
Very fast
Speed
Scale
Pattern
Repetit
ion
Aspect
Ratio
Curvin
ess
Orienta
tion
Stroke
Ord
er
Stroke
count
Finger c
ount
Task 3: Saliency of Gesture Features
![Page 43: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/43.jpg)
43
Task 2
Task 3
Task 2
Action-SpecificGesture Creation
Saliency ofGesture Features
A.
Task 1
Open-EndedGesture Creation
Number of fingers/strokes, stroke order aredistinguishable than speed or size
![Page 44: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/44.jpg)
Summary
Creating new gestures is hard for end-users• Tendency to focus on the familiar• Opaque nature of gesture recognizer
�
Objective features are more distinguishable• Finger/stroke count, stroke order are
more distinguishable than speed and scale
Quality of gesture sets can be improved• Users are not fully satisfied with their
gesture sets• Low recognition potential
![Page 45: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/45.jpg)
Memorability
Efficiency
Accessibility
�
Potential Benefits of Allowing End-User Customization
Take-away Message
Systematic Support is Needed for End-User
Customization
![Page 46: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/46.jpg)
Future Work
Mixed-initiative support for customization
Feedback
EditsTrain
System Gesture set User
![Page 47: The Challenges and Potential of End- User Gesture Customization Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information.](https://reader038.fdocuments.us/reader038/viewer/2022103007/56649e985503460f94b9b95f/html5/thumbnails/47.jpg)
47
The Challenges and Potential of End-User Gesture Customization
Uran Oh1 and Leah Findlater2
1 Department of Computer Science2 College of Information StudiesUniversity of Maryland, College Park
�
[email protected] | [email protected]
Thank you for listening
Questions?