Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web
-
Upload
markel-vigo -
Category
Technology
-
view
299 -
download
1
description
Transcript of Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web
![Page 1: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/1.jpg)
{
Detecting Problematic Interactions on the Web
The COPE Project: Coping strategies on the Web
markel.vigo@{ehu.es | manchester.ac.uk}
@markelvigoDonostia, 18/11/2013
![Page 2: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/2.jpg)
The Web is a complex environment
Motivation
![Page 3: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/3.jpg)
Motivation
Ceaparu,I., Lazar,J., Bessiere,K., Robinson,J., and Shneiderman,B. (2004) Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17(3), 333–356.
![Page 4: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/4.jpg)
Design Unfamiliar conventions Lack of skills
Problems
![Page 5: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/5.jpg)
Post-hoc analysis: 87% detection accuracy (Feild, 2010)- Self-reported frustration- Usage metrics: query length, task time.
Feild, H.A., Allan, J. and Jones, R. (2010) Predicting searcher frustration. ACM SIGIR ’10, 34–41.
Detecting problems
![Page 6: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/6.jpg)
Task-driven vs. emergent tasks
Top-down vs. bottom up Constrained vs. ecological SERP vs. navigation and more
Detecting problems
![Page 7: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/7.jpg)
Reactions are cognitive markers that indicate problems
Ask for help (Novick, 2007) Other means (Ceaparu, 2004)
- "I found an alternative solution"- "I figure out how to fix myself"
Novick, D. G., Elizalde, E., and Bean, N. (2007) Toward a more accurate view of when and how people seek help with computer applications. ACM SIGDOC '07, 95–102. Ceaparu, I., Lazar, J., Bessiere, K., Robinson, J., and Shneiderman, B. (2004) Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17(3), 333–356.
Reactions to problems
![Page 8: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/8.jpg)
Reactions are cognitive markers that indicate problems
If we are able to automatically detect coping we can provide the means to overcome the situation
Identifying strategies
![Page 9: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/9.jpg)
Challenges- Low frequency- Non-overtly manifested- Ecological setting (get out of the lab!)
Consequences- Significant amount of observations into the wild
- Time and personnel++
Identifying strategies
![Page 10: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/10.jpg)
We can transfer strategies from populations who cope more frequent and overtly to general audiences
Hypothesis
![Page 11: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/11.jpg)
Overlap of problems between:- The constrains imposed by reduced screens and keyboards (situational impairments)
- Problems encountered by motor impaired desktop users
Same scope, different magnitude
Evidence
Yesilada,Y., Harper,S., Chen,T., Trewin,S. (2010) Small-device users situationally impaired by input. Computers in Human Behavior 26 (3), 427–435. Simon Harper, Yeliz Yesilada, Tianyi Chen (2011) Mobile device impairment...similar problems, similar solutions? Behaviour & IT 30(5),673-690
![Page 12: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/12.jpg)
MAGNITUDE
There is an accessibility continuum with overlapping- Problems- Strategies- Solutions
Hypothesis
SCOPE
lower access
end
higher access end
people with disabilities
able bodied users
situational impairment
![Page 13: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/13.jpg)
HypothesisMAGNITUDE
SCOPE
lower access
end
higher access end
Our goal is more ambitious: explore the commonalities of distant populations.- Visually disabled users who cope- Able bodied users encountering difficulties
physically impaired
mobile users
Yesilada et al. (2010)Harper et al. (2011)
visually disabled
able bodied users
![Page 14: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/14.jpg)
Our proposal4 step method:
1. Observation and identification of strategies
2. Implementation of algorithms to detect strategies
3. Deployment into the wild4. Run user studies
Go to step 2 until obtaining reliable
results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 15: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/15.jpg)
Step 1. Observationand identification of strategies
2 independent ethnographic studies and user tests
24 screen reader and screen magnifier users
8 coping strategies (17 implementations) were identifiedVigo, M and Harper, S. (2013) Coping tactics employed by visually disabled users on the web.
International Journal of Human-Computer Studies 71(11), 1013–1025.
![Page 16: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/16.jpg)
Step 1. Observationand identification of strategiesT1. Asking for assistanceT2. Impulsive clicking
- I21. Deliberately clicking on low-scented links
- I22. Clicking on any link- I23. Clicking on any link as long as
it is accessible
T3. Exploration tactics- I31. Persevering- I32. Escaping from content by
moving down- I33. Move down without listening
to content- I34. Move around
T4. Narrowing down search
Vigo, M and Harper, S. (2013) Coping tactics employed by visually disabled users on the web. International Journal of Human-Computer Studies 71(11), 1013–1025.
T5. Gaining orientation- I51. Gaining local orientation- I52. Gaining global orientation
T6. Redoing- I61. Re-checking- I62. Re-typing- I63. Re-tracing- I64. Re-starting
T7. Not operating- I71. Delegating on assistive
technology- I72. Swapping assistive
technology- I73. Waiting
T8. Giving up
![Page 17: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/17.jpg)
Step 1. Further analysis I We analyse the skill acquisition process of visually disabled users over time
A decrease in last resort tactics is observed across sessions
In parallel, users exhibit more exploration tactics and explore more websites
Vigo, M and Harper, S. (under review) A snapshot of the first encounters of visually disabled users with the Web.
![Page 18: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/18.jpg)
Step 1. Further analysis I While effectiveness increases, coping success does not
Misuse of tactics may be indicative of a larger repertoire or increased confidence.
Vigo, M and Harper, S. (under review) A snapshot of the first encounters of visually disabled users with the Web.
![Page 19: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/19.jpg)
Step 1. Further analysis II Users are not always driven by information scent
Navigation is driven by the need to overcome an interaction problem
Users escape from the current web patch- Users click on misleading links if by doing this problems are averted.
- Low satisfacing levels: in extraordinary situations any web patch is “good enough”.
Vigo, M and Harper, S. (2013) Challenging information foraging theory: screen reader users are not always driven by information scent. ACM Conference on Hypertext and Social Media, 60-68
![Page 20: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/20.jpg)
Our proposal4 step method:
1. Observation and identification of strategies ✔
2. Implementation of algorithms to detect strategies
3. Deployment into the wild4. Run user studies
Go to step 2 until obtaining reliable
results
Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 21: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/21.jpg)
Step 2. Implementation ofalgorithms to detect strategies
I61. Re-checking: quick revisitationswebpagei→wpj→wpiwpj
I63. Re-tracing: users retrace the steps in a sequence of pages. wpj→wpk→wpl→wpm→wpl→wpk→wpj→wpk→wpl→wpm
![Page 22: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/22.jpg)
Our proposal4 step method:
1. Observation and identification of strategies ✔
2. Implementation of algorithms to detect strategies ✔
3. Deployment into the wild4. Run user studies
Go to step 2 until obtaining reliable
results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 23: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/23.jpg)
Step 3. Deployment intothe wild
WebTactics
t1(){..}
DB
User Website Manager
/Researcher
1.1 Detection algorithms are injected onto web pages.
1.2 Each algorithm keeps track of determined sequences of events and actions.
3. This event is asynchronously sent to a remote location.
2. When the use of a tactic is detected a notification is triggered.
4. Reports from users are viewed by interested parties.
ti(){..}
tn(){..}
id timestamp URL tactic
ccgu1331569030153 1333922552190 http://www.bbc.co.uk/sport/ t5
ccgu1331569030153 1333922556391 http://www.bbc.co.uk/sport/ t1
Vigo, M. and Harper, S. (2013) Evaluating Accessibility-in-use. W4A '13, article 7.
![Page 24: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/24.jpg)
Step 3. Deployment intothe wild
Vigo, M. and Harper, S. (2013) Evaluating Accessibility-in-use. W4A '13, article 7.
![Page 25: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/25.jpg)
Our proposal4 step method:
1. Observation and identification of strategies ✔
2. Implementation of algorithms to detect strategies ✔
3. Deployment into the wild ✔4. Run user studies
Go to step 2 until obtaining reliable
results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 26: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/26.jpg)
Step 4. Run user studies 18 participants, 10 days 126 retraces and 67 rechecks Tabbed browsing was interfering
Feedback on false positives:- “I’m browsing across tabs”- “I’m comparing different web pages”- “I’m navigating through different tabs”
Vigo, M. and Harper, S. (in progress) Detecting navigation tactics to cope on the World 'Wild' Web. Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 27: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/27.jpg)
Our proposal4 step method:
1. Observation and identification of strategies ✔
2. Implementation of algorithms to detect strategies ✔
3. Deployment into the wild ✔4. Run user studies ✔
Go to step 2 until obtaining reliable
results
Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 28: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/28.jpg)
Step 2. Implementation ofalgorithms to detect strategies
I61. Re-checking: quick revisitationswebpagei→wpj→wpiwpj
I63. Re-tracing: users retrace the steps in a sequence of pages. wpj→wpk→wpl→wpm→wpl→wpk→wpj→wpk→wpl→wpm
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
NON-TABBED
![Page 29: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/29.jpg)
Our proposal4 step method:
1. Observation and identification of strategies ✔
2. Implementation of algorithms to detect strategies ✔✔
3. Deployment into the wild ✔✔4. Run user studies ✔
Go to step 2 until obtaining reliable
results Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
![Page 30: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/30.jpg)
Step 4. Run user studies 20 participants, 10 days 24 retraces and 16 rechecks
Vigo, M. and Harper, S. (in progress) Detecting navigation tactics to cope on the World 'Wild' Web. Vigo, M. and Harper, S. (2013) Considering people with disabilities as überusers for eliciting generalisable coping strategies on the web. ACM Web Science, 441-444.
I63. Retracing I61. Rechecking
![Page 31: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/31.jpg)
Conclusions
There is an overlap on problems and strategies
More evidence supporting the accessibility continuum
Implications for accessibility-in-use
![Page 32: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/32.jpg)
Future work Ongoing work
- 4th iteration with users- Testing with visually disabled users
2nd stage of the COPE project- Intervening in the interface- Long term objective- Triangulation
![Page 33: Detecting Problematic Interactions on the Web. The COPE Project: Coping strategies on the Web](https://reader038.fdocuments.us/reader038/viewer/2022110302/54951f55b4795917158b45f2/html5/thumbnails/33.jpg)
{
Detecting Problematic Interactions on the Web
markel.vigo@{ehu.es | manchester.ac.uk}
@markelvigoDonostia, 18/11/2013
questions?