MobiMed: Comparing Object Identification Techniques on Smartphones
-
Upload
distributed-multimodal-information-processing-group -
Category
Technology
-
view
161 -
download
2
description
Transcript of MobiMed: Comparing Object Identification Techniques on Smartphones
Technische Universität München Distributed Multimodal Information Processing Group
MobiMed: Comparing Object Identification Techniques
on Smartphones
Andreas Möller1, Stefan Diewald1, Luis Roalter1, Matthias Kranz2
1Technische Universität München, Germany 2Luleå University of Technology, Department of Computer Science,
Electrical and Space Engineering, Luleå, Sweden
October 15, 2012 NordiCHI, Copenhagen, Denmark
Technische Universität München Distributed Multimodal Information Processing Group
Outline
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 2
Background and Motivation
Scenario and Prototype
User Study
Discussion and Conclusion
Technische Universität München Distributed Multimodal Information Processing Group
Background and Motivation
• Idea of bridging the gap between the physical and the virtual world for easier interaction and additional functionality – Connect physical objects with virtual representations by tags
(Want et al., 1999) – Physical mobile interaction (Rukzio, 2006)
• Investigation and comparison of different interaction techniques done earlier, BUT: – meanwhile outdated technologies (e.g. IR) – older comparisons based on (nowadays) limited hardware
(VGA cameras, small screens, slow mobile CPUs) – new technologies have emerged (e.g. vision-based approaches) – user knowledge and experience has changed
Suggesting a new comparison of (state-of-the-art) interaction techniques
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 3
Technische Universität München Distributed Multimodal Information Processing Group
Outline
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 4
Background and Motivation
Scenario and Prototype
User Study
Discussion and Conclusion
Technische Universität München Distributed Multimodal Information Processing Group
Scenario for Physical Mobile Interaction
• MobiMed: identifying medication packages with the smartphone
• Target groups: active people pursuing a healthy lifestyle, elderly people
• Physical mobile interaction to get information on drugs – package insert – side effects – active ingredients – cross-correlations
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 5
Technische Universität München Distributed Multimodal Information Processing Group
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 6
Investigated Interaction Types
Pointing (tag-less vision-based identification)
Touching (radio tags, e.g. NFC or RFID)
Scanning (visual tags, e.g. bar codes)
Text Input (e.g. name, ID, …)
Technische Universität München Distributed Multimodal Information Processing Group
Excursus: Pointing (Vision-based Recognition)
• Image processing is used to detect visual features of an image
• A query in feature space returns similar images from a reference database
• Good choice of feature type allows very reliable results (e.g. MSER) – High distinctiveness (e.g. by
using text-related features) – Scale invariance (works at
different distances) – Rotation invariance (works at
different angles) • Enabled by rise in mobile CPU
performance (multi-core...)
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 7
Technische Universität München Distributed Multimodal Information Processing Group
Prototype
• Implementation as Android application • 47,000 drugs in query database • 100,000 reference images
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 8
Technische Universität München Distributed Multimodal Information Processing Group
Outline
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 9
Background and Motivation
Scenario and Prototype
User Study
Discussion and Conclusion
Technische Universität München Distributed Multimodal Information Processing Group
Research Questions
• RQ1: What advantages and disadvantages of identification techniques, as presented in MobiMed, can be determined?
– ...in terms of effectiveness? large-scale, online – ...in terms of efficiency? lab
• RQ2: Which method is preferred by users? – ...a priori? large-scale, online – ...after practical use? lab
• RQ3: What potential do people see for MobiMed as a whole? – ...a priori? large-scale, online – ...after practical use? lab
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 10
Technische Universität München Distributed Multimodal Information Processing Group
Methodology
• Online study – Human Intelligence Task at Amazon mTurk – 149 participants
• 74 females, 75 males • 17-79 years (average: 31, standard deviation: 11)
– Questionnaire survey
• Lab study – 16 participants
• 6 females, 10 males • 22-69 years (average: 31, standard deviation: 12)
– Experimental task + Questionnaire survey • Identification of 10 packages
with each of four methods • Within-subjects design, permuted order
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 11
Technische Universität München Distributed Multimodal Information Processing Group
Results: RQ1 (Individual Method Comparison)
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 12
Method Advantages Disadvantages
Scanning Quick, precise, high familiarity
Visual code + camera required, need to find and focus on code
Touching Hassle-free, fool-proof, quick
NFC augmentation and NFC-capable phone required, privacy skepticism
Pointing Intuitive to use, „most human form“ of interaction, works from any angle, works also with catalog/website images, no product tagging required
Computational demand, ambiguous results possible
Text Highest familiarity, accurate, search term flexibility
High amount of typing, misspelling, slow, difficult
Technische Universität München Distributed Multimodal Information Processing Group
Results: RQ1 (Efficiency)
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 13
Technische Universität München Distributed Multimodal Information Processing Group
Results: RQ2 (User Preferences)
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 14
Observations/interpretations: • Touching was only #3 in online survey, but rated best in lab study • Possible explanation: low familiarity (as soon as people used it, they liked it)
-3 = strongly disagree, +3 =strongly agree
Technische Universität München Distributed Multimodal Information Processing Group
Results: RQ3 (Utility of Tool in Scenario)
• Information sources on drugs: – Doctor/pharmacist (75%) – Package insert (69%) – Books/internet (56%)
• Would you be interested in MobiMed as alternative source for drug information? 88%
• Would you use a system such as MobiMed? 82%
• Average amount of money subjects would spend: $8.40 (aged >25: $14.01)
• Suggestions for additional features – Price comparison – Active ingredient analysis – Self-diagnose – Personalized medication
management
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 15
Technische Universität München Distributed Multimodal Information Processing Group
Results: RQ3 (Usability of Prototype)
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 16
Technische Universität München Distributed Multimodal Information Processing Group
Outline
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 17
Background and Motivation
Scenario and Prototype
User Study
Discussion and Conclusion
Technische Universität München Distributed Multimodal Information Processing Group
Discussion and Conclusion
• Physical Mobile Interaction is popular and efficient – Was preferred over conventional (text) search – Was faster than text search
• Touching and Scanning evaluated best – Fastest and most popular physical mobile interaction methods – Touching faster and more popular than scanning in lab study – Scanning more popular in online survey (familiarity)
• Vision-based Search (pointing) as future alternative? – Natural; works for any object (no augmentation needed) – Reliability/speed improvement needed, but almost as fast as scanning
• Best method depends on intended scenario • General demand for medical apps
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 18
Technische Universität München Distributed Multimodal Information Processing Group
Thank you for your attention! Questions?
? ? Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 19
[email protected] www.vmi.ei.tum.de/team/andreas-moeller.html
Technische Universität München Distributed Multimodal Information Processing Group
References
• Slide 3: – Rukzio, E. Physical mobile interactions: Mobile devices as pervasive mediators for interactions
with the real world. PhD thesis, 2006 – Want, R., Fishkin, K., Gujar, A., and Harrison, B. Bridging physical and virtual worlds with
electronic tags. In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, ACM (1999), 370–377.
• Slide 10: https://www.mturk.com/mturk/welcome
• All other images: Microsoft ClipArt 2012
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 20
Technische Universität München Distributed Multimodal Information Processing Group
Paper Reference
• Please find the associated paper at: http://dx.doi.org/10.1145/2399016.2399022
• Please cite this work as follows: • Andreas Möller, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012.
MobiMed: comparing object identification techniques on smartphones. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). ACM, New York, NY, USA, 31-40. DOI=10.1145/2399016.2399022 http://doi.acm.org/10.1145/2399016.2399022
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 21
Technische Universität München Distributed Multimodal Information Processing Group
If you use BibTex, please use the following entry to cite this work:
Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 22
@inproceedings{Moller:2012:MCO:2399016.2399022, author = {M\"{o}ller, Andreas and Diewald, Stefan and Roalter, Luis and Kranz, Matthias}, title = {MobiMed: comparing object identification techniques on smartphones}, booktitle = {Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design}, series = {NordiCHI '12}, year = {2012}, isbn = {978-1-4503-1482-4}, location = {Copenhagen, Denmark}, pages = {31--40}, numpages = {10}, url = {http://doi.acm.org/10.1145/2399016.2399022}, doi = {10.1145/2399016.2399022}, acmid = {2399022}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {object identification, physical mobile interaction, pointing, scanning, touching}, }