MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational...
Transcript of MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational...
![Page 1: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/1.jpg)
MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND CHALLENGES
LEIGH CLARK UNIVERSITY COLLEGE DUBLIN
@lmhclark@cogsis@hci_ucd
The CogSIS Project
![Page 2: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/2.jpg)
CUI 2019: Conversational User Interfaces conference
22nd-23rd August 2019; Dublin, Ireland
cui2019.com
Mapping theoretical and methodological perspectives for understanding speech interface interactions
CHI 2019 part-day workshop: 4th May; Glasgow, UK
http://speech-interaction.org/chi2019/
UPCOMING EVENTS
![Page 3: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/3.jpg)
Ben Cowan Philip Doyle
Diego Garaialde Justin Edwards
![Page 4: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/4.jpg)
Emer Gilmartin
Trinity College Dublin
Stephan Schlögl
MCI Centre Innsbruck
Jens Edlund
KTH Stockholm
Matthew Aylett
CereProc Ltd
João Cabral
Trinity College Dublin
Cosmin Munteanu
University of Toronto Mississauga
![Page 5: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/5.jpg)
![Page 6: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/6.jpg)
![Page 7: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/7.jpg)
![Page 8: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/8.jpg)
https://www.amazon.com/AmazonBasics-Microwave-Compact-Works-Alexa/dp/B07894S727
![Page 9: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/9.jpg)
MAP OUT:
PUBLICATION TRENDS RESEARCH METHODS RESEARCH THEMES
RESEARCH AIMS
![Page 10: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/10.jpg)
speech interface; voice user interface; voice system; human computer dialog*; human machine dialog*; natural language dialog* system; natural language interface; conversational interface; conversational
agent; conversational system; conversational dialog* system; automated dialog* system; interactive voice response system; spoken
dialog* system; spoken human machine interaction; human system dialog*; intelligent personal assistant
+
ABSTRACT, TITLE & KEYWORD SEARCH
SEARCH TERMS & DATABASES
![Page 11: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/11.jpg)
INCLUSION/EXCLUSION CRITERIA
1181
68
INCLUDE EXCLUDE
Speech focused
Full conference / journal papers
English
Embodiment
No interaction evaluation
Non-full / non- peer reviewed
![Page 12: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/12.jpg)
RESEARCH METHODS
![Page 13: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/13.jpg)
DIRECTION OF COMMUNICATION
User-system dialogue (44)
User input only (16)
System output only (12)
![Page 14: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/14.jpg)
User attitudes 36
Task performance 33
Lexis & syntax 20
Perceived usability 18
System usage 15
User recall 7
Physiological data 3
Other 11
CONCEPTS MEASURED
![Page 15: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/15.jpg)
RESEARCH THEMES
![Page 16: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/16.jpg)
Synthesis
Content
SYSTEM SPEECH PRODUCTION
![Page 17: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/17.jpg)
Keyboard and/or mouse
Digital pen
MODALITY COMPARISON
![Page 18: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/18.jpg)
General production
Addressee identification
Alignment
USER SPEECH PRODUCTION
![Page 19: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/19.jpg)
ASSISTIVE TECHNOLOGY & ACCESSIBILITY
https://www.nationaldeafcenter.org/topics/assistive-technology
Tabletop designs - physicians, deaf patients & interpreters
Mobile interface - limited hand dexterity
Voiced-based browser plugin - blind users
![Page 20: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/20.jpg)
DESIGN INSIGHT
Early design insight - speech to access GUI-based software
Interface for a large-scale game
![Page 21: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/21.jpg)
IPA EXPERIENCE
Disparity between people’s mental models of IPAs & reality of interaction
Human likeness can negatively affect IUX
Embarrassment of public use
Structure of multiple user interaction w/ Siri
![Page 22: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/22.jpg)
CHALLENGES & ONGOING RESEARCH
![Page 23: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/23.jpg)
CHALLENGES & ONGOING RESEARCH
MORE THEORETICAL UNDERSTANDING FOR:
1. LANGUAGE PRODUCTION TO SYSTEMS 2. PERCEPTION OF SYSTEMS 3. DESIGN IN LIGHT OF THESE
![Page 24: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/24.jpg)
Proliferation of humanlike voices in non-human artefacts can create
unrealistic expectations of capabilities
Moore, R. K. (2017). Appropriate Voices for Artefacts: Some Key Insights. In 1st International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots.
![Page 25: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/25.jpg)
![Page 26: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/26.jpg)
![Page 27: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/27.jpg)
POLITENESS & FACE
Politeness linked to concept of face (Goffman, 1952; 1967)
Social self-image dependent on societal norms and rules
Usually best interest to save face
![Page 28: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/28.jpg)
No politeness
Connect… Give each piece a twist… Attach… …so it’s in line with the feet Locate… …so the end is closest to the top of the body
Politeness
Just connect…. Just give each piece a little bit of a twist… Basically, attach…. …so it’s more or less in line with the feet Now just locate…. …the end should be closest to the top of the body
POLITENESS EXAMPLE
![Page 29: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/29.jpg)
https://fineartamerica.com/featured/i-dont-care-if-she-is-a-tape-dispenser-i-love-sam-gross.html
How similar is appropriate?
![Page 30: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/30.jpg)
KEY POINTS
1. Speech HCI fragmented
2. More theoretical development/application can help improve cohesion in the field
3. Theories can help explain & understand, but can also be redefined and re-conceptualised in HCI
4. Current work at HCI @ UCD looking at UCD looking design choices affecting partners models, language production, user perception; humanlike ≠ always good aim
![Page 31: MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational User Interfaces conference 22nd-23rd August 2019; Dublin, Ireland cui2019.com Mapping](https://reader036.fdocuments.us/reader036/viewer/2022071218/605369f17e9f1261627e8663/html5/thumbnails/31.jpg)
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., ... & Cowan, B. R. (2019). What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. arXiv preprint arXiv:1901.06525.
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., ... & Cowan, B. (2018). The State of Speech in HCI: Trends, Themes and Challenges. arXiv preprint arXiv:1810.06828.
Murad, C., Munteanu, C., Clark, L., & Cowan, B. R. (2018). Design guidelines for hands-free speech interaction. Mobile HCI 2018 Adjunct (pp. 269-276). ACM.
Clark, L. (2018). Social boundaries of appropriate speech in HCI: a politeness perspective. BCS HCI 2018.
Clark, L., Cabral, J. & Cowan, B.R. (2018). The CogSIS Project: Examining the Cognitive Effects of Speech Interface Synthesis. BCS HCI 2018.
Large, D. R., Clark, L., Quandt, A., Burnett, G., & Skrypchuk, L. (2017). Steering the conversation: a linguistic exploration of natural language interactions with a digital assistant during simulated driving. Applied Ergonomics, 63, 53-61.
Clark, L., Ofemile, A., Adolphs, S. & Rodden, T. (2016). A Multimodal Approach to Assessing User Experiences with Agent Helpers. ACM Transactions on Interactive Intelligent Systems (TIIS), 6(4) 29.
@lmhclark@cogsis@hci_ucd