MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational...
Transcript of MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND … · 2019. 2. 1. · CUI 2019: Conversational...
MAPPING SPEECH RESEARCH IN HCI: TRENDS, THEMES AND CHALLENGES
LEIGH CLARK UNIVERSITY COLLEGE DUBLIN
@lmhclark@cogsis@hci_ucd
The CogSIS Project
CUI 2019: Conversational User Interfaces conference
22nd-23rd August 2019; Dublin, Ireland
cui2019.com
Mapping theoretical and methodological perspectives for understanding speech interface interactions
CHI 2019 part-day workshop: 4th May; Glasgow, UK
http://speech-interaction.org/chi2019/
UPCOMING EVENTS
Ben Cowan Philip Doyle
Diego Garaialde Justin Edwards
Emer Gilmartin
Trinity College Dublin
Stephan Schlögl
MCI Centre Innsbruck
Jens Edlund
KTH Stockholm
Matthew Aylett
CereProc Ltd
João Cabral
Trinity College Dublin
Cosmin Munteanu
University of Toronto Mississauga
https://www.amazon.com/AmazonBasics-Microwave-Compact-Works-Alexa/dp/B07894S727
MAP OUT:
PUBLICATION TRENDS RESEARCH METHODS RESEARCH THEMES
RESEARCH AIMS
speech interface; voice user interface; voice system; human computer dialog*; human machine dialog*; natural language dialog* system; natural language interface; conversational interface; conversational
agent; conversational system; conversational dialog* system; automated dialog* system; interactive voice response system; spoken
dialog* system; spoken human machine interaction; human system dialog*; intelligent personal assistant
+
ABSTRACT, TITLE & KEYWORD SEARCH
SEARCH TERMS & DATABASES
INCLUSION/EXCLUSION CRITERIA
1181
68
INCLUDE EXCLUDE
Speech focused
Full conference / journal papers
English
Embodiment
No interaction evaluation
Non-full / non- peer reviewed
RESEARCH METHODS
DIRECTION OF COMMUNICATION
User-system dialogue (44)
User input only (16)
System output only (12)
User attitudes 36
Task performance 33
Lexis & syntax 20
Perceived usability 18
System usage 15
User recall 7
Physiological data 3
Other 11
CONCEPTS MEASURED
RESEARCH THEMES
Synthesis
Content
SYSTEM SPEECH PRODUCTION
Keyboard and/or mouse
Digital pen
MODALITY COMPARISON
General production
Addressee identification
Alignment
USER SPEECH PRODUCTION
ASSISTIVE TECHNOLOGY & ACCESSIBILITY
https://www.nationaldeafcenter.org/topics/assistive-technology
Tabletop designs - physicians, deaf patients & interpreters
Mobile interface - limited hand dexterity
Voiced-based browser plugin - blind users
DESIGN INSIGHT
Early design insight - speech to access GUI-based software
Interface for a large-scale game
IPA EXPERIENCE
Disparity between people’s mental models of IPAs & reality of interaction
Human likeness can negatively affect IUX
Embarrassment of public use
Structure of multiple user interaction w/ Siri
CHALLENGES & ONGOING RESEARCH
CHALLENGES & ONGOING RESEARCH
MORE THEORETICAL UNDERSTANDING FOR:
1. LANGUAGE PRODUCTION TO SYSTEMS 2. PERCEPTION OF SYSTEMS 3. DESIGN IN LIGHT OF THESE
Proliferation of humanlike voices in non-human artefacts can create
unrealistic expectations of capabilities
Moore, R. K. (2017). Appropriate Voices for Artefacts: Some Key Insights. In 1st International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots.
POLITENESS & FACE
Politeness linked to concept of face (Goffman, 1952; 1967)
Social self-image dependent on societal norms and rules
Usually best interest to save face
No politeness
Connect… Give each piece a twist… Attach… …so it’s in line with the feet Locate… …so the end is closest to the top of the body
Politeness
Just connect…. Just give each piece a little bit of a twist… Basically, attach…. …so it’s more or less in line with the feet Now just locate…. …the end should be closest to the top of the body
POLITENESS EXAMPLE
https://fineartamerica.com/featured/i-dont-care-if-she-is-a-tape-dispenser-i-love-sam-gross.html
How similar is appropriate?
KEY POINTS
1. Speech HCI fragmented
2. More theoretical development/application can help improve cohesion in the field
3. Theories can help explain & understand, but can also be redefined and re-conceptualised in HCI
4. Current work at HCI @ UCD looking at UCD looking design choices affecting partners models, language production, user perception; humanlike ≠ always good aim
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., ... & Cowan, B. R. (2019). What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. arXiv preprint arXiv:1901.06525.
Clark, L., Doyle, P., Garaialde, D., Gilmartin, E., Schlögl, S., Edlund, J., ... & Cowan, B. (2018). The State of Speech in HCI: Trends, Themes and Challenges. arXiv preprint arXiv:1810.06828.
Murad, C., Munteanu, C., Clark, L., & Cowan, B. R. (2018). Design guidelines for hands-free speech interaction. Mobile HCI 2018 Adjunct (pp. 269-276). ACM.
Clark, L. (2018). Social boundaries of appropriate speech in HCI: a politeness perspective. BCS HCI 2018.
Clark, L., Cabral, J. & Cowan, B.R. (2018). The CogSIS Project: Examining the Cognitive Effects of Speech Interface Synthesis. BCS HCI 2018.
Large, D. R., Clark, L., Quandt, A., Burnett, G., & Skrypchuk, L. (2017). Steering the conversation: a linguistic exploration of natural language interactions with a digital assistant during simulated driving. Applied Ergonomics, 63, 53-61.
Clark, L., Ofemile, A., Adolphs, S. & Rodden, T. (2016). A Multimodal Approach to Assessing User Experiences with Agent Helpers. ACM Transactions on Interactive Intelligent Systems (TIIS), 6(4) 29.
@lmhclark@cogsis@hci_ucd