“Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language...
Transcript of “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language...
![Page 1: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/1.jpg)
“Hello HARLIE”: Enabling Speech Monitoring Through Chatbot Conversations
HEALTH AND BIOSECURITY
Dr. David Ireland
![Page 2: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/2.jpg)
Remote Monitoring Research
Movement & mobility Well-being Voice & Language Medication Legal & ethics
2 | Hello Harlie | David Ireland
![Page 3: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/3.jpg)
Some People Have Trouble Communicating
Conversation initiation
Turn-taking
Topic management
Word-retrieval & memory
3 | Hello Harlie | David Ireland
![Page 4: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/4.jpg)
What about using a chatbot?
Hello Harlie | David Ireland4 |
A computer program designed to simulate conversation with a
human
![Page 5: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/5.jpg)
Research Questions
1.Can chatbots help the health clinicians and researchers?
2.Can chatbots provide people with a way to privately practise their communication?
Hello Harlie | David Ireland5 |
![Page 6: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/6.jpg)
H.A.R.L.I.E. Human and Robot Language Interaction Experiment
•Full voice operation
•Small-talk topics
•Ask for speech tasks
•Remote monitoring
6 | Hello Harlie | David Ireland
![Page 7: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/7.jpg)
Hello HARLIE
7 | Hello Harlie | David Ireland
![Page 8: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/8.jpg)
Conversing with HARLIE
Hello Harlie | David Ireland8 |
![Page 9: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/9.jpg)
Goodbye HARLIE
Hello Harlie | David Ireland9 |
![Page 10: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/10.jpg)
Conversation Features
1.Drive the dialog toward a goal
2.Recall previous conversations
3.Change the subject
4.Keep context
5.Interface with other computers
Hello Harlie | David Ireland10 |
![Page 11: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/11.jpg)
Demo Chat With HARLIE
Hello Harlie | David Ireland11 |
![Page 12: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/12.jpg)
Focus Group Studies
Hello Harlie | David Ireland12 |
![Page 13: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/13.jpg)
Focus Group Studies
Hello Harlie | David Ireland13 |
![Page 14: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/14.jpg)
Hello Harlie | David Ireland14 |
Visualising Interaction
![Page 15: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/15.jpg)
Hello Harlie | David Ireland15 |
Visualising Interaction
![Page 16: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/16.jpg)
Hello Harlie | David Ireland16 |
Visualising Interaction
![Page 17: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/17.jpg)
Large Volume of Useable Data Collected
•156 Users (51 from focus groups)
•390 conversations (30,000+ words)
•13 hours of collected human audio
•20 minutes of collected speech tasks
Hello Harlie | David Ireland17 |
![Page 18: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/18.jpg)
What can we measure?
•Speech / syllable rate
•Vocabulary range
•Mid-sentence pauses
•Fundamental frequency variations
•Vowel articulation
Hello Harlie | David Ireland18 |
![Page 19: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/19.jpg)
Developments to Date
•i-HARLIE (iOS version)
•Human-computer interaction studies
•Clinical workshop
Hello Harlie | David Ireland19 |
![Page 20: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/20.jpg)
Thank you to all contributors• Dr. Jacki Liddle (Occupational Therapist)
• Ms Alana Campbell (Speech Pathologist)
• Dr Dana Bradford (Neuroscientist)
• Ms. Olivia Rushin (Psychologist)
• Dr. Christina Atay (Psychologist)
• Pr. Janet Wiles (Complex Systems / Engineering)
• Pr. David Copland (Speech Pathologist)
• Dr. Adam Vogel (Speech Pathologist)
• Centre of Excellence in the Dynamics of Language for funding.
• Community groups
Hello Harlie | David Ireland20 |
![Page 21: “Hello HARLIE”: Enabling Speech Monitoring Through Chatbot … · Human and Robot Language Interaction Experiment •Full voice operation •Small-talk topics •Ask for speech](https://reader034.fdocuments.us/reader034/viewer/2022050305/5f6e00080aa29b49931f7941/html5/thumbnails/21.jpg)
Australian E-Health Research CentreDr. David Ireland
t +61 7 3253 3604
e [email protected] www.csiro.au
HEALTH AND BIOSECURITY
Thank you