The Cocktail Party Effect. An inclusive vision of conversational interactions.
-
Upload
isabella-loddo -
Category
Design
-
view
68 -
download
2
Transcript of The Cocktail Party Effect. An inclusive vision of conversational interactions.
![Page 1: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/1.jpg)
Isabella Loddo Università Iuav di Venezia
NTT Data Italia
Dario Martini Università degli Studi
della Repubblica di S.Marino
The cocktail party effect. An inclusive vision of conversational interactions
![Page 2: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/2.jpg)
Myopia
![Page 3: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/3.jpg)
Diversity
Different People
Different Devices
![Page 4: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/4.jpg)
Interaction Model
Sullivan & Igoe, 2004Model Reality
![Page 5: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/5.jpg)
Impairment
Microsoft, 2016
![Page 6: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/6.jpg)
Captchas
![Page 7: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/7.jpg)
Testing
Benchmarking Reaction
![Page 8: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/8.jpg)
Testing + Feedback
![Page 9: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/9.jpg)
Assistive devices
“For most of us, technology makes things easier. For a person with disability, it makes things possible.”
Judy Heumann
![Page 10: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/10.jpg)
Assistive Technologies
![Page 11: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/11.jpg)
Comparison
BulkyFragile
Expensive
EmbeddedConsistent
Cheap
Higher adoption BRAILLE SCREEN-READERS
![Page 12: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/12.jpg)
Talking Machines
1889
![Page 13: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/13.jpg)
Talking Machines
1939 >
synthesis recording
1927 >
![Page 14: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/14.jpg)
1939 >
synthesis recording
1927 >
…
…
formant speech synthesis
concatenative speech synthesis
Talking Machines
![Page 15: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/15.jpg)
Speech Synthesis
formant speech synthesis
concatenative speech synthesis
no database required
artificial sounding
more intelligible than human speech
large database
natural sounding
intelligible as human speech
![Page 16: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/16.jpg)
Expression
![Page 17: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/17.jpg)
Prosodyphrasing
pitch
loudness
tempo
rhythm Liu, 2006
![Page 18: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/18.jpg)
phrasing
pitch
loudness
tempo
rhythm Hewlett et al, 2006
Prosody
![Page 19: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/19.jpg)
phrasing
pitch
loudness
tempo
rhythm
Prosody
talking shouting
![Page 20: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/20.jpg)
phrasing
pitch
loudness
tempo
rhythm
Oliveira, 2012
Prosodyenhance storytelling
express own speech style
Dlugan, 2012
![Page 21: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/21.jpg)
phrasing
pitch
loudness
tempo
rhythm Hammen et al, 1994
Prosody
![Page 22: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/22.jpg)
Intelligibility
Trouvain, 2007
concatenative
formant
Average 100
com
preh
ensio
n
rate (syllabes / second)
95
90
85
80
75
2 4 6 8 10 12 14 16 18
concatenative
formant
Blind
![Page 23: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/23.jpg)
Speech Rates
Asakawa et al, 2003
50%80%50%80%sighted blind
350
wpm
300
400
450
500
250
200
150
100
50
0
software peak rate
software avg. rate
1.6 X
2.8 X
![Page 24: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/24.jpg)
Relevant Scanning
On the web, scanning tasks are more frequent than linear readings.
![Page 25: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/25.jpg)
75%50%
15%
Mobility
Mobile devices have become the main usage scenario.
100
75
50
25
02009 2010 2011 2012 2013 2014 2015 2016
mobile screen reader usage (%)
![Page 26: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/26.jpg)
Situated listening
Users need to distinguish relevant information in real environments.
![Page 27: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/27.jpg)
Cocktail party effect
1953
![Page 28: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/28.jpg)
Guerreiro et al, 2015
Concurrent voices
10090
70
50
30
10
com
preh
ensio
n
information bandwidth
80
60
40
20
01 X 1,5 X 2 X 2,5 X 3 X 3,5 X 4 X 5 X4,5 X 5,5 X 6 X
1 voiceVoices
2 voices
3 voices
![Page 29: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/29.jpg)
Concurrent voices
information bandwidth is increased equally in sighted and blind subjects
the maximum increase also depends on age
voice differences are preferred, but not essential
are very promising for web searches, news, assisted navigation, multi-touch simulations
Guerreiro et al, 2016
![Page 30: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/30.jpg)
From Reading to Conversation
![Page 31: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/31.jpg)
human level
Word Error Rate
-18% / year
thanks to artificial intelligence cloud computing
25
20
15
10
5
02008 2009 2010 2011 2012 2013 2014 2015 2016 2017
introduction of deep learning
![Page 32: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/32.jpg)
Mutual Recognition
“Normal people, when they think about speech recognition, they want the whole thing. They want recognition, they want understanding and they want an action to be taken.”
Hsiao-Wuen Hon, Microsoft Research
![Page 33: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/33.jpg)
Meaning
![Page 34: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/34.jpg)
Mental models
screen reader: tool-like
vs
voice assistant: human-like
Qvarfordt, 2004
![Page 35: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/35.jpg)
Black Mirror, 2013
Human-like
![Page 36: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/36.jpg)
Human-like?
![Page 37: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/37.jpg)
Affective Interaction
2013
![Page 38: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/38.jpg)
Conclusions
Screen Readers are the most popular assistive technology
1
![Page 39: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/39.jpg)
Conclusions
Naturalness has improved, but blind users still prefer formant synthesis
2
![Page 40: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/40.jpg)
Conclusions
Screen Readers are slow in relevant scanning tasks
3
![Page 41: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/41.jpg)
Conclusions
Screen Readers can be improved by applying the Cocktail Party Effect
4
![Page 42: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/42.jpg)
Conclusions
Voice spatiality and formant synthesis can improve mobile experience
5
![Page 43: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/43.jpg)
Scenario
Doctor
Crossing
Tickets
ParkingVegetablesBank
![Page 44: The Cocktail Party Effect. An inclusive vision of conversational interactions.](https://reader031.fdocuments.us/reader031/viewer/2022030310/58f9a9ab760da3da068b710a/html5/thumbnails/44.jpg)
Thank youQuestions?
ISABELLA LODDO Università Iuav di Venezia NTT DATA Italia [email protected]
DARIO MARTINI Università degli Studi della Repubblica di San Marino [email protected]