Post on 22-Mar-2018
ICONICPROSODY 1
Iconicprosodyinstoryreading
MarcusPerlman
UniversityofWisconsin‐Madison
DepartmentofPsychology
NathanielClark
UniversityofCalifornia‐SantaCruz
PsychologyDepartment
MarleneJohanssonFalck
UmeåUniversity
DepartmentofLanguageStudies
Keywords:prosody;vocalgesture;speechproduction;iconicity
Marcus Perlman
1202 E. Johnson St.
University of Wisconsin, Madison
Madison, WI 53706
Email: mperlman@wisc.edu
ICONICPROSODY 2
Abstract
Recentexperimentshaveshownthatpeopleiconicallymodulatetheirprosody
correspondingwiththeirutterance’ssemanticmeaning(e.g.Shintel,etal.,2006).This
articlereportsfindingsfromastoryreadingtaskthatexpandstheinvestigationoficonic
prosodytoabstractmeaningsinadditiontoconcreteones.Participantsreadstoriesthat
contrastedalongconcreteandabstractsemanticdimensionsofspeed(e.g.afastdrive,slow
careerprogress)andsize(e.g.asmallgrasshopper,animportantcontract).Participants
readfaststoriesatafasterratethanslowstories,andbigstorieswithalowerpitchthan
smallstories.Theeffectofspeedwasdistributedacrossthestories,includinginportions
thatwereidenticalacrossstories,whereasthesizeeffectwaslocalizedtosize‐related
words.Overall,thesefindingsenrichthedocumentationoficonicityinspokenlanguage,
andbearonourunderstandingoftherelationshipbetweengestureandspeech.
ICONICPROSODY 3
1.Introduction
Aspeopletalk,theycommonlyusetheirhandstoproduceiconicgesturesthatare
relatedtothemeaningtheyareexpressingverbally(Kendon,2004;McNeill,1992).
Typically,theiconicformsofthesegesturesareunderstoodtoreflectsomemore‐or‐less
detailedaspectofthesensorimotorimagerythatisassociatedwiththemeaningofthe
spokenutterance.Forexample,considertheutteranceofmycologistDavidArora,inwhich
hedescribesthe“longnicestem”ofaparticularvarietyofmushroom
(http://www.youtube.com/watch?v=VSn3aGGzG1M).Asthespeakerarticulatestheword
“long”,hesimultaneouslydepictsthestemwithhishands,curlingthefingersofhisright
handandtracingthestem’slengthdownwardfromhisraisedleftpalm.
Notably,inthisexample,Aroraalsoappearstodepicttheimageofthelongstem
throughthedynamic“shape”ofhisvoice.Incoordinationwiththemanualgesture,he
salientlyextendsthespokendurationoftheadjective“long”,stretchingthewordiniconic
reflectionofthestem’slength.Thisobservationsuggeststhattheconceptualizationof
length—inthiscaseinstantiatedinthecontextofamushroomstem—canbeiconically
realizednotjustinco‐speechmanualgestures,butalsoiniconicmodulationsofthe
temporalpatterningofspeech.
Recentexperimentalstudiesshowthattheuseoficonicprosodyisnotjustlimited
toisolatedinstancesinassociationwithmanualiconicgestures.Forexample,inthelab,
Englishspeakershavebeenshowntoreliablyproduceiconicmodulationsoftheirspeech
ratewhendescribingthespeedofanevent,andintheirpitchwhendescribingthevertical
ICONICPROSODY 4
movementofanobject(Perlman,2010;Shintel,Nusbaum,&Okrent,2006).Thesestudies
showamotivatedrelationshipbetweenthemeaningthatpeopleareexpressingandthe
prosodicformsthattheyproduceintheirspeech.Specifically,participantstendtotalk
fasterwhendescribingafasteventandslowerwhendescribingaslowevent.Theytalk
withahigherpitchwhenreferringtoanobjectthatismovingupwardandalowerpitch
whenreferringtoonethatismovingdownward.Inthispaper,werefergenerallytosuch
motivatedcorrespondencesbetweentheprosodicformofanutteranceanditsmeaningas
iconic,andnotethaticoniccorrespondencesmaybevariablydetailed,abstract,schematic,
cross‐modal,metaphoric,metonymic,etc.(Similargeneralusageisrecognizedinthe
manualgestureliterature,e.g.,Cienki&Müller,2008;McNeill,2005)
Todate,researchoniconicprosodyislargelyexploratory.Ithasbeeninvestigated
injustasmallsetofsemanticdomains,withtheapplicationofonlyafewmethodsand
analyses.Remarkablylittleisknownabouthowspeakersuseiconicprosodyinthewild.
Yetthisresearchissignificantasitcontributestothegrowingdocumentationofthe
prevalenceoficonicityinspokenlanguages,inadditiontosigned.Inlanguagesacrossthe
world,iconicityisincreasinglyreportedinphenomenalikephonologicalorsound
symbolism,phonesthemes,onomatopoeia,andideophones(Dingemanse,2012;Nuckolls,
1996;Perniss,Thompson,&Vigliocco,2010).Linguistsandanthropologistsdescribe
substantialiconiclexiconsfromBantulanguagesinAfrica(Childs,1994),non‐Pama‐
NyunganAustralianAboriginallanguages(Alpher,2001;McGregor,2001;Schultze‐Berndt,
2001),Japanese,KoreanandSoutheastAsianlanguages(Diffloth,1972;Watson,2001),
QuechalanguagesofSouthAmerica(Nuckolls,1996),andBalto‐Finniclanguages(Mikone,
2001).Moreover,theiconicityintheseconventionalspokenformsrelatestoawiderange
ICONICPROSODY 5
ofmeanings,including,forexample,shape,mannerofmotion,texture,size,brightness,
distance,psychologicalandmentalstates,andtemporalaspect.Theserichiconiclexicons
pointtomoredynamicspokeniconicgestures,suchasiconicprosody,intheprocessesof
theirformation(Perlman,Dale,&Lupyan,2014).
Thepresentstudybuildsonpreviousresearchoniconicprosodyintwoprimary
ways.First,itprovidesfurtherevidenceofspeedrelatedmodulationsofspeechratewithin
amorenaturalisticspeakingtaskthanthatusedbyShintelandcolleagues(2006).Second,
itextendstheinvestigationoficonicprosodytothesemanticdomainofsize.Inboththe
speedandsizedomains,ourresultscontributefinerdetailsrelatedtothetemporal
patterningoficonicprosodyandthekindsofconcreteandabstractmeaningsthatmay
elicitit.
1.1.Iconicityinprosody
Mostpreviousresearchonprosodyinspeechproductionhasnotexaminedhow
variablesassociateddirectlywiththesemanticmeaningofanutterancemightinfluenceits
prosodicform.Althoughboundariesbetweensemanticsandpragmaticsmaybe
contentious,welimitouruseof“semantic”heretomeaningsthatareovertlyexpressedby
thewordsandphrasesofanutterance,suchasmannerofmotion(e.g.fast/slow),theshape
ofanentity(e.g.big/small),oranentity’sspatialpositionanddirectionofmovement(e.g.
up/down).Incontrasttothesekindsofsemanticdistinctions,pastresearchhasgenerally
focusedonquestionsrelatedtohowprosodydemarcatesthesyntacticstructureofan
utterance(e.g.,Ferreira,1993),howitdirectsattentionthroughstressandpitchaccenton
ICONICPROSODY 6
focusedelements(e.g.,Levelt,Roelofs,&Meyers,1999),andhowitexpresses
paralinguisticinformationaboutaspeaker’sinternalstateslikeemotionandattitude(e.g.,
Bolinger,1986;BryantandFoxTree,2002;Cosmides,1983).Overall,thislargebodyof
researchshowsthattheprosodicformofanutteranceissignificantlydeterminedand
constrainedbyamultitudeoffactors,includingthesyntaxandphonologyoftheutterance,
aswellasparalinguisticfactorslikethespeaker’sattention,emotion,andattitude.For
instance,Levelt(1989:180)summarizesthatprosodyisdeterminedbytheinteractionsof
thephrasalorganizationofwordstodetermineintonationalunitboundaries,themoodand
modalityoftheutterancetodetermineintonationalpatternsandboundarytones,andthe
assignmentofpitchaccenttofocusedelements.Accordingtothisview,theprocessesthat
determineprosodyarenotdirectlyinfluencedbytheconceptualizationoftheutterance.
However,somerecentresultsshowthat,undersomeconditions,prosodyisalso
measurablyinfluencedbyvariablesrelatingtothesemanticmeaningoftheutterance.One
pioneeringstudyrecordedparticipantsastheydescribedthedirectionofmotionofan
animateddotonacomputerscreen,usingthecarrierphrase,“Thedotismoving
[left/right/up/down]”(Shinteletal.,2006).Inafirstexperiment,participantswerefound
toincreaseordecreasetheirfundamentalfrequencyastheydescribedanupwardor
downwardmovingdot,respectively.Inasecondexperiment,thedotmovedtotheleftor
right,whilealsotravelingateitherafastoraslowrate.Participantsspokefasterwhen
describingtheleftwardorrightwardmovementoffastmovingdotscomparedtoslow
ones,eventhoughthespeedofthedotwasincidentaltothecommunicativetask.
Anotherstudyinvestigatedwhetheradultsspeakingininfantdirectedspeech(IDS)
wouldmodulatetheprosodyoftheirvoiceastheyspokesentencesrelatingtothe
ICONICPROSODY 7
antonymicpairsofmeaningshappy/sad,hot/cold,big/small,tall/short,yummy/yucky,
andstrong/weak(Nygaard,Herold,&Namy,2009).Threespeakerswereshownanonce
word,like“foppick”or“tillen,”foreachmeaning,andtheninstructedtosay,“Canyouget
the[nonceword]one?”asiftoaninfant.Analysisrevealedconsistentdifferencesalong
differentcombinationsofacousticparameterswithineachpairofantonyms.Forexample,
themeaningbigwascharacterizedbylowerpitch,longerduration,andhigherintensity
thansmall,whiletallwascharacterizedbyalongerdurationandgreaterpitchvariation
thanshort.Theseeffectsweregenerallyfoundatthelevelsofbothwordandsentence.
Speakersalsoproducesemanticallymotivatedmodulationsoftheirprosodyinmore
spontaneouscontexts.Perlman(2010)askedparticipantstowatchaseriesofshortvideo
clipsshowingfastorslow‐pacedeventsandthendescribethemopen‐endedlytoan
experimenter.Onaverage,speakersspokefasteracrosstheircompletedescriptionsoffast
eventscomparedtoslowevents.Inadditiontotheseoveralldifferences,theyalso
produceddistinctmodulationsintempowhenarticulatingadverbialphrasesaboutspeed,
suchasreallyfastorveryslowly.
Adifferentflavorofexperimentaskedparticipantstoproducethevowel/a/ina
go/no‐gotaskinresponsetostimulithatvariedalongthreedimensions:shape(trianglevs.
dodecagon),luminance(whitevs.black),andsize(smallvs.large;PariseandPavani,
2011).Thestudyfoundthatparticipantspronouncedthesyllableindifferentways
accordingtothestimuli,articulatingitwithhigherintensitywhenrespondingto
dodecagonscomparedtotrianglesandwhiteshapescomparedtoblackones,andwitha
higherthirdformantfortrianglescomparedtododecagons.Size,however,wasnotfound
tohaveaneffectonarticulation.
ICONICPROSODY 8
Takentogether,theresultsofthesestudiesshowthatpeoplehaveatendencyto
modulatetheirprosodyiniconiccorrespondencewithcertainmeanings.Yet,ourempirical
knowledgeoficonicprosodyremainsextremelylimited.Forexample,weknowlittleabout
thekindsofmeaningsthatarelikelytoinfluenceprosody.Wealsoknowlittleabouthow
iconicprosodymanifestsinanutterance,includingtheacousticcharacteristicsthatare
mostrelevant,aswellasitstemporalpatterning.
Gesturescholarshaveproposedthatonereasonforthisempiricalgapmaybe
relatedtothemethodologicalchallengeofstudyingiconicprosody(Duncan,2003;McNeill,
2005).Whereasmanualgesturesaredistinctlyidentifiableascommunicativemovements
(Kendon,2004),prosodyisproducedwithinthearticulatorymovementsofconventional
speechforms.Theprosodicformisfurtherdeterminedbyahostofotherfactorssuchas
syntaxandthespeaker’semotional,attitudinalandattentionalstate.Asaconsequence,it
canbechallengingtoidentifyandmeasureiconicprosodyandseparateitfromtheseother
variables.
2.Presentstudy
Towardsovercomingthischallenge,thepresentarticlereportstheresultsfroman
experimentthatusedastoryreadingtasktoinvestigateiconicprosodyinthesemantic
domainsofspeedandsize.Participantswererecordedastheyreadaloudaseriesofshort
storiestoapartner.Onesetofstoriesinvolvedeitherfastorslowspeedofmovement,and
theother,smallorlargesize.Wealsoexaminedwhetherabstractmeaningsmightelicit
iconicprosody,includingstorieswithineachsetthatinvolvedabstractinstantiationsofthe
ICONICPROSODY 9
targetedsemanticdomain(e.g.slowprogressorabigcontract).Wegenerallyexpected
thatparticipantswouldexpressthespeedofthestoriesthroughtherateofspeech,
whereassizewouldbeexpressedthroughitsintonationalpattern.Inparticular,
participantswouldspeakwithafasterarticulationrateinfaststoriescomparedtoslow
ones,andwithahigherpitchinsmallstoriescomparedtobigones.Thetaskalsoallowed
ustoexaminemorepreciselythedynamicsofhowtheconceptsofspeedandsizemightbe
manifestedinthetemporalandintonationalpatternsofspeech,forexample,whether
iconicallyfasterorhigher‐pitchedspeechoccursonlyintheimmediatecontextofwords
like‘fast’or‘small,’orwhetheritisdispersedmorewidelythroughoutanutterance.
Ourpredictionsregardingspeedandarticulationratewerebasedonpriorresearch
showingthatspeakerssometimesproducesloworfastmodulationsofarticulationrate
whendescribing,respectively,asloworfastevent(Perlman,2010;Shinteletal.,2006).
Priorresearchalsomotivatedourpredictionsregardinganiconicrelationshipbetween
sizeandpitch,asinphenomenalikesize‐soundsymbolism(Jacobson&Waugh,1979;
Ohala,1984;Tsur,2006;butseeParise&Pavani,2011).Peopleroutinelyassociatehigh‐
pitchedsoundswithsmallsizeandlow‐pitchedsoundswithlargesize.Inonewell‐
documentedcase,frontvowelswithhighsecondformants(e.g./i/)areassociatedwith
small,whilebackvowelswithlowsecondformants(e.g./ɔ/)areassociatedwithlarge(e.g.
Sapir,1929;andseeUltan,1978fortheprevalenceofthisassociationacrossthelexiconsof
variouslanguages).Additionally,experimentalresultsshowthatadultsspontaneously
modulatetheirpitchasaniconicexpressionofsizewhenreadingsentencesininfant
directedspeech(Nygaardetal.,2009).
ICONICPROSODY 10
2.1.Method
2.1.1.Participants
Sixty‐sevenundergraduatestudents(35forspeed,32forsize)attendingthe
UniversityofCalifornia,SantaCruzparticipatedinthestudyinexchangeforcoursecredit.
Allparticipantswereself‐reportednativespeakersofEnglish.
2.1.2.Stimulianddesign
Eightcarefullymatchedpairsofshortstoriesinstantiatingsemanticcontrastsof
speed(fastvs.slow)andsize(bigvs.small)werecreatedandformattedintoMicrosoft
PowerPointpresentations(SeeTable1).Thefirstslideofeachstorypresentedan
identifyingcharacterandtitle,andthestory’scontentwascontainedinthreesubsequent
slidesthatincludedanintroduction,aplot,andaconclusion.Onlytheintroductionand
plotcontainedwordsexplicitlyreferringtospeedorsize;theconclusionswereidentical
betweenpairedstories.Thusthestoriescontainedasetofcontrastingphrasesthatdiffered
betweenconditions,andtheremainingnon‐contrasting,sharedcontentthatwasidentical
betweenconditions.
Inadditiontothemainsemanticcontrast,thetargetstoriesalsovariedinmapping
type.Twoofthefourstoriesforeachdomainwereconcrete,concerningobservableevents
andphysicalobjects.Forspeed,thismeantrealmovementtakingplaceatameasurable
rate,suchasfastrunningversusslowwalking,andforsize,thismeantlargeorsmall
physicalobjects,likehouses.Theothertwostoriesineachdomainwereabstractand
concernedmoreabstractmeaningsdescribedintermsofspeedandsize,suchascareer
ICONICPROSODY 11
progress(e.g.“afasttracktosuccess”)orthesignificanceofanevent(e.g.,“areallybig
deal”).
Table1:Examplestorymaterialsforspeedcontrast
Fast Slow Syl. Sectionlabelandinfo
Martha Martha Character‐‐notanalyzed
ARapidRun ASluggishWalk Title‐‐notanalyzed
Marthaisafastrunner(4).Shealwaysexercises,andtodaysheisoutthere.
Marthaisaslowwalker(4).Sherarelyexercises,buttodaysheisoutthere.
21 Intro‐‐analyzed,instantiatescontrast,italictextanalyzedforcontrastingphrases
Shetakesoffquickly(4)throughtheneighborhood.Shedashespastsomehouses(7)andcontinuestowardsaschool.Shespeedstoanear‐sprint(5)andreachesthefinalstretch.
Shelaborsslowly(4)throughtheneighborhood.Shestrugglespastsomehouses(7)andcontinuestowardsaschool.Sheslowstoanearstop(5)butreachesthefinalstretch.
37 Plot‐‐analyzed,instantiatescontrast,italictextanalyzedforcontrastingphrases
Sheisreachingtheend. Sheisreachingtheend. 6 Conclusion‐‐analyzed,doesnotinstantiatecontrast
Fivefillerstorieswerealsocreatedforeachsemanticdomain.Thefillersparalleled
thetargetstoriesinstructure,butwereunrelatedtospeedorsizeandalsodifferedin
meaningfromeachother.Thetargetandfillerstorieswerecompiledtogetherinto
PowerPointpresentationsoffourcounterbalancedlistseachforthesizeandspeed
domains.Eachlistcontainedfourtargetstoriesrepresentingthefactorialcombinationof
semanticvalence(eitherfastvs.sloworbigvs.small)andconcreteness(concretevs
abstract).Thesewereinterspersedamongthedomain’sfivefillers,foratotalofnine
storiesperlist.Afulllistofstimuliappearsintheappendix.TableA‐1displaysstories
involvingspeed,andTableA‐2displaysstoriesinvolvingsize.
2.1.3.Procedure
ICONICPROSODY 12
Subjectsparticipatedinthestudyinpairs,butwerescheduledindependentlyand
didnotknoweachother.Onememberofthepairwasassignedtoalistcontainingspeed
stories,andtheothertoalistforsize.Participantsreadtheirrespectivelistofstoriesto
theirpartnerslide‐by‐slideviaaself‐pacedPowerPointpresentation,whichwasdisplayed
tothereaderonalaptopcomputer.Inthreeinstances,onlyoneparticipantarrivedfora
scheduledsession.Inthesecases,theparticipantreadfromoneofthespeedliststoan
undergraduateconfederate.
Tomakethetaskmorecommunicativeandmeaningful,participantswereinstructed
“tobeaninterestingstoryteller,”andweretoldthattheywouldbeansweringquestions
aboutthestoriestheirpartnerreadtothem.Tohelpthereadergainfamiliaritywiththe
storyandengagewithitsmeaning,eachstorywasrepeatedtwiceconsecutivelyinthe
presentation.Thefirsttimethroughparticipantswereinstructedtoreadthestorysilently,
andthesecondtimetoreaditaloudtotheirpartner.Afteronepartnerreadthroughtheir
listofstories,thepairswitchedroles,andthenewreaderreadtheirstoriesfromtheother
domain.Participantsconcludedthesessionbyfillingoutapost‐experimentquestionnaire.
Intotal,atypicalsessionlastedlessthan25minutes.
2.1.4.Analysis
Thereadingswererecordedwithaflatboundarymicrophonethatwassetonthe
tableinfrontofthereaderandconnectedtoadigitalrecorder.Therecordingswere
analyzedusingPraatphoneticanalysissoftware(Boersma,2001).
Foreachstory,theboundariesofsectionsandcontrastingtargetphraseswere
markedinaPraatTextGrid.Disfluencies,suchasfalsestartsandrepetitions,weremarked
ICONICPROSODY 13
andlaterexcisedfromtheanalysis.Therecordedreadings(.wavfile)andassociated
TextGridannotationswerefedintoaPraatscriptthatcomputedthedurationsandpitches
ofeachmarkedinterval,includingbothcontrastingandsharedphrases.Articulationrates
werecalculatedbydividingthetotalnumberofsyllablesoftherelevantportionsbythe
totaldurationofthoseportions(seeAppendixtablesA‐1andA‐2forsyllablecountsof
storysections).
Pitchforthecontrastingintervalsofeachstorywascomputedastheduration‐
weightedaveragepitchofallcontrastingphrases;pitchforthesharedelementswasa
duration‐weightedaverageofthepitchesoftheremaining,sharedintervals.Duration‐
weightedaverageswerecomputedbymultiplyingeachphrase’smeanpitchbyitsduration,
summingtheseproducts,anddividingthesumbythetotaldurationofthephrases.In
Praat,theautocorrelationalgorithmforF0measurementsissensitivetosourcesofnoise,
likeaccidentaldoublingorhalvingofthefundamentalfrequencyduringmodalvoicing,and
inaccuratemeasurementsofamodalvoicing(likecreakyvoicingorwhispering,whichare
inherentlynoisier).DuetothesedifficultiesofaccuratelymeasuringF0,eachparticipant’s
recordingwasanalyzedtwice.Thefirstpassofthepitchtrackerusedthedefaultsettings
ontherawrecordingsfromeachparticipant.Thisservedtoprovideabaselinepitchfor
eachparticipant,basedontheiraveragepitchacrossthewholetask,whichwasusedinthe
secondpassofthepitchtrackertorefinethesettingsformoreaccuratetracking.
Forthesecondpass,severaladjustmentsweremadetotherecordingandtothe
defaultsettingsofPraat’spitchtrackingalgorithm.First,eachparticipant’srecordingwas
low‐passfiltered,withtheupperboundofthepassbandsetateightsemitones(2/3
octave)abovethespeaker’saveragepitchacrossthewholerecording.Thisfilter
ICONICPROSODY 14
guaranteesanabsenceofdoublingfortheupperoctaveofthespeakers’constrainedvocal
range,startingat4semitonesbelowtheirbaselinepitch.Thefilteralsoremovesformants,
reducingthepotentialinfluenceofsupralaryngealarticulationontrackedF0.Further,to
excludeamodal(breathy,creaky)voicingandreducepitchhalving,theSilenceThresholdof
Praat’spitchtrackingalgorithmwassetto0.075(fromadefaultof0.03)andthefloorof
thepitchtrackerwindowwassetto8semitonesbelowthespeaker’spreviously
determinedbaseline(Boersma,1993).
2.2.Resultsanddiscussion
2.2.1.Speedstories
Thedatafromoneparticipantwasdiscardedbecauseofexcessivedisfluenciesand
failuretoreadaportionofastory,leaving34readers.Tables2and3showsummary
statisticsforarticulationratesandpitchesofthesestories.
Table2:Summarystatisticsfromarticulationratedata(syl/s)fromspeedstories
StoryType
Condition Sections Overall Concretestories Abstractstories
Overall 5.00(0.20) 4.79(0.22) 5.20(0.29)
Fast‐‐Mean(SD) Sharedphrases 4.79(0.30) 4.63(0.36) 4.95(0.36)
Contrastingphrases 5.20(0.44) 4.95(0.47) 5.45(0.56)
Overall 4.86(0.20) 4.58(0.25) 5.13(0.33)
Slow‐‐Mean(SD) Sharedphrases 4.67(0.20) 4.37(0.26) 4.97(0.27)
Contrastingphrases 5.04(0.49) 4.79(0.61) 5.29(0.65)
Note:n=34participants.
First,athree‐wayrepeatedmeasuresANOVAwasperformed,witharticulationrate
asthedependentvariableandsemanticvalence(itemsfromfastorslowstories),
concreteness(itemsfromconcreteorabstractstories),andcontrast(contrastingorshared
ICONICPROSODY 15
storyintervals)asfixedwithin‐subjectfactors.Thetestrevealedasignificantmaineffect
forspeed,F(1,32)=4.24,p=.048,partialη2=.11,butnointeractionsofspeedwith
mappingorcontrast.Onaverage,participants’articulationrateswerefasterwhenreading
thefaststoriescomparedtotheslowones.
Table3:Summarystatisticsfrompitchdata(Hz)fromspeedstories
StoryType
Condition Sections Overall Concretestories Abstractstories
Overall 173.13(2.30) 171.50(3.70) 175.21(3.65)
Fast‐‐Mean(SD) Sharedphrases 170.74(3.15) 169.31(3.64) 172.25(5.69)
Contrastingphrases 175.97(3.87) 173.80(6.95) 178.34(6.99)
Overall 172.56(2.29) 172.49(3.39) 172.68(3.65)
Slow‐‐Mean(SD) Sharedphrases 170.82(2.48) 171.37(3.01) 170.33(4.20)
Contrastingphrases 174.36(2.48) 173.72(7.14) 175.18(7.22)
Note:n=34participants.
Athree‐wayrepeatedmeasureANOVAwasalsoperformedwithpitchasthe
dependentvariableandthesamesetoffixedwithin‐subjectfactors.TheANOVAshowed
nomaineffectofspeedandnointeractionsbetweenspeedandtheotherfactors.
Tosummarize,wepredictedthatparticipantswouldspeakwithafasterarticulation
ratewhenreadingstoriesinthefastcondition,butthattheirreadingsoffastandslow
storieswouldnotdifferintermsofpitch.Thispredictionissupportedbyourdata:stories
inthefastconditionwerereadaloudatasignificantlyfasterarticulationratethanstories
inthecorrespondingslowcondition,andtherewerenosignificantdifferencesinpitch
betweenfastandslowstories.Further,thelackofinteractionbetweensemanticvalence
andcontrastsuggeststhatthemodulationsinarticulationratearenotconcentratedwithin
explicitphrasesaboutspeed,butareinsteadmoredispersedacrossthestory.
2.2.2.Sizestories
ICONICPROSODY 16
Threeparticipantswereremovedbecauseofexcessivedisfluencies,leaving29
speakers.Tables4and5showsummarystatisticsofthepitchesandarticulationratesfrom
thesestories.BecausepitchmeasurementsexpressedinHertzaregenerallylog‐normally
distributed(Johnson,1997),Table4reportsthegeometricratherthanarithmeticmeanand
standarddeviation.
Table4:Summarystatisticsforpitchdata(Hz)fromsizestories
StoryType
Condition Sections Overall Concretestories Abstractstories
Overall 170.92(2.03) 173.37(5.54) 168.51(4.00)
Small‐‐Mean(SD) Sharedphrases 172.43(3.36) 175.78(6.82) 169.13(4.27)
Contrastingphrases 169.43(4.37) 170.98(8.47) 167.89(5.33)
Overall 168.15(2.01) 169.44(5.92) 166.87(4.21)
Big‐‐Mean(SD) Sharedphrases 171.76(3.27) 173.74(6.90) 169.79(4.43)
Contrastingphrases 164.62(3.61) 165.24(7.66) 164.00(6.42)
Note:n=29participants.
Table5:Summarystatisticsforarticulationratedata(syl/s)fromsizestories
StoryType
Condition Sections Overall Concretestories Abstractstories
Overall 4.94(0.23) 4.85(0.32) 5.02(0.40)
Small‐‐Mean(SD) Sharedphrases 5.08(0.52) 5.02(0.68) 5.13(0.55)
Contrastingphrases 4.80(0.69) 4.68(0.90) 4.92(0.99)
Overall 4.97(0.23) 4.83(0.21) 5.11(0.42)
Big‐‐Mean(SD) Sharedphrases 4.95(0.55) 4.87(0.63) 5.03(0.68)
Contrastingphrases 5.00(0.43) 4.80(0.76) 5.20(0.44)
Note:n=29participants.
First,athree‐wayrepeated‐measuresANOVAwasperformed,withlog‐transformed
pitchasthedependentvariable,andsemanticvalence(itemsfrombigorsmallstories),
concreteness(itemsfromconcreteorabstractstories),andcontrast(contrastingorshared
phrases)asfixedwithin‐subjectfactors.
Thethree‐wayANOVArevealedasignificantmaineffectforsize,F(1,27)=13.70,p
<.001,partialη2=.33.Therewasalsoasignificantinteractionbetweensizeandcontrast,
ICONICPROSODY 17
F(1,27)=7.22,p=.021,partialη2=.21.Followupt‐testsshowedthat,forcontrasting
phrases,participants’pitchesweresignificantlyhigherinthesmallconditioncomparedto
thebigcondition,t(28)=3.86,p<.001,whileforthesharedphrases,therewasno
significantdifferenceinpitchbetweenbigandsmallstories.
Nextweconductedathree‐wayANOVAwitharticulationrateasthedependent
variable,andthesamefixedfactorsasabove.Theresultsdidnotshowareliablemain
effectofsize,noraninteractionwithmappingtype.Therewas,however,asignificant
interactionbetweensizeandcontrast.Follow‐upt‐testsshowedthat,whilethedirection
ofdifferencewasreversedbetweensharedandcontrastingphrases,neithersharednor
contrastingphrasesofsizestoriesdifferedsignificantlyinarticulationrate.
Tosummarizetheseresults,wepredictedthatparticipantswouldspeakwitha
higherpitchwhenreadingstoriesinthesmallcondition,butthattheirreadingsofsmall
andbigstorieswouldnotdifferintermsofarticulationrate.Insupportofthisprediction,
wefoundthatcontrastingphrasesofsmallstorieswerespokenwithahigherpitchthan
thoseof‘large’stories,overbothconcreteandabstractinstantiationsofsize.Theshared
sectionsofthesestoriesdidnotdifferacrossbigandsmallstories.Thusthepitcheffect
associatedwithbigvs.smallwasrestrictedtotheimmediatecontextinwhichtheconcept
wasmentioned,andwasnotpresentforthepartsofthestorythatweresharedacross
conditions.Forarticulationrate,althoughwefoundaninteractionbetweensemantic
valenceandcontrast,follow‐uptestsfailedtorevealanysignificantpairwisedifferences
betweenthecontrastingorsharedelementsofbigandsmallversions,indicatingno
systematicinfluenceofsemanticvalenceonarticulationrate.
2.2.3.Alternativeexplanations
ICONICPROSODY 18
Althoughinlinewithourpredictions,thefactthatonlycontrastingsizephrases
variedinpitchpresentsthepossibleconfoundthatthedifferenceisanartifactofdifferent
phonologicalcontentbetweentheconditions,ratherthanthecontrastingmeanings.(Note,
however,thatthiscannotaccountforthearticulationrateeffectofspeedstories,which
occurredinthephonologically‐identicalsharedphrasesaswellascontrastingones.)There
areintrinsicpitchdifferencesassociatedwithconsonantvoicing.Forexample,/ta/hasa
higherintrinsicF0than/da/(Ohde,1984),thoughthiseffectissmall,shortinduration,and
localizedtohigh‐pitchedcontextsearlyintheutterance(Hanson,2009).Moreimportantly,
vowelcharacterisalsoassociatedwithintrinsicpitchdifferences.Forexample,/bi/hasa
higherintrinsicF0than/ba/(WhalenandLevitt,1995).Thusitisimportanttotest
whetherthepitcheffectcouldhavebeenduetolow‐levelmechanicsofarticulation,rather
thantothesemanticdifferencesbetweenconditions.
Toaddressthispossibleconfound,wereasonedthat,ifthephonological
characteristicsofcontrastingphrasesareresponsibleforthepitchdifferences,thenphrase
pairswhosephonologicalcontentbiasesthesmallmemberofthepairtowardshigherpitch
shouldhavesubstantiallygreaterdifferencesthanphrasepairswhosephonological
contentisdistributedwithanoppositebias.Forexample,thephrasepair,“itty‐bitty
houses”versus“gargantuanhouses”shouldhavealargerpitchdifferencethanthepair,“a
heftypricetopay”versus“asmallpricetopay.”
Thiswastestedbyfirstquantifyingthedegreeofphonologicalcontrastalongthe
potentiallytroublesomefeatures.Aphonologicalbiasscorewascomputedforeachphrase
pairasaratiooftheoddsofthehigher‐versuslower‐pitchedphonetypesforthesmall
phraseofeachpair,versusthesameoddsforthebigphraseofeachpair(seeTable6;χ2
ICONICPROSODY 19
testsshowednosignificantdifferenceinthedistributionofvowelandconsonanttypes
betweensmallandlargephrases).Forvoicelessness,weusedthecountsofvoicelessand
voicedconsonantsinthesmallandbigphrasesofeachofthe13pairstocalculatetheodds
ratiosofvoicelessconsonantspreferentiallyoccurringinthesmallphraseofeachpair.To
avoidundefinedodds‐ratios,weaddedacountof0.1toanycategorythathadrecordeda0,
andsubtracted0.1fromthecomplementarycategory.(Forexample,thephrase“reallybig
deal”containsnovoicelessconsonants,butitwascountedashaving0.1voiceless
consonantsand5.9voicedones.)Wealsocomputed13similaroddsratiosforvowels,
usingthecountsofhigh‐fronttoothervowelsinsmallversusbigmembersofthepair.
Next,pitchdifferencescoreswerecalculatedforeachphrase.Thefillerstoriesin
eachparticipant’srecordingprovidedabaselinelog‐transformedpitchwithwhichto
normalizeforindividualdifferencesacrossproductionsofagivenphrase,andthenall
participants’productionsofthatphrasewereaveragedtogether.Thedifferencescorewas
computedasthedifferencebetweentheaveragelog‐transformedpitchofthesmall
memberofthepairandtheaveragelog‐transformedpitchofthebigmemberofthepair.
Finally,thesepitchdifferencescoreswereregressedonthelogarithmsofthe
consonantoddsratiosandthevoweloddsratios,andtheirinteraction.Thesethree
predictorsfailedtoaccountforasignificantportionofthevarianceinpairwisedifferences,
multiple‐R2=.09,F(3,9)=0.30,n.s.Thus,theregressionfailedtoprovideevidencethat
phonologicaldifferencesbetweenthephrasepairsofsizestoriesrelatedtotheobserved
differencesinpitchacrossthepairs.
ICONICPROSODY 20
Table6.Distributionofconsonantsandvowelsacrosscontrastingmaterialfromsizestories
Small Big Odds‐ratio
Consonants Voiceless 47 38 1.50
Voiced 52 63
Vowels High‐front 14 14 1.06
Other 17 18
Note:Theoddsratiocomparestheoddsofthehigher‐pitchedfeatureoccuringinsmallvsbigphrases.
Anotherpossibleexplanationforourresultsisthatparticipantsmighthavebeen
consciouslyawareofthespeedandsizemanipulationsinthestories.Giventhe
experimentalcontextofreadingstoriestooneanother,theymighthavethendeliberately
modifiedtheirspeechiniconicemphasisofthesemeanings.However,post‐experiment
questionnairesmitigatethisconcerntosomeextent.Whenparticipantswereaskedwhat
theythoughtthestudywasaboutandwhethertheynoticedanythinginparticularabout
thestories,themajoritymadenomentionofnoticingparticularmeaningsinthestories.
The34%whodidsuggestedthatthestorieshadsomethingtodowithemotionalcontrasts
likehappyvs.sad,andnotasingleparticipantnoticedthesemanticcontrastsofspeedor
size.Thus,theresultsofthequestionnairesuggestthatparticipantsemployedtheseiconic
modulationsspontaneouslyinthetask,withoutconsciousdeliberation.
3.Generaldiscussion
Thepurposeofthepresentstudywastoadvanceinvestigationintothe
phenomenonoficonicprosodyinspeechproduction.Ourstoryreadingtaskfocusedon
meaningsrelatedtotheconceptsofspeedandsize,consideringbothconcreteandabstract
ICONICPROSODY 21
sensesoftheseterms.Wefoundthatwhenparticipantsreadaloudstoriesrelatingto
speed,theirreadingsoffaststoriesweresignificantlyfasterthanslowcounterpartstories.
Thisdifferencewassignificantacrossbothcontrastingandsharedphrases.Thelackof
interactionbetweenspeedandcontrastsuggeststhattheinfluenceofspeedwas
distributedacrossthestory,includingportionsinwhichspeedwasnotexplicitly
mentioned.Participants’readingsofsizestoriesalsoshowedaniconiceffectinprosody,
withsmallstoriesreadwithahigherpitchthanlargestories.Inthiscase,thedifference
waslocalizedtophrasesmakingexplicitmentionofsize;sharedsectionsofsizestoriesdid
notsignificantlydifferinpitch.Notably,wedidnotfindpitcheffectsinthespeedstories,or
articulationrateeffectsinthesizestories.
Asawhole,theseresultsreinforcepreviousresearchshowingthatspeakers
spontaneouslymodulatetheirprosodyiniconicrelationtothemeaningtheyare
expressing.Further,thedistinctionbetweenthewidely‐dispersedarticulationrate
differencesforspeedstories,andthemorenarrowlyconcentratedpitchdifferencesforsize
storiessuggeststhatthemodulationsarenotduesimplytoautomatic,linearactivationof
wordmeanings,whichwouldgenerateonlylocalizedeffectsinthecontrastingphrases.
Rather,itappearsthatthedifferentconceptsofspeedandsizemaylendthemselvesto
differenttemporalpatternsoficonicprosody.Onepossibleexplanationofthesedifferent
temporalpatternsisthattheyreflecttheconceptualizationofspeedasadynamic,ongoing
propertyofeventsthatlaststhroughoutthestories,andsizeasamorestatic,localized
propertyofobjectsandotherentitiesthatisrelevantonlywhentheyareinfocus.
Anadditionalgoalofthestudywastoexaminewhetherpeopleproduceiconic
modulationsofprosodywhenexpressingabstractsensesofspeed(e.g.fast/slowcareer
ICONICPROSODY 22
progress)andsize(e.g.,small/bigdeal).Wefailedtofindsignificantinteractionsof
concretenesswitheitherspeedorsize.Thus,althoughtheconcretesenseselicited
numericallylargereffects,especiallyforspeed,theresultsprovidetentativeevidencethat
abstractmeaningshavethepotentialtoelicitconceptuallymotivatedmodulationsof
prosody.
Whilethesmallnumberofitemsinthepresentexperimentdoesnotpermitmuch
generalization,theseresultsontheuseoficonicprosodytoexpressabstractsensesof
concretetermsoffersomedirectionforfurtherinquiry.Onepossibilityisthatsome
abstractconceptsmaytendtomanifestmoredistinctlyiniconicprosodythanothers.For
example,vocaliconicityforconcreteobjectpropertieslikesize,wherewefoundevidence
thattheactivatedconceptisconcentratedonthearticulationofparticularsize‐related
words,andwhichhasawell‐establishedsymbolicphonetics,maybemoreflexiblyapplied
torelatedabstractconcepts.Alternatively,itmaybethatthevocaliconicityunderlying
moreabstractmeaningsisgenerallyattenuatedindegreeoractivatedwithmore
variability(cf.Bergenetal.,2007).
3.1.Originsandsemanticscopeoficonicprosodyasgesture
Somescholarshavepreviouslyproposedthaticonicmodulationsofprosody,such
asthoseexaminedhere,maybeconsideredqualitativelyasaformofgesture(Liddel,2003;
McNeill,2005;Okrent,2002;Perlman,2010;Shintel,Nusbaum,&Okrent,2006).This
accountposesthatspeakersconceptualizeandexpressnon‐vocaldomainslikespeedand
size,inpart,byiconicmovementsoftheirvocaltract.Indeed,theintroductoryexampleof
ICONICPROSODY 23
DavidAroragesturingandvocalizingtodepictthelengthofamushroom’s“longstem”
hintsatacloseconnectionbetweeniconicmanualgesturesandiconicprosody.However,
thisraisesthequestionofhowiconicprosodymightcometobeincorporatedintoone’s
conceptualizationofnon‐vocaldomainslikespeedandsize.Wehypothesizethatsuch
associationsarelikelytodevelopbetweennon‐vocalconceptsandprosodicqualitiesofthe
voicetotheextentthattheyarecorrelatedinexperience—includingexperiencewith
culturalconventions—andlendthemselvestoabstractstructuralcorrespondences(cf.,
primarymetaphors;Grady,1999).
Forexample,peoplemightdevelopamappingbetweenspeedandspeechrateas
thesedomainsarelikelytobecorrelatedinexperience.Aspeopleactfast,theyare
generallymoreinclinedtotalkfast—acorrelationthatfollowsfromthefindingthat
physiologicalarousalisassociatedwithafasterspeechrate(BanseandScherer,1996).In
supportofthisidea,ithasbeenfoundthatradioplay‐by‐playofliveactioninasoccer
matchroughlytracksthepaceofactioninthegame.Play‐by‐playnarrationexhibitsa
fasterspeechratethanthatofspeechprovidingbackgroundinformationorcolor
commentary,withspeechrateincreasing(accomplishedbyshorterandlessfrequent
pauses)assuspensebuilds,andnormalizingafterdramaticeventslikegoals(Kern,2010).
Thiscorrelationisalsosupportedbyworkonlisteners’processingofspeechrateiconicity,
whichshowsthatlistenersaresensitivetothearticulationrateofthespeech,withshorter
reactiontimesinatrue‐falsejudgmenttaskafterfastspeech,butonlywhenthespeech
describedanurgentorhurriedscenario(ShintelandNusbaum,2008).Theseresultspoint
tohowanacceleratedorslowedarticulationratemightcometoembodyone’smore
generalconceptoffastorslowmotion.Thus,similartotheproductionofmanualiconic
ICONICPROSODY 24
gestures,whenpeoplethinkandtalkaboutfastorslowspeed,theirconceptualization
mightmanifestthroughiconicprosodythatreflectsthiscorrespondence.
Aplausibleaccountalsoexistsfortheexperientialcorrelationbetweenpitchand
size,whichisthoroughlyreviewedbyOhala(1984).Asthebasisforthiscorrelation,the
physicsofsoundentailthatlargerobjectsgenerallymakelower‐frequencysoundsthan
smallerones.Thepatternalsoappliestothepitchofanimalvocalizations,withlarger
animalsgenerallyproducinglowerpitchedsoundsandsmalleranimalsproducinghigher
pitchedsounds(afactlexicalizedinthephrase“pip‐squeak”).Moreover,thisexperiential
correlationhasbeenarguedtobesignificantintheevolutionofritualizedvocalsignalsin
manyvertebrates(Morton,1977).Acrossspeciesasdiverseasdogs,chickadees,
rhinoceroses,andfrogs,lower‐pitchedsoundsareusedforthreatdisplays(whenan
individualmightbenefitfromothers’reactingasifitislargerthanitreallyis),while
higher‐pitchedsoundsareusedinnonaggressivecontexts(whenitcanbebeneficialtobe
smallandnon‐threatening).Thesepatternssuggestthattheassociationbetweenpitchand
sizemaybegainedthroughexperience,butmayalsohaveareinforcingbasisthatis
inheritedfromourevolution.
Thus,throughsuchexperientialcorrelations,people’sconceptsofqualitieslike
speedandsizemightcometobeembodiedinthearticulatorymovementsoftheirvocal
tract.Weproposethat,likemanualiconicgestures,thevocalembodimentofthese
conceptualizationsbecomesactivewhenpeopletalkandthinkaboutconceptslikespeed
andsize,andthaticonicprosodyisproducedwhenthisactivityreachesacertainthreshold
(cf.Hostetter&Alibali,2008).
ICONICPROSODY 25
Consideredtogether,ourresults,alongwiththoseofotherrecentstudies,pointto
severalmappingsthatappeartobeembodiediniconicprosody,includingspeed,
verticality,andnowsize.Yetwhatothersemanticdomainsmightelicittheproductionof
iconicprosody?Comparedtothevisuospatialmodalityutilizedbymanualgestures,some
scholarshavemadetheargumentthatthevocal‐auditorymodalityisseverelylimitedinits
potentialforiconicrepresentation(Armstrong&Wilcox,2007;Hockett,1978;Liddell,
2003;Tomasello,2008;alsoseePinker&Jackendoff,2005onvocalimitation).AsHockett
reasons(1978:274):
Whenarepresentationofsomefour‐dimensionalhunkoflifehastobe
compressedintothesingledimensionofspeech,mosticonicityisnecessarily
squeezedout.Inone‐dimensionalprojection,anelephantisindistinguishable
fromawoodshed.Speechperforceislargelyarbitrary.
Whilesuchreasoningmayseemintuitive,itisuncleartheextenttowhichitisbiasedbythe
methodological(andcasualobservational)challengeofidentifyingiconicprosodyand
separatingitfromthe“non‐iconic”partofthespeechsignal,includingsyntacticand
phonologicalfactors.Indeed,onemightcounterthatspeechtooismultidimensional,
consistingofvariablepropertieslikefundamentalfrequency,intensity,duration,aswellas
morecomplexqualitiesoftimbrelikeharmonics‐to‐noiseratio.
Incontrasttothisview,substantialexperimentalworkinsoundsymbolismand
crosslinguisticresearchontherichsystemsofonomatopoeiainvariouslanguagespointsto
manyotherpotentialdomainsforiconicprosody(Dingemanse,2012;Hinton,Nichols,&
Ohala,1994;Nuckolls,1999;Pernissetal.,2010).Thisresearch,whichimplicatesarolefor
vocaliconicityinthediachronicdevelopmentoflanguages,hasfoundthattheconventional
ICONICPROSODY 26
gesturesofspeechareusedintheiconicexpressionofadisparatearrayofmeanings
relatingtoshape,mannerofmotion,physicaltexture,brightness,distance,andgender,
amongnumerousotherconcepts.Furtherresearchshouldaimtoobservethewaysin
whichthesevariousconceptsmightmanifestmoredynamicallyintheiconicprosodyof
speech(aswellasinexaggerationofthephoneticfeaturesofspeech;Feist,2013;Perlman,
2010).
4.Conclusion
Achallengeforresearchersistodevelopmethodsandanalysesthatfacilitatethe
identificationandmeasurementoficonicprosody,especiallywithinmorenaturalistic
contextsandacrosslanguages.Thegoalofthepresentstudywastoexpandthescopeof
semanticdomainsexaminedinresearchoniconicprosodyandtoenableamorefine‐
grainedexaminationofthephenomenon.Notably,theresultsshowdifferenttemporal
patternsoficonicprosodybetweentheconceptualdomainsofspeedandsize,which
presentsasignificantconsiderationasresearchersseektounderstandhowiconicprosody
mightmanifestoverthetimecourseofanutterance.Moregenerally,thefindings
contributetoasmallbutgrowingsetofstudiessuggestingthaticonicprosodyinthe
expressionofspeedisreadilyelicitedinEnglishspeakersinarangeofcontexts,fromthe
productionofcannedphrasesdescribingamovingdotinacomputerexperiment,to
readingstories,tothespontaneousdescriptionofvideoclips.Thesestudiesadditionally
indicatethaticonicprosodyrelatingtosomeotherconceptualdomains,likesizeand
verticality,mightalsobeelicitedwithsomefacility.
ICONICPROSODY 27
Yetbeyondthesefewexperiments,verylittleisknownabouticonicprosodyinthe
wild.Weknowalmostnothing,forexample,regardingtheonlinerelationshipbetween
manualiconicgestureandiconicprosodyduringtheproductionofspeech,suchastheir
relativefrequencyofoccurrenceorwhethertheymighttendtoco‐occurwitheachother.
Ultimately,ifwearetounderstandtherelationshipsbetweenspeech,gesture,andthe
conceptualprocessesthatareinvolvedintheirproduction,itiscrucialthatwegaina
deeperandmorecomprehensiveunderstandingoficonicprosodyandothervocaliconic
phenomena.
References
Armstrong,D.F.&Wilcox,S.(2007).Thegesturaloriginoflanguage.NewYork:Oxford
UniversityPress.
Banse,R.&Scherer,K.R.(1996).Acousticprofilesinvocalemotionexpression.Journalof
PersonalityandSocialPsychology70(3),614‐636.
Bergen,B.,Lindsay,S.,Matlock,T.,&Narayanan,S.(2007).Spatialandlinguisticaspectsof
visualimageryinsentencecomprehension.CognitiveScience,31,733‐764.
Boersma,P.(1993).Accurateanalysisofthefundamentalfrequencyandtheharmonics‐to‐
noiseratioofasampledsound.InstituteforPhoneticScience,Universityof
Amsterdam,Proceedings17,97‐110.
Boersma,P.(2001).Praat,asystemfordoingphoneticsbycomputer.GlotInternational,5,
341‐345.
ICONICPROSODY 28
Bolinger,D.(1986).Intonationanditsparts:MelodyinspokenEnglish.PaloAlto,CA:
StanfordUniversityPress.
Bryant,G.A.&FoxTree,J.E.(2002).Recognizingverbalironyinspontaneousspeech.
MetaphorandSymbol,17(2),99‐117.
Cienki,A.&Müller,C.(2008).Metaphor,gesture,andthought.InR.W.Gibbs,Jr.(Ed.),The
Cambridgehandbookofmetaphorandthought.Cambridge,UK:Cambridge
UniversityPress,483‐501
Cosmides,L.(1983).Invariancesintheacousticexpressionofemotionduringspeech.
JournalofExperimentalPsychology:HumanPerceptionandPerformance,9,864‐881.
Dingemanse,M.(2012).Advancesinthecross‐linguisticstudyofideophones.Language
andLinguisticsCompass,6,654‐672.
Duncan,S.D.(2003).Gestureinlanguage:issuesforsignlanguageresearch.InK.Emmorey
(Ed.),Perspectivesonclassifierconstructionsinsignedlanguages.Mahwah,NJ:
LawrenceErlbaumAssociates,259‐268.
Ferreira,F.(1993).Creationofprosodyduringsentenceproduction.PsychologicalReview,
100,233‐253.
Feist,J.(2013).“Soundsymbolism”inEnglish.JournalofPragmatics,45,104‐118.
Grady,J.(1999).Atypologyofmotivationforconceptualmetaphor:correlationvs.
resemblance.InG.Steen,R.Gibbs(Eds.),MetaphorinCognitiveLinguistics.
Philadelphia:JohnBenjamins,79–100.
Hanson,H.M.(2009).Effectofobstruentconsonantsonfundamentalfrequencyatvowel
onsetinEnglish.JournaloftheAcousticalSocietyofAmerica,125(1),425‐441.
ICONICPROSODY 29
Hinton,L.,Nichols,J.&Ohala,JJ.(Eds.).(1994).Soundsymbolism.Cambridge,UK:
CambridgeUniversityPress.
Hockett,C.F.(1978).InsearchofJove'sbrow.Americanspeech,53(4),243‐313.
Hostetter,A.&Alibali,M.W.(2008).Visibleembodiment:Gesturesassimulatedaction.
PsychonomicBulletin&Review,15,495‐514.
Jakobson,R.&Waugh,L.R.(1979).Thesoundshapeoflanguage.Bloomington:Indiana
UniversityPress.
Johnson,K.(1997).Auditoryandacousticphonetics.Oxford:Blackwell.
Kendon,A.(2004).Gesture:Visibleactionasutterance.Cambridge,UK:Cambridge
UniversityPress.
Kern,F.(2010).Speakingdramatically:Theprosodyinradiolivecommentariesoffootball
games.InSelting,M.,Barth‐Weingarten,D.,Reber,E.(eds.),ProsodyinInteraction.
Amsterdam:JohnBenjamins,217‐238.
Levelt,W.J.M.(1989).Speaking:Fromintentiontoarticulation.Cambridge,MA:MITPress.
Levelt,W.,Roelofs,A.,&Meyer,A.(1999).Atheoryoflexicalaccessinspeechproduction.
BehavioralandBrainSciences,22,1‐75.
Liddell,S.K.(2003).Grammar,gesture,andmeaninginAmericanSignLanguage.
Cambridge,UK:CambridgeUniversityPress.
McNeill,D.(1992).Handandmind:Whatgesturesrevealaboutthought.Chicago:Chicago
UniversityPress.
McNeill,D.(Ed.).(2000).Languageandgesture.Cambridge,UK:CambridgeUniversity
Press.
ICONICPROSODY 30
McNeill,D.(2005).Gestureandthought.Chicago:ChicagoUniversityPress.
Morton,E.W.(1977).Ontheoccurrenceandsignificanceofmotivation‐structuralrulesin
somebirdandmammalsounds.AmericanNaturalist,111,855‐869.
Nuckolls,J.(1999).Thecaseforsoundsymbolism.AnnualReviewofAnthropology,28,
225‐252.
Nygaard,L.C.,Herold,D.S.,&Namy,L.L.(2009).Thesemanticsofprosody:Listeners’use
ofacousticcorrelatestowordmeaning.CognitiveScience,33,127‐146.
Ohala,J.J.(1984).Anethologicalperspectiveoncommoncross‐languageutilizationofF0
ofvoice.Phonetica,41,1‐16.
Ohde,R.N.(1984).Fundamentalfrequencyasanacousticcorrelateofstopconsonant
voicing.JournaloftheAcousticalSocietyofAmerica,75(1),224‐230.
Okrent,A.(2002).Amodality‐freenotionofgestureandhowitcanhelpuswiththe
morphemevs.gesturequestioninsignlanguagelinguistics.InR.P.Meier,K.
Cormier,&D.Quinto‐Pozos(Eds.),Modalityandstructureinsignedandspoken
languages.Cambridge,UK:CambridgeUniversityPress,175‐198.
Parise,C.V.&Pavani,F.(2011).Evidenceofsoundsymbolisminsimplevocalizations.
ExperimentalBrainResearch,214,373‐380.
Perlman,M.(2010).Talkingfast:Theuseofspeechrateasiconicgesture.InF.Perrill,V.
Tobin,&M.Turner(Eds.),Meaning,Form,andBody(pp.245‐262).Stanford,CA:
CSLIPublications.
Perlman,M.,Dale,R.,&Lupyan,G.(2014).Vocalcharades:Theemergenceofconventions
invocalcommunication.InE.A.Cartmill,S.Roberts,H.Lyn,&H.Cornish(Eds.),The
ICONICPROSODY 31
EvolutionofLanguage:Proceedingsofthe10thInternationalConference
(EVOLANGX).NewJersey:WorldScientific.
Perniss,P.,Thompson,R.L.,&Vigliocco,G.(2010).Iconicityasageneralpropertyof
language:Evidencefromspokenandsignedlanguages.FrontiersinPsychology,1,1‐
14.
Pinker,S.&Jackendoff,R.(2005).Thefacultyoflanguage:What’sspecialaboutit?
Cognition,95,201‐236.
Shintel,H.&Nusbaum,H.C.(2008).Movingtothespeedofsound:Contextmodulationof
theeffectofacousticpropertiesofspeech.CognitiveScience,105(6),1063‐1078.
Shintel,H.,Nusbaum,H.C.,&Okrent,A.(2006).Analogacousticexpressioninspeech
communication.JournalofMemoryandLanguage,55,167‐177.
Tomasello,M.(2008).Originsofhumancommunication.Cambridge,MA:MITPress.
Tsur,R.(2006).Size‐soundsymbolismrevisited.JournalofPragmatics,38,905‐924.
Ultan,R.(1969).Size‐soundsymbolism.InJ.H.Greenberg,C.A.Ferguson,andE.A.
Moravcsik(Eds.)UniversalofHumanLanguage,vol.2:Phonology(pp.527‐568).
Stanford:UniversityPress.
Whalen,D.W.,&Levitt,A.G.(1995).TheuniversalityofintrinsicF0ofvowels.Journalof
Phonetics,23,349‐366.
Acknowledgements
ICONICPROSODY 32
Thestudywasdonewhilethethirdauthor,MarleneJohanssonFalck,wasfundedbythe
SwedishResearchCouncil(VR37127).Hercontributiontowritingthismanuscriptwas
fundedbytheRoyalSwedishAcademyofLetters,HistoryandAntiquities,supportedbya
grantfromtheKnutandAliceWallenbergFoundation(KAW2009.0295).
ICONICPROSODY 33
Appendix1TableS‐1:Speedstimuliwithsyllablecounts
Section Syl. Fasttext Slowtext
Character Martha Martha
Title ARapidRun ASluggishWalk
Intro 21 MarthaisafastrunnerShealwaysexercises,andtodaysheisoutthere.
MarthaisaslowwalkerSherarelyexercises,buttodaysheisoutthere.
ContrastingPhrase 4 afastrunner aslowrunner
Plot 37 Shetakesoffquicklythroughtheneighborhood.Shedashespastsomehousesandcontinuestowardsaschool.Shespeedstoanear‐sprintandreachesthefinalstretch.
Shelaborsslowlythroughtheneighborhood.Shestrugglespastsomehousesandcontinuestowardsaschool.Sheslowstoanearstopbutreachesthefinalstretch.
ContrastingPhrase 4 takesoffquickly laborsslowly
ContrastingPhrase 5 dashespastsomehouses strugglespastsomehouses
ContrastingPhrase 5 speedstoanear‐sprint slowstoanearstop
Conclusion 6 Sheisreachingtheend. Sheisreachingtheend.
Character Bob Bob
Title RacingthroughTraffic InchingThroughTraffic
Intro 18 Bobisamiddle‐agedprofessionalman.Heisdrivingtowork,butseemstobeflying
Bobisamiddle‐agedprofessionalman.Heisdrivingtowork,butseemstobecreeping
ContrastingPhrase 6 butseemstobeflying butseemstobecreeping
Plot 39 Thetrafficislight,andBobquicklyracesdownthehighway,dartinginandoutbetweentheothercars.Hekeepsracinguntilheseestheofficebuilding.
Thetrafficisthick,andBobslowlyinchesdownthehighway,crawlinginandoutbetweentheothercars.Hekeepsinchinguntilheseestheofficebuilding.
ContrastingPhrase 5 Bobquicklyracesdown Bobslowlyinchesdown
ContrastingPhrase 5 dartinginandout crawlinginandout
Conclusion 5 Heisgettingthere. Heisgettingthere.
Character Diana Diana
Title SpeedingDownaRoadtoNowhere LimpingDownaPathtoNowhere
Intro 25 Dianaisawomaninherforties.Herlifehassuddenlytakensomeunfortunateturns.
Dianaisawomaninherforties.Herlifehasgraduallytakensomeunfortunateturns.
ContrastingPhrase 6 hassuddenlytaken hasgraduallytaken
Plot 51 SheisspeedingdownaroadtonowhereTheeconomyisbad.Hercompanyhasgoneoutofbusiness.Herskillsarenolongerneededonthejobmarket.Sheblazesontowardsanuncertainfuture.
SheislimpingdownapathtonowhereTheeconomyisbad.Hercompanyhasgoneoutofbusiness.Herskillsarenolongerneededonthejobmarket.Shehobblesontowardsanuncertainfuture.
ContrastingPhrase 7 sheisspeedingdownaroad sheislimpingdownapath
ICONICPROSODY 34
ContrastingPhrase 4 sheblazeson shehobbleson
Conclusion 6 Herlifeischanging. Herlifeischanging.
Character Peter Peter
Title TheFastTracktoSuccess TheSlowPathtoSuccess
Intro 22 Peterisa35‐year‐oldbusinessman.Heistakingafasttracktosuccess.
Peterisa35‐year‐oldbusinessman.Heistakingaslowpathtosuccess.
ContrastingPhrase 3 afasttrack aslowpath
Plot 41 HiscareerisquicklymovingforwardHeisabouttobepromoted.Hehasanicelifeandapleasanthome.Heisheadingswiftlyintherightdirection.
HiscareerisslowlymovingforwardHeisabouttobepromoted.Hehasanicelifeandapleasanthome.Heisheadingslowlyintherightdirection.
ContrastingPhrase 6 quicklymovingforward slowlymovingforward
ContrastingPhrase 4 headingswiftly headingslowly
Conclusion 6 Heisreachinghisgoals. Heisreachinghisgoals.
Note:"Bob"and"Martha"itemswereconsideredconcrete,while"Diana"and"Peter"itemswereconsideredabstract.
ICONICPROSODY 35
Appendix2
TableA‐2:SizestimuliwithphonoligcalbiasmeasuresforcontrastingContrastingPhrases
Section Syl Bigtext Smalltext C‐Bias V‐Bias
Character Phyllis Phyllis
Title AGiantElephant ATinyGrasshopper
Intro 19 Phyllisisahumungouselephant.Sheisontheplainsearchingforfood.
Phyllisisaminisculegrasshopper.Sheisontheplainsearchingforfood.
ContrastingPhrase 7 ahumongouselephant aminisculegrasshopper 1.00 1.25
Plot 17 Phyllislumbersthroughthegrass,castingamassiveshadowasshegoes.
Phyllishiphopsthroughthegrass,castingatinyshadowasshegoes.
ContrastingPhrase 5 lumbersthroughthegrass hiphopsthroughthegrass 149.50 5.33
ContrastingPhrase 5 amassiveshadow atinyshadow 199.00 1.50
Conclusion 6 Shekeepslookingforfood. Shekeepslookingforfood.
Character Bonnie Bonnie
Title AGargantuanHome AnItty‐BittyHome
Intro 14 Bonnieisathome.Shelivesinaverygianthouse.
Bonnieisathome.Shelivesinaverytinyhouse.
ContrastingPhrase 5 verygianthouse veryteenyhouse 4.00 1.00
Plot 28 Thehugehouseisinaneighborhoodwithothergargantuanhouses.Eachhouseisonagreatbigplotofland.
Thetinyhouseisinaneighborhoodwithotheritty‐bittyhouses.Eachhouseisonalittleplotofland.
ContrastingPhrase 2 hugehouse tinyhouse 199.00 1.00
ContrastingPhrase 6 gargantuanhouses itty‐bittyhouses 398.00 3.50
ContrastingPhrase 6 agreatbigplotofland alittleplotofland 0.50 1.29
Conclusion 5 Thishouseisherhome. Thishouseisherhome.
Character Joe Joe
Title AReallyBigDeal A PrettySmallDeal
Intro 20 Joeisthinkingaboutsigningarecordcontract.Itisareallybigdeal.
Joeisthinkingaboutsigningarecordcontract.Itisaprettysmalldeal.
ContrastingPhrase 4 reallybigdeal prettysmalldeal 1.00 359.40
Plot 23/22 Itwillhaveahugeimpactonhiscareer.Joeismakingamonumentaldecision.
Itwillhavelittleimpactonhiscareer.Joeismakingatrivialdecision.
ContrastingPhrase 5 haveahugeimpact havelittleimpact 4.00 0.60
ContrastingPhrase 8/7 amonumentaldecision atrivialdecision 1.00 1.33
Conc. 5 Hesignsthecontract. Hesignsthecontract.
Character George George
Title AHeftyPricetoPay ASmallPricetoPay
Intro 25 Georgeisageneral.Hehasjustsentabattalionintoheavybattletotakeoverabridge.
Georgeisageneral.Hehasjustsentabattalionintoalightskirmishtotakeoverabridge.
ICONICPROSODY 36
ContrastingPhrase 4 heavybattle lightskirmish 1.00 2.00
Plot 23/22 Thecasualtieswillbevast.Itisaheftypricetopay,buttheenemymustbestopped.
Thecasualtieswillbeslight.Itisasmallpricetopay,andtheenemymustbestopped.
ContrastingPhrase 3 willbevast willbeslight 1.00 1.00
ContrastingPhrase 7/6 aheftypricetopay asmallpricetopay 0.01 0.24
Conclusion 5 Theymovetowardsthebridge. Theymovetowardsthebridge.
Note:"Phyllis"and"Bonnie"itemswereconsideredconcrete,while"Joe"and"George"itemswereconsideredabstract.Note:Consonantbiasscore(C‐bias)istheratiooftheoddsofvoicelessconsonantsinthesmallphrasetotheiroddsinthebigphrase.Vowelbiasscore(V‐Bias)istheratiooftheoddsofhigh‐frontvowelsinthesmallphrasetotheiroddsinthebigphrase.