Haskins Laboratories Status Report on Speech Research1989, SR-99/100, 148-161

Physical Interaction and Association by Contiguity inMemory for the Words and Melodies of Songs*

Robert G. Crowder,t Mary Louise Serafine,t and Bruno H. Repp

Six experiments investigated two explanations for the integration effect in memory forsongs (Serafine et at, 1984, 1986). The integration effect is the finding that recognition ofthe melody (or text) of a song is better in the presence of the text (or melody) with which ithad been heard originally than in the presence of a different text (or melody). Oneexplanation for this finding is the physical interaction hypothesis, which holds that onecomponent of a song exerts subtle but memorable physical changes on the othercomponent, making the latter different from what it would be with a different companion.Experiments 1, 2, and 3 investigated the influence that words could exert on the subtlemusical character of a melody. A second explanation for the integration effect is theassociation-by-contiguity hypothesis-that any two events experienced in close temporalproximity may become connected in memory such that each acts as a recall cue for theother. Experiments 4, 5, and 6 investigated the degree to which both successive andsimultaneous presentations of spoken text with hummed melody would give rise toassociation of the two components. The results gave encouragement for both explanationsand are discussed in terms of the distinction between encoding specificity and independentassociative bonding.

Stimuli obviously have multiple features. Twoexamples are that ordinary objects have both colorand shape and that songs have both melody andtext. Questions about memory representations ofthese theoretically separable but seeminglyrelated components of a song-melody and text­motivated our earlier investigations (Serafine,Crowder, & Repp, 1984; Serafine, Davidson,Crowder, & Repp, 1986). We hypothesized that asong might be represented in memory in threeways: (1) independent storage of components (theseparate entities perceived and stored so thatmemory for one is uninfluenced by the other); (2)holistic storage (the two components so thoroughlyconnected in perception and memory that one isremembered only in the presence of the other);and (3) integrated storage (the two components

This research was supported by NSF Grant GB 86 08344 toR. Crowder and by NICHD Grant HD01994 to HaskinsLaboratories. The authors are grateful for the assistance ofWilliam Flack in testing subjects and to Shari R. Speer fordiscussion of earlier versions of this paper.

148

related in memory such that one component isbetter recognized in the presence of the other thanotherwise). The holistic hypothesis is obviouslyfalse in the general case since people oftenrecognize the melodies of familiar songs whenthey hear them performed on solo instruments, orwith unfamiliar verses. What this informalobservation leaves open, however, is whether thememory representation consists of independent orintegrated components.

In earlier studies we reported evidence for whatwe called an integration effect in memory formelody and text. Using a recognition task, wefound that melodies were better recognized whenheard with the same words (as originally heard)than with different words, even when the differentwords fit the melody and were equally familiar tothe subject Similarly, we found that the words ofsongs were better recognized in test songscontaining the original melody than in thosecontaining a different but equally familiar melody.The procedure we employed was as follows:Subjects heard a serial presentation of up to 24

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 149

unfamiliar folksong excerpts, each heard onlyonce. A recognition test followed immediately, inwhich subjects were typically asked to indicate,for each excerpt, whether they had heard exactlythat melody (or text) before, ignoring the currenttext (or melody). The test excerpts consisted of oldsongs (exactly as heard in the presentation) andvarious types of new songs (for example, oldmelody with new words), including a type wetermed mismatch songs-that is, an old melodywith old words that had been paired with adifferent melody in the original presentation. Thecritical comparison, of course, was betweenmelody recognition when old songs were heardand when mismatch songs were heard, that is,when the melody was paired with its originalcompanion as opposed to a different, but equallyfamiliar, companion. This comparison, then,avoided the potentially biasing effect thatcompletely new, unfamiliar words would have onrecognition of a truly remembered melody. Whatwe have termed the integration effect is thefinding that both melody and text recognitionwere better in the case of old songs than inmismatch songs. We concentrated on thefacilitating effect of identical words on recognitionof melodies because recognition of words was insome cases almost at ceiling.

The effect was robust. It was not eliminated byinstructing the subjects, on their initial hearing,to focus attention on the melody (and ignore thewords), nor was it eliminated by hearing adifferent singer on the recognition test than hadsung in the original presentation (Serafine et al.,1984). Moreover, the effect was not due to aparticular experimental artifact, the potentiallyconfusing effect of hearing the melody withseemingly "wrong" words; the wrong words did notmake melody recognition suffer, as against anappropriate baseline, but the right wordsfacilitated it (Serafine et al., 1986).

The integration was not accounted for by asemantic connotation imposed on the melody bythe meanings of the words (Serafine et al., 1986)because the integration effect was found even insongs employing nonsense syllables onpresentation and test. A melody heard only once,then, was better recognized in the presence of itsoriginal nonsense text than with different butequally familiar nonsense. This latter observationseems inconsistent with a meaning-interactionhypothesis that might have considerable intuitiveappeal.

In the present studies, we explore further thesource of the integration effect. Two hypotheses,

not necessarily incompatible, are under test here,the physical interaction hypothesis and theassociation-by-contiguityl hypothesis. The first ofthese asserts that when a song is sung, the wordsimpose subtle effects upon the melody notes,slightly affecting their acoustic properties such asthe onsets, durations, and offsets. We have termedthese effects "submelodic" because they wouldleave unaffected the pitches as they would benotated or conceived in composition. For example,some words might impose a staccato articulationand others a legato phrasing. If this hypothesiswere correct, then a melody sung with oneparticular text would in fact be a somewhatdifferent melody than it were when sung withanother text. It would not be surprising to findthat the melody were then better recognized withthe same words both times than with changedwords. A similar argument could be made fortexts.

The association-by-contiguity hypothesis assertsthat two events that occur in close temporalproximity (contiguously or simultaneously) tend tobe associated in memory, though neither wasnecessarily changed by virtue of having enteredinto this association. If this hypothesis werecorrect, then in the limit text and melody would bejust as well associated if they were experiencedsimultaneously but separately (e.g., words spokenand hummed melodies) as if they were given as asong.

In the present research, the first threeexperiments addressed the submelodic hypothesis(a special case of the physical-interactionhypothesis in which words affect musicalproperties of the melody) and the latter threeexperiments addressed the association hypothesis.All experiments employed our usual generalprocedure: Subjects heard folksong excerptsfollowed immediately by a melody recognition testwhere test items contained controlledcombinations of song components. All experimentsused variations of the musical materials anddesign described below.

General Method

Musical materials were based on 40 Americanfolksongs (from Erdei, 1974, see Serafine et al.,1984, 1986) which, in earlier experiments, wefound were virtually all unfamiliar to our subjects.There were 20 pairs of song excerpts, each pairselected so that melodies and texts wereinterchangeable, having rhythmic compatibility.Figure 1 shows such a pair.

150 Crowder et al.

Interchangeability of melodies and texts wascrucial to the construction of test items in which asong contained a different melody or text thanthat heard originally in the presentation. Thus,each text contained a stress pattern suitable foreither melody, and both texts within a paircontained the same number of syllables. The

exceptions were Song Pairs 11 and 17, where onetext was shorter by one syllable and required thecommon "slur" across two tones (see "sleep" inFigure 1, Melody B). Given interchangeablecomponents, each pair potentially yielded fivetypes of test items, examples of which are shownin Figure 2:

Melody Text

A ~, I! J ( I C r ~r,~ r P' }~...

a When the train comes a-lanK. When the train comes a- lanK·

b Hush a- bye. don't you cry, go to sleep lit- tle babe.

B ~ r' , f' IE" ,:== @ ~b Hush a- bye. don't you cry. go to sleep lit-tie babe.

a When the train comes a- long ,l-/hen the train comes ;:I-lonK.

Figure 1. Sample pairs of songs with interchangeable texts. (Aa and Bbdenote original songs; Ab and Ba denotederivatives.)

SAMPLE PRESENTATION ITEMS SAMPLE TEST ITEMS

go both Jack. Bnd Joe set sall--'cro88 the foUl.•J r I c' r r r I aJOJ

I'm just . poor way- far-ing strang-er.

&t j t . r F I j cO j rJHere comes il blue- bird through the--- win- dow.

a

b

oOne yeAr

'iF Err r rWhat '01111 we do with

I j 0 j rthe old sow's hide--?

Hold mv mule whlle I dance Ja-sey.Hold my mule while [ dance.

~~ J g I 0 U I I JI .OJ

jWho's that t.:ip- ping i3t the win- dow? ,> > >

'T n n J I J J J IOJ

~lilr- y hnd . bn- by, Lord.

Ma- ma buy me a chln-ey doll. Ma- rna buy me a chin-ey doll.

Hold my ,mule whlll! I dance Jo-sey,Hold fII'J mule \lb1le 1 dance.

d 6~ n J J J~ I J J I I

Who's that tap-ping at the win- dow?

Ha- ma buy me a chin-ey doll, Ka- .. buy me a chin-ey doll.

Figure 2. Sample presentation and test items (Code for test items: a - new melody/new words, b - old melody/newwords, c- new melody/old words, d- old melody/old words, mismatched, e- old songs).

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 151

The five types were a - new melody/new words,b - old melody/new words, c- new melody/oldwords, d- old melody/old words, mismatched, e- oldsongs.

In addition, test items might consist of old ornew melodies alone, that is, hummed melodieswithout words. The present experiments employedthree to five types of test items, but in each casethe critical comparison was between old songs andmismatch songs, where the latter items allowed usto test recognition of one component in thepresence of a different component that hadnevertheless been heard in the presentation andwas equally familiar.

The songs were sung in the alto range by thesecond author, recorded onto a master tape anddubbed onto sets of experimental tapes with a 5­sec interval of silence between presentation itemsand a 10-sec response interval after each testitem. A silent metronome set at one beat per secfacilitated performance at an even tempo, and apiano tone (not heard by the subjects) ensuredpitch accuracy at the start of each song.

Subjective tempos across the songs were notuniform, however, due to normal rhythmic andmetric variations (e.g., "double time"). Thepresented songs ranged in total duration fromabout 4 to about 10 sec, with a mean of 6.4 andstandard deviation of 2.01. All songs used C as thetonic, although there were variations in mode(Dorian, major and minor), and starting tone.Only slight alterations were made in the originalfolk melodies or texts (e.g., "across" changed to"cross"), in order to ensure rhythmicinterchangeability of materials.

The same general design was used in allexperiments. The presentation and test sequencesalways utilized the song pairs in the same order.On the presentation tapes, half the songs weremelodies with their original folksong texts (typeAa in Figure 1) and half used the borrowed,interchangeable text (type Ab in Figure 1). Eachmismatch item on the test tapes required twosongs in the presentation sequence (since themelody of one would be tested with the text of theother). Whenever two such songs occurred in thepresentation, they followed one anotherimmediately on the tape. Natural sources ofvariation among these songs include length,nature of the melody, tempo, and subject matter ofthe text, to name only a few characteristics. Thesefactors were completely controlled, however, bycounterbalancing across different subjects groups.

Experiment 1The aim of Experiment 1 was to test the

submelodic hypothesis by employing, on the testtape, songs that contained different texts but thesame phonetic and prosodic pattern as thoseemployed on the original presentation. We derivedphonetically similar texts by translating each textinto a corresponding nonsense text, where vowelswere left intact and consonants were changed to areasonably close phonetic neighbor (/b/=!g/; /k/=/tI,etc.). The presentation consisted of songs withnonsense texts, and the test consisted of obviouslydifferent but phonetically similar texts, that is,the real words. If the submelodic hypothesis werecorrect, the integration effect should be obtained.That is, a melody should be better recognizedwhen it appears with words that are phoneticallysimilar to the nonsense words with which it wasoriginally heard than with words whose phoneticderivatives are equally familiar but had beenheard originally with a different melody.

Subjects heard a presentation of 24 songs withnonsense texts followed by a 20-item melodyrecognition test containing four each of thefollowing types of items:

(a) "old songs" (old melody with real words thatare phonetically similar to the nonsense textheard with that melody in thepresentation).2

(b) "mismatch songs" (old melody with realwords that are phonetically similar to anonsense text heard with a different melodyin the presentation);

(c) new melody/"old words" (new melody withreal words that are phonetically similar to anonsense text heard in the presentation);

(d) old melody hummed, (that is, withoutwords);

(e) new melody hummed.The main question was whether melody

recognition would be better in the "old song" thanin the "mismatch" condition. The hummed testitems provided a baseline for melody recognition.

MethodMaterials

Using the songs described under GeneralMethod, each text was translated intophonetically similar nonsense using the rulesdescribed in Experiment 1 of Serafine et a1.(1986). The following are examples of translatedtexts written following English orthography:

152 Crowder et al.

Yes/no responses with confidence ratings weretranslated into single scores with a theoreticalrange of 1 to 6, where 1 represents very confidentno (did not hear melody) and 6 represents veryconfident yes (did hear melody). The mean ratingsfor "old song," "mismatch song," new melody "oldwords," old hummed melody, and new hummedmelody, respectively, were 3.99, 4.44, 3.56, 4.00,and 2.92. An analysis of variance with subjects asthe sampling variable showed conditions to be a

DesignFive sets' of presentation and test tapes were

constructed, each administered to a differentgroup of subjects. Presentations consisted of 24song excerpts with nonsense texts, and testsequences consisted of 20 items containing realwords, where each of the five types of itemsoccurred 4 times. Across five subject groups, eachpresentation item was tested in each of the fiveconditions. For example, the first presentationitem was tested as an "old song" in one group, as a"mismatch song" in another group, as an oldmelody hummed in another group, and so on.Because each "mismatch song" required hearingtwo songs in the presentation, the presentationtapes consisted of 20 song excerpts plus 4additional ones for the "mismatch" condition.

ProcedureTesting was conducted individually in a quiet

laboratory with tapes heard over loudspeakers.Subjects were instructed to listen to a presen­tation of 24 excerpts that would sound likefolksongs except that the texts had.been changedto nonsense. They were told that their "memorywould be tested later" but not informed that onlymelody recognition would be tested. The testimmediately followed the presentation. Subjectswere told that test items would consist ofhummedmelodies or songs with real words, but in all casesthey were only to indicate whether they had''heard that exact melody before--that is, just themusical portion." Subjects indicated "yes" or "no"on the answer sheet and gave a confidence ratingthat ranged from 1 to 3.

SubjectsTwenty Yale undergraduates with unde­

termined levels of musical training were equallydivided among the five groups.

Results and Discussion

Original:Nonsense:Original:Nonsense:

Cobbler, cobbler, make my shoe.Togglue, Togglue, nate nie choo.Cape Cod girls they have no combs.Tade top berf shey jaze mo tong.

significant source of variance, F(4,76)=9.45, p <.001. The Newman-Keuls procedure was used toidentify which comparisons produced thesignificant overall effects. Evidence that melodyrecognition was above chance was provided by thefact that the mean rating for old melodieshummed (4.00) exceeds that for new hummedmelodies (2.92),p < .01, as well as by the fact therating of "mismatch songs" (4.44) was reliablygreater than the condition with new melodies and"old words" (3.56). However, "mismatch songs"generated a higher mean rating (4.44) than did"old songs" (3.99), contrary to our expectationsbased on the submelodic interpretation of theintegration effect. Thus, phonetically similar "oldsong" texts did not enhance recognition for theoriginal melodies.

In this experiment, then, the submelodichypothesis was not supported. No evidenceemerged that the nonsense words in thepresentation imposed effects on the melodieswhich would allow the phonetically similar realwords to enhance melody recognition. As a test ofthe submelodic hypothesis, however, thisexperiment seemed in retrospect to have beencompromised·by the fact that subjects heard realwords on the test after having heard nonsense inthe presentation. Possibly the surprise of a fullsemantic experience after originally studyingnonsense may have been distracting. InExperiments 2 and 3 we addressed this problem.

Experiment 2One aim of the present experiment was to verify

that the integration effect could be obtained withnewly-constructed nonsense texts that would benecessary for Experiment 3, where the submelodichypothesis was tested again. In an earlier study(Experiment 1 of Serafine et aI., 1986) we hadshown, as noted above, that the integration effectwas robust with nonsense texts of the sort used inExperiment 1 of the present paper. In the presentand following experiments, however, somewhatdifferent rules were employed for the constructionof nonsense texts, and we sought to verify that theresulting new materials would give rise to theintegration effect. Following a presentation ofsong excerpts with nonsense words, a melodyrecognition test employed only three types of testitems:

(a) old songs (old melody, old nonsense words)exactly as heard in the presentation);

(b) mismatch songs (old melody with oldnonsense words that had been sung to adifferent melody in the presentation);

(c) new melody/old words.

Physicallnteradion and Association by Contiguity in Memory for the Words and Melodies of Songs 153

The critical comparison was that between theold song and mismatch conditions. The mainprediction was that melody recognition would beenhanced by the presence of the original oldnonwords (in the old songs) over that obtainedwith the different but equally familiar nonwords(in the mismatch songs).

MethodMaterials

The design of Experiment 2 required onlyeighteen of the 20 song pairs we had available.(Two pairs were omitted on the grounds that theyhad proven at least somewhat familiar to somesubjects in Experiment 1). Because, in Experiment3 reported below, two nonsense texts wererequired for each real text, it was necessary toemploy new rules for translating real words intononsense. The new rules, as follows, were similarto, but more detailed than, those of Experiment 1and allowed fewer deviations. For example, thevoiced/unvoiced distinction was was preservedacross transformations:

(1) Vowels remain the same, and the followingvowel-liquid sequences are treated as intact: lerlas in Mary, larl as in far, /xli as in will, I~rl as inlore, l~rJ as in boy, laul as in how, leIII as in pail,lall as in doll, 1~I1as in awl, IAnz/ as in runs.

(2) Consonants are interchanged according tothe following list of phonetic similarities. Forexample, if /hI occurs in a real word, the twocorresponding nonsense words use Id/ and Ig/,respectively:

Ib/=/d/=/g/InJ=Im!=/IIIr/=/l/=/j/ or Iw/; Iw/=lj/=Irl or /IIIp/=ltJ=/kJ1f/=/9/=1hI or lsi; 191 =/f/=IJI or lsi; 1hI=/91 =/f/; Is/=

IJI=/f/1z/=/v/=161Itr/=/kw/=/pIlIpr/=/tw/=/kIl/kr/=/tw/=/pIlIstJ=/sk/=/splIsIl=IJw/=/frlIskw/=/str/=/fipIl/bIl=/dw/=/grlIsp/=/stJ=/sk/terminal Inl=terminal Iml=terminal IrY

Special cases of translated vowel/consonantcombination:

1a-I1=Ia-m!=Ia-n! (e.g., girl =berm, dern)lir/=/iIl=/inJ (e.g., here = seal=feen)

lend/=/emd/ =/emblIAnJ=/Am!=/A IIterminal l~rY=/~n!=/~m!(e.g., song=

fawn, shawm)terminal lni = /xm! = lIn!

(3) Interior I&-I or Ia-n! is treated as a vowel, butin terminal position is interchanged as follows:1&-1=!-;JnJ=!-;J1I as in anger=aften=able.

(4) A terminal lsi or Iz/, when a plural marker,may be retained (untranslated) if the resultingnonsense is too difficult to pronounce.

(5) The sounds ItJ/ and /~ I are omitted from allreal texts because three suitable phoneticcorrespondences do not exist. Thus, minor changesin some real texts were made (e.g., Joe changed toMoe, chase to run).

The following is an example of a translated text,·written in the form (regular orthography) used bythe singer:

Original: Cobbler, cobbler make my shoeNonsense 1: Poggrel, poggrel nate nie foo.Nonsense 2: Toddwen, toddwen lape lie thoo.

Only one set of nonsense texts was used in thisexperiment.

DesignThe design was comparable to that of

Experiment 1. Three sets of presentation and testtapes were administered to different sets ofsubjects. Across the three subject groups, eachpresentation item was tested in each of the threeconditions: old song, mismatch, and newmelody/old words. The presentation tapesconsisted of 24 songs, and test sequences consistedof 18 items, six each of the three conditions.

ProcedureThe testing procedure was comparable to that of

Experiment 1. Subjects were instructed to listento a presentation of folksong excerpts withnonsense texts, were told that their "memorywould be tested later," and following thepresentation were given the melody recognitiontest in which they were to indicate whether theyhad "heard this exact melody before--that is, justthe musical portion." They were not told whattypes of items to expect on the test except that thenonsense folksongs would be similar to those onthe presentation.

SubjectsFifteen Yale undergraduates with undetermined

levels of musical training were equally dividedamong the three groups.

154 Crowder et al.

Results and DiscussionAs in Experiment 1, responses were translated

into 6-point ratings where 1 represents veryconfident no (did not hear melody) and 6represents very confident yes (did hear melody).Mean ratings for the old song, mismatch, and newmelody/old words conditions were 4.85, 3.63, and2.59 respectively. The results of two omnibusanalyses of variance were significant: Withsubjects as the sampling variable, F(2,28)=42.66,p < .001, and with the 18 test items as thesampling variable, F(2,34)=4l.76, p < .00l.Newman-Keuls tests revealed that melodyrecognition was significantly better in the old songthan in the mismatch condition, both acrosssubjects (p < .01) and across items (p < .01).

Thus the integration effect was confirmed withthese new materials, verifying that the presence oforiginal old words--even nonsense words absentof semantic meaning-facilitates melodyrecognition over that obtained with the differentbut equally familiar words, in the mismatchsongs. Besides vindicating our new stimuli, theresults of Experiment 2 provide welcomereplication for one of our most important previousresults: In this new experiment, mean ratings forthe old song and mismatch conditions,respectively, were 4.85 and 3.63; correspondingmeans from Experiment 1 of Serafine et al. (1986)were 4.47 and 3.76.

Experiment 3The aim of Experiment 3 was to retest the

submelodic hypothesis, which had not beenconfirmed in Experiment 1. To avoid thepotentially distracting use of both nonsense (atpresentation) and real words (at test), which mayhave influenced the outcome of Experiment 1, weemployed two different sets ofphonetically derivednonsense texts, based on the same real words(which were never used in this experiment). Thepresentation consisted of folksong excerpts withnonsense texts. The test consisted of folksongexcerpts whose texts were phonetically similar tothose in the presentation but nevertheless were,in all cases, different nonsense. (As in Experiment1, the phonetic derivative of an old song was calledan "old song," etc.) Test items were of three types:

(a) "old songs" (old melody with nonsense wordsphonetically similar to the old nonsensetext);

(b) "mismatch songs" (old melody with nonsensewords phonetically similar to an oldnonsense text from a different song in thepresentation); .

(c) new melody/"old words" (new melody withnonsense words phonetically similar to anold nonsense text).

If the submelodic hypothesis were correct, thatis, if words impose subtle and memorable effectsupon their melodies, then a melody should bebetter recognized when it is heard with nonsensewords that are phonetically related to thenonsense with which that melody was originallypresented than when heard with nonsense that isnot phonetically related to the original. In otherwords, melody recognition in "old songs" shouldexceed that in "mismatch songs."

MethodMaterials

The materials were those described underExperiment 2. Both sets of nonsense texts,(phonetic derivatives of the original folksongtexts) were employed.

DesignThe design was comparable to that of

Experiment 2, except that the test items, insteadof comprising old songs, mismatch songs, and newmelodies with old words, used the phoneticallyderived "old songs," "mismatch songs," and newmelodies with "old words," where our quotationmarks indicate that exact repetition of the verbaltexts between learning and test never occurred. Asin earlier experiments, counterbalancing acrosssubjects groups was employed to control fornatural variations in the songs. The presentationconsisted of 24 items and the tests consisted of 18items.

After 12 of the 30 subjects had been tested, aninadvertent error in the test tapes was detected.Two song pairs containeci faulty material for thecondition new melody/"old words," although theother two conditions were correct. Thus, scores forthose 12 subjects were based on four (instead ofsix) items in the new melody "old words"condition.

ProcedureThe procedure was analogous to that of

Experiments 1 and 2. At test, subjects were toldthat the texts of songs may sound similar to ordifferent from those heard before, but they were toattend only to the melody and indicate recognition(yes or no) and a confidence rating on the answersheet.

SubjectsThirty adults with undetermined levels of

musical training were paid to participate andwere equally divided among the three groups.

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 155

Results and DiscussionAs before, melody recognition ratings had a

possible range of from 1 to 6. Means for the "oldsong," "mismatch," and new melody with "oldwords" conditions were 4.26, 3.88, and 3.05,respectively. With subjects as the samplingvariable, the result of an analysis of variance wassignificant, F(2,58)=28.18, p < .001, and Newman­Keuls tests indicated that melody recognitionunder the "old song" condition was significantlybetter than that under the "mismatch" condition,p < .05.

With items as the sampling variable, ananalysis of variance was performed on meansgenerated only by the 18 subjects who hadcompleted all items in all conditions. Thosemeans, for the "old song," "mismatch," and newmelody "old words" conditions respectively, were4.10, 3.71, and 3.04. The main effect wassignificant, F(2,34)=12.02, p < .001, and a prioricomparisons involving only the first twoconditions revealed significance at the .02 level.(The results ofpost hoc tests were not significant,however.)

The results of the present experiment show thatthe integration effect is obtainable withphonetically similar nonsense used at test. Oneplausible explanation for this result is that wordsexert specific, albeit subtle, and memorable effectson the melodies with which they are sung. Thesesubmelodic effects include the manner in whichconsonants (perhaps also vowels) affect the onset,duration, and offset of particular melody tones. Inour view, it seems indisputable that words exertvariable effects on melody tones, as can be easilyimagined, for example, in the case where twotones accompany the words "tip-top" as opposed to"ho-hum."

We think it not an accident that the presentexperiment showed evidence favorable to thesubmelodic hypothesis, whereas earlier effortswith a similar experimental design did not. Therules for deriving phonetically-similar nonsensetexts were more fastidious here than those usedbefore: For example, in these new materials werespected the voiced/voiceless distinction moreconsistently than under the old rules. Consonantswith stop closures were. distributed equally in theoriginal and derived versions, too. Thesedistinctions are just the sort that would beexpected to underlie a submelodic effect of wordson music.

Other interpretations of the integration effect,for example those to be considered below, mightalso be consistent with the evidence adduced here

for the submelodic hypothesis. ComparingExperiments 2 and 3 of the present series, we notea smaller, and statistically weaker integrationeffect in the latter, with the derived nonsensewords, than in the former, with the very samenonsense texts presented at learning and test.This is as it should be, by any commonsense view,for no scheme for deriving "similar" phonetic textscould possibly be as faithful a reinstatement ascomplete identity. On the other hand, we shouldnot exaggerate the triumph of the submelodichypothesis: At most, we can claim that we haveshown conclusively that some such factor isoperating somehow in our integrationexperiments, not that it is an answer as to thecomplete cause of the effect.

Introduction to Experiments 4 through 6The remaining three experiments investigated

the degree to which the melodies and texts mightbe associated in memory because of their closetemporal proximity, as successive events inExperiments 4 and 5, and as simultaneous eventsin Experiment 6. These experiment address whatwe referred to above as the association-by­contiguity hypothesis. The term association, byitself, may connote many things theoretically,such as rote learning, Pavlovian conditioning, andpre-cognitive, antediluvian mists of antiquity.However, its denotation is theoretically empty: Itsimply stands for an experimental fact, thatevents A and B stand in a particular empiricalrelationship because of their history of co­occurrence. The challenge for theory is torationalize the circumstances necessary for thatassociation to be formed and the nature of thebonding thereby achieved. Thus, our integrationresult illustrates some form of association,without doubt. The submelodic mechanism, forwhich we adduced some support in Experiments 1to 3, is not strictly an associative mechanism atall, but rather an effect of one element upon thenature of the other, namely that the occurrence ofA with B changed the physical nature of B. Wenow ask whether the temporal contiguity of A andB, words and melody respectively, is a sufficientcondition for their association when no possibilityexists for an overt influence of one upon thephysical integrity of the other, as with thesubmelodic mechanism.

In considering the theory of associations wehave relied upon the distinction in the respectivepsychologies ofJames Mill and ofJohn Stuart Millbetween mental compounding and mentalchemistry (see Boring, 1957, chapter 12). In the

156 Crowder et al.

former case two components retain theirindependent identities, yet are connected to oneanother. In the latter case the two components arethemselves altered by each other's presence. Ourconcept of the association-by-contiguity of melodyand text is like that of mental compounding: themelody and text are connected in memory, henceact as recall cues for each other, yet each is storedwith its independent integrity intact. By contrast,the submelodic hypothesis in the first threeexperiments is consistent with a somewhat morechemical form of bonding, for a melody and textchange each other physically when sung togetherin a song. A more purely mental chemistry couldbe an associative process in which, by co­occurrence in the mind, the memoryrepresentation of each is changed as against whatit would have been without a particularcompanion.

More recently, a similar distinction has beenarticulated by Horowitz and Manelis (1972), albeitwith a linguistic orientation, for adjective-nounphrases. They refer to a distinction between [­Bonding (where I stands for individual orindependent) and J-Bonding (where J stands forjoint). The former, illustrated by the phrases deep­chair or dark-wing, take their meaning as aphrase from the meanings of the constituentwords. The latter, illustrated by high-chair orright-wing, possesses idiomatic meaning thattranscends the meanings of the severalconstituents. As Horowitz and Manelis remarked(p. 222), I-Bonding owes allegiance to the Britishempiricist philosophers and J-Bonding to theGestalt tradition. Tulving's work on recognitionfailure in episodic memory (Tulving, 1983)illustrates the same properties as J-Bonding,wherein an element of an association can be onlypoorly recognized but can be well recalled giventhe original associate as a cue. In many ways we

. believe that these issues are raised in their moststark relief when the two constituents, such aswords and melodies, are fundamentally differentcognitive elements than when intraverbalassociations are at stake.

Experiment 4The present experiment investigated the

concept of contiguity as a sufficient condition forassociation. It assessed the degree to which a textcould serve as the retrieval cue for a melody, whenthe two had initially been heard in close temporalproximity (in this case successively), yet not as aproper song. Each component in Experiment 4·was strictly independent physically: Texts were

spoken and melodies were hummed. Using atechnique similar to those reported above, we gavesubjects a serial presentation of spoken texts, eachfollowed by its corresponding melody, hummed,and then each text-melody pair was followed by a10-sec interval of silence during which subjectswere to '"imagine" the song. A melody recognitiontest followed in which true (sung) songs andhummed melodies were heard. If subjects hadmanaged to imagine the songs, as instructedduring original presentation, they should havebehaved in the same way as subjects in our earlierexperiments, who had actually heard the songs.The test items were of five types (quotation marksindicate a deviation from what was heard in thepresentation):

(a) "old songs" (the text and melody of one pairfrom the presentation were sung together asa song in testing);

(b) "mismatch songs" (the text from one pairand the melody from a different pair weresung together as a song);

(c) "new melody/old words" (the text from onepair heard in the presentation was sungwith a new melody);

(d) old melody hummed (exactly as heard in thepresentation except not preceded by words);

(e) new melody hummed.

The main question was whether the "old song"condition would generate better melodyrecognition than would the "mismatch" condition.Such an advantage could derive from either of twoprocesses, corresponding to 1- or J-Bonding in theterminology of Horowitz and Manelis (1972). If thesimple contiguity hypothesis were correct, wewould expect that melodies and texts presented inclose succession would be connected in memory,hence could act as recall cues for one another.Thus, in a melody recognition test consisting oftrue (sung) songs, the melody should be betterrecognized when heard with the textwith which itis connected than with a different (mismatched)but equally familiar text. Likewise, as we saidabove, if subjects are able to fuse the melodies andtext mentally, using the 10 sec "imagine" period asthey were instructed, to create a song-like memoryrepresentation, then, too, the old song conditionwould produce better melody recognition than themismatch condition, as in the earlier experiments.Thus, the conditions of this experiment could notpermit a choice between these two hypotheses.That would require consultation of still furtherexperimental arrangements. However, a positive

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 157

outcome here would be a necessary, if insufficient,condition for either hypothesis.

Method

MaterialsThe 20 pairs offolksong excerpts were used with

their real (not nonsense) words. Tape recordings ofspoken texts and hummed melodies were made bythe same female alto that was the singer in ourearlier studies. Each spoken text generallyfollowed the rhythm of its companion melody,consumed approximately the same amount oftime, and had the character of poetic speechrather than normal, conversational speech.

DesignThe design was comparable to that of

Experiment 1, with five sets of tapes used tocounterbalance the test conditions for eachpresentation item across five subject groups. Thepresentations consisted of 24 text-melody pairs,where the spoken text and hummed melody ineach pair were separated by a one-sec interval andfollowed by 10 sec of silence. The test consisted of20 items, four each assigned to the five conditions.

ProcedureThe procedure was generally the same as that of

the earlier experiments, except that subjects weretold they would hear spoken texts and hummedmelodies from simple folksongs and that theywere to use the 10-sec interval to "imagine thewords and melody together as though someonewere singing them" and to "sing the song in yourhead." On the test, subjects were told to expecteither true (sung) songs or hummed melodies andto indicate melody recognition as in the otherexperiments.

SubjectsTwenty five adults with undetermined levels of

musical training were divided equally among thefive groups.

Results and DiscussionAs in earlier experiments, melody recognition

ratings had a theoretical range of 1 to 6. Meanratings respectively for the "old song," "mismatchsong," "old words/new melody," old melodyhummed, and new melody hummed were 4.24,4.08, 3.63, 4.28, and 3.23. The result of an analysisof variance with subjects as the sampling variablegave a statistically significant main effect ofcondition, F(4, 96) =5.38,p < .001. Newman-Keulstests showed that melody recognition, on its own,was better than chance; there was a significant

difference in the baseline comparison between themeans for old and new hummed melodies (4.28and 3.23 respectively), p <.05. This conclusion isbolstered by the fact that the melodies of "oldsongs" yielded a higher mean rating (4.24) thandid "new melodies with old words" (3.63), p < .05.However, no integration effect for texts andmelodies occurred. That is, the mean rating for"old songs" (4.24) was not significantly higherthan that for "mismatch songs" (4.08). There was,then, no advantage for melody recognitionconferred by the presence of original words overdifferent but equally familiar words.

Thus, we can report no evidence that a melodyand text heard in succession are better recognizedlater in each other's presence, even wheninstructions had been given to imagine the twopresentation components as a song. Severalreasons could possibly underlie the failure to findan association effect here. The problem may havebeen in the generation process, for one thing:Subjects may have been simply unable to imaginethe combined components as a unified song. Or,the imagination instructions themselves maysomehow have served to distract the subjects fromencoding the two components even as "normal"paired associates. Further, in this experiment anunprecedented inconsistency existed betweenwhat was heard in input (successive spokenspeech and hummed melodies) and what wastested for recognition in output (sung songs); thismay have been distracting. And of course it ispossible that spoken and hummed stimuli of thesort employed here are not conducive to either aprocess of song generation or of contiguousassociative bonding.

A special circumstance introduced by spokenspeech and hummed melodies is the introductionof elements in each stream that are foreign to theidentity of these as components within normalsongs: The prosody of normal speech necessarilyintroduces intonation gestures (phrasaldeclination for example) that would not becompatible with the sung version. The act ofhumming, likewise, cannot but introduce nasaland vowel segments that might otherwise notreside in the spoken text. Therefore, ourabstraction of the spoken and melodic streams inthe methodology of this set of studies is not anabsolutely neutral operation. As so often, negativeresults are ambiguous, but positive results (seebelow) speak with considerably greater force.

In our next experiment we focused on thepossibility that the instruction to generate songs,at presentation, had backfired even to the extent

158 Crowder et ai.

of preventing the formation of independent (1­bonded) associations.

Experiment 5Experiment 5 differed from Experiment 4 in two

ways: First, no instructions for imagining songswere given at presentation. Second, test itemscontained the same format of successivecomponents as had the presentation, that is,spoken words followed by hummed melodies.Abandoning the imagery instruction was intendedto determine whether a melody and text couldbecome independently associated in memory if thetwo components had been in close temporalproximity. Having established baseline melodyrecognition in Experiment 4, we saw no need torepeat the two hummed-melody conditions. Themelody recognition test consisted of successivetext/melody pairs under the following threeconditions:

(a) old songs (a spoken text followed by itshummed melody, exactly as heard in thepresentation); 3

(b) mismatch songs (a spoken text followed bythe hummed melody of a differenttextJmelody pair from presentation);

(c) old text with a new melody that had notbeen heard in the presentation.

MethodMaterials

Eighteen of the 20 song pairs used inExperiment 4 were used in the presentexperiment.

Design

The design was comparable to that of previousExperiments 2 and 3. Three sets of tapescounterbalanced the test conditions for eachpresentation item across three subject groups. Thepresentation consisted of 24 textJmelody pairs,and the test consisted of 18 textJmelody pairs, 6each of the three conditions.

ProcedureThe procedure was comparable to the earlier

experiments. Subjects were told to expect pairs ofspoken texts and hummed melodies on thepresentation and test. Melody recognition ratingswere obtained as before.

SubjectsFifteen adults with undetermined levels of

musical training were equally divided among thethree groups.

Results and DiscussionAs before, melody recognition ratings had a

theoretical range of 1 to 6. Means for the old song,mismatch, and old words/new melody conditionswere 4.22, 4.12, and 2.98 respectively. Clearly, nosignificant difference emerged between the oldsong and mismatch conditions. In other words,hummed melodies were not better recognizedwhen preceded by the same spoken text which hadpreceded that melody at presentation than whenpreceded by a different (yet equally familiar) text.Thus, no evidence was found that a link inmemory is engendered by the successivepresentation of independent texts and melodies.This leaves open the possibility that mentalcompounding is not the agency for the integrationeffect between melody and text reported in ourearlier experiments. If not, then the submelodichypothesis remains the only explanation for theeffect with evidence in its favor. However, alingering question is whether a simultaneouspresentation of spoken text and hummed melodycould give rise to an association in memory of thetwo components. This was addressed in thefollowing experiment.

Experiment 6The objective of this experiment was to assess

the degree to which a simultaneous presentationof spoken words and hummed melody could giverise to an integration of the two such that themelody was recognized better in the presence ofthe text with which it had originally beenpresented than in the presence of a different(equally familiar) text. This result would indicatethat our reasoning about independent associativebonding had been correct, in Experiments 4 and 5,but our realization of contiguity had beeninadequate.

The presentation episodes consisted of normalspoken texts and hummed melodies heardsimultaneously and binaurally (but notdichotically). We refer to these simultaneouspairings as "spoken songs." The later recognitiontests were of two types: Half the subjects heardonly spoken songs (as in presentation) and halfheard true, sung songs. Again, no instruction forthe generation of song-like representations wasgiven. So the question at hand was whether anassociation between contiguous components, if itoccurred, would influence melody recognition onlyif the test stimuli were like those of thepresentation or whether that association'sinfluence would extend also to the case of truesongs.

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 159

The melody recognition tests for both the(between-subject) conditions with spoken songsand true songs consisted of three within"subjectconditions. As before, the critical comparison wasthat between old songs and mismatch songs.

(a) old songs (same text and melody as washeard in the presentation);

(b) mismatch songs (the text of one pair and themelody of a different pair heard in thepresentation);

(c) old words with new melody.

Method

MaterialsEighteen of the 20 song pairs were used.A master tape, from which experimental tapes

were dubbed, was prepared by the same altosinger, as follows: Hummed melodies were firstrecorded in succession, each preceded by exactlyfour evenly spaced taps, also recorded onto thetape. The resulting signal was then fed into asecond tape recorder at the same time that spokentexts were recorded o~to a second tape. The singerlistened to the hummed melodies from the firsttape over headphones, using the four taps to fixthe onset of the hummed melody, and then spokethe text along with the melody, recording bothonto the second tape. Texts were generally spokenin the rhythm of the melody and also began andterminated in synchrony with it. Whenexperimental tapes were dubbed from the master,the four taps were omitted. The test tapesemploying true songs were the same as thoseemployed previously with these materials.

DesignThe design was exactly analogous to that of

Experiment 2, except that two sets of test tapeswere constructed, each administered to a differentgroup of subjects, one set with spoken songs andthe other with true songs.

ProcedureThe procedure was comparable to that of the

earlier experiments, with subjects told to expectthe spoken texts of simple folksongs to be heardsimultaneously with hummed melodies. At test,one group was told that items would be true, sungsongs, and the other group that test items wouldbe similar to presentation items. In all cases, ofcourse, instructions called for recognition basedonly on the melodies.

Subjects

Twenty-four adults with undetermined levels ofmusical training were equally divided between thetwo test groups.

Results and DiscussionMelody recognition ratings had a theoretical

range of from 1 to 6. Mean ratings for old songs,mismatch songs, and old words/new melodyrespectively were 4.56, 4.04, and 3.33 when thetest consisted of spoken. songs (as heard in thepresentation) and 4.35, 3.96, and 3.25 when thetest consisted of true songs. Two mixed analyses ofvariance were performed with type of test (spokenvs. true songs) as a between-subjects variable andthe same three conditions ("old song," "mismatch,"and new melody/"old words") as a within-subjectsvariable. With subjects as the sampling variable,only the main effect of conditions was significant,F(2,44)=21.68, p <.001; neither the type. of testmain effect nor the interaction was significant.The Newman-Keuls test indicated that combined"old song" ratings for both groups (4.46) exceededthat for "mismatch songs" (4.00), p < .05.Similarly, with items as the sampling variable,only differences among the three conditions weresignificant, F(2,68)=13.65, p < .001. The Newman­Keuls again test supported the difference between"old song" and "mismatch" ratings, P < .05.

Thus, by the reasoning of the fourth and fifthexperiments here, true temporal contiguity wasthe necessary condition for observing ourintegration effect. The most straightforwardinterpretation of that result is that, in Experiment6, conditions were favorable for the formation ofindependent associative links betweenconstituents that had not lost their individualidentity. Close temporal proximity, as inExperiments 4 and 5, was apparently not enough.. Here, for the first time in this series, we mayrule out the submelodic hypothesis, because thepairing manipulation cannot have had anysubstantial effect on the physical nature of eachconstituent.4 Likewise, the cognitive version of thesubmelodic hypothesis-J-Bonding indicative ofwhat we have called "mental chemistry"-receivedno encouragement from these last threeexperiments. What makes a difference is notwhether or not people try actively to integrate themelody and words in their minds, constructing anunheard song, but whether or not the two werestrictly simultaneous.

160 Crowder et al.

Straightforward as this interpretation is, we arenot so naive as to believe that one absentinteraction from a single experiment canoverthrow ideas as important as J-Bonding orEncoding Specificity. Besides the usual cautionthat we need more converging evidence on thispoint, it could be argued, albeit with someconsiderable added assumptions, that all subjects,in both groups of this experiment, left thepresentation sequence with self-generated songsas memory representations. Hearing a hummedmelody at the same time as one hears arhythmically-matching stream of words mightproduce the experience of a song, whether thesubject is trying to generate this or not. Thiswould account for the integration effect amongsubjects tested with real songs. To account for thesame effect in subjects tested with spoken songs,we need only observe that for these people, theconditions of acquisition and testing were exactlythe same, which could have outweighed thedisadvantage produced by the need for thesesubjects to generate song representations at test,as well as at acquisition.

An automatic process generating song-likerepresentations from simultaneous, compatibleverbal and musical streams would not beunexpected from a consideration of speechprocessing: In our ordinary lives, a simultaneousmixture of this sort is the rule rather than theexception, because the segmental features (inwords) are always overlaid upon supra-segmentalfeatures, including specifically variation infundamental frequency. For this ecological reason,simultaneous variation in pitch might be assignedto the prosodic aspect of speech automatically,even when the listener "knows" the verbal andtonal messages are nearly independent, as inlistening to songs, or spoken songs. Theseconsiderations lead us to the design of futureexperiments better exploiting the tonal prosodyand spoken message of integrated. languagecommunication.

General DiscussionAs for the integration effect in song, our

experiments in this and in the two previouspapers (Serafine et aI., 1984; Serafine et aI., 1986)have guided our thinking in a number of ways.First of all, and despite musicological folklore tothe contrary, the meaning of words seems to havea negligible role in the fact that melody and wordsof folksongs become stored in an integral fashion.Here again, in Experiment 2, the result withstoodnonsense materials devoid of conventionalmeaning.

Secondly, we have adduced statistically reliablesupport for the submelodic hypothesis, suggestingthat particular words can change the musical linesufficiently to influence recognition of themelodies later. It is no wonder people are slow torealize that "Baa, Baa, Black Sheep" and"Twinkle, Twinkle, Little Star" are words to thesame tune-they are not, musically, quite thesame tune, by virtue of the words to which eachhas been set.

Finally we have uncovered a number of factorsthat govern the size of the effect, somestatistically reliable on their own and others not.In retrospect, it was perhaps misguided for us tohave thought that a single factor would controlthe integration effect. Among the agencies forwhich evidence exists, we must include firsttemporal contiguity, as shown in Experiment 6here. Barring the unknown contribution ofautomatic fusion of text and melody in songs,hearing words and the melody at the same timeappears to affect their joint storage in the mannerof paired associates. But we should not discardcompletely those factors uncovered by earlierexperiments in this series as potential factors;even though they were not reliable on their own,they did measurably affect the size of the effect.Among these, we count instructions to attend onlyto melodies rather than to the whole songs atpresentation (Serafine et aI., 1984). Similarly, inthe same experiment, acoustic non-identity ofpresentati()n and test materials (different singers,respectively) had an effect in the direction thatwould have been predicted (though notsignificantly). Elsewhere (Serafine et aI., 1986 andhere, in Experiment 4) we found that melodies inthe presence of the wrong words did indeed have adistracting effect on melody recognition, beyondthe facilitation that the correct words had.

Putting all these factors together, we believe weknow well how to arrange conditions so as tomaximize, or minimize, the integration of wordsand melodies in recognition of songs. Thislaboratory control is not unsatisfactory asexplanation, provided one gives up the goal ofhaving only one crucial component.

REFERENCESBoring, E. G. (1957). A history of experimental psychology. New

York: Appleton-Century-Crofts.Erdei, P. (1974). American folksongs to read, sing, and play. New

York: Boosey and Hawkes.Horowitz, L. M., & Manelis, L. (1972). Toward a theory of

redintegrative memory: Adjective-noun phrases. In G. H.Bower (Ed.), The psychology of learning and motivation, 6 (pp. 195-224). New York: Academic Press. .

Physical Interaction and Association by Contiguity in Memory for the Words and Melodies of Songs 161

Serafine, M. L., Crowder, R. G., & Repp, B. H. (1984). Integrationof melody and text in memory for song. Cognition, 16,285-303.

Serafine, M. L., Davidson, J., Crowder, R. G., & Repp, B. H. (1986).On the nature of melody-text integration in memory for songs.Journal ofMemory and u,ngtUlge.

Tulving, E. (1983). Elements ofepisodic memory. New York: OxfordUniversity Press.

FOOTNOTES-Memory & Cognition, in press, (in a shorter version).tYale University.lWe are well aware that the dictionary meaning of the word

contiguity stipulates that the events in question be juxtaposed,or adjacent in time, but not overlapping or coterminous. Thisdeparts from usage of the term within psychology, wheresuccessive and simultaneous arrangements are both considered

contiguous. In this paper we remain with this latter usage eventhough the former might be more justifiable to some scholars.

2Throughout this paper, quotation marks on test item labelsindicate a deviation from the nomenclature described underGeneral Method. Here, for example, an "old song" is solabelled because it is the real-word phonetic equivalent of anold song and is not exactly what was heard in the presentation.

3The stimuli were, of course, in no sense true songs. However,we retain the same terminology as used in the otherexperiments.

4(:ertainly not in Experiment 4 and 5, where the two constituentsdid not overlap in time. In Experiment 6, with simultaneouscontiguity, masking-like effects could have existed between themelodies and texts. This perceptual interaction is not what wemean by physical interaction, which could not have occurred inany of these experiments.